Take the Poll | Join our Discussion
sponsored by
Connecting Components with PCIe
Peripheral Component Interconnect Express (PCIe) is a high-speed interface that connects peripheral hardware components like graphics cards, sound cards, Ethernet and Wi-Fi adapters, and high-speed SSDs to the motherboard. PCIe is also used as a physical network interface for high-throughput, low-latency communication among various compute nodes (CPUs, GPUs, FPGAs, custom-built ASIC accelerators) in heterogeneous computing systems.
The rapid adoption of artificial intelligence and machine learning (AI/ML) applications and the move to cloud-based workloads have significantly increased network traffic. The PCIe interface has evolved to better support these developments. PCIe has advanced as a point-to-point link-based interface to satisfy the I/O requirements across the cloud, enterprise, AI, embedded, IoT, automotive, and mobile market segments. After generations of advancements, the PCIe architecture has matured and is now capable of seamlessly delivering cost-effective, power-efficient, high-bandwidth, and low-latency solutions.
Let’s Take a Look Back
PCI (Peripheral Component Interconnect) was introduced in the early 1990s by engineers from Intel, AMD, and various other companies as a data I/O for computers, communication, and storage platforms. Through several updates, PCI eventually became PCI Express, a much improved technology when compared to its predecessors. Compared to previous version, PCIe features:
- Increased bus throughput
- Reduced footprint
- Lower I/O pin counts
- Enhanced performance
PCIe was initially named HSI (for High-Speed Interconnect), and later 3GIO (for 3rd Generation I/O), before being labeled with the PCI-SIG name, PCI Express. The PCI Special Interest Group (PCI-SIG) comprises more than 800 member companies and defines specifications and compliance tests that ensure the interoperability of PCIe systems. PCIe 1.0 (Gen1) was introduced in 2003. PCIe 1.0 supported a transfer rate of 2.5 GT/s, with a data rate of 250MB/s per lane. As other areas of technology improved, users sought higher speed and performance, which led to the development of PCIe 2.0 (Gen2) with a 5.0 GT/s per lane transfer speed and throughput doubled to 500MB/s per lane, in addition to several feature improvements.
Processing power and storage continued their rapid increase, leading to the introduction of PCIe 3.0 (Gen3) in 2010. PCIe 3.0 was a revolutionary product with a bit rate of 8 GT/s. However, as AI and Big Data grew in popularity, with their massive data and throughput requirements, the demand for even more speed and accuracy swelled. PCIe 4.0 (Gen4) was introduced in 2017, offering improved flexibility, scalability, and lower power consumption with a 16 GT/s bit rate, double the bandwidth offered by PCIe 3.0.
Advancements in technology never stall, however, and just two short years later the exponential increase of computing and memory capacity required the release of PCIe 5.0 (Gen5) in 2019. PCIe 5.0 is the latest standard operational in the market and offers another step up in performance to increase bandwidth and minimize communication latency in the data center and at the edge. It scales data rates up to 32 GT/s, another doubling in lane speed over the previous generation. This speed provides the necessary bandwidth to support demanding applications such as genomics, AI/ML training, video transcoding, and streaming games. Figure 1 illustrates how PCIe I/O bandwidth rates have doubled every three years.
Figure 1: The Evolution of PCIe (Source: PCI-SIG)
Source: https://pcisig.com
PCIe expansion slots on a PC motherboard are typically available in five types: PCIe x1, PCIe x2, PCIe x4, PCIe x8, and PCIe x16. These numbers signify the number of lanes allocated to each PCIe slot. The number of lanes of a PCIe slot corresponds to the amount of transferred data; the more lanes there are, the more data can be transferred. All PCIe card versions are backward-compatible, so that any version of PCIe card can work together with any version of PCIe slot on a motherboard, with bandwidth in the mode of the lowest version.
What’s in the Future for PCIe?
The PCI-SIG has already announced the progress of its sixth generation PCIe 6.0 specification. PCIe 6.0 is slated for release by the end of 2021, with hardware available by 2023. Data rates will be doubled; PCIe 6.0 will take a leap to 64 GT/s from the current 32 GT/s offered by PCIe 5.0. The upcoming update delivers a higher signaling rate and tighter signal integrity requirements than its predecessor. The earlier generations of PCIe followed NRZ (Non-Return to Zero) signaling; however, PCIe 6.0 will make the switch to PAM4 (Pulse Amplitude Modulation), a technology that allows the packing of more bits into a serial channel within the same timeframe.
Official highlights of PCIe 6.0 are as follows:
- A data rate of 64 GT/s speeds, doubling the 32 GT/s data rate of the PCIe 5.0 specification
- A move to PAM4 (Pulse Amplitude Modulation with four levels) encoding
- Low-latency Forward Error Correction (FEC) with additional mechanisms to improve bandwidth efficiency
- Backwards-compatibility with all previous specification generations
|
NRZ vs. PAM4 While previous generations of PCIe used NRZ signaling, PCIe 6.0 will make the switch to PAM4 technology. PAM4 is able to send more data with each clock cycle, but how does it do it? Let’s take a look at the differences between NRZ and PAM4.
What is NRZ? NRZ (Non-Return to Zero) is a form of digital data transmission where the 1 and 0 are represented by DC voltages. In a positive-logic NRZ, the high is represented by the more positive voltage. For example:
Logic 1: +5.0V Logic 0: +1.0V
In a negative-logic NRZ, the low is represented by the more negative voltage.
Logic 1: +1.0V Logic 0: +5.0V
For each clock cycle, 1 bit is sent, either a 1 or a 0.
What is PAM4? PAM4 (Pulse Amplitude Modulation with 4 Levels) manages to squeeze 2 bits into each clock cycle with a clever encoding mechanism. Let’s assume that there are two bitstreams, A and B. PAM4 takes the signal from B, divides it in half, and adds it to A.
As an example, for A = 1 and B = 0:
Signal A: +5.0V Signal B: +1.0V / 2 = +0.5V Signal A + B: +5.5V
This results in 4 potential levels:
A = 0 / B = 0: +1.5V (1.0 + 1.0 / 2) A = 0 / B = 1: +3.5V (1.0 + 5.0 / 2) A = 1 / B = 0: +5.5V (5.0 + 1.0 / 2) A = 1 / B = 1: +7.5V (5.0 + 5.0 / 2)
Figure 2: Comparison of NRZ vs PAM4 signals (Source: Kumoh National Institute of Technology)
Because we are now sending 2 bits with each clock cycle, the bandwidth is effectively doubled.
What are the potential disadvantages of using PAM4?
|
PCIe Connectors from Amphenol ICC
Connectors are an integral part of the PCIe ecosystem, and Amphenol ICC has developed solutions with specifications that meet the demands of PCIe standards. Amphenol ICC currently provides PCI Express Gen 3 Card Edge Connectors, PCIe M.2 Gen 3 and Gen 4 Card Edge Connectors, and PCI Express Gen 4 and Gen 5 Card Edge Connectors. PCIe connectors are designed to meet specifications such as impedance, insertion loss, and return loss limits. Also important is the strength to withstand the weight, shock, and vibration of a heavier component, such as a high-performance GPU card.
PCI Express® Gen 3 Card Edge Connectors
Amphenol ICC's 1.00 mm pitch, vertical card edge connectors enable PCIe signaling from 2.5 Gb/s (Gen 1) and 5 Gb/s (Gen 2), up to 8 Gb/s (Gen 3) per differential signal pair. These connectors have a modular design that allows for standard pin counts. The simplest bandwidth (x1) version supports a single PCIe lane and is frequently used in I/O cards in desktop PCs. The x4 and x8 connectors provide 64 and 98 contacts, respectively, for server I/O. The high bandwidth versions (x16 lanes and higher) are typically used for higher bandwidth applications such as high-performance GPUs in PCs or riser cards in servers.
Figure 3: PCIe Gen 3 Card Edge Connectors
Source: Amphenol ICC
PCIe® M.2 Gen 3 and Gen 4 Card Edge Connectors
Amphenol ICC's PCIe M.2 Gen 3 and Gen 4 connectors provide 67 contacts on 0.50 mm pitch. These connectors occupy less board space, offer more connector height options, and support higher data rates. They are designed for PCIe 3.0, USB 3.0, and SATA 3.0 applications, making them suitable for tablets, laptops, and low-profile storage and server applications. PCIe M.2 connectors also support higher data rate transmission with both single and double-sided modules.
Figure 4: PCIe M.2 Gen 3 and Gen 4 Card Edge Connectors
Source: Amphenol ICC
PCI Express® Gen 4 and Gen 5 Card Edge Connectors
PCIe Gen 4 and Gen 5 connectors from Amphenol ICC meet industry-standard PCIe 4.0 and 5.0 specifications. These 1.00 mm pitch, vertical card edge connectors enable all generations of PCI Express signaling in desktop PCs, workstations, and servers. Amphenol's vertical PCIe Gen 4 and Gen 5 range of connectors includes surface-mount (SMT), through-hole solder, press-fit (PF), and straddle-mount terminations.
Figure 5: Gen 4 and Gen 5 Card Edge Connectors
Source: Amphenol ICC
SAS/PCIe® 4.0 (U.2 and U.3) Connectors
Amphenol ICC's SAS PCIe 4.0 and 5.0 connectors adhere to the U.2 (SFF-8639) and U.3 (SFF-TA-1001) specifications, and are frequently used in data center storage flash arrays made up of enterprise solid-state drives (SSDs). The molded guidepost allows the device plug and receptacle to self-align during the mating process. With halogen-free high-temperature thermoplastic, these connectors can withstand a variety of thermal conditions. They also offer the durability of 500 mating cycles.
Figure 6: SAS/PCIe 4.0 (U.2 and U.3) Connectors
Source: Amphenol ICC
Summing Up
With a rich and successful history of navigating several technology transitions in a backwards-compatible manner, PCIe is well-positioned to lead the changing computing landscape. The advancement in PCIe technology has kept pace with the growing demand for high bandwidth applications, such as AI/ML, cloud computing, IoT, and industrial automation. PCIe standards, such as PCIe 4.0, PCIe 5.0, and the upcoming PCIe 6.0 are designed to keep the technology cost effective, scalable, and power efficient.