- Intel Advanced Vector Extensions 512 (Intel AVX-512)
- Innovation Engine,
- Internet Wide Area Remote Direct Memory Access (RDMA) Protocol (iWARP)
- Intel Volume Management Device (Intel VMD)
- Intel Platform Trust Technology (Intel PTT)
This series explores these new features to help explain why Intel says its new server platform represents the “biggest platform advancements in this decade.”
What Is iWARP?
Simply stated, iWARP allows an application to read or write a block of data from or to the memory of another application. The target application can be in another virtual machine (VM) or even a server on the other side of the planet. iWARP uses the standard TCP/IP protocol, making it ideal for storage solutions and hyper-converged architectures.
iWARP was designed in order to overcome three major sources of networking-related CPU overhead inherent to the Ethernet standard: transport processing, intermediate buffer copies, and application context switches. It does so by moving data management and network protocol processing to an accelerated, RDMA-capable Ethernet adapter. As a result, CPU cycles that were previously spent on traffic processing can be returned to applications, improving performance.
How Does It Work?
The protocol delivers high bandwidth and low latency by offloading transport processing from the system software and CPU onto specialized hardware, eliminating 40 percent of the CPU overhead generally attributed to networking. It does this through two methods: kernel bypass and direct data placement.
- Kernel bypass. In normal Ethernet traffic processing, system software relies on the CPU to align incoming packets into queues and to assemble the packets for transmission. When a queue is full, a CPU interrupt occurs and the CPU has to stop its packet processing to service the interrupt. All of this processing can drive up CPU utilization and impede network performance.
With kernel bypass, iWARP-capable network adapters directly manage RDMA connections, avoiding the system software as much as possible. RDMA establishes secure queue pairs between the source and destination computers, bypassing the host CPU, and transmitting packets directly to applications’ memory spaces.
- Direct data placement. With traditional Ethernet traffic, the host processor doesn’t know where the destination application resides, so it creates a queue in kernel space. It then copies every packet in the kernel queue into applications’ user spaces. iWARP eliminates these unnecessary copies. Because of its queue pairs, iWARP already knows where each packet is headed, so it can place packets directly into the destination applications’ user spaces.
Kernel bypass and direct data placement eliminate CPU-taxing network-related interrupts, copies, and queues, which results in lower latency, greater bandwidth, and lower CPU utilization. These benefits increase with the size of the message, so iWARP is particularly helpful for large storage blocks and VM migration.
How Do I Get It?
To take advantage of iWARP, organizations need Ethernet controllers that are iWARP capable, and applications must be written to take advantage of RDMA verbs. That’s where system software comes in. SMB Direct technology in the Windows Server 2012 operating system automatically detects iWARP-capable devices and uses any iWARP-capable path from source to destination for storage and VM migration.
The new Intel Xeon Scalable platform includes integrated Intel Ethernet with iWARP RDMA and provides up to four 10 gigabit Ethernet (GbE) ports for high data throughput and low-latency workloads. This makes the platform ideal for software-defined storage solutions, NVM Express (NVMe) over Fabrics solutions, and VM migrations.
To learn more about iWARP, including the difference between iWARP and a competing approach (RDMA over Converged Ethernet, or RoCE), watch this video. For a deeper technical discussion of iWARP, read this PDF by NetEffect.
And, as always, you can keep up on the latest activities at Prowess by reading the Prowess blog and by following Prowess on Twitter and LinkedIn.
