Boosting Performance: RDMA with GPU Memory via DMA-Buf

Boosting Performance: RDMA with GPU Memory via DMA-Buf

AD

APIMart

APIMart

AI API, 99.9% SLA. Your AI, Always On.Most models offer cost savings of 20–60%; nanobanana pro api costs only 0.03.

AI API, 99.9% SLA. Your AI, Always On.Most models offer cost savings of 20–60%; nanobanana pro api costs only 0.03.

Try Now

Table of Contents

1. Introduction
2. What is DMA?
3. Overview of DMA Operation
4. Memory Registration for System Memory
5. Using GPU Memory in DMA Operations
6. Peer-to-Peer DMA
7. Existing Solutions: Mellanox Pure Direct
8. Proposed Solution: DMA Buff as a Mechanism
9. Introduction to DMA Buff
10. Implementation of DMA Buff for GPU Memory
11. Changes Required in Software Stack
12. Prototype Implementation and testing
13. Next Steps and Submission to Linux

🖥️ Introduction

In this article, we will explore the concept of using DMA Buff as a mechanism to support GPU memory in DMA operations. DMA (Direct Memory Access) is a vital process in computer systems and allows efficient data transfer between devices and memory without involving the CPU. We will dive into the details of DMA and its operation, and how it can be combined with GPU memory to enhance performance. Additionally, we will discuss existing solutions like Mellanox Pure Direct and propose a new solution using DMA Buff. A prototype implementation and testing will be discussed, along with the necessary changes required in the software stack. Finally, we will Outline the next steps and the submission of the proposed solution to the Linux kernel.

⚙️ What is DMA?

DMA, or Direct Memory Access, is a mechanism in computer systems that allows devices to transfer data to and from memory without involvement from the CPU. It enables efficient data transfer by reducing the load on the CPU and improving system performance. In a typical DMA operation, an initiator device reads the data from the buffer and sends it over the network, while a target device performs a DMA write operation to write data to the buffer. DMA operations are commonly used in networking and storage devices for high-speed data transfers.

🔍 Overview of DMA Operation

To understand how DMA and GPU memory can be combined, let's first get an overview of a typical DMA operation. In the initiator side, a DMA read operation is performed by a device (such as a network interface card - NIC) to read data from a buffer. The data is then sent over the network. On the target side, a DMA write operation is performed to write data to the buffer. The direction of the DMA operation can be reversed, depending on the specific use case. In order to set up the DMA operation, memory pages need to be pinned to ensure that the physical memory is not accessed for other purposes during the DMA operation. This is typically done through memory registration, where the NIC driver provides the necessary information for address translation and provides the DMA address as a list to the NIC.

💡 Memory Registration for System Memory

In a typical DMA operation using system memory, memory pages need to be pinned to ensure that they are not accessed for other purposes during the DMA operation. This is achieved through a process called memory registration. The NIC driver provides all the necessary information for address translation to obtain the DMA address for the buffer. This information is then saved internally by the NIC in a memory translation table or a similar data structure.

💡 Using GPU Memory in DMA Operations

When it comes to using GPU memory in DMA operations, the process needs to be approached differently. GPU memory is local, and the NIC driver cannot directly Pin the GPU memory. Furthermore, the DMA address of the GPU memory is not known to the NIC driver, making it challenging to perform address translation. To overcome this, the NIC driver and the GPU driver need to work closely together and exchange information. The GPU driver, which has control over the GPU memory, needs to provide a mechanism for sharing the information of the GPU memory with the NIC driver. This can be achieved through a handshake process, where the information is exchanged during the memory registration phase.

🚦 Peer-to-Peer DMA

In certain scenarios, both the NIC and the GPU are connected to the PCIe network. In such cases, peer-to-peer DMA can be used to perform DMA transactions directly between the NIC and the GPU memory. Mellanox Pure Direct is an example of supporting this kind of operation. It provides a clocking interface for the kernel's DMA driver, and each GPU driver implements a set of API functions that are called by the RDMA (Remote Direct Memory Access) subsystem. These functions are responsible for detecting and associating the memory owned by the GPU with the DMA calls.

📜 Existing Solutions: Mellanox Pure Direct

Mellanox Pure Direct is a solution that enables peer-to-peer DMA transactions between NICs and GPUs connected via PCIe. It provides an interface for the kernel's DMA driver, allowing the GPU driver to implement a set of API functions. These functions are responsible for detecting and associating the GPU memory with DMA calls.

⚙️ Proposed Solution: DMA Buff as a Mechanism

Our proposed solution aims to use DMA Buff as a mechanism to exchange information between the NIC driver and the GPU driver. DMA Buff is a standard mechanism in the Linux kernel that allows sharing buffers between different device drivers. In our proposed solution, the GPU driver exports the GPU memory as a DMA Buff object, which is then imported by the NIC driver. This object contains all the necessary information for address translation and can be used for peer-to-peer DMA transactions between the NIC and the GPU memory.

🧰 Introduction to DMA Buff

DMA Buff is a standard mechanism in the Linux kernel for sharing buffers between different device drivers. It allows one driver (the exporter) to export a buffer as a DMA Buff object, which can be imported by another driver (the importer). The exporter defines a set of operations that are necessary for buffer management, including address translation. The importer can then use these operations to map the buffer and perform address translation.

🚧 Implementation of DMA Buff for GPU Memory

To support GPU memory in DMA operations using DMA Buff, several changes are required in the software stack. Firstly, the GPU software stack, including the GPU driver and the DMA Buff implementation, needs to be modified to optimize for peer-to-peer access. Secondly, changes are needed in the user-space library to provide an interface for obtaining a DMA Buff file descriptor. Additionally, the RDMA driver and the device interface need to be modified to support DMA Buff memory registration for GPU memory. Likewise, the user-space RDMA library needs to be updated to accommodate this new feature.

🧪 Prototype Implementation and Testing

A software prototype to support DMA Buff for GPU memory has been implemented and tested. The implementation is based on the Upstream Linux kernel version 4.6 and the most recent user-space RDMA core library. Intel GPUs with the i915 driver are used in the prototype, along with the Mellanox ConnectX-4 EDR NIC. The necessary changes to DMA Buff and the DMA driver have been added, and the prototype has been tested successfully, achieving the expected performance improvements.

🏭 Next Steps and Submission to Linux

The next steps for the proposed solution involve revising the code based on feedback from the Linux DMA mailing list. The code changes will be submitted to the upstream Linux kernel for review and inclusion. Additionally, developments in GPU driver and DMA Buff infrastructure need to be monitored to ensure all the necessary pieces are available upstream. Once all the pieces are in place, the proposed solution will be submitted to the Linux kernel, aiming for inclusion in future releases.

FAQ

Q: What is DMA Buff? A: DMA Buff is a standard mechanism in the Linux kernel for sharing buffers between different device drivers. It allows one driver to export a buffer as a DMA Buff object, which can be imported by another driver.

Q: How does DMA Buff support GPU memory in DMA operations? A: By using DMA Buff, the GPU driver exports the GPU memory as a DMA Buff object, which is then imported by the NIC driver. This object contains the necessary information for address translation and enables peer-to-peer DMA transactions between the NIC and the GPU memory.

Q: Can the proposed solution be used with other GPU drivers besides Intel? A: Yes, the proposed solution can be used with other GPU drivers as long as they support the necessary APIs for exporting GPU memory as DMA Buff objects.

Q: Is DMA Buff already supported in the upstream Linux kernel? A: DMA Buff is already supported in the upstream Linux kernel, but the proposed solution requires additional changes and optimizations to support GPU memory in DMA operations.

Q: What are the expected performance improvements with the proposed solution? A: The proposed solution aims to improve performance by enabling efficient peer-to-peer DMA transactions between the NIC and the GPU memory, reducing the CPU overhead and enhancing overall system performance.

Resources

• Linux Kernel
• Mellanox Pure Direct