Unlocking Performance: GPU Support in Apache Spark 3

AD

YouArt

YouArt

Agentic Creative OS

Agentic Creative OS

Try Now

Table of Contents

1. Introduction
2. GPU Support in Apache Spark 3
3. GPU Features in Apache Spark 3
   • Accelerated-Aware Scheduling
   • Discovery Script Example
   • Task Context API
4. Stage Level Scheduling
5. SQL Columnar Processing
6. Project Hydrogen Initiative
7. The Plugin for Accelerated DataFrame and SQL Processing
   • Performance Comparison with CPU
   • Performance Comparison with GPU
   • Cost Savings with GPU
8. Accelerated Shuffle in Apache Spark
   • CPU Centric Data Movement
   • GPU Centric Data Movement
   • UCX and Unified Communication
9. Results and Benefits
   • Performance Improvement with Accelerated Shuffle
   • Handling Data Spills
10. Next Steps and Future Development
    • Open Sourcing the Accelerator Plugin
    • Nested Types Support
    • GPU Direct Storage
    • Timezone Support for Timestamps
    • Higher Order Functions and UDFs
11. Resources

🚀 GPU Support in Apache Spark 3

Apache Spark 3 introduces powerful GPU support, aiming to accelerate various operations and boost overall performance. This article will explore the GPU features in Apache Spark 3, including the Accelerated-Aware Scheduling, the Task Context API, and the enhancements in SQL Columnar Processing. Furthermore, we will delve into the benefits of using the Plugin for Accelerated DataFrame and SQL Processing, which enables significant speed-ups and cost savings. We will also discuss the concept of Accelerated Shuffle and how it revolutionizes data movement in Apache Spark. Finally, we will highlight the next steps for this project and the future development plans, such as supporting nested types, GPU Direct Storage, and UDFs. Let's dive into the exciting world of GPU support in Apache Spark 3!

GPU Features in Apache Spark 3

Accelerated-Aware Scheduling

The introduction of Accelerated-Aware Scheduling in Apache Spark 3 allows GPUs to be scheduled as a separate resource. This feature enables users to request GPU resources at the Executor, Driver, and Task levels. Additionally, it facilitates the discovery of available GPU resources on the nodes. This feature is supported on YARN, Kubernetes, and Standalone load clusters, providing flexibility in resource allocation.

Discovery Script Example

To detect GPU resources on a node, a discovery script can be utilized. This script produces JSON-formatted strings that Spark uses to determine the available GPUs. An example Discovery Script utilizes the NVIDIA Speed Binary, which is installed with the NVIDIA Driver. The output of this script is marshaled into a format that Spark can utilize to identify the GPUs on the system.

Task Context API

The Task Context API enables tasks to retrieve their assigned GPU resources. Tasks can access the task context from within their Resources map, allowing them to look up the assigned addresses based on the accelerator type. These addresses are STRING representations that can be passed into frameworks like TensorFlow to identify which indices or addresses were assigned to a particular GPU. The task context can also be accessed by the driver through the Spark context, providing essential information about GPU assignments.

Stage Level Scheduling

Stage Level Scheduling addresses the issue of disjointed GPU usage in Spark applications, particularly those involving ETL and machine learning stages. By allowing resource requirements to be specified per RDD operation, Spark can dynamically allocate containers to meet the specific stage's resource needs. This feature eliminates the need for a uniform task requirement throughout the application and optimizes the usage of GPUs during ETL and machine learning stages. Although this feature missed the Spark 3.0 release, it will be available in Spark 3.1.

SQL Columnar Processing

Spark 3.0 introduces SQL Columnar Processing, extending Catalyst, the SQL engine in Spark. The Plugin interface in Catalyst enables columnar operations, replacing the traditional row-based operations. This extension allows for efficient processing by vectorized accelerators like SIMD units, FPGAs, and GPUs. Plugins can modify query plans and use Columnar Batches instead of Row-based RDDs or data structures. Leveraging the Columnar format enhances processing speed and performance.

Project Hydrogen Initiative

With Spark 3.0, the Project Hydrogen initiative comes to fruition. Traditionally, the pipeline for machine learning-related data preparation involved separate CPU and GPU clusters. However, with Spark 3, a unified architecture consolidates and simplifies the infrastructure. The single pipeline benefits from stage level scheduling, allowing ingestion, data preparation, and model training to be orchestrated by Spark. This consolidated platform built for AI removes the need for multiple clusters and streamlines the data processing workflow.

🏆 The Plugin for Accelerated DataFrame and SQL Processing

The Plugin developed for accelerated DataFrame and SQL processing on the GPU demonstrates impressive speed-ups and cost savings. By leveraging the power of GPUs, ETL operations can be significantly faster and more efficient. Comparative performance tests show that GPU processing outperforms CPU processing by a substantial margin. For example, running a TPCx-BB query on the CPU took approximately 228 seconds, while using the RAPIDS accelerator Plugin reduced it to just over eight seconds. These speed-ups lead to valuable cost savings, making GPU acceleration a compelling choice for data-intensive tasks.

Performance Comparison with CPU

Running queries on the CPU-only configuration can be time-consuming, especially for extensive ETL operations. However, by integrating the RAPIDS accelerator Plugin, the processing time is drastically reduced. The Plugin optimizes GPU resources, enabling faster data processing and analysis. Comparisons between CPU and GPU processing consistently demonstrate the superiority of GPU acceleration, with significant performance improvements across various workloads.

Performance Comparison with GPU

The RAPIDS accelerator Plugin, combined with the accelerated shuffle feature, further enhances GPU performance. By minimizing data movement and leveraging GPU-centric data transfers, shuffle operations become highly efficient. Tests conducted on TPCx-BB queries showcase exceptional speed-ups and demonstrate the significant impact of GPU-centric data movement in reducing processing time.

Cost Savings with GPU

GPU-enabled processing not only offers substantial performance advantages but also provides cost savings. By accelerating data processing on the GPU, organizations can achieve faster time-to-insights, leading to improved efficiency and reduced expenses. Comparative cost analyses consistently highlight the cost-effectiveness of GPU processing, enabling organizations to achieve more with the same resources.

Accelerated Shuffle in Apache Spark

Traditional shuffle operations in Apache Spark heavily rely on CPU-centric data movement, resulting in multiple data transfers across the PCI-e Bus and host memory. The introduction of the accelerated shuffle feature aims to optimize this data movement for GPU processing. Through techniques like caching data in GPU memory, utilizing Nvlink or RDMA for GPU-to-GPU transfers, and leveraging UCX for unified communication, shuffle operations become more efficient and minimize the involvement of CPUs. This GPU-centric data movement dramatically improves performance and reduces overhead.

CPU Centric Data Movement

In traditional Spark shuffle operations, data is transferred between tasks and stages through multiple CPU hops. The CPU fetches data from GPU or local storage, processes it, and transfers it across the network to the receiving node's CPU. This approach incurs additional latency and overhead, especially when involving multiple nodes.

GPU Centric Data Movement

The accelerated shuffle feature leverages GPU-centric data movement to optimize data transfers. By caching data in GPU memory and utilizing technologies like Nvlink and RDMA, the data movement between GPUs becomes faster and more efficient. Transfers occur directly between the GPUs, eliminating the need for CPU involvement and reducing the impact on the PCI-e Bus. This GPU-centric approach significantly improves data transfer speeds and overall shuffle performance.

UCX and Unified Communication

The accelerated shuffle feature utilizes UCX (Unified Communication X), an open-source consortium that focuses on optimizing communication transports. UCX provides various communication options like TCP, RDMA, shared memory, and IPC for GPU transfers. It supports zero-copy GPU transfers over RDMA, ensuring high-performance data movement. UCX allows for efficient communication between GPUs, eliminating unnecessary CPU involvement and reducing overhead.

Results and Benefits

The integration of GPU support in Apache Spark 3, along with the accelerated shuffle feature and the RAPIDS accelerator Plugin, delivers remarkable results and benefits for data processing and analytics.

Performance Improvement with Accelerated Shuffle

Accelerated shuffle significantly improves shuffle performance by leveraging GPU-centric data movement techniques. By reducing CPU involvement and optimizing data transfers between GPUs, the time required for shuffle operations is dramatically reduced. This boosts overall data processing efficiency and enables faster insights extraction.

Handling Data Spills

Data spills occur when GPU memory is insufficient to accommodate all the data during shuffle operations. In such cases, the accelerated shuffle feature spills data to host memory or disk. By utilizing UCX, RDMA, or other optimized transfers, the Plugin minimizes the impact of data spills on performance. Even in scenarios where data spills occur, the accelerated shuffle feature ensures efficient data movement, minimizing computational overhead and ensuring high throughput.

Next Steps and Future Development

The development of GPU support in Apache Spark 3 is an ongoing process aimed at enhancing performance, scalability, and usability. Future steps and developments include:

Open Sourcing the Accelerator Plugin

The RAPIDS accelerator Plugin will be open-sourced, allowing developers and organizations to benefit from GPU-accelerated DataFrame and SQL processing. This open-source initiative promotes collaboration, innovation, and wider adoption of GPU technologies in Apache Spark.

Nested Types Support

Efforts are underway to support nested types like Arrays, Structures, and Maps in DataFrame and SQL operations. The goal is to provide a seamless experience for working with complex data structures and enable efficient processing on the GPU.

GPU Direct Storage

The integration of GPU Direct Storage aims to leverage the power of GPUs for direct data transfers between GPU memory and storage banks. This technology reduces reliance on CPU intermediaries and further optimizes data movement during ETL and other operations.

Timezone Support for Timestamps

Support for timezone manipulation in timestamp operations is being developed to enable more comprehensive data processing. This enhancement will provide compatibility with different timezones and facilitate accurate analysis on timestamp data.

Higher Order Functions and UDFs

The GPU support in Apache Spark 3 aims to enhance support for higher-order functions and enable automatic translation of certain types of UDFs to GPU operations. Research efforts are focused on optimizing UDF execution on GPUs for additional performance gains.

As the GPU support in Apache Spark continues to evolve, these advancements and future developments will unlock new possibilities for data processing, analytics, and machine learning.

Resources

To explore more about GPU support in Apache Spark 3 and the RAPIDS accelerator Plugin, visit the following resources:

• NVIDIA.com/Spark: Main landing page for GPU support in Apache Spark
• Contact Us: Reach out to the NVIDIA Spark team for more information
• Adobe's Email Marketing Use-Case: Learn how Adobe leveraged the RAPIDS accelerator Plugin for their marketing intelligence services use-case pipeline
• NVIDIA.com/Spark-book: Access a free ebook covering GPU features in Apache Spark 3, the RAPIDS accelerator Plugin, and related details