Accelerate Deep Learning in Hybrid Cloud with Intel Analytics Zoo & Alluxio

Table of Contents:

1. Introduction
2. The Need for Deep Learning on Big Data Systems
3. Introducing Intel Analytics Zoo
4. The Architecture of Analytics Zoo
5. Key Features of Analytics Zoo
6. Benefits of Using Analytics Zoo
7. Introducing the Hybrid Solution with Electro
8. The Challenge of Remote Data Access
9. The Solution: Analytics Zoo + Electro
10. Performance Report and Results
11. Conclusion

Introduction

In today's rapidly growing digital landscape, the need for processing and analyzing large amounts of data has become crucial for businesses across various industries. As a software engineering team at Intel MMP, our focus lies in leveraging big data analytics using machine learning or deep learning techniques. In this article, we will introduce our hybrid solution that combines analytics and electro to provide accelerated deep learning on big data systems.

The Need for Deep Learning on Big Data Systems

With the exponential growth of data in recent years, large neural networks have demonstrated excellent performance in the fields of machine learning and deep learning. As a result, many industries have shown a keen interest in performing complex big data analytics using these techniques. However, integrating machine learning or deep learning systems into existing big data analytics pipelines presents a significant challenge. The deep learning system needs to seamlessly integrate with other components in the pipeline to create an end-to-end solution.

Introducing Intel Analytics Zoo

To address this challenge, Intel has developed the Intel Analytics Zoo, an open-source analytics super project. This unified AI platform is designed specifically for big data systems, providing a solution for simplifying the creation of end-to-end deep learning pipelines. Analytics Zoo allows users to create prototypes on their laptops using simple sample data and then Scale up their experiments on development clusters using historical data. Once the models are trained, they can be seamlessly deployed to production cluster environments with minimal code changes.

Analytics Zoo also provides direct access to production big data systems such as Hadoop, HBase, and more. It enables seamless deployments on distributed clusters, simplifying the process of scaling applications from a laptop to a production environment.

The Architecture of Analytics Zoo

Analytics Zoo is built on top of various compute environments, including laptops, Kubernetes clusters, Hadoop clusters, and cloud-based systems. It supports major deep learning frameworks like TensorFlow and PyTorch, as well as popular distributed analytics systems like Spark and BlinkDB. Additionally, Analytics Zoo provides support for end-to-end deep learning pipelines through its API, allowing users to perform distributed TensorFlow or Python training on Spark. It also offers a Rio Spark framework for building applications on top of Spark clusters.

Key Features of Analytics Zoo

Analytics Zoo provides several key features to enhance the machine learning and deep learning experience on big data systems:

1. Distributed TensorFlow and Python Training: Analytics Zoo allows users to perform distributed training of TensorFlow and Python models on Spark clusters, leveraging the scalability of Spark to accelerate training tasks.

2. Inference on Spark: Analytics Zoo offers a framework for performing distributed inference, making it easier to deploy and run models in real-time on Spark clusters.

3. Deep Learning Pipelines Compatible with Spark DataFrames: Analytics Zoo supports deep learning pipelines that are compatible with Spark DataFrames, enabling users to leverage Spark's powerful data processing capabilities in their pipelines.

4. Automatically Scalable AutoML Framework: Analytics Zoo provides a scalable AutoML framework for time series prediction. Users can leverage this framework for automatic feature selection, model selection, hyperparameter tuning, and training time series models using TensorFlow and Spark.

5. Distributed Inference: Analytics Zoo supports distributed inference, making it easier to serve TensorFlow, Python, and BigDL models with real-time streaming frameworks such as Kafka.

Benefits of Using Analytics Zoo

By leveraging Intel Analytics Zoo, users can enjoy several benefits:

1. Simplified Data Analytics on Big Data Systems: Analytics Zoo simplifies the creation of complex data analytics systems on big data platforms, providing a unified platform for both analytics and AI workloads.

2. Seamless Integration with Existing Pipelines: With Analytics Zoo, users can seamlessly integrate their machine learning and deep learning systems into existing big data analytics pipelines without significant code changes.

3. Accelerated Data Loading: The combination of Analytics Zoo and Electro provides accelerated data loading, reducing the overhead of accessing remote data for compute tasks and improving performance.

4. Scalable and Efficient Training: By leveraging the scalability of Spark clusters, Analytics Zoo enables distributed training of deep learning models, making it easier to train models on large datasets.

5. Real-Time Inference: Analytics Zoo supports distributed inference, allowing users to serve models in real-time on Spark clusters or other streaming frameworks, leading to faster and more efficient predictions.

6. Versatility and Flexibility: With support for major deep learning frameworks, distributed systems, and Python libraries, Analytics Zoo offers users flexibility and versatility to build complex analytics systems tailored to their specific needs.

7. Real-World Applications: Analytics Zoo has been successfully deployed and tested in real-world applications, receiving positive feedback from various companies and partners.

Introducing the Hybrid Solution with Electro

While the Analytics Zoo platform provides significant benefits for big data analytics, it still faces challenges in accessing remote data sources efficiently. As the size of the data increases, accessing remote data for compute tasks becomes a bottleneck, especially for workload types like deep learning and Spark ETL. To address this challenge, Intel has developed a hybrid solution that combines Analytics Zoo with Electro.

The Challenge of Remote Data Access

In hybrid cloud environments, where compute and data storage are disaggregated across different clusters or cloud services, accessing remote data poses a significant challenge. Deep learning and analytics workloads often need frequent access to data, but remote data access introduces overhead and latency. Manually copying data from remote storage to local compute systems is time-consuming, error-prone, and inefficient. Therefore, a solution that makes data immediately available for compute tasks is essential.

The Solution: Analytics Zoo + Electro

In the hybrid solution developed by Intel, Electro is used as a storage layer for accessing remote data. By combining Analytics Zoo and Electro on the same cluster, the solution accelerates data loading for Analytics Zoo applications. This approach is particularly beneficial for deep learning workloads, which require repetitive access to data during the model iteration process.

The hybrid solution can be deployed in two scenarios:

1. The Analytics and Electro cluster are both in the public cloud, while the storage is in a different data center. Electro provides accelerated data loading for Analytics Zoo applications by accessing the remote storage.
2. The Analytics and Electro clusters are deployed in a cloud environment, while the data storage (Hadoop system) is an on-premises cluster. Electro enables Analytics Zoo to directly access the on-premises Hadoop system, improving the loading performance for data-intensive tasks.

Performance Report and Results

To demonstrate the performance improvement achieved by the hybrid solution, we conducted an experiment on AWS using Intel Analytics Zoo and Electro. In the experiment, we used the Inception model on ImageNet and measured the data loading time for different scenarios.

The results showed that using Electro with Analytics Zoo led to a significant speedup in data loading compared to direct access to S3. On average, the loading time was 1.5 times faster with Electro. Moreover, the standard deviation was smaller, indicating more consistent and predictable performance.

Conclusion

In conclusion, Intel Analytics Zoo provides a unified platform for performing big data analytics and deep learning on various compute environments. By combining Analytics Zoo with Electro in a hybrid cloud environment, users can benefit from accelerated data loading and improved performance for deep learning workloads. This solution addresses the challenges of remote data access and provides a seamless integration of machine learning and deep learning systems into big data analytics pipelines. With its key features and benefits, Analytics Zoo offers a versatile and efficient platform for building end-to-end data analytics solutions.

Resources:

• Intel Analytics Zoo
• Electro
• AWS