Download as PDF, PPTX
Uploaded byDataWorks Summit
MLeap: Deploy Spark ML Pipelines to Production API Servers
MLeap is a framework designed to streamline the deployment of machine learning models from research to production, focusing on serialization and execution efficiency, especially outside the JVM. It allows users to train models in Spark, serialize them into MLeap bundles, and execute them via a REST API with minimal code changes. Future developments include expanding support for other ML frameworks and improving performance benchmarks.
In this document
Powered by AI
MLeap facilitates scaling machine learning from research to production. The presentation outlines its parts: introduction to MLeap, future roadmap, and a demo.
MLeap addresses issues such as lengthy production times for Spark models. Original requirements focused on eliminating re-coding and optimizing performance.
New requirements emphasize executing inference outside JVM and compatibility with other ML frameworks like Scikit-Learn and TensorFlow.
MLeap consists of a serialization framework and an execution engine for machine learning pipelines, facilitating smooth transitions from research to production.
The workflow involves writing ML pipelines in Spark, serializing them to MLeap bundles, and executing via a REST API—minimizing dependencies.
Explains 'bundle.ml' as a serialization format containing metadata, model execution data, and connections for model input/output.
Details on creating custom transformers for Spark and MLeap, and how serialization is handled in various formats like JSON and Protobuf.
Defines core concepts such as LeapFrame which denote how data is structured, including types and schemas essential for ML models.
MLeap Serving enables deployment on IoT and cloud APIs. Discusses compatibility with Scikit-Learn and TensorFlow.
MLeap shows significant performance improvements, reducing execution times to 4-15 ms compared to Spark's 0.2s-1s.
Future development plans include full streaming support and expanded compatibility with frameworks like Rust and Scikit-Learn.
Encourages community contributions, sharing experiences, and seeking product managers for MLeap to foster growth and collaboration.
Explores diverse programming language support for MLeap including Rust, Python, and others to enhance interoperability.
Demonstrates a live training session utilizing Spark Feature Pipeline with a linear regression model, showcasing real-time data transformation.
Thanking the audience, providing contact details, and encouraging engagement with MLeap community and projects.
MLeap: Deploy Spark ML Pipelines to Production API Servers
- 1. MLeap: Scaling MachineLearning From Research to Production https://github.com/combust/mleap twitter: @combustml
- 2. Intros Hollin Wilkins MikhailSemeniuk
- 3. Our Talk in3 Parts 1. What is MLeap? ○ Problem Statement + Architecture of serialization format and execution engine + Benchmarks 2. Future of MLeap/Product Roadmap ○ Beyond Spark and JVM 3. Demo: Train and deploy a streaming model to an API server with MLeap-Serving
- 4.
- 5. Initially Took too longto get Spark models out into production
- 6. Original MLeap Requirements -Has to eliminate re-coding of feature pipelines and models from research to production - Serving/inference system has to be fast, sub-20ms at worst - Should require minimal amount of new code to be written by the researcher to add new features/models - Should be a lightweight library that will allow users/organizations to customize as they see fit
- 7. Now Solve for morethan just Spark … we’ll talk about this later
- 8. New MLeap Requirements -Inference needs to happen outside of the JVM (Train in Spark, execute on an embedded device) - Should support other popular ML frameworks like Scikit-Learn, and TensorFlow
- 9. MLeap Architecture (high-level) ASerialization Framework For Machine Learning Pipelines An Execution Engine for Machine Learning Pipelines
- 10. From Research toProduction in 3 Steps 1. Continue to write your ML pipelines and training of models in Spark 2. Serialize your entire feature pipeline and model(s) to an MLeap bundle, called bundle.ml 3. Load the serialized pipeline to MLeap serving and execute via a REST-api, without any dependency on the Spark-context
- 11.
- 12. bundle.ml: Serialization Framework Vector Assembler Continuous FeatureVector Standard Scaler Vector Assembler Scaled Continuous Feature Vector String Indexer String Indexer OneHotEncoder Vector Assembler Categorical Feature Vector OneHotEncoder Linear Regression Continuous Features Categorical Feature (bundle 1)
- 13. bundle.ml: Serialization Framework Vector Assembler Continuous FeatureVector Standard Scaler Vector Assembler Scaled Continuous Feature Vector String Indexer String Indexer OneHotEncoder Vector Assembler Categorical Feature Vector OneHotEncoder Random Forest Regression Continuous Features Categorical Feature (bundle 2) PCA
- 14. bundle.ml: Structure Bundle.json -Root-level meta data about pipeline (version, names, etc.) Model.json - Data required to execute the model (coefficients, decision trees, intercepts, string lookups, etc.) Node.json - Connects input/output data for models to a LeapFrame (features for a logistic regression, prediction field for a random forest, etc.)
- 15. Custom Transformers Custom Spark Transformer CustomMLeap Transformer Bundle Spark Serializer Bundle MLeap Serializer MLeap Bundle - Define your model and node conversions - Bundle.ML handles serializing as either JSON or Protobuf - All transformers in MLeap are implemented in this way - Custom MLeap TFs: Unary/Binary, SVM, Imputer Logic
- 16. Core Concepts: DataFrame (LeapFrame) square_feet (Int) room_type (string) avg_rating (double) is_special (bool) 1200 House .93 true 800 Apartment .90 false 1. Schema defines names and types of columns 2. Rows to hold data
- 17.
- 18.
- 19. MLeap <> Scikit-Learn -Preprocessing (Scalers, Label/OneHot Encoder) - Base Models (Linear/Logistic) - Dimension Reduction (PCA) - Other: PCA, RF, GBRT - Pipelines/Feature Unions MLeap Bundles MLeap-JVM
- 20. MLeap <> TensorFlow SparkFeature Pipeline TensorFlow Transformer TF via JNI MLeap Serving TF via JNI
- 21. MLeap Benchmarks Default MLeapTransformer (Random Forest): MLeap Row Transformer (Random Forest):
- 22. MLeap Deployed inProd Deployed With MLeap MLeap 4-15ms Spark 0.2s-1s
- 23.
- 24. MLeap Roadmap MLeap <>Spark - Full Streaming Support - Fully typed transformer pipelines MLeap <> Scikit - Scikit <> Spark Transformer Parity - Protobuf Serialization - Deserialization MLeap <> Rust - MIR - mid-level intermediate representation - JIT/Interpreted Mode
- 25. Grow the Community DeployedWith MLeap 14 Contributors 21k Lines of Code Started 2016 - Become a contributor! - File an issue report - Write a cool demo using MLeap - Discuss the future of MLeap - Chat with us about your use case - Let us help if you run into any problems Deployed With MLeap - Looking for Product Managers for MLeap - Share your MLeap success story - Write a blog post about your MLeap project
- 26. MLeap Rust CPU CUDA/HIPOpenCL MLeap Rust Python Obj-C/Swift Go Ruby C#
- 27. Demo Train Spark FeaturePipeline + Linear Regression Model Apartment Listing Data MLeap Serving Client API Feature Pipeline Bundle Streaming Listing Data Transform Features Stream to Socket Online Linear Regression
- 28. Thank You! Hollin Wilkins e:hollin@combust.ml tw: @hollinwilkins https://github.com/combust/mleap Mikhail Semeniuk e: mikhail@combust.ml tw: @mikhailsemeniuk https://gitter.im/combust/mleap https://twitter.com/combustml