Download as PDF, PPTX
Uploaded byDatabricks
How to Utilize MLflow and Kubernetes to Build an Enterprise ML Platform
This document summarizes a presentation about utilizing MLFlow and Kubernetes to build an enterprise machine learning platform. It discusses challenges that motivated building such a platform, like lack of model management and difficult deployments. The solution presented abstracts data pipelines into modular components to standardize workflows. It also uses MLFlow to package and track models and experiments, and Kubernetes with Kubeflow to deploy models at scale. A demo shows implementing model serving with these tools.
More Related Content
Similar to How to Utilize MLflow and Kubernetes to Build an Enterprise ML Platform
![[Redis Released]- FalkorDB - Redis + Graph Agentic Memory’s Secret Sauce](https://cdn.slidesharecdn.com/ss_thumbnails/redisreleased-falkordbslidedeck-1125-251115194922-e1c0046b-thumbnail.jpg?width=640&height=640&fit=bounds)
How to Utilize MLflow and Kubernetes to Build an Enterprise ML Platform
- 1. WIFI SSID:SparkAISummit |Password: UnifiedAnalytics
- 2. Nick Pinckernell, ComcastApplied AI Research Utilizing MLFlow and Kubernetes to build an Enterprise ML Platform #UnifiedAnalytics #SparkAISummit
- 3. Topics TOPIC WHY? Example ofdata pipeline abstraction Modular components and reuse are important for abstracting complex systems Ways to package and track ML project and experiments Consistency and reproducibility is key for scale How Comcast uses Kubeflow to serve and deploy models and pipelines A tangible example to help you brainstorm about your organizations requirements 3#UnifiedAnalytics #SparkAISummit
- 4. Challenges and motivations •Before, there was no – model management or tracking – standardization for model packaging or deployments • Cumbersome deployment process – Deployment required code rewrite from research to operations – Days or weeks to deploy • Response and tradeoff: restrict model complexity 4#UnifiedAnalytics #SparkAISummit
- 5. Requirements Minimum requirements fromour organization • Zero code refactoring or rewriting between research ready models and production • Easier experiment and model tracking • Researchers need to deploy their own models • A/B testing for quick model enhancement testing in production • Ability to modularize and inject custom metrics and workflows at each step 5#UnifiedAnalytics #SparkAISummit
- 6. Solution – existingtechnologies 6#UnifiedAnalytics #SparkAISummit Research Model serving Images, containers
- 7. Data pipeline abstraction •Determine use cases • Identify commonalities for modularization • Abstract interfaces • Automate configuration 7#UnifiedAnalytics #SparkAISummit
- 8.
- 9.
- 10.
- 11.
- 12.
- 13.
- 14.
- 15.
- 16.
- 17.
- 18.
- 19.
- 20. Seldon inference graphs Allowsfor complex graphs • A/B testing • Ensembles • Multi-armed bandit • Custom combinations 20#UnifiedAnalytics #SparkAISummit https://github.com/SeldonIO/seldon-core/blob/release-0.2/notebooks/advanced_graphs.ipynb
- 21. Packaging and tracking 1.Researchers code and train models with Databricks, Spark 2. Experiments tracked with MLFlow 3. Packaging and model tracking with MLFlow and Kubeflow • MLFLow standard packaging formats • scikit-learn • h2o • TensorFlow • more 21#UnifiedAnalytics #SparkAISummit
- 22. An MLFlow experiment 22#UnifiedAnalytics#SparkAISummit
- 23. MLFlow – multipleexperiments 23#UnifiedAnalytics #SparkAISummit
- 24. MLFlow – multipleexperiments 24#UnifiedAnalytics #SparkAISummit
- 25.
- 26. Research and modelflow 26#UnifiedAnalytics #SparkAISummit
- 27. Research and modelflow 27#UnifiedAnalytics #SparkAISummit
- 28. Research and modelflow – at scale 28#UnifiedAnalytics #SparkAISummit
- 29. Model serving withKubeflow Considerations and requirements • Resilient • Highly available • Rate limiting • Shadow deployments • Auto-scaling (WIP) 29#UnifiedAnalytics #SparkAISummit Ambassador http://www.getambassador.io
- 30. Throughput Static number ofreplicas Determined after • Constant and burst load testing with Locust 30#UnifiedAnalytics #SparkAISummit
- 31. DEMO A demonstration of •MLFlow experiments – Serving the chosen model • Implementation of components – Consumer pod – Model pod – Producer logic (to simulate real requests) 31#UnifiedAnalytics #SparkAISummit
- 32. Choosing the run 32#UnifiedAnalytics#SparkAISummit
- 33. Choosing the model 33#UnifiedAnalytics#SparkAISummit
- 34. Implementing the model 34#UnifiedAnalytics#SparkAISummit
- 35. Implementing the consumer 35#UnifiedAnalytics#SparkAISummit
- 36. Implementing the producer 36#UnifiedAnalytics#SparkAISummit
- 37. Deploy the model •Define the YAML / JSON Seldon deployment • Build the image – s2i build -E environment_rest . seldonio/seldon-core-s2i-python3:0.6-SNAPSHOT sklearn-iris-mlflow:0.3 • Deploy – kubectl create -f sklearn_iris_deployment.json -n kubeflow 37#UnifiedAnalytics #SparkAISummit
- 38.
- 39. 39#UnifiedAnalytics #SparkAISummit COMCAST ISHIRING PHILADELPHIA WASHINGTON, D.C. SILICON VALLEY DENVER
- 40. DON’T FORGET TORATE AND REVIEW THE SESSIONS SEARCH SPARK + AI SUMMIT