Download as PDF, PPTX
Uploaded byRui Quintino
“Houston, we have a model...” Introduction to MLOps
The document introduces MLOps (Machine Learning Operations) and the need to operationalize machine learning models beyond just model deployment. It discusses challenges like data and model drift, retraining models, software dependencies, monitoring models in production, and the need for automation, testing, and reproducibility across the full machine learning lifecycle from data to deployment. An example MLOps workflow is shown using GitHub and Azure ML to enable experiment tracking, automation, and continuous integration and delivery of models.
More Related Content
Similar to “Houston, we have a model...” Introduction to MLOps
“Houston, we have a model...” Introduction to MLOps
- 1. Rui Quintino AI &Analytics Strategy Advisor DevScope 20210219 “Houston, we have a model...” Introduction to MLOps
- 2.
- 3. We have amodel... It’s great! YOU ARE HERE
- 4.
- 5.
- 6.
- 7. What about thedata Pipeline/preproc?
- 8. Or Data Drift-> causing Model Performance degradation? https://github.com/Azure/MachineLearningNotebooks/blob/master/how-to-use-azureml/work-with-data/datadrift-tutorial/datadrift-tutorial.ipynb
- 9. Plus... • How/who canwe retrain if needed (ex: data drift)? • What are the software requirements? • How can we later update lib/os versions? (ex: security) • How can we monitor model performance in production? • How do we ensure data checks in production? • How was the model trained? What was the performance? • How to scale? And Reproduce? • ...
- 10. Model & MLcode just tip of the iceberg...
- 11. Model & MLcode just tip of the iceberg... https://cloud.google.com/solutions/machine-learning/mlops-continuous-delivery-and-automation-pipelines-in-machine-learning
- 12. Introducing MLOps –Machine Learning & Operations
- 13. DevOps MLOps Code testing Codereproducibility App deployment Model retraining Model validation Model reproducibility Model deployment
- 14. Build app Collaborate Testapp Deploy app Monitor app App developer
- 15. Model reproducibility Modelretraining Model deployment Model validation Train model Validate model Deploy model Monitor model Retrain model Build app Collaborate Test app Release app Monitor app Audit /interpretability App developer Data scientist
- 16. GitHub & AzureML MLOps Demo
- 22. Lets try anew hyper param configuration...
- 27. Compute, Experiment, Model,Deployment in Azure ML
- 43. Tracking critical forMLOps, but also for Experimenting
- 44. With GitHub &CML • Automation for Experimentation • Ex: Smoke Test Locally (Fast) • Queue Reproducible in Github • Using notebooks Experimentation
- 45. GitHub Workflows tosupport Experiment Queues Check Continuous ML (CML) framework in Github
- 47. Branch for newexperiment, change code, queue in GitHub
- 48. Commit & Pushbranch to queue workflow in GitHub
- 49.
- 50. Notebook execution usingpapermill on Github agent
- 51. Train performance resultson commits (also PRs)
- 52. MLOps Recap •From Modelsto Pipelines •Execution of Pipelines produce models •Train for Deployment is fully automated/reproducible •Pipelines enable auto model retraining •Automation, Automation, Automation... •Monitoring for Data Drift, Concept Drift, Model Drift •Tracking, Metadata & Artifact Stores
- 53. Mature MLOps Example(Level 1) https://cloud.google.com/solutions/machine-learning/mlops-continuous-delivery-and-automation-pipelines-in-machine-learning
- 54. Not best fitfor interative/model development
- 55. Final notes •MLOps doesn’tguarantee ML success •MLOps should improve feedback and learning (not block DataScientists from business/data understanding) •Dont reinvent the wheel... •Still a young field Dont believe everything you read in docs (and ppt...) •When/If to introduce MLOps (it depends...)
- 56.
- 57.
- 58.
- 60. Also • https://azure.microsoft.com/en-us/services/machine- learning/mlops/ • https://cloud.google.com/solutions/machine-learning/mlops- continuous-delivery-and-automation-pipelines-in-machine- learning •https://github.com/microsoft/MLOps • https://databricks.com/blog/2020/10/13/using-mlops-with- mlflow-and-azure.html • https://github.com/Azure/MachineLearningNotebooks
- 61.
- 62. © 2016 DevScope.All rights reserved. Rua Passos Manuel Nº 223 – 4º Andar 4000-385 Porto T. +351 223 751 350/51 F. +351 223 751 352 Av. Sidónio Pais, Nº 2 – 3º Andar 1050-214 5 Lisboa info@devscope.net www.devscope.net
- 63.