Downloaded 38 times
Uploaded byVMware Tanzu
Integrate Machine Learning into Your Spring Application in Less than an Hour
The document discusses Amazon's Deep Java Library (DJL) designed to integrate machine learning into Spring Boot applications easily. It highlights the challenges of machine learning, such as the skills gap and complex model building, while emphasizing DJL's advantages for Java developers, including support for various ML frameworks and improved performance. The document also outlines DJL's functionalities, including model training, multi-GPU support, and ease of integration with existing applications.
In this document
Powered by AI
Introduction to using the Deep Java Library to integrate ML into Spring Boot apps in under an hour. Machine Learning is a priority across organizations in 2020.
Highlights common challenges in ML like skill gaps and complexity in model building while emphasizing the need for easier development and scaling.
Overview of AWS ML capabilities including vision, speech, and other services, highlighting the extensive support for various ML frameworks and tools.
Introduction to Deep Java Library (DJL) with features such as API compatibility, pre-trained models, and ease of use for Java developers.
DJL offers an easy interface and rapid model deployment with excellent performance metrics, including a significant reduction in inference times.
Real-world examples of DJL in action at Amazon, including advertising and customer behavior analytics, with efficiencies and framework flexibility.
Technical overview of integrating DJL within Spring Boot, including dependency management, auto-configuration, and model inference.
Discussion on the deployment cycle, collaboration with GitOps, future enhancements for DJL including Android support and federated learning.
More Related Content
Similar to Integrate Machine Learning into Your Spring Application in Less than an Hour
Integrate Machine Learning into Your Spring Application in Less than an Hour
- 1. 1© 2020 AmazonWeb Services, Inc. or its affiliates. All rights reserved | 1© 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved | With Deep Java Library Hermann Burgmeier Mikhail Shapirov Qing Lan Vaibhav Goel Integrate Machine Learning into Your Spring Boot Application in Less than an Hour
- 2. 2© 2020 AmazonWeb Services, Inc. or its affiliates. All rights reserved | Put machine learning in the hands of all developers
- 3. 3© 2020 AmazonWeb Services, Inc. or its affiliates. All rights reserved | MACHINE LEARNING IN 2020 • ML is a top priority for many organizations in 2020 • ML pulls through more compute and data (it can pull it away to another cloud too) • Tens of thousands of customers are already running ML on AWS. It’s real and happening now in your accounts.
- 4. 4© 2020 AmazonWeb Services, Inc. or its affiliates. All rights reserved | COMMON CHALLENGES • Skills gap—not enough people can build ML models • ML model building is a time-consuming and complex process • Finding the right business use cases that could benefit from ML
- 5. 5© 2020 AmazonWeb Services, Inc. or its affiliates. All rights reserved | Easier to build Easier to scale Easier to apply
- 6. 6© 2020 AmazonWeb Services, Inc. or its affiliates. All rights reserved | The AWS ML Stack Broadest and most complete set of Machine Learning capabilities VISION SPEECH TEXT SEARCH CHATBOTS PERSONALIZATION FORECASTING FRAUD DEVELOPMENT CONTACT CENTERS Ground Truth Augmented AI ML Marketplace Neo Built-in algorithms Notebooks Experiments Model training & tuning Debugger Autopilot Model hosting Model Monitor Deep Learning AMIs & Containers GPUs & CPUs Elastic Inference Inferentia (Inf1 instance) FPGA Amazon Rekognition Amazon Polly Amazon Transcribe +Medical Amazon Comprehend +Medical Amazon Translate Amazon Lex Amazon Personalize Amazon Forecast Amazon Fraud Detector Amazon CodeGuru AI SERVICES ML SERVICES ML FRAMEWORKS & INFRASTRUCTURE Amazon Textract Amazon Kendra Contact Lens For Amazon Connect SageMaker Studio IDE NEW NEW! NEW! NEW! NEW! NEW! NEW! NEW! NEW! NEW! NEW! Amazon SageMaker
- 7. 7© 2020 AmazonWeb Services, Inc. or its affiliates. All rights reserved | The AWS ML Stack Broadest and most complete set of Machine Learning capabilities VISION SPEECH TEXT SEARCH CHATBOTS PERSONALIZATION FORECASTING FRAUD DEVELOPMENT CONTACT CENTERS Ground Truth Augmented AI ML Marketplace Neo Built-in algorithms Notebooks Experiments Model training & tuning Debugger Autopilot Model hosting Model Monitor Deep Learning AMIs & Containers GPUs & CPUs Elastic Inference Inferentia (Inf1 instance) FPGA Amazon Rekognition Amazon Polly Amazon Transcribe +Medical Amazon Comprehend +Medical Amazon Translate Amazon Lex Amazon Personalize Amazon Forecast Amazon Fraud Detector Amazon CodeGuru AI SERVICES ML SERVICES ML FRAMEWORKS & INFRASTRUCTURE Amazon Textract Amazon Kendra Contact Lens For Amazon Connect SageMaker Studio IDE NEW NEW! NEW! NEW! NEW! NEW! NEW! NEW! NEW! NEW! NEW! Amazon SageMaker
- 8. 8© 2020 AmazonWeb Services, Inc. or its affiliates. All rights reserved | 8© 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved | Deep Learning in Java with DJL
- 9. 9© 2020 AmazonWeb Services, Inc. or its affiliates. All rights reserved | DJL – The Machine Learning framework in Java • API - Designed for Java • Multithread support and memory control • Engine Agnostic: Write Once, Run Anywhere (MXNet, PyTorch, TensorFlow) • ModelZoo: 70+ pretrained models for out of box deployment • AWS actively contributes to Open Source ML projects
- 10. 10© 2020 AmazonWeb Services, Inc. or its affiliates. All rights reserved | DJL – Build AI Application with no ML Experience • DJL on Android device • Easy Deployment • 70+ models out of box deployment in applications
- 11. 11© 2020 AmazonWeb Services, Inc. or its affiliates. All rights reserved | Why should you use DJL? • Most enterprise customers using Java • Existing Java API for Deep Learning is painful • Deploy ML in Java is challenging - Hard to maintain • Java community lacks DL standard • No single standard in Java DL packages • DL with strongly typed language • Service code - readability and reliability
- 12. 12© 2020 AmazonWeb Services, Inc. or its affiliates. All rights reserved | DJL – Key advantage • Easy: 10 line inference setup • Minimum dependency requirement • Fast: up to 2x performance boost on small model inference. Used by Ads, query understanding and image classification • Large scale: Offline data processing with 800M inference load on Apache Spark. Online streaming real-time processing (AWS Kinesis, Apache Flink …) • Stable: 100+ hour benchmark test on continuous inference call.
- 13. 13© 2020 AmazonWeb Services, Inc. or its affiliates. All rights reserved | DJL – Easy Interface for Java Developers • DJL Live Jshell • DJL interactive console
- 14. 14© 2020 AmazonWeb Services, Inc. or its affiliates. All rights reserved | DJL for training • Engine ready: MXNet and PyTorch • Built-in metrics: Built-in metrics system to easy collect training information • Dataset ready: Built-in dataset for CV and NLP • Model ready: Built-in CV and NLP models • Multi-GPU support: Support training on Multi-GPU • Transfer learning: Use your existing models to do transfer learning • Jupyter Notebook Support (CPU/GPU)
- 15. 15© 2020 AmazonWeb Services, Inc. or its affiliates. All rights reserved | 15© 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved | DJL Customer Study
- 16. 16© 2020 AmazonWeb Services, Inc. or its affiliates. All rights reserved | Amazon Advertising • Multiple offerings on and off Amazon pages • Sponsored ads (Sponsored Brands, Sponsored Display, Sponsored Products, Stores) • Display ads • Video ads • Amazon DSP • Sponsored Products • Help shoppers discover and find products across Amazon's vast product selection. • Advertisers promote their products on search result and product pages to help increase reach and grow their business.
- 17. 17© 2020 AmazonWeb Services, Inc. or its affiliates. All rights reserved | Sponsored Products
- 18. 18© 2020 AmazonWeb Services, Inc. or its affiliates. All rights reserved | Machine Learning in Sponsored Products • 100s of machine learning (ML) models deployed globally for many use cases like click-through rate prediction. • More use cases require ML in advertising. New data science teams and models emerge constantly. • Different teams prefer different ML frameworks. • Models are hosted in a Java-based model hosting service. • Modern software engineering techniques like strong typing and dependency Injection. • Java stack is well integrated in the operational safety frameworks.
- 19. 19© 2020 AmazonWeb Services, Inc. or its affiliates. All rights reserved | DJL Usage in Sponsored Products • With DJL the scientists building the model can choose the best ML framework for them and their needs. • New ML frameworks supported by DJL can be easily integrated into our stack. • DJL enables us to run models built with different ML frameworks side- by-side in the same JVM without infrastructure changes. • DJL Translators allows us to keep feature generation independent of ML framework used for a model.
- 20. 20© 2020 AmazonWeb Services, Inc. or its affiliates. All rights reserved | Amazon Customer Behavior Analytics • Provides best-in-class targeting for any Amazon product or service to drive customer engagement • Helps to provide personalized marketing to customers • Email campaigns • Social Ads • Website banners • Identify customer’s propensity to take action in various product categories
- 21. 21© 2020 AmazonWeb Services, Inc. or its affiliates. All rights reserved | Machine Learning in Customer Behavior Analytics • Uses a multi-label classification model in MXNet for building propensity models • A single model is used for thousand of product categories • Use high dimensional feature data • Hundreds of Thousands of features per customer • The team consists of a mix of software development engineers and research scientists • Science team prefers using Python • Engineering team prefers using Java/Scala
- 22. 22© 2020 AmazonWeb Services, Inc. or its affiliates. All rights reserved | DJL Usage in Customer Behavior Analytics • With DJL • Science team can use MXNet Python APIs to build the model • Engineering team uses DJL to run inference using Apache Spark with Scala • DJL decoupled our system from ML framework • DJL easily handled high dimensional data • DJL reduced batch inference time by 85% • From 24 hours to 3.5 hours
- 23. 23© 2020 AmazonWeb Services, Inc. or its affiliates. All rights reserved | DJL Spring Boot Starter • Provides more native experience for Spring Developers • Handles dependency management for inference • Auto-configuration option for model look up and predictor wiring
- 24. 24© 2020 AmazonWeb Services, Inc. or its affiliates. All rights reserved | DJL Spring Boot Dependency Management <parent> <artifactId>spring-boot-starter-parent</artifactId> <groupId>org.springframework.boot</groupId> </parent> <properties> <jna.version>5.3.0</jna.version> <!--Required to override default JNA version for Spring Boot parent--> </properties> <dependencies> <dependency> <groupId>ai.djl.spring</groupId> <artifactId>djl-spring-boot-starter-mxnet-linux_86_64</artifactId> <version>{djl.version}</version> </dependency> </dependencies>
- 25. 25© 2020 AmazonWeb Services, Inc. or its affiliates. All rights reserved | DJL Spring Boot Auto <Platform> Dependency plugins { id("org.springframework.boot") } repositories { mavenCentral() } dependencies { implementation("ai.djl.spring:djl-spring-boot-starter-pytorch-auto") ... }
- 26. 26© 2020 AmazonWeb Services, Inc. or its affiliates. All rights reserved | DJL Spring Boot Auto-Configuration Maven: <dependency> <groupId>ai.djl.spring</groupId> <artifactId>djl-spring-boot-starter-autoconfigure</artifactId> <version>{djl.version}</version> </dependency> Gradle: dependencies { implementation("ai.djl.spring:djl-spring-boot-starter-autoconfigure :${djl.starter.version}") ... }
- 27. 27© 2020 AmazonWeb Services, Inc. or its affiliates. All rights reserved | DJL Spring Boot Configuration - application.yml/application.properties Access to the model zoo allows selection of application types (or inference types) from the following set: • QUESTION_ANSWER(NLP.QUESTION_ANSWER), TEXT_CLASSIFICATION(NLP.TEXT_CLASSIFICATION), IMAGE_CLASSIFICATION(CV.IMAGE_CLASSIFICATION), OBJECT_DETECTION(CV.OBJECT_DETECTION), ACTION_RECOGNITION(CV.ACTION_RECOGNITION), INSTANCE_SEGMENTATION(CV.INSTANCE_SEGMENTATION), POSE_ESTIMATION(CV.POSE_ESTIMATION), SEMANTIC_SEGMENTATION(CV.SEMANTIC_SEGMENTATION);
- 28. 28© 2020 AmazonWeb Services, Inc. or its affiliates. All rights reserved | DJL Spring Boot Configuration djl: # Define application type application-type: OBJECT_DETECTION # Define input data type, a model may accept multiple input data type input-class: ai.djl.modality.cv.Image # Define output data type, a model may generate different out put output-class: ai.djl.modality.cv.output.DetectedObjects # Define filters that matches your application's need model-filter: size: 512 backbone: mobilenet1.0 # Override default pre-processing/post-processing behavior arguments: threshold: 0.7
- 29. 29© 2020 AmazonWeb Services, Inc. or its affiliates. All rights reserved | DJL Spring Boot: Using Predictors Inject predictor given for the specified model and input/output: @Resource private Supplier<Predictor<Image, DetectedObjects>> predictorProvider; @Resource private ImageFactory imageFactory; // convenience factory for images Use with try-with-resources: try (var predictor = predictorProvider.get()) { var results = predictor.predict(imageFactory.fromFile(("/myimage.jpg"))); for(var result : results.items()) { LOG.info("results: {}", result.toString()); } }
- 30. 30© 2020 AmazonWeb Services, Inc. or its affiliates. All rights reserved | Containerize your Spring Boot Application jib { from.image = "adoptopenjdk/openjdk13:debian" to.image = "{repository-spec}/djl- spring-boot-app" to.tags = setOf(version.toString().plus("-") .plus(inferredClassifier).plus("-") .plus(commitHash)) } --- $./gradlew jib
- 31. 31© 2020 AmazonWeb Services, Inc. or its affiliates. All rights reserved | CD Cycle 1. Build 2. Upload to Container Registry 3. Run on Kubernetes (staged)
- 32. 32© 2020 AmazonWeb Services, Inc. or its affiliates. All rights reserved | GitOps with EKS and Team Collaboration ArgoCD GitHub Repos App of apps
- 33. 33© 2020 AmazonWeb Services, Inc. or its affiliates. All rights reserved | DJL Spring Boot: What is next? - Integrate DJL functionality - Include training (and transfer learning) with DJL Spring Boot Starter - Create solutions and workshops with full CI/CD and training/inference feedback loop and multi-tenancy - Running on EKS with GPU optimized instances
- 34. 34© 2020 AmazonWeb Services, Inc. or its affiliates. All rights reserved | DJL - Future • Android Support: Creating Android-based training and inference apps with DJL • Book – Dive into Deep Learning (Java Edition) • Integrate with more AWS Services (EMR, Lambda, etc…) • Federate Learning with DJL (Work in Progress) • Reinforcement Learning with DJL (work in Progress) • Distributed Inference on GPU support
- 35. 35© 2020 AmazonWeb Services, Inc. or its affiliates. All rights reserved | References Resources and Documentation: https://djl.ai/ https://docs.djl.ai/ Source Code: https://github.com/awslabs/djl/ https://github.com/awslabs/djl-spring-boot-starter https://github.com/awslabs/djl-spring-boot-starter-demo