Download to read offline
![Unless otherwise indicated, these slides are © 2013 -2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by -nc/3.0/
Available TensorFlow Processors
• Object Detection – TF Object Detection API: [12] [13]
• Instance Segmentation, TF Object Detection API: [11]
• Semantic Segmentation, TF API: [8][9][10]
• Image Recognition (Inception V3), TensorFlow API: [7]
• Pose Estimation (OpenPose), TensorFlow API [5], [6]
• Twitter Sentiment Analysis, TensorFlow API: [4]
• Face Detection (MTCNN), ND4J,
ND4J-Tensorflow, JavaCV: [1], [2]
• Face Recognition (FaceNet), TensorFlow API: [3]](https://image.slidesharecdn.com/springone2018christiantzolov-machinescanlearnv2-181005173420/75/Machines-Can-Learn-a-Practical-Take-on-Machine-Intelligence-Using-Spring-Cloud-Data-Flow-and-TensorFlow-21-2048.jpg)
![Unless otherwise indicated, these slides are © 2013 -2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by -nc/3.0/
References
[1] Face Detection Lib: https://github.com/tzolov/mtcnn-java
[2] Face Detection Processor: https://github.com/tzolov/computer-vision/blob/master/spring-cloud-starter-stream-processor-face-detection-mtcnn
[3] Face Recognition: https://github.com/tzolov/computer-vision/tree/master/spring-cloud-starter-stream-processor-face-recognition
[4] Twitter Sentiment Analysis: https://github.com/spring-cloud-stream-app-starters/tensorflow/blob/master/spring-cloud-starter-stream-processor-twitter-sentiment
[5] Pose Estimation Processor: https://github.com/spring-cloud-stream-app-starters/tensorflow/tree/master/spring-cloud-starter-stream-processor-pose-estimation
[6] Pose Estimation Blog: https://content.pivotal.io/blog/making-data-science-accessible-to-developers-real-time-multi-person-human-pose-estimation-with-spring-
cloud-data-flow-and-tensorflow
[7] Image Recognition: https://github.com/spring-cloud-stream-app-starters/tensorflow/tree/master/spring-cloud-starter-stream-processor-image-recognition
[8] Semantic Segmentation Blog: https://goo.gl/dtwAfo
[9] Semantic Segmentation - Java2D: https://github.com/tzolov/computer-vision/tree/master/spring-cloud-starter-stream-processor-semantic-segmentation
[10] Semantic Segmentation - JavaCV: https://github.com/tzolov/computer-vision/tree/master/spring-cloud-starter-stream-processor-semantic-segmentation-cv
[11] Instance Segmentation - Object Detection Processor with Masks: https://github.com/spring-cloud-stream-app-starters/tensorflow/tree/master/spring-cloud-starter-
stream-processor-object-detection
[12] Object Detection Processor: https://github.com/spring-cloud-stream-app-starters/tensorflow/tree/master/spring-cloud-starter-stream-processor-object-detection
[13] Object Detection Blog: https://content.pivotal.io/blog/real-time-object-detection-with-spring-cloud-stream
32](https://image.slidesharecdn.com/springone2018christiantzolov-machinescanlearnv2-181005173420/75/Machines-Can-Learn-a-Practical-Take-on-Machine-Intelligence-Using-Spring-Cloud-Data-Flow-and-TensorFlow-27-2048.jpg)
![Unless otherwise indicated, these slides are © 2013 -2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by -nc/3.0/
Available TensorFlow Processors
• Object Detection – TF Object Detection API: [12] [13]
• Instance Segmentation, TF Object Detection API: [11]
• Semantic Segmentation, TF API: [8][9][10]
• Image Recognition (Inception V3), TensorFlow API: [7]
• Pose Estimation (OpenPose), TensorFlow API [5], [6]
• Twitter Sentiment Analysis, TensorFlow API: [4]
• Face Detection (MTCNN), ND4J,
ND4J-Tensorflow, JavaCV: [1], [2]
• Face Recognition (FaceNet), TensorFlow API: [3]](https://crownmelresort.com/image.slidesharecdn.com/springone2018christiantzolov-machinescanlearnv2-181005173420/75/Machines-Can-Learn-a-Practical-Take-on-Machine-Intelligence-Using-Spring-Cloud-Data-Flow-and-TensorFlow-21-2048.jpg)
![Unless otherwise indicated, these slides are © 2013 -2018 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by -nc/3.0/
References
[1] Face Detection Lib: https://github.com/tzolov/mtcnn-java
[2] Face Detection Processor: https://github.com/tzolov/computer-vision/blob/master/spring-cloud-starter-stream-processor-face-detection-mtcnn
[3] Face Recognition: https://github.com/tzolov/computer-vision/tree/master/spring-cloud-starter-stream-processor-face-recognition
[4] Twitter Sentiment Analysis: https://github.com/spring-cloud-stream-app-starters/tensorflow/blob/master/spring-cloud-starter-stream-processor-twitter-sentiment
[5] Pose Estimation Processor: https://github.com/spring-cloud-stream-app-starters/tensorflow/tree/master/spring-cloud-starter-stream-processor-pose-estimation
[6] Pose Estimation Blog: https://content.pivotal.io/blog/making-data-science-accessible-to-developers-real-time-multi-person-human-pose-estimation-with-spring-
cloud-data-flow-and-tensorflow
[7] Image Recognition: https://github.com/spring-cloud-stream-app-starters/tensorflow/tree/master/spring-cloud-starter-stream-processor-image-recognition
[8] Semantic Segmentation Blog: https://goo.gl/dtwAfo
[9] Semantic Segmentation - Java2D: https://github.com/tzolov/computer-vision/tree/master/spring-cloud-starter-stream-processor-semantic-segmentation
[10] Semantic Segmentation - JavaCV: https://github.com/tzolov/computer-vision/tree/master/spring-cloud-starter-stream-processor-semantic-segmentation-cv
[11] Instance Segmentation - Object Detection Processor with Masks: https://github.com/spring-cloud-stream-app-starters/tensorflow/tree/master/spring-cloud-starter-
stream-processor-object-detection
[12] Object Detection Processor: https://github.com/spring-cloud-stream-app-starters/tensorflow/tree/master/spring-cloud-starter-stream-processor-object-detection
[13] Object Detection Blog: https://content.pivotal.io/blog/real-time-object-detection-with-spring-cloud-stream
32](https://crownmelresort.com/image.slidesharecdn.com/springone2018christiantzolov-machinescanlearnv2-181005173420/75/Machines-Can-Learn-a-Practical-Take-on-Machine-Intelligence-Using-Spring-Cloud-Data-Flow-and-TensorFlow-27-2048.jpg)
Uploaded byChristian Tzolov
Machines Can Learn - a Practical Take on Machine Intelligence Using Spring Cloud Data Flow and TensorFlow
This document discusses using machine learning with Spring Cloud Data Flow and TensorFlow. It introduces machine learning concepts and how they differ from classical programming. It then describes using TensorFlow for deep learning inference in Java applications and building real-time predictive pipelines with Spring Cloud Data Flow. Several TensorFlow processors for tasks like object detection, image recognition, and sentiment analysis are presented.
More Related Content
Similar to Machines Can Learn - a Practical Take on Machine Intelligence Using Spring Cloud Data Flow and TensorFlow
Machines Can Learn - a Practical Take on Machine Intelligence Using Spring Cloud Data Flow and TensorFlow
- 1. Machines Can Learn- a Practical Take on Machine Intelligence Using Spring Cloud Data Flow and TensorFlow By Christian Tzolov @christzolov
- 2. Unless otherwise indicated,these slides are © 2013 -2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by -nc/3.0/ Disclaimer
- 3. Unless otherwise indicated,these slides are © 2013 -2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by -nc/3.0/ Safe Harbor Statement The following is intended to outline the general direction of Pivotal's offerings. It is intended for information purposes only and may not be incorporated into any contract. Any information regarding pre-release of Pivotal offerings, future updates or other planned modifications is subject to ongoing evaluation by Pivotal and is subject to change. This information is provided without warranty or any kind, express or implied, and is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions regarding Pivotal's offerings. These purchasing decisions should only be based on features currently available. The development, release, and timing of any features or functionality described for Pivotal's offerings in this presentation remain at the sole discretion of Pivotal. Pivotal has no obligation to update forward looking information in this presentation. 3
- 4. Unless otherwise indicated,these slides are © 2013 -2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by -nc/3.0/ Does It Matter?
- 5. Unless otherwise indicated,these slides are © 2013 -2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by -nc/3.0/ Rise of the machines • Machine Learning brings unprecedented abilities to the Software Engineering field • Provides different way to reason about problems • Solves un-programmable tasks 5 • There is a technological Gap ( e.g. BLAS/CPU, CUDA/GPU, C++/Python) • And cultural differences • How Machine Learning can enable us deliver richer business solutions?
- 6. Unless otherwise indicated,these slides are © 2013 -2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by -nc/3.0/ Real-Time Predictions with TensorFlow and Spring Cloud Data Flow (SCDF) Focus is on • Supervised, Deep Learning (DL) • TensorFlow and DL4J as DL toolkits • Model Inference (e.g. evaluation, prediction) • (Near) Real-Time - Streaming • Java and Spring 6
- 7. Unless otherwise indicated,these slides are © 2013 -2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by -nc/3.0/ How to Build SCDF applications like 7
- 8. Unless otherwise indicated,these slides are © 2013 -2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by -nc/3.0/ Can Machine Learn?
- 9. Unless otherwise indicated,these slides are © 2013 -2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by -nc/3.0/ Classical Programming 9 Classical Programming Rules Data Answers
- 10. Unless otherwise indicated,these slides are © 2013 -2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by -nc/3.0/ Machine Learning Paradigm 10 Machine Learning Rules’ Data* Answers* * Historical, annotated Datasets
- 11. Unless otherwise indicated,these slides are © 2013 -2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by -nc/3.0/ ML Lifecycle 11 Model Inference Output Predictions MODEL (e.g. Rules) New Data Model Training Annotated answers Historical Data Offline Online (Production) Software Engineering & Ops Data Science
- 12. Unless otherwise indicated,these slides are © 2013 -2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by -nc/3.0/ Remote Server Model Serving & Embedding 12 Data Source HTTP/REST, RPC Processor(s) Data Sink Data Source Processor(s) Model Data Sink Runtime Model Remote Serving In-Pipeline Embedded
- 13. Unless otherwise indicated,these slides are © 2013 -2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by -nc/3.0/ Deep Learning Inference with Java • Deep Learning Java Runtime • Portable Models 13 DL Java Runtime Input Data Output Predictions Pre-trained Model (e.g. Rules)
- 14. Unless otherwise indicated,these slides are © 2013 -2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by -nc/3.0/ Deep Learning Frameworks
- 15. Unless otherwise indicated,these slides are © 2013 -2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by -nc/3.0/ Deep Learning Frameworks • Tensorflow (Google) • PyTorch • CNTK (Microsoft) • Caffe2 (Facebook) • MXNet (Amazon) • DeepLearning4J • Keras 15 Deep Learning Java Runtime
- 16. Unless otherwise indicated,these slides are © 2013 -2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by -nc/3.0/ Real-Time Model Evaluation with Spring Cloud Data Flow
- 17. Unless otherwise indicated,these slides are © 2013 -2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by -nc/3.0/ Transport Middleware Stream A Spring Cloud Data Flow (SCDF) Toolkit for building data integration, real-time, and batch data processing pipelines • Multi-platform support (PCF, Kubernetes) • Comprehensive app lifecycle (Skipper) • Pipes & Filters like DSL: Source | Processor … | Sink Source Processor Processor Sink
- 18. Unless otherwise indicated,these slides are © 2013 -2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by -nc/3.0/ Localizing and identifying multiple objects in a single image • TF Object Detection API: http://bit.ly/tensorflow-object-detection-api • Pre-trained models: http://bit.ly/object-detection-api-model-zoo • Object Detection Processor Doc: http://bit.ly/scdf-object-detection-processor • Blog: https://content.pivotal.io/blog/real-time-object-detection-with-spring-cloud-stream Object Detection Processor file --directory=‘/tmp/scdf/od/input’ | object-detection --mode=header --model='file:/model.pb’ --labels='file:/label_map.pbtxt’ | image-viewer
- 19. Unless otherwise indicated,these slides are © 2013 -2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by -nc/3.0/ TensorFlow SCDF Processor 20 TensorFlowService Message payload headers OutputConverter InputConverter Message payload headers Map<String, Tensor> Load from file:// , http://, classpath:// TensorFlow Build/Train/Export TensorFlow Runtime TF Model Offline Real Time
- 20. Unless otherwise indicated,these slides are © 2013 -2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by -nc/3.0/ Object Detection - RT Demo
- 21. Unless otherwise indicated,these slides are © 2013 -2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by -nc/3.0/ Available TensorFlow Processors • Object Detection – TF Object Detection API: [12] [13] • Instance Segmentation, TF Object Detection API: [11] • Semantic Segmentation, TF API: [8][9][10] • Image Recognition (Inception V3), TensorFlow API: [7] • Pose Estimation (OpenPose), TensorFlow API [5], [6] • Twitter Sentiment Analysis, TensorFlow API: [4] • Face Detection (MTCNN), ND4J, ND4J-Tensorflow, JavaCV: [1], [2] • Face Recognition (FaceNet), TensorFlow API: [3]
- 22. Unless otherwise indicated,these slides are © 2013 -2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by -nc/3.0/ Quick Starters • TensorFlow Spring Cloud Stream App Starters: https://github.com/spring-cloud-stream-app-starters/tensorflow • Computer Vision Projects (experimental): https://github.com/tzolov/computer-vision • SCDF TensorFlow Demos (Docker): https://github.com/tzolov/scdf-tensorflow-demos • Real-time Object-Detection With Spring Cloud Stream https://content.pivotal.io/blog/real-time-object-detection-with-spring-cloud-stream • Real-time Object-Detection With Spring Cloud Stream https://content.pivotal.io/blog/real-time-object-detection-with-spring-cloud-stream • Real-Time Multi-Person, Human Pose Estimation with Spring Cloud Data Flow and TensorFlow http://bit.ly/scdf-pose-estimationn-blog
- 23. Unless otherwise indicated,these slides are © 2013 -2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by -nc/3.0/ Conclusions
- 24. Unless otherwise indicated,these slides are © 2013 -2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by -nc/3.0/ Convinced? • Data Science and Deep Learning mature at a rapid pace • Pivotal is committed to make them easily accessible for Spring/Java developers. • Way to start: Check the TensorFlow processors for Spring Cloud Data Flow • You feedback is appreciated!
- 25. Unless otherwise indicated,these slides are © 2013 -2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by -nc/3.0/ Next
- 26. Unless otherwise indicated,these slides are © 2013 -2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by -nc/3.0/ Unofficial agenda • Top level TensorFlow (or DL) project – move the utilities and existing implementations (such as Image Recognition, Object Detection, Sentiment Analysis) out of SCSt into a top level, generic projects. • Use with Spring Cloud Stream, Spring Cloud Function, Spring Cloud Batch • Research “Transfer Learning” possibilities • Evaluate Deeplearning4J, ND4J as Runtime alternative • ONNX Runtime to use models from other DL frameworks • REST, Spring Boot as remote DL serving application 31
- 27. Unless otherwise indicated,these slides are © 2013 -2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by -nc/3.0/ References [1] Face Detection Lib: https://github.com/tzolov/mtcnn-java [2] Face Detection Processor: https://github.com/tzolov/computer-vision/blob/master/spring-cloud-starter-stream-processor-face-detection-mtcnn [3] Face Recognition: https://github.com/tzolov/computer-vision/tree/master/spring-cloud-starter-stream-processor-face-recognition [4] Twitter Sentiment Analysis: https://github.com/spring-cloud-stream-app-starters/tensorflow/blob/master/spring-cloud-starter-stream-processor-twitter-sentiment [5] Pose Estimation Processor: https://github.com/spring-cloud-stream-app-starters/tensorflow/tree/master/spring-cloud-starter-stream-processor-pose-estimation [6] Pose Estimation Blog: https://content.pivotal.io/blog/making-data-science-accessible-to-developers-real-time-multi-person-human-pose-estimation-with-spring- cloud-data-flow-and-tensorflow [7] Image Recognition: https://github.com/spring-cloud-stream-app-starters/tensorflow/tree/master/spring-cloud-starter-stream-processor-image-recognition [8] Semantic Segmentation Blog: https://goo.gl/dtwAfo [9] Semantic Segmentation - Java2D: https://github.com/tzolov/computer-vision/tree/master/spring-cloud-starter-stream-processor-semantic-segmentation [10] Semantic Segmentation - JavaCV: https://github.com/tzolov/computer-vision/tree/master/spring-cloud-starter-stream-processor-semantic-segmentation-cv [11] Instance Segmentation - Object Detection Processor with Masks: https://github.com/spring-cloud-stream-app-starters/tensorflow/tree/master/spring-cloud-starter- stream-processor-object-detection [12] Object Detection Processor: https://github.com/spring-cloud-stream-app-starters/tensorflow/tree/master/spring-cloud-starter-stream-processor-object-detection [13] Object Detection Blog: https://content.pivotal.io/blog/real-time-object-detection-with-spring-cloud-stream 32
- 28.
Editor's Notes
- #6 Machine and Deep Learning (ML/DL) have brought unprecedented abilities to the software engineering field. ML/DL allows you to reason about and to solve otherwise “un-programmable” tasks such as computer-vision and language-processing. That’s why it’s so exciting to so many people. [1] (ML) is an approach for learning (data insights) without explicitly being programmed. [2] (DL) is a ML technique for learning features in data using Neural Networks. [3] TensorFlow is a DL framework for defining and executing computational graphs.
- #10 Software engineers are trained to think logically and mathematically; We USE ASSERTIONS to prove properties of our program are correct
- #11 With ML, the focus shifts from a mathematical science to a natural science: We're making observations about an uncertain world, running experiments, and Using STATISTICS, not logic, to analyze the results of the experiment. Machine learning changes the way you think about a problem. Software engineers are trained to think logically and mathematically; We use assertions to prove properties of our program are correct. With machine learning, the focus shifts from a mathematical science to a natural science: we're making observations about an uncertain world, running experiments, and using statistics, not logic, to analyze the results of the experiment.
- #12 Trains Models on Historical Datasets - Data Science (Offline) Run Pre-Trained models for predictive analytics - Software Engineers (Real-Time) Separation of Concerns
- #13 My goal is to show you how Spring Cloud Stream and Spring Cloud Data Flow can make this much easier for you.
- #14 My goal is to show you how Spring Cloud Stream and Spring Cloud Data Flow can make this much easier for you.
- #16 https://cv-tricks.com/deep-learning-2/tensorflow-or-pytorch/
- #21 My goal is to show you how Spring Cloud Stream and Spring Cloud Data Flow can make this much easier for you.
- #25 My goal is to show you how Spring Cloud Stream and Spring Cloud Data Flow can make this much easier for you.
- #30 business applications can seem a daunting task for most developers. But by using the right tools that abstract away some of the complexities of the data science itself, like Spring Cloud Data Flow and TensorFlow, it doesn’t have to be.