Download as PDF, PPTX
Uploaded byTill Rohrmann
Interactive Data Analysis with Apache Flink @ Flink Meetup in Berlin
The document discusses interactive data analysis using Apache Flink and Apache Zeppelin, highlighting their features such as exploratory data analysis, a read-evaluate-print loop, and web-based notebooks. It provides examples of word count analysis and linear regression to predict sales based on advertisement spending, as well as a classifier for insult detection from social commentary. The conclusion emphasizes the ease of interactive data analysis with these tools, especially in machine learning tasks.
More Related Content
Interactive Data Analysis with Apache Flink @ Flink Meetup in Berlin
- 1. Till Rohrmann Flink PMCmember trohrmann@apache.org @stsffap Interactive Data Analysis with Apache Flink
- 2.
- 3. Exploratory Data Analysis § Visualize data § Calculate main characteristics § Understand data and find possibly new hypothesis 2
- 4.
- 5. Read-Evaluate-Print Loop § NewScala shell offers REPL § Interactive queries § Let’s you explore data quickly 4
- 6.
- 7. Simple Scala ShellExample 6
- 8. Problems § No visualization § No saving or replaying of written code § No assistance à Bad IDE 7
- 9. Notebooks § Web-based interactive computation environment § Combines rich text, execution code, plots and rich media § Storytelling 8
- 10. Apache Zeppelin § Web-basedREPL with pluggable interpreters § Since 2014 in the Apache Incubator § Supported interpreters: • Flink • Spark • Python • Markdown • Many more … 9
- 11. Word Count withZeppelin § Find the 10 most frequent words with more than 4 letters in the King James version of the bible. 10
- 12.
- 13.
- 14.
- 15.
- 16. Linear regression § Let’spredict the influence of advertisement spending on sales § Input data set: http://www-bcf.usc.edu/~gareth/ISL/ Advertising.csv § Features: • TV advertisement money • Radio advertisement money • Newspaper advertisement money § Response: • Sales 15
- 17.
- 18.
- 19.
- 20.
- 21.
- 22.
- 23.
- 24.
- 25.
- 26. Classification § Let’s builda classifier for insult detection § Kaggle challenge https://www.kaggle.com/c/detecting- insults-in-social-commentary § Label: 1 – Insult, 0 – No insult § Feature: Comment text 25
- 27.
- 28.
- 29. Conclusion § Interactive dataanalysis is really easy with Apache Flink § Apache Zeppelin is great interactive notebook § Zeppelin and Flink play well together to solve machine learning tasks and more 28
- 30.
- 31.