Download as PDF, PPTX
Uploaded byJie-Han Chen
Data science-toolchain
The document outlines a data science toolchain presented by Jie-Han Chen, covering various programming languages and software such as Python, R, and Java, along with their strengths in data analysis and visualization. It discusses data collection methods, algorithm modeling, and machine learning libraries like Scikit-learn and TensorFlow. Additionally, it highlights tools for data visualization and frameworks like Scrapy for data extraction from websites.
More Related Content
Data science-toolchain
- 1. Data Science Toolchain presentedby Jie-Han Chen slide: https://goo.gl/1hXBGk
- 2.
- 3. Python Open Source Community Package WebService Good Readability Machine Learning
- 4. R Open Source Community Built-inStatistics Package Standalone computing & data analysis Slower than Python
- 5. High Performance Big Data PoorVisualization, Modeling Java
- 6. Matlab & Octave Powerfulbuilt-in math functions Simple Data Visualization tool Prototyping -5 0 5 10 -10 -10 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1 -5 0 5 10
- 7. Jupyter Notebook Support 40+programming language. eg: Python, R, Scala... Excellent for sharing your experiments Markdown, Latex example1 example2
- 8.
- 9.
- 10. Data Science Toolchains DataCollection Data Visualization Data Storage Algorithm & Modeling
- 12. Data Collection Using API:Facebook, Wikipedia Web Scraper
- 13.
- 14. Web Scraper HTTP request+ HTML Parser
- 15. HTTP: python-requests Better thanbuilt-in urllib Sessions with Cookie Persistence Thread-safety
- 16.
- 17.
- 18.
- 19.
- 20.
- 21.
- 22.
- 23.
- 24. Scrapy An open sourceand collaborative framework for extracting the data you need from websites.In a fast, simple, yet extensible way.
- 25. Scrapy $ scrapy startprojecttutorial
- 26.
- 27. path: /scrapy/dmoz.py crawler name:dmoz Scrapy
- 28.
- 29. $ scrapy crawldmoz Scrapy
- 31.
- 32.
- 33. "I believe thatvisualization is one of the most powerful means of achieving personal goals." Harvey Mackay
- 34.
- 35.
- 36.
- 37.
- 38. D3.js Data Visualization Project Interacive Webfrontend example1 example2
- 40. Bokeh Python, R, Scala,Julia Interactive Jupyter Notebook
- 41.
- 42.
- 44.
- 47.
- 48. Using GeoJSON withLeaflet , Configurable
- 49. Using GeoJSON withLeaflet
- 57. S3 1. Key-value 2. Permission 3.Data Visualization 4. Big Data (Spark)
- 58.
- 59. Algorithm & Modeling python-numpy+ python-pandas + scikit-learn libsvm spark-Mlib Weka Deep Learning
- 60. Numpy + Pandas +Scikit-learn
- 61.
- 62. Numpy - datastructure ndarray (n-dim array) ndim size shape dtype
- 63.
- 64.
- 65.
- 66.
- 67.
- 68.
- 69. numpy Series, DataFrame : csv,json ... nan
- 70.
- 71.
- 72.
- 73.
- 74.
- 75.
- 76.
- 77.
- 78.
- 79.
- 80.
- 81.
- 82.
- 84.
- 86. LIBSVM C Easy to use Supportmany programming languages Dataset
- 87. LIBSVM - install $git clone
- 88.
- 89.
- 90. LIBSVM - dataformat label index , attribute value , attribute
- 91. LIBSVM - dataformat
- 92.
- 94.
- 95. MLlib , Hadoop Java, Scala,R, Python
- 96.
- 97. MLlib , Hadoop Java, Scala,R, Python
- 98. Classification: logistic regression,naive Bayes,... Regression: generalized linear regression, survival regression,... Decision trees, random forests, and gradient-boosted trees Recommendation: alternating least squares (ALS) Clustering: K-means, Gaussian mixtures (GMMs),... Topic modeling: latent Dirichlet allocation (LDA) Frequent itemsets, association rules, and sequential pattern mining MLlib
- 99. Feature transformations: standardization, normalization,hashing,... ML Pipeline construction Model evaluation and hyper-parameter tuning ML persistence: saving and loading models and Pipelines MLlib
- 100. MLlib , Hadoop Java, Scala,R, Python
- 101.
- 103.
- 104.
- 105. Theano Base on Numpy Implementedby Cython Dynamic C code generation GPU & CUDA tensor, math expression A CPU and GPU Math Compiler in Python Theano tutorial: http://www.slideshare.net/SergiiGavrylov/theano-tutorial
- 106. Keras Theano, Tensorflow Support GPU prototype High-levelneural networks library
- 112.
- 114. Homework Github repo Datascience Database, Social Network Analytics, ML library, Deep Learning Platform ... READM.md: Repo Demo Code email: ita3051@gmail.com Google https://goo.gl/forms/PQPz8u2glyunQvfM2