Data Science Project Lifecycle

Data Science Project Lifecycle

• 1.
  Data Science ProjectLifecycle Jason Geng @Data Application Lab Miya Du @Data Science Association
• 2.
  Business Requirement Data Acquisition Data Preparation Hypothesis & Modeling Evaluation & Interpretation Deployment Operations Optimization
• 3.
  Business Requirements u Datascientists need to work with business people and those with expertise in understanding the data, understanding the business u Specify the business requirements u For instance, the healthcare data
• 4.
  e.g. ‘DISCWT’: ‘This thedischarge-level weight on the HCUP nationwide data to produce national estimates’ Understand the data: Understand the Business: Goal: Predict Readmission Rate Database: Healthcare: Readmissions Database Modeling
• 5.
  Data Collection u Datafrom product line u Purchase third party data u Social media (Facebook, LinkedIn) u Web crawling u Open source data (Opendata, U.S. Census Data) Challenge Data Storage Data Management
• 6.
  Legacy data OLTP WebLog Web Crawler Open Source Third Party Data Social Media Data XML CSV LOG SQL … Product Line Business Intelligence Data Science App
• 7.
  Data Preparation (DataWrangling) u Cleaning data (semantic errors, missing entries, or inconsistent formatting) u Challenge: data integration u 80% time in project workflow Data Source A Data Source B Data Source B ETL Data Warehouse
• 8.
  Feature Engineering Select or creating features Research feature relevance Experiment and validation Changethe feature set Go back to feature selection step
• 9.
  Modeling Reference Source: http://scikit-learn.org/stable/tutorial/machine_learning_map/
• 10.
  Deploy to ProductLine
• 11.
  Thank you! https://www.DataAppLab.com Feb 2017 PPT:Xiaolu Zhao @ Feb 16, 2017