Download as PDF, PPTX
Uploaded bySeth Familian
Automating Big Data (Dec-2016)
This document discusses automating big data analytics processes. It describes the traditional "old school" approach of manually extracting, loading and transforming raw data. The document then presents two "new school" examples that automate these processes. The first automates extracting core website and social media data, enriching email addresses, and generating segments and metrics. The second integrates data from 70+ APIs in real-time, performs custom aggregations, and enables behavioral segmentation and messaging. The document concludes by soliciting questions and feedback on working with big data.
More Related Content
Similar to Automating Big Data (Dec-2016)
Automating Big Data (Dec-2016)
- 1. Seth Familian Founder +Principal, Familian&1 AUTOMATING BIG DATA
- 2. INTRODUCTION SETH FAMILIAN FOUNDER +PRINCIPAL, FAMILIAN&1 2 Corporate Strategy User Experience Design Creative ProcraftinationTeaching + Education Growth Hacking
- 3.
- 4. “Long” Data AUTOMATING BIGDATA THE “OLD SCHOOL” APPROACH 4 Raw “Header” File 2 1 Affiliates Lookup File Update Loop Summary Index saved searches scheduled searches TRANSFORM Instantly Generates EXTRACT LOAD Channel Dashboards
- 5. AUTOMATING BIG DATA SEGMENT-CLEARBITENRICHMENT 5 RAW DATA EXTRACT Core Website Enrichment of email addresses
- 6. AUTOMATING BIG DATA “NEWSCHOOL” EXAMPLE 1 6 RAW DATA EXTRACT LOADTRANSFORM Traffic Sources & Session Stats RAW DATA Behavioral Segments, Funnels, Retention & LTV EXTRACT Additional aggregation and data refinement Core Website Social Engagement Footprint Unified social footprint metrics Enrichment of email addresses CRM data store for easy segmentation + analysis Additional context on Twitter followers More flexible segments, funnels + retention metrics
- 7. AUTOMATING BIG DATA “NEWSCHOOL” EXAMPLE 2 7 LOAD Custom dashboards synced with 70+ APIs Traffic Sources & Session Stats Realtime (RT) TRANSFORMRAW DATA EXTRACT Core Website Social Engagement Footprint heavy-duty query tools already in place App Databases Custom aggregation scripts Postgres or Redshift DB Daily Pull Internally-reported metrics summarized for triangulation Daily CSV Behavioral segmentation + in-app messaging RT Behavioral Segments, Funnels, Retention & LTV RT Unified social footprint metrics Unified app downloads & ratings metrics App Store Activity email address enrichment
- 8. AUTOMATING BIG DATA8 MORE ADVANCED AUTOMATION
- 9. DISCUSSION TIME WORKING WITHBIG DATA 9 QUESTIONS · FEEDBACK · IDEAS · INSIGHTS
- 10. A BIT ABOUTME 10MY GENERAL ASSEMBLY WORKSHOPS
- 11. THANK YOU KEEP INTOUCH! 11 SETH@FAMILIAN1.COM · @SETHFAM1