© Copyright 2017 NETMONASTERY Inc
Big Data Structures in the Cyber Security Puzzle
1
Shomiron DAS GUPTA, Founder, CEO
NETMONASTERY Inc.
Data Analytics in
Cyber Security
© Copyright 2017 NETMONASTERY Inc
Agenda … Setting Expectations!
2
NEXT 45 MINS
1. Cyber Security - How it works, today!
2. Data Analytics, the What and the Why
3. The Technical Aspects
4. The Pipeline
5. Opportunities - Gaps we’re aiming for
6. Demo
© Copyright 2017 NETMONASTERY Inc
Hello, who is speaking?
3
Founder of NETMONASTERY, we built DNIF - An
Integrated Threat Hunting Platform for the CSOC
Research on Detection, Hunting and …. ML
One of the few guys that does defense for a
living
GCIA 2000 - 18Yrs of Intrusion Detection,
Handling
WHAT I DO FOR A LIVING
@shomiron
© Copyright 2017 NETMONASTERY Inc
Cyber Security — The NUMBERS!
4
THE CURRENT WORLD CHALLENGES
■ Data growth 230% YoY
■ Mid Sized Enterprise - 30 TB Monthly
■ Enterprise Grade - 100+ TB Monthly
■ Government - 1.23 PB Monthly
■ Process, Percolation / Lag < 10s
■ Detect stuff we didn’t know about
© Copyright 2017 NETMONASTERY Inc
Cyber Security Solutions — Where were we?
5
THE CURRENT WORLD
Security Information and Event Management
■ It’s… last generation
■ Slow… can’t scale (database)
■ Rule based - KNOWN / KNOWNs
■ Beautiful UI - HiFi
■ Center of the Earth
© Copyright 2017 NETMONASTERY Inc
6
We FAIL*
EVERYDAY!!
© Copyright 2017 NETMONASTERY Inc
Data Analytics + Big Data Structures
7
AND THE MARRIAGE PARTY
1. Facets and dimensionality
2. Static / dynamic analytics
3. Why static is better
4. Then why are we doing dynamic?
5. The Marriage of Big Data with Analytics
6. Hadoop…. anybody?
© Copyright 2017 NETMONASTERY Inc
Technical Aspects
8
HOW IT WORKS
1. Horizontal scaling - Speed
2. Pipeline, let’s revisit Hadoop?
3. Data preparation - Parsing / Annotation
4. Enrichment - localization for context
5. Plugin framework - SOAR - Enrich, Validate, Respond
6. AI / ML ;)
7. Platform + Content
© Copyright 2017 NETMONASTERY Inc
9
The PIPELINE
DATA FLOW WITHIN
COLLECT PARSE ANNOTATE ENRICH
VISUALIZE CORRELATE
INDEX
VALIDATE RESPOND REMEDIATE
© Copyright 2017 NETMONASTERY Inc
10
The PIPELINE
DATA FLOW WITHIN
ADAPTER
DATASTORE
CORRELATOR
© Copyright 2017 NETMONASTERY Inc
Talking about DNIF
11
AND COMPETITION
1. SIEM - ArcSight, QRadar
2. Splunk, ELK
3. Hadoop (Cloudera, Hortonworks), Tableau, HANA, Watson
4. Free Parsers, SOAR, Packages / Plugins
5. Power of Collaboration
© Copyright 2017 NETMONASTERY Inc
Opportunities - Gaps we’re aiming for
12
THE BUSINESS ANGLE, YAY!
1. Managed Security Services
2. Threat Hunting
3. Managed Detection Response
4. SWIFT, IMPS, NEFT, CC, Exchange, Telecom Switching
5. The Data Lake Theory - WORKBENCH
Thank You
Oh BTW We are hiring
Write to us - hello@dnif.it
13

Data Analytics in Cyber Security

  • 1.
    © Copyright 2017NETMONASTERY Inc Big Data Structures in the Cyber Security Puzzle 1 Shomiron DAS GUPTA, Founder, CEO NETMONASTERY Inc. Data Analytics in Cyber Security
  • 2.
    © Copyright 2017NETMONASTERY Inc Agenda … Setting Expectations! 2 NEXT 45 MINS 1. Cyber Security - How it works, today! 2. Data Analytics, the What and the Why 3. The Technical Aspects 4. The Pipeline 5. Opportunities - Gaps we’re aiming for 6. Demo
  • 3.
    © Copyright 2017NETMONASTERY Inc Hello, who is speaking? 3 Founder of NETMONASTERY, we built DNIF - An Integrated Threat Hunting Platform for the CSOC Research on Detection, Hunting and …. ML One of the few guys that does defense for a living GCIA 2000 - 18Yrs of Intrusion Detection, Handling WHAT I DO FOR A LIVING @shomiron
  • 4.
    © Copyright 2017NETMONASTERY Inc Cyber Security — The NUMBERS! 4 THE CURRENT WORLD CHALLENGES ■ Data growth 230% YoY ■ Mid Sized Enterprise - 30 TB Monthly ■ Enterprise Grade - 100+ TB Monthly ■ Government - 1.23 PB Monthly ■ Process, Percolation / Lag < 10s ■ Detect stuff we didn’t know about
  • 5.
    © Copyright 2017NETMONASTERY Inc Cyber Security Solutions — Where were we? 5 THE CURRENT WORLD Security Information and Event Management ■ It’s… last generation ■ Slow… can’t scale (database) ■ Rule based - KNOWN / KNOWNs ■ Beautiful UI - HiFi ■ Center of the Earth
  • 6.
    © Copyright 2017NETMONASTERY Inc 6 We FAIL* EVERYDAY!!
  • 7.
    © Copyright 2017NETMONASTERY Inc Data Analytics + Big Data Structures 7 AND THE MARRIAGE PARTY 1. Facets and dimensionality 2. Static / dynamic analytics 3. Why static is better 4. Then why are we doing dynamic? 5. The Marriage of Big Data with Analytics 6. Hadoop…. anybody?
  • 8.
    © Copyright 2017NETMONASTERY Inc Technical Aspects 8 HOW IT WORKS 1. Horizontal scaling - Speed 2. Pipeline, let’s revisit Hadoop? 3. Data preparation - Parsing / Annotation 4. Enrichment - localization for context 5. Plugin framework - SOAR - Enrich, Validate, Respond 6. AI / ML ;) 7. Platform + Content
  • 9.
    © Copyright 2017NETMONASTERY Inc 9 The PIPELINE DATA FLOW WITHIN COLLECT PARSE ANNOTATE ENRICH VISUALIZE CORRELATE INDEX VALIDATE RESPOND REMEDIATE
  • 10.
    © Copyright 2017NETMONASTERY Inc 10 The PIPELINE DATA FLOW WITHIN ADAPTER DATASTORE CORRELATOR
  • 11.
    © Copyright 2017NETMONASTERY Inc Talking about DNIF 11 AND COMPETITION 1. SIEM - ArcSight, QRadar 2. Splunk, ELK 3. Hadoop (Cloudera, Hortonworks), Tableau, HANA, Watson 4. Free Parsers, SOAR, Packages / Plugins 5. Power of Collaboration
  • 12.
    © Copyright 2017NETMONASTERY Inc Opportunities - Gaps we’re aiming for 12 THE BUSINESS ANGLE, YAY! 1. Managed Security Services 2. Threat Hunting 3. Managed Detection Response 4. SWIFT, IMPS, NEFT, CC, Exchange, Telecom Switching 5. The Data Lake Theory - WORKBENCH
  • 13.
    Thank You Oh BTWWe are hiring Write to us - hello@dnif.it 13