ai based computer basic learning Lecture about Bigdata.ppt

Introduction to Big Data
Muhammad Asim Khan

Topics
 Scope: Big Data & Analytics
 Topics:
 Foundation of Data Analytics and Data Mining
 Hadoop/Map-Reduce Programming and Data Processing &
BigTable/Hbase/Cassandra
 Graph Database and Graph Analytics
2

What’s Big Data?
No single definition; here is from Wikipedia:
 Big data is the term for a collection of data sets so large and complex that it
becomes difficult to process using on-hand database management tools or
traditional data processing applications.
 The challenges include capture, curation, storage, search, sharing, transfer,
analysis, and visualization.
 The trend to larger data sets is due to the additional information derivable
from analysis of a single large set of related data, as compared to separate
smaller sets with the same total amount of data, allowing correlations to be
found to "spot business trends, determine quality of research, prevent
diseases, link legal citations, combat crime, and determine real-time roadway
traffic conditions.”
3

Volume (Scale)
 Data Volume
 44x increase from 2009 2020
 From 0.8 zettabytes to 35zb
 Data volume is increasing exponentially
5
Exponential increase in
collected/generated data

12+ TBs
of tweet data
every day
25+ TBs of
log data
every day
?
TBs
of
data
every
day
2+
billion
people on
the Web
by end
2011
30 billion RFID
tags today
(1.3B in 2005)
4.6
billion
camera
phones
world wide
100s of
millions
of GPS
enabled
devices sold
annually
76 million smart meters
in 2009…
200M by 2014

Maximilien Brice, © CERN
CERN’s Large Hydron Collider (LHC) generates 15 PB a year

The Earthscope
The Earthscope is the world's largest science
project. Designed to track North America's
geological evolution, this observatory records
data over 3.8 million square miles, amassing 67
terabytes of data. It analyzes seismic slips in the
San Andreas fault, sure, but also the plume of
magma underneath Yellowstone and much, much
more.
(http://www.msnbc.msn.com/id/44363598/ns/tec
hnology_and_science-
future_of_technology/#.TmetOdQ--uI)

Variety (Complexity)
 Relational Data (Tables/Transaction/Legacy Data)
 Text Data (Web)
 Semi-structured Data (XML)
 Graph Data
 Social Network, Semantic Web (RDF), …
 Streaming Data
 You can only scan the data once
 A single application can be generating/collecting many
types of data
 Big Public Data (online, weather, finance, etc)
9
To extract knowledge all these types of
data need to linked together

A Single View to the Customer
Customer
Social
Media
Gamin
g
Entertain
Bankin
g
Financ
e
Our
Known
History
Purchas
e

Velocity (Speed)
 Data is begin generated fast and need to be processed fast
 Online Data Analytics
 Late decisions  missing opportunities
 Examples
 E-Promotions: Based on your current location, your purchase
history, what you like  send promotions right now for store next
to you
 Healthcare monitoring: sensors monitoring your activities and
body  any abnormal measurements require immediate reaction
11

Real-time/Fast Data
 The progress and innovation is no longer hindered by the ability to collect data
 But, by the ability to manage, analyze, summarize, visualize, and discover knowledge from
the collected data in a timely manner and in a scalable fashion
12
Social media and networks
(all of us are generating data)
Scientific instruments
(collecting all sorts of data)
Mobile devices
(tracking all objects all the time)
Sensor technology and networks
(measuring all kinds of data)

Real-Time Analytics/Decision Requirement
Customer
Influence
Behavior
Product
Recommendations
that are Relevant
& Compelling
Friend Invitations
to join a
Game or Activity
that expands
business
Preventing Fraud
as it is Occurring
& preventing more
proactively
Learning why Customers
Switch to competitors
and their offers; in
time to Counter
Improving the
Marketing
Effectiveness of a
Promotion while it
is still in Play

Harnessing Big Data
 OLTP: Online Transaction Processing (DBMSs)
 OLAP: Online Analytical Processing (Data Warehousing)
 RTAP: Real-Time Analytics Processing (Big Data Architecture & technology)
15

The Model Has Changed…
 The Model of Generating/Consuming Data has Changed
16
Old Model: Few companies are generating data, all others are consuming data
New Model: all of us are generating data, and all of us are consuming data

What’s driving Big Data
17
- Ad-hoc querying and reporting
- Data mining techniques
- Structured data, typical sources
- Small to mid-size datasets
- Optimizations and predictive analytics
- Complex statistical analysis
- All types of data, and many sources
- Very large datasets
- More of a real-time

Big Data:
Batch Processing &
Distributed Data Store
Hadoop/Spark;
HBase/Cassandra
BI Reporting
OLAP &
Dataware house
Business Objects, SAS,
Informatica, Cognos other
SQL Reporting Tools
Interactive
Business
Intelligence &
In-memory RDBMS
QliqView, Tableau, HANA
Big Data:
Real Time &
Single View
Graph Databases
The Evolution of Business Intelligence
1990’s 2000’s 2010’s
Speed
Scale
Scale
Speed

Big Data Analytics
 Big data is more real-time in
nature than traditional DW
applications
 Traditional DW architectures (e.g.
Exadata, Teradata) are not well-
suited for big data apps
 Shared nothing, massively parallel
processing, scale out architectures
are well-suited for big data apps
19

ai based computer basic learning Lecture about Bigdata.ppt

More Related Content

Similar to ai based computer basic learning Lecture about Bigdata.ppt

More from ALAMGIRHOSSAIN256982

Recently uploaded

ai based computer basic learning Lecture about Bigdata.ppt