Download to read offline
Uploaded byNETUserGroupBern
Rest Fundamentals
This document discusses different data processing models and their characteristics for structured and unstructured data at varying scales. It compares batch processing, interactive analysis, and stream processing in terms of query runtime, data volume handled, programming models, typical users, and examples of originating projects and open source implementations. It then provides examples of how organizations can gain competitive advantage by analyzing different types of data using these approaches.
More Related Content
Rest Fundamentals
- 1. PLATFORM FOR BIGDATA, NOSQL AND RELATIONAL DATA. WHAT MAKES SENSE FOR ME? (+AZURE) michael.epprecht@microsoft.com
- 5.
- 10.
- 11. Batch Processing InteractiveAnalysis Stream Processing Query runtime Minutes to hours Milliseconds to minutes Never-ending Data volume TBs to PBs GBs to PBs Continuous stream Programming model MapReduce Queries DAG Users Developers Analysts and developers Developers Originating project Google MapReduce Google Dremel Twitter Storm Open source project Hadoop / Spark Drill / Shark / Impala Hbase Storm / Apache S4 /Kafka
- 14. How do Ioptimize my fleet based on weather and traffic patterns? What’s the social sentiment for my brand or products How do I better predict future outcomes?
- 15. GAIN COMPETITIVE ADVANTAGEBY MOVING FIRST AND FAST IN YOUR INDUSTRY Web app optimization Smart meter monitoring Equipment monitoring Advertising analysis Life sciences research Fraud detection Healthcare outcomes Weather forecasting Natural resource exploration Social network analysis Churn analysis Traffic flow optimization IT infrastructure optimization Legal discovery
- 20. persistent | distributed •In Memory • Efficient at Random Reads/Writes • Distributed, large scale data store • Utilizes Hadoop for persistence • Both HBase and Hadoop are distributed
- 21.
- 30. MANAGE ANY DATA,ANY SIZE, ANYWHERE 010101010101010101 1010101010101010 01010101010101 101010101010
- 32. Distributed Storage (HDFS) Hadoop architecture. DistributedProcessing (Map Reduce)
- 33. INSIGHTS FOR ALLUSERS THROUGH FAMILIAR TOOLS PB TB GB
- 36. An object containedwithin a user database Defines the scheme for the federation Represent the database being sharded Database that houses the federation object System managed SQL databases Contain part, or “slices” of data Orders_federation Orders_federation CREATE FEDERATION fed_name(fed_key_label fed_key_type distribution_type)
- 37. Orders_federation Orders_federation The keyused for data distribution int, bigint, guid, varbinary Represent a single instance of a federation key. All rows in all federated tables with the same federation key value. PK=5 PK=25 PK=35 PK=5 PK=25 PK=35
- 38. Contains only atomicunits for member’s key range Non-Federated table
- 39. SalesDB Orders_federation Orders_federation Orders_Fed [5000, 10000) ALTER FEDERATIONOrders_Fed SPLIT AT (tenant_id=7500) [5000, 7500) & [7500, 10000) SPLIT members to spread workloads over to more nodes DROP members to shrink back to fewer nodes
- 40. SalesDB Orders_federation Orders_federation Orders_Fed [5000, 7500) &[7500, 10000) USE FEDERATION Orders_Fed (tenant_id=7509) Ensure apps can discover where the data is just-in-time No “Shard Map” caching Guaranteed member routing
- 43.
- 49. MANAGE ANY DATA,ANY SIZE ANYWHERE 1010101010101010 01010101010101 101010101010 Hadoop Connectors & ETL
- 50. virtual machines websites cloud services SQL database noSQL database blob storage connect virtual network traffic manager . . . Frameworks Services Fabric Infrastructure N Central US, S Central US, N Europe, W Europe, E Asia, SE Asia + 24 Edge CDN Locations . . . . . . . . . . . . . . . Automated Managed Resources Elastic Usage Based