Downloaded 23 times
Uploaded byInfiniteGraph
PowerOfRelationshipsInBigData_SVNoSQL
The document discusses the challenges of big data analytics and how traditional SQL databases struggle with relationship management. It highlights Objectivity's solutions, including InfiniteGraph, which effectively handles complex connections between data entities using graph databases. The presentation outlines the importance of relationship analytics in various fields such as social network analysis, cybersecurity, and logistics.
More Related Content
Similar to PowerOfRelationshipsInBigData_SVNoSQL
![Support, Monitoring, Continuous Improvement & Scaling Agentic Automation [3/3]](https://cdn.slidesharecdn.com/ss_thumbnails/agenticcommunityseries-day3-cfd-251120170304-ddef8112-thumbnail.jpg?width=640&height=640&fit=bounds)
PowerOfRelationshipsInBigData_SVNoSQL
- 1. The Database The Powerof Relationships in Big Data Leon Guzenda - Objectivity, Inc. Silicon Valley NoSQL Meetup - 1/23/14 © Objectivity, Inc. 2014
- 2. Overview • The Problem •Current Big Data Analytics • Relationship Analytics • Leveraging NoSQL • Big Data Connection Platform • Solution Use Case Demo © Objectivity, Inc. 2014 !2
- 3. Objectivity, Inc. • Headquarteredin San Jose, CA • Over two decades of NoSQL and Big Data experience • Enables complex data virtualization and Big Data solutions for the enterprise • Software products: • Objectivity/DB • InfiniteGraph • InfiniteGraph Social App • Embedded in hundreds of enterprises, government organizations and products, with millions of deployments. © Objectivity, Inc. 2014 !3
- 4. A Typical Deployment ©Objectivity, Inc. 2014 !4
- 5. Current Big DataAnalytics © Objectivity, Inc. 2014 !5
- 6. The Problem Information Overload! •Making sense of it all takes time and $$$ • Which lead to a rush to Big Data Analytics © Objectivity, Inc. 2014 !6
- 7. Current Big DataAnalytics © Objectivity, Inc. 2014 !7
- 8. Leveraging NoSQL © Objectivity,Inc. 2014 !8
- 9. Not Only SQL- Four Main Technologies Simple © Objectivity, Inc. 2014 Highly Interconnected !9
- 10. Hadoop? Hadoop: Objectivity/DB & InfiniteGraph •Parallel processing using a divide and conquer or split and merge paradigm • Distributed processing with multithreading client processes and simple servers* • Sharded, distributed file system • Distributed, segmented Federated Database with a Single Logical View down to fine grain objects • Tuned for sequential scans and simple queries • Tuned for random access and powerful parallel queries • Not suitable for highly interconnected data sets (graphs) • Excel at handling very large graph structures with built-in relationship analytics * Process workflow could be driven using MapReduce © Objectivity, Inc. 2014 !10
- 11. Incremental Analytic ImprovementsAren’t Enough • All current solutions use the same basic architectural model. • None of the current solutions has an efficient way to store connections between entities in different silos. • Most analytic technology focuses on the content of the data nodes, rather than the many kinds of connections between the nodes and in those connections. • Why? Because most DBMSs are bad at handling relationships. • Object and Graph Databases can efficiently store, manage and query the many kinds of relationships hidden in the data. © Objectivity, Inc. 2014 !11
- 12. Relationship Analytics… A SQLShortcoming Table_A Table_B Table_C Table_D Table_E Table_F Table_G There are some kinds of complex relationship handling problems that SQL wasn't designed for. © Objectivity, Inc. 2014 !12
- 13. …Relationship Analytics A SQLShortcoming Table_A Table_B Table_C Table_D Table_E Table_F Table_G InfiniteGraph - The solution can be found with a few lines of code A3 © Objectivity, Inc. 2014 G4 !13
- 14. Graph Terminology ● VERTEX: Asingle node in a graph data structure ● EDGE: A connection between a pair of VERTICES ● PROPERTIES: Data items that belong to a particular Vertex or Edge ● WEIGHT: A quantity associated with a particular Edge ● GRAPH: A network of linked Vertex and Edge objects Vertex 1 City: San Francisco Pop: 812,826 © Objectivity, Inc. 2014 Edge 1 Road: I-101 Miles: 47.8 Vertex 2 City: San Jose Pop: 967,487 !14
- 15. Example 1 -Relationship Analytics MARKET ANALYSIS SOCIAL NETWORK ANALYSIS LOGISTICS HEALTHCARE INFORMATICS © Objectivity, Inc. 2014 !15
- 16. Finding The Links… Events/Places People/Orgs SituationX Combatant A A Called P A Seen Near X P Emailed S Situation Y Bank X P Called Q Q Seen Near T X Paid S Target T Civilian P P Called R R Seen Near T Cafe C Civilian Q A Banks at X S Seen Near T Facts Civilian R Civilian S © Objectivity, Inc. 2014 A Seen At Y A Eats At !16
- 17. …Finding The Links… EDGES VERTICES Events/Places People/Orgs Facts SituationX Combatant A A Called P A Seen Near X P Emailed S Situation Y Bank X P Called Q Q Seen Near T X Paid S Target T Civilian P P Called R R Seen Near T Cafe C Civilian Q A Banks at X S Seen Near T Civilian R Civilian S © Objectivity, Inc. 2014 A Seen At Y A Eats At !17
- 18. …Finding The Links… SituationX Seen Near Eats At Cafe C Combatant A Seen At Called Banks At Bank X Civilian P Called Civilian Q Called Emailed Civilian R Seen Near Seen Near Situation Y Paid Civilian S Seen Near Target T © Objectivity, Inc. 2014 !18
- 19. …Finding The Links… SituationX Seen Near Combatant A Seen At Called Situation Y Banks At SUSPECTS Bank X Civilian P Called Civilian Q Called Civilian R Seen Near Seen Near Target T © Objectivity, Inc. 2014 Emailed Paid Civilian S Seen Near NEEDS PROTECTION !19
- 20. …Finding The Links OTHER DATABASE(S) GRAPHDATABASE © Objectivity, Inc. 2014 !20
- 21. Example 2 -Finding Patterns in Open Source Data The Challenges ● Data Volumes ● Fast-Changing Data ● Sensitivity of Data ● Significance of Data © Objectivity, Inc. 2014 !21
- 22. Example 3 -Cybersecurity © Objectivity, Inc. 2014 !22
- 23. Big Data ConnectionPlatform © Objectivity, Inc. 2014 !23
- 24. Objectivity’s Disruptive BigData Architecture Uses Data Virtualization to hide the nodes and focus on the connections © Objectivity, Inc. 2014 !24
- 25. InfiniteGraph Distributed Parallel Loadand Queries Powerful Graph Queries X Start Start X Computational and Visualization Plugins Distributed Parallel Link Finding Latency Exceeded Start Start © Objectivity, Inc. 2014 Finish Custom Visualizer !25
- 26. Solution Use CaseDemo… Let’s see InfiniteGraph coupled with Oracle’s NoSQL Solution… © Objectivity, Inc. 2014 !26