Download as PDF, PPTX
![© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
My [insert your favorite DB] works for everything
General purpose Special purpose
One size fits all Efficiency at scale](https://image.slidesharecdn.com/dat305-how-to-choose-the-right-e1f2a414-97b1-47e6-8af4-683b683766e5-1743316450-190321091253/75/Building-with-AWS-Databases-Match-Your-Workload-to-the-Right-Database-AWS-Summit-Tel-Aviv-2019-11-2048.jpg)
![© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Indexing and
storing
documents with
support
for query on
any attribute
Self Contained
Flexible Schema
Document
Amazon
DocumentDB
User profiles
{
id: 181276,
username: "sue1942",
name: {first: "Susan",
last: "Benoit"}
}
{
id: 181276,
username: "sue1942",
name: {first: "Susan",
last: "Benoit"},
ExplodingAcorns: {
hi_score: 3185400,
global_rank: 5139,
bonus_levels: true
},
promotions: ["new user","5%",”acorn"]
}](https://image.slidesharecdn.com/dat305-how-to-choose-the-right-e1f2a414-97b1-47e6-8af4-683b683766e5-1743316450-190321091253/75/Building-with-AWS-Databases-Match-Your-Workload-to-the-Right-Database-AWS-Summit-Tel-Aviv-2019-22-2048.jpg)
![© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
My [insert your favorite DB] works for everything
General purpose Special purpose
One size fits all Efficiency at scale](https://crownmelresort.com/image.slidesharecdn.com/dat305-how-to-choose-the-right-e1f2a414-97b1-47e6-8af4-683b683766e5-1743316450-190321091253/75/Building-with-AWS-Databases-Match-Your-Workload-to-the-Right-Database-AWS-Summit-Tel-Aviv-2019-11-2048.jpg)
![© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Indexing and
storing
documents with
support
for query on
any attribute
Self Contained
Flexible Schema
Document
Amazon
DocumentDB
User profiles
{
id: 181276,
username: "sue1942",
name: {first: "Susan",
last: "Benoit"}
}
{
id: 181276,
username: "sue1942",
name: {first: "Susan",
last: "Benoit"},
ExplodingAcorns: {
hi_score: 3185400,
global_rank: 5139,
bonus_levels: true
},
promotions: ["new user","5%",”acorn"]
}](https://crownmelresort.com/image.slidesharecdn.com/dat305-how-to-choose-the-right-e1f2a414-97b1-47e6-8af4-683b683766e5-1743316450-190321091253/75/Building-with-AWS-Databases-Match-Your-Workload-to-the-Right-Database-AWS-Summit-Tel-Aviv-2019-22-2048.jpg)
Uploaded byAWS Summits
Building with AWS Databases: Match Your Workload to the Right Database | AWS Summit Tel Aviv 2019
The document discusses how to effectively match workloads to the appropriate Amazon Web Services (AWS) databases by considering factors such as database shape, size, and compute requirements. It highlights the importance of purpose-built databases for optimal application performance and scalability, emphasizing the varied use cases of different database types. Additionally, it provides insights on real-world applications, like Duolingo, which leverage AWS databases to support large-scale user interactions and data processing.
More Related Content
Similar to Building with AWS Databases: Match Your Workload to the Right Database | AWS Summit Tel Aviv 2019
Building with AWS Databases: Match Your Workload to the Right Database | AWS Summit Tel Aviv 2019
- 2. © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved.S U M M I T Building With AWS Databases: Match Your Workload To The Right Database Ronnen Slasky Head of Technology, AWS D A T 3 0 5
- 3. © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved.S U M M I T 1. Database Considerations 2. Common data models/use cases 3. Demo 4. Ledger Databases Agenda
- 4. © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved.S U M M I T Why did you choose this database? “Because we heard X is the best new thing.” “Because we have a site license for X.” “Because X is what we know how to use.”
- 5. © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved.S U M M I T Why should you choose this database? “Because this database is purpose built to support what my application is designed to do.”
- 6. © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved.S U M M I T A common question that I get is why do we offer so many database products? The answer for me is simple: Developers want their applications to be well architected and scale effectively. To do this, they need to be able to use multiple databases and data models within the same application. https://www.allthingsdistributed.com/2018/06/purpose-built-databases-in-aws.html
- 7. © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved.S U M M I T Database Workloads Data Considerations Shape Size Compute
- 8. © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved.S U M M I T Shape Purpose-Built For Optimized for When you need to Example Workload Row Store Operate on a record or group of records Payroll Column Store Aggregations, scans and joins Analytics Key-Value Store Query by key with high throughput & fast ingestion Tracking devices Document Store Index & store documents for query on any property Patient data Graph Store Persist and retrieve relationships Recommendations Time-Series Store Store and process data sequence Process Engine telemetry Unstructured Store Get and put of objects Store user reviews
- 9. © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved.S U M M I T Size Considerations Example Workload Size at limit – bounded or unbounded Number of employees – bounded Number of sensors – unbounded Working set size & caching 10-years of sales data but only the last 12-months is queried Session data for users of a streaming service Retrieval size Get one row Get one thousand rows Partition-able or monolithic Storage and processing of car location data is partition-able Company payroll data has no natural partition boundary
- 10. © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved.S U M M I T Compute Considerations Example Workload Compute functions Sum of sales for the last 12-months Get & Put data Rate of ingestion Location telemetry from cars added to the database every minute New employees records being added to the database Change rate Inventory counts are frequently updated Sales records are never updated Throughput Million users browsing a product catalogue every second 50 doctors looking at 300 patient records per day Latency Get the location of a car in 5 milliseconds Get the min, max & average deal size for the last 12-months in 5 seconds
- 11. © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved.S U M M I T My [insert your favorite DB] works for everything General purpose Special purpose One size fits all Efficiency at scale
- 12. © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved.S U M M I T Modern apps create new requirements Users: 1 million+ Data volume: TB–PB–EB Locality: Global Performance: Milliseconds–microseconds Request rate: Millions Access: Web, mobile, IoT, devices Scale: Up-down, Out-in Economics: Pay for what you use Developer access: No assembly requiredSocial mediaRide hailing Media streaming Dating
- 13. © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved.S U M M I T CHALLENGE Wanted to enable anyone to learn a language for free. SOLUTION Purpose-built databases from AWS: • DynamoDB: 31B items tracking which language exercises completed • Aurora: primary transactional database for user data • ElastiCache: instant access to common words and phrases Result: More people learning a language on Duolingo than entire US school system 300M total users 7B exercises per month
- 14. © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved.S U M M I T Relational Key-value Document In-memory Graph Referential integrity, ACID transactions, schema- on-write Low-latency, key lookups with high throughput and fast ingestion of data Indexing and storing documents with support for query on any attribute Microseconds latency, key- based queries, and specialized data structures Creating and navigating data relations easily and quickly Lift and shift, EMR, CRM, finance Real-time bidding, shopping cart, social Content management, personalization, mobile Leaderboards, real-time analytics, caching Fraud detection, social networking, recommendation engine Search Indexing and searching semistructured logs and data Product catalog, help and FAQs, full text Time-series Ledger Collect, store, and process data sequenced by time IoT applications, event tracking Complete, immutable, and verifiable history of all changes Systems of record, supply chain, health care, registrations, financial Common data categories and use cases
- 15. © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved.S U M M I T Relational Key-value Document In-memory Graph Search Amazon DynamoDB Amazon Neptune Amazon RDS Amazon Aurora CommercialCommunity Amazon ElastiCache Amazon Elasticsearch Service Amazon DocumentDB Time-series Ledger Amazon Timestream Amazon Quantum Ledger DatabaseMemcachedRedis AWS purpose-built databases
- 16. S U MM I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Relational Key-value Document Graph
- 17. © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved.S U M M I T Patient * Patient ID First Name Last Name Gender DOB * Doctor ID Visit * Visit ID * Patient ID * Hospital ID Date * Treatment ID Medical Treatment * Treatment ID Procedure How Performed Adverse Outcome Contraindication Doctor * Doctor ID First Name Last Name Medical Specialty * Hospital Affiliation Hospital * Hospital ID Name Address Rating Relational Referential integrity, ACID transactions, schema-on-write Lift and shift, ERP, CRM, finance Amazon Aurora Amazon RDS Amazon Redshift
- 18. © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved.S U M M I T Patient * Patient ID First Name Last Name Gender DOB * Doctor ID Visit * Visit ID * Patient ID * Hospital ID Date * Treatment ID Medical Treatment * Treatment ID Procedure How Performed Adverse Outcome Contraindication Doctor * Doctor ID First Name Last Name Medical Specialty * Hospital Affiliation Hospital * Hospital ID Name Address Rating // Number of patient visits each doctor completed last week SELECT d.first_name, d.last_name, count(*) FROM visit as v, hospital as h, doctor as d WHERE v.hospital_id = h.hospital_id AND h.hospital_id = d.hospital AND v.t_date > date_trunc('week’, CURRENT_TIMESTAMP - interval '1 week') GROUP BY d.first_name, d.last_name; Relational Referential integrity, ACID transactions, schema-on-write
- 19. © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved.S U M M I T Table 1 Table 2 Table N … … … … … … Partitions Partitions Partitions … … … Highly partitionable data Low-latency, key look-ups with high throughput and fast ingestion of data Real-time bidding, shopping cart, IoT device tracking Key-value Amazon DynamoDB
- 20. © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved.S U M M I T Gamers Primary Key Attributes Gamer Tag Type Hammer57 Rank Level Points Tier 87 4050 Elite Status Health Progress 90 30 Weapon Class Damage Range Taser 87% 50 FluffyDuffy Rank Level Points Tier 5 1072 Trainee Status Health Progress 37 8 // Status of Hammer57 GET { TableName:"Gamers", Key: { "GamerTag":"Hammer57", "Type":"Status” } } // Return all Hammer57 Gamers GamerTag = :a :a Hammer57 Key-value
- 21. © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved.S U M M I T Document (client) (app) (database) JSON !JSON != Indexing and storing documents with support for query on any attribute Content management, personalization, mobile Amazon DocumentDB
- 22. © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved.S U M M I T Indexing and storing documents with support for query on any attribute Self Contained Flexible Schema Document Amazon DocumentDB User profiles { id: 181276, username: "sue1942", name: {first: "Susan", last: "Benoit"} } { id: 181276, username: "sue1942", name: {first: "Susan", last: "Benoit"}, ExplodingAcorns: { hi_score: 3185400, global_rank: 5139, bonus_levels: true }, promotions: ["new user","5%",”acorn"] }
- 23. © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved.S U M M I T Graph Creating and navigating relations between data easily and quickly Fraud detection, social networking, recommendation engine Amazon Netptune
- 24. © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved.S U M M I T PURCHASED PURCHASED FOLLOWS PURCHASED KNOWS PRODUCT SPORT FOLLOWS Graph Creating and navigating relations between data easily and quickly Fraud detection, social networking, recommendation engine Amazon Netptune
- 25. © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved.S U M M I T // Product recommendation to a user gremlin> V().has(‘name’,’sara’).as(‘customer’).out(‘follows’).in(‘follows’).out(‘purchased’) ( (‘customer’)).dedup() (‘name’) ('name') KNO W S PURCHASED PURCHASED FOLLOWS PURCHASED KNOWS PRODUCT SPORT FOLLOWS FOLLOWS // Identify a friend in common and make a recommendation gremlin> g.V().has('name','mary').as(‘start’). both('knows').both('knows’). where(neq(‘start’)). dedup().by('name').properties('name') Graph
- 26. © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved.S U M M I T © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved. What museums should Alice visit while in Paris? Who painted the Mona Lisa? What artists have paintings in The Louvre? Graph
- 27. S U MM I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
- 28. © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved.S U M M I T Retail demo application Demo application: 1. Available today 2. On GitHub: /aws-samples/aws- bookstore-demo-app 3. One click CloudFormation deployment Search Indexing and searching semistructured logs and data Product search Amazon Neptune Amazon Elasticsearch Service Key-value High throughput, Low-latency reads and writes, endless scale Shopping cart, user profile Graph Quickly and easily create and navigate relationships between data Product recommendation In-memory Query by key with microsecond latency Product leaderboard DynamoDB ElastiCache
- 29. © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved.S U M M I T Retail demo application
- 30. S U MM I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
- 31. © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved.S U M M I T Amazon Quantum Ledger Database (QLDB) (Preview) Fully managed ledger database Track and verify history of all changes made to your application’s data Immutable Maintains a sequenced record of all changes to your data, which cannot be deleted or modified; you have the ability to query and analyze the full history Cryptographically verifiable Uses cryptography to generate a secure output file of your data’s history Easy to use Easy to use, letting you use familiar database capabilities like SQL APIs for querying the data Highly scalable Executes 2–3X as many transactions than ledgers in common blockchain frameworks
- 32. S U MM I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
- 33. © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved.S U M M I T What have we learned? Choose the right tool for the job Start from the application and work backwards Focus on shape, size and compute Demo Application On GitHub: /aws-samples/aws-bookstore-demo-app Keep on Building !
- 34. © 2019, AmazonWeb Services, Inc. or its affiliates. All rights reserved.S U M M I T Thank you! S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Ronnen Slasky Head of Technology Amazon Web Services http://bit.ly/2SGYHp4