Download as PDF, PPTX
Uploaded byconfluent
Data Driven Enterprise with Apache Kafka
The document discusses real-time streaming with Apache Kafka. It introduces Bill McLaren, a streaming data engineer at Confluent. It then provides background on Confluent, which was founded by the creators of Apache Kafka and has many Kafka committers. The rest of the document discusses challenges with traditional infrastructure-driven systems and how a data-driven approach using Apache Kafka's distributed event streaming platform can help by processing and storing streams of data in real-time.
More Related Content
Similar to Data Driven Enterprise with Apache Kafka
Data Driven Enterprise with Apache Kafka
- 1. © 2018, AmazonWeb Services, Inc. or its affiliates. All rights reserved. Bill McLaren Streaming Data Engineer, Confluent Data Driven Enterprise Real-time Streaming with Apache Kafka®
- 2. © 2018, AmazonWeb Services, Inc. or its affiliates. All rights reserved. 80% of active Kafka Committers Founded September 2014 Technology developed while at LinkedIn Founded by the creators of Apache Kafka About Confluent™ Jay Kreps CEO Cheryl Dalrymple CFO Todd Barnett VP WW Sales Neha Narkhede CTO, VP Engineering
- 3. © 2018, AmazonWeb Services, Inc. or its affiliates. All rights reserved. Infrastructure Driven Enterprise
- 4. © 2018, AmazonWeb Services, Inc. or its affiliates. All rights reserved. Challenges In Infrastructure Driven Enterprise Divergent • “Normalization” • Combined & Derived • Which one is the source of truth? • What happens if any attributes change? Duplicate • Applications • Orgs • Data centers Slow • Batch processes • Multiple hops add up • Slow in propagating changes Serial • How many hops? • How many different processes? • What happens if one breaks
- 5. © 2018, AmazonWeb Services, Inc. or its affiliates. All rights reserved. What is the cost Agility Cost Success
- 6. © 2018, AmazonWeb Services, Inc. or its affiliates. All rights reserved. Data Driven Enterprise Collapse the space between the data and the mission Time | Distance
- 7. © 2018, AmazonWeb Services, Inc. or its affiliates. All rights reserved. Data Driven Enterprise Think Event Streaming
- 8. © 2018, AmazonWeb Services, Inc. or its affiliates. All rights reserved. Apache Kafka: A Distributed Event Streaming Platform
- 9. © 2018, AmazonWeb Services, Inc. or its affiliates. All rights reserved. Data Driven Enterprise You have probably seen a diagram like that before
- 10. © 2018, AmazonWeb Services, Inc. or its affiliates. All rights reserved. Apache Kafka: An Event Streaming Platform Process streams of data in real time, as they occur 110101 010111 001101 100010 Publish and subscribe to streams of data similar to a message queue or enterprise messaging system 110101 010111 001101 100010 Store streams of data in a distributed highly available fault tolerant way 110101 010111 001101 100010
- 11. © 2018, AmazonWeb Services, Inc. or its affiliates. All rights reserved. Open Open Source Multi Cloud & On Premise Explosive Ecosystem
- 12. © 2018, AmazonWeb Services, Inc. or its affiliates. All rights reserved. Confluent Platform: a Streaming Platform based on Apache Kafka
- 13. © 2018, AmazonWeb Services, Inc. or its affiliates. All rights reserved. Please complete the session survey in the summit mobile app.
- 14. © 2018, AmazonWeb Services, Inc. or its affiliates. All rights reserved. Thank you!