Pooja Rallabhandi
04-29-2019
Pooja Rallabhandi
04-29-2019
Pooja Rallabhandi
§ Software Engineer @ Ticketmaster, Scottsdale
§ Work on Web and Stream based applications
§ Passionate about Women in Technology
§ LinkedIn : www.linkedin.com/in/poojaralla
§ Twitter: @pzralla Confluent CTO, Neha Narkhede @
Kafka Summit 2018
DATA
STORE
CLIENT A
Create Concert
CLIENT B
Sell Inventory
on Concert
CLIENT C
Create Inventory
for Concert
A
P
I
G
A
T
E
W
A
Y
Concert APIs
INVENTORY APIs
Poor performance Increase in Dependencies
Data Not Available
Data Server
Copyright © 2015 -2018 Confluent, Inc. All rights reserved
§ A data log that keeps track of changes as they happen
§ Maintains the order in which events occurred
§ Data is persisted in the log
§ Clean-up is configurable
§ An approach to reactive programming
§ Process data as it is being generated
§ Helps in real-time decision making
Copyright © 2015 -2018 Confluent, Inc. All rights reserved
Features
§ Distributed streaming platform
§ Data persistence
§ High throughput
§ Low Latency
Origin
§ Created @ LinkedIn in 2010
§ Open source, top-level Apache project since 2012
§ Confluent provides an Enterprise version
§ Messages as Key/Value pair
§ Topics
§ Partitions
§ Data Retention
§ Delete
§ Compact
CENTRALIZED
DATA STORE
Copyright © LinkedIn. All rights reserved
Copyright © Ticketmaster. All rights reserved
BEFORE NOW
PRODUCER
APPLICATION
DATA
TRANSFORMATION
map(), filter()
CONSUMER
APPLICATION
PRODUCER
APPLICATION
CONSUMER
APPLICATIONDATA
SOURCE
DATA SINKDATA COPY
§ Kafka streams commit logs from a system into Topics
§ Uses pull based model
§ High throughput and low latency
§ KStreams API for data transformation
§ Kafka Connect for data transfer
§ Download Apache Kafka: https://kafka.apache.org/downloads
§ Quickstart guide: https://kafka.apache.org/quickstart
§ Tutorials:
https://www.tutorialspoint.com/apache_kafka/
https://docs.confluent.io/current/tutorials/index.html
§ Confluent published Use Cases and Resources
https://www.confluent.io/resources/
Introduction to Kafka Streams

Introduction to Kafka Streams