How Apache Kafka helps to create Data Culture – How to Cross the Kafka Chasm

CROSSING THE KAFKA CHASM
GET YOUR ORGANISATION TO ADOPT EVENT-THINKING
Andreas Grütter | Head Architecture | Mobiliar
Patrick Bönzli | Co-Founder, CEO | SPOUD

The event-architecture vision of Mobiliar
Main challenges on our Kafka journey
A solution to scale event-thinking inside Mobiliar
WELCOME!
Andreas Grütter
Head IT + Architecture
Patrick Bönzli
Co-Founder + CEO

Four Levers Leading to Outstanding Results Since 2010
Profitable growth more than 800
millions Swiss francs above
market average
Nearly 314,000 new customers
gained since 2010.
Overall, around 46 % of our
policyholders recommend us to
others.
The average length of service of
Swiss Mobiliar's employees is 11
years.
Above-markt growth More customers Satisfied customers Loyal staff members
By comparison: more than all
non-life competitors together.
By comparison: Berne, Lausanne
and Neuchâtel together have around
314,000 inhabitants.
By comparison: Our competitors
are recommended by 22 % of their
customers.
Very low fluctuation rate
of around 5.5 %.

Also Known For..
chicken chicken
me

Digital Transformation - Our Way to the Future

The World of Insurance Is Event-Driven
Our business evolves around change in the lives of our customers, their
posessions and how they interact with a changing world.
♀

+ +
React to the World as It Unfolds

4 Starting Points for Event-Architecture
Synchronization
Sourcing
#event-database
Save data as a sequence
of events.
#integration-architecture
Decoupling through streams as
a pattern of integration.
#real-time-processing
Act on processed events
in real-time
#microservice
Synchronize actions between
distributed functions.
Legacy
World
Future
World

Challenges of an Enterprise Kafka Journey
Kafka in Production
Kafka Secured
Organisational Scaling
Transparency
+ Convenience
1
Maslov’s Pyramid of Kafka Needs
more
organisational
challenges
Kafka as a Service
Data as a Product
Stream as a Service
Growingwithpeopleanddata

Event-Streaming Vision of Mobiliar
Implementing event-driven architecture where business-
events are becoming the central nervous system.1
Decouple software components and teams, matching
our agile setup (>40 teams, 800 employees).2
Pioneering ‘Data as a Product’ culture in the event-
streaming subdomain.3

Kafka Setup in Mobiliar Today
Event-driven integration architecture
Digital
Marketplace

Kafka Use-Case Example
Real-Time Customer Analytics for Personalized Ads and Leads

Challenges of Organisational Scaling of Kafka
How do we onboard teams to scale
event-architectures.
Without additional transparency
and clear guidelines you end up in
chaos.

Chasm
Innovators
Early
Adopters Early Majority Late Majority Laggards
The Kafka Chasm in Enterprises
Tech-savvy people with a
bias for new technology.
These people want
complete solutions
and convenience.
Early-Adopters love
promising new stuff
Early-Majority needs visible,
practical benefits.

How
Early-Adopters
perceives Kafka
How
Early-Majority
perceives Kafka
Kafka Cluster
Topic A
Topic B
Topic A
Partition 1
Partition 2
Producers
Producer 1
Producer 2 MSG
Position Fleet Position Stream
Temperature Weather Stream
Wow, awesome technology! It’s a data subscription service!

How
Early-Adopters
perceives Kafka
How
Early-Majority
perceives Kafka
It’s a data subscription service!
How
Business
perceives Kafka
It’s data, people and use-cases!Wow, nice technology!
♀ ♀ ♂
Responsibilities
Business Cases
Producers
Producer 1
Producer 2
Kafka Cluster
Topic A
Topic B
Topic A
Partition 1
Partition 2
MSG
Position Fleet Position Stream

Data as a product.
We enable a
customer-oriented
view on data for a
subscription-based
technology.

Agoora Creates Data Customer Experience
As business I would like
to find + understand data.
As DevOps I would like to
access data in a simple way.
As DevOps I would want a high-
quality data product and service.
Simplify
access to
Data Products
Quality of
Data
and Service
Catalog
of Data
Products
for a ‘Data as a product’ culture

Scaling Event-Architecture in Mobiliar
We decided to integrate Agoora
To crate more transparency
To onboard teams to scale event-driven architecture

Integration Architecture of Agoora

Transparency – “Seeing is believing”
Bridging visibility gap between business and IT. Business
knows domain, business-cases and people. And this is
how we present data.
Domain
Business Cases
Data ♀ Responsibilities
#it-architects
#business

Convenience and Automation
We have built convenience layers around Kafka, incrementally abstracting
technology complexity.
Raw Kafka
#it-architects
#dev-ops
Customized Adaptors
Tool Ecosystem
Agoora
Promethius
Grafana
Promethius
Schema Registry
Kubernetes
Github
Control Center

Agoora
Custom
Actions
Automation
Automation to build a streams as a service
Create new Kafka topic.
Request permission to consume.
Request permission to consume.
Handle failure and risks.
Agoora
Custom
ActionsCustom
Actions
Custom
Actions
#dev-ops
Custom
Actions
Custom
Actions

Agoora is helping us to create the necessary transparency to onboard
new teams and scaling event-driven architectures in Mobiliar.

We are looking for innovative companies to join
our beta-testing community.
Join our beta-testing on agoora.com

How Apache Kafka helps to create Data Culture – How to Cross the Kafka Chasm

More Related Content

What's hot

Similar to How Apache Kafka helps to create Data Culture – How to Cross the Kafka Chasm

More from confluent

Recently uploaded

How Apache Kafka helps to create Data Culture – How to Cross the Kafka Chasm