Setting Up Apache Kafka
Local & Cluster Mode
Kafka Architecture Overview
• Brokers
• Producers
• Consumers
• Topics & Partitions
• Zookeeper (optional for Kafka <
2.8)
+91-96400 01789
contact@accentfuture.com
Kafka Setup – Local Mode
• Ideal for dev/testing
• Single-node Kafka + Zookeeper
• Basic requirements (Java, Kafka binaries)
🧩 Steps:
1.Download Kafka
2.Start Zookeeper
3.Start Kafka broker
4.Create a topic
5.Produce and consume messages
+91-96400 01789
contact@accentfuture.com
Kafka Local Mode – Demo
Commands
+91-96400 01789
contact@accentfuture.com
Kafka Setup – Cluster Mode
• Multi-node Kafka setup
• Zookeeper ensemble (or KRaft mode for
2.8+)
• Brokers across machines
• Use cases: high availability, production
workloads
+91-96400 01789
contact@accentfuture.com
Kafka Cluster – Architecture
Diagram
• 3 Kafka brokers
• 3 Zookeeper nodes
• Producers & consumers
+91-96400 01789
contact@accentfuture.com
Kafka Cluster Setup – Steps
•Set up multiple servers/VMs
•Configure unique broker.id, log.dirs,
listeners
•Point all brokers to same Zookeeper
•Start all brokers
•Verify with topic creation/replication
+91-96400 01789
contact@accentfuture.com
Kafka Configuration Files
server.properties key
settings:
• broker.id
• log.dirs
• zookeeper.connect
• listeners
+91-96400 01789
contact@accentfuture.com
Testing the Cluster
• Create topic with replication
factor > 1
• Produce messages
• Consume from multiple
consumers
• Simulate broker failure
+91-96400 01789
contact@accentfuture.com
Monitoring & Tools
• Kafka Manager /
AKHQ
• Prometheus +
Grafana
• Log monitoring
+91-96400 01789
contact@accentfuture.com
Common Issues & Fixes
• Port conflicts
• Zookeeper sync
issues
• Log retention
• Broker
unavailability
+91-96400 01789
contact@accentfuture.com
Summary & Best Practices
• Local = dev/test, Cluster =
production
• Keep configs modular
• Secure your cluster (SSL/SASL,
ACLs)
• Monitor everything
+91-96400 01789
contact@accentfuture.com
Contact Details
šŸ“§ contact@accentfuture.com
🌐 AccentFuture
šŸ“ž +91-96400 01789
KAFKA ONLINE TRAINING
+91-96400 01789
contact@accentfuture.com

Apache Kafka | Apache Kafka online training

  • 1.
    Setting Up ApacheKafka Local & Cluster Mode
  • 2.
    Kafka Architecture Overview •Brokers • Producers • Consumers • Topics & Partitions • Zookeeper (optional for Kafka < 2.8) +91-96400 01789 contact@accentfuture.com
  • 3.
    Kafka Setup –Local Mode • Ideal for dev/testing • Single-node Kafka + Zookeeper • Basic requirements (Java, Kafka binaries) 🧩 Steps: 1.Download Kafka 2.Start Zookeeper 3.Start Kafka broker 4.Create a topic 5.Produce and consume messages +91-96400 01789 contact@accentfuture.com
  • 4.
    Kafka Local Mode– Demo Commands +91-96400 01789 contact@accentfuture.com
  • 5.
    Kafka Setup –Cluster Mode • Multi-node Kafka setup • Zookeeper ensemble (or KRaft mode for 2.8+) • Brokers across machines • Use cases: high availability, production workloads +91-96400 01789 contact@accentfuture.com
  • 6.
    Kafka Cluster –Architecture Diagram • 3 Kafka brokers • 3 Zookeeper nodes • Producers & consumers +91-96400 01789 contact@accentfuture.com
  • 7.
    Kafka Cluster Setup– Steps •Set up multiple servers/VMs •Configure unique broker.id, log.dirs, listeners •Point all brokers to same Zookeeper •Start all brokers •Verify with topic creation/replication +91-96400 01789 contact@accentfuture.com
  • 8.
    Kafka Configuration Files server.propertieskey settings: • broker.id • log.dirs • zookeeper.connect • listeners +91-96400 01789 contact@accentfuture.com
  • 9.
    Testing the Cluster •Create topic with replication factor > 1 • Produce messages • Consume from multiple consumers • Simulate broker failure +91-96400 01789 contact@accentfuture.com
  • 10.
    Monitoring & Tools •Kafka Manager / AKHQ • Prometheus + Grafana • Log monitoring +91-96400 01789 contact@accentfuture.com
  • 11.
    Common Issues &Fixes • Port conflicts • Zookeeper sync issues • Log retention • Broker unavailability +91-96400 01789 contact@accentfuture.com
  • 12.
    Summary & BestPractices • Local = dev/test, Cluster = production • Keep configs modular • Secure your cluster (SSL/SASL, ACLs) • Monitor everything +91-96400 01789 contact@accentfuture.com
  • 13.
    Contact Details šŸ“§ contact@accentfuture.com 🌐AccentFuture šŸ“ž +91-96400 01789 KAFKA ONLINE TRAINING +91-96400 01789 contact@accentfuture.com