Read time: ~

Cheat Sheet

Daily-use reference for producer, consumer, and broker configs, CLI one-liners, Spring Kafka keys, a delivery-semantics matrix, and links back to the deep-dive modules.

This page is the quick reference you keep open while working. It collects the settings and commands from across the course into scannable tables, with a deep-dive link on each topic so you can jump to the full explanation when you need the reasoning rather than just the value.

---

1. Producer configs

Config	Typical value	Purpose
acks	all	Durability: wait for in-sync replicas
enable.idempotence	true	No duplicates from producer retries
retries / delivery.timeout.ms	high / 120000	Ride out transient failures
max.in.flight.requests.per.connection	5 (with idempotence)	Throughput without reordering
batch.size / linger.ms	16384+ / 5-20	Batching for throughput
compression.type	lz4 / zstd	Shrink network and disk
max.request.size	1048576	Cap per-record size

Deep dive: Producing Deeper and Reliable Producing.

---

2. Consumer configs

Config	Typical value	Purpose
group.id	stable per service	Group membership and offset ownership
enable.auto.commit	false	Commit after processing, not before
auto.offset.reset	earliest / latest	Behavior with no committed offset
max.poll.records	100-500	Bound per-batch processing time
max.poll.interval.ms	300000	Max time between polls before eviction
session.timeout.ms / heartbeat.interval.ms	45000 / 3000	Liveness detection
group.instance.id	per instance	Static membership, fewer rebalances
fetch.min.bytes / fetch.max.wait.ms	tune / 500	Throughput vs latency on fetch

Deep dive: Consuming Deeper and Rebalancing.

---

3. Broker and topic configs

Config	Typical value	Purpose
replication.factor	3	AZ resilience
min.insync.replicas	2	Safe acks=all writes with RF 3
retention.ms / retention.bytes	per topic	How long/large the log is kept
cleanup.policy	delete / compact	Time-based vs key-compacted
segment.bytes / segment.ms	defaults	Segment rolling

Deep dive: Storage Internals and MSK Architecture.

---

4. CLI one-liners

# Topic layout, replicas, ISR
kafka-topics.sh --bootstrap-server $B --describe --topic orders

# Create a durable topic
kafka-topics.sh --bootstrap-server $B --create --topic orders \
  --partitions 3 --replication-factor 3

# Consumer group lag
kafka-consumer-groups.sh --bootstrap-server $B --describe --group payment-service

# Reset offsets (group stopped; ALWAYS dry-run first)
kafka-consumer-groups.sh --bootstrap-server $B --group payment-service \
  --topic orders --reset-offsets --to-earliest --dry-run

# Effective topic config
kafka-configs.sh --bootstrap-server $B --describe --entity-type topics --entity-name orders

# Change retention
kafka-configs.sh --bootstrap-server $B --alter --entity-type topics \
  --entity-name orders --add-config retention.ms=604800000

# Produce / consume by hand
kafka-console-producer.sh --bootstrap-server $B --topic orders
kafka-console-consumer.sh --bootstrap-server $B --topic orders --from-beginning \
  --property print.key=true

Deep dive: Tooling Walkthrough.

---

5. Spring Kafka config keys

spring:
  kafka:
    bootstrap-servers: ${KAFKA_BOOTSTRAP}
    producer:
      acks: all
      properties:
        enable.idempotence: true
    consumer:
      group-id: payment-service
      enable-auto-commit: false
      auto-offset-reset: earliest
      max-poll-records: 100
    listener:
      ack-mode: manual        # commit after processing
      concurrency: 3          # threads per listener
    properties:
      security.protocol: SASL_SSL   # on MSK

Deep dive: First Producer and Consumer.

---

6. Delivery-semantics matrix

Semantic	Producer	Consumer	Use for
At most once	acks=0/1, no retries	Commit before processing	Disposable data (metrics)
At least once	acks=all, retries, idempotence	Commit after processing	Most business events (with idempotent consumer)
Exactly once	Transactions, acks=all	isolation.level=read_committed	Read-process-write within Kafka

Deep dive: Delivery Guarantees, Transactions and EOS, Idempotency and Ordering.

---

7. Developer deep-dive links

Topic	Module
Core concepts, mental model	Core Concepts
Brokers, partitions, ISR	Cluster Anatomy
KRaft and ZooKeeper	Control Plane
Local Docker lab	Local Lab
Schema Registry, Avro	Schema Registry
Retry, DLT	Retry and Error Handling
Event-driven design	Event-Driven Architecture
Outbox, CDC	Outbox and CDC
Kafka Streams	Kafka Streams
Kafka Connect	Kafka Connect
Security	Security
Observability	Observability
Performance	Performance Tuning
Testing	Testing
Incident playbooks	Alert Playbooks

---

Next:Capstone Project, where you build the whole system end to end.