Kafka Event-Driven Microservices and Topic Design

You now have every building block: producers, consumers, schemas, delivery semantics, and idempotency. This module steps back to architecture. It shows how to compose the Order, Payment, Inventory, and Notification services into an event-driven system that stays loosely coupled as it grows, and how to make the topic and key decisions that decide whether it scales and stays correct.

What you’ll be able to do after this module

Distinguish an event from a command and choose the right one.
Compare choreography and orchestration and pick per situation.
Decide between topic-per-event-type and topic-per-entity.
Select keys and partition counts for ordering and scale.
Version events without breaking existing consumers.

1. Events vs commands

Both travel as Kafka records, but they express different intent.

	Command	Event
Intent	“Do this”	“This happened”
Naming	Imperative: `ReserveStock`	Past tense: `OrderCreated`
Knows the recipient	Yes, one handler	No, any number of subscribers
Coupling	Higher	Lower

Events are the backbone of loose coupling. The Order service announces OrderCreated without knowing who reacts. Commands still have a place for directed work, but prefer events for cross-service reactions, because they let you add consumers later without touching the producer.

2. Choreography vs orchestration

Two ways to coordinate a multi-step flow across services.

Choreography: each service reacts to events and emits its own, with no central brain. Order emits OrderCreated, Payment reacts and emits PaymentSucceeded, Inventory reacts to that. Loosely coupled, but the end-to-end flow is implicit.
Orchestration: a coordinator service issues commands and tracks progress. Explicit and easy to follow, but the coordinator couples to every step.

flowchart TD
    subgraph choreo [Choreography]
        o1["Order"] -->|OrderCreated| p1["Payment"]
        p1 -->|PaymentSucceeded| i1["Inventory"]
    end
    subgraph orch [Orchestration]
        c["Saga coordinator"] -->|charge| p2["Payment"]
        c -->|reserve| i2["Inventory"]
    end

This course uses choreography for the main flow, which fits Kafka’s publish-and-subscribe model. Orchestration is reasonable for complex flows that need central visibility.

3. Topic design: per event type vs per entity

A recurring decision is how to slice topics.

Topic per event type:orders, payments, inventory, notifications. Each topic carries one kind of event. Easy to reason about, easy to set per-topic retention and compatibility.
Topic per entity: one topic carries all events about an entity (for example every order lifecycle event). Preserves order across event types for that entity, at the cost of mixed schemas on one topic.

The common default, used here, is topic per event type. When you genuinely need the full ordered history of one entity across event types, a per-entity topic (often compacted) is the tool, which ties back to log compaction from Storage Internals.

4. Keys and partitions for ordering and scale

As established in Idempotent Consumers, Ordering, and Duplicates, ordering is per partition, and the key decides the partition. So topic design is incomplete without a key decision.

Key by orderId on orders and payments, so all events for one order stay ordered while different orders parallelize.
Partition count sets the ceiling on consumer parallelism. Size it for peak throughput and future growth, because increasing it later remaps keys and disrupts ordering.

A useful rule: choose the key from the entity whose ordering you must preserve, then choose enough partitions to hit your throughput target with headroom.

5. Event versioning

Events outlive the code that first emitted them, so plan for change from the start.

Additive changes: add optional fields with defaults, which the Schema Registry compatibility modes from Schema Registry and Data Contracts enforce.
Breaking changes: when a change cannot be additive, introduce a new event version (for example a new topic orders.v2 or a versioned schema) and migrate consumers before retiring the old one.
Carry a version in the schema so a consumer can branch when semantics, not just fields, change.

The registry rejects an incompatible schema at registration, so most versioning discipline is enforced automatically once schemas are in place.

6. The full topology

Putting it together, the four services form a choreographed pipeline. Each service owns its topics and reacts to upstream events.

flowchart TD
    client["Client"] -->|POST /orders| order["Order service"]
    order -->|OrderCreated| ordersT["orders topic"]
    ordersT --> payment["Payment service"]
    payment -->|PaymentSucceeded / PaymentFailed| paymentsT["payments topic"]
    paymentsT --> inventory["Inventory service"]
    inventory -->|StockReserved / StockRejected| inventoryT["inventory topic"]
    ordersT --> notify["Notification service"]
    paymentsT --> notify
    inventoryT --> notify
    notify -->|notifications| notifT["notifications topic"]

The Notification service subscribes to several topics to keep the customer informed, which is the fan-out advantage of the log model: adding it required no change to any producer.

Checkpoint

You should now be able to:

Distinguish an event from a command and choose appropriately.
Compare choreography and orchestration and pick per situation.
Decide between topic-per-event-type and topic-per-entity.
Select keys and partition counts for ordering and scale.
Version events without breaking existing consumers.

Next:Transactional Outbox and CDC, where you close the gap between a database commit and a Kafka publish.