This article will describe the how the Apache Kafka work with components

Kafka and components:

Apache kafka is distributed event-streaming engine. It was designed by C/S(Client/Server) archtecture. Client means kafka producer and kafka consumer, which function include: Kafka producer and kafka consumer. Server means kafka broker, which function include: store data; cluster of computers; can also integrate with other systems.

Kafka Server:

  • Data written by producers is stored and organized via topics.
  • Topic are partitioned
  • messages / events stored in a given partition based on an event id

Source connector

Sink connector

Streamer

Transformer

Register Schema