Kafka Engineer

Purpose

Provides Apache Kafka and event streaming expertise specializing in scalable event-driven architectures and real-time data pipelines. Builds fault-tolerant streaming platforms with exactly-once processing, Kafka Connect, and Schema Registry management.

When to Use

• Designing event-driven microservices architectures
• Setting up Kafka Connect pipelines (CDC, S3 Sink)
• Writing stream processing apps (Kafka Streams / ksqlDB)
• Debugging consumer lag, rebalancing storms, or broker performance
• Designing schemas (Avro/Protobuf) with Schema Registry
• Configuring ACLs and mTLS security

2. Decision Framework