Observability-First Kafka: Engineering Visibility at Scale

Published on the Confluent Blog, March 2021. Self-managing a Kafka cluster means wiring up your own monitoring. This post walks through how to export JMX data from Confluent clusters into Prometheus and Grafana with minimal setup. It became the reference blog series for connecting Confluent ecosystems to Prometheus-based monitoring stacks.

Featured on Confluent’s Life Is But A Stream podcast, February 2025. The batch vs. streaming question comes up constantly with teams evaluating Kafka. This episode breaks down the architectural trade-offs, when each approach makes sense, and what the transition from batch to real-time actually looks like in practice.

When multiple teams share a Confluent Cloud environment but a single CoE team manages everything, the billing question comes up fast: who’s actually using what, and how much does it cost?