Observability

In Hindu tradition, Chitragupta is the deity who maintains a complete record of every being’s actions. The divine accountant. Fitting name for a system that tracks exactly who used what and how much it cost.

Presented at Conf42 Observability 2025. Kafka doesn’t fail cleanly. It stalls, lags, and misfires beneath the surface. This talk cuts through the noise to show what signals actually matter, how to catch issues early, and how to make Kafka observable without drowning in metrics.

When multiple teams share a Confluent Cloud environment but a single CoE team manages everything, the billing question comes up fast: who’s actually using what, and how much does it cost?

Most monitoring setups for Kafka require long and lengthy parsing configurations to get JMX metrics into your telemetry system. This project standardizes that pipeline across Prometheus/Grafana, New Relic, Elastic/Kibana, Datadog, and OpenTelemetry.

Published on the Confluent Blog, March 2021. Self-managing a Kafka cluster means wiring up your own monitoring. This post walks through how to export JMX data from Confluent clusters into Prometheus and Grafana with minimal setup. It became the reference blog series for connecting Confluent ecosystems to Prometheus-based monitoring stacks.