I want to create Kafka cluster in KRaft mode with two controllers and two brokers. I wrote configuration:
services: kafka-controller-1: image: 'bitnami/kafka:latest' environment: - KAFKA_CFG_NODE_ID=0 - KAFKA_KRAFT_CLUSTER_ID='4ea7fcb4a2fb4dc8a402d4103ee414eb' - KAFKA_CFG_CONTROLLER_QUORUM_VOTERS=0@kafka-controller-1:9092,1@kafka-controller-2:9092 - KAFKA_CFG_PROCESS_ROLES=controller - KAFKA_CFG_CONTROLLER_LISTENER_NAMES=CONTROLLER - KAFKA_CFG_LISTENERS=CONTROLLER://kafka-controller-1:9092 - KAFKA_CFG_ADVERTISED_LISTENERS=CONTROLLER://kafka-controller-1:9092 - KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP=CONTROLLER:PLAINTEXT kafka-controller-2: image: 'bitnami/kafka:latest' environment: - KAFKA_CFG_NODE_ID=1 - KAFKA_KRAFT_CLUSTER_ID='4ea7fcb4a2fb4dc8a402d4103ee414eb' - KAFKA_CFG_CONTROLLER_QUORUM_VOTERS=0@kafka-controller-1:9092,1@kafka-controller-2:9092 - KAFKA_CFG_PROCESS_ROLES=controller - KAFKA_CFG_CONTROLLER_LISTENER_NAMES=CONTROLLER - KAFKA_CFG_LISTENERS=CONTROLLER://kafka-controller-2:9092 - KAFKA_CFG_ADVERTISED_LISTENERS=CONTROLLER://kafka-controller-2:9092 - KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP=CONTROLLER:PLAINTEXT kafka-broker-1: image: 'bitnami/kafka:latest' environment: - KAFKA_CFG_NODE_ID=2 - KAFKA_KRAFT_CLUSTER_ID='4ea7fcb4a2fb4dc8a402d4103ee414eb' - KAFKA_CFG_CONTROLLER_QUORUM_VOTERS=0@kafka-controller-1:9092,1@kafka-controller-2:9092 - KAFKA_CFG_PROCESS_ROLES=broker - KAFKA_CFG_CONTROLLER_LISTENER_NAMES=CONTROLLER - KAFKA_CFG_LISTENERS=PLAINTEXT://kafka-broker-1:9092 - KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP=PLAINTEXT:PLAINTEXT,CONTROLLER:PLAINTEXT kafka-broker-2: image: 'bitnami/kafka:latest' environment: - KAFKA_CFG_NODE_ID=3 - KAFKA_KRAFT_CLUSTER_ID='4ea7fcb4a2fb4dc8a402d4103ee414eb' - KAFKA_CFG_CONTROLLER_QUORUM_VOTERS=0@kafka-controller-1:9092,1@kafka-controller-2:9092 - KAFKA_CFG_PROCESS_ROLES=broker - KAFKA_CFG_CONTROLLER_LISTENER_NAMES=CONTROLLER - KAFKA_CFG_LISTENERS=PLAINTEXT://kafka-broker-2:9092 - KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP=PLAINTEXT:PLAINTEXT,CONTROLLER:PLAINTEXT kafka-ui: container_name: kafka-ui image: 'provectuslabs/kafka-ui:latest' environment: - KAFKA_CLUSTERS_0_NAME=test-local - KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS=kafka-broker-1:9092 depends_on: - kafka-controller-1 - kafka-controller-2 - kafka-broker-1 - kafka-broker-2 ports: - '1354:8080' It works, UI shows me two brokers and one of them a controller. (I can't post image, need 10 reputation)
My questions is:
- Why I see only two brokers and no one of my controllers?
- Why one of brokers is controller if i configure other two controllers?
Additional:
- KAFKA_CFG_CONTROLLER_LISTENER_NAMES=CONTROLLERconfigured for brokers because without this I get error:Exception in thread "main" java.lang.IllegalArgumentException: requirement failed: controller.listener.names must contain at least one value when running KRaft with just the broker role
If I connect UI to controller
- KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS=kafka-controller-1:9092that UI can't request METADATA:2025-02-08 00:18:21 2025-02-07 21:18:21,979 INFO [kafka-admin-client-thread | kafka-ui-admin-1738963090-1] o.a.k.c.a.i.AdminMetadataManager: [AdminClient clientId=kafka-ui-admin-1738963090-1] Metadata update failed 2025-02-08 00:18:21 org.apache.kafka.common.errors.UnsupportedVersionException: The broker does not support METADATA