0

I want to create Kafka cluster in KRaft mode with two controllers and two brokers. I wrote configuration:

services: kafka-controller-1: image: 'bitnami/kafka:latest' environment: - KAFKA_CFG_NODE_ID=0 - KAFKA_KRAFT_CLUSTER_ID='4ea7fcb4a2fb4dc8a402d4103ee414eb' - KAFKA_CFG_CONTROLLER_QUORUM_VOTERS=0@kafka-controller-1:9092,1@kafka-controller-2:9092 - KAFKA_CFG_PROCESS_ROLES=controller - KAFKA_CFG_CONTROLLER_LISTENER_NAMES=CONTROLLER - KAFKA_CFG_LISTENERS=CONTROLLER://kafka-controller-1:9092 - KAFKA_CFG_ADVERTISED_LISTENERS=CONTROLLER://kafka-controller-1:9092 - KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP=CONTROLLER:PLAINTEXT kafka-controller-2: image: 'bitnami/kafka:latest' environment: - KAFKA_CFG_NODE_ID=1 - KAFKA_KRAFT_CLUSTER_ID='4ea7fcb4a2fb4dc8a402d4103ee414eb' - KAFKA_CFG_CONTROLLER_QUORUM_VOTERS=0@kafka-controller-1:9092,1@kafka-controller-2:9092 - KAFKA_CFG_PROCESS_ROLES=controller - KAFKA_CFG_CONTROLLER_LISTENER_NAMES=CONTROLLER - KAFKA_CFG_LISTENERS=CONTROLLER://kafka-controller-2:9092 - KAFKA_CFG_ADVERTISED_LISTENERS=CONTROLLER://kafka-controller-2:9092 - KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP=CONTROLLER:PLAINTEXT kafka-broker-1: image: 'bitnami/kafka:latest' environment: - KAFKA_CFG_NODE_ID=2 - KAFKA_KRAFT_CLUSTER_ID='4ea7fcb4a2fb4dc8a402d4103ee414eb' - KAFKA_CFG_CONTROLLER_QUORUM_VOTERS=0@kafka-controller-1:9092,1@kafka-controller-2:9092 - KAFKA_CFG_PROCESS_ROLES=broker - KAFKA_CFG_CONTROLLER_LISTENER_NAMES=CONTROLLER - KAFKA_CFG_LISTENERS=PLAINTEXT://kafka-broker-1:9092 - KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP=PLAINTEXT:PLAINTEXT,CONTROLLER:PLAINTEXT kafka-broker-2: image: 'bitnami/kafka:latest' environment: - KAFKA_CFG_NODE_ID=3 - KAFKA_KRAFT_CLUSTER_ID='4ea7fcb4a2fb4dc8a402d4103ee414eb' - KAFKA_CFG_CONTROLLER_QUORUM_VOTERS=0@kafka-controller-1:9092,1@kafka-controller-2:9092 - KAFKA_CFG_PROCESS_ROLES=broker - KAFKA_CFG_CONTROLLER_LISTENER_NAMES=CONTROLLER - KAFKA_CFG_LISTENERS=PLAINTEXT://kafka-broker-2:9092 - KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP=PLAINTEXT:PLAINTEXT,CONTROLLER:PLAINTEXT kafka-ui: container_name: kafka-ui image: 'provectuslabs/kafka-ui:latest' environment: - KAFKA_CLUSTERS_0_NAME=test-local - KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS=kafka-broker-1:9092 depends_on: - kafka-controller-1 - kafka-controller-2 - kafka-broker-1 - kafka-broker-2 ports: - '1354:8080' 

It works, UI shows me two brokers and one of them a controller. (I can't post image, need 10 reputation)

My questions is:

  1. Why I see only two brokers and no one of my controllers?
  2. Why one of brokers is controller if i configure other two controllers?

Additional:

  1. - KAFKA_CFG_CONTROLLER_LISTENER_NAMES=CONTROLLER configured for brokers because without this I get error:

    Exception in thread "main" java.lang.IllegalArgumentException: requirement failed: controller.listener.names must contain at least one value when running KRaft with just the broker role

  2. If I connect UI to controller - KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS=kafka-controller-1:9092 that UI can't request METADATA:

    2025-02-08 00:18:21 2025-02-07 21:18:21,979 INFO [kafka-admin-client-thread | kafka-ui-admin-1738963090-1] o.a.k.c.a.i.AdminMetadataManager: [AdminClient clientId=kafka-ui-admin-1738963090-1] Metadata update failed 2025-02-08 00:18:21 org.apache.kafka.common.errors.UnsupportedVersionException: The broker does not support METADATA

1
  • OTOH, one of the problems might be that you need a consensus per role. Also, an even number of members in Raft is a Very Bad Idea ™. Commented Jun 11 at 7:59

0

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.