在Ubuntu上配置Apache Kafka时,有几个关键要点需要注意:
下载并解压Zookeeper安装包:
wget https://downloads.apache.org/zookeeper/zookeeper-3.7.0/apache-zookeeper-3.7.0-bin.tar.gz tar -xzvf apache-zookeeper-3.7.0-bin.tar.gz sudo mv apache-zookeeper-3.7.0 /opt/zookeeper 编辑zoo.cfg配置文件:
cd /opt/zookeeper/conf cp zoo_sample.cfg zoo.cfg 设置以下参数:
dataDir=/opt/zookeeper/data clientPort=2181 启动Zookeeper:
/opt/zookeeper/bin/zkServer.sh start 验证Zookeeper是否启动成功:
netstat -an | grep 2181 安装OpenJDK:
sudo apt update sudo apt install openjdk-8-jdk 验证Java安装:
java -version 下载Kafka安装包:
wget https://downloads.apache.org/kafka/2.8.0/kafka_2.13-2.8.0.tgz 解压安装包:
sudo tar -xzvf kafka_2.13-2.8.0.tgz -C /usr/local 编辑server.properties文件:
cd /usr/local/kafka_2.13-2.8.0/config cp server.properties server.properties.template 修改以下配置项:
broker.id=0 listeners=PLAINTEXT://:9092 advertised.listeners=PLAINTEXT://your_server_ip:9092 log.dirs=/tmp/kafka zookeeper.connect=localhost:2181 为每个broker节点复制一份配置文件,并修改其中的broker.id以保证每个broker的唯一性。
启动Zookeeper:
/usr/local/kafka_2.13-2.8.0/bin/zookeeper-server-start.sh config/zookeeper.properties 启动Kafka Server:
/usr/local/kafka_2.13-2.8.0/bin/kafka-server-start.sh config/server.properties 创建一个主题:
/usr/local/kafka_2.13-2.8.0/bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test 发送与接收消息:
/usr/local/kafka_2.13-2.8.0/bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test /usr/local/kafka_2.13-2.8.0/bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic test --from-beginning /tmp/kafka目录下。-Xmx和-Xms参数来分配足够的堆内存,选择合适的垃圾回收器,如G1,调整堆内存大小,并启用JIT编译。num.network.threads和num.io.threads以提高性能,设置socket.send.buffer.bytes和socket.receive.buffer.bytes来增加网络数据传输的效率。ulimit -n 65536命令,更改TCP参数如net.core.somaxconn和net.ipv4.tcp_max_syn_backlog以提高网络性能。以上步骤涵盖了在Ubuntu上配置Kafka的基本流程,并根据实际需求可能需要进行更多的配置和优化。