DEV Community

Abdul wakeel
Abdul wakeel

Posted on

AWS Kafka with Node.js

#watercooler

To use AWS Kafka with Node.js, use the kafkajs library which provides a client for Kafka in Node.js. Here how to use kafkajs with AWS Kafka:

Install the kafkajs library using npm:

npm install kafkajs

Create a new instance of the Kafka client with the AWS configuration:

const { Kafka } = require('kafkajs') const kafka = new Kafka({ clientId: 'my-app', brokers: ['kafka-1.amazonaws.com:9092', 'kafka-2.amazonaws.com:9092'], ssl: true, sasl: { mechanism: 'scram-sha-256', username: 'my-username', password: 'my-password', }, })
Enter fullscreen mode Exit fullscreen mode

Create a producer to send messages to a topic:
javascript

const producer = kafka.producer() await producer.connect() await producer.send({ topic: 'my-topic', messages: [ { value: 'Hello Kafka!' }, ], }) await producer.disconnect()
Enter fullscreen mode Exit fullscreen mode

Create a consumer to receive messages from a topic:

const consumer = kafka.consumer({ groupId: 'my-group' }) await consumer.connect() await consumer.subscribe({ topic: 'my-topic', fromBeginning: true }) await consumer.run({ eachMessage: async ({ topic, partition, message }) => { console.log({ topic, partition, offset: message.offset, value: message.value.toString(), }) }, }) await consumer.disconnect()
Enter fullscreen mode Exit fullscreen mode

That's it! You should now be able to use AWS Kafka with Node.js using the kafkajs library.

Top comments (0)