Spring Boot and Kafka Integration: Building a Real-Time Messaging Application

🎓 Top 15 Udemy Courses (80-90% Discount): My Udemy Courses - Ramesh Fadatare — All my Udemy courses are real-time and project oriented courses.

▶️ Subscribe to My YouTube Channel (178K+ subscribers): Java Guides on YouTube

▶️ For AI, ChatGPT, Web, Tech, and Generative AI, subscribe to another channel: Ramesh Fadatare on YouTube

Integrating Apache Kafka with Spring Boot offers a powerful combination for creating efficient, scalable real-time messaging applications. In this blog post, we'll walk through the steps to set up a simple messaging system using Spring Boot and Kafka, highlighting the usage of a Kafka producer and consumer service.

Prerequisites 

Before we start, ensure that you have the following: 

• Java and Spring Boot set up in your development environment. 
• Apache Kafka is installed and running on your machine or a server. 
• Basic understanding of Spring Boot and RESTful services.

Step 1: Setting Up Dependencies 

Include the following Maven dependencies in your pom.xml:

<dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-web</artifactId> </dependency> <dependency> <groupId>org.springframework.kafka</groupId> <artifactId>spring-kafka</artifactId> </dependency>

Step 2: Kafka Configuration 

Configure the Kafka properties in your application.properties file:

spring.kafka.bootstrap-servers=localhost:9092 spring.kafka.consumer.group-id=myGroup spring.kafka.consumer.auto-offset-reset=earliest spring.kafka.consumer.key-deserializer=org.apache.kafka.common.serialization.StringDeserializer spring.kafka.consumer.value-deserializer=org.apache.kafka.common.serialization.StringDeserializer 

Step 3: Creating Kafka Producer Service

Now, let's set up the KafkaProducerService:

import org.springframework.beans.factory.annotation.Autowired; import org.springframework.kafka.core.KafkaTemplate; import org.springframework.stereotype.Service; @Service public class KafkaProducerService { private final KafkaTemplate<String, String> kafkaTemplate; @Autowired public KafkaProducerService(KafkaTemplate<String, String> kafkaTemplate) { this.kafkaTemplate = kafkaTemplate; } public void sendMessage(String topic, String message) { kafkaTemplate.send(topic, message); } } 

Step 4: Implementing a REST Controller

To send messages, implement a REST controller that uses KafkaProducerService:

import org.springframework.beans.factory.annotation.Autowired; import org.springframework.web.bind.annotation.PostMapping; import org.springframework.web.bind.annotation.RequestParam; import org.springframework.web.bind.annotation.RestController; @RestController public class MessageController { private final KafkaProducerService producerService; @Autowired public MessageController(KafkaProducerService producerService) { this.producerService = producerService; } @PostMapping("/send") public String sendMessage(@RequestParam String message) { producerService.sendMessage("myTopic", message); return "Message sent to Kafka topic"; } } 

Step 5: Creating Kafka Consumer Service

Next, create a Kafka consumer service to receive messages:

@Service public class KafkaConsumerService { @KafkaListener(topics = "myTopic", groupId = "myGroup") public void listen(String message) { System.out.println("Received message in group 'myGroup': " + message); } } 

Step 6: Testing the Setup

To test, send a message using a tool like Postman or a curl command:

curl -X POST "http://localhost:8080/send?message=Hello Kafka!"

The consumer service should log:

Received message in group 'myGroup': Hello Kafka!

Conclusion

This basic setup illustrates how to create a real-time messaging application using Spring Boot and Kafka. This integration allows you to leverage Kafka’s capabilities for handling large-scale, real-time data while benefiting from Spring Boot's ease of development and configuration. Remember to adapt and expand upon this foundation to suit your application's specific needs and Kafka configurations.