Kafka message sending fails with 'class SslBundleSslEngineFactory could not be found' #44414
Description
Spring Boot Kafka SSL Issue on EKS Deployment using Spring boot 3.4.3(tried 3.3.5 and 3.4.2)
We are facing an issue where the Kafka producer works fine in local environments (IntelliJ, mvn spring-boot:run, and java -jar). However, after deploying to EKS, we encounter the following exception:
Exception occurred: org.apache.kafka.common.config.ConfigException: Invalid value org.springframework.boot.autoconfigure.kafka.SslBundleSslEngineFactory for configuration ssl.engine.factory.class: Class org.springframework.boot.autoconfigure.kafka.SslBundleSslEngineFactory could not be found.. Stack Trace: [org.apache.kafka.common.config.ConfigDef.parseType(ConfigDef.java:778), org.apache.kafka.common.config.ConfigDef.parseValue(ConfigDef.java:531), org.apache.kafka.common.config.ConfigDef.parse(ConfigDef.java:524), org.apache.kafka.common.config.AbstractConfig.<init>(AbstractConfig.java:114), org.apache.kafka.common.config.AbstractConfig.<init>(AbstractConfig.java:134), org.apache.kafka.clients.producer.ProducerConfig.<init>(ProducerConfig.java:643), org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:295), org.springframework.kafka.core.DefaultKafkaProducerFactory.createRawProducer(DefaultKafkaProducerFactory.java:944), org.springframework.kafka.core.DefaultKafkaProducerFactory.createKafkaProducer(DefaultKafkaProducerFactory.java:826), org.springframework.kafka.core.DefaultKafkaProducerFactory.doCreateProducer(DefaultKafkaProducerFactory.java:793), org.springframework.kafka.core.DefaultKafkaProducerFactory.createProducer(DefaultKafkaProducerFactory.java:768), org.springframework.kafka.core.DefaultKafkaProducerFactory.createProducer(DefaultKafkaProducerFactory.java:762), org.springframework.kafka.core.KafkaTemplate.getTheProducer(KafkaTemplate.java:976), org.springframework.kafka.core.KafkaTemplate.doSend(KafkaTemplate.java:828), org.springframework.kafka.core.KafkaTemplate.observeSend(KafkaTemplate.java:805), org.springframework.kafka.core.KafkaTemplate.send(KafkaTemplate.java:608), Issue Details:
Kafka producer initialization delayed: Instead of initializing at application startup, it only starts when sending a message, leading to the SslBundleSslEngineFactory class not being found.
Temporary workaround: We manually create the producer in a @PostConstruct method inside a KafkaConfig class.
@Slf4j @Configuration @RequiredArgsConstructor public class KafkaConfig { private final ProducerFactory<String, String> producerFactory; @PostConstruct public void initializeProducer() { try { Producer<String, String> producer = producerFactory.createProducer(); // Workaround to initialize Kafka producer log.info("Kafka Producer initialized successfully: {}", producer); } catch (Exception e) { log.error("Failed to initialize Kafka Producer", e); } } } Spring Boot Configuration (application.yml)
spring: application: name: test ssl: bundle: pem: kafkaCert: keystore: certificate: classpath:KafkaKeystoreCert.pem private-key: classpath:KafkaKeystoreKey.pem truststore: certificate: classpath:KafkaTruststorePem.pem kafka: bootstrap-servers: ${KafkaBootstrapServers} ssl: bundle: kafkaCert security: protocol: SSL client-id: client-id producer: retries: 3 key-serializer: org.apache.kafka.common.serialization.StringSerializer value-serializer: org.apache.kafka.common.serialization.StringSerializer Request for a Proper Solution
Do we have a proper fix for this issue instead of the workaround?
Is there a missing dependency or classpath issue specific to the environment?
Are there configurations that need adjustment to ensure Kafka SSL settings load correctly?
Any best practices for ensuring SslBundleSslEngineFactory is properly available at runtime?