SimpleAsyncUncaughtExceptionHandler : Unexpected exception occurred invoking async method during kafka junit tests - spring

Question

Scenario: I want to test my kafka producer against a Kafka Docker Container instead of KafkaEmbedded.
Goal: I want to start kafka container from my docker-compose before executing a simple producer test.
Searching around I found a plugin that can run docker-compose ( docker-compose-rule) and I am following its guidance.
My straight question: does this exception give a clue what is going wrong
2020-03-04 18:14:39.420 INFO 15372 --- [extShutdownHook] o.s.s.concurrent.ThreadPoolTaskExecutor : Shutting down ExecutorService 'taskExecutor'
2020-03-04 18:14:39.421 INFO 15372 --- [extShutdownHook] o.a.k.clients.producer.KafkaProducer : [Producer clientId=producer-1] Closing the Kafka producer with timeoutMillis = 30000 ms.
2020-03-04 18:14:39.428 ERROR 15372 --- [kaMsgExecutor-2] .a.i.SimpleAsyncUncaughtExceptionHandler : Unexpected exception occurred invoking async method: public void com.arquiteturareferencia.kafkaassyncsend.producer.Producer.send(java.lang.String,java.lang.String)
org.apache.kafka.common.errors.InterruptException: java.lang.InterruptedException
at org.apache.kafka.clients.producer.KafkaProducer.doSend(KafkaProducer.java:937) ~[kafka-clients-2.3.1.jar:na]
Surrounding doubt: I read from this logs firstly "Shutting down ExecutorService 'taskExecutor'" then "Closing the Kafka producer" and finally "[kaMsgExecutor-2] .a.i.SimpleAsyncUncaughtExceptionHandler". Does it mean the root cause is either ExecutorService or kafka producer isn't available at the moment of test? If so, what I have to settup in other to make it available while running the test?
Here is my Junit test:
import org.junit.ClassRule;
import org.junit.jupiter.api.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.test.context.junit4.SpringRunner;
import com.arquiteturareferencia.kafkaassyncsend.producer.Producer;
import com.palantir.docker.compose.DockerComposeRule;
import com.palantir.docker.compose.connection.waiting.HealthChecks;
#RunWith(SpringRunner.class)
#SpringBootTest
class KafkaAssyncSendApplicationTests {
#Autowired
private Producer p;
#ClassRule
public static DockerComposeRule docker = DockerComposeRule.builder()
.file("src/test/resources/docker-compose.yml")
.waitingForService("kafka", HealthChecks.toHaveAllPortsOpen())
.build();
#Test
public void testSend() throws Exception {
p.send("test", "Messagem demonstrativa assincrona 2");
}
}
My producer config
#EnableAsync
#Configuration
public class KafkaProducerConfig {
private static final Logger LOGGER = LoggerFactory.getLogger(KafkaProducerConfig.class);
#Value("${kafka.brokers}")
private String servers;
#Bean
public Executor taskExecutor() {
ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
executor.setCorePoolSize(2);
executor.setMaxPoolSize(2);
executor.setQueueCapacity(500);
executor.setThreadNamePrefix("KafkaMsgExecutor-");
executor.initialize();
return executor;
}
#Bean
public Map<String, Object> producerConfigs() {
Map<String, Object> props = new HashMap<>();
props.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, servers);
props.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class);
props.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, JsonDeserializer.class);
return props;
}
}
Producer:
#Service
public class Producer {
private static final Logger LOGGER = LoggerFactory.getLogger(Producer.class);
#Autowired
private KafkaTemplate<String, String> kafkaTemplate;
#Async
public void send(String topic, String message) {
ListenableFuture<SendResult<String, String>> future = kafkaTemplate.send(topic, message);
future.addCallback(new ListenableFutureCallback<SendResult<String, String>>() {
#Override
public void onSuccess(final SendResult<String, String> message) {
LOGGER.info("sent message= " + message + " with offset= " + message.getRecordMetadata().offset());
}
#Override
public void onFailure(final Throwable throwable) {
}
});
}
}
My docker-compose.yml in case it matters
version: "3.2"
services:
zookeeper:
image: wurstmeister/zookeeper
ports:
- "2181:2181"
kafka:
image: wurstmeister/kafka
ports:
- "9092:9092"
environment:
KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
KAFKA_LISTENERS:
"INTERNAL://kafka:9090,\
EXTERNAL://:9092"
KAFKA_ADVERTISED_LISTENERS:
"INTERNAL://kafka:9090,\
EXTERNAL://192.168.99.100:9092"
KAFKA_LISTENER_SECURITY_PROTOCOL_MAP:
"INTERNAL:PLAINTEXT,\
EXTERNAL:PLAINTEXT"
KAFKA_INTER_BROKER_LISTENER_NAME: INTERNAL
Full Exception logs while testing:
2020-03-04 18:14:28.132 INFO 15372 --- [ main] c.a.k.KafkaAssyncSendApplicationTests : Starting KafkaAssyncSendApplicationTests on SPANOT149 with PID 15372 (started by Cast in C:\WSs\pockafka\kafka-assync-send)
2020-03-04 18:14:28.135 INFO 15372 --- [ main] c.a.k.KafkaAssyncSendApplicationTests : No active profile set, falling back to default profiles: default
2020-03-04 18:14:29.072 INFO 15372 --- [ main] o.s.s.concurrent.ThreadPoolTaskExecutor : Initializing ExecutorService
2020-03-04 18:14:29.089 INFO 15372 --- [ main] o.s.s.concurrent.ThreadPoolTaskExecutor : Initializing ExecutorService 'taskExecutor'
2020-03-04 18:14:29.281 INFO 15372 --- [ main] c.a.k.KafkaAssyncSendApplicationTests : Started KafkaAssyncSendApplicationTests in 1.424 seconds (JVM running for 3.408)
2020-03-04 18:14:29.339 INFO 15372 --- [kaMsgExecutor-1] o.a.k.clients.producer.ProducerConfig : ProducerConfig values:
acks = 1
batch.size = 16384
bootstrap.servers = [192.168.99.100:9092]
buffer.memory = 33554432
client.dns.lookup = default
client.id =
compression.type = none
connections.max.idle.ms = 540000
delivery.timeout.ms = 120000
enable.idempotence = false
interceptor.classes = []
key.serializer = class org.apache.kafka.common.serialization.StringSerializer
linger.ms = 0
max.block.ms = 60000
max.in.flight.requests.per.connection = 5
max.request.size = 1048576
metadata.max.age.ms = 300000
metric.reporters = []
metrics.num.samples = 2
metrics.recording.level = INFO
metrics.sample.window.ms = 30000
partitioner.class = class org.apache.kafka.clients.producer.internals.DefaultPartitioner
receive.buffer.bytes = 32768
reconnect.backoff.max.ms = 1000
reconnect.backoff.ms = 50
request.timeout.ms = 30000
retries = 2147483647
retry.backoff.ms = 100
sasl.client.callback.handler.class = null
sasl.jaas.config = null
sasl.kerberos.kinit.cmd = /usr/bin/kinit
sasl.kerberos.min.time.before.relogin = 60000
sasl.kerberos.service.name = null
sasl.kerberos.ticket.renew.jitter = 0.05
sasl.kerberos.ticket.renew.window.factor = 0.8
sasl.login.callback.handler.class = null
sasl.login.class = null
sasl.login.refresh.buffer.seconds = 300
sasl.login.refresh.min.period.seconds = 60
sasl.login.refresh.window.factor = 0.8
sasl.login.refresh.window.jitter = 0.05
sasl.mechanism = GSSAPI
security.protocol = PLAINTEXT
send.buffer.bytes = 131072
ssl.cipher.suites = null
ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1]
ssl.endpoint.identification.algorithm = https
ssl.key.password = null
ssl.keymanager.algorithm = SunX509
ssl.keystore.location = null
ssl.keystore.password = null
ssl.keystore.type = JKS
ssl.protocol = TLS
ssl.provider = null
ssl.secure.random.implementation = null
ssl.trustmanager.algorithm = PKIX
ssl.truststore.location = null
ssl.truststore.password = null
ssl.truststore.type = JKS
transaction.timeout.ms = 60000
transactional.id = null
value.serializer = class org.apache.kafka.common.serialization.StringSerializer
2020-03-04 18:14:29.527 INFO 15372 --- [kaMsgExecutor-1] o.a.kafka.common.utils.AppInfoParser : Kafka version: 2.3.1
2020-03-04 18:14:29.529 INFO 15372 --- [kaMsgExecutor-1] o.a.kafka.common.utils.AppInfoParser : Kafka commitId: 18a913733fb71c01
2020-03-04 18:14:29.529 INFO 15372 --- [kaMsgExecutor-1] o.a.kafka.common.utils.AppInfoParser : Kafka startTimeMs: 1583356469521
2020-03-04 18:14:39.420 INFO 15372 --- [extShutdownHook] o.s.s.concurrent.ThreadPoolTaskExecutor : Shutting down ExecutorService 'taskExecutor'
2020-03-04 18:14:39.421 INFO 15372 --- [extShutdownHook] o.a.k.clients.producer.KafkaProducer : [Producer clientId=producer-1] Closing the Kafka producer with timeoutMillis = 30000 ms.
2020-03-04 18:14:39.428 ERROR 15372 --- [kaMsgExecutor-2] .a.i.SimpleAsyncUncaughtExceptionHandler : Unexpected exception occurred invoking async method: public void com.arquiteturareferencia.kafkaassyncsend.producer.Producer.send(java.lang.String,java.lang.String)
org.apache.kafka.common.errors.InterruptException: java.lang.InterruptedException
at org.apache.kafka.clients.producer.KafkaProducer.doSend(KafkaProducer.java:937) ~[kafka-clients-2.3.1.jar:na]
at org.apache.kafka.clients.producer.KafkaProducer.send(KafkaProducer.java:856) ~[kafka-clients-2.3.1.jar:na]
at org.springframework.kafka.core.DefaultKafkaProducerFactory$CloseSafeProducer.send(DefaultKafkaProducerFactory.java:592) ~[spring-kafka-2.3.6.RELEASE.jar:2.3.6.RELEASE]
at org.springframework.kafka.core.KafkaTemplate.doSend(KafkaTemplate.java:404) ~[spring-kafka-2.3.6.RELEASE.jar:2.3.6.RELEASE]
at org.springframework.kafka.core.KafkaTemplate.send(KafkaTemplate.java:216) ~[spring-kafka-2.3.6.RELEASE.jar:2.3.6.RELEASE]
at com.arquiteturareferencia.kafkaassyncsend.producer.Producer.send(Producer.java:23) ~[classes/:na]
at com.arquiteturareferencia.kafkaassyncsend.producer.Producer$$FastClassBySpringCGLIB$$78adf947.invoke(<generated>) ~[classes/:na]
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) ~[spring-core-5.2.4.RELEASE.jar:5.2.4.RELEASE]
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:769) ~[spring-aop-5.2.4.RELEASE.jar:5.2.4.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) ~[spring-aop-5.2.4.RELEASE.jar:5.2.4.RELEASE]
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:747) ~[spring-aop-5.2.4.RELEASE.jar:5.2.4.RELEASE]
at org.springframework.aop.interceptor.AsyncExecutionInterceptor.lambda$invoke$0(AsyncExecutionInterceptor.java:115) ~[spring-aop-5.2.4.RELEASE.jar:5.2.4.RELEASE]
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[na:na]
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[na:na]
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[na:na]
at java.base/java.lang.Thread.run(Thread.java:834) ~[na:na]
Caused by: java.lang.InterruptedException: null
at java.base/java.lang.Object.wait(Native Method) ~[na:na]
at org.apache.kafka.common.utils.SystemTime.waitObject(SystemTime.java:60) ~[kafka-clients-2.3.1.jar:na]
at org.apache.kafka.clients.producer.internals.ProducerMetadata.awaitUpdate(ProducerMetadata.java:97) ~[kafka-clients-2.3.1.jar:na]
at org.apache.kafka.clients.producer.KafkaProducer.waitOnMetadata(KafkaProducer.java:999) ~[kafka-clients-2.3.1.jar:na]
at org.apache.kafka.clients.producer.KafkaProducer.doSend(KafkaProducer.java:876) ~[kafka-clients-2.3.1.jar:na]
... 15 common frames omitted
2020-03-04 18:14:39.428 ERROR 15372 --- [kaMsgExecutor-1] .a.i.SimpleAsyncUncaughtExceptionHandler : Unexpected exception occurred invoking async method: public void com.arquiteturareferencia.kafkaassyncsend.producer.Producer.send(java.lang.String,java.lang.String)
org.apache.kafka.common.errors.InterruptException: java.lang.InterruptedException
at org.apache.kafka.clients.producer.KafkaProducer.doSend(KafkaProducer.java:937) ~[kafka-clients-2.3.1.jar:na]
at org.apache.kafka.clients.producer.KafkaProducer.send(KafkaProducer.java:856) ~[kafka-clients-2.3.1.jar:na]
at org.springframework.kafka.core.DefaultKafkaProducerFactory$CloseSafeProducer.send(DefaultKafkaProducerFactory.java:592) ~[spring-kafka-2.3.6.RELEASE.jar:2.3.6.RELEASE]
at org.springframework.kafka.core.KafkaTemplate.doSend(KafkaTemplate.java:404) ~[spring-kafka-2.3.6.RELEASE.jar:2.3.6.RELEASE]
at org.springframework.kafka.core.KafkaTemplate.send(KafkaTemplate.java:216) ~[spring-kafka-2.3.6.RELEASE.jar:2.3.6.RELEASE]
at com.arquiteturareferencia.kafkaassyncsend.producer.Producer.send(Producer.java:23) ~[classes/:na]
at com.arquiteturareferencia.kafkaassyncsend.producer.Producer$$FastClassBySpringCGLIB$$78adf947.invoke(<generated>) ~[classes/:na]
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) ~[spring-core-5.2.4.RELEASE.jar:5.2.4.RELEASE]
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:769) ~[spring-aop-5.2.4.RELEASE.jar:5.2.4.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) ~[spring-aop-5.2.4.RELEASE.jar:5.2.4.RELEASE]
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:747) ~[spring-aop-5.2.4.RELEASE.jar:5.2.4.RELEASE]
at org.springframework.aop.interceptor.AsyncExecutionInterceptor.lambda$invoke$0(AsyncExecutionInterceptor.java:115) ~[spring-aop-5.2.4.RELEASE.jar:5.2.4.RELEASE]
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[na:na]
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[na:na]
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[na:na]
at java.base/java.lang.Thread.run(Thread.java:834) ~[na:na]
Caused by: java.lang.InterruptedException: null
at java.base/java.lang.Object.wait(Native Method) ~[na:na]
at org.apache.kafka.common.utils.SystemTime.waitObject(SystemTime.java:60) ~[kafka-clients-2.3.1.jar:na]
at org.apache.kafka.clients.producer.internals.ProducerMetadata.awaitUpdate(ProducerMetadata.java:97) ~[kafka-clients-2.3.1.jar:na]
at org.apache.kafka.clients.producer.KafkaProducer.waitOnMetadata(KafkaProducer.java:999) ~[kafka-clients-2.3.1.jar:na]
at org.apache.kafka.clients.producer.KafkaProducer.doSend(KafkaProducer.java:876) ~[kafka-clients-2.3.1.jar:na]
... 15 common frames omitted