kafka sync producer takes longer time on the first request - spring-boot

Question

I am using spring cloud stream Kafka sync producer in a spring boot micro service. every time we deploy the service the very first call to kafka takes more than 20 seconds to publish the message to Topic. but all the subsequent calls takes hardly 3 to 4 miliseconds. This issue also happens randomly and is intermittent but mostly happens when we restart the service.
we are using kafka version 0.9.0.1 and gradle dependencies as below
dependencies {
compile('org.springframework.cloud:spring-cloud-starter-stream-kafka')
}
dependencyManagement {
imports {
mavenBom "org.springframework.cloud:spring-cloud-dependencies:Camden.SR3"
}
}
here is the application. yml
spring:
cloud:
stream:
bindings:
output:
content-type: application/json
destination: SOPOrderReceiveTopic
kafka:
binder:
brokers: "localhost:9092,localhost:9093"
headers: eventType
requiredAcks: -1
zkNodes: "localhost:2181"
bindings:
output:
producer:
configuration:
max:
block:
ms: 20000
reconnect:
backoff:
ms: 5000
request:
timeout:
ms: 30000
retries: 3
retry:
backoff:
ms: 10000
timeout:
ms: 30000
sync: true
I am using org.springframework.cloud.stream.messaging.Source as output channel and this is the method used to publish message
public void publish(Message event) {
try {
boolean result = source.output().send(event, orderEventConfig.getTimeoutMs());
logger.log(LoggingEventType.INFORMATION, "MESSAGE SENT TO KAFKA : " + result);
} catch (Exception publishingExceptionMessage) {
logger.log(LoggingEventType.ERROR, "publish event to kafka failed!", publishingExceptionMessage);
throw new PublishEventException("publish event to kafka failed for eventPayload: " + event.getPayload(),
ThreadVariables.getTenantId());
}
}
I am aware that sync producer is slower is terms of performance as it guarantees the order and durability of message but why only the first request takes so long? is this issue a known issue ? is it fixed in the latest kafka version. can somebody suggest. thanks

Answer 1

It looks like there is an issue with the version for spring cloud stream downloaded using below dependency ,
imports {
mavenBom "org.springframework.cloud:spring-cloud-dependencies:Camden.SR3"
}
Try upgrading the spring cloud stream and check. It should fix the latency in the first publishing call on kafka server after springboot service startup.
dependencies {
compile('org.springframework.cloud:spring-cloud-stream-binder-kafka')
}
ext { springCloudVersion = 'Dalston.RELEASE' }
dependencyManagement {
imports {
mavenBom "org.springframework.cloud:spring-cloud-dependencies:${springCloudVersion}"
}
}