Spring Embedded Kafka using Kafka 2.2.x - apache-kafka-streams

Question

The Embedded Kafka Broker for Spring does not work when upgrading Spring-Kafka 2.2.x which is required for kafka-client and kafka-stream 2.2.3. It seems to require a meta.properties file in the log.dir directory, and requires setting a broker.id. However, it throws an exception for a binding address. Has anyone got this to work?
at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:391)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:312)
at org.springframework.boot.test.context.SpringBootContextLoader.loadContext(SpringBootContextLoader.java:120)
at org.springframework.test.context.cache.DefaultCacheAwareContextLoaderDelegate.loadContextInternal(DefaultCacheAwareContextLoaderDelegate.java:99)
at org.springframework.test.context.cache.DefaultCacheAwareContextLoaderDelegate.loadContext(DefaultCacheAwareContextLoaderDelegate.java:117)
... 26 common frames omitted
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'embeddedKafkaBroker' defined in com.talentreef.notification.legacy.streams.topology.ApplicantPoolStreamTest$ApplicantPoolStreamTestConfig: Invocation of init method failed; nested exception is org.apache.kafka.common.KafkaException: Socket server failed to bind to localhost:50659: Address already in use.
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1778)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:593)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:515)
at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:320)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:318)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:199)
at org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:277)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1251)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1171)
at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:593)
... 44 common frames omitted
Caused by: org.apache.kafka.common.KafkaException: Socket server failed to bind to localhost:50659: Address already in use.
at kafka.network.Acceptor.openServerSocket(SocketServer.scala:573)
at kafka.network.Acceptor.<init>(SocketServer.scala:451)
at kafka.network.SocketServer.kafka$network$SocketServer$$createAcceptor(SocketServer.scala:245)
at kafka.network.SocketServer$$anonfun$createDataPlaneAcceptorsAndProcessors$1.apply(SocketServer.scala:215)
at kafka.network.SocketServer$$anonfun$createDataPlaneAcceptorsAndProcessors$1.apply(SocketServer.scala:214)
at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
at kafka.network.SocketServer.createDataPlaneAcceptorsAndProcessors(SocketServer.scala:214)
at kafka.network.SocketServer.startup(SocketServer.scala:114)
at kafka.server.KafkaServer.startup(KafkaServer.scala:253)
at kafka.utils.TestUtils$.createServer(TestUtils.scala:140)
at kafka.utils.TestUtils.createServer(TestUtils.scala)
at org.springframework.kafka.test.EmbeddedKafkaBroker.afterPropertiesSet(EmbeddedKafkaBroker.java:223)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1837)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1774)
... 54 common frames omitted
Caused by: java.net.BindException: Address already in use
at java.base/sun.nio.ch.Net.bind0(Native Method)
at java.base/sun.nio.ch.Net.bind(Net.java:461)
at java.base/sun.nio.ch.Net.bind(Net.java:453)
at java.base/sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:227)
at java.base/sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:80)
at java.base/sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:73)
at kafka.network.Acceptor.openServerSocket(SocketServer.scala:569)
... 68 common frames omitted
broker.properties
listeners=PLAINTEXT://localhost:9092
log.dir=/tmp/data/logs
auto.create.topics.enable=true
port=9092
Sample Test
#RunWith(SpringRunner.class)
#SpringBootTest(classes = SomeSampleTestConfig.class)
#DirtiesContext
#EmbeddedKafka(
partitions = 1,
topics = {
"someRandomTopic"},
brokerPropertiesLocation = "classpath:/broker.properties"
)
public class SampleStreamTest {
#Autowired
private EmbeddedKafkaBroker embeddedKafkaBroker;
#Test
public void shouldTestStream() {
}
}

Answer 1

listeners=PLAINTEXT://localhost:9092
Caused by: java.net.BindException: Address already in use
You can't use the same port if you already have a broker listening on 9092
You also must override all the kafka jar versions, as discussed in the documentation.