Kafka starts throwing replication factor error as soon as spring boot app connects to it - spring-boot

Question

Kafka starts flooding logs with the the error
INFO [Admin Manager on Broker 2]: Error processing create topic request CreatableTopic(name='__consumer_offsets', numPartitions=50, replicationFactor=3, assignments=[], configs=[CreateableTopicConfig(name='compression.type', value='producer'), CreateableTopicConfig(name='cleanup.policy', value='compact'), CreateableTopicConfig(name='segment.bytes', value='104857600')]) (kafka.server.ZkAdminManager)
org.apache.kafka.common.errors.InvalidReplicationFactorException: Replication factor: 3 larger than available brokers: 1.
As soon as the spring boot application connects. Here's the Kafka config I pass via application yaml
spring:
cloud:
stream:
instanceIndex: 0
kafka:
default:
producer:
topic:
replication-factor: 1
configuration:
key:
serializer: org.apache.kafka.common.serialization.StringSerializer
spring:
json:
add:
type:
headers: 'false'
value:
serializer: org.springframework.kafka.support.serializer.JsonSerializer
max:
block:
ms: '100'
consumer:
topic:
replication-factor: 1
configuration:
key:
deserializer: org.apache.kafka.common.serialization.StringDeserializer
spring:
json:
trusted:
packages: com.message
value:
deserializer: org.springframework.kafka.support.serializer.JsonDeserializer
binder:
autoCreateTopics: 'false'
brokers: localhost:19092
replicationFactor: 1
bindings:
consume-in-0:
group: ${category}-${spring.application.name}-consumer-${runtime-env}
destination: alm-tom-${runtime-env}
publish-out-0:
destination: ${category}-${spring.application.name}-${runtime-env}
I don't see any other config to control consumer-offsets topic replication factor

Answer 1

The error is coming from the broker itself. It will only start logging this when you create your first consumer group...
Check the server.properties file for offsets.topic.replication.factor.
Similarly, the transactions topic will have its own replication factor that needs reduced (assuming you're using a newer Kafka version)