Spring Boot application reads same messages from Kafka whenever started - spring-boot

Question

I have Zookeeper and Apache Kafka servers running on my Windows computer. The problem is with a Spring Boot application: it reads the same messages from Kafka whenever I start it. It means the offset is not being saved. How do I fix it?
Versions are: kafka_2.12-2.4.0, Spring Boot 2.5.0.
In Kafka listener bean, I have
factory.getContainerProperties().setAckMode(AckMode.MANUAL_IMMEDIATE);
Consumer config values printed on console when Spring Boot starts are:
2021-06-10 18:21:11.008 INFO 7036 --- [ main] o.a.k.clients.consumer.ConsumerConfig : ConsumerConfig values:
allow.auto.create.topics = true
auto.commit.interval.ms = 5000
auto.offset.reset = latest
bootstrap.servers = [http://localhost:9092]
check.crcs = true
client.dns.lookup = use_all_dns_ips
client.id = consumer-group_id1-1
client.rack =
connections.max.idle.ms = 540000
default.api.timeout.ms = 60000
enable.auto.commit = false
exclude.internal.topics = true
fetch.max.bytes = 52428800
fetch.max.wait.ms = 500
fetch.min.bytes = 1
group.id = group_id1
group.instance.id = null
heartbeat.interval.ms = 3000
interceptor.classes = []
internal.leave.group.on.close = true
internal.throw.on.fetch.stable.offset.unsupported = false
isolation.level = read_uncommitted
key.deserializer = class org.apache.kafka.common.serialization.StringDeserializer
max.partition.fetch.bytes = 1048576
max.poll.interval.ms = 300000
max.poll.records = 500
metadata.max.age.ms = 300000
metric.reporters = []
metrics.num.samples = 2
metrics.recording.level = INFO
metrics.sample.window.ms = 30000
partition.assignment.strategy = [class org.apache.kafka.clients.consumer.RangeAssignor]
receive.buffer.bytes = 65536
reconnect.backoff.max.ms = 1000
reconnect.backoff.ms = 50
request.timeout.ms = 30000
retry.backoff.ms = 100
sasl.client.callback.handler.class = null
sasl.jaas.config = null
sasl.kerberos.kinit.cmd = /usr/bin/kinit
sasl.kerberos.min.time.before.relogin = 60000
sasl.kerberos.service.name = null
sasl.kerberos.ticket.renew.jitter = 0.05
sasl.kerberos.ticket.renew.window.factor = 0.8
sasl.login.callback.handler.class = null
sasl.login.class = null
sasl.login.refresh.buffer.seconds = 300
sasl.login.refresh.min.period.seconds = 60
sasl.login.refresh.window.factor = 0.8
sasl.login.refresh.window.jitter = 0.05
sasl.mechanism = GSSAPI
security.protocol = PLAINTEXT
security.providers = null
send.buffer.bytes = 131072
session.timeout.ms = 10000
socket.connection.setup.timeout.max.ms = 127000
socket.connection.setup.timeout.ms = 10000
ssl.cipher.suites = null
ssl.enabled.protocols = [TLSv1.2, TLSv1.3]
ssl.endpoint.identification.algorithm = https
ssl.engine.factory.class = null
ssl.key.password = null
ssl.keymanager.algorithm = SunX509
ssl.keystore.certificate.chain = null
ssl.keystore.key = null
ssl.keystore.location = null
ssl.keystore.password = null
ssl.keystore.type = JKS
ssl.protocol = TLSv1.3
ssl.provider = null
ssl.secure.random.implementation = null
ssl.trustmanager.algorithm = PKIX
ssl.truststore.certificates = null
ssl.truststore.location = null
ssl.truststore.password = null
ssl.truststore.type = JKS
value.deserializer = class org.apache.kafka.common.serialization.StringDeserializer
UPDATE 11-Jun-2021
Using #nipuna's suggestion, in my Kafka consumer config, I set
props.put(ConsumerConfig.ENABLE_AUTO_COMMIT_CONFIG, true);
However, this was not compatible with ack mode manual_immediate, so I deleted the following line to use the default ack mode (batch, I think).
factory.getContainerProperties().setAckMode(AckMode.MANUAL_IMMEDIATE);

Answer 1

Your issue is here enable.auto.commit = false. If you are not manually committing offset after consuming messages, You should configure this to true
If this is set to false, after consuming messages from Kafka, there is no feedback to Kafka whether you read or not. Then after you restart your consumer it will send messages from the start. If you enable this, your consumer make sure it will automatically send your last read offset to Kafka. Then Kafka saved that offset in __consumer_offsets topic with your consumer group_id, topic you consumed and partition.
Then after you restart the consumer, Kafka read your last position from __consumer_offsets topic and send from there.