I created Spring boot multi module project with lettuce.
My config :
#Bean
public LettuceConnectionFactory lettuceConnectionFactory(){
return new LettuceConnectionFactory(host, port);
}
#Bean
public RedisTemplate<String, FullMessage> redisTemplate() {
RedisTemplate<String, FullMessage> template = new RedisTemplate<>();
template.setConnectionFactory(lettuceConnectionFactory());
return template;
}
I use RedisTemplate other module asynchronously:
#Async
#EventListener(ApplicationReadyEvent.class)
public void check() {
while (true){
try{
List<ObjectRecord<String, FullMessage>> records = redisTemplate
.opsForStream()
.read(FullMessage.class, StreamOffset.fromStart(key));
...
}
This code doesn`t work in multi module project but working another project(not multi module)
I am gettting the following error(infinity):
java.lang.IllegalStateException: Cannot connect, Event executor group is terminated.
at io.lettuce.core.AbstractRedisClient.initializeChannelAsync(AbstractRedisClient.java:283)
at io.lettuce.core.RedisClient.connectStatefulAsync(RedisClient.java:314)
at io.lettuce.core.RedisClient.connectStandaloneAsync(RedisClient.java:271)
at io.lettuce.core.RedisClient.connect(RedisClient.java:204)
at org.springframework.data.redis.connection.lettuce.StandaloneConnectionProvider.lambda$getConnection$1(StandaloneConnectionProvider.java:115)
at java.base/java.util.Optional.orElseGet(Optional.java:369)
at org.springframework.data.redis.connection.lettuce.StandaloneConnectionProvider.getConnection(StandaloneConnectionProvider.java:115)
at org.springframework.data.redis.connection.lettuce.LettuceConnectionFactory$SharedConnection.getNativeConnection(LettuceConnectionFactory.java:1197)
at org.springframework.data.redis.connection.lettuce.LettuceConnectionFactory$SharedConnection.getConnection(LettuceConnectionFactory.java:1178)
at org.springframework.data.redis.connection.lettuce.LettuceConnectionFactory.getSharedConnection(LettuceConnectionFactory.java:942)
at org.springframework.data.redis.connection.lettuce.LettuceConnectionFactory.getConnection(LettuceConnectionFactory.java:353)
at org.springframework.data.redis.core.RedisConnectionUtils.doGetConnection(RedisConnectionUtils.java:132)
at org.springframework.data.redis.core.RedisConnectionUtils.getConnection(RedisConnectionUtils.java:95)
at org.springframework.data.redis.core.RedisConnectionUtils.getConnection(RedisConnectionUtils.java:82)
at org.springframework.data.redis.core.RedisTemplate.execute(RedisTemplate.java:215)
at org.springframework.data.redis.core.RedisTemplate.execute(RedisTemplate.java:188)
at org.springframework.data.redis.core.AbstractOperations.execute(AbstractOperations.java:96)
at org.springframework.data.redis.core.DefaultStreamOperations.read(DefaultStreamOperations.java:217)
at org.springframework.data.redis.core.StreamOperations.read(StreamOperations.java:319)
at org.springframework.data.redis.core.StreamOperations.read(StreamOperations.java:290)
at uz.test.websocket.config.MainFIFOListener.check(MainFIFOListener.java:49)
at uz.test.websocket.config.MainFIFOListener$$FastClassBySpringCGLIB$$a659b30d.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:749)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at org.springframework.aop.interceptor.AsyncExecutionInterceptor.lambda$invoke$0(AsyncExecutionInterceptor.java:115)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
Related
There is spring-boot application with kafka dependecy,
there are two Kafka topics and need to read messages from them
tacocloud.orders.topic
tacocloud.tacos.topic
And already successful sent messages in it
Configured kafka config for listen this topics like this
package com.example.tacocloud.config;
import com.example.tacocloud.model.jpa.Order;
import com.example.tacocloud.model.jpa.Taco;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.boot.context.properties.EnableConfigurationProperties;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.kafka.annotation.EnableKafka;
import org.springframework.kafka.config.ConcurrentKafkaListenerContainerFactory;
import org.springframework.kafka.core.DefaultKafkaConsumerFactory;
import org.springframework.kafka.core.DefaultKafkaProducerFactory;
import org.springframework.kafka.core.KafkaTemplate;
import org.springframework.kafka.core.ProducerFactory;
import org.springframework.kafka.listener.ContainerProperties;
import org.springframework.kafka.listener.KafkaMessageListenerContainer;
import org.springframework.kafka.listener.MessageListener;
import java.util.HashMap;
import java.util.Map;
#Slf4j
#Configuration
#EnableKafka
#EnableConfigurationProperties
public class KafkaConfig {
// Order
#Bean
#ConfigurationProperties("spring.kafka.consumer.order")
public Map<String, Object> orderConsumerConfig() {
return new HashMap<>();
}
#Bean
public DefaultKafkaConsumerFactory<Object, Order> orderConsumerFactory(#Qualifier("orderConsumerConfig")
Map<String, Object> orderConsumerConfig) {
return new DefaultKafkaConsumerFactory<>(orderConsumerConfig);
}
#Bean
public ConcurrentKafkaListenerContainerFactory<String, Order> order1KafkaMessageListenerContainer(
#Qualifier("orderConsumerConfig") Map<String, Object> orderConsumerConfig,
#Qualifier("orderConsumerFactory") DefaultKafkaConsumerFactory orderConsumerFactory) {
ConcurrentKafkaListenerContainerFactory factory = new ConcurrentKafkaListenerContainerFactory();
factory.setConsumerFactory(orderConsumerFactory);
return factory;
}
// Taco
#Bean
#ConfigurationProperties("spring.kafka.consumer.taco")
public Map<String, Object> tacoConsumerConfig() {
return new HashMap<>();
}
#Bean
public DefaultKafkaConsumerFactory tacoConsumerFactory(
#Qualifier("tacoConsumerConfig") Map<String, Object> tacoConsumerConfig) {
return new DefaultKafkaConsumerFactory<>(tacoConsumerConfig);
}
#Bean
public ConcurrentKafkaListenerContainerFactory tacoConcurrentMessageListenerContainer(
#Qualifier("tacoConsumerConfig") Map<String, Object> tacoConsumerConfig,
#Qualifier("tacoConsumerFactory") DefaultKafkaConsumerFactory tacoConsumerFactory) {
ConcurrentKafkaListenerContainerFactory factory = new ConcurrentKafkaListenerContainerFactory();
factory.setConsumerFactory(tacoConsumerFactory);
return factory;
}
}
So, there are two DefaultKafkaConsumerFactory and two ConcurrentKafkaListenerContainerFactory
Aften that, created a service with #KafkaListener log messages:
package com.example.tacocloud.service;
import org.springframework.kafka.annotation.EnableKafka;
import org.springframework.kafka.annotation.KafkaListener;
import org.springframework.stereotype.Service;
#Service
#EnableKafka
public class KafkaService {
#KafkaListener(topics = "tacocloud.orders.topic", groupId = "one")
public void order() {
System.out.println("Order");
}
#KafkaListener(topics ="tacocloud.tacos.topic", groupId = "two")
public void taco() {
System.out.println("Taco");
}
}
application.yml file
spring:
kafka:
consumer:
order:
topic: tacocloud.orders.topic
"[bootstrap.servers]": localhost:29888
"[key.deserializer]": org.apache.kafka.common.serialization.StringDeserializer
"[value.deserializer]": com.example.tacocloud.model.serialization.OrderDeserializer
template:
"[default.topic]": tacocloud.orders.topic
taco:
topic: tacocloud.tacos.topic
"[bootstrap.servers]": localhost:29888
"[key.deserializer]": org.apache.kafka.common.serialization.StringDeserializer
"[value.deserializer]": com.example.tacocloud.model.serialization.TacoDeserializer
template:
"[default.topic]": tacocloud.tacos.topic
But, when I start my spring-boot application, I see error message(((
2022-04-15 21:38:35.918 ERROR 13888 --- [ restartedMain]
o.s.boot.SpringApplication : Application run failed
org.springframework.context.ApplicationContextException: Failed to
start bean
'org.springframework.kafka.config.internalKafkaListenerEndpointRegistry';
nested exception is org.apache.kafka.common.config.ConfigException:
Missing required configuration "key.deserializer" which has no default
value. at
org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:181)
~[spring-context-5.3.16.jar:5.3.16] at
org.springframework.context.support.DefaultLifecycleProcessor.access$200(DefaultLifecycleProcessor.java:54)
~[spring-context-5.3.16.jar:5.3.16] at
org.springframework.context.support.DefaultLifecycleProcessor$LifecycleGroup.start(DefaultLifecycleProcessor.java:356)
~[spring-context-5.3.16.jar:5.3.16] at
java.base/java.lang.Iterable.forEach(Iterable.java:75) ~[na:na] at
org.springframework.context.support.DefaultLifecycleProcessor.startBeans(DefaultLifecycleProcessor.java:155)
~[spring-context-5.3.16.jar:5.3.16] at
org.springframework.context.support.DefaultLifecycleProcessor.onRefresh(DefaultLifecycleProcessor.java:123)
~[spring-context-5.3.16.jar:5.3.16] at
org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:935)
~[spring-context-5.3.16.jar:5.3.16] at
org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:586)
~[spring-context-5.3.16.jar:5.3.16] at
org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:145)
~[spring-boot-2.6.4.jar:2.6.4] at
org.springframework.boot.SpringApplication.refresh(SpringApplication.java:740)
~[spring-boot-2.6.4.jar:2.6.4] at
org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:415)
~[spring-boot-2.6.4.jar:2.6.4] at
org.springframework.boot.SpringApplication.run(SpringApplication.java:303)
~[spring-boot-2.6.4.jar:2.6.4] at
org.springframework.boot.SpringApplication.run(SpringApplication.java:1312)
~[spring-boot-2.6.4.jar:2.6.4] at
org.springframework.boot.SpringApplication.run(SpringApplication.java:1301)
~[spring-boot-2.6.4.jar:2.6.4] at
com.example.tacocloud.TacoCloudApplication.main(TacoCloudApplication.java:10)
~[classes/:na] at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
Method) ~[na:na] at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
~[na:na] at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
~[na:na] at
java.base/java.lang.reflect.Method.invoke(Method.java:566) ~[na:na]
at
org.springframework.boot.devtools.restart.RestartLauncher.run(RestartLauncher.java:49)
~[spring-boot-devtools-2.6.4.jar:2.6.4] Caused by:
org.apache.kafka.common.config.ConfigException: Missing required
configuration "key.deserializer" which has no default value. at
org.apache.kafka.common.config.ConfigDef.parseValue(ConfigDef.java:493)
~[kafka-clients-2.8.0.jar:na] at
org.apache.kafka.common.config.ConfigDef.parse(ConfigDef.java:483)
~[kafka-clients-2.8.0.jar:na] at
org.apache.kafka.common.config.AbstractConfig.(AbstractConfig.java:108)
~[kafka-clients-2.8.0.jar:na] at
org.apache.kafka.common.config.AbstractConfig.(AbstractConfig.java:129)
~[kafka-clients-2.8.0.jar:na] at
org.apache.kafka.clients.consumer.ConsumerConfig.(ConsumerConfig.java:640)
~[kafka-clients-2.8.0.jar:na] at
org.apache.kafka.clients.consumer.KafkaConsumer.(KafkaConsumer.java:665)
~[kafka-clients-2.8.0.jar:na] at
org.springframework.kafka.core.DefaultKafkaConsumerFactory.createRawConsumer(DefaultKafkaConsumerFactory.java:416)
~[spring-kafka-2.8.3.jar:2.8.3] at
org.springframework.kafka.core.DefaultKafkaConsumerFactory.createKafkaConsumer(DefaultKafkaConsumerFactory.java:384)
~[spring-kafka-2.8.3.jar:2.8.3] at
org.springframework.kafka.core.DefaultKafkaConsumerFactory.createConsumerWithAdjustedProperties(DefaultKafkaConsumerFactory.java:360)
~[spring-kafka-2.8.3.jar:2.8.3] at
org.springframework.kafka.core.DefaultKafkaConsumerFactory.createKafkaConsumer(DefaultKafkaConsumerFactory.java:327)
~[spring-kafka-2.8.3.jar:2.8.3] at
org.springframework.kafka.core.DefaultKafkaConsumerFactory.createConsumer(DefaultKafkaConsumerFactory.java:304)
~[spring-kafka-2.8.3.jar:2.8.3] at
org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.(KafkaMessageListenerContainer.java:758)
~[spring-kafka-2.8.3.jar:2.8.3] at
org.springframework.kafka.listener.KafkaMessageListenerContainer.doStart(KafkaMessageListenerContainer.java:344)
~[spring-kafka-2.8.3.jar:2.8.3] at
org.springframework.kafka.listener.AbstractMessageListenerContainer.start(AbstractMessageListenerContainer.java:442)
~[spring-kafka-2.8.3.jar:2.8.3] at
org.springframework.kafka.listener.ConcurrentMessageListenerContainer.doStart(ConcurrentMessageListenerContainer.java:209)
~[spring-kafka-2.8.3.jar:2.8.3] at
org.springframework.kafka.listener.AbstractMessageListenerContainer.start(AbstractMessageListenerContainer.java:442)
~[spring-kafka-2.8.3.jar:2.8.3] at
org.springframework.kafka.config.KafkaListenerEndpointRegistry.startIfNecessary(KafkaListenerEndpointRegistry.java:331)
~[spring-kafka-2.8.3.jar:2.8.3] at
org.springframework.kafka.config.KafkaListenerEndpointRegistry.start(KafkaListenerEndpointRegistry.java:276)
~[spring-kafka-2.8.3.jar:2.8.3] at
org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:178)
~[spring-context-5.3.16.jar:5.3.16] ... 19 common frames omitted
Process finished with exit code 0
Thank you for a sample.
So, I opened it locally and placed a break point into this bean definition:
#Bean
public DefaultKafkaConsumerFactory<Object, Order> orderConsumerFactory(#Qualifier("orderConsumerConfig")
Map<String, Object> orderConsumerConfig) {
return new DefaultKafkaConsumerFactory<Object, Order>(orderConsumerConfig);
}
That orderConsumerConfig map looks like this:
orderConsumerConfig = {LinkedHashMap#8587} size = 1
"orderConsumerConfig" -> {HashMap#8600} size = 5
key = "orderConsumerConfig"
value = {HashMap#8600} size = 5
"key.deserializer" -> "org.apache.kafka.common.serialization.StringDeserializer"
"template" -> {LinkedHashMap#8611} size = 1
"topic" -> "tacocloud.orders.topic"
"bootstrap.servers" -> "localhost:29888"
"value.deserializer" -> "sample.kafka.serializer.OrderDeserializer"
so, that's indeed not a surprise that your KafkaConsumer is not initialized properly. Your target map config is hidden under that orderConsumerConfig entry of this injected map.
Would you mind to share with me from where did you take an idea of the #ConfigurationProperties on the Map bean? And how that supposed to be used as dependency injection?
I wanted to do something similar (configure multiple ConsumerFactories) based on properties.
I used #ConfigurationProperties to create a Map<String,String> instead of Map<String,Object> and then added the items of that map into a new Map<String,Object>. Not sure why Spring-Boot loaded the Map<String,Object> that way.
#Bean
#ConfigurationProperties("taco-cart.kafka")
public Map<String, String> tacoCartKafkaProperties() {
return new HashMap<>();
}
#Bean
public ConsumerFactory<String, TacoCart> tacoCartConsumerFactory(#Qualifier("tacoCartKafkaProperties") Map<String, String> tacoCartKafkaProperties) {
// Convert map.
Map<String, Object> config = new HashMap<>();
config.putAll(tacoCartKafkaProperties);
return new DefaultKafkaConsumerFactory<>(config);
}
Everything is mostly in the title.
I have a specific channel to send data to PubSub, using Spring integration and this information about GCP PubSub
I don't have any problem locally, or on the QA env.
But, in the prod, I have the following error :
org.springframework.messaging.MessageDeliveryException: failed to send Message to channel
'pubSubFlow.channel#1'; nested exception is java.lang.IllegalStateException: The [bean
'pubSubFlow.channel#1'; defined in: 'class path resource [fr/auchan/lark/tracking/api/v1
/pubsub/PubSubRequestIntegration.class]'; from source: 'bean method pubSubFlow'] doesn't
have subscribers to accept messages
org.springframework.integration.support.utils.IntegrationUtils.wrapInDeliveryExceptionIfNecessary(IntegrationUtils.java:167) ~[spring-integration-core-5.3.5.RELEASE.jar:5.3.5.RELEASE]
org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:600) ~[spring-integration-core-5.3.5.RELEASE.jar:5.3.5.RELEASE]
org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:520) ~[spring-integration-core-5.3.5.RELEASE.jar:5.3.5.RELEASE]
org.springframework.integration.channel.FluxMessageChannel.lambda$subscribeTo$2(FluxMessageChannel.java:83) ~[spring-integration-core-5.3.5.RELEASE.jar:5.3.5.RELEASE]
reactor.core.publisher.FluxPeekFuseable$PeekFuseableSubscriber.onNext(FluxPeekFuseable.java:189) ~[reactor-core-3.3.13.RELEASE.jar:3.3.13.RELEASE]
reactor.core.publisher.FluxPublishOn$PublishOnSubscriber.runAsync(FluxPublishOn.java:439) ~[reactor-core-3.3.13.RELEASE.jar:3.3.13.RELEASE]
reactor.core.publisher.FluxPublishOn$PublishOnSubscriber.run(FluxPublishOn.java:526) ~[reactor-core-3.3.13.RELEASE.jar:3.3.13.RELEASE]
reactor.core.scheduler.WorkerTask.call(WorkerTask.java:84) ~[reactor-core-3.3.13.RELEASE.jar:3.3.13.RELEASE]
reactor.core.scheduler.WorkerTask.call(WorkerTask.java:37) ~[reactor-core-3.3.13.RELEASE.jar:3.3.13.RELEASE]
java.base/java.util.concurrent.FutureTask.run(Unknown Source) ~[na:na]
java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) ~[na:na]
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) ~[na:na]
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ~[na:na]
java.base/java.lang.Thread.run(Unknown Source) ~[na:na]
Caused by: java.lang.IllegalStateException: The [bean 'pubSubFlow.channel#1'; defined in: 'class path resource [PubSubRequestIntegration.class]'; from source: 'bean method pubSubFlow'] doesn't have subscribers to accept messages
org.springframework.util.Assert.state(Assert.java:97) ~[spring-core-5.2.12.RELEASE.jar:5.2.12.RELEASE]
org.springframework.integration.channel.FluxMessageChannel.doSend(FluxMessageChannel.java:61) ~[spring-integration-core-5.3.5.RELEASE.jar:5.3.5.RELEASE]
org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:570) ~[spring-integration-core-5.3.5.RELEASE.jar:5.3.5.RELEASE]
12 common frames omitted
Below is my channel declaration, and the use of ServiceActivator as written is the PubSub Guidelines (see the link above).
#Bean
public MessageChannel dataChannel() {
return MessageChannels.publishSubscribe(Executors.newCachedThreadPool()).get();
}
#Bean
public MessageChannel pubSubChannel() {
return MessageChannels.publishSubscribe(Executors.newCachedThreadPool()).get();
}
#Bean
public IntegrationFlow pubSubFlow(
MessageChannel dataChannel,
MessageChannel pubSubChannel) {
return IntegrationFlows
.from(dataChannel)
.fluxTransform(this::toPubSubFormat)
.channel(pubSubChannel)
.get();
}
#Bean
#ServiceActivator(inputChannel = "pubSubChannel")
public PubSubMessageHandler sendToPubSub(PubSubTemplate pubSubTemplate) {
PubSubMessageHandler adapter = new PubSubMessageHandler(pubSubTemplate,
pubSubIntegrationProperties.getTopic());
adapter.setPublishCallback(
new ListenableFutureCallback<>() {
#Override
public void onFailure(Throwable throwable) {
log.warn("There was the following error sending the message. " + throwable);
}
#Override
public void onSuccess(String result) {
log.debug("Message was sent via the outbound channel adapter to {} : {}", pubSubIntegrationProperties.getTopic(), result);
}
});
return adapter;
}
Did I miss something? Why is the pubSubChannel marked as having no subscribers?
Thanks for the help
I am developing Spring Boot + Batch XML based approach. In this example, I have created following classes. When I simply load or class the Spring Batch Job. I get the below error.
I web search links like : Migration to Spring Boot 2 and using Spring Batch 4 , but it did not solve my problem.
Could anyone please guide what exact solution needs to be applied here ?
Error:
Caused by: java.lang.IllegalArgumentException: Unable to deserialize the execution context
at org.springframework.batch.core.repository.dao.JdbcExecutionContextDao$ExecutionContextRowMapper.mapRow(JdbcExecutionContextDao.java:325) ~[spring-batch-core-4.0.1.RELEASE.jar:4.0.1.RELEASE]
at org.springframework.batch.core.repository.dao.JdbcExecutionContextDao$ExecutionContextRowMapper.mapRow(JdbcExecutionContextDao.java:309) ~[spring-batch-core-4.0.1.RELEASE.jar:4.0.1.RELEASE]
at org.springframework.jdbc.core.RowMapperResultSetExtractor.extractData(RowMapperResultSetExtractor.java:93) ~[spring-jdbc-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.springframework.jdbc.core.RowMapperResultSetExtractor.extractData(RowMapperResultSetExtractor.java:60) ~[spring-jdbc-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:667) ~[spring-jdbc-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:605) ~[spring-jdbc-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:657) ~[spring-jdbc-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:688) ~[spring-jdbc-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:700) ~[spring-jdbc-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:756) ~[spring-jdbc-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.springframework.batch.core.repository.dao.JdbcExecutionContextDao.getExecutionContext(JdbcExecutionContextDao.java:112) ~[spring-batch-core-4.0.1.RELEASE.jar:4.0.1.RELEASE]
at org.springframework.batch.core.explore.support.SimpleJobExplorer.getJobExecutionDependencies(SimpleJobExplorer.java:202) ~[spring-batch-core-4.0.1.RELEASE.jar:4.0.1.RELEASE]
at org.springframework.batch.core.explore.support.SimpleJobExplorer.getJobExecutions(SimpleJobExplorer.java:83) ~[spring-batch-core-4.0.1.RELEASE.jar:4.0.1.RELEASE]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_162]
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) ~[na:1.8.0_162]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) ~[na:1.8.0_162]
at java.lang.reflect.Method.invoke(Unknown Source) ~[na:1.8.0_162]
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343) ~[spring-aop-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:197) ~[spring-aop-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) ~[spring-aop-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.springframework.batch.core.configuration.annotation.SimpleBatchConfiguration$PassthruAdvice.invoke(SimpleBatchConfiguration.java:127) ~[spring-batch-core-4.0.1.RELEASE.jar:4.0.1.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185) ~[spring-aop-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212) ~[spring-aop-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at com.sun.proxy.$Proxy70.getJobExecutions(Unknown Source) ~[na:na]
at org.springframework.batch.core.JobParametersBuilder.getNextJobParameters(JobParametersBuilder.java:264) ~[spring-batch-core-4.0.1.RELEASE.jar:4.0.1.RELEASE]
at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.execute(JobLauncherCommandLineRunner.java:162) ~[spring-boot-autoconfigure-2.0.4.RELEASE.jar:2.0.4.RELEASE]
at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.executeLocalJobs(JobLauncherCommandLineRunner.java:179) ~[spring-boot-autoconfigure-2.0.4.RELEASE.jar:2.0.4.RELEASE]
at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.launchJobFromProperties(JobLauncherCommandLineRunner.java:134) ~[spring-boot-autoconfigure-2.0.4.RELEASE.jar:2.0.4.RELEASE]
at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.run(JobLauncherCommandLineRunner.java:128) ~[spring-boot-autoconfigure-2.0.4.RELEASE.jar:2.0.4.RELEASE]
at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:800) [spring-boot-2.0.4.RELEASE.jar:2.0.4.RELEASE]
... 5 common frames omitted
Caused by: com.fasterxml.jackson.databind.exc.MismatchedInputException: Unexpected token (START_OBJECT), expected VALUE_STRING: need JSON String that contains type id (for subtype of java.lang.Object)
at [Source: (ByteArrayInputStream); line: 1, column: 9] (through reference chain: java.util.HashMap["map"])
at com.fasterxml.jackson.databind.exc.MismatchedInputException.from(MismatchedInputException.java:59) ~[jackson-databind-2.9.6.jar:2.9.6]
at com.fasterxml.jackson.databind.DeserializationContext.wrongTokenException(DeserializationContext.java:1498) ~[jackson-databind-2.9.6.jar:2.9.6]
at com.fasterxml.jackson.databind.DeserializationContext.reportWrongTokenException(DeserializationContext.java:1273) ~[jackson-databind-2.9.6.jar:2.9.6]
at com.fasterxml.jackson.databind.jsontype.impl.AsArrayTypeDeserializer._locateTypeId(AsArrayTypeDeserializer.java:151) ~[jackson-databind-2.9.6.jar:2.9.6]
at com.fasterxml.jackson.databind.jsontype.impl.AsArrayTypeDeserializer._deserialize(AsArrayTypeDeserializer.java:96) ~[jackson-databind-2.9.6.jar:2.9.6]
at com.fasterxml.jackson.databind.jsontype.impl.AsArrayTypeDeserializer.deserializeTypedFromAny(AsArrayTypeDeserializer.java:71) ~[jackson-databind-2.9.6.jar:2.9.6]
at com.fasterxml.jackson.databind.deser.std.UntypedObjectDeserializer$Vanilla.deserializeWithType(UntypedObjectDeserializer.java:712) ~[jackson-databind-2.9.6.jar:2.9.6]
at com.fasterxml.jackson.databind.deser.std.MapDeserializer._readAndBindStringKeyMap(MapDeserializer.java:529) ~[jackson-databind-2.9.6.jar:2.9.6]
at com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:364) ~[jackson-databind-2.9.6.jar:2.9.6]
at com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:29) ~[jackson-databind-2.9.6.jar:2.9.6]
at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4013) ~[jackson-databind-2.9.6.jar:2.9.6]
at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3077) ~[jackson-databind-2.9.6.jar:2.9.6]
at org.springframework.batch.core.repository.dao.Jackson2ExecutionContextStringSerializer.deserialize(Jackson2ExecutionContextStringSerializer.java:70) ~[spring-batch-core-4.0.1.RELEASE.jar:4.0.1.RELEASE]
at org.springframework.batch.core.repository.dao.Jackson2ExecutionContextStringSerializer.deserialize(Jackson2ExecutionContextStringSerializer.java:50) ~[spring-batch-core-4.0.1.RELEASE.jar:4.0.1.RELEASE]
at org.springframework.batch.core.repository.dao.JdbcExecutionContextDao$ExecutionContextRowMapper.mapRow(JdbcExecutionContextDao.java:322) ~[spring-batch-core-4.0.1.RELEASE.jar:4.0.1.RELEASE]
... 34 common frames omitted
CommonConfig.java
#Configuration
#ComponentScan("com.XXXX")
#EnableBatchProcessing
#EnableScheduling
#PropertySource("classpath:database.properties")
#ImportResource({ "classpath:jobs/XYZ.xml"})
public class CommonConfig {
#Bean
BatchConfigurer configurer(#Qualifier("dataSource") DataSource dataSource) {
return new DefaultBatchConfigurer(dataSource);
}
}
XYZ.xml
<bean id="databaseConfig" class="com.XX.config.DatabaseConfig" />
<bean id="transactionManager" class="org.springframework.batch.support.transaction.ResourcelessTransactionManager" />
<bean id="job_Repository" class="org.springframework.batch.core.repository.support.MapJobRepositoryFactoryBean">
<property name="transactionManager" ref="transactionManager" />
</bean>
<bean id="jobLauncher" class="org.springframework.batch.core.launch.support.SimpleJobLauncher">
<property name="jobRepository" ref="job_Repository" />
</bean>
<bean id="stepScope" class="org.springframework.batch.core.scope.StepScope">
<property name="autoProxy" value="true"/>
</bean>
<batch:job id="myXYZBatchJob">
<batch:step id="XYZContextStep" >
<batch:tasklet ref=XYZContextTasklet" />
</batch:step>
</batch:job>
......
........
</bean>
DatabaseConfig.java
#Configuration
#EnableTransactionManagement
#ComponentScan({ "com.XXX" })
#EnableJpaRepositories(basePackages = {"com.XX.repository", "com.XX.custom.repository"},
entityManagerFactoryRef = "entityManagerFactory",
transactionManagerRef = "transactionManager")
public class DatabaseConfig {
#Bean
#Primary
#ConfigurationProperties("abc.datasource")
public DataSourceProperties dataSourceProperties() {
return new DataSourceProperties();
}
#Primary
#Bean(name = "dataSource")
#ConfigurationProperties(prefix = "abc.datasource")
public DataSource dataSource() {
return dataSourceProperties().initializeDataSourceBuilder().build();
}
#Primary
#Bean(name = "entityManagerFactory")
public LocalContainerEntityManagerFactoryBean entityManagerFactory(#Qualifier("dataSource") DataSource dataSource) {
LocalContainerEntityManagerFactoryBean em = new LocalContainerEntityManagerFactoryBean();
em.setDataSource(dataSource);
em.setPackagesToScan(new String[] { "com.XX.Entity" });
em.setPersistenceUnitName("devcloud");
JpaVendorAdapter vendorAdapter = new HibernateJpaVendorAdapter();
em.setJpaVendorAdapter(vendorAdapter);
return em;
}
#Primary
#Bean(name = "transactionManager")
public PlatformTransactionManager transactionManager(#Qualifier("entityManagerFactory") EntityManagerFactory entityManagerFactory) {
return new JpaTransactionManager(entityManagerFactory);
}
}
EDIT-1:
I am using Spring Boot Parent version 2.0.4.RELEASE, which is latest. It pulls all the Spring versions including Context version of 5.0.8.RELEASE.
Request you to post code snippet and detailed description to support your answer.
Edit-2:
public class BillingConfig implements BatchConfigurer{
private PlatformTransactionManager transactionManager;
private JobRepository jobRepository;
private JobLauncher jobLauncher;
private JobExplorer jobExplorer;
#Override
public JobRepository getJobRepository() {
return jobRepository;
}
#Override
public PlatformTransactionManager getTransactionManager() {
return transactionManager;
}
#Override
public JobLauncher getJobLauncher() throws JobExecutionAlreadyRunningException, JobRestartException, JobInstanceAlreadyCompleteException, JobParametersInvalidException {
return jobLauncher;
}
#Override
public JobExplorer getJobExplorer() {
return jobExplorer;
}
#PostConstruct
void initialize() throws Exception {
if (this.transactionManager == null) {
this.transactionManager = new ResourcelessTransactionManager();
}
// A FactoryBean that automates the creation of a SimpleJobRepository using non-persistent in-memory DAO implementations.
MapJobRepositoryFactoryBean jobRepositoryFactory = new MapJobRepositoryFactoryBean(this.transactionManager);
jobRepositoryFactory.afterPropertiesSet();
this.jobRepository = jobRepositoryFactory.getObject();
// A FactoryBean that automates the creation of a SimpleJobRepository using non-persistent in-memory DAO implementations.
MapJobExplorerFactoryBean jobExplorerFactory = new MapJobExplorerFactoryBean(jobRepositoryFactory);
jobExplorerFactory.afterPropertiesSet();
this.jobExplorer = jobExplorerFactory.getObject();
this.jobLauncher = createJobLauncher();
}
private JobLauncher createJobLauncher() throws Exception {
SimpleJobLauncher simpleJobLauncher = new SimpleJobLauncher();
simpleJobLauncher.setJobRepository(jobRepository);
simpleJobLauncher.afterPropertiesSet();
return simpleJobLauncher;
}
}
This error happens when the execution context of your job is serialized with version 3 (using XStream by default) and then deserialized with version 4 (using Jackson by default). So either downgrade Spring Batch to version 3 or configure your job repository to use the XStreamExecutionContextStringSerializer.
In your case, you have already defined a bean of type BatchConfigurer, so you can override the createJobRepository method and configure the XStream serializer. For example:
#Bean
BatchConfigurer configurer(#Qualifier("dataSource") DataSource dataSource, PlatformTransactionManager transactionManager) {
return new DefaultBatchConfigurer(dataSource) {
#Override
protected JobRepository createJobRepository() throws Exception {
JobRepositoryFactoryBean factory = new JobRepositoryFactoryBean();
factory.setDataSource(dataSource);
factory.setTransactionManager(transactionManager);
factory.setSerializer(new XStreamExecutionContextStringSerializer());
factory.afterPropertiesSet();
return factory.getObject();
}
};
}
Try cleaning up the spring batch tables.
Use following link :
--cleaning spring batch tables ----
http://forum.spring.io/forum/spring-projects/batch/122103-clean-spring-batch-metadata-tables
Since XStreamExecutionContextStringSerializer is deprecated with the latest Spring BATCH release.
If you are adding a complex object to Execution Context Check if the object you are trying to add is Serializable.
For example, if your class looks like:
class Sample{
String data;
List<DataItem> dataItems;
}
Make sure DataItem which is present inside the object is also Serializable.
This helped me solve my issue.
I am trying to create a RedisTemplate bean which will have the updated value serializer to serialize an object in JSON format in redis.
#Configuration
class RedisConfig {
#Bean(name = ["redisTemplate"])
#Primary
fun template(factory: RedisConnectionFactory): RedisTemplate<Any, Any> {
val template = RedisTemplate<Any, Any>()
template.connectionFactory = factory
template.valueSerializer = Jackson2JsonRedisSerializer(Object::class.java)
template.afterPropertiesSet()
return template
}
}
As per my understanding, spring should use the JSON serializer to serialize the object returned by the methods marked with Cacheable annotation. Despite this configuration, spring seems to be using the default Java serializer as this exception confirms this fact.
java.io.NotSerializableException: en.prateekj.vds.dto.Task
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1184)
at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:348)
at java.util.ArrayList.writeObject(ArrayList.java:766)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:1128)
at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1496)
at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432)
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178)
at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:348)
at org.springframework.core.serializer.DefaultSerializer.serialize(DefaultSerializer.java:46)
at org.springframework.core.serializer.support.SerializingConverter.convert(SerializingConverter.java:63)
at org.springframework.core.serializer.support.SerializingConverter.convert(SerializingConverter.java:35)
at org.springframework.data.redis.serializer.JdkSerializationRedisSerializer.serialize(JdkSerializationRedisSerializer.java:94)
at org.springframework.data.redis.serializer.DefaultRedisElementWriter.write(DefaultRedisElementWriter.java:43)
at org.springframework.data.redis.serializer.RedisSerializationContext$SerializationPair.write(RedisSerializationContext.java:219)
at org.springframework.data.redis.cache.RedisCache.serializeCacheValue(RedisCache.java:238)
at org.springframework.data.redis.cache.RedisCache.put(RedisCache.java:144)
at org.springframework.cache.interceptor.AbstractCacheInvoker.doPut(AbstractCacheInvoker.java:87)
at org.springframework.cache.interceptor.CacheAspectSupport$CachePutRequest.apply(CacheAspectSupport.java:770)
at org.springframework.cache.interceptor.CacheAspectSupport.execute(CacheAspectSupport.java:398)
at org.springframework.cache.interceptor.CacheAspectSupport.execute(CacheAspectSupport.java:314)
at org.springframework.cache.interceptor.CacheInterceptor.invoke(CacheInterceptor.java:61)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:689)
Am I missing any configuration or something by which spring is not able to determine what RedisTemplate to use?
You have probably solved it meanwhile, but for further answer seekers.
According to spring data redis reference:
By default, RedisCache and RedisTemplate are configured to use Java native serialization.
From stacktrace I can see that you are actually using Redis for caching, so you need to configure RedisCache and not RedisTemplate. RedisCache is not picking up your #Bean because it is not using RedisTemplate internally.
Example how you can do it in Java:
#EnableCaching
#Configuration
public class CacheConfig {
#Bean
#Primary
public RedisCacheConfiguration defaultCacheConfig(ObjectMapper objectMapper) {
return RedisCacheConfiguration.defaultCacheConfig()
.serializeKeysWith(SerializationPair.fromSerializer(new StringRedisSerializer()))
.serializeValuesWith(SerializationPair.fromSerializer(new GenericJackson2JsonRedisSerializer(objectMapper)));
}
}
I am struggling trying to find a solution to this and I'm hoping that someone out there can help.
We have a Spring/Hibernate/Wicket/Tomcat webapp. We use Spring Batch for executing jobs in the background. Some execute every minute and check a database table in an external system to see if there are new records. So there are several jobs (maybe 8 or so) that are executing on some fixed interval. A couple of those jobs we have to do some manual queries to ensure there isn't a second one running at the same time.
The problem is that we are intermittently getting dead lock exceptions when Spring Batch is trying to update the job execution status, or some other framework state in the database which is SQL Server. The job state is then hung in whatever status it was in at that moment. So the jobs that ensure they only have one instance running at a time end up never running because it appears there is a job instance still running.
I'm considering moving to an in memory hsqldb database for the Spring Batch JobRepository only however, that may come with it's own set of problems so I'd like to at least see what others have done to address this.
EDIT
Also one thing I'm unsure about is whether retry logic will handle such a thing. I know it works for user code inside of steps but I'm unsure whether the database activity the framework performs between steps will be handled with retry logic. If someone could clarify that I would appreciate it.
I'll post my stacktrace and spring config below. We are using spring-batch 3.0.7-RELEASE, spring-core 4.2.6.RELEASE. Thanks in advance for the help!
#Configuration
#EnableScheduling //Enables the #Scheduled annotation
#EnableBatchProcessing
public class BatchConfig implements BatchConfigurer
{
#Autowired
private PlatformTransactionManager transactionManager;
#Autowired
private JobRepository jobRepository;
#Autowired
private DataSource dataSource;
private #Value("${batch.maxPoolSize}") String maxPoolSize;
private #Value("${batch.corePoolSize}") String corePoolSize;
private #Value("${batch.queueCapacity}") String queueCapacity;
#Bean
public JobOperator jobOperator() throws Exception
{
SimpleJobOperator jobOperator = new SimpleJobOperator();
jobOperator.setJobExplorer(getJobExplorer());
jobOperator.setJobRepository(getJobRepository());
jobOperator.setJobRegistry(jobRegistry());
jobOperator.setJobLauncher(getJobLauncher());
return jobOperator;
}
#Primary
#Bean
#Override
public JobLauncher getJobLauncher() throws Exception
{
SimpleJobLauncher jobLauncher = new SimpleJobLauncher();
jobLauncher.setJobRepository(getJobRepository());
jobLauncher.setTaskExecutor(asyncJobTaskExecutor());//Needed for launching jobs from webapp
return jobLauncher;
}
#Bean
public ThreadPoolTaskExecutor asyncJobTaskExecutor()
{
ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
executor.setMaxPoolSize(Integer.valueOf(maxPoolSize));
executor.setCorePoolSize(Integer.valueOf(corePoolSize));
executor.setQueueCapacity(Integer.valueOf(queueCapacity));
return executor;
}
#Bean
public JobLauncher syncJobLauncher() throws Exception
{
SimpleJobLauncher jobLauncher = new SimpleJobLauncher();
jobLauncher.setJobRepository(getJobRepository());
jobLauncher.setTaskExecutor(new SyncTaskExecutor());//Needed for launching jobs from quartz if you want to ensure more than one job doesn't execute at a time
return jobLauncher;
}
#Bean
public JobRegistry jobRegistry()
{
return new MapJobRegistry();
}
#Bean
#Override
public JobExplorer getJobExplorer() throws Exception
{
JobExplorerFactoryBean jobExplorerFactoryBean = new JobExplorerFactoryBean();
jobExplorerFactoryBean.setDataSource(this.dataSource);
jobExplorerFactoryBean.afterPropertiesSet();
return jobExplorerFactoryBean.getObject();
}
#Override
public JobRepository getJobRepository() throws Exception
{
JobRepositoryFactoryBean factory = new JobRepositoryFactoryBean();
factory.setDataSource(dataSource);
factory.setTransactionManager(transactionManager);
factory.afterPropertiesSet();
return factory.getObject();
}
#Bean
public JobService jobService() throws Exception
{
SimpleJobServiceFactoryBean factory = new SimpleJobServiceFactoryBean();
factory.setJobRepository(jobRepository);
factory.setJobLauncher(getJobLauncher());
factory.setJobLocator(jobRegistry());
factory.setDataSource(dataSource);
factory.setJobExplorer(getJobExplorer());
factory.setTransactionManager(transactionManager);
factory.afterPropertiesSet();
return factory.getObject();
}
#Bean
public JobListener jobListener()
{
return new JobListener();
}
#Override
public PlatformTransactionManager getTransactionManager() throws Exception
{
return transactionManager;
}
}
Here's an example error. It's not always in the exact same spot but this one seems to be the most prominent.
2017-05-28 02:35:00,975 ERROR [asyncJobTaskExecutor-5] o.s.b.c.j.AbstractJob [AbstractJob.java:335] Encountered fatal error executing job
org.springframework.dao.DeadlockLoserDataAccessException: PreparedStatementCallback; SQL [UPDATE BATCH_JOB_EXECUTION set START_TIME = ?, END_TIME = ?, STATUS = ?, EXIT_CODE = ?, EXIT_MESSAGE = ?, VERSION = ?, CREATE_TIME = ?, LAST_UPDATED = ? where JOB_EXECUTION_ID = ? and VERSION = ?]; Transaction (Process ID 59) was deadlocked on lock resources with another process and has been chosen as the deadlock victim. Rerun the transaction.; nested exception is java.sql.SQLException: Transaction (Process ID 59) was deadlocked on lock resources with another process and has been chosen as the deadlock victim. Rerun the transaction.
at org.springframework.jdbc.support.SQLErrorCodeSQLExceptionTranslator.doTranslate(SQLErrorCodeSQLExceptionTranslator.java:263) ~[spring-jdbc-4.2.6.RELEASE.jar:4.2.6.RELEASE]
at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:73) ~[spring-jdbc-4.2.6.RELEASE.jar:4.2.6.RELEASE]
at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:645) ~[spring-jdbc-4.2.6.RELEASE.jar:4.2.6.RELEASE]
at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:866) ~[spring-jdbc-4.2.6.RELEASE.jar:4.2.6.RELEASE]
at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:927) ~[spring-jdbc-4.2.6.RELEASE.jar:4.2.6.RELEASE]
at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:932) ~[spring-jdbc-4.2.6.RELEASE.jar:4.2.6.RELEASE]
at org.springframework.batch.core.repository.dao.JdbcJobExecutionDao.updateJobExecution(JdbcJobExecutionDao.java:224) ~[spring-batch-core-3.0.7.RELEASE.jar:3.0.7.RELEASE]
at org.springframework.batch.core.repository.support.SimpleJobRepository.update(SimpleJobRepository.java:162) ~[spring-batch-core-3.0.7.RELEASE.jar:3.0.7.RELEASE]
at sun.reflect.GeneratedMethodAccessor625.invoke(Unknown Source) ~[na:na]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_40]
at java.lang.reflect.Method.invoke(Method.java:497) ~[na:1.8.0_40]
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:302) ~[spring-aop-4.2.6.RELEASE.jar:4.2.6.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190) ~[spring-aop-4.2.6.RELEASE.jar:4.2.6.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) ~[spring-aop-4.2.6.RELEASE.jar:4.2.6.RELEASE]
at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:99) ~[spring-tx-4.2.6.RELEASE.jar:4.2.6.RELEASE]
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:281) ~[spring-tx-4.2.6.RELEASE.jar:4.2.6.RELEASE]
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96) ~[spring-tx-4.2.6.RELEASE.jar:4.2.6.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) ~[spring-aop-4.2.6.RELEASE.jar:4.2.6.RELEASE]
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:208) ~[spring-aop-4.2.6.RELEASE.jar:4.2.6.RELEASE]
at com.sun.proxy.$Proxy75.update(Unknown Source) ~[na:na]
at sun.reflect.GeneratedMethodAccessor625.invoke(Unknown Source) ~[na:na]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_40]
at java.lang.reflect.Method.invoke(Method.java:497) ~[na:1.8.0_40]
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:302) ~[spring-aop-4.2.6.RELEASE.jar:4.2.6.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190) ~[spring-aop-4.2.6.RELEASE.jar:4.2.6.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) ~[spring-aop-4.2.6.RELEASE.jar:4.2.6.RELEASE]
at org.springframework.batch.core.configuration.annotation.SimpleBatchConfiguration$PassthruAdvice.invoke(SimpleBatchConfiguration.java:127) ~[spring-batch-core-3.0.7.RELEASE.jar:3.0.7.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) ~[spring-aop-4.2.6.RELEASE.jar:4.2.6.RELEASE]
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:208) ~[spring-aop-4.2.6.RELEASE.jar:4.2.6.RELEASE]
at com.sun.proxy.$Proxy75.update(Unknown Source) ~[na:na]
at org.springframework.batch.core.job.AbstractJob.updateStatus(AbstractJob.java:422) ~[spring-batch-core-3.0.7.RELEASE.jar:3.0.7.RELEASE]
at org.springframework.batch.core.job.AbstractJob.execute(AbstractJob.java:301) ~[spring-batch-core-3.0.7.RELEASE.jar:3.0.7.RELEASE]
at org.springframework.batch.core.launch.support.SimpleJobLauncher$1.run(SimpleJobLauncher.java:135) [spring-batch-core-3.0.7.RELEASE.jar:3.0.7.RELEASE]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_40]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_40]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_40]
Caused by: java.sql.SQLException: Transaction (Process ID 59) was deadlocked on lock resources with another process and has been chosen as the deadlock victim. Rerun the transaction.
at net.sourceforge.jtds.jdbc.SQLDiagnostic.addDiagnostic(SQLDiagnostic.java:368) ~[jtds-1.2.4.jar:1.2.4]
at net.sourceforge.jtds.jdbc.TdsCore.tdsErrorToken(TdsCore.java:2820) ~[jtds-1.2.4.jar:1.2.4]
at net.sourceforge.jtds.jdbc.TdsCore.nextToken(TdsCore.java:2258) ~[jtds-1.2.4.jar:1.2.4]
at net.sourceforge.jtds.jdbc.TdsCore.getMoreResults(TdsCore.java:632) ~[jtds-1.2.4.jar:1.2.4]
at net.sourceforge.jtds.jdbc.JtdsStatement.processResults(JtdsStatement.java:584) ~[jtds-1.2.4.jar:1.2.4]
at net.sourceforge.jtds.jdbc.JtdsStatement.executeSQL(JtdsStatement.java:546) ~[jtds-1.2.4.jar:1.2.4]
at net.sourceforge.jtds.jdbc.JtdsPreparedStatement.executeUpdate(JtdsPreparedStatement.java:506) ~[jtds-1.2.4.jar:1.2.4]
at com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.executeUpdate(NewProxyPreparedStatement.java:105) ~[c3p0-0.9.1.2.jar:0.9.1.2]
at org.springframework.jdbc.core.JdbcTemplate$2.doInPreparedStatement(JdbcTemplate.java:873) ~[spring-jdbc-4.2.6.RELEASE.jar:4.2.6.RELEASE]
at org.springframework.jdbc.core.JdbcTemplate$2.doInPreparedStatement(JdbcTemplate.java:866) ~[spring-jdbc-4.2.6.RELEASE.jar:4.2.6.RELEASE]
at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:629) ~[spring-jdbc-4.2.6.RELEASE.jar:4.2.6.RELEASE]
... 33 common frames omitted
I'm going to answer my own question since a couple of people have asked what I did to fix this. I don't work for the same company so I don't have access to code but here's a summary of what I remember doing.
Create an implementation of JobRepository that wraps another JobRepository and provides retries for the specific exceptions that are causing problems.
Some pseudo code:
class RetryingJobRepository implements JobRepository
{
private JobRepository delegate;
public RetryingJobRepository(JobRepository delegate, RetryTemplate retryTemplate)
{
this.delegate = delegate;
}
public JobExecution createJobExecution(JobInstance jobInstance, JobParameters jobParameters, String jobConfigurationLocation)
{
retryTemplate.execute(context -> {
delegate.createJobExecution(jobInstance, jobParameters, jobConfigurationLocation);
});
}
//...Do the same pattern for other JobRepository methods here
}
You can probably do this with aspect oriented programming or with annotations. I prefer explicit code like above. But it's up to you. The idea is the same however you choose to implement it.
I hope this helps others. I also hope the spring batch developers provide a solution to this out of the box or at least some better guidelines on how to avoid it.