I'm using Spring WebClient to communicate with other web services in a Java 11/Spring Boot 2.2.6 web application.
The following is my bean config for WebClient:
#Bean
public WebClient.Builder webClientBuilder() {
String connectionProviderName = "customConnectionProvider";
int maxConnections = 1000;
int acquireTimeout = 45;
ConnectionProvider connectionProvider = ConnectionProvider.builder(connectionProviderName)
.maxConnections(maxConnections)
.pendingAcquireTimeout(Duration.ofSeconds(acquireTimeout))
.build();
HttpClient httpClient = HttpClient.create(connectionProvider);
return WebClient.builder().clientConnector(new ReactorClientHttpConnector(httpClient));
}
private Mono<ClientHttpResponse> getWebClientFactory(HttpMethod httpMethod, URI uri, Function<? super ClientHttpRequest, Mono<Void>> monoFunction) {
TcpClient tcpClient = TcpClient
.create()
.option(ChannelOption.CONNECT_TIMEOUT_MILLIS, 60000)
.doOnConnected(connection -> {
connection.addHandlerLast(new ReadTimeoutHandler(300000, TimeUnit.MILLISECONDS));
connection.addHandlerLast(new WriteTimeoutHandler(300000, TimeUnit.MILLISECONDS));
});
return new ReactorClientHttpConnector(HttpClient.from(tcpClient)).connect(httpMethod, uri, monoFunction);
}
I do a blocking invocation using WebClient like below, because the rest of my application is blocking (servlet-based)
webClient.post().uri("http://localhost:8080/api", iceId).bodyValue(request).accept(MediaType.APPLICATION_JSON).retrieve().bodyToMono(String.class).block();
Problem:
When the number of requests served by the app is low, there doesn't seem to be any problem with this approach. But when it goes over 20+ requests per second, then the webClient method call above starts throwing java.util.concurrent.RejectedExecutionException for some of the invocations, and starts working again after some time.
The stacktrace looks like this:
java.util.concurrent.RejectedExecutionException: event executor terminated at
io.netty.util.concurrent.SingleThreadEventExecutor.reject(SingleThreadEventExecutor.java:926) ~[netty-common-4.1.48.Final.jar!/:4.1.48.Final] at
io.netty.util.concurrent.SingleThreadEventExecutor.offerTask(SingleThreadEventExecutor.java:353) ~[netty-common-4.1.48.Final.jar!/:4.1.48.Final] at
io.netty.util.concurrent.SingleThreadEventExecutor.addTask(SingleThreadEventExecutor.java:346) ~[netty-common-4.1.48.Final.jar!/:4.1.48.Final] at
io.netty.util.concurrent.SingleThreadEventExecutor.execute(SingleThreadEventExecutor.java:828) ~[netty-common-4.1.48.Final.jar!/:4.1.48.Final] at
io.netty.util.concurrent.SingleThreadEventExecutor.execute(SingleThreadEventExecutor.java:818) ~[netty-common-4.1.48.Final.jar!/:4.1.48.Final] at
io.netty.channel.AbstractChannel$AbstractUnsafe.register(AbstractChannel.java:471) ~[netty-transport-4.1.48.Final.jar!/:4.1.48.Final] at
io.netty.channel.SingleThreadEventLoop.register(SingleThreadEventLoop.java:87) ~[netty-transport-4.1.48.Final.jar!/:4.1.48.Final] at
io.netty.channel.SingleThreadEventLoop.register(SingleThreadEventLoop.java:81) ~[netty-transport-4.1.48.Final.jar!/:4.1.48.Final] at
reactor.netty.resources.ColocatedEventLoopGroup.register(ColocatedEventLoopGroup.java:71) ~[reactor-netty-0.9.6.RELEASE.jar!/:0.9.6.RELEASE] at
io.netty.bootstrap.AbstractBootstrap.initAndRegister(AbstractBootstrap.java:323) ~[netty-transport-4.1.48.Final.jar!/:4.1.48.Final] at
io.netty.bootstrap.Bootstrap.doResolveAndConnect(Bootstrap.java:155) ~[netty-transport-4.1.48.Final.jar!/:4.1.48.Final] at
io.netty.bootstrap.Bootstrap.connect(Bootstrap.java:116) ~[netty-transport-4.1.48.Final.jar!/:4.1.48.Final] at
reactor.netty.resources.PooledConnectionProvider$PooledConnectionAllocator.lambda$connectChannel$0(PooledConnectionProvider.java:224) ~[reactor-netty-0.9.6.RELEASE.jar!/:0.9.6.RELEASE] at
reactor.core.publisher.MonoCreate.subscribe(MonoCreate.java:57) ~[reactor-core-3.3.4.RELEASE.jar!/:3.3.4.RELEASE] at
reactor.core.publisher.Mono.subscribe(Mono.java:4210) ~[reactor-core-3.3.4.RELEASE.jar!/:3.3.4.RELEASE] at
reactor.core.publisher.Mono.subscribeWith(Mono.java:4316) ~[reactor-core-3.3.4.RELEASE.jar!/:3.3.4.RELEASE] at
reactor.core.publisher.Mono.subscribe(Mono.java:4182) ~[reactor-core-3.3.4.RELEASE.jar!/:3.3.4.RELEASE] at
reactor.core.publisher.Mono.subscribe(Mono.java:4118) ~[reactor-core-3.3.4.RELEASE.jar!/:3.3.4.RELEASE] at
reactor.netty.internal.shaded.reactor.pool.SimplePool.drainLoop(SimplePool.java:201) ~[reactor-netty-0.9.6.RELEASE.jar!/:0.9.6.RELEASE] at
reactor.netty.internal.shaded.reactor.pool.SimplePool.drain(SimplePool.java:172) ~[reactor-netty-0.9.6.RELEASE.jar!/:0.9.6.RELEASE] at
reactor.netty.internal.shaded.reactor.pool.SimplePool.doAcquire(SimplePool.java:132) ~[reactor-netty-0.9.6.RELEASE.jar!/:0.9.6.RELEASE] at
reactor.netty.internal.shaded.reactor.pool.AbstractPool$Borrower.request(AbstractPool.java:351) ~[reactor-netty-0.9.6.RELEASE.jar!/:0.9.6.RELEASE] at
reactor.netty.resources.PooledConnectionProvider$DisposableAcquire.onSubscribe(PooledConnectionProvider.java:498) ~[reactor-netty-0.9.6.RELEASE.jar!/:0.9.6.RELEASE] at
reactor.netty.internal.shaded.reactor.pool.SimplePool$QueueBorrowerMono.subscribe(SimplePool.java:323) ~[reactor-netty-0.9.6.RELEASE.jar!/:0.9.6.RELEASE] at
reactor.netty.resources.PooledConnectionProvider.disposableAcquire(PooledConnectionProvider.java:199) ~[reactor-netty-0.9.6.RELEASE.jar!/:0.9.6.RELEASE] at
reactor.netty.resources.PooledConnectionProvider.lambda$acquire$3(PooledConnectionProvider.java:160) ~[reactor-netty-0.9.6.RELEASE.jar!/:0.9.6.RELEASE] at
reactor.core.publisher.MonoCreate.subscribe(MonoCreate.java:57) ~[reactor-core-3.3.4.RELEASE.jar!/:3.3.4.RELEASE] at
reactor.netty.http.client.HttpClientConnect$MonoHttpConnect.lambda$subscribe$0(HttpClientConnect.java:320) ~[reactor-netty-0.9.6.RELEASE.jar!/:0.9.6.RELEASE] at
reactor.core.publisher.MonoCreate.subscribe(MonoCreate.java:57) ~[reactor-core-3.3.4.RELEASE.jar!/:3.3.4.RELEASE] at
reactor.core.publisher.FluxRetryPredicate$RetryPredicateSubscriber.resubscribe(FluxRetryPredicate.java:124) ~[reactor-core-3.3.4.RELEASE.jar!/:3.3.4.RELEASE] at
reactor.core.publisher.MonoRetryPredicate.subscribeOrReturn(MonoRetryPredicate.java:51) ~[reactor-core-3.3.4.RELEASE.jar!/:3.3.4.RELEASE] at
reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:48) ~[reactor-core-3.3.4.RELEASE.jar!/:3.3.4.RELEASE] at
reactor.netty.http.client.HttpClientConnect$MonoHttpConnect.subscribe(HttpClientConnect.java:323) ~[reactor-netty-0.9.6.RELEASE.jar!/:0.9.6.RELEASE] at
reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:55) ~[reactor-core-3.3.4.RELEASE.jar!/:3.3.4.RELEASE] at
reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:52) ~[reactor-core-3.3.4.RELEASE.jar!/:3.3.4.RELEASE] at
reactor.core.publisher.Mono.subscribe(Mono.java:4210) ~[reactor-core-3.3.4.RELEASE.jar!/:3.3.4.RELEASE] at
reactor.core.publisher.Mono.block(Mono.java:1665) ~[reactor-core-3.3.4.RELEASE.jar!/:3.3.4.RELEASE] at
... calling code ...
org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) ~[spring-core-5.2.5.RELEASE.jar!/:5.2.5.RELEASE] at
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771) ~[spring-aop-5.2.5.RELEASE.jar!/:5.2.5.RELEASE] at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) ~[spring-aop-5.2.5.RELEASE.jar!/:5.2.5.RELEASE] at
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) ~[spring-aop-5.2.5.RELEASE.jar!/:5.2.5.RELEASE] at
org.springframework.retry.interceptor.RetryOperationsInterceptor$1.doWithRetry(RetryOperationsInterceptor.java:91) ~[spring-retry-1.2.5.RELEASE.jar!/:na] at
org.springframework.retry.support.RetryTemplate.doExecute(RetryTemplate.java:287) ~[spring-retry-1.2.5.RELEASE.jar!/:na] at
org.springframework.retry.support.RetryTemplate.execute(RetryTemplate.java:164) ~[spring-retry-1.2.5.RELEASE.jar!/:na] at
org.springframework.retry.interceptor.RetryOperationsInterceptor.invoke(RetryOperationsInterceptor.java:118) ~[spring-retry-1.2.5.RELEASE.jar!/:na] at
org.springframework.retry.annotation.AnnotationAwareRetryOperationsInterceptor.invoke(AnnotationAwareRetryOperationsInterceptor.java:153) ~[spring-retry-1.2.5.RELEASE.jar!/:na] at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.2.5.RELEASE.jar!/:5.2.5.RELEASE] at
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) ~[spring-aop-5.2.5.RELEASE.jar!/:5.2.5.RELEASE] at
org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691) ~[spring-aop-5.2.5.RELEASE.jar!/:5.2.5.RELEASE] at
...calling code...
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[na:na] at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[na:na] at
java.base/java.lang.Thread.run(Thread.java:834) ~[na:na]
This seems to be a very common usage of WebClient, has anyone else faced a similar issue? If so, how was it resolved?
I think that you meant to set read/write timeouts at 300 seconds with the getWebClientFactory method. But note that the getWebClientFactory method is neither called anywhere in your provided code, nor exposed as a Bean. However, even if you were to expose that method, you're likely to run into http/tcp connection pool problems as described by this user in this comment.
Try the following configuration instead:
#Bean
public WebClient.Builder webClientBuilder() {
String connectionProviderName = "customConnectionProvider";
int maxConnections = 1000;
int acquireTimeout = 45;
ConnectionProvider connectionProvider = ConnectionProvider.builder(connectionProviderName)
.maxConnections(maxConnections)
.pendingAcquireTimeout(Duration.ofSeconds(acquireTimeout))
.build();
HttpClient httpClient = HttpClient.create(connectionProvider)
.tcpConfiguration(tcpClient -> tcpClient.option(ChannelOption.CONNECT_TIMEOUT_MILLIS, 60000)
.doOnConnected(connection -> {
connection.addHandlerLast(new ReadTimeoutHandler(300));
connection.addHandlerLast(new WriteTimeoutHandler(300));
}));
return WebClient.builder().clientConnector(new ReactorClientHttpConnector(httpClient));
}
Related
I have a Spring Boot app that sends and receives messages on a GCP PubSub queue. Basically I try to store in the DB some of the information that I get in the message. However when I try to commit this information or saveAndFlush() I get the error RollbackException: Error while committing the transaction. If I persists any other information that it's not triggered when the app receives an inbound message it works correctly. If I try saveAndFlush instead of save I get a NullPointerException.
What am I missing?
Thanks.
Error stack
Caused by: org.springframework.transaction.TransactionSystemException: Could not commit JPA transaction; nested exception is javax.persistence.RollbackException: Error while committing the transaction
at org.springframework.orm.jpa.JpaTransactionManager.doCommit(JpaTransactionManager.java:543)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:743)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:711)
at org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:633)
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:386)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:118)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:139)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.data.jpa.repository.support.CrudMethodMetadataPostProcessor$CrudMethodMetadataPopulatingMethodInterceptor.invoke(CrudMethodMetadataPostProcessor.java:178)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212)
at com.sun.proxy.$Proxy153.save(Unknown Source)
at com.sap.service.deployments.DeploymentService.lambda$saveLogs$16(DeploymentService.java:235)
at java.util.Optional.map(Optional.java:215)
at com.sap.service.deployments.DeploymentService.saveLogs(DeploymentService.java:233)
at com.sap.service.deployments.DeploymentService.processDeploymentExecutionResponse(DeploymentService.java:404)
at com.sap.service.deployments.DeploymentService.messageReceiver(DeploymentService.java:245)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.messaging.handler.invocation.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:171)
at org.springframework.messaging.handler.invocation.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:120)
at org.springframework.integration.handler.support.MessagingMethodInvokerHelper$HandlerMethod.invoke(MessagingMethodInvokerHelper.java:1105)
at org.springframework.integration.handler.support.MessagingMethodInvokerHelper.invokeHandlerMethod(MessagingMethodInvokerHelper.java:583)
at org.springframework.integration.handler.support.MessagingMethodInvokerHelper.processInternal(MessagingMethodInvokerHelper.java:478)
at org.springframework.integration.handler.support.MessagingMethodInvokerHelper.process(MessagingMethodInvokerHelper.java:356)
at org.springframework.integration.handler.MethodInvokingMessageProcessor.processMessage(MethodInvokingMessageProcessor.java:108)
... 24 more
Caused by: javax.persistence.RollbackException: Error while committing the transaction
at org.hibernate.internal.ExceptionConverterImpl.convertCommitException(ExceptionConverterImpl.java:81)
at org.hibernate.engine.transaction.internal.TransactionImpl.commit(TransactionImpl.java:104)
at org.springframework.orm.jpa.JpaTransactionManager.doCommit(JpaTransactionManager.java:534)
... 54 more
Caused by: java.lang.NullPointerException
at com.sap.config.AuditorAwareImpl.getCurrentAuditor(AuditorAwareImpl.java:13)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:205)
at com.sun.proxy.$Proxy164.getCurrentAuditor(Unknown Source)
at org.springframework.data.auditing.AuditingHandler.lambda$touchAuditor$6(AuditingHandler.java:193)
at java.util.Optional.map(Optional.java:215)
at org.springframework.data.auditing.AuditingHandler.touchAuditor(AuditingHandler.java:191)
at org.springframework.data.auditing.AuditingHandler.lambda$touch$0(AuditingHandler.java:165)
at java.util.Optional.map(Optional.java:215)
at org.springframework.data.auditing.AuditingHandler.touch(AuditingHandler.java:163)
at org.springframework.data.auditing.AuditingHandler.markModified(AuditingHandler.java:143)
at org.springframework.data.jpa.domain.support.AuditingEntityListener.touchForUpdate(AuditingEntityListener.java:112)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.hibernate.jpa.event.internal.ListenerCallback.performCallback(ListenerCallback.java:35)
at org.hibernate.jpa.event.internal.CallbackRegistryImpl.callback(CallbackRegistryImpl.java:95)
at org.hibernate.jpa.event.internal.CallbackRegistryImpl.preUpdate(CallbackRegistryImpl.java:69)
at org.hibernate.event.internal.DefaultFlushEntityEventListener.invokeInterceptor(DefaultFlushEntityEventListener.java:366)
at org.hibernate.event.internal.DefaultFlushEntityEventListener.handleInterception(DefaultFlushEntityEventListener.java:348)
at org.hibernate.event.internal.DefaultFlushEntityEventListener.scheduleUpdate(DefaultFlushEntityEventListener.java:299)
at org.hibernate.event.internal.DefaultFlushEntityEventListener.onFlushEntity(DefaultFlushEntityEventListener.java:170)
at org.hibernate.event.internal.AbstractFlushingEventListener.flushEntities(AbstractFlushingEventListener.java:232)
at org.hibernate.event.internal.AbstractFlushingEventListener.flushEverythingToExecutions(AbstractFlushingEventListener.java:92)
at org.hibernate.event.internal.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:39)
at org.hibernate.event.service.internal.EventListenerGroupImpl.fireEventOnEachListener(EventListenerGroupImpl.java:102)
at org.hibernate.internal.SessionImpl.doFlush(SessionImpl.java:1360)
at org.hibernate.internal.SessionImpl.managedFlush(SessionImpl.java:451)
at org.hibernate.internal.SessionImpl.flushBeforeTransactionCompletion(SessionImpl.java:3210)
at org.hibernate.internal.SessionImpl.beforeTransactionCompletion(SessionImpl.java:2378)
at org.hibernate.engine.jdbc.internal.JdbcCoordinatorImpl.beforeTransactionCompletion(JdbcCoordinatorImpl.java:447)
at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl.beforeCompletionCallback(JdbcResourceLocalTransactionCoordinatorImpl.java:183)
at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl.access$300(JdbcResourceLocalTransactionCoordinatorImpl.java:40)
at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl$TransactionDriverControlImpl.commit(JdbcResourceLocalTransactionCoordinatorImpl.java:281)
at org.hibernate.engine.transaction.internal.TransactionImpl.commit(TransactionImpl.java:101)
... 55 more
AuditorAwareImpl
import org.springframework.data.domain.AuditorAware;
import org.springframework.security.core.context.SecurityContextHolder;
import java.util.Optional;
class AuditorAwareImpl implements AuditorAware<String> {
#Override
public Optional<String> getCurrentAuditor() {
SamlUserDetails userDetails = (SamlUserDetails) SecurityContextHolder.getContext().getAuthentication().getPrincipal(); //NullPointerExc on this line
return Optional.of(userDetails.getFullname() + " (" + userDetails.getAttribute("uid") + ")");
}
}
This is the configuration of the Inbound PubSub
#Component
public class PubSubInbound {
#Autowired
private DeploymentService deploymentService;
#Bean
public MessageChannel inputMessageChannel() {
return new PublishSubscribeChannel();
}
// Create an inbound channel adapter to listen to the subscription `sub-one` and send
// messages to the input message channel.
#Bean
public PubSubInboundChannelAdapter inboundChannelAdapter(
#Qualifier("inputMessageChannel") MessageChannel messageChannel,
PubSubTemplate pubSubTemplate) {
PubSubInboundChannelAdapter adapter =
new PubSubInboundChannelAdapter(pubSubTemplate, "cerberusSubscription");
adapter.setOutputChannel(messageChannel);
adapter.setAckMode(AckMode.MANUAL);
adapter.setPayloadType(String.class);
return adapter;
}
#Bean
#ServiceActivator(inputChannel = "pubsubInputChannel")
public MessageHandler messageReceiver() {
return message -> {
System.out.println("Message arrived! Payload: " + new String((byte[]) message.getPayload()));
BasicAcknowledgeablePubsubMessage originalMessage =
message.getHeaders().get(GcpPubSubHeaders.ORIGINAL_MESSAGE, BasicAcknowledgeablePubsubMessage.class);
originalMessage.ack();
deploymentService.persistLogs(
Long.parseLong(message.getHeaders().get("deploymentId", String.class)),
new String((byte[]) message.getPayload()));
};
}
}
DeploymentService
#Service
public class DeploymentService {
#Autowired
private DeploymentRepository deploymentRepository;
....
public Optional<Deployment> persistLogs(Long deploymentId, String logs){
Optional<Deployment> deploymentOpt = deploymentRepository.findById(deploymentId);
return deploymentOpt.map(deployment -> {
deployment.setLogs(logs);
return deploymentRepository.save(deployment); //ERROR HERE
});
}
....
DeploymentRepository
#Repository
public interface DeploymentRepository extends JpaRepository<Deployment, Long> { }
Solved.
SecurityContextHolder.getContext().getAuthentication()
was null when the message is coming from the GCP PubSub. Reworking the getCurrentAuditor did the trick.
#Override
public Optional<String> getCurrentAuditor() {
if(SecurityContextHolder.getContext().getAuthentication()!=null){
SamlUserDetails userDetails = (SamlUserDetails) SecurityContextHolder.getContext().getAuthentication().getPrincipal();
return Optional.of(userDetails.getFullname() + " (" + userDetails.getAttribute("uid") + ")");
}else return Optional.empty();
}
I find myself in an environment where the rabbitmq's are already provided as given infrastructure.
A Service A is writing into the rabbit queues and Service B is reading from the queues. The reading part as a consumer via Spring Cloud Stream Binder is working like a charme after intensive reading the documentation and setting it up properly.
However i am not able to set up a producer which is going to write into the rabbit queues.
Setup
RabbitMq (already up and running)
1 Exchange: myExchange
3 Queues: myQueueA, myQueueB, myQueueC (under myExchange linked)
Producer-Service (above mentioned Service A)
#Slf4j
#Component
#RequiredArgsConstructor
#EnableBinding(RabbitChannelSource.class)
public class RabbitSender
{
private final RabbitChannelSource rabbitChannelSource;
public void sendMessage()
{
Message<String> msg = MessageBuilder.withPayload("TEEEEST").build();
rabbitChannelSource.myOutput().send(msg);
}
public interface RabbitChannelSource
{
String MY_OUTPUT_BINDING = "my-output";
#Output(MY_OUTPUT_BINDING)
MessageChannel myOutput();
}
I am trying to make it work first for one queue, but ideally i would set the properties right for all three queues.
spring.rabbitmq.host=localhost
spring.rabbitmq.port=5672
spring.rabbitmq.username=user
spring.rabbitmq.password=pass
## use existing rabbitmq via bindings
spring.cloud.stream.bindings.my-output.destination=my-output
#spring.cloud.stream.bindings.my-output.group=myQueueA
spring.cloud.stream.bindings.my-output.producer.required-groups=myQueueA
spring.cloud.stream.rabbit.bindings.my-output.producer.bind-queue=false
spring.cloud.stream.rabbit.bindings.my-output.producer.declare-exchange=false
spring.cloud.stream.rabbit.bindings.my-output.producer.queueNameGroupOnly=true
bind-queue=false and declare-exchange=false is necessary since i have rabbit infrastructure.
But however i am getting always the same exception, i could not figure out why. I mean i know why, because there is no appropriate channel istantiated to send messages to. So i am suspecting it has something to do with the application.properties.
org.springframework.messaging.MessageDeliveryException: Dispatcher has no subscribers for channel 'application.my-output'.; nested exception is org.springframework.integration.MessageDispatchingException: Dispatcher has no subscribers, failedMessage=GenericMessage [payload=byte[7], headers={contentType=application/json, id=98698b4c-61fa-596d-736e-f630d3ba4626, timestamp=1605105272723}]
at org.springframework.integration.channel.AbstractSubscribableChannel.doSend(AbstractSubscribableChannel.java:77)
at org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:453)
at org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:403)
at de.techem.emsreceiver.rabbitmq.RabbitSender.sendMessage(RabbitSender.java:25)
at de.techem.emsreceiver.event.TriggeredEmsImport.execute(TriggeredImport.java:95)
at de.techem.emsreceiver.event.TriggeredEmsImport$$FastClassBySpringCGLIB$$d84f31a4.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:769)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:747)
at org.springframework.aop.aspectj.AspectJAfterThrowingAdvice.invoke(AspectJAfterThrowingAdvice.java:62)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:747)
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:747)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:689)
at de.techem.emsreceiver.event.TriggeredEmsImport$$EnhancerBySpringCGLIB$$a0878aed.execute(<generated>)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.springframework.context.event.ApplicationListenerMethodAdapter.doInvoke(ApplicationListenerMethodAdapter.java:305)
at org.springframework.context.event.ApplicationListenerMethodAdapter.processEvent(ApplicationListenerMethodAdapter.java:190)
at org.springframework.context.event.ApplicationListenerMethodAdapter.onApplicationEvent(ApplicationListenerMethodAdapter.java:153)
at org.springframework.context.event.SimpleApplicationEventMulticaster.doInvokeListener(SimpleApplicationEventMulticaster.java:172)
at org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:165)
at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:139)
at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:403)
at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:409)
at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:360)
at org.springframework.boot.context.event.EventPublishingRunListener.running(EventPublishingRunListener.java:103)
at org.springframework.boot.SpringApplicationRunListeners.running(SpringApplicationRunListeners.java:77)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:330)
at org.springframework.boot.builder.SpringApplicationBuilder.run(SpringApplicationBuilder.java:140)
at org.springframework.cloud.stream.binder.DefaultBinderFactory.getBinderInstance(DefaultBinderFactory.java:320)
at org.springframework.cloud.stream.binder.DefaultBinderFactory.doGetBinder(DefaultBinderFactory.java:209)
at org.springframework.cloud.stream.binder.DefaultBinderFactory.getBinder(DefaultBinderFactory.java:140)
at org.springframework.cloud.stream.binding.BindingService.getBinder(BindingService.java:379)
at org.springframework.cloud.stream.binding.BindingService.bindProducer(BindingService.java:268)
at org.springframework.cloud.stream.binding.BindingService.bindProducer(BindingService.java:291)
at org.springframework.cloud.stream.binding.AbstractBindableProxyFactory.createAndBindOutputs(AbstractBindableProxyFactory.java:136)
at org.springframework.cloud.stream.binding.OutputBindingLifecycle.doStartWithBindable(OutputBindingLifecycle.java:58)
at java.base/java.util.LinkedHashMap$LinkedValues.forEach(LinkedHashMap.java:608)
at org.springframework.cloud.stream.binding.AbstractBindingLifecycle.start(AbstractBindingLifecycle.java:57)
at org.springframework.cloud.stream.binding.OutputBindingLifecycle.start(OutputBindingLifecycle.java:34)
at org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:182)
at org.springframework.context.support.DefaultLifecycleProcessor.access$200(DefaultLifecycleProcessor.java:53)
at org.springframework.context.support.DefaultLifecycleProcessor$LifecycleGroup.start(DefaultLifecycleProcessor.java:360)
at org.springframework.context.support.DefaultLifecycleProcessor.startBeans(DefaultLifecycleProcessor.java:158)
at org.springframework.context.support.DefaultLifecycleProcessor.onRefresh(DefaultLifecycleProcessor.java:122)
at org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:894)
at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.finishRefresh(ServletWebServerApplicationContext.java:162)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:553)
at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:141)
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:747)
at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:397)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:315)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1226)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1215)
at de.techem.emsreceiver.EmsReceiverApplication.main(EmsReceiverApplication.java:21)
I want to be able to write into myQueueA, myQueueB or myQueueC based on a routingKey. So if i set routingKey inside my application to myQA the message should be send to myQueueA, for myQB then to myQueueB, and for myQC then to myQueueC.
So three different routing keys lead to the corresponding rabbit queue.
I am happy to any input since i have tried many things which did not lead to me any success. Thank you!
Perhaps you are trying to use the binding before it is bound? This works fine for me:
spring.cloud.stream.bindings.output.destination=my-output
spring.cloud.stream.rabbit.bindings.output.producer.declare-exchange=false
spring.cloud.stream.rabbit.bindings.output.producer.routing-key-expression=headers['routeTo']
#SpringBootApplication
#EnableBinding(Source.class)
public class So64788954Application {
public static void main(String[] args) {
SpringApplication.run(So64788954Application.class, args);
}
#Autowired
Source source;
#Bean
ApplicationRunner runner() {
return args -> {
source.output().send(MessageBuilder.withPayload("foo")
.setHeader("routeTo", "myQA")
.build());
source.output().send(MessageBuilder.withPayload("bar")
.setHeader("routeTo", "myQB")
.build());
source.output().send(MessageBuilder.withPayload("baz")
.setHeader("routeTo", "myQC")
.build());
};
}
#RabbitListener(queues = { "myQueueA", "myQueueB", "myQueueC" } )
void listen(String in, #Header(AmqpHeaders.CONSUMER_QUEUE) String queue) {
System.out.println(in + ", from: " + queue);
}
}
foo, from: myQueueA
bar, from: myQueueB
baz, from: myQueueC
You don't need required groups on the producer side when you bring your own queues/exchanges.
The #RabbitListener` is just there to consume the messages sent to the 3 queues.
By the way, the annotation model is deprecated; the functional programming model is now preferred, where we use a StreamBridge for output (and Consumer<?> or Function<?, ?> to consume and process respectively).
Here is the equivalent of the above:
spring.cloud.stream.rabbit.bindings.my-output.producer.declare-exchange=false
spring.cloud.stream.rabbit.bindings.my-output.producer.routing-key-expression=headers['routeTo']
#SpringBootApplication
public class So64788954Application {
public static void main(String[] args) {
SpringApplication.run(So64788954Application.class, args);
}
#Autowired
StreamBridge bridge;
#Bean
ApplicationRunner runner() {
return args -> {
bridge.send("my-output", MessageBuilder.withPayload("foo")
.setHeader("routeTo", "myQA")
.build());
bridge.send("my-output", MessageBuilder.withPayload("bar")
.setHeader("routeTo", "myQB")
.build());
bridge.send("my-output", MessageBuilder.withPayload("baz")
.setHeader("routeTo", "myQC")
.build());
};
}
#RabbitListener(queues = { "myQueueA", "myQueueB", "myQueueC" } )
void listen(String in, #Header(AmqpHeaders.CONSUMER_QUEUE) String queue) {
System.out.println(in + ", from: " + queue);
}
}
After Gary's suggestion i get the same error as in my first post.
application.properties
# --- spring cloud ---
spring.rabbitmq.host=localhost
spring.rabbitmq.port=5672
spring.rabbitmq.username=username
spring.rabbitmq.password=password
## use existing rabbitmq via bindings
spring.cloud.stream.bindings.output.destination=my-output
spring.cloud.stream.rabbit.bindings.output.producer.declare-exchange=false
spring.cloud.stream.rabbit.bindings.output.producer.routing-key-expression=headers['routeTo']
RabbitTrigger.java
#Slf4j
#Component
#RequiredArgsConstructor
public class RabbitTrigger
{
private final RabbitSender rabbitSender;
#EventListener(ApplicationReadyEvent.class)
public void execute()
{
rabbitSender.testSending();
}
}
RabbitSender.java
#Slf4j
#Configuration
#RequiredArgsConstructor
#EnableBinding(Source.class)
public class RabbitSender
{
private final Source source;
public void testSending()
{
Message<String> testMessage = MessageBuilder.withPayload("TEEEEST")
.setHeader("routeTo", "CRE")
.build();
source.output().send(testMessage);
}
But i get the same exception as i was getting in my initial post. I assume i did not consider a crucial part to this?
2020-11-25 16:27:17,116 [main] ERROR o.s.boot.SpringApplication - Application run failed
org.springframework.messaging.MessageDeliveryException: Dispatcher has no subscribers for channel 'application.output'.; nested exception is org.springframework.integration.MessageDispatchingException: Dispatcher has no subscribers, failedMessage=GenericMessage [payload=byte[7], headers={routeTo=CRE, id=15479810-375f-38e8-a692-e84c6ede01d7, contentType=application/json, timestamp=1606318037108}]
at org.springframework.integration.channel.AbstractSubscribableChannel.doSend(AbstractSubscribableChannel.java:77)
at org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:453)
at org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:403)
at de.techem.emsreceiver.rabbitmq.RabbitSender.testSending(RabbitSender.java:34)
at de.techem.emsreceiver.event.RabbitTrigger.execute(RabbitTrigger.java:96)
at de.techem.emsreceiver.event.RabbitTrigger$$FastClassBySpringCGLIB$$d84f31a4.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:769)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:747)
at org.springframework.aop.aspectj.AspectJAfterThrowingAdvice.invoke(AspectJAfterThrowingAdvice.java:62)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:747)
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:747)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:689)
at de.techem.emsreceiver.event.RabbitTrigger$$EnhancerBySpringCGLIB$$d00eff49.execute(<generated>)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.springframework.context.event.ApplicationListenerMethodAdapter.doInvoke(ApplicationListenerMethodAdapter.java:305)
at org.springframework.context.event.ApplicationListenerMethodAdapter.processEvent(ApplicationListenerMethodAdapter.java:190)
at org.springframework.context.event.ApplicationListenerMethodAdapter.onApplicationEvent(ApplicationListenerMethodAdapter.java:153)
at org.springframework.context.event.SimpleApplicationEventMulticaster.doInvokeListener(SimpleApplicationEventMulticaster.java:172)
at org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:165)
at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:139)
at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:403)
at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:409)
at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:360)
at org.springframework.boot.context.event.EventPublishingRunListener.running(EventPublishingRunListener.java:103)
at org.springframework.boot.SpringApplicationRunListeners.running(SpringApplicationRunListeners.java:77)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:330)
at org.springframework.boot.builder.SpringApplicationBuilder.run(SpringApplicationBuilder.java:140)
at org.springframework.cloud.stream.binder.DefaultBinderFactory.getBinderInstance(DefaultBinderFactory.java:320)
at org.springframework.cloud.stream.binder.DefaultBinderFactory.doGetBinder(DefaultBinderFactory.java:209)
at org.springframework.cloud.stream.binder.DefaultBinderFactory.getBinder(DefaultBinderFactory.java:140)
at org.springframework.cloud.stream.binding.BindingService.getBinder(BindingService.java:379)
at org.springframework.cloud.stream.binding.BindingService.bindProducer(BindingService.java:268)
at org.springframework.cloud.stream.binding.BindingService.bindProducer(BindingService.java:291)
at org.springframework.cloud.stream.binding.AbstractBindableProxyFactory.createAndBindOutputs(AbstractBindableProxyFactory.java:136)
at org.springframework.cloud.stream.binding.OutputBindingLifecycle.doStartWithBindable(OutputBindingLifecycle.java:58)
at java.base/java.util.LinkedHashMap$LinkedValues.forEach(LinkedHashMap.java:608)
at org.springframework.cloud.stream.binding.AbstractBindingLifecycle.start(AbstractBindingLifecycle.java:57)
at org.springframework.cloud.stream.binding.OutputBindingLifecycle.start(OutputBindingLifecycle.java:34)
at org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:182)
at org.springframework.context.support.DefaultLifecycleProcessor.access$200(DefaultLifecycleProcessor.java:53)
at org.springframework.context.support.DefaultLifecycleProcessor$LifecycleGroup.start(DefaultLifecycleProcessor.java:360)
at org.springframework.context.support.DefaultLifecycleProcessor.startBeans(DefaultLifecycleProcessor.java:158)
at org.springframework.context.support.DefaultLifecycleProcessor.onRefresh(DefaultLifecycleProcessor.java:122)
at org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:894)
at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.finishRefresh(ServletWebServerApplicationContext.java:162)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:553)
at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:141)
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:747)
at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:397)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:315)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1226)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1215)
at de.techem.emsreceiver.EmsReceiverApplication.main(EmsReceiverApplication.java:21)
Caused by: org.springframework.integration.MessageDispatchingException: Dispatcher has no subscribers
at org.springframework.integration.dispatcher.UnicastingDispatcher.doDispatch(UnicastingDispatcher.java:139)
at org.springframework.integration.dispatcher.UnicastingDispatcher.dispatch(UnicastingDispatcher.java:106)
at org.springframework.integration.channel.AbstractSubscribableChannel.doSend(AbstractSubscribableChannel.java:73)
... 60 common frames omitted
Setting up the producer is way more complicated for me than it was for the consumer part.
I'm using Spring Batch which loads data from Oracle and put it into the MongoDB. I'm looking to use the Spring Cloud Data Flow, but SCDF doesn't have support for the MongoDB.
Is there any way if we can maintain the SCDF Metadata into the Postgres (as its has good support) or H2, in this example I used H2, and run the actual batch jobs which load data from Oracle to MongoDB.
When I tried to run this combination, I get the below error, certainly some config changes needs to be tweek. Any quick workaround ?
Error:
org.springframework.batch.item.ItemStreamException: Failed to initialize the reader
at org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader.open(AbstractItemCountingItemStreamItemReader.java:152) ~[spring-batch-infrastructure-4.1.1.RELEASE.jar!/:4.1.1.RELEASE]
at org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader$$FastClassBySpringCGLIB$$ebb633d0.invoke(<generated>) ~[spring-batch-infrastructure-4.1.1.RELEASE.jar!/:4.1.1.RELEASE]
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) ~[spring-core-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:749) ~[spring-aop-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) [spring-aop-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:136) ~[spring-aop-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:124) ~[spring-aop-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) [spring-aop-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:688) ~[spring-aop-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
at org.springframework.batch.item.database.JdbcCursorItemReader$$EnhancerBySpringCGLIB$$3c62d122.open(<generated>) ~[spring-batch-infrastructure-4.1.1.RELEASE.jar!/:4.1.1.RELEASE]
at org.springframework.batch.item.support.CompositeItemStream.open(CompositeItemStream.java:103) ~[spring-batch-infrastructure-4.1.1.RELEASE.jar!/:4.1.1.RELEASE]
at org.springframework.batch.core.step.tasklet.TaskletStep.open(TaskletStep.java:311) ~[spring-batch-core-4.1.1.RELEASE.jar!/:4.1.1.RELEASE]
at org.springframework.batch.core.step.AbstractStep.execute(AbstractStep.java:200) ~[spring-batch-core-4.1.1.RELEASE.jar!/:4.1.1.RELEASE]
at org.springframework.batch.core.job.SimpleStepHandler.handleStep(SimpleStepHandler.java:148) [spring-batch-core-4.1.1.RELEASE.jar!/:4.1.1.RELEASE]
at org.springframework.batch.core.job.AbstractJob.handleStep(AbstractJob.java:399) [spring-batch-core-4.1.1.RELEASE.jar!/:4.1.1.RELEASE]
at org.springframework.batch.core.job.SimpleJob.doExecute(SimpleJob.java:135) [spring-batch-core-4.1.1.RELEASE.jar!/:4.1.1.RELEASE]
at org.springframework.batch.core.job.AbstractJob.execute(AbstractJob.java:313) [spring-batch-core-4.1.1.RELEASE.jar!/:4.1.1.RELEASE]
at org.springframework.batch.core.launch.support.SimpleJobLauncher$1.run(SimpleJobLauncher.java:144) [spring-batch-core-4.1.1.RELEASE.jar!/:4.1.1.RELEASE]
at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:50) [spring-core-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
at org.springframework.batch.core.launch.support.SimpleJobLauncher.run(SimpleJobLauncher.java:137) [spring-batch-core-4.1.1.RELEASE.jar!/:4.1.1.RELEASE]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_171]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_171]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_171]
at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_171]
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343) [spring-aop-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198) [spring-aop-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) [spring-aop-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
at org.springframework.batch.core.configuration.annotation.SimpleBatchConfiguration$PassthruAdvice.invoke(SimpleBatchConfiguration.java:127) [spring-batch-core-4.1.1.RELEASE.jar!/:4.1.1.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) [spring-aop-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212) [spring-aop-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
at com.sun.proxy.$Proxy66.run(Unknown Source) [na:na]
at com.mastercard.customer.data.management.refdata.ReferenceMongoBatchApplication.run(ReferenceMongoBatchApplication.java:63) [classes!/:1.0]
at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:813) [spring-boot-2.1.3.RELEASE.jar!/:2.1.3.RELEASE]
at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:797) [spring-boot-2.1.3.RELEASE.jar!/:2.1.3.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:324) [spring-boot-2.1.3.RELEASE.jar!/:2.1.3.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1260) [spring-boot-2.1.3.RELEASE.jar!/:2.1.3.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1248) [spring-boot-2.1.3.RELEASE.jar!/:2.1.3.RELEASE]
at com.mastercard.customer.data.management.refdata.ReferenceMongoBatchApplication.main(ReferenceMongoBatchApplication.java:51) [classes!/:1.0]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_171]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_171]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_171]
at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_171]
at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48) [reference-mongo-batch-1.0.jar:1.0]
at org.springframework.boot.loader.Launcher.launch(Launcher.java:87) [reference-mongo-batch-1.0.jar:1.0]
at org.springframework.boot.loader.Launcher.launch(Launcher.java:50) [reference-mongo-batch-1.0.jar:1.0]
at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:51) [reference-mongo-batch-1.0.jar:1.0]
application.properties
# ORACLE DATASOURCE
spring.datasource.url=jdbc:oracle:thin:#//XXXXXX:1527/XXXX
spring.datasource.username=user
spring.datasource.password=password
spring.datasource.driver-class-name=oracle.jdbc.OracleDriver
spring.datasource.validation-query= select 1
#spring.datasource.hikari.validationTimeout=30000
spring.data.mongodb.host=localhost
spring.data.mongodb.port=27017
spring.data.mongodb.database=mgdb_drefdata_99
Config
#Configuration
public class EmployeeJob {
#Value( "${spring.chunk.size}")
private String chunkSize;
#Autowired
private JobBuilderFactory jobBuilderFactory;
#Autowired
private StepBuilderFactory stepBuilderFactory;
#Autowired
private JdbcCursorItemReader<Employee> EmployeeReader;
#Autowired
public AsyncItemProcessor<Employee, Employee> asyncItemProcessor;
#Autowired
public AsyncItemWriter<Employee> asyncItemWriter;
#Bean
public EmployeeStepExecuListner EmployeeStepExecuListner() {
return new EmployeeStepExecuListner();
}
#Bean("readEmployeeJob")
#Primary
public Job readEmployeeJob() {
return jobBuilderFactory.get("readEmployeeJob")
.incrementer(new RunIdIncrementer())
.start(EmployeeStepOne())
.build();
}
#SuppressWarnings({ "unchecked", "rawtypes" })
#Bean
public Step EmployeeStepOne() {
return stepBuilderFactory.get("EmployeeStepOne")
.<Employee, Employee>chunk(Integer.parseInt(chunkSize))
.reader(EmployeeReader)
.processor((ItemProcessor) asyncItemProcessor)
.writer(asyncItemWriter)
.build();
}
}
Config
#Configuration
public class EmployeeBatchConfig {
#Autowired
#Qualifier(value="oracleDS")
private DataSource dataSource;
#Bean(destroyMethod = "")
#StepScope
public JdbcCursorItemReader<Employee> EmployeeReader() throws Exception {
JdbcCursorItemReader<Employee> reader = new JdbcCursorItemReader<>();
reader.setDataSource(this.dataSource);
reader.setSql(SELECT_Employee_SQL);
reader.setRowMapper(new EmployeeRowMapper());
reader.afterPropertiesSet();
return reader;
}
#Bean
public ItemProcessor<Employee, Employee> EmployeeProcessor() {
return new EmployeeProcessor();
}
#Bean
public AsyncItemProcessor<Employee, Employee> asyncItemProcessor() throws Exception{
AsyncItemProcessor<Employee, Employee> asyncItemProcessor = new AsyncItemProcessor<>();
asyncItemProcessor.setDelegate(EmployeeProcessor());
asyncItemProcessor.setTaskExecutor(new SimpleAsyncTaskExecutor());
asyncItemProcessor.afterPropertiesSet();
return asyncItemProcessor;
}
#Bean
public EmployeeWriter EmployeeWriter() {
return new EmployeeWriter();
}
#Bean
public AsyncItemWriter<Employee> asyncItemWriter() throws Exception{
AsyncItemWriter<Employee> asyncItemWriter = new AsyncItemWriter<>();
asyncItemWriter.setDelegate(EmployeeWriter());
asyncItemWriter.afterPropertiesSet();
return asyncItemWriter;
}
}
Currently I am writing Spring boot application to move data from Database in source A to another Database in source B. To do so, I am trying to configure Spring with two datasource and tried to retrive data from source A concurrently and insert to source B. I could retrieve data from source A but the problem is that I could not insert those data to source B. It seems like I didn't configure transactional correctly with EntityManager. Please check my below code and help me figure out this.
Here is my datasource configuration class:
#Configuration
#EnableJpaRepositories(
basePackages = ["com.gdce.xgen.repo.psql"],
entityManagerFactoryRef = "psqlEntityManager",
transactionManagerRef = "psqlTransactional")
#EnableTransactionManagement
#ComponentScan("com.gdce.xgen.repo.psql")
class PsqlDataSource {
#Bean("psqlEntityManager")
fun entityManagerFactory(#Qualifier("psqlDataSource1") dataSource: DataSource): LocalContainerEntityManagerFactoryBean {
val em = LocalContainerEntityManagerFactoryBean()
val vendorAdapter: JpaVendorAdapter = HibernateJpaVendorAdapter()
em.dataSource = dataSource
em.jpaVendorAdapter = vendorAdapter
em.setPackagesToScan("com.gdce.xgen.model.psql")
em.setJpaProperties(hibernateProperties())
return em
}
#Bean("psqlDataSource1")
#ConfigurationProperties("psql.datasource.main.configuration")
fun dataSource(#Qualifier("psqlDataSourceProperties") dataSourceProperties: DataSourceProperties): DataSource {
return dataSourceProperties.initializeDataSourceBuilder().type(HikariDataSource::class.java).build()
}
#Bean("psqlDataSourceProperties")
#ConfigurationProperties("psql.datasource.main")
fun dataSourceProperties(): DataSourceProperties {
return DataSourceProperties()
}
#Bean("psqlTransactional")
fun transactionManager(#Qualifier("psqlEntityManager") entityManager: EntityManager, #Qualifier("psqlDataSource1") dataSource: DataSource): PlatformTransactionManager {
val transactionManager = JpaTransactionManager(entityManager.entityManagerFactory)
transactionManager.dataSource = dataSource
return transactionManager
}
private fun hibernateProperties(): Properties {
val hibernateProperties = Properties()
hibernateProperties.setProperty("hibernate.jdbc.lob.non_contextual_creation", "true")
hibernateProperties.setProperty("hibernate.show_sql", "true")
hibernateProperties.setProperty("hibernate.hbm2ddl.auto", "update")
hibernateProperties.setProperty("hibernate.jdbc.batch_size", "40000")
hibernateProperties.setProperty("hibernate.dialect", "org.hibernate.dialect.PostgreSQLDialect")
hibernateProperties.setProperty("hibernate.connection.pool_size", "20")
hibernateProperties.setProperty("hibernate.order_inserts", "true")
hibernateProperties.setProperty("hibernate.order_updates", "true")
// hibernateProperties.setProperty("hibernate.format_sql", "true")
return hibernateProperties
}
}
Here is the class I use to insert data into destination data source
#Service
class InsertRepo {
#Qualifier("psqlEntityManager")
// #Autowired // also tried this but not working and saw many people use #PersistenceContext and tried it too
#PersistenceContext
private lateinit var entityManager: EntityManager
fun migrate(records: ArrayList<Any>) {
records.parallelStream().forEach { record ->
run {
entityManager.persist(record)
}
}
}
}
Here an error output:
Exception in thread "DefaultDispatcher-worker-1" javax.persistence.TransactionRequiredException: No EntityManager with actual transaction available for current thread - cannot reliably process 'persist' call
at org.springframework.orm.jpa.SharedEntityManagerCreator$SharedEntityManagerInvocationHandler.invoke(SharedEntityManagerCreator.java:298)
at com.sun.proxy.$Proxy83.persist(Unknown Source)
at com.gdce.xgen.repo.psql.InsertRepo$migrate$1.accept(InsertRepo.kt:18)
at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1654)
at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
at java.base/java.util.stream.ForEachOps$ForEachTask.compute(ForEachOps.java:290)
at java.base/java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:746)
at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
at java.base/java.util.concurrent.ForkJoinTask.doInvoke(ForkJoinTask.java:408)
at java.base/java.util.concurrent.ForkJoinTask.invoke(ForkJoinTask.java:736)
at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateParallel(ForEachOps.java:159)
at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateParallel(ForEachOps.java:173)
at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:233)
at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:497)
at java.base/java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:661)
at com.gdce.xgen.repo.psql.InsertRepo.migrate(InsertRepo.kt:16)
at com.gdce.xgen.controller.TestController$find$$inlined$map$lambda$1.invokeSuspend(TestController.kt:46)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
at kotlinx.coroutines.DispatchedTask.run(Dispatched.kt:241)
at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:594)
at kotlinx.coroutines.scheduling.CoroutineScheduler.access$runSafely(CoroutineScheduler.kt:60)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:740)
If i modify my InsertRepo to:
#Service
class InsertRepo {
#Qualifier("psqlEntityManager")
#PersistenceContext
private lateinit var entityManager: EntityManager
#javax.transaction.Transactional
fun migrate(records: ArrayList<Any>) {
records.parallelStream().forEach { record ->
run {
entityManager.persist(record)
}
}
}
}
Then I got this error log:
Exception in thread "DefaultDispatcher-worker-1" javax.persistence.TransactionRequiredException
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
at java.base/java.util.concurrent.ForkJoinTask.getThrowableException(ForkJoinTask.java:603)
at java.base/java.util.concurrent.ForkJoinTask.reportException(ForkJoinTask.java:678)
at java.base/java.util.concurrent.ForkJoinTask.invoke(ForkJoinTask.java:737)
at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateParallel(ForEachOps.java:159)
at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateParallel(ForEachOps.java:173)
at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:233)
at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:497)
at java.base/java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:661)
at com.gdce.xgen.repo.psql.InsertRepo.migrate(InsertRepo.kt:18)
at com.gdce.xgen.repo.psql.InsertRepo$$FastClassBySpringCGLIB$$aa68e733.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:769)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:747)
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:353)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:99)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:747)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:689)
at com.gdce.xgen.repo.psql.InsertRepo$$EnhancerBySpringCGLIB$$85ac330b.migrate(<generated>)
at com.gdce.xgen.controller.TestController$find$$inlined$map$lambda$1.invokeSuspend(TestController.kt:46)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
at kotlinx.coroutines.DispatchedTask.run(Dispatched.kt:241)
at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:594)
at kotlinx.coroutines.scheduling.CoroutineScheduler.access$runSafely(CoroutineScheduler.kt:60)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:740)
Caused by: javax.persistence.TransactionRequiredException: No EntityManager with actual transaction available for current thread - cannot reliably process 'persist' call
I have also tried to use #org.springframework.transaction.annotation.Transactional then i got this error log:
Exception in thread "DefaultDispatcher-worker-1" javax.persistence.TransactionRequiredException
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
at java.base/java.util.concurrent.ForkJoinTask.getThrowableException(ForkJoinTask.java:603)
at java.base/java.util.concurrent.ForkJoinTask.reportException(ForkJoinTask.java:678)
at java.base/java.util.concurrent.ForkJoinTask.invoke(ForkJoinTask.java:737)
at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateParallel(ForEachOps.java:159)
at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateParallel(ForEachOps.java:173)
at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:233)
at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:497)
at java.base/java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:661)
at com.gdce.xgen.repo.psql.InsertRepo.migrate(InsertRepo.kt:17)
at com.gdce.xgen.repo.psql.InsertRepo$$FastClassBySpringCGLIB$$aa68e733.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:769)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:747)
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:353)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:99)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:747)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:689)
at com.gdce.xgen.repo.psql.InsertRepo$$EnhancerBySpringCGLIB$$82673045.migrate(<generated>)
at com.gdce.xgen.controller.TestController$find$$inlined$map$lambda$1.invokeSuspend(TestController.kt:46)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
at kotlinx.coroutines.DispatchedTask.run(Dispatched.kt:241)
at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:594)
at kotlinx.coroutines.scheduling.CoroutineScheduler.access$runSafely(CoroutineScheduler.kt:60)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:740)
Caused by: javax.persistence.TransactionRequiredException: No EntityManager with actual transaction available for current thread - cannot reliably process 'persist' call
at org.springframework.orm.jpa.SharedEntityManagerCreator$SharedEntityManagerInvocationHandler.invoke(SharedEntityManagerCreator.java:298)
at com.sun.proxy.$Proxy83.persist(Unknown Source)
at com.gdce.xgen.repo.psql.InsertRepo$migrate$1.accept(InsertRepo.kt:19)
at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1654)
at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
at java.base/java.util.stream.ForEachOps$ForEachTask.compute(ForEachOps.java:290)
at java.base/java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:746)
at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)
at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)
at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:177)
I have tried to get transactional from entityManager but i got some error:
Exception in thread "DefaultDispatcher-worker-1" java.lang.IllegalStateException: Not allowed to create transaction on shared EntityManager - use Spring transactions or EJB CMT instead
at org.springframework.orm.jpa.SharedEntityManagerCreator$SharedEntityManagerInvocationHandler.invoke(SharedEntityManagerCreator.java:264)
at com.sun.proxy.$Proxy83.getTransaction(Unknown Source)
at com.gdce.xgen.repo.psql.InsertRepo.migrate(InsertRepo.kt:17)
at com.gdce.xgen.repo.psql.InsertRepo$$FastClassBySpringCGLIB$$aa68e733.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:769)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:747)
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:353)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:99)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:747)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:689)
at com.gdce.xgen.repo.psql.InsertRepo$$EnhancerBySpringCGLIB$$3c37a126.migrate(<generated>)
at com.gdce.xgen.controller.TestController$find$$inlined$map$lambda$1.invokeSuspend(TestController.kt:46)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
at kotlinx.coroutines.DispatchedTask.run(Dispatched.kt:241)
at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:594)
at kotlinx.coroutines.scheduling.CoroutineScheduler.access$runSafely(CoroutineScheduler.kt:60)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:740)
We are using Spring Integration Java DSL and intermittently we are observing stackoverflow error , and from stack-trace , we observe the infinite loop too . But unable to figure out why its happening intermittently and why not always .
Stacktrace we see is:
at org.springframework.integration.handler.MethodInvokingMessageProcessor.processMessage(MethodInvokingMessageProcessor.java:93)
... 992 more
Caused by: java.lang.StackOverflowError
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:170)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at java.net.ManagedSocketInputStreamAPSIHighPerformanceNew.read(ManagedSocketInputStreamAPSIHighPerformanceNew.java:100)
at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)
at sun.security.ssl.InputRecord.read(InputRecord.java:503)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:973)
at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:930)
at sun.security.ssl.AppInputStream.read(AppInputStream.java:105)
at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139)
at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155)
at org.apache.http.impl.BHttpConnectionBase.fillInputBuffer(BHttpConnectionBase.java:344)
at org.apache.http.impl.BHttpConnectionBase.isStale(BHttpConnectionBase.java:364)
at org.apache.http.impl.conn.CPool.validate(CPool.java:69)
at org.apache.http.impl.conn.CPool.validate(CPool.java:43)
at org.apache.http.pool.AbstractConnPool.getPoolEntryBlocking(AbstractConnPool.java:247)
at org.apache.http.pool.AbstractConnPool.access$000(AbstractConnPool.java:64)
at org.apache.http.pool.AbstractConnPool$2.getPoolEntry(AbstractConnPool.java:192)
at org.apache.http.pool.AbstractConnPool$2.getPoolEntry(AbstractConnPool.java:185)
at org.apache.http.pool.PoolEntryFuture.get(PoolEntryFuture.java:107)
at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.leaseConnection(PoolingHttpClientConnectionManager.java:276)
at org.apache.http.impl.conn.PoolingHttpClientConnectionManager$1.get(PoolingHttpClientConnectionManager.java:263)
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:190)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
at org.springframework.ws.transport.http.HttpComponentsConnection.onSendAfterWrite(HttpComponentsConnection.java:120)
at org.springframework.ws.transport.AbstractWebServiceConnection.send(AbstractWebServiceConnection.java:48)
at org.springframework.ws.client.core.WebServiceTemplate.sendRequest(WebServiceTemplate.java:658)
at org.springframework.ws.client.core.WebServiceTemplate.doSendAndReceive(WebServiceTemplate.java:606)
at org.springframework.ws.client.core.WebServiceTemplate.sendAndReceive(WebServiceTemplate.java:555)
at org.springframework.integration.ws.MarshallingWebServiceOutboundGateway.doHandle(MarshallingWebServiceOutboundGateway.java:87)
at org.springframework.integration.ws.AbstractWebServiceOutboundGateway.handleRequestMessage(AbstractWebServiceOutboundGateway.java:188)
at org.springframework.integration.handler.AbstractReplyProducingMessageHandler$AdvisedRequestHandler.handleRequestMessage(AbstractReplyProducingMessageHandler.java:178)
at sun.reflect.GeneratedMethodAccessor729.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:333)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at org.springframework.integration.handler.advice.AbstractRequestHandlerAdvice$1.execute(AbstractRequestHandlerAdvice.java:75)
at com.bofa.paycom.ci.advice.TxnTraceAdvice.doInvoke(TxnTraceAdvice.java:71)
at org.springframework.integration.handler.advice.AbstractRequestHandlerAdvice.invoke(AbstractRequestHandlerAdvice.java:70)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:213)
at com.sun.proxy.$Proxy1080.handleRequestMessage(Unknown Source)
at org.springframework.integration.handler.AbstractReplyProducingMessageHandler.doInvokeAdvisedRequestHandler(AbstractReplyProducingMessageHandler.java:127)
at org.springframework.integration.handler.AbstractReplyProducingMessageHandler.handleMessageInternal(AbstractReplyProducingMessageHandler.java:112)
at org.springframework.integration.handler.AbstractMessageHandler.handleMessage(AbstractMessageHandler.java:127)
at org.springframework.integration.dispatcher.AbstractDispatcher.tryOptimizedDispatch(AbstractDispatcher.java:116)
at org.springframework.integration.dispatcher.UnicastingDispatcher.doDispatch(UnicastingDispatcher.java:148)
at org.springframework.integration.dispatcher.UnicastingDispatcher.dispatch(UnicastingDispatcher.java:121)
at org.springframework.integration.channel.AbstractSubscribableChannel.doSend(AbstractSubscribableChannel.java:89)
at org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:423)
at org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:373)
at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:115)
at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:45)
at org.springframework.messaging.core.AbstractMessageSendingTemplate.send(AbstractMessageSendingTemplate.java:105)
at org.springframework.integration.handler.AbstractMessageProducingHandler.sendOutput(AbstractMessageProducingHandler.java:292)
at org.springframework.integration.handler.AbstractMessageProducingHandler.produceOutput(AbstractMessageProducingHandler.java:212)
at org.springframework.integration.handler.AbstractMessageProducingHandler.sendOutputs(AbstractMessageProducingHandler.java:129)
at org.springframework.integration.handler.AbstractReplyProducingMessageHandler.handleMessageInternal(AbstractReplyProducingMessageHandler.java:115)
at org.springframework.integration.handler.AbstractMessageHandler.handleMessage(AbstractMessageHandler.java:127)
at org.springframework.integration.dispatcher.AbstractDispatcher.tryOptimizedDispatch(AbstractDispatcher.java:116)
at org.springframework.integration.dispatcher.UnicastingDispatcher.doDispatch(UnicastingDispatcher.java:148)
at org.springframework.integration.dispatcher.UnicastingDispatcher.dispatch(UnicastingDispatcher.java:121)
at org.springframework.integration.channel.AbstractSubscribableChannel.doSend(AbstractSubscribableChannel.java:89)
at org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:423)
at org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:373)
at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:115)
at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:45)
at org.springframework.messaging.core.AbstractMessageSendingTemplate.send(AbstractMessageSendingTemplate.java:105)
at org.springframework.integration.handler.AbstractMessageProducingHandler.sendOutput(AbstractMessageProducingHandler.java:292)
at org.springframework.integration.handler.AbstractMessageProducingHandler.produceOutput(AbstractMessageProducingHandler.java:212)
at org.springframework.integration.handler.AbstractMessageProducingHandler.sendOutputs(AbstractMessageProducingHandler.java:129)
at org.springframework.integration.handler.AbstractReplyProducingMessageHandler.handleMessageInternal(AbstractReplyProducingMessageHandler.java:115)
at org.springframework.integration.handler.AbstractMessageHandler.handleMessage(AbstractMessageHandler.java:127)
at org.springframework.integration.dispatcher.AbstractDispatcher.tryOptimizedDispatch(AbstractDispatcher.java:116)
at org.springframework.integration.dispatcher.UnicastingDispatcher.doDispatch(UnicastingDispatcher.java:148)
at org.springframework.integration.dispatcher.UnicastingDispatcher.dispatch(UnicastingDispatcher.java:121)
at org.springframework.integration.channel.AbstractSubscribableChannel.doSend(AbstractSubscribableChannel.java:89)
at org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:423)
at org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:373)
at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:115)`enter code here`
at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:45)
at org.springframework.messaging.core.AbstractMessageSendingTemplate.send(AbstractMessageSendingTemplate.java:105)
at org.springframework.integration.handler.AbstractMessageProducingHandler.sendOutput(AbstractMessageProducingHandler.java:292)
at org.springframework.integration.handler.AbstractMessageProducingHandler.produceOutput(AbstractMessageProducingHandler.java:212)
at org.springframework.integration.handler.AbstractMessageProducingHandler.sendOutputs(AbstractMessageProducingHandler.java:129)
at org.springframework.integration.handler.AbstractReplyProducingMessageHandler.handleMessageInternal(AbstractReplyProducingMessageHandler.java:115)
at org.springframework.integration.handler.AbstractMessageHandler.handleMessage(AbstractMessageHandler.java:127)
at org.springframework.integration.dispatcher.AbstractDispatcher.tryOptimizedDispatch(AbstractDispatcher.java:116)
at org.springframework.integration.dispatcher.UnicastingDispatcher.doDispatch(UnicastingDispatcher.java:148)
at org.springframework.integration.dispatcher.UnicastingDispatcher.dispatch(UnicastingDispatcher.java:121)
These set of lines is being looped :
<pre><code>
at org.springframework.integration.handler.AbstractMessageProducingHandler.sendOutput(AbstractMessageProducingHandler.java:292)
at org.springframework.integration.handler.AbstractMessageProducingHandler.produceOutput(AbstractMessageProducingHandler.java:212)
at org.springframework.integration.handler.AbstractMessageProducingHandler.sendOutputs(AbstractMessageProducingHandler.java:129)
at org.springframework.integration.handler.AbstractReplyProducingMessageHandler.handleMessageInternal(AbstractReplyProducingMessageHandler.java:115)
at org.springframework.integration.handler.AbstractMessageHandler.handleMessage(AbstractMessageHandler.java:127)
at org.springframework.integration.dispatcher.AbstractDispatcher.tryOptimizedDispatch(AbstractDispatcher.java:116)
at org.springframework.integration.dispatcher.UnicastingDispatcher.doDispatch(UnicastingDispatcher.java:148)
at org.springframework.integration.dispatcher.UnicastingDispatcher.dispatch(UnicastingDispatcher.java:121)
at org.springframework.integration.channel.AbstractSubscribableChannel.doSend(AbstractSubscribableChannel.java:89)
at org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:423)
at org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:373)
at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:115)
at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:45)
at org.springframework.messaging.core.AbstractMessageSendingTemplate.send(AbstractMessageSendingTemplate.java:105)
Any help or pointers will help .
My Integration flow looks as below :
#Bean
public IntegrationFlow lxxDownstreamNotificationFlow() {
LOGGER.info("Inside the downstream notification flow - lcmDownstreamNotificationFlow()");
return IntegrationFlows.from(lxxDownstreamNotificationChannel)
.handle(lxxNotificationProcessor)
.gateway(requestxxxxxxxEventLogger)
.channel("lxxCoreReplyChannel")
.get();
}
I could not find any flow/component where inputChannel & OutPutChannel is same . But we have some common utility flow , which is used , somewhere it sense me is this causing the issue .
Gateway which I mentioned earlier is routing to below integration flow
#Component
public final class EventLoggingFlow implements IntegrationFlow {
private Long eventId;
private String message;
#Autowired
private EventPublishingHandler eventPublishingHandler;
public EventLoggingFlow() {
}
public EventLoggingFlow(Long eventId, String message) {
super();
this.eventId = eventId;
this.message = message;
}
#Override
public void configure(IntegrationFlowDefinition<?> t) {
t .fixedSubscriberChannel()
.enrichHeaders(new Consumer<HeaderEnricherSpec>() {
#Override
public void accept(HeaderEnricherSpec t) {
t.headerExpression(HeaderKeys.SOURCE, "payload.sourceChannel");
}
})
.enrich(new EventEnricherSpec(eventId, message))
.publishSubscribeChannel(eventPublishingHandler)
.handle(new BridgeHandler());
}
}
Publish/Subscribe flow is implemented as below :
#Component
#DependsOn("eventSubcriberFlow")
public class EventPublishingHandler implements Consumer<PublishSubscribeSpec> {
#Autowired
private EventSubcriberFlow eventSubcriberFlow;
#Override
public void accept(PublishSubscribeSpec t) {
t.subscribe(eventSubcriberFlow);
}
}
#Component
public class EventSubcriberFlow implements IntegrationFlow {
#Autowired
private EventSubscribingHandler eventSubscribingHandler;
#Override
public void configure(IntegrationFlowDefinition<?> flow) {
flow.fixedSubscriberChannel()
.handle(eventSubscribingHandler)
.channel("nullChannel");
}
}
Is fixedSubscriberChannel() causing a looping ?
The typical error for this kind of behavior is about sending outputs to the channel where its flow ends up on this channel as well and resend message again. It’s hard to say where, but according stack trace there is something with calling SOAP service and you don’t show that.