I am at a loss how to debug this issue, so not even sure which code snippets to put up. Here is the basic setup:
I have to read from a REST API and write to a SOAP API. As far as I can tell the reading and writing are working without any issue. I have 3 sets of reads with 3 matching sets of writes that can happen in parallel so I have built them using the FlowBuilder like so:
final Flow firstFlow = new FlowBuilder<Flow>("firstFlow").from(Step1).end();
final Flow secondFlow = new FlowBuilder<Flow>("secondFlow").from(Step2).end();
final Flow thirdFlow = new FlowBuilder<Flow>("thirdFlow").from(Step3).end();
Flow mdmFlow = builder2.split(new SimpleAsyncTaskExecutor()).add(firstFlow, secondFlow, thirdFlow).build();
FlowBuilder<SimpleFlow> builder2 = new FlowBuilder<SimpleFlow>("attribsAndItemsFlow");
Then I build it like so:
return jobs.get("feedControlJob")
.incrementer(new RunIdIncrementer())
.listener(listener)
.preventRestart()
.start(mdmFlow)
.end()
.build();
In the listener I have:
public void beforeJob(JobExecution jobExecution) {
log.info("Starting Batch Job...");
}
Which does put the line in the log, so I know the listener is being loaded properly. I also have:
#Override
public void afterJob(JobExecution jobExecution) {
log.info("Batch job finished, saving state.");
// some stuf to write files here
}
That log.info never runs so I am sure it is not getting there. However, the last line of my writer is:
log.debug("Finished writes.");
That does write to the log for all three flows. When I try walking through with the debugger I quickly end up in sections of code that STS does not have the source for.
Best I can tell is that some place between the end of the last writer exit, and before the After Job event, something is wrong. I get no errors in the console.
I ran it with log level set to debug, here is the output starting with my last line of code:
2016-08-30 16:46:37.046 DEBUG 11260 --- [cTaskExecutor-2] com.dcsg.writeobject.MdmItemWriterClass : Finished writes.
2016-08-30 16:46:37.046 DEBUG 11260 --- [cTaskExecutor-2] o.s.b.c.step.item.ChunkOrientedTasklet : Inputs not busy, ended: true
2016-08-30 16:46:37.046 DEBUG 11260 --- [cTaskExecutor-2] o.s.batch.core.step.tasklet.TaskletStep : Applying contribution: [StepContribution: read=20, written=20, filtered=0, readSkips=0, writeSkips=0, processSkips=0, exitStatus=EXECUTING]
2016-08-30 16:46:37.046 DEBUG 11260 --- [cTaskExecutor-2] o.s.j.d.DataSourceTransactionManager : Participating in existing transaction
2016-08-30 16:46:37.047 DEBUG 11260 --- [cTaskExecutor-2] o.s.jdbc.core.JdbcTemplate : Executing prepared SQL update
2016-08-30 16:46:37.047 DEBUG 11260 --- [cTaskExecutor-2] o.s.jdbc.core.JdbcTemplate : Executing prepared SQL statement [UPDATE BATCH_STEP_EXECUTION_CONTEXT SET SHORT_CONTEXT = ?, SERIALIZED_CONTEXT = ? WHERE STEP_EXECUTION_ID = ?]
2016-08-30 16:46:37.048 DEBUG 11260 --- [cTaskExecutor-2] o.s.jdbc.core.JdbcTemplate : SQL update affected 1 rows
2016-08-30 16:46:37.049 DEBUG 11260 --- [cTaskExecutor-2] o.s.batch.core.step.tasklet.TaskletStep : Saving step execution before commit: StepExecution: id=0, version=1, name=GG Step, status=STARTED, exitStatus=EXECUTING, readCount=20, filterCount=0, writeCount=20 readSkipCount=0, writeSkipCount=0, processSkipCount=0, commitCount=1, rollbackCount=0, exitDescription=
2016-08-30 16:46:37.049 DEBUG 11260 --- [cTaskExecutor-2] o.s.j.d.DataSourceTransactionManager : Participating in existing transaction
2016-08-30 16:46:37.049 DEBUG 11260 --- [cTaskExecutor-2] o.s.jdbc.core.JdbcTemplate : Executing prepared SQL update
2016-08-30 16:46:37.049 DEBUG 11260 --- [cTaskExecutor-2] o.s.jdbc.core.JdbcTemplate : Executing prepared SQL statement [UPDATE BATCH_STEP_EXECUTION set START_TIME = ?, END_TIME = ?, STATUS = ?, COMMIT_COUNT = ?, READ_COUNT = ?, FILTER_COUNT = ?, WRITE_COUNT = ?, EXIT_CODE = ?, EXIT_MESSAGE = ?, VERSION = ?, READ_SKIP_COUNT = ?, PROCESS_SKIP_COUNT = ?, WRITE_SKIP_COUNT = ?, ROLLBACK_COUNT = ?, LAST_UPDATED = ? where STEP_EXECUTION_ID = ? and VERSION = ?]
2016-08-30 16:46:37.050 DEBUG 11260 --- [cTaskExecutor-2] o.s.jdbc.core.JdbcTemplate : SQL update affected 1 rows
2016-08-30 16:46:37.050 DEBUG 11260 --- [cTaskExecutor-2] o.s.jdbc.core.JdbcTemplate : Executing prepared SQL query
2016-08-30 16:46:37.050 DEBUG 11260 --- [cTaskExecutor-2] o.s.jdbc.core.JdbcTemplate : Executing prepared SQL statement [SELECT VERSION FROM BATCH_JOB_EXECUTION WHERE JOB_EXECUTION_ID=?]
2016-08-30 16:46:37.051 DEBUG 11260 --- [cTaskExecutor-2] o.s.j.d.DataSourceTransactionManager : Initiating transaction commit
2016-08-30 16:46:37.051 DEBUG 11260 --- [cTaskExecutor-2] o.s.j.d.DataSourceTransactionManager : Committing JDBC transaction on Connection [org.hsqldb.jdbc.JDBCConnection#557528a5]
2016-08-30 16:46:37.051 DEBUG 11260 --- [cTaskExecutor-2] o.s.j.d.DataSourceTransactionManager : Releasing JDBC Connection [org.hsqldb.jdbc.JDBCConnection#557528a5] after transaction
2016-08-30 16:46:37.052 DEBUG 11260 --- [cTaskExecutor-2] o.s.jdbc.datasource.DataSourceUtils : Returning JDBC Connection to DataSource
2016-08-30 16:46:37.052 DEBUG 11260 --- [cTaskExecutor-2] o.s.batch.repeat.support.RepeatTemplate : Repeat is complete according to policy and result value.
2016-08-30 16:46:37.052 DEBUG 11260 --- [cTaskExecutor-2] o.s.batch.core.step.AbstractStep : Step execution success: id=0
2016-08-30 16:46:37.052 DEBUG 11260 --- [cTaskExecutor-2] o.s.j.d.DataSourceTransactionManager : Creating new transaction with name [org.springframework.batch.core.repository.support.SimpleJobRepository.updateExecutionContext]: PROPAGATION_REQUIRED,ISOLATION_DEFAULT
2016-08-30 16:46:37.052 DEBUG 11260 --- [cTaskExecutor-2] o.s.j.datasource.SimpleDriverDataSource : Creating new JDBC Driver Connection to [jdbc:hsqldb:mem:testdb]
2016-08-30 16:46:37.052 DEBUG 11260 --- [cTaskExecutor-2] o.s.j.d.DataSourceTransactionManager : Acquired Connection [org.hsqldb.jdbc.JDBCConnection#5e762da3] for JDBC transaction
2016-08-30 16:46:37.052 DEBUG 11260 --- [cTaskExecutor-2] o.s.j.d.DataSourceTransactionManager : Switching JDBC Connection [org.hsqldb.jdbc.JDBCConnection#5e762da3] to manual commit
2016-08-30 16:46:37.053 DEBUG 11260 --- [cTaskExecutor-2] o.s.jdbc.core.JdbcTemplate : Executing prepared SQL update
2016-08-30 16:46:37.053 DEBUG 11260 --- [cTaskExecutor-2] o.s.jdbc.core.JdbcTemplate : Executing prepared SQL statement [UPDATE BATCH_STEP_EXECUTION_CONTEXT SET SHORT_CONTEXT = ?, SERIALIZED_CONTEXT = ? WHERE STEP_EXECUTION_ID = ?]
2016-08-30 16:46:37.054 DEBUG 11260 --- [cTaskExecutor-2] o.s.jdbc.core.JdbcTemplate : SQL update affected 1 rows
2016-08-30 16:46:37.054 DEBUG 11260 --- [cTaskExecutor-2] o.s.j.d.DataSourceTransactionManager : Initiating transaction commit
2016-08-30 16:46:37.054 DEBUG 11260 --- [cTaskExecutor-2] o.s.j.d.DataSourceTransactionManager : Committing JDBC transaction on Connection [org.hsqldb.jdbc.JDBCConnection#5e762da3]
2016-08-30 16:46:37.054 DEBUG 11260 --- [cTaskExecutor-2] o.s.j.d.DataSourceTransactionManager : Releasing JDBC Connection [org.hsqldb.jdbc.JDBCConnection#5e762da3] after transaction
2016-08-30 16:46:37.054 DEBUG 11260 --- [cTaskExecutor-2] o.s.jdbc.datasource.DataSourceUtils : Returning JDBC Connection to DataSource
2016-08-30 16:46:37.054 DEBUG 11260 --- [cTaskExecutor-2] o.s.j.d.DataSourceTransactionManager : Creating new transaction with name [org.springframework.batch.core.repository.support.SimpleJobRepository.update]: PROPAGATION_REQUIRED,ISOLATION_DEFAULT
2016-08-30 16:46:37.054 DEBUG 11260 --- [cTaskExecutor-2] o.s.j.datasource.SimpleDriverDataSource : Creating new JDBC Driver Connection to [jdbc:hsqldb:mem:testdb]
2016-08-30 16:46:37.054 DEBUG 11260 --- [cTaskExecutor-2] o.s.j.d.DataSourceTransactionManager : Acquired Connection [org.hsqldb.jdbc.JDBCConnection#5a915472] for JDBC transaction
2016-08-30 16:46:37.055 DEBUG 11260 --- [cTaskExecutor-2] o.s.j.d.DataSourceTransactionManager : Switching JDBC Connection [org.hsqldb.jdbc.JDBCConnection#5a915472] to manual commit
2016-08-30 16:46:37.055 DEBUG 11260 --- [cTaskExecutor-2] o.s.jdbc.core.JdbcTemplate : Executing prepared SQL update
2016-08-30 16:46:37.055 DEBUG 11260 --- [cTaskExecutor-2] o.s.jdbc.core.JdbcTemplate : Executing prepared SQL statement [UPDATE BATCH_STEP_EXECUTION set START_TIME = ?, END_TIME = ?, STATUS = ?, COMMIT_COUNT = ?, READ_COUNT = ?, FILTER_COUNT = ?, WRITE_COUNT = ?, EXIT_CODE = ?, EXIT_MESSAGE = ?, VERSION = ?, READ_SKIP_COUNT = ?, PROCESS_SKIP_COUNT = ?, WRITE_SKIP_COUNT = ?, ROLLBACK_COUNT = ?, LAST_UPDATED = ? where STEP_EXECUTION_ID = ? and VERSION = ?]
2016-08-30 16:46:37.056 DEBUG 11260 --- [cTaskExecutor-2] o.s.jdbc.core.JdbcTemplate : SQL update affected 1 rows
2016-08-30 16:46:37.056 DEBUG 11260 --- [cTaskExecutor-2] o.s.jdbc.core.JdbcTemplate : Executing prepared SQL query
2016-08-30 16:46:37.056 DEBUG 11260 --- [cTaskExecutor-2] o.s.jdbc.core.JdbcTemplate : Executing prepared SQL statement [SELECT VERSION FROM BATCH_JOB_EXECUTION WHERE JOB_EXECUTION_ID=?]
2016-08-30 16:46:37.056 DEBUG 11260 --- [cTaskExecutor-2] o.s.j.d.DataSourceTransactionManager : Initiating transaction commit
2016-08-30 16:46:37.057 DEBUG 11260 --- [cTaskExecutor-2] o.s.j.d.DataSourceTransactionManager : Committing JDBC transaction on Connection [org.hsqldb.jdbc.JDBCConnection#5a915472]
2016-08-30 16:46:37.057 DEBUG 11260 --- [cTaskExecutor-2] o.s.j.d.DataSourceTransactionManager : Releasing JDBC Connection [org.hsqldb.jdbc.JDBCConnection#5a915472] after transaction
2016-08-30 16:46:37.057 DEBUG 11260 --- [cTaskExecutor-2] o.s.jdbc.datasource.DataSourceUtils : Returning JDBC Connection to DataSource
2016-08-30 16:46:37.057 DEBUG 11260 --- [cTaskExecutor-2] o.s.batch.core.step.AbstractStep : Step execution complete: StepExecution: id=0, version=3, name=GG Step, status=COMPLETED, exitStatus=COMPLETED, readCount=20, filterCount=0, writeCount=20 readSkipCount=0, writeSkipCount=0, processSkipCount=0, commitCount=1, rollbackCount=0
2016-08-30 16:46:37.057 DEBUG 11260 --- [cTaskExecutor-2] o.s.j.d.DataSourceTransactionManager : Creating new transaction with name [org.springframework.batch.core.repository.support.SimpleJobRepository.updateExecutionContext]: PROPAGATION_REQUIRED,ISOLATION_DEFAULT
2016-08-30 16:46:37.057 DEBUG 11260 --- [cTaskExecutor-2] o.s.j.datasource.SimpleDriverDataSource : Creating new JDBC Driver Connection to [jdbc:hsqldb:mem:testdb]
2016-08-30 16:46:37.057 DEBUG 11260 --- [cTaskExecutor-2] o.s.j.d.DataSourceTransactionManager : Acquired Connection [org.hsqldb.jdbc.JDBCConnection#5802f2e9] for JDBC transaction
2016-08-30 16:46:37.057 DEBUG 11260 --- [cTaskExecutor-2] o.s.j.d.DataSourceTransactionManager : Switching JDBC Connection [org.hsqldb.jdbc.JDBCConnection#5802f2e9] to manual commit
2016-08-30 16:46:37.058 DEBUG 11260 --- [cTaskExecutor-2] o.s.jdbc.core.JdbcTemplate : Executing prepared SQL update
2016-08-30 16:46:37.058 DEBUG 11260 --- [cTaskExecutor-2] o.s.jdbc.core.JdbcTemplate : Executing prepared SQL statement [UPDATE BATCH_JOB_EXECUTION_CONTEXT SET SHORT_CONTEXT = ?, SERIALIZED_CONTEXT = ? WHERE JOB_EXECUTION_ID = ?]
2016-08-30 16:46:37.058 DEBUG 11260 --- [cTaskExecutor-2] o.s.jdbc.core.JdbcTemplate : SQL update affected 1 rows
2016-08-30 16:46:37.059 DEBUG 11260 --- [cTaskExecutor-2] o.s.j.d.DataSourceTransactionManager : Initiating transaction commit
2016-08-30 16:46:37.059 DEBUG 11260 --- [cTaskExecutor-2] o.s.j.d.DataSourceTransactionManager : Committing JDBC transaction on Connection [org.hsqldb.jdbc.JDBCConnection#5802f2e9]
2016-08-30 16:46:37.059 DEBUG 11260 --- [cTaskExecutor-2] o.s.j.d.DataSourceTransactionManager : Releasing JDBC Connection [org.hsqldb.jdbc.JDBCConnection#5802f2e9] after transaction
2016-08-30 16:46:37.059 DEBUG 11260 --- [cTaskExecutor-2] o.s.jdbc.datasource.DataSourceUtils : Returning JDBC Connection to DataSource
2016-08-30 16:46:37.059 DEBUG 11260 --- [cTaskExecutor-2] o.s.b.core.job.flow.support.SimpleFlow : Completed state=thirdFlow.GG Step with status=COMPLETED
2016-08-30 16:46:37.059 DEBUG 11260 --- [cTaskExecutor-2] o.s.b.core.job.flow.support.SimpleFlow : Handling state=thirdFlow.COMPLETED
2016-08-30 16:46:37.059 DEBUG 11260 --- [cTaskExecutor-2] o.s.b.core.job.flow.support.SimpleFlow : Completed state=thirdFlow.COMPLETED with status=COMPLETED
After that, it just hangs indefinitely.
EDIT: A Thread Dump was requested, so here is one from while it is hanging:
2016-09-01 13:42:00
Full thread dump Java HotSpot(TM) 64-Bit Server VM (25.66-b18 mixed mode):
"SimpleAsyncTaskExecutor-1" #16 prio=5 os_prio=0 tid=0x00000000211c2000 nid=0x157c runnable [0x000000001eb0d000]
java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(Unknown Source)
at java.net.SocketInputStream.read(Unknown Source)
at java.net.SocketInputStream.read(Unknown Source)
at sun.security.ssl.InputRecord.readFully(Unknown Source)
at sun.security.ssl.InputRecord.read(Unknown Source)
at sun.security.ssl.SSLSocketImpl.readRecord(Unknown Source)
- locked <0x00000006c1eba630> (a java.lang.Object)
at sun.security.ssl.SSLSocketImpl.readDataRecord(Unknown Source)
at sun.security.ssl.AppInputStream.read(Unknown Source)
- locked <0x00000006c1ebb978> (a sun.security.ssl.AppInputStream)
at java.io.BufferedInputStream.fill(Unknown Source)
at java.io.BufferedInputStream.read1(Unknown Source)
at java.io.BufferedInputStream.read(Unknown Source)
- locked <0x00000006c1ebb950> (a java.io.BufferedInputStream)
at sun.net.www.http.HttpClient.parseHTTPHeader(Unknown Source)
at sun.net.www.http.HttpClient.parseHTTP(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(Unknown Source)
- locked <0x00000006c1ef2178> (a sun.net.www.protocol.https.DelegateHttpsURLConnection)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)
- locked <0x00000006c1ef2178> (a sun.net.www.protocol.https.DelegateHttpsURLConnection)
at java.net.HttpURLConnection.getResponseCode(Unknown Source)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(Unknown Source)
at org.springframework.http.client.SimpleClientHttpResponse.getRawStatusCode(SimpleClientHttpResponse.java:48)
at org.springframework.http.client.AbstractClientHttpResponse.getStatusCode(AbstractClientHttpResponse.java:33)
at org.springframework.web.client.DefaultResponseErrorHandler.getHttpStatusCode(DefaultResponseErrorHandler.java:56)
at org.springframework.web.client.DefaultResponseErrorHandler.hasError(DefaultResponseErrorHandler.java:50)
at org.springframework.web.client.RestTemplate.handleResponse(RestTemplate.java:629)
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:597)
at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:557)
at org.springframework.web.client.RestTemplate.exchange(RestTemplate.java:475)
at com.dcsg.mdm.MdmItemItemReaderClass.getResultSetId(MdmItemItemReaderClass.java:173)
at com.dcsg.mdm.MdmItemItemReaderClass.init(MdmItemItemReaderClass.java:84)
at com.dcsg.mdm.MdmItemItemReaderClass.read(MdmItemItemReaderClass.java:107)
at com.dcsg.mdm.MdmItemItemReaderClass.read(MdmItemItemReaderClass.java:1)
at org.springframework.batch.core.step.item.SimpleChunkProvider.doRead(SimpleChunkProvider.java:91)
at org.springframework.batch.core.step.item.SimpleChunkProvider.read(SimpleChunkProvider.java:157)
at org.springframework.batch.core.step.item.SimpleChunkProvider$1.doInIteration(SimpleChunkProvider.java:116)
at org.springframework.batch.repeat.support.RepeatTemplate.getNextResult(RepeatTemplate.java:374)
at org.springframework.batch.repeat.support.RepeatTemplate.executeInternal(RepeatTemplate.java:215)
at org.springframework.batch.repeat.support.RepeatTemplate.iterate(RepeatTemplate.java:144)
at org.springframework.batch.core.step.item.SimpleChunkProvider.provide(SimpleChunkProvider.java:110)
at org.springframework.batch.core.step.item.ChunkOrientedTasklet.execute(ChunkOrientedTasklet.java:69)
at org.springframework.batch.core.step.tasklet.TaskletStep$ChunkTransactionCallback.doInTransaction(TaskletStep.java:406)
at org.springframework.batch.core.step.tasklet.TaskletStep$ChunkTransactionCallback.doInTransaction(TaskletStep.java:330)
at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:133)
at org.springframework.batch.core.step.tasklet.TaskletStep$2.doInChunkContext(TaskletStep.java:271)
at org.springframework.batch.core.scope.context.StepContextRepeatCallback.doInIteration(StepContextRepeatCallback.java:81)
at org.springframework.batch.repeat.support.RepeatTemplate.getNextResult(RepeatTemplate.java:374)
at org.springframework.batch.repeat.support.RepeatTemplate.executeInternal(RepeatTemplate.java:215)
at org.springframework.batch.repeat.support.RepeatTemplate.iterate(RepeatTemplate.java:144)
at org.springframework.batch.core.step.tasklet.TaskletStep.doExecute(TaskletStep.java:257)
at org.springframework.batch.core.step.AbstractStep.execute(AbstractStep.java:200)
at org.springframework.batch.core.job.SimpleStepHandler.handleStep(SimpleStepHandler.java:148)
at org.springframework.batch.core.job.flow.JobFlowExecutor.executeStep(JobFlowExecutor.java:64)
at org.springframework.batch.core.job.flow.support.state.StepState.handle(StepState.java:67)
at org.springframework.batch.core.job.flow.support.SimpleFlow.resume(SimpleFlow.java:169)
at org.springframework.batch.core.job.flow.support.SimpleFlow.start(SimpleFlow.java:144)
at org.springframework.batch.core.job.flow.support.state.SplitState$1.call(SplitState.java:93)
at org.springframework.batch.core.job.flow.support.state.SplitState$1.call(SplitState.java:90)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
"RMI TCP Accept-0" #13 daemon prio=5 os_prio=0 tid=0x000000001e36f800 nid=0xbf8 runnable [0x000000001f8df000]
java.lang.Thread.State: RUNNABLE
at java.net.DualStackPlainSocketImpl.accept0(Native Method)
at java.net.DualStackPlainSocketImpl.socketAccept(Unknown Source)
at java.net.AbstractPlainSocketImpl.accept(Unknown Source)
at java.net.PlainSocketImpl.accept(Unknown Source)
- locked <0x00000006c0046480> (a java.net.SocksSocketImpl)
at java.net.ServerSocket.implAccept(Unknown Source)
at java.net.ServerSocket.accept(Unknown Source)
at sun.management.jmxremote.LocalRMIServerSocketFactory$1.accept(Unknown Source)
at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(Unknown Source)
at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
"RMI TCP Accept-26293" #12 daemon prio=5 os_prio=0 tid=0x000000001e381000 nid=0x15e0 runnable [0x000000001f7ae000]
java.lang.Thread.State: RUNNABLE
at java.net.DualStackPlainSocketImpl.accept0(Native Method)
at java.net.DualStackPlainSocketImpl.socketAccept(Unknown Source)
at java.net.AbstractPlainSocketImpl.accept(Unknown Source)
at java.net.PlainSocketImpl.accept(Unknown Source)
- locked <0x00000006c00469e0> (a java.net.SocksSocketImpl)
at java.net.ServerSocket.implAccept(Unknown Source)
at java.net.ServerSocket.accept(Unknown Source)
at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(Unknown Source)
at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
"RMI TCP Accept-0" #11 daemon prio=5 os_prio=0 tid=0x000000001e388800 nid=0x2ba4 runnable [0x000000001ed3e000]
java.lang.Thread.State: RUNNABLE
at java.net.DualStackPlainSocketImpl.accept0(Native Method)
at java.net.DualStackPlainSocketImpl.socketAccept(Unknown Source)
at java.net.AbstractPlainSocketImpl.accept(Unknown Source)
at java.net.PlainSocketImpl.accept(Unknown Source)
- locked <0x00000006c0046e70> (a java.net.SocksSocketImpl)
at java.net.ServerSocket.implAccept(Unknown Source)
at java.net.ServerSocket.accept(Unknown Source)
at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(Unknown Source)
at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
"Service Thread" #9 daemon prio=9 os_prio=0 tid=0x000000001e018000 nid=0x1e24 runnable [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"C1 CompilerThread2" #8 daemon prio=9 os_prio=2 tid=0x000000001df94800 nid=0x28b0 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"C2 CompilerThread1" #7 daemon prio=9 os_prio=2 tid=0x000000001df92800 nid=0x2150 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"C2 CompilerThread0" #6 daemon prio=9 os_prio=2 tid=0x000000001c89d000 nid=0x2d8c waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"Attach Listener" #5 daemon prio=5 os_prio=2 tid=0x000000001c89b800 nid=0x2de4 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"Signal Dispatcher" #4 daemon prio=9 os_prio=2 tid=0x000000001c898800 nid=0x2d18 runnable [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"Finalizer" #3 daemon prio=8 os_prio=1 tid=0x000000001c834000 nid=0x2988 in Object.wait() [0x000000001da1f000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000006c0089980> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(Unknown Source)
- locked <0x00000006c0089980> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(Unknown Source)
at java.lang.ref.Finalizer$FinalizerThread.run(Unknown Source)
"Reference Handler" #2 daemon prio=10 os_prio=2 tid=0x000000001c832000 nid=0x2e58 in Object.wait() [0x000000001d90f000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000006c00899c0> (a java.lang.ref.Reference$Lock)
at java.lang.Object.wait(Unknown Source)
at java.lang.ref.Reference$ReferenceHandler.run(Unknown Source)
- locked <0x00000006c00899c0> (a java.lang.ref.Reference$Lock)
"main" #1 prio=5 os_prio=0 tid=0x0000000002baa000 nid=0x2a64 waiting on condition [0x0000000002f7d000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000006c1fc8ea0> (a java.util.concurrent.FutureTask)
at java.util.concurrent.locks.LockSupport.park(Unknown Source)
at java.util.concurrent.FutureTask.awaitDone(Unknown Source)
at java.util.concurrent.FutureTask.get(Unknown Source)
at org.springframework.batch.core.job.flow.support.state.SplitState.handle(SplitState.java:113)
at org.springframework.batch.core.job.flow.support.SimpleFlow.resume(SimpleFlow.java:169)
at org.springframework.batch.core.job.flow.support.SimpleFlow.start(SimpleFlow.java:144)
at org.springframework.batch.core.job.flow.FlowJob.doExecute(FlowJob.java:134)
at org.springframework.batch.core.job.AbstractJob.execute(AbstractJob.java:306)
at org.springframework.batch.core.launch.support.SimpleJobLauncher$1.run(SimpleJobLauncher.java:135)
at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:50)
at org.springframework.batch.core.launch.support.SimpleJobLauncher.run(SimpleJobLauncher.java:128)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:302)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at org.springframework.batch.core.configuration.annotation.SimpleBatchConfiguration$PassthruAdvice.invoke(SimpleBatchConfiguration.java:127)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:208)
at com.sun.proxy.$Proxy33.run(Unknown Source)
at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.execute(JobLauncherCommandLineRunner.java:216)
at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.executeLocalJobs(JobLauncherCommandLineRunner.java:233)
at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.launchJobFromProperties(JobLauncherCommandLineRunner.java:125)
at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.run(JobLauncherCommandLineRunner.java:119)
at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:800)
at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:784)
at org.springframework.boot.SpringApplication.afterRefresh(SpringApplication.java:771)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:307)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1185)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1174)
at com.dcsg.mdm.MdmAttributeControlApplication.main(MdmAttributeControlApplication.java:28)
"VM Thread" os_prio=2 tid=0x000000001c82c800 nid=0x2d14 runnable
"GC task thread#0 (ParallelGC)" os_prio=0 tid=0x0000000002bbe800 nid=0x2bd0 runnable
"GC task thread#1 (ParallelGC)" os_prio=0 tid=0x0000000002bc0000 nid=0x272c runnable
"GC task thread#2 (ParallelGC)" os_prio=0 tid=0x0000000002bc2000 nid=0x1ca8 runnable
"GC task thread#3 (ParallelGC)" os_prio=0 tid=0x0000000002bc5800 nid=0x2b3c runnable
"VM Periodic Task Thread" os_prio=2 tid=0x000000001e371000 nid=0x2a00 waiting on condition
JNI global references: 237
It turns out that at least in some cases, RestTemplate.exchange() can get into a state where it never returns. When this happens, the ItemReader also never returns, and no errors are reported. The Batch process gets to the end, and sees one (or more) ItemReader process is still unfinished, and waits indefinitely for that reader to return. I have even let it sit overnight, it never gives up waiting.
I resolved this by adding a time out to the RestTemplate call. To set the timeout you need to create a ClientHttpRequest like so:
private ClientHttpRequestFactory getClientHttpRequestFactory() {
HttpComponentsClientHttpRequestFactory clientHttpRequestFactory = new HttpComponentsClientHttpRequestFactory();
clientHttpRequestFactory.setConnectTimeout(this.timeout);
return clientHttpRequestFactory;
}
Then when you create the RestTemplage, you do it like so:
RestTemplate restTemplate = new RestTemplate(getClientHttpRequestFactory());
I had assumed that the RestTemplate had some default timeout, but that does not appear to be the case.
I was finally able to track this down thanks to Hansjoerg's suggestion above to study the thread dump.
Related
What is wrong with this Config ? I am not able to overwrite an existing file in the sftp server location (new and old file name are the same). the directory has 777 as chmod too. I have added the error logs below. I have attached the atmoz dockercompose file for you to analyze this issue.
#Bean
#ServiceActivator(inputChannel = "toSftpChannel")
public MessageHandler toSftpChannelHandler() {
SftpOutboundGateway sftpOutboundGateway =
new SftpOutboundGateway(
sftpDestinationSessionFactory(),
AbstractRemoteFileOutboundGateway.Command.PUT.getCommand(),
"headers['remote-target-dir']");
sftpOutboundGateway.setRemoteDirectoryExpression(PARSER.parseExpression("headers['remote-target-dir']"));
sftpOutboundGateway.setUseTemporaryFileName(true);
sftpOutboundGateway.setTemporaryFileSuffix(".temp");
sftpOutboundGateway.setFileExistsMode(FileExistsMode.REPLACE);
sftpOutboundGateway.setOutputChannelName("deleteLocalFilesChannel");
sftpOutboundGateway.setAdviceChain(List.of(retryAdvice));
return sftpOutboundGateway;
}
file_remoteFile=SampleIntegrationFile.txt, timestamp=1675720242276}]
2023-02-07T07:55:18.854+10:00 INFO 9752 --- [ scheduling-1] c.d.s.sftpTosftp.middle.ProcessMessage : Input File Path: C:\tmp\sftp-inbound\sftpToSftpData\SampleIntegrationFile.txt
2023-02-07T07:55:18.854+10:00 INFO 9752 --- [ scheduling-1] c.d.s.sftpTosftp.middle.ProcessMessage : Input File Headers: [file_remoteHostPort=localhost:22, file_name=SampleIntegrationFile.txt, file_remoteDirectory=//sftp/Outgoing/sftpToSftp/, file_originalFile=\tmp\sftp-inbound\sftpToSftpData\SampleIntegrationFile.txt, id=35841395-ac31-6636-966c-fc3cc0d04329, file_relativePath=SampleIntegrationFile.txt, file_remoteFile=SampleIntegrationFile.txt, timestamp=1675720518854]
2023-02-07T07:55:18.854+10:00 DEBUG 9752 --- [ scheduling-1] o.s.retry.support.RetryTemplate : Retry: count=0
2023-02-07T07:55:21.891+10:00 DEBUG 9752 --- [ scheduling-1] o.s.retry.support.RetryTemplate : Checking for rethrow: count=1
2023-02-07T07:55:21.891+10:00 DEBUG 9752 --- [ scheduling-1] o.s.retry.support.RetryTemplate : Retry: count=1
2023-02-07T07:55:21.916+10:00 WARN 9752 --- []-nio2-thread-2] o.a.s.c.k.AcceptAllServerKeyVerifier : Server at localhost/127.0.0.1:22 presented unverified RSA key: SHA256:kQ7FBDG5+ZBmGHEsYW4RQmhmKHptYsN0MsAeQ7Cx1So
2023-02-07T07:55:22.014+10:00 WARN 9752 --- [-nio2-thread-13] o.a.s.c.session.ClientConnectionService : globalRequest(ClientConnectionService[ClientSessionImpl[admin#localhost/127.0.0.1:22]])[hostkeys-00#openssh.com, want-reply=false] failed (SshException) to process: EdDSA provider not supported
2023-02-07T07:55:25.250+10:00 DEBUG 9752 --- [ scheduling-1] o.s.retry.support.RetryTemplate : Checking for rethrow: count=2
2023-02-07T07:55:25.250+10:00 DEBUG 9752 --- [ scheduling-1] o.s.retry.support.RetryTemplate : Retry: count=2
2023-02-07T07:55:25.271+10:00 WARN 9752 --- []-nio2-thread-4] o.a.s.c.k.AcceptAllServerKeyVerifier : Server at localhost/127.0.0.1:22 presented unverified RSA key: SHA256:kQ7FBDG5+ZBmGHEsYW4RQmhmKHptYsN0MsAeQ7Cx1So
2023-02-07T07:55:25.327+10:00 WARN 9752 --- [-nio2-thread-13] o.a.s.c.session.ClientConnectionService : globalRequest(ClientConnectionService[ClientSessionImpl[admin#localhost/127.0.0.1:22]])[hostkeys-00#openssh.com, want-reply=false] failed (SshException) to process: EdDSA provider not supported
2023-02-07T07:55:28.561+10:00 DEBUG 9752 --- [ scheduling-1] o.s.retry.support.RetryTemplate : Checking for rethrow: count=3
2023-02-07T07:55:28.561+10:00 DEBUG 9752 --- [ scheduling-1] o.s.retry.support.RetryTemplate : Retry: count=3
2023-02-07T07:55:28.584+10:00 WARN 9752 --- []-nio2-thread-8] o.a.s.c.k.AcceptAllServerKeyVerifier : Server at localhost/127.0.0.1:22 presented unverified RSA key: SHA256:kQ7FBDG5+ZBmGHEsYW4RQmhmKHptYsN0MsAeQ7Cx1So
2023-02-07T07:55:28.683+10:00 WARN 9752 --- [-nio2-thread-13] o.a.s.c.session.ClientConnectionService : globalRequest(ClientConnectionService[ClientSessionImpl[admin#localhost/127.0.0.1:22]])[hostkeys-00#openssh.com, want-reply=false] failed (SshException) to process: EdDSA provider not supported
2023-02-07T07:55:31.958+10:00 DEBUG 9752 --- [ scheduling-1] o.s.retry.support.RetryTemplate : Checking for rethrow: count=4
2023-02-07T07:55:31.958+10:00 DEBUG 9752 --- [ scheduling-1] o.s.retry.support.RetryTemplate : Retry: count=4
2023-02-07T07:55:31.979+10:00 WARN 9752 --- [-nio2-thread-10] o.a.s.c.k.AcceptAllServerKeyVerifier : Server at localhost/127.0.0.1:22 presented unverified RSA key: SHA256:kQ7FBDG5+ZBmGHEsYW4RQmhmKHptYsN0MsAeQ7Cx1So
2023-02-07T07:55:32.072+10:00 WARN 9752 --- [-nio2-thread-13] o.a.s.c.session.ClientConnectionService : globalRequest(ClientConnectionService[ClientSessionImpl[admin#localhost/127.0.0.1:22]])[hostkeys-00#openssh.com, want-reply=false] failed (SshException) to process: EdDSA provider not supported
2023-02-07T07:55:32.315+10:00 DEBUG 9752 --- [ scheduling-1] o.s.retry.support.RetryTemplate : Checking for rethrow: count=5
2023-02-07T07:55:32.315+10:00 DEBUG 9752 --- [ scheduling-1] o.s.retry.support.RetryTemplate : Retry failed last attempt: count=5
2023-02-07T07:55:32.318+10:00 ERROR 9752 --- [ scheduling-1] o.s.integration.handler.LoggingHandler : org.springframework.messaging.MessagingException: Failed to handle, failedMessage=GenericMessage [payload=\tmp\sftp-inbound\sftpToSftpData\SampleIntegrationFile.txt, headers={file_remoteHostPort=localhost:22, file_name=SampleIntegrationFile.txt, file_remoteDirectory=//sftp/Outgoing/sftpToSftp/, remote-target-dir=/sftp/Incoming/sftpToSftp/, file_originalFile=\tmp\sftp-inbound\sftpToSftpData\SampleIntegrationFile.txt, id=cd33b5fa-4fab-93c3-d46a-5955f858ce0a, file_relativePath=\tmp\sftp-inbound\sftpToSftpData\SampleIntegrationFile.txt, file_remoteFile=SampleIntegrationFile.txt, timestamp=1675720518854}]
at org.springframework.integration.handler.advice.AbstractRequestHandlerAdvice$CallbackImpl.cloneAndExecute(AbstractRequestHandlerAdvice.java:175)
at org.springframework.integration.handler.advice.RequestHandlerRetryAdvice.lambda$doInvoke$1(RequestHandlerRetryAdvice.java:86)
at org.springframework.retry.support.RetryTemplate.doExecute(RetryTemplate.java:329)
at org.springframework.retry.support.RetryTemplate.execute(RetryTemplate.java:255)
at org.springframework.integration.handler.advice.RequestHandlerRetryAdvice.doInvoke(RequestHandlerRetryAdvice.java:86)
at org.springframework.integration.handler.advice.AbstractRequestHandlerAdvice.invoke(AbstractRequestHandlerAdvice.java:67)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:218)
at jdk.proxy3/jdk.proxy3.$Proxy60.handleRequestMessage(Unknown Source)
at org.springframework.integration.handler.AbstractReplyProducingMessageHandler.doInvokeAdvisedRequestHandler(AbstractReplyProducingMessageHandler.java:155)
at org.springframework.integration.handler.AbstractReplyProducingMessageHandler.handleMessageInternal(AbstractReplyProducingMessageHandler.java:139)
at org.springframework.integration.handler.AbstractMessageHandler.doHandleMessage(AbstractMessageHandler.java:105)
at org.springframework.integration.handler.AbstractMessageHandler.handleMessage(AbstractMessageHandler.java:73)
at org.springframework.integration.dispatcher.AbstractDispatcher.tryOptimizedDispatch(AbstractDispatcher.java:115)
at org.springframework.integration.dispatcher.UnicastingDispatcher.doDispatch(UnicastingDispatcher.java:133)
at org.springframework.integration.dispatcher.UnicastingDispatcher.dispatch(UnicastingDispatcher.java:106)
at org.springframework.integration.channel.AbstractSubscribableChannel.doSend(AbstractSubscribableChannel.java:72)
at org.springframework.integration.channel.AbstractMessageChannel.sendInternal(AbstractMessageChannel.java:373)
at org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:327)
at org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:297)
at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:187)
at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:166)
at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:47)
at org.springframework.messaging.core.AbstractMessageSendingTemplate.send(AbstractMessageSendingTemplate.java:109)
at org.springframework.integration.handler.AbstractMessageProducingHandler.sendOutput(AbstractMessageProducingHandler.java:484)
at org.springframework.integration.handler.AbstractMessageProducingHandler.doProduceOutput(AbstractMessageProducingHandler.java:339)
at org.springframework.integration.handler.AbstractMessageProducingHandler.produceOutput(AbstractMessageProducingHandler.java:268)
at org.springframework.integration.handler.AbstractMessageProducingHandler.sendOutputs(AbstractMessageProducingHandler.java:232)
at org.springframework.integration.handler.AbstractReplyProducingMessageHandler.handleMessageInternal(AbstractReplyProducingMessageHandler.java:142)
at org.springframework.integration.handler.AbstractMessageHandler.doHandleMessage(AbstractMessageHandler.java:105)
at org.springframework.integration.handler.AbstractMessageHandler.handleMessage(AbstractMessageHandler.java:73)
at org.springframework.integration.dispatcher.AbstractDispatcher.tryOptimizedDispatch(AbstractDispatcher.java:115)
at org.springframework.integration.dispatcher.UnicastingDispatcher.doDispatch(UnicastingDispatcher.java:133)
at org.springframework.integration.dispatcher.UnicastingDispatcher.dispatch(UnicastingDispatcher.java:106)
at org.springframework.integration.channel.AbstractSubscribableChannel.doSend(AbstractSubscribableChannel.java:72)
at org.springframework.integration.channel.AbstractMessageChannel.sendInternal(AbstractMessageChannel.java:373)
at org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:327)
at org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:297)
at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:187)
at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:166)
at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:47)
at org.springframework.messaging.core.AbstractMessageSendingTemplate.send(AbstractMessageSendingTemplate.java:109)
at org.springframework.integration.endpoint.SourcePollingChannelAdapter.handleMessage(SourcePollingChannelAdapter.java:196)
at org.springframework.integration.endpoint.AbstractPollingEndpoint.messageReceived(AbstractPollingEndpoint.java:474)
at org.springframework.integration.endpoint.AbstractPollingEndpoint.doPoll(AbstractPollingEndpoint.java:460)
at org.springframework.integration.endpoint.AbstractPollingEndpoint.pollForMessage(AbstractPollingEndpoint.java:412)
at org.springframework.integration.endpoint.AbstractPollingEndpoint.lambda$createPoller$4(AbstractPollingEndpoint.java:348)
at org.springframework.integration.util.ErrorHandlingTaskExecutor.lambda$execute$0(ErrorHandlingTaskExecutor.java:57)
at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:50)
at org.springframework.integration.util.ErrorHandlingTaskExecutor.execute(ErrorHandlingTaskExecutor.java:55)
at org.springframework.integration.endpoint.AbstractPollingEndpoint.lambda$createPoller$5(AbstractPollingEndpoint.java:341)
at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54)
at org.springframework.scheduling.concurrent.ReschedulingRunnable.run(ReschedulingRunnable.java:96)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: org.springframework.messaging.MessageDeliveryException: Error handling message for file [C:\tmp\sftp-inbound\sftpToSftpData\SampleIntegrationFile.txt -> SampleIntegrationFile.txt], failedMessage=GenericMessage [payload=\tmp\sftp-inbound\sftpToSftpData\SampleIntegrationFile.txt, headers={file_remoteHostPort=localhost:22, file_name=SampleIntegrationFile.txt, file_remoteDirectory=//sftp/Outgoing/sftpToSftp/, remote-target-dir=/sftp/Incoming/sftpToSftp/, file_originalFile=\tmp\sftp-inbound\sftpToSftpData\SampleIntegrationFile.txt, id=cd33b5fa-4fab-93c3-d46a-5955f858ce0a, file_relativePath=\tmp\sftp-inbound\sftpToSftpData\SampleIntegrationFile.txt, file_remoteFile=SampleIntegrationFile.txt, timestamp=1675720518854}]
at org.springframework.integration.file.remote.RemoteFileTemplate.doSend(RemoteFileTemplate.java:367)
at org.springframework.integration.file.remote.RemoteFileTemplate.lambda$send$0(RemoteFileTemplate.java:314)
at org.springframework.integration.file.remote.RemoteFileTemplate.execute(RemoteFileTemplate.java:452)
at org.springframework.integration.file.remote.RemoteFileTemplate.send(RemoteFileTemplate.java:314)
at org.springframework.integration.file.remote.RemoteFileTemplate.send(RemoteFileTemplate.java:302)
at org.springframework.integration.file.remote.gateway.AbstractRemoteFileOutboundGateway.put(AbstractRemoteFileOutboundGateway.java:831)
at org.springframework.integration.file.remote.gateway.AbstractRemoteFileOutboundGateway.lambda$doPut$13(AbstractRemoteFileOutboundGateway.java:817)
at org.springframework.integration.file.remote.RemoteFileTemplate.invoke(RemoteFileTemplate.java:484)
at org.springframework.integration.file.remote.gateway.AbstractRemoteFileOutboundGateway.doPut(AbstractRemoteFileOutboundGateway.java:816)
at org.springframework.integration.file.remote.gateway.AbstractRemoteFileOutboundGateway.doPut(AbstractRemoteFileOutboundGateway.java:812)
at org.springframework.integration.file.remote.gateway.AbstractRemoteFileOutboundGateway.handleRequestMessage(AbstractRemoteFileOutboundGateway.java:589)
at org.springframework.integration.handler.AbstractReplyProducingMessageHandler$AdvisedRequestHandler.handleRequestMessage(AbstractReplyProducingMessageHandler.java:208)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:568)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:196)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at org.springframework.integration.handler.advice.AbstractRequestHandlerAdvice$CallbackImpl.cloneAndExecute(AbstractRequestHandlerAdvice.java:166)
... 58 more
Caused by: org.springframework.messaging.MessagingException: Failed to write to '/sftp/Incoming/sftpToSftp/SampleIntegrationFile.txt.temp' while uploading the file
at org.springframework.integration.file.remote.RemoteFileTemplate.sendFileToRemoteDirectory(RemoteFileTemplate.java:573)
at org.springframework.integration.file.remote.RemoteFileTemplate.doSend(RemoteFileTemplate.java:353)
... 77 more
Caused by: SFTP error (SSH_FX_FAILURE): Failure
at org.apache.sshd.sftp.client.impl.AbstractSftpClient.throwStatusException(AbstractSftpClient.java:217)
at org.apache.sshd.sftp.client.impl.AbstractSftpClient.checkResponseStatus(AbstractSftpClient.java:212)
at org.apache.sshd.sftp.client.impl.AbstractSftpClient.checkResponseStatus(AbstractSftpClient.java:186)
at org.apache.sshd.sftp.client.impl.AbstractSftpClient.checkCommandStatus(AbstractSftpClient.java:164)
at org.apache.sshd.sftp.client.impl.AbstractSftpClient.rename(AbstractSftpClient.java:647)
at org.apache.sshd.sftp.client.SftpClient.rename(SftpClient.java:617)
at org.springframework.integration.sftp.session.SftpSession.rename(SftpSession.java:162)
at org.springframework.integration.file.remote.session.CachingSessionFactory$CachedSession.rename(CachingSessionFactory.java:252)
at org.springframework.integration.file.remote.RemoteFileTemplate.doSend(RemoteFileTemplate.java:606)
at org.springframework.integration.file.remote.RemoteFileTemplate.sendFileToRemoteDirectory(RemoteFileTemplate.java:570)
My Atmoz dockercompose file
I'm using Spring Cloud Streams and the default Spring Retry mechanism only by using properties.
I currently have an application which retries correctly (up to the max retries) when MyCustomException is thrown in my listener. Then, it is sent to the DLQ - this piece is working as expected with the config below.
Question is: I'd like to get the other "non-retriable exceptions" to the DLQ as well (for instance, when we throw AnotherCustomException or even a RuntimeException).
Right now they are not being retried (and this is correct and working as expected) but the issue for my case is that they are not being redirected to myDeadLetterQueue.
The config used in the properties file are the following.
spring.cloud.stream.kafka.bindings.input.consumer.enableDlq = true
spring.cloud.stream.kafka.bindings.input.consumer.dlq-name = myDeadLetterQueue
spring.cloud.stream.bindings.input.consumer.defaultRetryable = false
spring.cloud.stream.bindings.input.consumer.retryableExceptions.com.packagename.common.kafka.exception.MyCustomException = true
Is there any other configuration missing for this scenario?
but the issue for my case is that they are not being redirected to myDeadLetterQueue.
That doesn't make sense; it's how it works; non-retyable exceptions are sent right to the DLQ topic without being retried; I just tested it without any problems:
#SpringBootApplication
#EnableBinding(Sink.class)
public class So58390786Application {
public static void main(String[] args) {
SpringApplication.run(So58390786Application.class, args);
}
#StreamListener(Sink.INPUT)
public void listen(String in) {
System.out.println(in);
if ("state".equals(in)) {
throw new IllegalStateException("ise");
}
throw new IllegalArgumentException("iae");
}
#Bean
public ApplicationRunner runner(KafkaTemplate<byte[], byte[]> template) {
return args -> {
template.send("so58390786", "state".getBytes());
template.send("so58390786", "arg".getBytes());
template.send("so58390786", "state".getBytes());
};
}
#KafkaListener(id = "so58390786slq", topics = "so58390786dlq")
public void listenDlq(String in) {
System.out.println("dlq:" + in);
}
}
and
spring.cloud.stream.bindings.input.destination=so58390786
spring.cloud.stream.bindings.input.group=so58390786
spring.cloud.stream.kafka.bindings.input.consumer.enableDlq=true
spring.cloud.stream.kafka.bindings.input.consumer.dlq-name=so58390786dlq
spring.cloud.stream.bindings.input.consumer.defaultRetryable = false
spring.cloud.stream.bindings.input.consumer.retryable-exceptions.java.lang.IllegalStateException=true
logging.level.root=warn
spring.kafka.consumer.auto-offset-reset=earliest
and
state
state
state
2019-10-15 09:22:46.765 ERROR 57129 --- [container-0-C-1] o.s.integration.handler.LoggingHandler : org.springframework.messaging.MessagingException: Exception thrown while invoking So58390786Application#listen[1 args]; nested exception is java.lang.IllegalStateException: ise, failedMessage=GenericMessage [payload=byte[5], headers={kafka_offset=12, kafka_consumer=org.apache.kafka.clients.consumer.KafkaConsumer#5c959af1, deliveryAttempt=3, kafka_timestampType=CREATE_TIME, kafka_receivedMessageKey=null, kafka_receivedPartitionId=0, kafka_receivedTopic=so58390786, kafka_receivedTimestamp=1571145763737, contentType=application/json, kafka_groupId=so58390786}]
at org.springframework.cloud.stream.binding.StreamListenerMessageHandler.handleRequestMessage(StreamListenerMessageHandler.java:64)
at org.springframework.integration.handler.AbstractReplyProducingMessageHandler.handleMessageInternal(AbstractReplyProducingMessageHandler.java:127)
at org.springframework.integration.handler.AbstractMessageHandler.handleMessage(AbstractMessageHandler.java:177)
at org.springframework.integration.dispatcher.AbstractDispatcher.tryOptimizedDispatch(AbstractDispatcher.java:115)
at org.springframework.integration.dispatcher.UnicastingDispatcher.doDispatch(UnicastingDispatcher.java:133)
at org.springframework.integration.dispatcher.UnicastingDispatcher.dispatch(UnicastingDispatcher.java:106)
at org.springframework.integration.channel.AbstractSubscribableChannel.doSend(AbstractSubscribableChannel.java:73)
at org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:453)
at org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:403)
at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:187)
at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:166)
at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:47)
at org.springframework.messaging.core.AbstractMessageSendingTemplate.send(AbstractMessageSendingTemplate.java:109)
at org.springframework.integration.endpoint.MessageProducerSupport.sendMessage(MessageProducerSupport.java:198)
at org.springframework.integration.kafka.inbound.KafkaMessageDrivenChannelAdapter.sendMessageIfAny(KafkaMessageDrivenChannelAdapter.java:384)
at org.springframework.integration.kafka.inbound.KafkaMessageDrivenChannelAdapter.access$300(KafkaMessageDrivenChannelAdapter.java:75)
at org.springframework.integration.kafka.inbound.KafkaMessageDrivenChannelAdapter$IntegrationRecordMessageListener.onMessage(KafkaMessageDrivenChannelAdapter.java:443)
at org.springframework.integration.kafka.inbound.KafkaMessageDrivenChannelAdapter$IntegrationRecordMessageListener.onMessage(KafkaMessageDrivenChannelAdapter.java:417)
at org.springframework.kafka.listener.adapter.RetryingMessageListenerAdapter.lambda$onMessage$0(RetryingMessageListenerAdapter.java:120)
at org.springframework.retry.support.RetryTemplate.doExecute(RetryTemplate.java:287)
at org.springframework.retry.support.RetryTemplate.execute(RetryTemplate.java:211)
at org.springframework.kafka.listener.adapter.RetryingMessageListenerAdapter.onMessage(RetryingMessageListenerAdapter.java:114)
at org.springframework.kafka.listener.adapter.RetryingMessageListenerAdapter.onMessage(RetryingMessageListenerAdapter.java:40)
at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.doInvokeOnMessage(KafkaMessageListenerContainer.java:1592)
at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.invokeOnMessage(KafkaMessageListenerContainer.java:1575)
at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.doInvokeRecordListener(KafkaMessageListenerContainer.java:1534)
at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.doInvokeWithRecords(KafkaMessageListenerContainer.java:1487)
at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.invokeRecordListener(KafkaMessageListenerContainer.java:1401)
at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.invokeListener(KafkaMessageListenerContainer.java:1165)
at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.pollAndInvoke(KafkaMessageListenerContainer.java:949)
at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.run(KafkaMessageListenerContainer.java:884)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.IllegalStateException: ise
at com.example.demo.So58390786Application.listen(So58390786Application.java:25)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.messaging.handler.invocation.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:171)
at org.springframework.messaging.handler.invocation.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:120)
at org.springframework.cloud.stream.binding.StreamListenerMessageHandler.handleRequestMessage(StreamListenerMessageHandler.java:55)
... 33 more
arg
2019-10-15 09:22:46.776 ERROR 57129 --- [container-0-C-1] o.s.integration.handler.LoggingHandler : org.springframework.messaging.MessagingException: Exception thrown while invoking So58390786Application#listen[1 args]; nested exception is java.lang.IllegalArgumentException: iae, failedMessage=GenericMessage [payload=byte[3], headers={kafka_offset=13, kafka_consumer=org.apache.kafka.clients.consumer.KafkaConsumer#5c959af1, deliveryAttempt=1, kafka_timestampType=CREATE_TIME, kafka_receivedMessageKey=null, kafka_receivedPartitionId=0, kafka_receivedTopic=so58390786, kafka_receivedTimestamp=1571145763741, contentType=application/json, kafka_groupId=so58390786}]
at org.springframework.cloud.stream.binding.StreamListenerMessageHandler.handleRequestMessage(StreamListenerMessageHandler.java:64)
at org.springframework.integration.handler.AbstractReplyProducingMessageHandler.handleMessageInternal(AbstractReplyProducingMessageHandler.java:127)
at org.springframework.integration.handler.AbstractMessageHandler.handleMessage(AbstractMessageHandler.java:177)
at org.springframework.integration.dispatcher.AbstractDispatcher.tryOptimizedDispatch(AbstractDispatcher.java:115)
at org.springframework.integration.dispatcher.UnicastingDispatcher.doDispatch(UnicastingDispatcher.java:133)
at org.springframework.integration.dispatcher.UnicastingDispatcher.dispatch(UnicastingDispatcher.java:106)
at org.springframework.integration.channel.AbstractSubscribableChannel.doSend(AbstractSubscribableChannel.java:73)
at org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:453)
at org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:403)
at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:187)
at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:166)
at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:47)
at org.springframework.messaging.core.AbstractMessageSendingTemplate.send(AbstractMessageSendingTemplate.java:109)
at org.springframework.integration.endpoint.MessageProducerSupport.sendMessage(MessageProducerSupport.java:198)
at org.springframework.integration.kafka.inbound.KafkaMessageDrivenChannelAdapter.sendMessageIfAny(KafkaMessageDrivenChannelAdapter.java:384)
at org.springframework.integration.kafka.inbound.KafkaMessageDrivenChannelAdapter.access$300(KafkaMessageDrivenChannelAdapter.java:75)
at org.springframework.integration.kafka.inbound.KafkaMessageDrivenChannelAdapter$IntegrationRecordMessageListener.onMessage(KafkaMessageDrivenChannelAdapter.java:443)
at org.springframework.integration.kafka.inbound.KafkaMessageDrivenChannelAdapter$IntegrationRecordMessageListener.onMessage(KafkaMessageDrivenChannelAdapter.java:417)
at org.springframework.kafka.listener.adapter.RetryingMessageListenerAdapter.lambda$onMessage$0(RetryingMessageListenerAdapter.java:120)
at org.springframework.retry.support.RetryTemplate.doExecute(RetryTemplate.java:287)
at org.springframework.retry.support.RetryTemplate.execute(RetryTemplate.java:211)
at org.springframework.kafka.listener.adapter.RetryingMessageListenerAdapter.onMessage(RetryingMessageListenerAdapter.java:114)
at org.springframework.kafka.listener.adapter.RetryingMessageListenerAdapter.onMessage(RetryingMessageListenerAdapter.java:40)
at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.doInvokeOnMessage(KafkaMessageListenerContainer.java:1592)
at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.invokeOnMessage(KafkaMessageListenerContainer.java:1575)
at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.doInvokeRecordListener(KafkaMessageListenerContainer.java:1534)
at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.doInvokeWithRecords(KafkaMessageListenerContainer.java:1487)
at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.invokeRecordListener(KafkaMessageListenerContainer.java:1401)
at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.invokeListener(KafkaMessageListenerContainer.java:1165)
at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.pollAndInvoke(KafkaMessageListenerContainer.java:949)
at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.run(KafkaMessageListenerContainer.java:884)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.IllegalArgumentException: iae
at com.example.demo.So58390786Application.listen(So58390786Application.java:27)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.messaging.handler.invocation.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:171)
at org.springframework.messaging.handler.invocation.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:120)
at org.springframework.cloud.stream.binding.StreamListenerMessageHandler.handleRequestMessage(StreamListenerMessageHandler.java:55)
... 33 more
state
dlq:state
dlq:arg
state
state
2019-10-15 09:22:49.784 ERROR 57129 --- [container-0-C-1] o.s.integration.handler.LoggingHandler : org.springframework.messaging.MessagingException: Exception thrown while invoking So58390786Application#listen[1 args]; nested exception is java.lang.IllegalStateException: ise, failedMessage=GenericMessage [payload=byte[5], headers={kafka_offset=14, kafka_consumer=org.apache.kafka.clients.consumer.KafkaConsumer#5c959af1, deliveryAttempt=3, kafka_timestampType=CREATE_TIME, kafka_receivedMessageKey=null, kafka_receivedPartitionId=0, kafka_receivedTopic=so58390786, kafka_receivedTimestamp=1571145763741, contentType=application/json, kafka_groupId=so58390786}]
at org.springframework.cloud.stream.binding.StreamListenerMessageHandler.handleRequestMessage(StreamListenerMessageHandler.java:64)
at org.springframework.integration.handler.AbstractReplyProducingMessageHandler.handleMessageInternal(AbstractReplyProducingMessageHandler.java:127)
at org.springframework.integration.handler.AbstractMessageHandler.handleMessage(AbstractMessageHandler.java:177)
at org.springframework.integration.dispatcher.AbstractDispatcher.tryOptimizedDispatch(AbstractDispatcher.java:115)
at org.springframework.integration.dispatcher.UnicastingDispatcher.doDispatch(UnicastingDispatcher.java:133)
at org.springframework.integration.dispatcher.UnicastingDispatcher.dispatch(UnicastingDispatcher.java:106)
at org.springframework.integration.channel.AbstractSubscribableChannel.doSend(AbstractSubscribableChannel.java:73)
at org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:453)
at org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:403)
at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:187)
at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:166)
at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:47)
at org.springframework.messaging.core.AbstractMessageSendingTemplate.send(AbstractMessageSendingTemplate.java:109)
at org.springframework.integration.endpoint.MessageProducerSupport.sendMessage(MessageProducerSupport.java:198)
at org.springframework.integration.kafka.inbound.KafkaMessageDrivenChannelAdapter.sendMessageIfAny(KafkaMessageDrivenChannelAdapter.java:384)
at org.springframework.integration.kafka.inbound.KafkaMessageDrivenChannelAdapter.access$300(KafkaMessageDrivenChannelAdapter.java:75)
at org.springframework.integration.kafka.inbound.KafkaMessageDrivenChannelAdapter$IntegrationRecordMessageListener.onMessage(KafkaMessageDrivenChannelAdapter.java:443)
at org.springframework.integration.kafka.inbound.KafkaMessageDrivenChannelAdapter$IntegrationRecordMessageListener.onMessage(KafkaMessageDrivenChannelAdapter.java:417)
at org.springframework.kafka.listener.adapter.RetryingMessageListenerAdapter.lambda$onMessage$0(RetryingMessageListenerAdapter.java:120)
at org.springframework.retry.support.RetryTemplate.doExecute(RetryTemplate.java:287)
at org.springframework.retry.support.RetryTemplate.execute(RetryTemplate.java:211)
at org.springframework.kafka.listener.adapter.RetryingMessageListenerAdapter.onMessage(RetryingMessageListenerAdapter.java:114)
at org.springframework.kafka.listener.adapter.RetryingMessageListenerAdapter.onMessage(RetryingMessageListenerAdapter.java:40)
at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.doInvokeOnMessage(KafkaMessageListenerContainer.java:1592)
at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.invokeOnMessage(KafkaMessageListenerContainer.java:1575)
at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.doInvokeRecordListener(KafkaMessageListenerContainer.java:1534)
at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.doInvokeWithRecords(KafkaMessageListenerContainer.java:1487)
at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.invokeRecordListener(KafkaMessageListenerContainer.java:1401)
at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.invokeListener(KafkaMessageListenerContainer.java:1165)
at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.pollAndInvoke(KafkaMessageListenerContainer.java:949)
at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.run(KafkaMessageListenerContainer.java:884)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.IllegalStateException: ise
at com.example.demo.So58390786Application.listen(So58390786Application.java:25)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.messaging.handler.invocation.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:171)
at org.springframework.messaging.handler.invocation.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:120)
at org.springframework.cloud.stream.binding.StreamListenerMessageHandler.handleRequestMessage(StreamListenerMessageHandler.java:55)
... 33 more
dlq:state
EDIT
Turn on debug logging for retry to see what you see;
logging.level.org.springframework.retry=debug
I get this...
2019-10-15 09:35:38.856 DEBUG 60391 --- [ main] o.s.retry.support.RetryTemplate : Retry: count=0
2019-10-15 09:35:38.967 DEBUG 60391 --- [container-0-C-1] o.s.retry.support.RetryTemplate : Retry: count=0
state
2019-10-15 09:35:38.973 DEBUG 60391 --- [container-0-C-1] o.s.r.backoff.ExponentialBackOffPolicy : Sleeping for 1000
2019-10-15 09:35:39.977 DEBUG 60391 --- [container-0-C-1] o.s.retry.support.RetryTemplate : Checking for rethrow: count=1
2019-10-15 09:35:39.978 DEBUG 60391 --- [container-0-C-1] o.s.retry.support.RetryTemplate : Retry: count=1
state
2019-10-15 09:35:39.979 DEBUG 60391 --- [container-0-C-1] o.s.r.backoff.ExponentialBackOffPolicy : Sleeping for 2000
2019-10-15 09:35:41.982 DEBUG 60391 --- [container-0-C-1] o.s.retry.support.RetryTemplate : Checking for rethrow: count=2
2019-10-15 09:35:41.982 DEBUG 60391 --- [container-0-C-1] o.s.retry.support.RetryTemplate : Retry: count=2
state
2019-10-15 09:35:41.983 DEBUG 60391 --- [container-0-C-1] o.s.retry.support.RetryTemplate : Checking for rethrow: count=3
2019-10-15 09:35:41.983 DEBUG 60391 --- [container-0-C-1] o.s.retry.support.RetryTemplate : Retry failed last attempt: count=3
...
2019-10-15 09:35:41.994 DEBUG 60391 --- [container-0-C-1] o.s.retry.support.RetryTemplate : Retry: count=0
arg
2019-10-15 09:35:41.994 DEBUG 60391 --- [container-0-C-1] o.s.retry.support.RetryTemplate : Checking for rethrow: count=1
m2019-10-15 09:35:41.994 DEBUG 60391 --- [container-0-C-1] o.s.retry.support.RetryTemplate : Retry failed last attempt: count=1
for our integration-test profile we want to disable Spring's automatic reconnecting to fail fast.
Attempting to connect to: [rabbitmq-invalid-host:5672]
Error creating consumer; retrying in 5000
org.springframework.amqp.AmqpConnectException: org.springframework.amqp.AmqpIOException: java.net.UnknownHostException: rabbitmq-invalid-host
the following settings do not have any effect yet:
spring.rabbitmq.connection-timeout=1000
spring.rabbitmq.requested-heartbeat=1
spring.rabbitmq.listener.direct.missing-queues-fatal=true
spring.rabbitmq.listener.direct.retry.enabled=false
spring.rabbitmq.template.retry.enabled=false
any idea how to make amqp give up or even destroy the whole app-context?
full stacktrace:
2019-06-11 14:56:31.596 INFO 3025 --- [ main] o.s.a.r.c.CachingConnectionFactory : Attempting to connect to: [rabbitmq-invalid-host:5672]
2019-06-11 14:56:31.698 INFO 3025 --- [ main] o.s.a.r.c.CachingConnectionFactory : Attempting to connect to: [rabbitmq-invalid-host:5672]
2019-06-11 14:56:31.902 INFO 3025 --- [ main] o.s.a.r.c.CachingConnectionFactory : Attempting to connect to: [rabbitmq-invalid-host:5672]
2019-06-11 14:56:32.305 INFO 3025 --- [ main] o.s.a.r.c.CachingConnectionFactory : Attempting to connect to: [rabbitmq-invalid-host:5672]
2019-06-11 14:56:33.108 INFO 3025 --- [ main] o.s.a.r.c.CachingConnectionFactory : Attempting to connect to: [rabbitmq-invalid-host:5672]
2019-06-11 14:56:34.709 INFO 3025 --- [ main] o.s.a.r.c.CachingConnectionFactory : Attempting to connect to: [rabbitmq-invalid-host:5672]
2019-06-11 14:56:36.476 INFO 3025 --- [ntContainer#0-1] o.s.a.r.c.CachingConnectionFactory : Attempting to connect to: [rabbitmq-invalid-host:5672]
2019-06-11 14:56:36.477 ERROR 3025 --- [ntContainer#0-1] o.s.a.r.l.DirectMessageListenerContainer : Error creating consumer; retrying in 5000
org.springframework.amqp.AmqpConnectException: org.springframework.amqp.AmqpIOException: java.net.UnknownHostException: rabbitmq-invalid-host
at org.springframework.amqp.rabbit.listener.DirectMessageListenerContainer.doConsumeFromQueue(DirectMessageListenerContainer.java:667) [spring-rabbit-2.1.6.RELEASE.jar:2.1.6.RELEASE]
at org.springframework.amqp.rabbit.listener.DirectMessageListenerContainer.consumeFromQueue(DirectMessageListenerContainer.java:645) [spring-rabbit-2.1.6.RELEASE.jar:2.1.6.RELEASE]
at org.springframework.amqp.rabbit.listener.DirectMessageListenerContainer.startConsumers(DirectMessageListenerContainer.java:550) [spring-rabbit-2.1.6.RELEASE.jar:2.1.6.RELEASE]
at org.springframework.amqp.rabbit.listener.DirectMessageListenerContainer.lambda$actualStart$1(DirectMessageListenerContainer.java:410) [spring-rabbit-2.1.6.RELEASE.jar:2.1.6.RELEASE]
at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_212]
Caused by: org.springframework.amqp.AmqpIOException: java.net.UnknownHostException: rabbitmq-invalid-host
at org.springframework.amqp.rabbit.support.RabbitExceptionTranslator.convertRabbitAccessException(RabbitExceptionTranslator.java:71) ~[spring-rabbit-2.1.6.RELEASE.jar:2.1.6.RELEASE]
at org.springframework.amqp.rabbit.connection.AbstractConnectionFactory.createBareConnection(AbstractConnectionFactory.java:509) ~[spring-rabbit-2.1.6.RELEASE.jar:2.1.6.RELEASE]
at org.springframework.amqp.rabbit.connection.CachingConnectionFactory.createConnection(CachingConnectionFactory.java:700) ~[spring-rabbit-2.1.6.RELEASE.jar:2.1.6.RELEASE]
at org.springframework.amqp.rabbit.listener.DirectMessageListenerContainer.doConsumeFromQueue(DirectMessageListenerContainer.java:663) [spring-rabbit-2.1.6.RELEASE.jar:2.1.6.RELEASE]
... 4 common frames omitted
Caused by: java.net.UnknownHostException: rabbitmq-invalid-host
at java.net.InetAddress.getAllByName0(InetAddress.java:1281) ~[na:1.8.0_212]
at java.net.InetAddress.getAllByName(InetAddress.java:1193) ~[na:1.8.0_212]
at java.net.InetAddress.getAllByName(InetAddress.java:1127) ~[na:1.8.0_212]
at com.rabbitmq.client.DnsRecordIpAddressResolver.resolveIpAddresses(DnsRecordIpAddressResolver.java:83) ~[amqp-client-5.4.3.jar:5.4.3]
at com.rabbitmq.client.DnsRecordIpAddressResolver.getAddresses(DnsRecordIpAddressResolver.java:73) ~[amqp-client-5.4.3.jar:5.4.3]
at com.rabbitmq.client.ConnectionFactory.newConnection(ConnectionFactory.java:1098) ~[amqp-client-5.4.3.jar:5.4.3]
at com.rabbitmq.client.ConnectionFactory.newConnection(ConnectionFactory.java:1054) ~[amqp-client-5.4.3.jar:5.4.3]
at com.rabbitmq.client.ConnectionFactory.newConnection(ConnectionFactory.java:994) ~[amqp-client-5.4.3.jar:5.4.3]
at org.springframework.amqp.rabbit.connection.AbstractConnectionFactory.createBareConnection(AbstractConnectionFactory.java:462) ~[spring-rabbit-2.1.6.RELEASE.jar:2.1.6.RELEASE]
... 6 common frames omitted
2019-06-11 14:56:37.912 INFO 3025 --- [ main] o.s.a.r.c.CachingConnectionFactory : Attempting to connect to: [rabbitmq-invalid-host:5672]
You can listen for the exception in an event listener...
#SpringBootApplication
public class So56270299Application {
public static void main(String[] args) {
SpringApplication.run(So56270299Application.class, args);
}
#EventListener
public void events(ListenerContainerConsumerFailedEvent event) {
System.out.println(event);
if (event.getThrowable().getCause() instanceof UnknownHostException) {
System.exit(1);
}
}
#RabbitListener(queues = "so56270299")
public void listen(String in) {
}
}
and
spring.rabbitmq.host=junk
and
2019-06-11 09:15:09.205 INFO 96620 --- [ntContainer#0-1] o.s.a.r.c.CachingConnectionFactory : Attempting to connect to: [junk:5672]
2019-06-11 09:15:09.212 INFO 96620 --- [ main] com.example.So56270299Application : Started So56270299Application in 0.704 seconds (JVM running for 1.071)
ListenerContainerConsumerFailedEvent [reason=Consumer raised exception, attempting restart, fatal=false, throwable=org.springframework.amqp.AmqpIOException: java.net.UnknownHostException: junk, container=SimpleMessageListenerContainer [concurrentConsumers=1, queueNames=[so56270299]]]
2019-06-11 09:15:14.255 INFO 96620 --- [ Thread-2] o.s.a.r.l.SimpleMessageListenerContainer : Waiting for workers to finish.
2019-06-11 09:15:14.255 INFO 96620 --- [ Thread-2] o.s.a.r.l.SimpleMessageListenerContainer : Successfully waited for workers to finish.
2019-06-11 09:15:14.257 INFO 96620 --- [ Thread-2] o.s.a.r.l.SimpleMessageListenerContainer : Shutdown ignored - container is not active already
Summary: Job restarts/retries even when reader/processor/writer succeed.
My step is defined as follows:
return stepBuilder.get("job.transaction-export.step1")
// .startLimit(stepStartLimit)
.<AfxEntity, AfxEntity> chunk(chunkSize)
.reader(reader)
.processor(processor)
.writer(writer)
// .faultTolerant()
// .backOffPolicy(exponentialRandomBackOffPolicy)
// .retry(HttpServerErrorException.class)
// .retry(UnknownHttpStatusCodeException.class)
// .retry(ResourceAccessException.class)
// .noRetry(HttpClientErrorException.class)
.build();
At some point I want certain HTTP errors to trigger a retry, but at this point, all of that code is commented out. I just have simple classes implementing the Item* interfaces, and a chunk size of 1.
I have done nothing that I can tell to cause the repeat policy to be more than once.
My application is a Spring Boot application that has several JMS listeners. When a listener gets a message, it dispatches the appropriate Job with the message contents.
This job is an export. The JMS message provides the ID of the record to be exported. The ItemReader (annotated with #StepScope) pulls the record into a Map. The ItemProcessor adds additional data to the Map. The ItemWriter does an HTTP POST with Basic Auth with the data gathered previously.
The test class, in a separate process, registers REST listener and posts a JMS message. When the REST method gets the message it exits. This means success. (Test needs to be enhanced.)
What is happening right now is that, in spite of a successful (no exceptions/errors, auth passes, data is exchanged both ways) REST transaction, the Batch framework repeats the step. Since the test class exits after responding to the REST request, the new POSTs fail. I let it run once and it went to over 120 retries.
The Question: In case of success, how do I keep the retry/repeat mechanism from being triggered?
The logs:
2016-03-03 14:19:44.952 TRACE 26954 --- [enerContainer-1] c.a.a.e.t.TransactionMQListener : Received message ActiveMQTextMessage {message found in here} with listener com.abc.afx.exporter.transaction.TransactionMQListener$$EnhancerBySpringCGLIB$$11c2ee83#480f78d0.
2016-03-03 14:19:44.955 DEBUG 26954 --- [enerContainer-1] c.a.a.e.t.TransactionMQListener : jobParameters = {job parameters here}
2016-03-03 14:19:44.956 DEBUG 26954 --- [enerContainer-1] BatchConfiguration$ReferenceTargetSource : Initializing lazy target object
2016-03-03 14:19:45.847 INFO 26954 --- [enerContainer-1] o.s.b.c.l.support.SimpleJobLauncher : Job: [SimpleJob: [name=job.transaction-export]] launched with the following parameters: [{job parameters here, launchTime=1457032784898}]
2016-03-03 14:19:45.848 DEBUG 26954 --- [enerContainer-1] o.s.batch.core.job.AbstractJob : Job execution starting: JobExecution: id=371, version=0, startTime=null, endTime=null, lastUpdated=Thu Mar 03 14:19:45 EST 2016, status=STARTING, exitStatus=exitCode=UNKNOWN;exitDescription=, job=[JobInstance: id=370, version=0, Job=[job.transaction-export]], jobParameters=[{job parameters here, launchTime=1457032784898}]
2016-03-03 14:19:45.850 DEBUG 26954 --- [enerContainer-1] BatchConfiguration$ReferenceTargetSource : Initializing lazy target object
// step starts here
2016-03-03 14:19:46.658 INFO 26954 --- [enerContainer-1] o.s.batch.core.job.SimpleStepHandler : Executing step: [job.transaction-export.step1]
2016-03-03 14:19:46.658 DEBUG 26954 --- [enerContainer-1] o.s.batch.core.step.AbstractStep : Executing: id=372
2016-03-03 14:19:46.810 DEBUG 26954 --- [enerContainer-1] o.s.batch.core.scope.StepScope : Creating object in scope=step, name=scopedTarget.reader
2016-03-03 14:19:46.817 DEBUG 26954 --- [enerContainer-1] o.s.batch.core.scope.StepScope : Registered destruction callback in scope=step, name=scopedTarget.reader
2016-03-03 14:19:46.817 DEBUG 26954 --- [enerContainer-1] o.s.batch.core.scope.StepScope : Creating object in scope=step, name=scopedTarget.reader.transaction
2016-03-03 14:19:46.819 DEBUG 26954 --- [enerContainer-1] o.s.batch.core.scope.StepScope : Registered destruction callback in scope=step, name=scopedTarget.reader.transaction
2016-03-03 14:19:46.947 DEBUG 26954 --- [enerContainer-1] o.s.batch.repeat.support.RepeatTemplate : Starting repeat context.
2016-03-03 14:19:46.947 DEBUG 26954 --- [enerContainer-1] o.s.batch.repeat.support.RepeatTemplate : Repeat operation about to start at count=1
2016-03-03 14:19:46.948 DEBUG 26954 --- [enerContainer-1] o.s.b.c.s.c.StepContextRepeatCallback : Preparing chunk execution for StepContext: org.springframework.batch.core.scope.context.StepContext#d9db406
2016-03-03 14:19:46.948 DEBUG 26954 --- [enerContainer-1] o.s.b.c.s.c.StepContextRepeatCallback : Chunk execution starting: queue size=0
2016-03-03 14:19:46.950 DEBUG 26954 --- [enerContainer-1] BatchConfiguration$ReferenceTargetSource : Initializing lazy target object
2016-03-03 14:19:46.981 DEBUG 26954 --- [enerContainer-1] o.s.batch.repeat.support.RepeatTemplate : Starting repeat context.
2016-03-03 14:19:46.981 DEBUG 26954 --- [enerContainer-1] o.s.batch.repeat.support.RepeatTemplate : Repeat operation about to start at count=1
2016-03-03 14:19:46.992 TRACE 26954 --- [enerContainer-1] c.a.a.e.t.job.reader.TransactionReader : In TransactionReader.read
2016-03-03 14:19:47.034 TRACE 26954 --- [enerContainer-1] c.a.a.e.t.job.reader.TransactionReader : paymentTransaction = {map data in here}
2016-03-03 14:19:47.034 DEBUG 26954 --- [enerContainer-1] o.s.batch.repeat.support.RepeatTemplate : Repeat is complete according to policy and result value.
2016-03-03 14:19:47.035 TRACE 26954 --- [enerContainer-1] c.a.a.e.t.j.p.TransactionProcessor : In TransactionProcessor.process with paymentTransaction {map data in here}
2016-03-03 14:19:47.038 TRACE 26954 --- [enerContainer-1] c.a.a.e.t.j.p.TransactionProcessor : returnAfxEntity = {map data in here}
2016-03-03 14:19:47.038 DEBUG 26954 --- [enerContainer-1] o.s.batch.core.scope.StepScope : Creating object in scope=step, name=scopedTarget.writer.transaction-xyz
2016-03-03 14:19:47.041 DEBUG 26954 --- [enerContainer-1] o.s.batch.core.scope.StepScope : Registered destruction callback in scope=step, name=scopedTarget.writer.transaction-xyz
2016-03-03 14:19:47.068 TRACE 26954 --- [enerContainer-1] c.a.a.e.t.j.w.v.TransactionXyzWriter : POSTing to http://localhost:19999/transactionExportJob
2016-03-03 14:19:47.307 TRACE 26954 --- [enerContainer-1] c.a.a.e.t.j.w.v.TransactionXyzWriter : jsonObject = {"success":"true"}
2016-03-03 14:19:47.308 DEBUG 26954 --- [enerContainer-1] o.s.b.c.step.item.ChunkOrientedTasklet : Inputs not busy, ended: false
2016-03-03 14:19:47.308 DEBUG 26954 --- [enerContainer-1] o.s.batch.core.step.tasklet.TaskletStep : Applying contribution: [StepContribution: read=1, written=1, filtered=0, readSkips=0, writeSkips=0, processSkips=0, exitStatus=EXECUTING]
2016-03-03 14:19:47.341 DEBUG 26954 --- [enerContainer-1] o.s.batch.core.step.tasklet.TaskletStep : Saving step execution before commit: StepExecution: id=372, version=1, name=job.transaction-export.step1, status=STARTED, exitStatus=EXECUTING, readCount=1, filterCount=0, writeCount=1 readSkipCount=0, writeSkipCount=0, processSkipCount=0, commitCount=1, rollbackCount=0, exitDescription=
// repeat starts here
2016-03-03 14:19:47.466 DEBUG 26954 --- [enerContainer-1] o.s.batch.repeat.support.RepeatTemplate : Repeat operation about to start at count=2
2016-03-03 14:19:47.466 DEBUG 26954 --- [enerContainer-1] o.s.b.c.s.c.StepContextRepeatCallback : Preparing chunk execution for StepContext: org.springframework.batch.core.scope.context.StepContext#d9db406
2016-03-03 14:19:47.466 DEBUG 26954 --- [enerContainer-1] o.s.b.c.s.c.StepContextRepeatCallback : Chunk execution starting: queue size=0
2016-03-03 14:19:47.496 DEBUG 26954 --- [enerContainer-1] o.s.batch.repeat.support.RepeatTemplate : Starting repeat context.
2016-03-03 14:19:47.497 DEBUG 26954 --- [enerContainer-1] o.s.batch.repeat.support.RepeatTemplate : Repeat operation about to start at count=1
2016-03-03 14:19:47.497 TRACE 26954 --- [enerContainer-1] c.a.a.e.t.job.reader.TransactionReader : In TransactionReader.read
2016-03-03 14:19:47.534 TRACE 26954 --- [enerContainer-1] c.a.a.e.t.job.reader.TransactionReader : paymentTransaction = {map details here}
2016-03-03 14:19:47.534 DEBUG 26954 --- [enerContainer-1] o.s.batch.repeat.support.RepeatTemplate : Repeat is complete according to policy and result value.
2016-03-03 14:19:47.534 TRACE 26954 --- [enerContainer-1] c.a.a.e.t.j.p.TransactionProcessor : In TransactionProcessor.process with paymentTransaction {map details here}
2016-03-03 14:19:47.534 TRACE 26954 --- [enerContainer-1] c.a.a.e.t.j.p.TransactionProcessor : returnAfxEntity = {map details here}
2016-03-03 14:19:47.542 TRACE 26954 --- [enerContainer-1] c.a.a.e.t.j.w.v.TransactionXyzWriter : POSTing to http://localhost:19999/transactionExportJob
2016-03-03 14:19:47.546 DEBUG 26954 --- [enerContainer-1] o.s.batch.core.step.tasklet.TaskletStep : Applying contribution: [StepContribution: read=1, written=0, filtered=0, readSkips=0, writeSkips=0, processSkips=0, exitStatus=EXECUTING]
2016-03-03 14:19:47.546 DEBUG 26954 --- [enerContainer-1] o.s.batch.core.step.tasklet.TaskletStep : Rollback for RuntimeException: org.springframework.web.client.ResourceAccessException: I/O error on POST request for "http://localhost:19999/transactionExportJob": Unexpected end of file from server; nested exception is java.net.SocketException: Unexpected end of file from server
2016-03-03 14:19:47.601 DEBUG 26954 --- [enerContainer-1] o.s.batch.repeat.support.RepeatTemplate : Handling exception: org.springframework.web.client.ResourceAccessException, caused by: org.springframework.web.client.ResourceAccessException: I/O error on POST request for "http://localhost:19999/transactionExportJob": Unexpected end of file from server; nested exception is java.net.SocketException: Unexpected end of file from server
2016-03-03 14:19:47.601 DEBUG 26954 --- [enerContainer-1] o.s.batch.repeat.support.RepeatTemplate : Handling fatal exception explicitly (rethrowing first of 1): org.springframework.web.client.ResourceAccessException: I/O error on POST request for "http://localhost:19999/transactionExportJob": Unexpected end of file from server; nested exception is java.net.SocketException: Unexpected end of file from server
2016-03-03 14:19:47.608 ERROR 26954 --- [enerContainer-1] o.s.batch.core.step.AbstractStep : Encountered an error executing step job.transaction-export.step1 in job job.transaction-export
org.springframework.web.client.ResourceAccessException: I/O error on POST request for "http://localhost:19999/transactionExportJob": Unexpected end of file from server; nested exception is java.net.SocketException: Unexpected end of file from server
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:607) ~[spring-web-4.2.4.RELEASE.jar:4.2.4.RELEASE]
at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:557) ~[spring-web-4.2.4.RELEASE.jar:4.2.4.RELEASE]
at org.springframework.web.client.RestTemplate.exchange(RestTemplate.java:475) ~[spring-web-4.2.4.RELEASE.jar:4.2.4.RELEASE]
at com.abc.afx.exporter.transaction.job.writer.xyz.TransactionXyzWriter.write(TransactionXyzWriter.java:89) ~[classes/:na]
at com.abc.afx.exporter.transaction.job.writer.xyz.TransactionXyzWriter$$FastClassBySpringCGLIB$$7020a21.invoke(<generated>) ~[classes/:na]
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) ~[spring-core-4.2.4.RELEASE.jar:4.2.4.RELEASE]
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:720) ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) [spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:133) ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:121) ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) [spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:655) ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
at com.abc.afx.exporter.transaction.job.writer.xyz.TransactionXyzWriter$$EnhancerBySpringCGLIB$$3eabc04e.write(<generated>) ~[classes/:na]
at org.springframework.batch.core.step.item.SimpleChunkProcessor.writeItems(SimpleChunkProcessor.java:175) ~[spring-batch-core-3.0.6.RELEASE.jar:3.0.6.RELEASE]
at org.springframework.batch.core.step.item.SimpleChunkProcessor.doWrite(SimpleChunkProcessor.java:151) ~[spring-batch-core-3.0.6.RELEASE.jar:3.0.6.RELEASE]
at org.springframework.batch.core.step.item.SimpleChunkProcessor.write(SimpleChunkProcessor.java:274) ~[spring-batch-core-3.0.6.RELEASE.jar:3.0.6.RELEASE]
at org.springframework.batch.core.step.item.SimpleChunkProcessor.process(SimpleChunkProcessor.java:199) ~[spring-batch-core-3.0.6.RELEASE.jar:3.0.6.RELEASE]
at org.springframework.batch.core.step.item.ChunkOrientedTasklet.execute(ChunkOrientedTasklet.java:75) ~[spring-batch-core-3.0.6.RELEASE.jar:3.0.6.RELEASE]
at org.springframework.batch.core.step.tasklet.TaskletStep$ChunkTransactionCallback.doInTransaction(TaskletStep.java:406) ~[spring-batch-core-3.0.6.RELEASE.jar:3.0.6.RELEASE]
at org.springframework.batch.core.step.tasklet.TaskletStep$ChunkTransactionCallback.doInTransaction(TaskletStep.java:330) ~[spring-batch-core-3.0.6.RELEASE.jar:3.0.6.RELEASE]
at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:133) ~[spring-tx-4.2.4.RELEASE.jar:4.2.4.RELEASE]
at org.springframework.batch.core.step.tasklet.TaskletStep$2.doInChunkContext(TaskletStep.java:271) ~[spring-batch-core-3.0.6.RELEASE.jar:3.0.6.RELEASE]
at org.springframework.batch.core.scope.context.StepContextRepeatCallback.doInIteration(StepContextRepeatCallback.java:81) ~[spring-batch-core-3.0.6.RELEASE.jar:3.0.6.RELEASE]
at org.springframework.batch.repeat.support.RepeatTemplate.getNextResult(RepeatTemplate.java:374) ~[spring-batch-infrastructure-3.0.6.RELEASE.jar:3.0.6.RELEASE]
at org.springframework.batch.repeat.support.RepeatTemplate.executeInternal(RepeatTemplate.java:215) ~[spring-batch-infrastructure-3.0.6.RELEASE.jar:3.0.6.RELEASE]
at org.springframework.batch.repeat.support.RepeatTemplate.iterate(RepeatTemplate.java:144) ~[spring-batch-infrastructure-3.0.6.RELEASE.jar:3.0.6.RELEASE]
at org.springframework.batch.core.step.tasklet.TaskletStep.doExecute(TaskletStep.java:257) ~[spring-batch-core-3.0.6.RELEASE.jar:3.0.6.RELEASE]
at org.springframework.batch.core.step.AbstractStep.execute(AbstractStep.java:200) ~[spring-batch-core-3.0.6.RELEASE.jar:3.0.6.RELEASE]
at org.springframework.batch.core.job.SimpleStepHandler.handleStep(SimpleStepHandler.java:148) [spring-batch-core-3.0.6.RELEASE.jar:3.0.6.RELEASE]
at org.springframework.batch.core.job.AbstractJob.handleStep(AbstractJob.java:392) [spring-batch-core-3.0.6.RELEASE.jar:3.0.6.RELEASE]
at org.springframework.batch.core.job.SimpleJob.doExecute(SimpleJob.java:135) [spring-batch-core-3.0.6.RELEASE.jar:3.0.6.RELEASE]
at org.springframework.batch.core.job.AbstractJob.execute(AbstractJob.java:306) [spring-batch-core-3.0.6.RELEASE.jar:3.0.6.RELEASE]
at org.springframework.batch.core.launch.support.SimpleJobLauncher$1.run(SimpleJobLauncher.java:135) [spring-batch-core-3.0.6.RELEASE.jar:3.0.6.RELEASE]
at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:50) [spring-core-4.2.4.RELEASE.jar:4.2.4.RELEASE]
at org.springframework.batch.core.launch.support.SimpleJobLauncher.run(SimpleJobLauncher.java:128) [spring-batch-core-3.0.6.RELEASE.jar:3.0.6.RELEASE]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_51]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_51]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_51]
...
Caused by: java.net.SocketException: Unexpected end of file from server
at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:792) ~[na:1.8.0_51]
at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:647) ~[na:1.8.0_51]
at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1535) ~[na:1.8.0_51]
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1440) ~[na:1.8.0_51]
at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480) ~[na:1.8.0_51]
at org.springframework.http.client.SimpleClientHttpResponse.getRawStatusCode(SimpleClientHttpResponse.java:48) ~[spring-web-4.2.4.RELEASE.jar:4.2.4.RELEASE]
at org.springframework.http.client.AbstractClientHttpResponse.getStatusCode(AbstractClientHttpResponse.java:33) ~[spring-web-4.2.4.RELEASE.jar:4.2.4.RELEASE]
at org.springframework.web.client.DefaultResponseErrorHandler.getHttpStatusCode(DefaultResponseErrorHandler.java:56) ~[spring-web-4.2.4.RELEASE.jar:4.2.4.RELEASE]
at org.springframework.web.client.DefaultResponseErrorHandler.hasError(DefaultResponseErrorHandler.java:50) ~[spring-web-4.2.4.RELEASE.jar:4.2.4.RELEASE]
at org.springframework.web.client.RestTemplate.handleResponse(RestTemplate.java:629) ~[spring-web-4.2.4.RELEASE.jar:4.2.4.RELEASE]
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:597) ~[spring-web-4.2.4.RELEASE.jar:4.2.4.RELEASE]
... 63 common frames omitted
2016-03-03 14:19:47.758 DEBUG 26954 --- [enerContainer-1] o.s.b.c.r.dao.JdbcStepExecutionDao : Truncating long message before update of StepExecution, original message is: org.springframework.web.client.ResourceAccessException: I/O error on POST request for "http://localhost:19999/transactionExportJob": Unexpected end of file from server; nested exception is java.net.SocketException: Unexpected end of file from server
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:607)
at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:557)
at org.springframework.web.client.RestTemplate.exchange(RestTemplate.java:475)
at com.abc.afx.exporter.transaction.job.writer.xyz.TransactionXyzWriter.write(TransactionXyzWriter.java:89)
at com.abc.afx.exporter.transaction.job.writer.xyz.TransactionXyzWriter$$FastClassBySpringCGLIB$$7020a21.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:720)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:133)
at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:121)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:655)
at com.abc.afx.exporter.transaction.job.writer.xyz.TransactionXyzWriter$$EnhancerBySpringCGLIB$$3eabc04e.write(<generated>)
at org.springframework.batch.core.step.item.SimpleChunkProcessor.writeItems(SimpleChunkProcessor.java:175)
at org.springframework.batch.core.step.item.SimpleChunkProcessor.doWrite(SimpleChunkProcessor.java:151)
at org.springframework.batch.core.step.item.SimpleChunkProcessor.write(SimpleChunkProcessor.java:274)
at org.springframework.batch.core.step.item.SimpleChunkProcessor.process(SimpleChunkProcessor.java:199)
at org.springframework.batch.core.step.item.ChunkOrientedTasklet.execute(ChunkOrientedTasklet.java:75)
at org.springframework.batch.core.step.tasklet.TaskletStep$ChunkTransactionCallback.doInTransaction(TaskletStep.java:406)
at org.springframework.batch.core.step.tasklet.TaskletStep$ChunkTransactionCallback.doInTransaction(TaskletStep.java:330)
at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:133)
at org.springframework.batch.core.step.tasklet.TaskletStep$2.doInChunkContext(TaskletStep.java:271)
at org.springframework.batch.core.scope.context.StepContextRepeatCallback.doInIteration(StepContextRepeatCallback.java:81)
at org.springframework.batch.repeat.support.RepeatTemplate.getNextResult(RepeatTemplate.java:374)
at org.springframework.batch.repeat.support.RepeatTemplate.executeInternal(RepeatTemplate.java:215)
at org.springframework.batch.repeat.support.RepeatTemplate.iterate(RepeatTemplate.java:144)
at org.springframework.batch.core.step.tasklet.TaskletStep.doExecute(TaskletStep.java:257)
at org.springframework.batch.core.step.AbstractStep.execute(AbstractStep.java:200)
at org.springframework.batch.core.job.SimpleStepHandler.handleStep(SimpleStepHandler.java:148)
at org.springframework.batch.core.job.AbstractJob.handleStep(AbstractJob.java:392)
at org.springframework.batch.core.job.SimpleJob.doExecute(SimpleJob.java:135)
at org.springframework.batch.core.job.AbstractJob.execute(AbstractJob.java:306)
at org.springframework.batch.core.launch.support.SimpleJobLauncher$1.run(SimpleJobLauncher.java:135)
at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:50)
at org.springframework.batch.core.launch.support.SimpleJobLauncher.run(SimpleJobLauncher.java:128)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
...
Caused by: java.net.SocketException: Unexpected end of file from server
at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:792)
at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:647)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1535)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1440)
at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480)
at org.springframework.http.client.SimpleClientHttpResponse.getRawStatusCode(SimpleClientHttpResponse.java:48)
at org.springframework.http.client.AbstractClientHttpResponse.getStatusCode(AbstractClientHttpResponse.java:33)
at org.springframework.web.client.DefaultResponseErrorHandler.getHttpStatusCode(DefaultResponseErrorHandler.java:56)
at org.springframework.web.client.DefaultResponseErrorHandler.hasError(DefaultResponseErrorHandler.java:50)
at org.springframework.web.client.RestTemplate.handleResponse(RestTemplate.java:629)
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:597)
... 63 more
2016-03-03 14:19:47.892 DEBUG 26954 --- [enerContainer-1] o.s.batch.core.step.AbstractStep : Step execution complete: StepExecution: id=372, version=3, name=job.transaction-export.step1, status=FAILED, exitStatus=FAILED, readCount=2, filterCount=0, writeCount=1 readSkipCount=0, writeSkipCount=0, processSkipCount=0, commitCount=1, rollbackCount=1
2016-03-03 14:19:48.017 DEBUG 26954 --- [enerContainer-1] o.s.batch.core.job.AbstractJob : Upgrading JobExecution status: StepExecution: id=372, version=3, name=job.transaction-export.step1, status=FAILED, exitStatus=FAILED, readCount=2, filterCount=0, writeCount=1 readSkipCount=0, writeSkipCount=0, processSkipCount=0, commitCount=1, rollbackCount=1, exitDescription=org.springframework.web.client.ResourceAccessException: I/O error on POST request for "http://localhost:19999/transactionExportJob": Unexpected end of file from server; nested exception is java.net.SocketException: Unexpected end of file from server
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:607)
at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:557)
at org.springframework.web.client.RestTemplate.exchange(RestTemplate.java:475)
at com.abc.afx.exporter.transaction.job.writer.xyz.TransactionXyzWriter.write(TransactionXyzWriter.java:89)
at com.abc.afx.exporter.transaction.job.writer.xyz.TransactionXyzWriter$$FastClassBySpringCGLIB$$7020a21.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:720)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:133)
at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:121)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:655)
at com.abc.afx.exporter.transaction.job.writer.xyz.TransactionXyzWriter$$EnhancerBySpringCGLIB$$3eabc04e.write(<generated>)
at org.springframework.batch.core.step.item.SimpleChunkProcessor.writeItems(SimpleChunkProcessor.java:175)
at org.springframework.batch.core.step.item.SimpleChunkProcessor.doWrite(SimpleChunkProcessor.java:151)
at org.springframework.batch.core.step.item.SimpleChunkProcessor.write(SimpleChunkProcessor.java:274)
at org.springframework.batch.core.step.item.SimpleChunkProcessor.process(SimpleChunkProcessor.java:199)
at org.springframework.batch.core.step.item.ChunkOrientedTasklet.execute(ChunkOrientedTasklet.java:75)
at org.springframework.batch.core.step.tasklet.TaskletStep$ChunkTransactionCallback.doInTransaction(TaskletStep.java:406)
at org.springframework.batch.core.step.tasklet.TaskletStep$ChunkTransactionCallback.doInTransaction(TaskletStep.java:330)
at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:133)
at org.springframework.batch.core.step.tasklet.TaskletStep$2.doInChunkContext(TaskletStep.java:271)
at org.springframework.batch.core.scope.context.StepContextRepeatCallback.doInIteration(StepContextRepeatCallback.java:81)
at org.springframework.batch.repeat.support.RepeatTemplate.getNextResult(RepeatTemplate.java:374)
at org.springframework.batch.repeat.support.RepeatTemplate.executeInternal(RepeatTemplate.java:215)
at org.springframework.batch.repeat.support.RepeatTemplate.iterate(RepeatTemplate.java:144)
at org.springframework.batch.core.step.tasklet.TaskletStep.doExecute(TaskletStep.java:257)
at org.springframework.batch.core.step.AbstractStep.execute(AbstractStep.java:200)
at org.springframework.batch.core.job.SimpleStepHandler.handleStep(SimpleStepHandler.java:148)
at org.springframework.batch.core.job.AbstractJob.handleStep(AbstractJob.java:392)
at org.springframework.batch.core.job.SimpleJob.doExecute(SimpleJob.java:135)
at org.springframework.batch.core.job.AbstractJob.execute(AbstractJob.java:306)
at org.springframework.batch.core.launch.support.SimpleJobLauncher$1.run(SimpleJobLauncher.java:135)
at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:50)
at org.springframework.batch.core.launch.support.SimpleJobLauncher.run(SimpleJobLauncher.java:128)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
...
Caused by: java.net.SocketException: Unexpected end of file from server
... 74 more
...
, job=[JobInstance: id=370, version=0, Job=[job.transaction-export]], jobParameters=[{job parameters here, launchTime=1457032784898}]
2016-03-03 14:19:48.018 TRACE 26954 --- [enerContainer-1] .p.j.PatientScheduleJobExecutionListener : Job {job parameters here, launchTime=1457032784898} has completed.
You can try to use step listener and then decide repeat a step or finish the job. Check this answer.
I currently have a SI project that listen on a JMS queue do some processing and depending on the messages send emails, write files, ...
Now I want to add a daily digest feature on the sending emails part. I want email messages to be stored somewhere and that once a day all messages that have the same destination email are grouped, content is concatenated and email is sent.
What would be the best design solution for this?
I checked the aggregator concept but I have few questions:
I need persistent storage of aggregated messages. Aggregated messages are only sent once a day so I don't want to loose messages. I guess I should use JdbcMessageStore?
I need transactions from Input queue -> aggregator then another transactions from aggregator to email output. Is this possible and if yes how should I configure?
Thanks for your help.
Cheers
EDIT
I now tried as here:
<int:aggregator id="templatingDailyAggregator"
input-channel="templatingDailyAggregatorInputChannel"
output-channel="templatingDailyAggregatorOutputChannel"
message-store="templatingEmailAggregatorStore"
correlation-strategy-expression="headers['templatingEmailGroupingCategory']+payload.emailMessage.email"
release-strategy-expression="false"
send-partial-result-on-expiry="true"
expire-groups-upon-completion="true"
>
<int:expire-transactional transaction-manager="templatingAggregatorStoreTransactionManager"/>
</int:aggregator>
<int-jdbc:message-store id="templatingEmailAggregatorStore" data-source="templatingEmailAggregatorStoreDataSource" />
<bean id="templatingAggregatorStoreTransactionManager"
class="org.springframework.jdbc.datasource.DataSourceTransactionManager">
<property name="dataSource" ref="templatingEmailAggregatorStoreDataSource" />
</bean>
<!-- MySQL DB DataSource -->
<bean id="templatingEmailAggregatorStoreDataSource"
class="org.springframework.jdbc.datasource.DriverManagerDataSource">
<property name="driverClassName" value="com.mysql.jdbc.Driver" />
<property name="url" value="MYURL" />
<property name="username" value="MYUSER" />
<property name="password" value="MYPASS" />
</bean>
<bean id="templatingEmailAggregatorStoreReaper" class="org.springframework.integration.store.MessageGroupStoreReaper">
<property name="messageGroupStore" ref="templatingEmailAggregatorStore"/>
<property name="timeout" value="10"/>
</bean>
<task:scheduled-tasks scheduler="templatingAggregatorScheduler">
<task:scheduled ref="templatingEmailAggregatorStoreReaper" method="run" cron="0 */2 * * * * "/>
</task:scheduled-tasks>
<task:scheduler id="templatingAggregatorScheduler"/>
<int:transformer id="templatingDailyDigestTransformer" ref="templatingDailyDigestTransformerBean" input-channel="templatingDailyAggregatorOutputChannel" method="processMessage" output-channel="emailOutputChannel"/>
But have the problem that if I have an exception after the aggregation part (e.g. during the sending of the email) I would expect the MessageGroup consumption to be rolled back and thus still be present in the database. However this is not the case the MessageGroup is consumed, is not present in the DB anymore and thus is lost.
EDIT 2
I added the transformer after the aggregator in the above XML. Currently this transformer simply raise an exception for testing crashed cases.
Here is the stacktrace I optain:
DEBUG [exec-1] - Expiring all messages older than timeout=10 from message group store: org.springframework.integration.jdbc.JdbcMessageStore#3e082583
DEBUG [exec-1] - Executing prepared SQL query
DEBUG [exec-1] - Executing prepared SQL statement [SELECT distinct GROUP_KEY as CREATED from INT_MESSAGE_GROUP where REGION=?]
DEBUG [exec-1] - Fetching JDBC Connection from DataSource
DEBUG [exec-1] - Creating new JDBC DriverManager Connection to [jdbc:mysql://localhost:3306/aggregator]
DEBUG [exec-1] - Returning JDBC Connection to DataSource
DEBUG [exec-1] - Executing prepared SQL query
DEBUG [exec-1] - Executing prepared SQL statement [SELECT MESSAGE_ID, MESSAGE_BYTES, CREATED_DATE from INT_MESSAGE where MESSAGE_ID in (SELECT MESSAGE_ID from INT_GROUP_TO_MESSAGE where GROUP_KEY = ?) and REGION=? ORDER BY CREATED_DATE]
DEBUG [exec-1] - Fetching JDBC Connection from DataSource
DEBUG [exec-1] - Creating new JDBC DriverManager Connection to [jdbc:mysql://localhost:3306/aggregator]
DEBUG [exec-1] - Returning BLOB as bytes
DEBUG [exec-1] - Returning JDBC Connection to DataSource
DEBUG [exec-1] - Executing prepared SQL query
DEBUG [exec-1] - Executing prepared SQL statement [SELECT COMPLETE, LAST_RELEASED_SEQUENCE, CREATED_DATE, UPDATED_DATE from INT_MESSAGE_GROUP where GROUP_KEY = ? and REGION=?]
DEBUG [exec-1] - Fetching JDBC Connection from DataSource
DEBUG [exec-1] - Creating new JDBC DriverManager Connection to [jdbc:mysql://localhost:3306/aggregator]
DEBUG [exec-1] - Returning JDBC Connection to DataSource
DEBUG [exec-1] - Executing prepared SQL query
DEBUG [exec-1] - Executing prepared SQL statement [SELECT MESSAGE_ID, MESSAGE_BYTES, CREATED_DATE from INT_MESSAGE where MESSAGE_ID in (SELECT MESSAGE_ID from INT_GROUP_TO_MESSAGE where GROUP_KEY = ?) and REGION=? ORDER BY CREATED_DATE]
DEBUG [exec-1] - Fetching JDBC Connection from DataSource
DEBUG [exec-1] - Creating new JDBC DriverManager Connection to [jdbc:mysql://localhost:3306/aggregator]
DEBUG [exec-1] - Returning BLOB as bytes
DEBUG [exec-1] - Returning JDBC Connection to DataSource
DEBUG [exec-1] - Executing prepared SQL query
DEBUG [exec-1] - Executing prepared SQL statement [SELECT COMPLETE, LAST_RELEASED_SEQUENCE, CREATED_DATE, UPDATED_DATE from INT_MESSAGE_GROUP where GROUP_KEY = ? and REGION=?]
DEBUG [exec-1] - Fetching JDBC Connection from DataSource
DEBUG [exec-1] - Creating new JDBC DriverManager Connection to [jdbc:mysql://localhost:3306/aggregator]
DEBUG [exec-1] - Returning JDBC Connection to DataSource
INFO [exec-1] - Expiring MessageGroup with correlationKey[fb90fe78-c3df-3793-9ee8-acae4924bebe]
DEBUG [exec-1] - Prematurely releasing partially complete group with key [fb90fe78-c3df-3793-9ee8-acae4924bebe] to: templatingDailyAggregatorOutputChannel
DEBUG [exec-1] - Completing group with correlationKey [fb90fe78-c3df-3793-9ee8-acae4924bebe]
DEBUG [exec-1] - org.springframework.integration.transformer.MessageTransformingHandler#2fdde28c received message: GenericMessage [payload=[ch.post.ehealth.extcom.templating.core.PreAggregatorEmailMessage#563b5cce], headers={jms_timestamp=1426090799820, extcomPluginDestination=templatingPluginInputChannel, extcomId=5782d059-c88d-44ae-82a1-0b738b43e821, jms_messageId=ID:some-vm-45363-1421229178337-3:9:1:1:4, timestamp=1426090920046, id=b8330bed-fc74-c8d5-6838-a3116a05ab39, history=jmsInputAdapter,inputChannel,xmlToSpringIntTransformer,pluginRouterChannel,pluginRouter,templatingPluginInputChannel,templatingTransformer,templatingPluginOutputChannel,templatingOutputRouter,templatingEmailOutputChannel,templatingEmailGroupingRouter,templatingPreAggregatorChannel,templatingPreAggregatorTransformer,templatingDailyAggregatorInputChannel,templatingDailyAggregator,templatingDailyAggregatorOutputChannel, JdbcMessageStore.CREATED_DATE=1426090800772, jms_type=, jms_redelivered=false, priority=0, templatingEmailGrouping=DAILY, jms_correlationId=, JdbcMessageStore.SAVED=true, templatingEmailGroupingCategory=DAILY}]
DEBUG [exec-1] - Executing prepared SQL query
DEBUG [exec-1] - Executing prepared SQL statement [select MESSAGE_ID, CREATED_DATE from INT_MESSAGE where MESSAGE_ID in (select MESSAGE_ID from INT_GROUP_TO_MESSAGE where GROUP_KEY=? and REGION=?) ORDER BY CREATED_DATE]
DEBUG [exec-1] - Fetching JDBC Connection from DataSource
DEBUG [exec-1] - Creating new JDBC DriverManager Connection to [jdbc:mysql://localhost:3306/aggregator]
DEBUG [exec-1] - Returning JDBC Connection to DataSource
DEBUG [exec-1] - Executing prepared SQL query
DEBUG [exec-1] - Executing prepared SQL statement [SELECT MESSAGE_ID, CREATED_DATE, MESSAGE_BYTES from INT_MESSAGE where MESSAGE_ID=? and REGION=?]
DEBUG [exec-1] - Fetching JDBC Connection from DataSource
DEBUG [exec-1] - Creating new JDBC DriverManager Connection to [jdbc:mysql://localhost:3306/aggregator]
DEBUG [exec-1] - Returning BLOB as bytes
DEBUG [exec-1] - Returning JDBC Connection to DataSource
DEBUG [exec-1] - Executing prepared SQL update
DEBUG [exec-1] - Executing prepared SQL statement [DELETE from INT_MESSAGE where MESSAGE_ID=? and REGION=?]
DEBUG [exec-1] - Fetching JDBC Connection from DataSource
DEBUG [exec-1] - Creating new JDBC DriverManager Connection to [jdbc:mysql://localhost:3306/aggregator]
DEBUG [exec-1] - SQL update affected 1 rows
DEBUG [exec-1] - Returning JDBC Connection to DataSource
DEBUG [exec-1] - Executing prepared SQL update
DEBUG [exec-1] - Executing prepared SQL statement [DELETE from INT_GROUP_TO_MESSAGE where GROUP_KEY=? and REGION=?]
DEBUG [exec-1] - Fetching JDBC Connection from DataSource
DEBUG [exec-1] - Creating new JDBC DriverManager Connection to [jdbc:mysql://localhost:3306/aggregator]
DEBUG [exec-1] - Removing relationships for the group with group key=fb90fe78-c3df-3793-9ee8-acae4924bebe
DEBUG [exec-1] - SQL update affected 1 rows
DEBUG [exec-1] - Returning JDBC Connection to DataSource
DEBUG [exec-1] - Executing prepared SQL update
DEBUG [exec-1] - Executing prepared SQL statement [DELETE from INT_MESSAGE_GROUP where GROUP_KEY=? and REGION=?]
DEBUG [exec-1] - Fetching JDBC Connection from DataSource
DEBUG [exec-1] - Creating new JDBC DriverManager Connection to [jdbc:mysql://localhost:3306/aggregator]
DEBUG [exec-1] - Marking messages with group key=fb90fe78-c3df-3793-9ee8-acae4924bebe
DEBUG [exec-1] - SQL update affected 1 rows
DEBUG [exec-1] - Returning JDBC Connection to DataSource
ERROR [exec-1] - Exception in expiry callbackjava.lang.RuntimeException: test crash
at ch.post.ehealth.extcom.templating.core.DailyDigestTransformer.processMessage(DailyDigestTransformer.java:33) ~[extcom-templating.jar:na]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.7.0_71]
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) ~[na:1.7.0_71]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) ~[na:1.7.0_71]
at java.lang.reflect.Method.invoke(Unknown Source) ~[na:1.7.0_71]
at org.springframework.expression.spel.support.ReflectiveMethodExecutor.execute(ReflectiveMethodExecutor.java:63) ~[spring-expression-4.0.8.RELEASE.jar:4.0.8.RELEASE]
at org.springframework.expression.spel.ast.MethodReference.getValueInternal(MethodReference.java:95) ~[spring-expression-4.0.8.RELEASE.jar:4.0.8.RELEASE]
at org.springframework.expression.spel.ast.MethodReference.access$000(MethodReference.java:44) ~[spring-expression-4.0.8.RELEASE.jar:4.0.8.RELEASE]
at org.springframework.expression.spel.ast.MethodReference$MethodValueRef.getValue(MethodReference.java:258) ~[spring-expression-4.0.8.RELEASE.jar:4.0.8.RELEASE]
at org.springframework.expression.spel.ast.CompoundExpression.getValueInternal(CompoundExpression.java:84) ~[spring-expression-4.0.8.RELEASE.jar:4.0.8.RELEASE]
at org.springframework.expression.spel.ast.SpelNodeImpl.getTypedValue(SpelNodeImpl.java:112) ~[spring-expression-4.0.8.RELEASE.jar:4.0.8.RELEASE]
at org.springframework.expression.spel.standard.SpelExpression.getValue(SpelExpression.java:111) ~[spring-expression-4.0.8.RELEASE.jar:4.0.8.RELEASE]
at org.springframework.integration.util.AbstractExpressionEvaluator.evaluateExpression(AbstractExpressionEvaluator.java:164) ~[spring-integration-core-4.1.2.RELEASE.jar:na]
at org.springframework.integration.util.MessagingMethodInvokerHelper.processInternal(MessagingMethodInvokerHelper.java:276) ~[spring-integration-core-4.1.2.RELEASE.jar:na]
at org.springframework.integration.util.MessagingMethodInvokerHelper.process(MessagingMethodInvokerHelper.java:142) ~[spring-integration-core-4.1.2.RELEASE.jar:na]
at org.springframework.integration.handler.MethodInvokingMessageProcessor.processMessage(MethodInvokingMessageProcessor.java:75) ~[spring-integration-core-4.1.2.RELEASE.jar:na]
Wrapped by: org.springframework.messaging.MessageHandlingException: ; nested exception is java.lang.RuntimeException: test crash
at org.springframework.integration.handler.MethodInvokingMessageProcessor.processMessage(MethodInvokingMessageProcessor.java:78) ~[spring-integration-core-4.1.2.RELEASE.jar:na]
at org.springframework.integration.transformer.AbstractMessageProcessingTransformer.transform(AbstractMessageProcessingTransformer.java:64) ~[spring-integration-core-4.1.2.RELEASE.jar:na]
at org.springframework.integration.transformer.MessageTransformingHandler.handleRequestMessage(MessageTransformingHandler.java:68) ~[spring-integration-core-4.1.2.RELEASE.jar:na]
Wrapped by: org.springframework.integration.transformer.MessageTransformationException: ; nested exception is org.springframework.messaging.MessageHandlingException: ; nested exception is java.lang.RuntimeException: test crash
at org.springframework.integration.transformer.MessageTransformingHandler.handleRequestMessage(MessageTransformingHandler.java:74) ~[spring-integration-core-4.1.2.RELEASE.jar:na]
at org.springframework.integration.handler.AbstractReplyProducingMessageHandler.handleMessageInternal(AbstractReplyProducingMessageHandler.java:99) ~[spring-integration-core-4.1.2.RELEASE.jar:na]
at org.springframework.integration.handler.AbstractMessageHandler.handleMessage(AbstractMessageHandler.java:78) ~[spring-integration-core-4.1.2.RELEASE.jar:na]
at org.springframework.integration.dispatcher.AbstractDispatcher.tryOptimizedDispatch(AbstractDispatcher.java:116) ~[spring-integration-core-4.1.2.RELEASE.jar:na]
at org.springframework.integration.dispatcher.UnicastingDispatcher.doDispatch(UnicastingDispatcher.java:101) ~[spring-integration-core-4.1.2.RELEASE.jar:na]
at org.springframework.integration.dispatcher.UnicastingDispatcher.dispatch(UnicastingDispatcher.java:97) ~[spring-integration-core-4.1.2.RELEASE.jar:na]
at org.springframework.integration.channel.AbstractSubscribableChannel.doSend(AbstractSubscribableChannel.java:77) ~[spring-integration-core-4.1.2.RELEASE.jar:na]
at org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:277) ~[spring-integration-core-4.1.2.RELEASE.jar:na]
at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:115) ~[spring-messaging-4.1.4.RELEASE.jar:4.1.4.RELEASE]
at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:45) ~[spring-messaging-4.1.4.RELEASE.jar:4.1.4.RELEASE]
at org.springframework.messaging.core.AbstractMessageSendingTemplate.send(AbstractMessageSendingTemplate.java:95) ~[spring-messaging-4.1.4.RELEASE.jar:4.1.4.RELEASE]
at org.springframework.integration.handler.AbstractMessageProducingHandler.sendOutput(AbstractMessageProducingHandler.java:248) ~[spring-integration-core-4.1.2.RELEASE.jar:na]
at org.springframework.integration.handler.AbstractMessageProducingHandler.produceOutput(AbstractMessageProducingHandler.java:171) ~[spring-integration-core-4.1.2.RELEASE.jar:na]
at org.springframework.integration.handler.AbstractMessageProducingHandler.sendOutputs(AbstractMessageProducingHandler.java:119) ~[spring-integration-core-4.1.2.RELEASE.jar:na]
at org.springframework.integration.aggregator.AbstractCorrelatingMessageHandler.completeGroup(AbstractCorrelatingMessageHandler.java:657) ~[spring-integration-core-4.1.2.RELEASE.jar:na]
at org.springframework.integration.aggregator.AbstractCorrelatingMessageHandler.completeGroup(AbstractCorrelatingMessageHandler.java:642) ~[spring-integration-core-4.1.2.RELEASE.jar:na]
at org.springframework.integration.aggregator.AbstractCorrelatingMessageHandler.expireGroup(AbstractCorrelatingMessageHandler.java:619) ~[spring-integration-core-4.1.2.RELEASE.jar:na]
at org.springframework.integration.aggregator.AbstractCorrelatingMessageHandler.forceComplete(AbstractCorrelatingMessageHandler.java:543) ~[spring-integration-core-4.1.2.RELEASE.jar:na]
at org.springframework.integration.aggregator.AbstractCorrelatingMessageHandler$ForceReleaseMessageGroupProcessor.processMessageGroup(AbstractCorrelatingMessageHandler.java:721) ~[spring-integration-core-4.1.2.RELEASE.jar:na]
at org.springframework.integration.aggregator.AbstractCorrelatingMessageHandler$1.execute(AbstractCorrelatingMessageHandler.java:168) ~[spring-integration-core-4.1.2.RELEASE.jar:na]
at org.springframework.integration.store.AbstractMessageGroupStore.expire(AbstractMessageGroupStore.java:169) [spring-integration-core-4.1.2.RELEASE.jar:na]
at org.springframework.integration.store.AbstractMessageGroupStore.expireMessageGroups(AbstractMessageGroupStore.java:113) [spring-integration-core-4.1.2.RELEASE.jar:na]
at org.springframework.integration.store.MessageGroupStoreReaper.run(MessageGroupStoreReaper.java:115) [spring-integration-core-4.1.2.RELEASE.jar:na]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.7.0_71]
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) ~[na:1.7.0_71]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) ~[na:1.7.0_71]
at java.lang.reflect.Method.invoke(Unknown Source) ~[na:1.7.0_71]
at org.springframework.scheduling.support.ScheduledMethodRunnable.run(ScheduledMethodRunnable.java:65) [spring-context-4.0.8.RELEASE.jar:4.0.8.RELEASE]
at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54) [spring-context-4.0.8.RELEASE.jar:4.0.8.RELEASE]
at org.springframework.scheduling.concurrent.ReschedulingRunnable.run(ReschedulingRunnable.java:81) [spring-context-4.0.8.RELEASE.jar:4.0.8.RELEASE]
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) [na:1.7.0_71]
at java.util.concurrent.FutureTask.run(Unknown Source) [na:1.7.0_71]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(Unknown Source) [na:1.7.0_71]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) [na:1.7.0_71]
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) [na:1.7.0_71]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) [na:1.7.0_71]
at java.lang.Thread.run(Unknown Source) [na:1.7.0_71]
DEBUG [exec-1] - Executing prepared SQL query
DEBUG [exec-1] - Executing prepared SQL statement [SELECT MESSAGE_ID, MESSAGE_BYTES, CREATED_DATE from INT_MESSAGE where MESSAGE_ID in (SELECT MESSAGE_ID from INT_GROUP_TO_MESSAGE where GROUP_KEY = ?) and REGION=? ORDER BY CREATED_DATE]
DEBUG [exec-1] - Fetching JDBC Connection from DataSource
DEBUG [exec-1] - Creating new JDBC DriverManager Connection to [jdbc:mysql://localhost:3306/aggregator]
DEBUG [exec-1] - Returning JDBC Connection to DataSource
DEBUG [exec-1] - Executing prepared SQL query
DEBUG [exec-1] - Executing prepared SQL statement [SELECT COMPLETE, LAST_RELEASED_SEQUENCE, CREATED_DATE, UPDATED_DATE from INT_MESSAGE_GROUP where GROUP_KEY = ? and REGION=?]
DEBUG [exec-1] - Fetching JDBC Connection from DataSource
DEBUG [exec-1] - Creating new JDBC DriverManager Connection to [jdbc:mysql://localhost:3306/aggregator]
DEBUG [exec-1] - Returning JDBC Connection to DataSource
DEBUG [exec-1] - Group expiry candidate (fb90fe78-c3df-3793-9ee8-acae4924bebe) has changed - it may be reconsidered for a future expiration
ERROR [exec-1] - Unexpected error occurred in scheduled task.java.lang.RuntimeException: test crash
at ch.post.ehealth.extcom.templating.core.DailyDigestTransformer.processMessage(DailyDigestTransformer.java:33) ~[extcom-templating.jar:na]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.7.0_71]
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) ~[na:1.7.0_71]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) ~[na:1.7.0_71]
at java.lang.reflect.Method.invoke(Unknown Source) ~[na:1.7.0_71]
at org.springframework.expression.spel.support.ReflectiveMethodExecutor.execute(ReflectiveMethodExecutor.java:63) ~[spring-expression-4.0.8.RELEASE.jar:4.0.8.RELEASE]
at org.springframework.expression.spel.ast.MethodReference.getValueInternal(MethodReference.java:95) ~[spring-expression-4.0.8.RELEASE.jar:4.0.8.RELEASE]
at org.springframework.expression.spel.ast.MethodReference.access$000(MethodReference.java:44) ~[spring-expression-4.0.8.RELEASE.jar:4.0.8.RELEASE]
at org.springframework.expression.spel.ast.MethodReference$MethodValueRef.getValue(MethodReference.java:258) ~[spring-expression-4.0.8.RELEASE.jar:4.0.8.RELEASE]
at org.springframework.expression.spel.ast.CompoundExpression.getValueInternal(CompoundExpression.java:84) ~[spring-expression-4.0.8.RELEASE.jar:4.0.8.RELEASE]
at org.springframework.expression.spel.ast.SpelNodeImpl.getTypedValue(SpelNodeImpl.java:112) ~[spring-expression-4.0.8.RELEASE.jar:4.0.8.RELEASE]
at org.springframework.expression.spel.standard.SpelExpression.getValue(SpelExpression.java:111) ~[spring-expression-4.0.8.RELEASE.jar:4.0.8.RELEASE]
at org.springframework.integration.util.AbstractExpressionEvaluator.evaluateExpression(AbstractExpressionEvaluator.java:164) ~[spring-integration-core-4.1.2.RELEASE.jar:na]
at org.springframework.integration.util.MessagingMethodInvokerHelper.processInternal(MessagingMethodInvokerHelper.java:276) ~[spring-integration-core-4.1.2.RELEASE.jar:na]
at org.springframework.integration.util.MessagingMethodInvokerHelper.process(MessagingMethodInvokerHelper.java:142) ~[spring-integration-core-4.1.2.RELEASE.jar:na]
at org.springframework.integration.handler.MethodInvokingMessageProcessor.processMessage(MethodInvokingMessageProcessor.java:75) ~[spring-integration-core-4.1.2.RELEASE.jar:na]
Wrapped by: org.springframework.messaging.MessageHandlingException: ; nested exception is java.lang.RuntimeException: test crash
at org.springframework.integration.handler.MethodInvokingMessageProcessor.processMessage(MethodInvokingMessageProcessor.java:78) ~[spring-integration-core-4.1.2.RELEASE.jar:na]
at org.springframework.integration.transformer.AbstractMessageProcessingTransformer.transform(AbstractMessageProcessingTransformer.java:64) ~[spring-integration-core-4.1.2.RELEASE.jar:na]
at org.springframework.integration.transformer.MessageTransformingHandler.handleRequestMessage(MessageTransformingHandler.java:68) ~[spring-integration-core-4.1.2.RELEASE.jar:na]
Wrapped by: org.springframework.integration.transformer.MessageTransformationException: ; nested exception is org.springframework.messaging.MessageHandlingException: ; nested exception is java.lang.RuntimeException: test crash
at org.springframework.integration.transformer.MessageTransformingHandler.handleRequestMessage(MessageTransformingHandler.java:74) ~[spring-integration-core-4.1.2.RELEASE.jar:na]
at org.springframework.integration.handler.AbstractReplyProducingMessageHandler.handleMessageInternal(AbstractReplyProducingMessageHandler.java:99) ~[spring-integration-core-4.1.2.RELEASE.jar:na]
at org.springframework.integration.handler.AbstractMessageHandler.handleMessage(AbstractMessageHandler.java:78) ~[spring-integration-core-4.1.2.RELEASE.jar:na]
at org.springframework.integration.dispatcher.AbstractDispatcher.tryOptimizedDispatch(AbstractDispatcher.java:116) ~[spring-integration-core-4.1.2.RELEASE.jar:na]
at org.springframework.integration.dispatcher.UnicastingDispatcher.doDispatch(UnicastingDispatcher.java:101) ~[spring-integration-core-4.1.2.RELEASE.jar:na]
at org.springframework.integration.dispatcher.UnicastingDispatcher.dispatch(UnicastingDispatcher.java:97) ~[spring-integration-core-4.1.2.RELEASE.jar:na]
at org.springframework.integration.channel.AbstractSubscribableChannel.doSend(AbstractSubscribableChannel.java:77) ~[spring-integration-core-4.1.2.RELEASE.jar:na]
at org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:277) ~[spring-integration-core-4.1.2.RELEASE.jar:na]
at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:115) ~[spring-messaging-4.1.4.RELEASE.jar:4.1.4.RELEASE]
at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:45) ~[spring-messaging-4.1.4.RELEASE.jar:4.1.4.RELEASE]
at org.springframework.messaging.core.AbstractMessageSendingTemplate.send(AbstractMessageSendingTemplate.java:95) ~[spring-messaging-4.1.4.RELEASE.jar:4.1.4.RELEASE]
at org.springframework.integration.handler.AbstractMessageProducingHandler.sendOutput(AbstractMessageProducingHandler.java:248) ~[spring-integration-core-4.1.2.RELEASE.jar:na]
at org.springframework.integration.handler.AbstractMessageProducingHandler.produceOutput(AbstractMessageProducingHandler.java:171) ~[spring-integration-core-4.1.2.RELEASE.jar:na]
at org.springframework.integration.handler.AbstractMessageProducingHandler.sendOutputs(AbstractMessageProducingHandler.java:119) ~[spring-integration-core-4.1.2.RELEASE.jar:na]
at org.springframework.integration.aggregator.AbstractCorrelatingMessageHandler.completeGroup(AbstractCorrelatingMessageHandler.java:657) ~[spring-integration-core-4.1.2.RELEASE.jar:na]
at org.springframework.integration.aggregator.AbstractCorrelatingMessageHandler.completeGroup(AbstractCorrelatingMessageHandler.java:642) ~[spring-integration-core-4.1.2.RELEASE.jar:na]
at org.springframework.integration.aggregator.AbstractCorrelatingMessageHandler.expireGroup(AbstractCorrelatingMessageHandler.java:619) ~[spring-integration-core-4.1.2.RELEASE.jar:na]
at org.springframework.integration.aggregator.AbstractCorrelatingMessageHandler.forceComplete(AbstractCorrelatingMessageHandler.java:543) ~[spring-integration-core-4.1.2.RELEASE.jar:na]
at org.springframework.integration.aggregator.AbstractCorrelatingMessageHandler$ForceReleaseMessageGroupProcessor.processMessageGroup(AbstractCorrelatingMessageHandler.java:721) ~[spring-integration-core-4.1.2.RELEASE.jar:na]
at org.springframework.integration.aggregator.AbstractCorrelatingMessageHandler$1.execute(AbstractCorrelatingMessageHandler.java:168) ~[spring-integration-core-4.1.2.RELEASE.jar:na]
at org.springframework.integration.store.AbstractMessageGroupStore.expire(AbstractMessageGroupStore.java:169) ~[spring-integration-core-4.1.2.RELEASE.jar:na]
at org.springframework.integration.store.AbstractMessageGroupStore.expireMessageGroups(AbstractMessageGroupStore.java:113) ~[spring-integration-core-4.1.2.RELEASE.jar:na]
at org.springframework.integration.store.MessageGroupStoreReaper.run(MessageGroupStoreReaper.java:115) ~[spring-integration-core-4.1.2.RELEASE.jar:na]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.7.0_71]
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) ~[na:1.7.0_71]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) ~[na:1.7.0_71]
at java.lang.reflect.Method.invoke(Unknown Source) ~[na:1.7.0_71]
at org.springframework.scheduling.support.ScheduledMethodRunnable.run(ScheduledMethodRunnable.java:65) ~[spring-context-4.0.8.RELEASE.jar:4.0.8.RELEASE]
at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54) ~[spring-context-4.0.8.RELEASE.jar:4.0.8.RELEASE]
at org.springframework.scheduling.concurrent.ReschedulingRunnable.run(ReschedulingRunnable.java:81) [spring-context-4.0.8.RELEASE.jar:4.0.8.RELEASE]
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) [na:1.7.0_71]
at java.util.concurrent.FutureTask.run(Unknown Source) [na:1.7.0_71]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(Unknown Source) [na:1.7.0_71]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) [na:1.7.0_71]
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) [na:1.7.0_71]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) [na:1.7.0_71]
at java.lang.Thread.run(Unknown Source) [na:1.7.0_71]
DEBUG [exec-1] - Expiring all messages older than timeout=10 from message group store: org.springframework.integration.jdbc.JdbcMessageStore#3e082583
DEBUG [exec-1] - Executing prepared SQL query
DEBUG [exec-1] - Executing prepared SQL statement [SELECT distinct GROUP_KEY as CREATED from INT_MESSAGE_GROUP where REGION=?]
DEBUG [exec-1] - Fetching JDBC Connection from DataSource
DEBUG [exec-1] - Creating new JDBC DriverManager Connection to [jdbc:mysql://localhost:3306/aggregator]
DEBUG [exec-1] - Returning JDBC Connection to DataSource
Well, looks like you go right way.
You can enclose you aggregator to the TX for JMS and JDBC and group messages by desired correlationKey, e.g. correlation-key-expression.
Since you don't want to release groups until some daily event (e.g. cron trigget) you should mark your aggregator with options:
release-strategy-expression="false"
send-partial-result-on-expiry="true"
expire-groups-upon-completion="true"
the first one: do not allow the normal release
the second: send the aggregator result to the output-channel, but not discard-channel on expire
the third: remove groups from the MessageStore to allow to form the fresh on for the same correlationKey
To make it worked you should configure MessageGroupStoreReaper:
<bean id="reaper" class="org.springframework.integration.store.MessageGroupStoreReaper">
<property name="messageGroupStore" ref="messageStore"/>
<property name="timeout" value="10"/>
</bean>
<task:scheduled-tasks scheduler="scheduler">
<task:scheduled ref="reaper" method="run" cron="0 0 * * * * "/>
</task:scheduled-tasks>
<task:scheduler id="scheduler"/>
The Reaper invokes the MessageGroupCallback which is registered during aggregator initialization. That callback invokes forceComplete on the aggregator to allow to work all those mentioned options.
Starting with SI-4.1 <aggregator> supports <expire-transactional> sub-element which allow to wrap that forceComplete to the TX, like you asked.
Previously we need to wrap to the TX (e.g. using <tx:advice>) the MessageGroupStoreReaper.run() method.
Hope I am clear.
UPDATE
My apologies, I see that we have a bug. In documentation about <expire-transactional> we say:
Allows a transaction to be started for the forceComplete operation. It is initiated from a group-timeout(-expression) or by a MessageGroupStoreReaper and is not applied to the normal add/release/discard operations. Only this sub-element or <expire-advice-chain/> is allowed.
But it isn't true. The code looks likes:
if (this.groupTimeoutExpression != null && !CollectionUtils.isEmpty(this.forceReleaseAdviceChain)) {
ProxyFactory proxyFactory = new ProxyFactory(processor);
for (Advice advice : this.forceReleaseAdviceChain) {
proxyFactory.addAdvice(advice);
}
return (MessageGroupProcessor) proxyFactory.getProxy(getApplicationContext().getClassLoader());
}
So, we apply the <expire-transactional> only if group-timeout(-expression) is provided, but it isn't a case for the MessageGroupStoreReaper.
Feel free to raise a JIRA issue on the matter and we'll take care of it soon.
In meanwhile as a work around you should use <tx:advice> for the MessageGroupStoreReaper.run().