The workflow of my application is adding a message header, then routing the message depending on another header and sending the message further downstream. My tests for this workflow succeed individually but not when executed together, although it seems that I have reset all relevant objects.
The spring integration workflow:
<int:header-enricher input-channel="workflowStart" output-channel="headerEnriched">
<int:header name="correlationId" expression="headers.id.toString()" />
</int:header-enricher>
<int:header-value-router input-channel="headerEnriched" header-name="messageType">
<int:mapping value="stp" channel="stpChannel" />
<int:mapping value="nonStp" channel="nonStpChannel" />
</int:header-value-router>
<int:publish-subscribe-channel id="nonStpChannel" />
<int:publish-subscribe-channel id="stpChannel" apply-sequence="true"/>
<int:chain input-channel="nonStpChannel" output-channel="systemChannel1" id="nonStpChainBlockA">
<!-- do something -->
</int:chain>
<int:chain input-channel="stpChannel" output-channel="systemChannel3" id="stpChainBlockA">
<!-- do something -->
</int:chain>
The Java test class:
#Autowired
private MessageChannel workflowStart;
#Autowired
private MessageHandler systemChannel1OutputHandler;
#Autowired
private MessageHandler systemChannel3OutputHandler;
#Value("/xml/tradeStp.xml")
private Resource stpMessageResource;
private String stpMessage;
#Value("/xml/tradeNonStp.xml")
private Resource nonStpMessageResource;
private String nonStpMessage;
#Before
public void setup() throws Exception {
reset(systemChannel1OutputHandler);
reset(systemChannel3OutputHandler);
stpMessage = IOUtils.toString(stpMessageResource.getInputStream());
nonStpMessage = IOUtils.toString(nonStpMessageResource.getInputStream());
}
#Test
public void nonStpMessageWorkflow1Test() {
Message<String> m = MessageBuilder
.withPayload(nonStpMessage)
.setHeaderIfAbsent("messageType", "nonStp")
.build();
workflowStart.send(m);
verify(systemChannel1OutputHandler, times(1)).handleMessage(any(Message.class));
}
#Test
public void stpMessageWorkflow2Test() {
Message<String> m = MessageBuilder
.withPayload(stpMessage)
.setHeaderIfAbsent("messageType", "stp")
.build();
workflowStart.send(m);
verify(systemChannel3OutputHandler, times(1)).handleMessage(any(Message.class));
}
And the test context:
<import resource="classpath:/spring/workflow.xml"/>
<int:outbound-channel-adapter channel="systemChannel1" ref="systemChannel1OutputHandler" method="handleMessage"/>
<bean id="systemChannel1OutputHandler" class="org.mockito.Mockito" factory-method="mock">
<constructor-arg value="org.springframework.integration.core.MessageHandler"/>
</bean>
<int:outbound-channel-adapter channel="systemChannel3" ref="systemChannel3OutputHandler" method="handleMessage"/>
<bean id="systemChannel3OutputHandler" class="org.mockito.Mockito" factory-method="mock">
<constructor-arg value="org.springframework.integration.core.MessageHandler"/>
</bean>
<int:outbound-channel-adapter channel="headerEnriched" ref="headerEnricherOutputHandler" method="handleMessage"/>
<bean id="headerEnricherOutputHandler" class="org.mockito.Mockito" factory-method="mock">
<constructor-arg value="org.springframework.integration.core.MessageHandler"/>
</bean>
If I run both tests together, the first test succeeds, while the second one fails with the message:
Wanted but not invoked:
messageHandler.handleMessage(<any>);
-> at com.company.integration.com.company.export.config.InterfaceTest.stpMessageWorkflow2Test(InterfaceTest.java:81)
Actually, there were zero interactions with this mock.
I have tried to debug in various ways. The problem seems to be just after the header-value-router and before the chain. The last debug output for the second test is a "postSend" on the headerEnriched channel.
Any advice is very welcome.
UPDATE:
I failed to include an additional outputHandler that I have in my context for another test, which I have included now. By removing the headerEnricherOutputHandler all tests run fine. However, I still don't understand the reason why this causes problems when tests are being run together.
I've had problems with Mockito using any(class). Where you have
verify(systemChannel3OutputHandler, times(1)).handleMessage(any(Message.class));
does it work with if you remove the class, like:
verify(systemChannel3OutputHandler, times(1)).handleMessage(any());
I cannot reproduce your issue with the following, which does not appear materially different to your tests...
<int:header-value-router input-channel="headerEnriched" header-name="messageType">
<int:mapping value="stp" channel="stpChannel" />
<int:mapping value="nonStp" channel="nonStpChannel" />
</int:header-value-router>
<int:publish-subscribe-channel id="nonStpChannel" />
<int:publish-subscribe-channel id="stpChannel" apply-sequence="true"/>
<int:chain input-channel="stpChannel" output-channel="systemChannel1">
<int:transformer expression="'bar'"/>
</int:chain>
<int:chain input-channel="nonStpChannel" output-channel="systemChannel3">
<int:transformer expression="'bar'"/>
</int:chain>
<int:channel id="systemChannel1" />
<int:channel id="systemChannel3" />
<int:outbound-channel-adapter channel="systemChannel1" ref="systemChannel1OutputHandler" method="handleMessage"/>
<bean id="systemChannel1OutputHandler" class="org.mockito.Mockito" factory-method="mock">
<constructor-arg value="org.springframework.integration.core.MessageHandler"/>
</bean>
<int:outbound-channel-adapter channel="systemChannel3" ref="systemChannel3OutputHandler" method="handleMessage"/>
<bean id="systemChannel3OutputHandler" class="org.mockito.Mockito" factory-method="mock">
<constructor-arg value="org.springframework.integration.core.MessageHandler"/>
</bean>
.
#ContextConfiguration
#RunWith(SpringJUnit4ClassRunner.class)
public class Foo {
#Autowired
private MessageChannel headerEnriched;
#Autowired
private MessageHandler systemChannel1OutputHandler;
#Autowired
private MessageHandler systemChannel3OutputHandler;
#Test
public void test10() {
Message<String> foo = MessageBuilder.withPayload("foo")
.setHeader("messageType", "stp").build();
headerEnriched.send(foo);
verify(systemChannel1OutputHandler).handleMessage(any(Message.class));
}
#Test
public void test30() {
Message<String> foo = MessageBuilder.withPayload("foo")
.setHeader("messageType", "nonStp").build();
headerEnriched.send(foo);
verify(systemChannel3OutputHandler).handleMessage(any(Message.class));
}
}
Both tests run fine for me.
Related
Migrating existing legacy application to java 11 and spring 5 and seems to be ServiceActivator method not invoking. I see publisher method invoking and printing log statement but no logs printing from ServiceActivator class
spring 5.3.14
spring-integration-* 5.5.7
XML Configuration:
<int:channel id="employeeServicesChannel">
<int:dispatcher task-executor="employeeServicesExecutor" />
</int:channel>
<bean id="employeeServicesExecutor"
class="org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor">
<property name="corePoolSize" value="10" />
<property name="maxPoolSize" value="40" />
<property name="queueCapacity" value="10" />
<property name="keepAliveSeconds" value="0"></property>
</bean>
public class EmployeeServicesPublisher {
#Publisher(channel = "employeeServicesChannel")
public EmployeeServicesDto publishEmployeeServicesRequest(EmployeeServicesDto requestDto) {
logger.info(" employee service request :" + requestDto);
return requestDto;
}
#MessageEndpoint
public class EmployeeServicesServerGateway {
#ServiceActivator(inputChannel = "employeeServicesChannel", outputChannel =
"esDynamicOutputChannel")
public EmployeeServicesDto processRequest(EmployeeServicesDto employeeServicesDto) {
logger.info("===================================");
logger.info(" request Id:" + employeeServicesDto.getRequestId());
return employeeServicesDto;
}
[EDIT] Uploading complete configs:
rabbit.xml which dequeues from rabbit
<rabbit:connection-factory id="amqpConnectionFactoryInbound"
host="${rabbit.host}" port="${rabbit.port}"
username="${rabbit.username}" password="${rabbit.password}" channel-
cache-size="5"
connection-factory="rabbitConnectionFactoryInbound"/>
<beans:bean id="rabbitConnectionFactoryInbound"
class="com.rabbitmq.client.ConnectionFactory">
<beans:property name="requestedHeartbeat"
value="60" />
</beans:bean>
<!-- Inbound Adapter to AMQP RabbitMq and write to file -->
<int-amqp:inbound-channel-adapter id="rabbitMQInboundChannelAdapter"
channel="rabbitInboundMessageChannel"
concurrent-consumers="8" task-
executor="rabbit-executor" connection-
factory="amqpConnectionFactoryInbound"
message-converter="byteArrayToStringConverter" queue-
names="${rabbit.queue}" acknowledge-mode="MANUAL" error-
channel="errorChannelId"
prefetch-count="25" />
<header-enricher input-channel="rabbitInboundMessageChannel" output-
channel="rabbitOutboundboundMessageChannel">
<int:header name="Operation" value="${operation.rabbit}" />
<int:header name="GUID" expression="#{
'T(java.util.UUID).randomUUID().toString()' }" />
<int:header name="operationStartTime" expression="#{
'T(java.lang.System).currentTimeMillis()' }" />
</header-enricher>
<int:channel id="rabbitOutboundboundMessageChannel">
<int:interceptors>
<int:wire-tap channel="loggerChannel" />
</int:interceptors>
</int:channel>
<task:executor id="rabbit-executor" rejection-policy="CALLER_RUNS"
pool-size="10-30"
queue-capacity="25" />
</beans:beans>
The message is then sent to router channel: router.xml
<int:header-enricher input-channel="rabbitOutboundboundMessageChannel"
output-channel="routerChannel">
<int:header name="Operation" value="${operation.router}"
overwrite="true" />
<int:header name="file_name" expression="headers['GUID'] + '.xml'" />
<int:header name="operationStartTime" expression="#{
'T(java.lang.System).currentTimeMillis()' }"
overwrite="true" />
<int:error-channel ref="errorChannelId" />
</int:header-enricher>
<int:recipient-list-router id="rabbitMsgrouter" input-
channel="routerChannel">
<int:recipient channel="fileBackupChannel" selector-expression="new
String(payload).length()>0" />
<int:recipient channel="transformerChannel" />
</int:recipient-list-router>
<int:channel id="transformerChannel">
<int:interceptors>
<int:wire-tap channel="loggerChannel" />
</int:interceptors>
</int:channel>
<int:channel id="fileBackupChannel"/>
<int:channel id="loggerChannel"/>
</beans>
The message is now sent to persister.xml and transformer.xml. The following is persister.xml and I want to ack if persistence is successful. There are other downstream processes after transformer.xml
<int:header-enricher input-channel="fileBackupChannel" output-
channel="fileSaveChannel">
<int:header name="Operation" value="${operation.filePersister}"
overwrite="true" />
<int:header name="replyChannel" value="nullChannel" />
<int:header name="operationStartTime" expression="#{
'T(java.lang.System).currentTimeMillis()' }" />
<int:error-channel ref="errorChannelId" />
</int:header-enricher>
<int-file:outbound-gateway id="fileBackUpChannelAdapter"
directory="${file.location}"
request-channel="fileSaveChannel" reply-channel="rabbitAckChannel"/>
<int:service-activator input-channel="rabbitAckChannel" output-
channel="nullChannel" ref="ackRabbit" method="handleRabbitAcks" />
<bean id="ackRabbit"
class="com.expedia.dataloader.rabbit.RabbitAcknowledgement"/>
<int:channel id="rabbitAckChannel">
<int:interceptors>
<int:wire-tap channel="loggerChannel" />
</int:interceptors>
</int:channel>
<int:channel id="loggerChannel"/>
<int:channel id="fileSaveChannel"/>
</beans>
I'm having trouble manually acking payloads from rabbitmq.
This is my work flow:
1. Get message from rabbit using inbound-channel-adapter:
<int-amqp:inbound-channel-adapter id="rabbitMQInboundChannelAdapter"
channel="rabbitInboundMessageChannel"
concurrent-consumers="${rabbit.concurrentConsumers}" task-
executor="rabbit-executor" connection-
factory="amqpConnectionFactoryInbound"
message-converter="byteArrayToStringConverter" queue-
names="${rabbit.queue}" acknowledge-mode="MANUAL" error-
channel="errorChannelId"
prefetch-count="${rabbit.prefetchCount}" />
2. Persist message to disk using outbound-gateway:
<int-file:outbound-gateway id="fileBackUpChannelAdapter"
directory="${file.location}"
request-channel="fileSaveChannel" reply-channel="loggerChannel" />
3. ack from rabbit when persister (step 2) succeeds.
for step (3), i wrote the following code:
public class RabbitAcknowledgement {
public void handleRabbitAcks(Message<?> message) throws IOException {
com.rabbitmq.client.Channel channel = (Channel)
message.getHeaders().get("amqp_channel");
long deliveryTag = (long) message.getHeaders().get("amqp_deliveryTag");
channel.basicAck(deliveryTag, false);
}
which I'm calling from spring via:
<int:service-activator input-
channel="rabbitOutboundboundMessageChannel" output-
channel="routerChannel" ref="ackRabbit" method="handleRabbitAcks" />
This doesn't work and the the rabbit payloads in my queue are not acked.
My questions are:
Do I need MANUAL ack in this scenario?
What am I doing wrong?
It should work fine; I just ran a quick test and it works for me...
#SpringBootApplication
public class So44666444Application implements CommandLineRunner {
public static void main(String[] args) {
SpringApplication.run(So44666444Application.class, args).close();
}
#Autowired
private RabbitTemplate template;
private final CountDownLatch latch = new CountDownLatch(1);
#Override
public void run(String... args) throws Exception {
this.template.convertAndSend("foo", "bar");
latch.await();
}
#Bean
public AmqpInboundChannelAdapter adapter(ConnectionFactory cf) {
AmqpInboundChannelAdapter adapter = new AmqpInboundChannelAdapter(listenerContainer(cf));
adapter.setOutputChannelName("ack");
return adapter;
}
#Bean
public AbstractMessageListenerContainer listenerContainer(ConnectionFactory cf) {
SimpleMessageListenerContainer container = new SimpleMessageListenerContainer(cf);
container.setAcknowledgeMode(AcknowledgeMode.MANUAL);
container.setQueueNames("foo");
return container;
}
#ServiceActivator(inputChannel = "ack")
public void ack(#Header(AmqpHeaders.CHANNEL) Channel channel, #Header(AmqpHeaders.DELIVERY_TAG) Long tag)
throws IOException {
System.out.println("Acking: " + tag);
channel.basicAck(tag, false);
latch.countDown();
}
}
If I set a breakpoint on the basicAck, I see the message as unacked on the console; stepping over to the next line and the message is removed.
I'm new to spring integration and caching and wanted to figure out how to add object that I received from outbound-gateway into cache. Unable to figure out the configurations which are required.
From following configuration, object that I received from rest api is getting logged:
INFO: com.domain.IpAddress#74589991
I'm planning to use ehcache/caffiene, any pointers will be helpful.
<int:channel id="requestChannel"/>
<int:channel id="outboundChannel"/>
<int:channel id="replyChannel"/>
<int:channel id="autoCompleteJsonResponseChannel"/>
<int:gateway id="restResponseService" default-request-channel="requestChannel" service-interface="com.domain.service.RestResponseService" />
<int-http:outbound-gateway id="out" request-channel="requestChannel"
http-method="GET"
url="https://api.ipify.org/?format=json"
expected-response-type="java.lang.String"
reply-channel="autoCompleteJsonResponseChannel"
/>
<int:json-to-object-transformer
input-channel="autoCompleteJsonResponseChannel"
type="com.domain.IpAddress"
output-channel="outboundChannel" />
<int:logging-channel-adapter id="outboundChannel" level="INFO"/>
Edit 2:
Now I changed the outbound gateway as suggested:
<int-http:outbound-gateway id="out" request-channel="requestChannel"
http-method="GET"
url="https://api.ipify.org/?format=json"
expected-response-type="java.lang.String"
reply-channel="autoCompleteJsonResponseChannel">
<int-http:request-handler-advice-chain>
<cache:advice>
<cache:caching cache="myCache">
<cache:cacheable method="getCurrentIpAddress" />
</cache:caching>
</cache:advice>
</int-http:request-handler-advice-chain>
</int-http:outbound-gateway>
and defined the ehcache configuration as follows:
<bean id="cacheManager" class="org.springframework.cache.ehcache.EhCacheCacheManager" >
<property name="cacheManager" ref="ehcache" />
</bean>
<!-- Ehcache library setup -->
<bean id="ehcache" class="org.springframework.cache.ehcache.EhCacheManagerFactoryBean" >
<property name="configLocation" value="classpath:ehcache.xml" />
<property name="shared" value="true"/>
</bean>
In my service class, defined the cacheable methods:
package com.domain.service;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.cache.annotation.CacheEvict;
import org.springframework.cache.annotation.Cacheable;
import org.springframework.stereotype.Service;
import com.domain.IpAddress;
import com.domain.RestResponse;
#Service("restResponseService")
public class RestResponseServiceImpl implements RestResponseService{
#Autowired
private IpAddress ipAddress;
#Cacheable("myCache")
public IpAddress getCurrentIpAddress(String name) {
// TODO Auto-generated method stub
return ipAddress;
}
#CacheEvict(value = "myCache", allEntries = true)
public void refreshAllResults() {
// TODO Auto-generated method stub
}
}
But still caching is not working. What I'm missing here.
You can do like this in your gateway:
<int-http:outbound-gateway>
<int-http:request-handler-advice-chain>
<cache:advice>
<cache:caching cache="foo">
<cache:cacheable method="handle*Message" key="#a0.payload"/>
</cache:caching>
</cache:advice>
</int-http:request-handler-advice-chain>
</int-http:outbound-gateway>
In other words, since Caching in Spring in fully based on the AOP, it isn't so complicated to figure out that request-handler-advice-chain can use any Advice implementation. In this case we make <cache:cacheable> advice to work for us via proper method selection and caching key as an incoming payload.
I am trying to convert the spring batch configuration from xml based to annotation based.
Below is my xml based configuration.
<bean id="jobRepository" class="org.springframework.batch.core.repository.support.MapJobRepositoryFactoryBean" />
<bean id="jobLauncher" class="org.springframework.batch.core.launch.support.SimpleJobLauncher">
<property name="jobRepository" ref="jobRepository" />
</bean>
<!-- Step will need a transaction manager -->
<bean id="transactionManager"
class="org.springframework.batch.support.transaction.ResourcelessTransactionManager" />
<bean id="dbMapper" class="org.test.DBValueMapper">
</bean>
<bean id="dbMapperFlatfile" class="org.test.FlatFileRowMapper">
</bean>
<bean id="paramSetter" class="org.test.DBParamSetter">
</bean>
<bean id="dbReader" class="org.test.DBValueReader"
scope="step">
<property name="paramSetter" ref="paramSetter"/>
<property name="verifyCursorPosition" value="false" />
<property name="dataSource" ref="dataSource" />
<property name="sql" value="#{jobParameters['SQL_QUERY']}" />
<property name="rowMapper" ref="dbMapper" />
<property name="fetchSize" value="5000" />
</bean>
<bean id="dbWriterIO" class="org.test.TemplateWritterIO"
scope="step">
<property name="velocityEngine" ref="velocityEngine" />
<!-- <property name="rptConfig" value="#{jobParameters['RPT_CONFIGVAL']}" /> -->
<property name="headerCallback" ref="dbWriterIO" />
<property name="footerCallback" ref="dbWriterIO" />
</bean>
<batch:job id="fileGenJobNio">
<batch:step id="fileGenJobStempNio">
<batch:tasklet>
<batch:chunk reader="dbReader" writer="dbWriterNIO"
commit-interval="5000">
</batch:chunk>
</batch:tasklet>
</batch:step>
</batch:job>
Below is the equivalent Java based configuration:
#EnableBatchProcessing
#Import({ServiceConfiguration.class})
public class SRBatchGenerator extends DefaultBatchConfigurer{
#Autowired
private JobBuilderFactory jobBuilders;
#Autowired
private StepBuilderFactory stepBuilders;
#Autowired
private VelocityEngine velocityEngine;
#Autowired
private DBValueMapper mapper;
#Autowired
private DbHelper dbhelper;
#Autowired
private DataSource datasource;
#Bean
public Step step(){
return stepBuilders.get("step")
.chunk(5000)
.reader(reader())
//.processor(processor())
.writer(writer())
//.listener(logProcessListener())
.faultTolerant()
//.skipLimit(10)
//.skip(UnknownGenderException.class)
//.listener(logSkipListener())
.build();
}
#Bean
public Job fileGeneratorJob(){
return jobBuilders.get("fileGeneratorJob")
//.listener(protocolListener())
.start(step())
.build();
}
#Bean
public DBValueMapper mapper(){
return new DBValueMapper();
}
#Bean
#StepScope
public DBValueReader3 reader(){
String query="Select Test1,Test2,test3,test4 from RPt_TEST";
DBValueReader3 dbread = new DBValueReader3();
dbread.setSql(query);
dbread.setRowMapper(mapper);
dbread.setDataSource(datasource);
return dbread;
}
#Bean
#StepScope
public TemplateWritterIO writer(){
TemplateWritterIO writer=new TemplateWritterIO();
writer.setVelocityEngine(velocityEngine);
return writer;
}
#Override
protected JobRepository createJobRepository() throws Exception {
MapJobRepositoryFactoryBean factory =
new MapJobRepositoryFactoryBean();
factory.afterPropertiesSet();
return (JobRepository) factory.getObject();
}
}
When I execute my Job using xml based it took 27sec to write 1 Million record into Flat file.
But to write same 1 million record, Java based job took about 2 hours to write.
I am not sure what I am missing here. Can anyone help me or guide me why it is slow in Java based configuration.
I have such method
#JmsListener(containerFactory = "jmsListenerContainerFactory", destination = "myQName")
public void rceive(MySerializableObject message) {
log.info("received: {}", message);
}
and such config on xml
<jms:annotation-driven />
<bean id="jmsListenerContainerFactory" class="org.springframework.jms.config.DefaultJmsListenerContainerFactory">
<property name="connectionFactory" ref="pooledConnectionFactory" />
<property name="concurrency" value="3-10" />
</bean>
<bean id="jmsConnectionFactory" class="org.apache.activemq.ActiveMQConnectionFactory">
<property name="brokerURL" value="${brokerURL}" />
</bean>
<bean id="pooledConnectionFactory" class="org.apache.activemq.pool.PooledConnectionFactory">
<property name="maxConnections" value="5" />
<property name="maximumActiveSessionPerConnection" value="500" />
<property name="connectionFactory" ref="jmsConnectionFactory" />
</bean>
<bean id="jmsContainerFactory" class="org.springframework.jms.config.DefaultJmsListenerContainerFactory">
<property name="connectionFactory" ref="pooledConnectionFactory" />
<property name="concurrency" value="3-10" />
</bean>
<bean id="jmsTemplate" class="org.springframework.jms.core.JmsTemplate" p:connectionFactory-ref="pooledConnectionFactory" />
Seems consumer was not created. I can send messages but cannot receive them.
Any idea whats wrong here?
Just tested your config and it works well. Only difference that I make a class with #JmsListener as a <bean> in that context:
<bean class="org.springframework.integration.jms.JmsListenerAnnotationTests$TestService"/>
#ContextConfiguration
#RunWith(SpringJUnit4ClassRunner.class)
#DirtiesContext
public class JmsListenerAnnotationTests {
#Autowired
private JmsTemplate jmsTemplate;
#Autowired
private TestService testService;
#Test
public void test() throws InterruptedException {
this.jmsTemplate.convertAndSend("myQName", "foo");
assertTrue(this.testService.receiveLatch.await(10, TimeUnit.SECONDS));
assertNotNull(this.testService.received);
assertEquals("foo", this.testService.received);
}
public static class TestService {
private CountDownLatch receiveLatch = new CountDownLatch(1);
private Object received;
#JmsListener(containerFactory = "jmsListenerContainerFactory", destination = "myQName")
public void receive(String message) {
this.received = message;
this.receiveLatch.countDown();
}
}
}
<jms:annotation-driven /> makes the #JmsListener infrastructure available, but to force Spring to see those methods your classes should be beans anyway.
For example <component-scan> for the package with #Service classes.
Cheers!