I am new to Spring Integration & Redis, So my apology if I am making a naive mistake.
My requirement is as below -
Need to implement a message queue. For dispatching money to the user based on some events.
The queue should not be volatile and Ensuring Atomicity. If I restarted the server or it crashed it should not lose the event message. This also includes messages which are currently in progress.
The queue should deliver the message granted once and only once. It will be a multithreaded(workers) and multi-server environment.
My progress till now is - configured Spring Integration and Spring Integration Redis in my spring project. My Spring Integration config as below -
<int-redis:queue-outbound-channel-adapter
id="event-outbound-channel-adapter"
channel="eventChannelJson"
serializer="serializer"
auto-startup="true" connection-factory="redisConnectionFactory"
queue="my-event-queue" />
<int:gateway id="eventChannelGateway"
service-interface="com.test.RedisChannelGateway"
error-channel="errorChannel" default-request-channel="eventChannel">
<int:default-header name="topic" value="queue"/>
</int:gateway>
<int:channel id="eventChannelJson"/>
<int:channel id="eventChannel">
<int:queue/>
</int:channel>
<bean id="serializer" class="org.springframework.data.redis.serializer.StringRedisSerializer"/>
<int:object-to-json-transformer input-channel="eventChannel"
output-channel="eventChannelJson"/>
<int-redis:queue-inbound-channel-adapter id="event-inbound-channel-adapter"
channel="eventChannelJson" queue="my-event-queue"
serializer="serializer" auto-startup="true"
connection-factory="redisConnectionFactory"/>
<bean id="serializer" class="org.springframework.data.redis.serializer.StringRedisSerializer"/>
<int:json-to-object-transformer input-channel="eventChannelJson"
output-channel="eventChannel"
type="com.test.PostPublishedEvent"/>
<int:service-activator input-channel="eventChannel" ref="RedisEventProcessingService"
method="process">
<int:poller fixed-delay="10" time-unit="SECONDS" max-messages-per-poll="500"/>
</int:service-activator>
I read an article on similar topic where they were using redis RPOPLPUSH for this purpose. But I am unable too figure out how to do it in Spring Integration.
Link for the article is - https://redis.io/commands/RPOPLPUSH
Please advise me regading this. I will realy appricate your help.
Spring integration does not have a component that utilizes that operation.
To use it, you should wrap a RedisTemplate in a <int:service-activator/> and call one of its rightPopAndLeftPush() methods.
Related
Could you tell me the way to log the SOAP message of , even if the message is not correct format ?
<sws:interceptors>
<bean class="com.capgemini.manulife.integration.interceptor.LogInterceptor" />
</sws:interceptors>
<!-- inbound -->
<ws:inbound-gateway id="cas-inbound-gateway" request-channel="casRequestChannel" reply-channel="casResponseChannel"
marshaller="casMarshaller" unmarshaller="casMarshaller" />
<int:channel id="casRequestChannel">
<int:interceptors>
<int:wire-tap channel="SOAPLogChannel"/>
</int:interceptors>
</int:channel>
<int:channel id="SOAPLogChannel" />
<int:logging-channel-adapter id="logger" expression="payload" level="INFO" channel="SOAPLogChannel"/>
As you see, I already using PayloadLoggingInterceptor (LogInterceptor extends PayloadLoggingInterceptor ), but it did not go through PayloadLoggingInterceptor
Thank you and best regards,
It's not Spring Integration question, but more up to SOAP directly.
So, if your XML isn't with correct format, you don't have choice unless check it manually before unmarshalling.
So, you have to rely on the simple <ws:inbound-gateway> do the <int-xml:validating-filter> (or some other check) and only after that perform <int-xml:unmarshalling-transformer>.
Yes, we can add Filter to Spring Web Service using FilterRegistrationBean ^^
I want to create programmatically the following XML config on demand:
<int-mqtt:message-driven-channel-adapter id="inboundAdapter"
client-id="${mqtt.client.id}"
url="${mqtt.broker.url}"
topics="${mqtt.subscribed.topics}"
client-factory="clientFactory"
channel="input-channel-1" converter="customConverter" />
<int:channel id="input-channel-1">
<int:queue/>
<int:interceptors>
<int:wire-tap channel="logger"/>
<int:ref bean="messageListener"/>
</int:interceptors>
</int:channel>
<int:channel id="logger" />
<int:logging-channel-adapter channel="logger"
auto-startup="true" level="INFO" id="loggerAdapter" log-full-message="true" />
What i can do is the following
CustomMqttPahoMessageDrivenChannelAdapter adapter = new CustomMqttPahoMessageDrivenChannelAdapter(url, clientId, topic);
adapter.setOutputChannel(outputChannel);
adapter.setConverter(ctx.getBean("customConverter", MyPahoMessageConverter.class));
Now I need to add interceptor bean through which each client will get notified when message arrives according to their subscribed topics respectively.
What I am trying to achieve is:
1) Create mqtt adapter when a client connects to server.(each client will subscribe to different topics as per configuration)
2) Dispose mqtt adapter when client disconnects.
Can anyone help me on this?
It's not clear what you are trying to do; what do you have downstream of the input-channel-1 in the XML configuration.
What does messageListener do?
It is an anti-pattern to put business logic in a channel; unless it's something really lightweight, consider invoking it using a <service-activator/> instead - possibly by making input-channel-1 a pub-sub channel.
To answer your simple question, to add the interceptor, you can use outputChannel.addInterceptor(ctx.getBean("messageListener", ChannelInterceptor.class));.
When message Listener in Spring integration is started, it is pulling messages and removing from the source queue, How to stop the delete of messages from source queue?
Make your listener transactional and rollback transaction, when the message from queue will be in your hands:
<int-jms:message-driven-channel-adapter channel="input"
connection-factory="connectionFactory"
transaction-manager="transactionManager"
destination-name="MY-QUEUE"/>
<int:publish-subscribe-channel id="input" />
<int:service-activator input-channel="input" order="1" ref="service" output-channel="output"/>
<int:outbound-channel-adapter channel="input" order="1"
expression="T(org.springframework.transaction.interceptor.TransactionAspectSupport)
.currentTransactionStatus().setRollbackOnly()"/>
But here is need to understand what are you going to do with that remained message in the queue: it becomes available for Listener on the next poll...
I have a production application that seems to be losing messages intermittently. One example that I have identified is the sending system logs 3 messages that are sent milliseconds apart. The receiving system logs that it got message 1 and message 3...but not message 2. The queue's depth is 0. I see no errors in the logs of the application that reads the messages to indicate something bad happened.
I have verified that there are no other "rogue" clients creating a race condition for reading messages.
Here is my configuration; we have a primary and secondary/failover queue...
<bean id="primaryProficiencyInboundQueue" class="com.ibm.mq.jms.MQQueue">
<property name="baseQueueManagerName" value="${lsm.primary.outbound.manager}"/>
<property name="baseQueueName" value="${lsm.proficiency.inbound.queue}"/>
</bean>
<bean id="secondaryProficiencyInboundQueue" class="com.ibm.mq.jms.MQQueue">
<property name="baseQueueManagerName" value="${lsm.secondary.outbound.manager}"/>
<property name="baseQueueName" value="${lsm.proficiency.inbound.queue}"/>
</bean>
<int:channel id="inboundProficiencyChannel">
<int:interceptors>
<int:wire-tap channel="logger" />
</int:interceptors>
</int:channel>
<!-- adapter that connects the inbound proficiency queues to the inboundProficiencyChannel -->
<jms:message-driven-channel-adapter id="primaryProficiencyInboundAdapter"
connection-factory="primaryConnectionFactory"
destination="primaryProficiencyInboundQueue"
channel="inboundProficiencyChannel" />
<jms:message-driven-channel-adapter id="secondaryProficiencyInboundAdapter"
connection-factory="secondaryConnectionFactory"
destination="secondaryProficiencyInboundQueue"
channel="inboundProficiencyChannel" />
<int:service-activator id="proficiencyInboundActivator"
input-channel="inboundProficiencyChannel">
<bean class="com.myactivator.LSMInboundProficiencyActivator" />
</int:service-activator>
Is it possible that something is failing silently?
One other noteworthy thing is that there are two production servers that are both configured as above....meaning that there are two servers that have JMS adapters configured to read the messages. I mentioned above that I see message 1 and message 3 being read. In that case, I see server one process message one and server two process mesage two...so it seems that having two servers configured like above has no negative impact. Any other thoughts on what could be happening or how I can debug the case of the missing/dropped messages?
Version information:
Spring - 3.0.5.RELEASE
Spring Integration - 2.0.3.RELEASE
Nothing springs to mind that would cause lost messages; I don't recall that ever being reported; but 2.0.3 is quite ancient (3rd birthday coming up soon). We don't support 2.0.x any more but you should at least update to 2.0.6 which is the last release of 2.0.x. But preferably move up to the latest 3.0.0 release (or at least 2.2.6).
we are using Spring Integration 2.1 for persisting messages into database sent by clients.
There is a queue which will be filled by a custom adapter. The configured service activator polls this queue and releases the message to a spring managed #Repository bean. All errors will be captured to an error channel and will be handled by a service. The configuration works so far fine.
My concern is that if the database is not available the service-activators polls all incoming message from the queue and puts them into the error channel. Is there a way to prevent the service-activator to poll the message if the database is obviously not available, for example by sending a test query ?
My configuraton:
<int:channel id="inChannel">
<int:queue />
</int:channel>
<bean id="service" class="some.service.Service" />
<int:service-activator ref="service"
method="write" input-channel="inChannel">
<int:poller fixed-rate="100" task-executor="srvTaskExecutor"
receive-timeout="90" error-channel="errChannel" />
</int:service-activator>
<task:executor id="srvTaskExecutor" pool-size="2-10"
queue-capacity="0" rejection-policy="DISCARD" />
<int:channel id="errChannel" />
<int:service-activator input-channel="errChannel"
ref="errorService" method="write"/>
Regards.
If you give the polling service-activator an "id", you can refer to that instance and call start() or stop() on it based on the DB being available or not. Most likely you'd want to set auto-startup="false" on that service-activator as well.
Additionally, you can even define a "control-bus" element and then send messages like "myActivator.start()" and "myActivator.stop()" to that control bus' input-channel.
Hope that helps,
Mark