How can I publish messages with different topics programmatically?
<mqtt:outbound-channel-adapter id="mqttOut"
auto-startup="true"
client-id="foo"
url="tcp://localhost:1883"
client-factory="clientFactory"
default-qos="0"
default-retained="false"
default-topic="bar"
async="true"
async-events="true" />
I tried Spring integration MQTT publish & subscribe to multiple topics, but were not able to configure.
Also tried with MqttPahoMessageHandlerAdapter which has a publish() but protected.
Going with org.eclipse.paho.client.mqttv3.MqttAsyncClient and org.eclipse.paho.client.mqttv3.MqttCallback is very easy. But I would like to stick with spring all the way.
Appreciate if somebody can points me to a correct direction.
Declare a <publish-subscribe-channel id="toMqtt" />; set it as the channel attribute on each outbound channel adapter; the message will be sent to each adapter.
You can do that with Spring Integration anyway! Having a lot of EIP components implementation and Spring power on board (injection, SpEL etc,), plus switching on a bit of imagination, we can reach any end-application requirements even without any Java code.
So, <mqtt:outbound-channel-adapter> allows determine topic at runtime. Instead of default-topic you should supply MqttHeaders.TOPIC MessageHeader.
So, if you have a requirement to send the same message to several topics, you just build a copy of that message for each topic. The <splitter> can help us:
<int:splitter input-channel="enricheMessage" output-channel="sendMessage" apply-sequence="false">
<int-groovy:script>
['topic1', 'topic2', 'topic3'].collect {
org.springframework.integration.support.MessageBuilder.withPayload(payload)
.copyHeaders(headers)
.setHeader(org.springframework.integration.mqtt.support.MqttHeaders.TOPIC, it)
.build()
}
</int-groovy:script>
</int:splitter>
sendMessage can be ExecutorChannel to achieve the parallel publishing.
UPDATE
You can achieve the same iteration and message enrichment logic with similar Java code using ref and method on <splitter>.
Of course, we can do that even with SpEL , but it will look a bit complex with Collection Projection.
Related
CLARIFICATION:
Thanks to #JustinBertram comment I realized that this question does not make sense.
STOMP protocol does not support selectors by itself, you have to use brokers such as ActiveMQ that implement it. STOMP supports headers that can be used by brokers for filtering messages by selectors.
In my case, I'm not using any broker, just frontend with Angular + Stomp + SocksJS and backend with Spring Boot, so I can't use selectors.
The documentation of STOMP protocol does not make this clear to me and I got confused. See these references:
the specification:
Stomp brokers may support the selector header which allows you to
specify an SQL 92 selector on the message headers which acts as a
filter for content based routing.
this article:
The subscribe() method takes an optional headers argument to specify
additional headers when subscribing to a destination:
var headers = {ack: 'client', 'selector': "location = 'Europe'"};
client.subscribe("/queue/test", message_callback, headers);
The client specifies that it will handle the message acknowledgement
and is interested to receive only messages matching the selector
location = 'Europe'.
I'm implementing a backend in Spring Boot. For two-way communications with the frontend I'm using stomp over websockets.
I have followed this Spring Boot + Angular example
It works, but one of my requirements is that the backend has to send messages with selectors so that the frontend subscribes to a topic and only receives the filtered data, to avoid performance issues with real time data.
i.e. { 'selector': "location = 'Europe'" }
For that purpose, I'm trying to make the backend send the messages with selectors, but I can't make it work.
I have followed this article to implement the frontend with selectors and it works correctly, the problem is only the backend.
I tried with #SendTo annotation but it seems it doesn't have any params for that as per the article:
#MessageMapping("/hello")
#SendTo("/topic/greetings")
public Greeting greeting(HelloMessage message) throws Exception {
Thread.sleep(1000);
return new Greeting("Hello, " + HtmlUtils.htmlEscape(message.getName()) + "!");
}
Also I tried with the MessagingTemplate, but I don't know how to set the selector properties in the header:
http://assets.spring.io/wp/WebSocketBlogPost.html
MessageSendingOperations<String> messagingTemplate;
messagingTemplate.convertAndSend(destination, quote);
I really appreciate any help, I have read many articles and docs but I don't find anything talking in particular about this with a solution.
Well, it's possible to use the JMS' selectors with Spring (Boot) Websocket and a STOMP client. I found a native way.
The key thing is that the selector is applied to the org.springframework.messaging.Message instance, and it uses Spring's Spel language to apply the condition (it's not the JMS SQL-like).
So using the default SimpMessagingTemplate, in the backend you can send header variables like this:
this.messagingTemplate.convertAndSend(
"/topic/something", //your destination
payload, //any kind of payload (body)
Map.of("id", 1) //header with key/value
);
In the frontend, to enter a selector that will be evaluated by the org.springframework.messaging.simp.broker.DefaultSubscriptionRegistry.filterSubscriptions, you must declare your Stomp/WebSockets headers as:
{"selector": "headers['nativeHeaders']['id'][0] == '999'"}
Yeah, it's horrible but it works.
As the default Message is GenericMessage, the headers are processed in a new key called "nativeHeaders".
The ['key'],[0] and == are Spring's Spel sintaxes.
Go ahead and filter your messages on the backend, not in the frontend, please!
The latest version of the STOMP specification doesn't include any specific statement about selectors and their syntax because it's really up to the broker implementation as to what is supported here. The specification now just states:
STOMP servers MAY support additional server specific headers to customize the delivery semantics of the subscription. Consult your server's documentation for details.
Brokers like ActiveMQ 5.x and ActiveMQ Artemis support the selector STOMP header and the syntax & behavior of the selector is based on JMS selectors.
Selectors in JMS are for selecting messages on consumption and are configured by the consuming client. You can't set the selector when sending the message.
JMS selectors select messages based on the headers or properties of the message, although some implementations go beyond this and allow selecting based on the content of the message itself. Therefore, if you want to have a selector location = 'Europe' on a consumer then you should set a header on the message when it is sent with the name location and the value of Europe.
The convertAndSend method is overloaded and provides a couple of ways to set a header:
Pass a map of key/value pairs to the convertAndSend method.
Implement a MessagePostProcessor and pass that to the convertAndSend method. Inside your post-processor you can invoke the javax.jms.Message#setStringProperty() method.
I have the use case that I need to wait 2 hours before consuming messages from an AMQP (we use Rabbit) queue.
EDIT: To clarify my use case... I need each individual message to wait 2 hours before being read. E.g. Message 1 arrives at 10am and Message 2 arrives at 10:15. I need Message 1 to be read at 12p and Message 2 to be read at 12:15p.
We are using Spring Integration 3.x.
The int-amqp:inbound-channel-adapter is message driven and doesn't have a polling option from what I can find.
A couple things I've thought of:
Set auto-startup to false and manually start the inbound channel adapter using a quartz job.
Create my own custom SimpleMessageListenerContainer that is based on polling (not sure how easy this would be)
Configure a "delay queue" in rabbitmq using this method: How to create a delayed queue in RabbitMQ?
EDIT: add 4th option: Use delayer to delay each message for 2 hours: http://docs.spring.io/spring-integration/docs/3.0.2.RELEASE/reference/html/messaging-endpoints-chapter.html#delayer
Any suggestions?
We don't currently have a polling inbound adapter. #1 is easy. For #2, the simplest would be to use a RabbitTemplate and invoke receive() from an inbound-channel-adapter in a POJO.
I would go with #1; you don't need quartz, you can use a simple Spring scheduled task and a control bus to start the adapter.
Another trick is about to use PollableAmqpChannel:
<int-amqp:channel id="myQueueName" message-driven="false"/>
and provide the <poller> for the subscriber to that channel.
There is no reason to send messages to that channel (because you will poll messages from Rabbit Queue) and, right, it looks like anti-pattern, but it is a hook how to avoid any workarounds with direct RabbitTemplate usage via SpEL.
UPDATE
<delayer> can help you, but it depends of your requirements. If you don't want to poll messages from RabbitMQ, you should use the workaround above. But if you just don't want to process message until some time is elapsed, you can just 'delay' it for that time.
Don't forget to add persistent message-store to avoid losing messages during that period and unexpected application failure.
FYI, how I solved the issue. (Used solution #3).
<rabbit:queue name="delayQueue" durable="true">
<rabbit:queue-arguments>
<entry key="x-message-ttl">
<value type="java.lang.Long">7200000</
</entry>
<entry key="x-dead-letter-exchange" value="finalDestinationTopic"/>
<entry key="x-dead-letter-routing-key" value="finalDestinationQueue"/>
</rabbit:queue-arguments>
</rabbit:queue>
<rabbit:topic-exchange name="finalDestinationTopic">
<rabbit:bindings>
<rabbit:binding queue="finalDestinationQueue" pattern="finalDestinationQueue"/>
</rabbit:bindings>
</rabbit:topic-exchange>
I'm working on abstracting out any sort of messaging framework for some code I'm working on. Basically, I'm using a combination of Spring AOP and Spring Integration to generate messages without the Java code knowing anything about RabbitMQ, JMS, or even Spring Integration. That said, what I'm using to generate the messages is contained in its own .jar, and it re-used by several other areas of the application. I currently have the messaging system set up such that the channels on which messages are sent are specified by the code that calls the system (i.e., channels are generated automatically based on the external method invocation) by specifying the channel name in the message header and using a header-value router to create the channels if they don't exist. My issue comes in on the endpoint of these channels - the intention of the current structure is to allow Spring to change to any messaging structure as requirements specify or change. I know how to take a static channel and use outbound channel converters/gateways to send it to a pre-specified RabbitMQ/JMS queue and process from there; what I'm struggling with is how to tell Spring that I need every channel created by the router to have a RabbitMQ (or whatever other messaging system gets implemented) outbound channel adapter that's dynamically generated based on the channel name since we don't know channel names beforehand.
Is this possible? And if not, would you mind providing input as to what could perhaps be a better way?
Thanks ahead of time!
Here's a basic template of what my config file looks like - I have an initial channel ("messageChannel") which gets sent to a publish-subscribe-channel and queuing channel depending on one of the message headers and is routed from there.
<!--Header value based channel configurations-->
<int:channel id="messageChannel" />
<int:channel id="queue" />
<int:publish-subscribe-channel id="topic" />
<!--Header-based router to route to queue or topic channels-->
<int:header-value-router input-channel="messageChannel"
header-name="#{ T(some.class.with.StringConstants).CHANNEL_TYPE}" />
<!--Re-routes messages according to their destination and messaging type-->
<int:header-value-router input-channel="queue"
header-name="#{ T(some.class.with.StringConstants).MESSAGE_DESTINATION}" />
<int:header-value-router input-channel="topic"
header-name="#{ T(some.class.with.StringConstants).MESSAGE_DESTINATION}" />
<!--AOP configuration - picks up on any invocation of some.class.which.generates.Messages.generateMessage()
from a Spring-managed context.-->
<aop:config>
<aop:pointcut id="eventPointcut"
expression="execution(* some.class.which.generates.Messages.generateMessage(..))" />
<aop:advisor advice-ref="interceptor" pointcut-ref="eventPointcut"/>
</aop:config>
<int:publishing-interceptor id="interceptor" default-channel="messageChannel">
<int:method pattern="generateMessage" payload="#return" channel="messageChannel" />
</int:publishing-interceptor>
See the dynamic-ftp sample; it uses a dynamic router that creates new outbound endpoints/channels on demand.
I am using
<int:inbound-channel-adapter id="dummyMessageA" channel="messages" method="getMessage" auto-startup="true" ref="messageGenerator">
<int:poller error-channel="errorChannel" fixed-rate="10000"/>
</int:inbound-channel-adapter>
<int:inbound-channel-adapter id="dummyNotif" channel="notifs" method="gtNotif" auto-startup="true" ref="notifGenerator">
<int:poller error-channel="errorChannel" fixed-rate="10000"/>
</int:inbound-channel-adapter>
These inbound channels are independent but when I deploy my Web Application, Only the second inbound channel adapter is taken into consideration (although the other one was working before adding the dummyNotif). Is this normal, should I add something in the config (NB : I don't aggregate the messages)
My guess you catched this issue https://jira.spring.io/browse/INT-3240 - 'Inbound Channel Adapter Parser doesn't generate unique bean Id for MessageSources'. That's mean that you use Spring Integration 3.0.
So, just upgrade to the latest - 3.0.2.RELEASE - and let us know.
UPDATE
Regarding the same id for several beans. By default Spring allow to do it and the last bean wins. All others will be ignored and skipped.
It can be disabled by AbstractRefreshableApplicationContext#setAllowBeanDefinitionOverriding(false).
From other side if you setup DEBUG logging level for org.springframework category you'll the message in the logs that your beans are overriden.
as far as your question is concerned, Spring Integration allows to have multiple inbound-channel-adapter definition in a single context.
However, from your comments, seems that you have some different issue in your configuration multiple Service Activators with same Id.
It can be disabled as #Artem described in his answer.
so I'm new to Spring Integration, and mostly to Spring as well, so I might not be up on all of the terminology, but I'm running across the following scenario:
I have a small Spring Integration application with three SI flows... each flow has its own Gateway, and each Gateway has its own request channel and reply channel. These flows receive a null invocation (for all intensive purposes... basically just a 'GO' signal / empty message) and reply with a status message, depending upon the (trivial) business logic results.
I would now like to wire each of these flows together to run in one 'master flow', given one request, without taking away their ability to run separately, and I'd like to wire it up completely through annotation / XML (IE. given a controller that invokes the main gateway's service interface, no additional code needs to be written outside of annotation / XML configuration.)
Is this feasible, what Integration components should I be using to do so, and/or should I just be adjusting the expected channels for each of these gateways to be meeting each other end-to-end (and if so, how would that strategy compensate to allow each of the flows to be called on a case-by-case basis)?
In addition, if this is not feasible, would it be appropriate to use a service activator to invoke each of the child flows? I wanted to avoid coding more, but if that is the only option, I guess that it'll have to do.
Thanks!
Probably the simplest way to do this is use Spring Profiles (a Spring 3.1 feature). When deployed in stand-alone mode, the final element can be a "bridge to nowhere"...
<int:bridge input-channel="app1Final" />
... when the final element in a flow has no output channel, the message is returned to the gateway's reply channel. If you prefer to explicitly configure the bridge to point to the gateway's reply-channel, that's ok too; it just not needed.
In the "linked" profile, you configure the bridge thus...
<int:bridge input-channel="app1Final" output-channel="app2Inbound"/>
...where app2Inbound is the same as that app's gateway's request-channel.
<beans profile="default">
<int:bridge input-channel="app1Final" />
</beans>
<beans profile="linked">
<int:bridge input-channel="app1Final" output-channel="app2Inbound"/>
</beans>
To run with the linked profile, set system property 'spring.profiles.active' to 'linked'