I have to use azurite as broker, unfortunately I can't use azure service bus yet. So I was forced to use
<dependency>
<groupId>com.microsoft.azure</groupId>
<artifactId>azure-servicebus-jms-spring-boot-starter</artifactId>
<version>2.3.3</version>
</dependency>
But with azurite container for JmsListener.
#JmsListener(destination = QUEUE_NAME, containerFactory = "jmsListenerContainerFactory")
public void receiveMessage(String message) {
logger.info("Received message: {}", message);
}
Inside factory, deeply nested in stackTrace is ConnectionStringBuilder which appends "amqps://";
and removes port number. The problem is that I don't know how to change messaging protocol in azurite for amqps from http and If it is not possible. How to overrite classes to use http not not amqps?
Azurite is intended for Azure Queue Storage, not Azure Service Bus. Azure Queue Storage is HTTP/S only and the tool was never designed to work with AMQP/S based services.
Related
I read everywhere #RefreshScope for cloud bus applications work with RabbitMQ and Kafka. But in my case, I am using AWS Parameter store. I want all my client instances to be refreshed automatically without rebuilding servers on AWS Console.
I created AWS Eventbridge from Paramstore to notify Kinesis Stream but I am not able to figure out how can it notify all my client nodes instead of load balancer refresh to only one node(instance).
Thank you for responding in advance.
I've never worked with AWS Eventbridge / Kinesis, however:
#RefreshScope is something that belongs to spring cloud and not not AWS.
More precisely, beans defined with this scope will be re-loaded by spring without reloading the whole application context "dynamically" when configuration changes in spring boot cloud configuration service. Usually this means that you don't have to restart the application.
Now, spring boot microservice should be deployed with actuator that exposes refresh endpoint. Calling this endpoint manually will cause all the #RefreshScope beans to reload.
Here is the source code of the RefreshEndpoint:
#Endpoint(id = "refresh")
public class RefreshEndpoint {
private ContextRefresher contextRefresher;
public RefreshEndpoint(ContextRefresher contextRefresher) {
this.contextRefresher = contextRefresher;
}
#WriteOperation
public Collection<String> refresh() {
Set<String> keys = this.contextRefresher.refresh();
return keys;
}
}
As you see, its merely invokes contextRefresher.refresh(), ContextRefresher is a bean that you can inject in your custom code that will listen to the changes coming from AWS Parameter store (it should somehow invoke it directly or maybe send some message that you could consume or something).
If you're using spring-cloud-bus (disclaimer, I've never worked with it) it exposes the bus-refresh endpoint as well (pretty similar mechanism to what I've described), read spring-cloud-bus documentation for more details.
Thank you Team for sharing info.
Here is what I did to make it work. Added these two libraries to my project
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-stream-binder-kinesis</artifactId>
<version>1.1.0.RELEASE</version>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-bus</artifactId>
</dependency>
And added these two entries into bootstrap.properties
cloud.aws.region.static=us-east-1
cloud.aws.stack.auto = false
And refreshing using this endpoint (/bus-refresh)
I am attempting to send data through IOWebSocketChannel in Flutter.io to a WebSocket created in Spring-Boot.
In spring-boot I have created the typical WebSocket config and controllers that are dealing with client's manipulation of my servers WebSocket. I will post them below just for reference.
WebSocketConfiguration.java
#Configuration
#EnableWebSocketMessageBroker
public class WebSocketConfiguration implements WebSocketMessageBrokerConfigurer{
#Override
public void registerStompEndpoints(StompEndpointRegistry registry){
registry.addEndpoint("/websocket")
.setAllowedOrigins("*") // allow us to connect to ws://localhost:8080/websocket with the default Spring port configuration.
.withSockJS(); // allows a client which does not support WebSocket natively mimic a WebSocket over an HTTP connection
}
#Override
public void configureMessageBroker(MessageBrokerRegistry registry){ //The configureMessageBroker method sets up a simple (in-memory) message broker for our application
registry.enableSimpleBroker("/topic"); //topic to be routed back to client
registry.setApplicationDestinationPrefixes("/app"); //This configuration allows Spring to understand that any message sent to a WebSocket channel name prefixed with /app should be routed to a #MessageMapping in our application.
}
}
WebSocketController.java
#Controller
public class WebSocketController {
private static final Logger LOGGER = LoggerFactory.getLogger(WebSocketController.class);
#MessageMapping("/send")
#SendTo("/topic/messages")
public Message send(Message message) {
LOGGER.info(String.format("Received message [%s]", message.toString()));
LocalDateTime timestamp = LocalDateTime.now();
return new Message(message.getFrom(), message.getMessage(), timestamp);
}
}
Now When I try using IOWebSocketChannel I perform the typical protocol of connecting to my configured websocket. Below is the code
final channel = IOWebSocketChannel.connect(
"ws://10.0.2.2:8080/websocket"
);
I have then created a method that is supposed to send data to my websocket so I attempt to connect to that endpoint which you see is created in WebSocketController.java called app/send/. Below is the code:
void _sendMessage() {
IOWebSocketChannel channel = IOWebSocketChannel.connect('ws://10.0.2.2:8080/app/send');
channel.sink.add(
json.encode({
"message": "bars",
})
);
}
Now when I check my Spring-Boot server nothing is logged, however, whenever I hot reload in Flutter Spring Boot and my connection to the websocket times out, tomcat server returns this:
So my question is if anybody has been able to make a breakthrough with sending data through websockets from Flutter into Spring-Boot using IOWebSocketChannel? I am also wondering if anyone has found a way to successfully use a STOMP protocol in Flutter.io? I was using stomp_client as it seemed like it was going to do the trick, however correct if I'm wrong, but flutter was giving me errors saying that there doesn't exist any html files, so I'm assuming that library is only for dart in the web.
Your Spring configuration looks good. But client-side you need some tweaks.
I spent some time to figure this out with the https://pub.dev/packages/stomp package. Use a modified version of the connect function provided here. Make sure to use this custom implementation of the connect function.
Future<StompClient> client = customStomp.connect('ws://10.0.2.2:8080/websocket', ...)
Once connected, according to your configuration, you can then send message on the following destination: /app/send.
I have a Spring Boot app (Jhipster) that uses STOMP over WebSockets to communicate information from the server to users.
I recently added an ActiveMQ server to handle scaling the app horizontally, with an Amazon auto-scaling group / load-balancer.
I make use the convertAndSendToUser() method, which works on single instances of the app to locate the authenticated users' "individual queue" so only they receive the message.
However, when I launch the app behind the load balancer, I am finding that messages are only being sent to the user if the event is generated on the server that their websocket-proxy connection (to the broker) is established on?
How do I ensure the message goes through ActiveMQ to whichever instance of the app that the user is actually "connected too" regardless of which instance receives, say an HTTP Request that executes the convertAndSendToUser() event?
For reference here is my StompBrokerRelayMessageHandler:
#Bean
public AbstractBrokerMessageHandler stompBrokerRelayMessageHandler() {
StompBrokerRelayMessageHandler handler = (StompBrokerRelayMessageHandler) super.stompBrokerRelayMessageHandler();
handler.setTcpClient(new Reactor2TcpClient<>(
new StompTcpFactory(orgProperties.getAws().getAmazonMq().getStompRelayHost(),
orgProperties.getAws().getAmazonMq().getStompRelayPort(), orgProperties.getAws().getAmazonMq
().getSsl())
));
return handler;
}
#Override
public void configureMessageBroker(MessageBrokerRegistry config) {
config.enableStompBrokerRelay("/queue", "/topic")
.setSystemLogin(orgProperties.getAws().getAmazonMq().getStompRelayHostUser())
.setSystemPasscode(orgProperties.getAws().getAmazonMq().getStompRelayHostPass())
.setClientLogin(orgProperties.getAws().getAmazonMq().getStompRelayHostUser())
.setClientPasscode(orgProperties.getAws().getAmazonMq().getStompRelayHostPass());
config.setApplicationDestinationPrefixes("/app");
}
I have found the name corresponding to the queue that is generated on ActiveMQ by examining the headers in the SessionSubscribeEvent, that is generated in the listener when a user subscribes to a user-queue, as simpSessionId.
#Override
#EventListener({SessionSubscribeEvent.class})
public void onSessionSubscribeEvent(SessionSubscribeEvent event) {
log.debug("Session Subscribe Event:" +
"{}", event.getMessage().getHeaders().toString());
}
Corresponding queues' can be found in ActiveMQ, in the format: {simpDestination}-user{simpSessionId}
Could I save the sessionId in a key-value pair and just push messages onto that topic channel?
I also found some possibilities of setting ActiveMQ specific STOMP properties in the CONNECT/SUBSCRIBE frame to create durable subscribers if I set these properties will Spring than understand the routing?
client-id & subcriptionName
Modifying the MessageBrokerReigstry config resolved the issue:
config.enableStompBrokerRelay("/queue", "/topic")
.setUserDestinationBroadcast("/topic/registry.broadcast")
Based on this paragraph in the documentation section 4.4.13:
In a multi-application server scenario a user destination may remain
unresolved because the user is connected to a different server. In
such cases you can configure a destination to broadcast unresolved
messages to so that other servers have a chance to try. This can be
done through the userDestinationBroadcast property of the
MessageBrokerRegistry in Java config and the
user-destination-broadcast attribute of the message-broker element in
XML
I did not see any documentation on "why" /topic/registry.broadcast was the correct "topic" destination, but I am finding various iterations of it:
websocket sessions sample doesn't cluster.. spring-session-1.2.2
What is MultiServerUserRegistry in spring websocket?
Spring websocket - sendToUser from a cluster does not work from backup server
In Spring Integration, I am using a couple of channel adapters for sending/receiving messages from a server socket. I always create client connections with the following adapters:
#Bean
public TcpReceivingChannelAdapter tcpIn(AbstractClientConnectionFactory connectionFactory) throws Exception {
TcpReceivingChannelAdapter receiver = new TcpReceivingChannelAdapter();
receiver.setOutputChannel(fromTcp ());
receiver.setConnectionFactory(connectionFactory);
return receiver;
}
#Bean
#ServiceActivator(inputChannel = "toTcp")
public TcpSendingMessageHandler tcpOut(AbstractClientConnectionFactory connectionFactory) throws Exception {
TcpSendingMessageHandler sender = new TcpSendingMessageHandler();
sender.setConnectionFactory(connectionFactory);
sender.setClientMode(true);
return sender;
}
The problem here is the server is answering in the same socket to the remote port (my opened socket port) . For example, If I connect a socket to 127.0.0.1:4444 the server is answering to my opened port (dynamic with Socket tcp) 6873 instead 4444.It is using the same socket.
A quick answer could be to use a TcpOutboundGateway but I have a couple of problems with this scenario:
I need to manage the connection events without taking into account
send /read operations. For instance, I have to auto-connect and open
the Socket before sending any message.
According to the documentation:
for high-volume messages, consider using a collaborating pair of
channel adapters. However, you will need to provide collaboration
logic.
Which component should I use for request/response sockets in a high-volume scenario?
Just read the Reference Manual from the words:
To achieve high-volume throughput (avoiding the pitfalls of using gateways as mentioned above) you may consider configuring a pair of collaborating outbound and inbound channel adapters.
Also take a look into TCP Multiplex Sample:
This sample demonstrates how to configure collaborating channel adapters, on both the client and server sides, and one technique for correlating the responses to the corresponding request.
I am trying to streaming time series data using Springframework SimpMessagingTemplate (default Stomp implementation) to broadcast messages to a topic that the SockJS client subscribed to. However, the messages is received out of order. The server is single thread and messages are sent in ascending order by their timestamps. The client somehow received the messages out of the order.
I am using the latest release version of both stompjs and springframework (4.1.6 release).
looks like there is a built in striped executor, so just enable it:
#Override
protected void configureMessageBroker(MessageBrokerRegistry registry) {
// ...
registry.setPreservePublishOrder(true);
}
https://docs.spring.io/spring/docs/current/spring-framework-reference/web.html#websocket-stomp-ordered-messages
Found the root cause of this issue. The messages were sending in "correct" order from the application implementation perspective (I.e, convertAndSend() are called in one thread or at least thread safe fashion"). However, Springframework web socket uses reactor-tcp implementation which will process the messages on clientOutboundChannel from the thread pool. Thus the messages can be written to the tcp socket in different order that they are arrived. When I configured the web socket to limit 1 thread for the clientOutboundChannel, the order is preserved.
This problem is not in the SocketJS but a limitation of current Spring web socket design.
It's Spring web socket design problem. To receive messages in valid order you have to set corePoolSize of websocket clients to 1.
#Configuration
#EnableWebSocketMessageBroker
public class WebSocketMessageBrokerConfiguration extends AbstractWebSocketMessageBrokerConfigurer {
#Override
public void configureClientOutboundChannel(ChannelRegistration registration) {
registration.taskExecutor().corePoolSize(1);
}
#Override
public void configureClientInboundChannel(ChannelRegistration registration) {
registration.taskExecutor().corePoolSize(1);
}
}
UPDATE
Please see #Jason's answer. Spring 5.1 has a setPreservePublishOrder() to order the messages based on their client ID.
I experienced this issue as well. I don't like to limit my thread pool size to 1 for this will cause an overhead on my application. Instead, I used a StripedExecutorService to process messages coming in and out of my application. This type of executor service guarantees ordered processing of messages for tasks that have same stripe. For me, I use WebSocket session ID as stripe. Register this executor via ChannelRegistration.taskExecutor() on your inbound, broker, and outbound channel and this will guarantee ordered messages. Choose your stripe wisely.