In Spring Integration, I am using a couple of channel adapters for sending/receiving messages from a server socket. I always create client connections with the following adapters:
#Bean
public TcpReceivingChannelAdapter tcpIn(AbstractClientConnectionFactory connectionFactory) throws Exception {
TcpReceivingChannelAdapter receiver = new TcpReceivingChannelAdapter();
receiver.setOutputChannel(fromTcp ());
receiver.setConnectionFactory(connectionFactory);
return receiver;
}
#Bean
#ServiceActivator(inputChannel = "toTcp")
public TcpSendingMessageHandler tcpOut(AbstractClientConnectionFactory connectionFactory) throws Exception {
TcpSendingMessageHandler sender = new TcpSendingMessageHandler();
sender.setConnectionFactory(connectionFactory);
sender.setClientMode(true);
return sender;
}
The problem here is the server is answering in the same socket to the remote port (my opened socket port) . For example, If I connect a socket to 127.0.0.1:4444 the server is answering to my opened port (dynamic with Socket tcp) 6873 instead 4444.It is using the same socket.
A quick answer could be to use a TcpOutboundGateway but I have a couple of problems with this scenario:
I need to manage the connection events without taking into account
send /read operations. For instance, I have to auto-connect and open
the Socket before sending any message.
According to the documentation:
for high-volume messages, consider using a collaborating pair of
channel adapters. However, you will need to provide collaboration
logic.
Which component should I use for request/response sockets in a high-volume scenario?
Just read the Reference Manual from the words:
To achieve high-volume throughput (avoiding the pitfalls of using gateways as mentioned above) you may consider configuring a pair of collaborating outbound and inbound channel adapters.
Also take a look into TCP Multiplex Sample:
This sample demonstrates how to configure collaborating channel adapters, on both the client and server sides, and one technique for correlating the responses to the corresponding request.
Related
Every session data passed into the socket is broadcasted to all users since every session subscribes to the UnicastProcessor eventPublisher.
How can I send by event data to a single session id and not to all of them?
#Override
public Mono<Void> handle(WebSocketSession session) {
WebSocketMessageSubscriber subscriber = new WebSocketMessageSubscriber(eventPublisher);
session.receive()
.map(WebSocketMessage::getPayloadAsText)
.map(this::toEvent)
.subscribe(subscriber::onNext, subscriber::onError, subscriber::onComplete);
return session.send(outputEvents.map(session::textMessage));
}
My use-case requires me to include both options for broadcasting any changed state with any client to all sockets connected plus the abillity to send response to a specific client (sessionId) that send a request within a specific event
Github link
or should It be routed to 2 different handlers from the same websocket path?
note that from javascript
new WebSocket(url/path) creates a socket connection
there is no way to change the path without creating or instantiating a new WebSocket object which is not wanted.
I'm not interested in creating for every browser client 2 sockets...
so my goal is to base the server connection via 1 single websocket path
#Bean
public HandlerMapping webSocketMapping(UnicastProcessor<Event> eventPublisher, Flux<Event> events) {
Map<String, Object> map = new HashMap<>();
map.put("/websocket/chat", new ChatSocketHandler(eventPublisher, events));
SimpleUrlHandlerMapping simpleUrlHandlerMapping = new SimpleUrlHandlerMapping();
simpleUrlHandlerMapping.setUrlMap(map);
//Without the order things break :-/
simpleUrlHandlerMapping.setOrder(10);
return simpleUrlHandlerMapping;
}
if so would be glad to see an example of such solution
With servlet based web socket, it's possible because you can connect websocket to messaging brokers. Then the messaging broker will take care of sending messages to specific client.
But with webflux based websocket that spring is provided , I couldn't manage to do bring messaging brokers into action. It seems that there is no support yet for it in spring webflux.
Find a sample with servlet stack here:
https://github.com/bmd007/RealtimeNoteSharing.git
I have a Spring Boot app (Jhipster) that uses STOMP over WebSockets to communicate information from the server to users.
I recently added an ActiveMQ server to handle scaling the app horizontally, with an Amazon auto-scaling group / load-balancer.
I make use the convertAndSendToUser() method, which works on single instances of the app to locate the authenticated users' "individual queue" so only they receive the message.
However, when I launch the app behind the load balancer, I am finding that messages are only being sent to the user if the event is generated on the server that their websocket-proxy connection (to the broker) is established on?
How do I ensure the message goes through ActiveMQ to whichever instance of the app that the user is actually "connected too" regardless of which instance receives, say an HTTP Request that executes the convertAndSendToUser() event?
For reference here is my StompBrokerRelayMessageHandler:
#Bean
public AbstractBrokerMessageHandler stompBrokerRelayMessageHandler() {
StompBrokerRelayMessageHandler handler = (StompBrokerRelayMessageHandler) super.stompBrokerRelayMessageHandler();
handler.setTcpClient(new Reactor2TcpClient<>(
new StompTcpFactory(orgProperties.getAws().getAmazonMq().getStompRelayHost(),
orgProperties.getAws().getAmazonMq().getStompRelayPort(), orgProperties.getAws().getAmazonMq
().getSsl())
));
return handler;
}
#Override
public void configureMessageBroker(MessageBrokerRegistry config) {
config.enableStompBrokerRelay("/queue", "/topic")
.setSystemLogin(orgProperties.getAws().getAmazonMq().getStompRelayHostUser())
.setSystemPasscode(orgProperties.getAws().getAmazonMq().getStompRelayHostPass())
.setClientLogin(orgProperties.getAws().getAmazonMq().getStompRelayHostUser())
.setClientPasscode(orgProperties.getAws().getAmazonMq().getStompRelayHostPass());
config.setApplicationDestinationPrefixes("/app");
}
I have found the name corresponding to the queue that is generated on ActiveMQ by examining the headers in the SessionSubscribeEvent, that is generated in the listener when a user subscribes to a user-queue, as simpSessionId.
#Override
#EventListener({SessionSubscribeEvent.class})
public void onSessionSubscribeEvent(SessionSubscribeEvent event) {
log.debug("Session Subscribe Event:" +
"{}", event.getMessage().getHeaders().toString());
}
Corresponding queues' can be found in ActiveMQ, in the format: {simpDestination}-user{simpSessionId}
Could I save the sessionId in a key-value pair and just push messages onto that topic channel?
I also found some possibilities of setting ActiveMQ specific STOMP properties in the CONNECT/SUBSCRIBE frame to create durable subscribers if I set these properties will Spring than understand the routing?
client-id & subcriptionName
Modifying the MessageBrokerReigstry config resolved the issue:
config.enableStompBrokerRelay("/queue", "/topic")
.setUserDestinationBroadcast("/topic/registry.broadcast")
Based on this paragraph in the documentation section 4.4.13:
In a multi-application server scenario a user destination may remain
unresolved because the user is connected to a different server. In
such cases you can configure a destination to broadcast unresolved
messages to so that other servers have a chance to try. This can be
done through the userDestinationBroadcast property of the
MessageBrokerRegistry in Java config and the
user-destination-broadcast attribute of the message-broker element in
XML
I did not see any documentation on "why" /topic/registry.broadcast was the correct "topic" destination, but I am finding various iterations of it:
websocket sessions sample doesn't cluster.. spring-session-1.2.2
What is MultiServerUserRegistry in spring websocket?
Spring websocket - sendToUser from a cluster does not work from backup server
I have a (legacy) TCP service that has multiple processes. Each process runs on the same host, but on a different port. The service is single threaded, so the way to increase throughput is to round-robin each request across each of the ports.
I am providing an AMQP exposure to this legacy application. Its very simple - take a string off the AMQP queue, pass it to the application, and return the response string to the AMQP reply queue.
This works great on a single port. However, i'd like to fan out the requests across all the ports.
Spring Integration seems to only provide AbstractClientConnectionFactory implementations that either connect directly to a single host/port (TcpNetClientConnectionFactory) or maintain a pool of connections to a single host/port (CachingClientConnectionFactory). There arent any that pool connections between a single host and multiple ports.
I have attempted to write my own AbstractClientConnectionFactory that maintains a pool of AbstractClientConnectionFactory objects and round-robins between them. However, I have struck several issues to do with handing the TCP connections when the target service goes away or the network is interrupted that I have not been able to solve.
There is also the approach taken by this question: Spring Integration 4 - configuring a LoadBalancingStrategy in Java DSL but the solution to that was to hardcode the number of endpoints. In my case, the number of endpoints is only known at runtime and is a user-configurable setting.
So, basically I need to create a TcpOutboundGateway per port dynamically at runtime and somehow register it in my IntegrationFlow. I have attempted the following:
#Bean
public IntegrationFlow xmlQueryWorkerIntegrationFlow() {
SimpleMessageListenerContainer inboundQueue = getMessageListenerContainer();
DirectChannel rabbitReplyChannel = MessageChannels.direct().get();
IntegrationFlowBuilder builder = IntegrationFlows
.from(Amqp.inboundGateway(inboundQueue)
.replyChannel(rabbitReplyChannel))
/* SOMEHOW DO THE ROUND ROBIN HERE */
//I have tried:
.channel(handlerChannel()) //doesnt work, the gateways dont get started and the message doesnt get sent to the gateway
//and I have also tried:
.handle(gateway1)
.handle(gateway2) //doesnt work, it chains the handlers instead of round-robining between them
//
.transform(new ObjectToStringTransformer())
.channel(rabbitReplyChannel);
return builder.get();
}
#Bean
//my attempt at dynamically adding handlers to the same channel and load balancing between them
public DirectChannel handlerChannel() {
DirectChannel channel = MessageChannels.direct().loadBalancer(new RoundRobinLoadBalancingStrategy()).get();
for (AbstractClientConnectionFactory factory : generateConnections()) {
channel.subscribe(generateTcpOutboundGateway(factory));
}
return channel;
}
Does anyone know how I can solve this problem?
See the dynamic ftp sample - in essence each outbound gateway goes in its own application context and the dynamic router routes to the appropriate channel (for which the outbound adapter is created on demand if necessary).
Although the sample uses XML, you can do the same thing with java configuration, or even with the Java DSL.
See my answer to a similar question for multiple IMAP mail adapters using Java configuration and then a follow-up question.
I am trying to streaming time series data using Springframework SimpMessagingTemplate (default Stomp implementation) to broadcast messages to a topic that the SockJS client subscribed to. However, the messages is received out of order. The server is single thread and messages are sent in ascending order by their timestamps. The client somehow received the messages out of the order.
I am using the latest release version of both stompjs and springframework (4.1.6 release).
looks like there is a built in striped executor, so just enable it:
#Override
protected void configureMessageBroker(MessageBrokerRegistry registry) {
// ...
registry.setPreservePublishOrder(true);
}
https://docs.spring.io/spring/docs/current/spring-framework-reference/web.html#websocket-stomp-ordered-messages
Found the root cause of this issue. The messages were sending in "correct" order from the application implementation perspective (I.e, convertAndSend() are called in one thread or at least thread safe fashion"). However, Springframework web socket uses reactor-tcp implementation which will process the messages on clientOutboundChannel from the thread pool. Thus the messages can be written to the tcp socket in different order that they are arrived. When I configured the web socket to limit 1 thread for the clientOutboundChannel, the order is preserved.
This problem is not in the SocketJS but a limitation of current Spring web socket design.
It's Spring web socket design problem. To receive messages in valid order you have to set corePoolSize of websocket clients to 1.
#Configuration
#EnableWebSocketMessageBroker
public class WebSocketMessageBrokerConfiguration extends AbstractWebSocketMessageBrokerConfigurer {
#Override
public void configureClientOutboundChannel(ChannelRegistration registration) {
registration.taskExecutor().corePoolSize(1);
}
#Override
public void configureClientInboundChannel(ChannelRegistration registration) {
registration.taskExecutor().corePoolSize(1);
}
}
UPDATE
Please see #Jason's answer. Spring 5.1 has a setPreservePublishOrder() to order the messages based on their client ID.
I experienced this issue as well. I don't like to limit my thread pool size to 1 for this will cause an overhead on my application. Instead, I used a StripedExecutorService to process messages coming in and out of my application. This type of executor service guarantees ordered processing of messages for tasks that have same stripe. For me, I use WebSocket session ID as stripe. Register this executor via ChannelRegistration.taskExecutor() on your inbound, broker, and outbound channel and this will guarantee ordered messages. Choose your stripe wisely.
In our current application, we use Spring Websockets over STOMP. We are looking to scale horizontally. Are there any best practices on how we should handle websocket traffic over multiple tomcat instances and how can we maintain session info across multiple nodes.Is there a working sample that one can refer to?
Horizontally scaling WebSockets is actually very different than horizontally scaling stateless/stateful HTTP only based applications.
Horizontally Scaling Stateless HTTP app: just spin up some application instances in different machines and put a load balancer in front of them. There are quite a lot different load balancer solutions such as HAProxy, Nginx, etc. If you are on a cloud environment such as AWS you could also have managed solutions such as Elastic Load Balancer.
Horizontally Scaling Stateful HTTP app: it would be great if we could have all applications being stateless everytime, but unfortunately that's not always possible. So, when dealing with stateful HTTP apps, you must care about the HTTP session, which is a basically a local storage for each different client where the web server can store data that is kept across different HTTP requests (such as when dealing with a Shopping Cart). Well, in this case, when scaling horizontally you should be aware that, as I said, it's a LOCAL storage, so ServerA will not be able to handle an HTTP session that is on ServerB. In other words, if for any reason Client1 that is being served by ServerA starts suddenly to be served by ServerB, his HTTP session will be lost (and his shopping cart will be gone!). The reasons could be a node failure or even a deployment.
In order to address this issue, you can't keep HTTP sessions only locally, that is, you must store them on another external component. That are several components that would be able to handle this, such as any relational database, but that would be actually an overhead. Some NoSQL databases can handle this key-value behavior very well, such as Redis.
Now, with the HTTP session being stored on Redis, if a client starts to be served by another server, it will fetch the client's HTTP session from Redis and load it into its memory, so everything will continue working and the user will not lost his HTTP session anymore.
You can use Spring Session to easily store the HTTP session on Redis.
Horizontally Scaling WebSocket app: When a WebSocket connection is established, the server must keep the connection opened with the client so that they can exchange data in both directions. When a client is listening to a destination such as "/topic/public.messages" we say the client is subscribed to this destination. In Spring, when you use the simpleBroker approach, the subscriptions are kept in memory, so what happens for instance if Client1 is being served by ServerA and wants to send a message using WebSocket to Client2 being served by ServerB? You already know the answer! The message will not be delivered to Client2 because Server1 not even know about the Client2's subscription.
So, in order to address this issue, again you have to externalize the WebSockets subscriptions. As you are using STOMP as a subprotocol, you need an external component that can act as a external STOMP broker. There are quite a lot tools able to do this, but I would suggest RabbitMQ.
Now, you must change your Spring configuration so that it will not keep the subscriptions in-memory. Instead, it will delegate the subscriptions to a external STOMP broker. You can easily achieve this with some basic configurations such as enableStompBrokerRelay.
The important thing to note is that HTTP session is different than WebSocket session. Using Spring Session to store the HTTP session in Redis has absolutely nothing to do with horizontally scaling WebSockets.
I've coded a complete Web Chat Application with Spring Boot (and much more) that uses RabbitMQ as a Full External STOMP Broker and it's public on GitHub so please clone it, run the app in your machine and see the code details.
When it comes to a WebSocket connection loss, there's not much that Spring can do. Actually, the reconnection must be requested by the client side implementing a reconnection callback function, for instance (that's the WebSocket handshake flow, the client must start the handshake, not the server). There are some client side libraries that can handle this transparently for you. That's not SockJS case. In the Chat Application I also implemented this reconnection feature.
Your requirement can be divided into 2 sub tasks:
Maintain session info across multiple nodes: You can try Spring Sessions clustering backed by Redis (see: HttpSession with Redis). This very very simple and already has support for Spring Websockets (see: Spring Session & WebSockets).
Handle websockets traffic over multiple tomcat instances: There are several ways to do that.
The first way: Using a full-featured broker (eg: ActiveMQ) and try new feature Support multiple WebSocket servers (from: 4.2.0 RC1)
The second way: Using a full-feature broker and implement a distributed UserSessionRegistry (eg: Using Redis :D ). The default implementation DefaultUserSessionRegistry using an in-memory storage.
Updated: I've written a simple implementation using Redis, try it if you are interested
To configure a full-featured broker (broker relay), you can try:
public class WebSocketConfig extends AbstractWebSocketMessageBrokerConfigurer {
...
#Autowired
private RedisConnectionFactory redisConnectionFactory;
#Override
public void configureMessageBroker(MessageBrokerRegistry config) {
config.enableStompBrokerRelay("/topic", "/queue")
.setRelayHost("localhost") // broker host
.setRelayPort(61613) // broker port
;
config.setApplicationDestinationPrefixes("/app");
}
#Bean
public UserSessionRegistry userSessionRegistry() {
return new RedisUserSessionRegistry(redisConnectionFactory);
}
...
}
and
import java.util.Set;
import org.springframework.data.redis.connection.RedisConnectionFactory;
import org.springframework.data.redis.core.BoundHashOperations;
import org.springframework.data.redis.core.BoundSetOperations;
import org.springframework.data.redis.core.RedisOperations;
import org.springframework.data.redis.core.RedisTemplate;
import org.springframework.data.redis.core.StringRedisTemplate;
import org.springframework.data.redis.serializer.StringRedisSerializer;
import org.springframework.messaging.simp.user.UserSessionRegistry;
import org.springframework.util.Assert;
/**
* An implementation of {#link UserSessionRegistry} backed by Redis.
* #author thanh
*/
public class RedisUserSessionRegistry implements UserSessionRegistry {
/**
* The prefix for each key of the Redis Set representing a user's sessions. The suffix is the unique user id.
*/
static final String BOUNDED_HASH_KEY_PREFIX = "spring:websockets:users:";
private final RedisOperations<String, String> sessionRedisOperations;
#SuppressWarnings("unchecked")
public RedisUserSessionRegistry(RedisConnectionFactory redisConnectionFactory) {
this(createDefaultTemplate(redisConnectionFactory));
}
public RedisUserSessionRegistry(RedisOperations<String, String> sessionRedisOperations) {
Assert.notNull(sessionRedisOperations, "sessionRedisOperations cannot be null");
this.sessionRedisOperations = sessionRedisOperations;
}
#Override
public Set<String> getSessionIds(String user) {
Set<String> entries = getSessionBoundHashOperations(user).members();
return (entries != null) ? entries : Collections.<String>emptySet();
}
#Override
public void registerSessionId(String user, String sessionId) {
getSessionBoundHashOperations(user).add(sessionId);
}
#Override
public void unregisterSessionId(String user, String sessionId) {
getSessionBoundHashOperations(user).remove(sessionId);
}
/**
* Gets the {#link BoundHashOperations} to operate on a username
*/
private BoundSetOperations<String, String> getSessionBoundHashOperations(String username) {
String key = getKey(username);
return this.sessionRedisOperations.boundSetOps(key);
}
/**
* Gets the Hash key for this user by prefixing it appropriately.
*/
static String getKey(String username) {
return BOUNDED_HASH_KEY_PREFIX + username;
}
#SuppressWarnings("rawtypes")
private static RedisTemplate createDefaultTemplate(RedisConnectionFactory connectionFactory) {
Assert.notNull(connectionFactory, "connectionFactory cannot be null");
StringRedisTemplate template = new StringRedisTemplate(connectionFactory);
template.setKeySerializer(new StringRedisSerializer());
template.setValueSerializer(new StringRedisSerializer());
template.afterPropertiesSet();
return template;
}
}
Maintain session info across multiple nodes:
Suppose we have 2 server host, backed up with load balancer.
Websockets are socket connection from browser to specific server host.eg host1
Now if host1 goes down, socket connection from load balancer - host 1 will break.
How spring will reopen same websocket connection from load balancer to host 2 ? browser should not open new websocket connection