Spring STOMP over WebSockets not scheduling heartbeats - stomp

We have a Spring over WebSockets connection that we're passing a CONNECT frame:
CONNECT\naccept-version:1.2\nheart-beat:10000,10000\n\n\u0000
Which the handler acknowledges, starts a new session, and than returns:
CONNECTED
version:1.2
heart-beat:0,0
However, we want the heart-beats so we can keep the WebSocket open. We're not using SockJS.
I stepped through the Spring Message Handler:
StompHeaderAccessor [headers={simpMessageType=CONNECT, stompCommand=CONNECT, nativeHeaders={accept-version=[1.2], heart-beat=[5000,0]}, simpSessionAttributes={}, simpHeartbeat=[J#5eba717, simpSessionId=46e855c9}]
After it gets the heart-beat (native header), it sets what looks like a memory address simpHeartbeat=[J#5eba717, simpSessionId=46e855c9}]
Of note, after the broker authenticates:
Processing CONNECT session=46e855c9 (the sessionId here is different than simpSessionId)?
When running earlier TRACE debugging I saw a notice "Scheduling heartbeat..." or something to that effect...though I'm not seeing it now?
Any idea what's going on?
Thanks
I have found the explanation in the documentation:
SockJS Task Scheduler stats from thread pool of the SockJS task
scheduler which is used to send heartbeats. Note that when heartbeats
are negotiated on the STOMP level the SockJS heartbeats are disabled.
Are SockJS heartbeats different than STOMP heart-beats?

Starting Spring 4.2 you can have full control, from the server side, of the heartbeat negotiation outcome using Stomp over SockJS with the built-in SimpleBroker:
public class WebSocketConfigurer extends AbstractWebSocketMessageBrokerConfigurer {
#Override
public void configureMessageBroker(MessageBrokerRegistry config) {
ThreadPoolTaskScheduler te = new ThreadPoolTaskScheduler();
te.setPoolSize(1);
te.setThreadNamePrefix("wss-heartbeat-thread-");
te.initialize();
config.enableSimpleBroker("/")
/**
* Configure the value for the heartbeat settings. The first number
* represents how often the server will write or send a heartbeat.
* The second is how often the client should write. 0 means no heartbeats.
* <p>By default this is set to "0, 0" unless the {#link #setTaskScheduler
* taskScheduler} in which case the default becomes "10000,10000"
* (in milliseconds).
* #since 4.2
*/
.setHeartbeatValue(new long[]{heartbeatServer, heartbeatClient})
.setTaskScheduler(te);
}
#Override
public void registerStompEndpoints(StompEndpointRegistry registry) {
registry.addEndpoint(.....)
.setAllowedOrigins(....)
.withSockJS();
}
}

Yes SockJS heartbeats are different. Fundamentally the same thing but their purpose in the SockJS protocol are to ensure that the connection doesn't look like it's "dead" in which case proxies can close it pro-actively. More generally a heartbeat allows each side to detect connectivity issues pro-actively and clean up resources.
When using STOMP and SockJS at the transport layer there is no need to have both which is why the SockJS heartbeats are turned off if STOMP heartbeats are in use. However you're not using SockJS here.
You're not showing any configuration but my guess is that you're using the built-in simple broker which does not automatically send heartbeats. When configuring it you will see an option to enable heartbeats and you also need to set a task scheduler.
#Configuration
#EnableWebSocketMessageBroker
public class WebSocketConfig implements WebSocketMessageBrokerConfigurer {
#Override
public void registerStompEndpoints(StompEndpointRegistry registry) {
// ...
}
#Override
public void configureMessageBroker(MessageBrokerRegistry registry) {
registry.enableStompBrokerRelay(...)
.setTaskScheduler(...)
.setHeartbeat(...);
}
}

We got same problem with Spring, Websockets, STOMP and Spring Sessions - no heartbeats and Spring session may expire while websocket doesn't receive messages on server side. We ended up with enable STOMP heartbeats from browser every 20000ms and add SimpMessageType.HEARTBEAT to Spring sessionRepositoryInterceptor matches to keep Spring session last access time updated on STOMP heartbeats without messages. We had to use AbstractSessionWebSocketMessageBrokerConfigurer as a base to enable in-build Spring session and websocket session binding. Spring manual, second example. In official example Spring session is updated on inbound websocket CONNECT/MESSAGE/SUBSCRIBE/UNSUBSCRIBE messages, but not heartbeats, that's why we need to re-configure 2 things - enable at least inbound heartbeats and adjust Spring session to react to websocket heartbeats
public class WebSocketConfig extends AbstractSessionWebSocketMessageBrokerConfigurer<ExpiringSession> {
#Autowired
SessionRepositoryMessageInterceptor sessionRepositoryInterceptor;
#Override
public void configureMessageBroker(MessageBrokerRegistry config) {
sessionRepositoryInterceptor.setMatchingMessageTypes(EnumSet.of(SimpMessageType.CONNECT,
SimpMessageType.MESSAGE, SimpMessageType.SUBSCRIBE,
SimpMessageType.UNSUBSCRIBE, SimpMessageType.HEARTBEAT));
config.setApplicationDestinationPrefixes(...);
config.enableSimpleBroker(...)
.setTaskScheduler(new DefaultManagedTaskScheduler())
.setHeartbeatValue(new long[]{0,20000});
}
}
Another way we tried is some re-implementing of SessionRepositoryMessageInterceptor functionality to update Spring sessions last access time on outbound websocket messages plus maintain websocket session->Spring session map via listeners, but code above did the trick.

Related

Having trouble sending data to my websocket created in Spring-Boot from Flutter

I am attempting to send data through IOWebSocketChannel in Flutter.io to a WebSocket created in Spring-Boot.
In spring-boot I have created the typical WebSocket config and controllers that are dealing with client's manipulation of my servers WebSocket. I will post them below just for reference.
WebSocketConfiguration.java
#Configuration
#EnableWebSocketMessageBroker
public class WebSocketConfiguration implements WebSocketMessageBrokerConfigurer{
#Override
public void registerStompEndpoints(StompEndpointRegistry registry){
registry.addEndpoint("/websocket")
.setAllowedOrigins("*") // allow us to connect to ws://localhost:8080/websocket with the default Spring port configuration.
.withSockJS(); // allows a client which does not support WebSocket natively mimic a WebSocket over an HTTP connection
}
#Override
public void configureMessageBroker(MessageBrokerRegistry registry){ //The configureMessageBroker method sets up a simple (in-memory) message broker for our application
registry.enableSimpleBroker("/topic"); //topic to be routed back to client
registry.setApplicationDestinationPrefixes("/app"); //This configuration allows Spring to understand that any message sent to a WebSocket channel name prefixed with /app should be routed to a #MessageMapping in our application.
}
}
WebSocketController.java
#Controller
public class WebSocketController {
private static final Logger LOGGER = LoggerFactory.getLogger(WebSocketController.class);
#MessageMapping("/send")
#SendTo("/topic/messages")
public Message send(Message message) {
LOGGER.info(String.format("Received message [%s]", message.toString()));
LocalDateTime timestamp = LocalDateTime.now();
return new Message(message.getFrom(), message.getMessage(), timestamp);
}
}
Now When I try using IOWebSocketChannel I perform the typical protocol of connecting to my configured websocket. Below is the code
final channel = IOWebSocketChannel.connect(
"ws://10.0.2.2:8080/websocket"
);
I have then created a method that is supposed to send data to my websocket so I attempt to connect to that endpoint which you see is created in WebSocketController.java called app/send/. Below is the code:
void _sendMessage() {
IOWebSocketChannel channel = IOWebSocketChannel.connect('ws://10.0.2.2:8080/app/send');
channel.sink.add(
json.encode({
"message": "bars",
})
);
}
Now when I check my Spring-Boot server nothing is logged, however, whenever I hot reload in Flutter Spring Boot and my connection to the websocket times out, tomcat server returns this:
So my question is if anybody has been able to make a breakthrough with sending data through websockets from Flutter into Spring-Boot using IOWebSocketChannel? I am also wondering if anyone has found a way to successfully use a STOMP protocol in Flutter.io? I was using stomp_client as it seemed like it was going to do the trick, however correct if I'm wrong, but flutter was giving me errors saying that there doesn't exist any html files, so I'm assuming that library is only for dart in the web.
Your Spring configuration looks good. But client-side you need some tweaks.
I spent some time to figure this out with the https://pub.dev/packages/stomp package. Use a modified version of the connect function provided here. Make sure to use this custom implementation of the connect function.
Future<StompClient> client = customStomp.connect('ws://10.0.2.2:8080/websocket', ...)
Once connected, according to your configuration, you can then send message on the following destination: /app/send.

Spring Websocket with Stomp full featured broker and failover

I have those configuration for spring and a full feature stomp broker (ActiveMQ):
#Configuration
#EnableWebSocketMessageBroker
public class WebsocketConfig extends AbstractWebSocketMessageBrokerConfigurer {
private static Logger LOG = org.slf4j.LoggerFactory.getLogger(WebsocketConfig.class);
#Override
public void configureMessageBroker(MessageBrokerRegistry config) {
config.enableStompBrokerRelay("/topic/", "/queue/");
config.setApplicationDestinationPrefixes("/app");
config.setUserDestinationPrefix("/user");
}
#Override
public void registerStompEndpoints(StompEndpointRegistry registry) {
registry.addEndpoint("/socket").withSockJS();
}
}
Naively, I though spring used my current ActiveMQ configuration but in reality it tries to connect into a server located in localhost with a default stomp port. I discovered that is possible to change this configuration by typing:
config.enableStompBrokerRelay("/topic/", "/queue/")
.setRelayHost("activeMQHOST")
.setRelayPort(9999);
Thats fine, but currently I have a failover setup with two brokers (master/flave with shared file system). How can I configure such setup for the stomp broker relay?
If not possible, I thought in the following solutions:
Use the simple (in memory) broker, which doesn't advisable
The ActiveMQ is used for several operations (not restricted to websockets) and I don't know if it is recommended to mix stomp/websockets queues with other feature's queues. Thinking on it, I can create another ActiveMQ instance (maybe using the VM transport.
The second option is advisable?

SockJS receive stomp messages from spring websocket out of order

I am trying to streaming time series data using Springframework SimpMessagingTemplate (default Stomp implementation) to broadcast messages to a topic that the SockJS client subscribed to. However, the messages is received out of order. The server is single thread and messages are sent in ascending order by their timestamps. The client somehow received the messages out of the order.
I am using the latest release version of both stompjs and springframework (4.1.6 release).
looks like there is a built in striped executor, so just enable it:
#Override
protected void configureMessageBroker(MessageBrokerRegistry registry) {
// ...
registry.setPreservePublishOrder(true);
}
https://docs.spring.io/spring/docs/current/spring-framework-reference/web.html#websocket-stomp-ordered-messages
Found the root cause of this issue. The messages were sending in "correct" order from the application implementation perspective (I.e, convertAndSend() are called in one thread or at least thread safe fashion"). However, Springframework web socket uses reactor-tcp implementation which will process the messages on clientOutboundChannel from the thread pool. Thus the messages can be written to the tcp socket in different order that they are arrived. When I configured the web socket to limit 1 thread for the clientOutboundChannel, the order is preserved.
This problem is not in the SocketJS but a limitation of current Spring web socket design.
It's Spring web socket design problem. To receive messages in valid order you have to set corePoolSize of websocket clients to 1.
#Configuration
#EnableWebSocketMessageBroker
public class WebSocketMessageBrokerConfiguration extends AbstractWebSocketMessageBrokerConfigurer {
#Override
public void configureClientOutboundChannel(ChannelRegistration registration) {
registration.taskExecutor().corePoolSize(1);
}
#Override
public void configureClientInboundChannel(ChannelRegistration registration) {
registration.taskExecutor().corePoolSize(1);
}
}
UPDATE
Please see #Jason's answer. Spring 5.1 has a setPreservePublishOrder() to order the messages based on their client ID.
I experienced this issue as well. I don't like to limit my thread pool size to 1 for this will cause an overhead on my application. Instead, I used a StripedExecutorService to process messages coming in and out of my application. This type of executor service guarantees ordered processing of messages for tasks that have same stripe. For me, I use WebSocket session ID as stripe. Register this executor via ChannelRegistration.taskExecutor() on your inbound, broker, and outbound channel and this will guarantee ordered messages. Choose your stripe wisely.

Spring Websocket in a tomcat cluster

In our current application, we use Spring Websockets over STOMP. We are looking to scale horizontally. Are there any best practices on how we should handle websocket traffic over multiple tomcat instances and how can we maintain session info across multiple nodes.Is there a working sample that one can refer to?
Horizontally scaling WebSockets is actually very different than horizontally scaling stateless/stateful HTTP only based applications.
Horizontally Scaling Stateless HTTP app: just spin up some application instances in different machines and put a load balancer in front of them. There are quite a lot different load balancer solutions such as HAProxy, Nginx, etc. If you are on a cloud environment such as AWS you could also have managed solutions such as Elastic Load Balancer.
Horizontally Scaling Stateful HTTP app: it would be great if we could have all applications being stateless everytime, but unfortunately that's not always possible. So, when dealing with stateful HTTP apps, you must care about the HTTP session, which is a basically a local storage for each different client where the web server can store data that is kept across different HTTP requests (such as when dealing with a Shopping Cart). Well, in this case, when scaling horizontally you should be aware that, as I said, it's a LOCAL storage, so ServerA will not be able to handle an HTTP session that is on ServerB. In other words, if for any reason Client1 that is being served by ServerA starts suddenly to be served by ServerB, his HTTP session will be lost (and his shopping cart will be gone!). The reasons could be a node failure or even a deployment.
In order to address this issue, you can't keep HTTP sessions only locally, that is, you must store them on another external component. That are several components that would be able to handle this, such as any relational database, but that would be actually an overhead. Some NoSQL databases can handle this key-value behavior very well, such as Redis.
Now, with the HTTP session being stored on Redis, if a client starts to be served by another server, it will fetch the client's HTTP session from Redis and load it into its memory, so everything will continue working and the user will not lost his HTTP session anymore.
You can use Spring Session to easily store the HTTP session on Redis.
Horizontally Scaling WebSocket app: When a WebSocket connection is established, the server must keep the connection opened with the client so that they can exchange data in both directions. When a client is listening to a destination such as "/topic/public.messages" we say the client is subscribed to this destination. In Spring, when you use the simpleBroker approach, the subscriptions are kept in memory, so what happens for instance if Client1 is being served by ServerA and wants to send a message using WebSocket to Client2 being served by ServerB? You already know the answer! The message will not be delivered to Client2 because Server1 not even know about the Client2's subscription.
So, in order to address this issue, again you have to externalize the WebSockets subscriptions. As you are using STOMP as a subprotocol, you need an external component that can act as a external STOMP broker. There are quite a lot tools able to do this, but I would suggest RabbitMQ.
Now, you must change your Spring configuration so that it will not keep the subscriptions in-memory. Instead, it will delegate the subscriptions to a external STOMP broker. You can easily achieve this with some basic configurations such as enableStompBrokerRelay.
The important thing to note is that HTTP session is different than WebSocket session. Using Spring Session to store the HTTP session in Redis has absolutely nothing to do with horizontally scaling WebSockets.
I've coded a complete Web Chat Application with Spring Boot (and much more) that uses RabbitMQ as a Full External STOMP Broker and it's public on GitHub so please clone it, run the app in your machine and see the code details.
When it comes to a WebSocket connection loss, there's not much that Spring can do. Actually, the reconnection must be requested by the client side implementing a reconnection callback function, for instance (that's the WebSocket handshake flow, the client must start the handshake, not the server). There are some client side libraries that can handle this transparently for you. That's not SockJS case. In the Chat Application I also implemented this reconnection feature.
Your requirement can be divided into 2 sub tasks:
Maintain session info across multiple nodes: You can try Spring Sessions clustering backed by Redis (see: HttpSession with Redis). This very very simple and already has support for Spring Websockets (see: Spring Session & WebSockets).
Handle websockets traffic over multiple tomcat instances: There are several ways to do that.
The first way: Using a full-featured broker (eg: ActiveMQ) and try new feature Support multiple WebSocket servers (from: 4.2.0 RC1)
The second way: Using a full-feature broker and implement a distributed UserSessionRegistry (eg: Using Redis :D ). The default implementation DefaultUserSessionRegistry using an in-memory storage.
Updated: I've written a simple implementation using Redis, try it if you are interested
To configure a full-featured broker (broker relay), you can try:
public class WebSocketConfig extends AbstractWebSocketMessageBrokerConfigurer {
...
#Autowired
private RedisConnectionFactory redisConnectionFactory;
#Override
public void configureMessageBroker(MessageBrokerRegistry config) {
config.enableStompBrokerRelay("/topic", "/queue")
.setRelayHost("localhost") // broker host
.setRelayPort(61613) // broker port
;
config.setApplicationDestinationPrefixes("/app");
}
#Bean
public UserSessionRegistry userSessionRegistry() {
return new RedisUserSessionRegistry(redisConnectionFactory);
}
...
}
and
import java.util.Set;
import org.springframework.data.redis.connection.RedisConnectionFactory;
import org.springframework.data.redis.core.BoundHashOperations;
import org.springframework.data.redis.core.BoundSetOperations;
import org.springframework.data.redis.core.RedisOperations;
import org.springframework.data.redis.core.RedisTemplate;
import org.springframework.data.redis.core.StringRedisTemplate;
import org.springframework.data.redis.serializer.StringRedisSerializer;
import org.springframework.messaging.simp.user.UserSessionRegistry;
import org.springframework.util.Assert;
/**
* An implementation of {#link UserSessionRegistry} backed by Redis.
* #author thanh
*/
public class RedisUserSessionRegistry implements UserSessionRegistry {
/**
* The prefix for each key of the Redis Set representing a user's sessions. The suffix is the unique user id.
*/
static final String BOUNDED_HASH_KEY_PREFIX = "spring:websockets:users:";
private final RedisOperations<String, String> sessionRedisOperations;
#SuppressWarnings("unchecked")
public RedisUserSessionRegistry(RedisConnectionFactory redisConnectionFactory) {
this(createDefaultTemplate(redisConnectionFactory));
}
public RedisUserSessionRegistry(RedisOperations<String, String> sessionRedisOperations) {
Assert.notNull(sessionRedisOperations, "sessionRedisOperations cannot be null");
this.sessionRedisOperations = sessionRedisOperations;
}
#Override
public Set<String> getSessionIds(String user) {
Set<String> entries = getSessionBoundHashOperations(user).members();
return (entries != null) ? entries : Collections.<String>emptySet();
}
#Override
public void registerSessionId(String user, String sessionId) {
getSessionBoundHashOperations(user).add(sessionId);
}
#Override
public void unregisterSessionId(String user, String sessionId) {
getSessionBoundHashOperations(user).remove(sessionId);
}
/**
* Gets the {#link BoundHashOperations} to operate on a username
*/
private BoundSetOperations<String, String> getSessionBoundHashOperations(String username) {
String key = getKey(username);
return this.sessionRedisOperations.boundSetOps(key);
}
/**
* Gets the Hash key for this user by prefixing it appropriately.
*/
static String getKey(String username) {
return BOUNDED_HASH_KEY_PREFIX + username;
}
#SuppressWarnings("rawtypes")
private static RedisTemplate createDefaultTemplate(RedisConnectionFactory connectionFactory) {
Assert.notNull(connectionFactory, "connectionFactory cannot be null");
StringRedisTemplate template = new StringRedisTemplate(connectionFactory);
template.setKeySerializer(new StringRedisSerializer());
template.setValueSerializer(new StringRedisSerializer());
template.afterPropertiesSet();
return template;
}
}
Maintain session info across multiple nodes:
Suppose we have 2 server host, backed up with load balancer.
Websockets are socket connection from browser to specific server host.eg host1
Now if host1 goes down, socket connection from load balancer - host 1 will break.
How spring will reopen same websocket connection from load balancer to host 2 ? browser should not open new websocket connection

Spring 4 STOMP over Websockets- How to setup login and passcode properly

I'm playing around with Spring 4 Stomp over Websockets. Now I'm trying to put login and password in my configuration.
#Override
public void configureMessageBroker(MessageBrokerRegistry registry) {
//registry.enableSimpleBroker("/queue/", "/topic/");
//Enable MQ
StompBrokerRelayRegistration relay=registry.enableStompBrokerRelay("/queue/", "/topic/");
relay.setSystemLogin("login");
relay.setSystemPasscode("passcode");
//relay.setClientLogin("login");
//relay.setClientPasscode("passcode");
registry.setApplicationDestinationPrefixes("/app");
}
But then when I try to connect with different login and passcode, I can still connect. Here's my javascript.
$scope.initSockets = function() {
$scope.socket.client = new SockJS('/Html5GameApp');
$scope.socket.stomp = Stomp.over($scope.socket.client);
$scope.socket.stomp.connect("sample","sample",function(frame) {
console.log('Connected: ' + frame);
$scope.socket.stomp.subscribe("/queue/stomp.data", $scope.liveGameData);
});
$scope.socket.client.onclose = $scope.reconnect;
};
Am I doing wrong with my configuration?How will I setup it properly.Thanks
Your application is made of 3 "systems" or "actors" in this scenario:
the browsers
the Spring application
the broker (e.g. RabbitMQ)
If you take a look at StompBrokerRelayRegistration's javadoc, you'll see that:
system credentials are for the shared "system" connection and are used to send messages to the STOMP broker from within the application, i.e. messages not associated with a specific client session (e.g. REST/HTTP request handling method).
client credentials are used when creating connections to the STOMP broker on behalf of connected clients.
If you're actually trying to enforce access security in your application, you could take a look at the portfolio sample and its security config. In a nutshell, security is enforced during the HTTP Upgrade phase in this example.

Resources