First started Redis Server then started application.As soon as messages appears in the Redis stream those are getting consumed properly. In mean while if restarts the Redis Server as application is already running it is not consuming new messages from the Redis Server.
Could someone please assist on this.
Is there any more configuration needs to be done, to continue to process message after redis server restart.
I'm using StreamMessageListenerContainer with consumergroup.
#Bean
public Subscription listener(RedisStreamConsumer streamListener, RedisConnectionFactory redisConnectionFactory) throws InterruptedException {
StreamMessageListenerContainer<String, MapRecord<String, Object, String>> listenerContainer =
StreamMessageListenerContainer.create(redisTemplate().getConnectionFactory(),
StreamMessageListenerContainer.StreamMessageListenerContainerOptions.builder()
.hashKeySerializer(new StringRedisSerializer()).hashValueSerializer(new StringRedisSerializer())
.pollTimeout(Duration.ofMillis(100))
.build());
Subscription subscription = listenerContainer.receive(Consumer.from(groupName, consumerName),
StreamOffset.create(consumerstreamName, ReadOffset.lastConsumed()), streamListener);
subscription.await(Duration.ofSeconds(2));
listenerContainer.start();
return subscription;
}
I have same Hazelcast server project running at 3 different server's, two of them are able to form the cluster but the third server does not join. I have created Hazelcast server project using spring-boot. Here are my Spring Boot Hazelcast config.
#Bean
public Config hazelCastConfig() {
Config config = new Config();
config.getNetworkConfig().setPortAutoIncrement(true);
config.setClusterName("myHazelcastStore");
NetworkConfig network = config.getNetworkConfig();
JoinConfig join = network.getJoin();
join.getMulticastConfig().setEnabled(true);
return config;
}
#Bean
public HazelcastInstance hazelcastInstance(Config hazelCastConfig) {
return Hazelcast.newHazelcastInstance(hazelCastConfig);
}
#Bean
public Map<String, EmployeeAccount> employeeMap(HazelcastInstance hazelcastInstance) {
return hazelcastInstance.getMap("employeeMap");
}
I am using latest stable version of Hazelcast IMDG v4.2.2. I have enabled multicast in Hazelcast config. My IP for three server's are:
192.168.1.10
192.168.1.25
192.168.34.122
Here two are in same series and one is different. Is this the reason. Or is there any limitation in free edition of Hazelcast as only 2 server's can form the cluster?
There is no limitation in Hazelcast Open Source (free) version. You can create as big a cluster as you want.
Concerning your issue, Multicast should work correctly. If it does not, I would check the following parts:
Try to use static TCP/IP configuration (if it does not work, then it's a connectivity issue, not a discoverability issue).
If TCP/IP works, then the next thing to check is if Multicast packets work in your network (sometimes they may be blocked).
If you still have an issue, could you attach Hazeclast logs?
I have setup an Artemis HA-Custer example locally on my computer to learn how it's basically working. Now I want to prepare it to be pushed in a kubernetes cluster. Therefore I want to change the way of the initial membership discovery for the broker nodes, so I can use it in cloud, too. I want to use JMS and JGroups with "jdbc_ping". Actually I am not sure, if I am doing it right, so maybe you can tell me if not.
So far the brokers have successfully put their infos in the db-table and are apparently connected. When I try the following connectionFactory from my java application, it starts without errors and connects with the brokers. But in some points I am not sure, if it acts correctly.
#Bean
public ConnectionFactory connectionFactory() {
TransportConfiguration transportConfiguration = new TransportConfiguration(NettyConnectorFactory.class.getName());
ConnectionFactory cf = ActiveMQJMSClient.createConnectionFactoryWithHA(JMSFactoryType.CF, transportConfiguration);
return cf;
}
So the single point of question is now, how to setup the connectionFactory for the use of JGroups correctly.
UPDATE:
INFO 24528 --- [enerContainer-1] o.s.j.l.DefaultMessageListenerContainer : JMS message listener invoker needs to establish shared Connection
ERROR 24528 --- [enerContainer-1] o.s.j.l.DefaultMessageListenerContainer : Could not refresh JMS Connection for destination 'TestA' - retrying using FixedBackOff{interval=5000, currentAttempts=0, maxAttempts=unlimited}. Cause: Failed to create session factory; nested exception is ActiveMQInternalErrorException[errorType=INTERNAL_ERROR message=AMQ219004: Failed to initialise session factory]
The ActiveMQ Artemis documentation covers this:
Lastly, the jgroups scheme is supported which provides an alternative to the udp scheme for server discovery. The URL pattern is either jgroups://channelName?file=jgroups-xml-conf-filename where jgroups-xml-conf-filename refers to an XML file on the classpath that contains the JGroups configuration or it can be jgroups://channelName?properties=some-jgroups-properties. In both instance the channelName is the name given to the jgroups channel created.
In your code you can do something like this:
#Bean
public ConnectionFactory connectionFactory() {
return new ActiveMQConnectionFactory("jgroups://channelName?file=jgroups-xml-conf-filename");
}
In your case the client will need access to the same database that the broker's are using in order to use that information for discovery.
How to get Websphere MQ Queue depth in case of Multi Instanced environment.
For a single instance we are getting a queue depth using the MQManager like this:
#SuppressWarnings("unchecked")
private MQQueueManager createQueueManager() throws MQException {
MQEnvironment.channel = channel;
MQEnvironment.port = port;
MQEnvironment.hostname = host;
MQEnvironment.properties.put(MQC.TRANSPORT_PROPERTY, MQC.TRANSPORT_MQSERIES);
return new MQQueueManager(manager);
}
For Websphere MQ in multi-instancing environment how to perform the same ?
You can do it either of these ways:
Use a load balancer which is capable of routing the TCP connections based on the availability of the queue manager instances behind it, and connect to the address of the load balancer instead of directly to the queue manager.
Use a Client Channel Definition Table to specify the parameters for the queue manager connection. You will need to configure a queue manager group containing the instances of your queue manager and connect using the CCDT:
https://www-01.ibm.com/support/knowledgecenter/SSFKSJ_7.1.0/com.ibm.mq.doc/ja11090_.htm
In our current application, we use Spring Websockets over STOMP. We are looking to scale horizontally. Are there any best practices on how we should handle websocket traffic over multiple tomcat instances and how can we maintain session info across multiple nodes.Is there a working sample that one can refer to?
Horizontally scaling WebSockets is actually very different than horizontally scaling stateless/stateful HTTP only based applications.
Horizontally Scaling Stateless HTTP app: just spin up some application instances in different machines and put a load balancer in front of them. There are quite a lot different load balancer solutions such as HAProxy, Nginx, etc. If you are on a cloud environment such as AWS you could also have managed solutions such as Elastic Load Balancer.
Horizontally Scaling Stateful HTTP app: it would be great if we could have all applications being stateless everytime, but unfortunately that's not always possible. So, when dealing with stateful HTTP apps, you must care about the HTTP session, which is a basically a local storage for each different client where the web server can store data that is kept across different HTTP requests (such as when dealing with a Shopping Cart). Well, in this case, when scaling horizontally you should be aware that, as I said, it's a LOCAL storage, so ServerA will not be able to handle an HTTP session that is on ServerB. In other words, if for any reason Client1 that is being served by ServerA starts suddenly to be served by ServerB, his HTTP session will be lost (and his shopping cart will be gone!). The reasons could be a node failure or even a deployment.
In order to address this issue, you can't keep HTTP sessions only locally, that is, you must store them on another external component. That are several components that would be able to handle this, such as any relational database, but that would be actually an overhead. Some NoSQL databases can handle this key-value behavior very well, such as Redis.
Now, with the HTTP session being stored on Redis, if a client starts to be served by another server, it will fetch the client's HTTP session from Redis and load it into its memory, so everything will continue working and the user will not lost his HTTP session anymore.
You can use Spring Session to easily store the HTTP session on Redis.
Horizontally Scaling WebSocket app: When a WebSocket connection is established, the server must keep the connection opened with the client so that they can exchange data in both directions. When a client is listening to a destination such as "/topic/public.messages" we say the client is subscribed to this destination. In Spring, when you use the simpleBroker approach, the subscriptions are kept in memory, so what happens for instance if Client1 is being served by ServerA and wants to send a message using WebSocket to Client2 being served by ServerB? You already know the answer! The message will not be delivered to Client2 because Server1 not even know about the Client2's subscription.
So, in order to address this issue, again you have to externalize the WebSockets subscriptions. As you are using STOMP as a subprotocol, you need an external component that can act as a external STOMP broker. There are quite a lot tools able to do this, but I would suggest RabbitMQ.
Now, you must change your Spring configuration so that it will not keep the subscriptions in-memory. Instead, it will delegate the subscriptions to a external STOMP broker. You can easily achieve this with some basic configurations such as enableStompBrokerRelay.
The important thing to note is that HTTP session is different than WebSocket session. Using Spring Session to store the HTTP session in Redis has absolutely nothing to do with horizontally scaling WebSockets.
I've coded a complete Web Chat Application with Spring Boot (and much more) that uses RabbitMQ as a Full External STOMP Broker and it's public on GitHub so please clone it, run the app in your machine and see the code details.
When it comes to a WebSocket connection loss, there's not much that Spring can do. Actually, the reconnection must be requested by the client side implementing a reconnection callback function, for instance (that's the WebSocket handshake flow, the client must start the handshake, not the server). There are some client side libraries that can handle this transparently for you. That's not SockJS case. In the Chat Application I also implemented this reconnection feature.
Your requirement can be divided into 2 sub tasks:
Maintain session info across multiple nodes: You can try Spring Sessions clustering backed by Redis (see: HttpSession with Redis). This very very simple and already has support for Spring Websockets (see: Spring Session & WebSockets).
Handle websockets traffic over multiple tomcat instances: There are several ways to do that.
The first way: Using a full-featured broker (eg: ActiveMQ) and try new feature Support multiple WebSocket servers (from: 4.2.0 RC1)
The second way: Using a full-feature broker and implement a distributed UserSessionRegistry (eg: Using Redis :D ). The default implementation DefaultUserSessionRegistry using an in-memory storage.
Updated: I've written a simple implementation using Redis, try it if you are interested
To configure a full-featured broker (broker relay), you can try:
public class WebSocketConfig extends AbstractWebSocketMessageBrokerConfigurer {
...
#Autowired
private RedisConnectionFactory redisConnectionFactory;
#Override
public void configureMessageBroker(MessageBrokerRegistry config) {
config.enableStompBrokerRelay("/topic", "/queue")
.setRelayHost("localhost") // broker host
.setRelayPort(61613) // broker port
;
config.setApplicationDestinationPrefixes("/app");
}
#Bean
public UserSessionRegistry userSessionRegistry() {
return new RedisUserSessionRegistry(redisConnectionFactory);
}
...
}
and
import java.util.Set;
import org.springframework.data.redis.connection.RedisConnectionFactory;
import org.springframework.data.redis.core.BoundHashOperations;
import org.springframework.data.redis.core.BoundSetOperations;
import org.springframework.data.redis.core.RedisOperations;
import org.springframework.data.redis.core.RedisTemplate;
import org.springframework.data.redis.core.StringRedisTemplate;
import org.springframework.data.redis.serializer.StringRedisSerializer;
import org.springframework.messaging.simp.user.UserSessionRegistry;
import org.springframework.util.Assert;
/**
* An implementation of {#link UserSessionRegistry} backed by Redis.
* #author thanh
*/
public class RedisUserSessionRegistry implements UserSessionRegistry {
/**
* The prefix for each key of the Redis Set representing a user's sessions. The suffix is the unique user id.
*/
static final String BOUNDED_HASH_KEY_PREFIX = "spring:websockets:users:";
private final RedisOperations<String, String> sessionRedisOperations;
#SuppressWarnings("unchecked")
public RedisUserSessionRegistry(RedisConnectionFactory redisConnectionFactory) {
this(createDefaultTemplate(redisConnectionFactory));
}
public RedisUserSessionRegistry(RedisOperations<String, String> sessionRedisOperations) {
Assert.notNull(sessionRedisOperations, "sessionRedisOperations cannot be null");
this.sessionRedisOperations = sessionRedisOperations;
}
#Override
public Set<String> getSessionIds(String user) {
Set<String> entries = getSessionBoundHashOperations(user).members();
return (entries != null) ? entries : Collections.<String>emptySet();
}
#Override
public void registerSessionId(String user, String sessionId) {
getSessionBoundHashOperations(user).add(sessionId);
}
#Override
public void unregisterSessionId(String user, String sessionId) {
getSessionBoundHashOperations(user).remove(sessionId);
}
/**
* Gets the {#link BoundHashOperations} to operate on a username
*/
private BoundSetOperations<String, String> getSessionBoundHashOperations(String username) {
String key = getKey(username);
return this.sessionRedisOperations.boundSetOps(key);
}
/**
* Gets the Hash key for this user by prefixing it appropriately.
*/
static String getKey(String username) {
return BOUNDED_HASH_KEY_PREFIX + username;
}
#SuppressWarnings("rawtypes")
private static RedisTemplate createDefaultTemplate(RedisConnectionFactory connectionFactory) {
Assert.notNull(connectionFactory, "connectionFactory cannot be null");
StringRedisTemplate template = new StringRedisTemplate(connectionFactory);
template.setKeySerializer(new StringRedisSerializer());
template.setValueSerializer(new StringRedisSerializer());
template.afterPropertiesSet();
return template;
}
}
Maintain session info across multiple nodes:
Suppose we have 2 server host, backed up with load balancer.
Websockets are socket connection from browser to specific server host.eg host1
Now if host1 goes down, socket connection from load balancer - host 1 will break.
How spring will reopen same websocket connection from load balancer to host 2 ? browser should not open new websocket connection