Stomp Client not always fetching data even if connection is established - websocket

I am sending data from spring boot to client using stomp client and web socket. It is able to send data to the first user but as soon as user increases it is fetching data for only some users. This seems weird because its behavior should be same for all the users. I have found out after extensive researching that the reason for this is because i am connecting to a queue ('/user/queue') and have more than one client listening to it. How to avoid this problem or is it impossible to solve this issue.
My controller code-
#Controller
public class ScheduledUpdatesOnTopic {
#Autowired
public SimpMessageSendingOperations messagingTemplate;
#Autowired
private SimpMessagingTemplate template;
DateFormat df = new SimpleDateFormat("yyyy/MM/dd HH:mm:ss");
Date date = new Date();
String json[][] = {{"Lokesh Gupta","34","India",df.format(date)},{"Meenal","23","Pakistan",df.format(date)},{"Gongo","12","Indonesia",df.format(date)},{"Abraham","17","US",df.format(date)},{"Saddam","56","Iraq",df.format(date)},{"Yimkov","67","Japan",df.format(date)},{"Salma","22","Pakistan",df.format(date)},{"Georgia","28","Russia",df.format(date)},{"Jaquline","31","Sri Lanka",df.format(date)},{"Chenchui","78","China",df.format(date)}};
String t[] = {"Lokesh Gupta","34","India","11/8/2017"};
String temp[][];
int p=0;
int count=0;
private MessageHeaderInitializer headerInitializer;
#MessageMapping("/hello")
public void start(SimpMessageHeaderAccessor accessor) throws Exception
{
String applicantId=accessor.getSessionId();
System.out.println("session id " + applicantId);
this.messagingTemplate.convertAndSendToUser(applicantId,"/queue/cache",json,createHeaders(applicantId));
}
private MessageHeaders createHeaders(String sessionId) {
SimpMessageHeaderAccessor headerAccessor = SimpMessageHeaderAccessor.create(SimpMessageType.MESSAGE);
if (getHeaderInitializer() != null) {
getHeaderInitializer().initHeaders(headerAccessor);
}
headerAccessor.setSessionId(sessionId);
headerAccessor.setLeaveMutable(true);
return headerAccessor.getMessageHeaders();
}
public MessageHeaderInitializer getHeaderInitializer() {
return this.headerInitializer;
}
public void setHeaderInitializer(MessageHeaderInitializer headerInitializer) {
this.headerInitializer = headerInitializer;
}
And client side html is-
var socket = new SockJS('/gs-guide-websocket');
var stompClient = Stomp.over(socket);
stompClient.connect({ }, function(frame) {
console.log('Connected this ' + frame);
stompClient.subscribe("/user/queue/cache", function(data) {
// code to display this data..........
});
I have to use queue because that is the only way to send data to particular session ids. Any help will be appreciated !!

It sounds like you need to use the "Request-Reply" messaging pattern.
When the client connects to the server on the common queue, it includes a private return address. This return address can be used to generate a new private message queue name for use by the server and client exclusively (since they are the only 2 that know the private return address. The server can then send the client data over the private message queue.
The return address could be a random UUID for example, and the private queue name could be /queue/private. .
This "Request-Reply" messaging pattern is more formally explained here, among other useful messaging patterns:
http://www.enterpriseintegrationpatterns.com/patterns/messaging/ReturnAddress.html

Related

Nifi Custom Processor errors with a "ControllerService found was not a WebSocket ControllerService but a com.sun.proxy.$Proxy75"

*** Update: I have changed my approach as described in my answer to the question, due to which the original issue reported becomes moot. ***
I'm trying to develop a Nifi application that provides a WebSocket interface to Kakfa. I could not accomplish this using the standard Nifi components as I have tried below (it may not make sense but intuitively this is what I want to accomplish):
I have now created a custom Processor "ReadFromKafka" that I intend to use as shown in the image below. "ReadFromKafka" would use the same implementation as the standard "PutWebSocket" component but would read messages from a Kafka Topic and send as response to the WebSocket client.
I have provided a code snippet of the implementation below:
#SystemResourceConsideration(resource = SystemResource.MEMORY)
public class ReadFromKafka extends AbstractProcessor {
public static final PropertyDescriptor PROP_WS_SESSION_ID = new PropertyDescriptor.Builder()
.name("websocket-session-id")
.displayName("WebSocket Session Id")
.description("A NiFi Expression to retrieve the session id. If not specified, a message will be " +
"sent to all connected WebSocket peers for the WebSocket controller service endpoint.")
.required(true)
.addValidator(StandardValidators.NON_BLANK_VALIDATOR)
.expressionLanguageSupported(ExpressionLanguageScope.FLOWFILE_ATTRIBUTES)
.defaultValue("${" + ATTR_WS_SESSION_ID + "}")
.build();
public static final PropertyDescriptor PROP_WS_CONTROLLER_SERVICE_ID = new PropertyDescriptor.Builder()
.name("websocket-controller-service-id")
.displayName("WebSocket ControllerService Id")
.description("A NiFi Expression to retrieve the id of a WebSocket ControllerService.")
.required(true)
.addValidator(StandardValidators.NON_BLANK_VALIDATOR)
.expressionLanguageSupported(ExpressionLanguageScope.FLOWFILE_ATTRIBUTES)
.defaultValue("${" + ATTR_WS_CS_ID + "}")
.build();
public static final PropertyDescriptor PROP_WS_CONTROLLER_SERVICE_ENDPOINT = new PropertyDescriptor.Builder()
.name("websocket-endpoint-id")
.displayName("WebSocket Endpoint Id")
.description("A NiFi Expression to retrieve the endpoint id of a WebSocket ControllerService.")
.required(true)
.addValidator(StandardValidators.NON_BLANK_VALIDATOR)
.expressionLanguageSupported(ExpressionLanguageScope.FLOWFILE_ATTRIBUTES)
.defaultValue("${" + ATTR_WS_ENDPOINT_ID + "}")
.build();
public static final PropertyDescriptor PROP_WS_MESSAGE_TYPE = new PropertyDescriptor.Builder()
.name("websocket-message-type")
.displayName("WebSocket Message Type")
.description("The type of message content: TEXT or BINARY")
.required(true)
.addValidator(StandardValidators.NON_BLANK_VALIDATOR)
.defaultValue(WebSocketMessage.Type.TEXT.toString())
.expressionLanguageSupported(ExpressionLanguageScope.FLOWFILE_ATTRIBUTES)
.build();
public static final Relationship REL_SUCCESS = new Relationship.Builder()
.name("success")
.description("FlowFiles that are sent successfully to the destination are transferred to this relationship.")
.build();
public static final Relationship REL_FAILURE = new Relationship.Builder()
.name("failure")
.description("FlowFiles that failed to send to the destination are transferred to this relationship.")
.build();
private static final List<PropertyDescriptor> descriptors;
private static final Set<Relationship> relationships;
static{
final List<PropertyDescriptor> innerDescriptorsList = new ArrayList<>();
innerDescriptorsList.add(PROP_WS_SESSION_ID);
innerDescriptorsList.add(PROP_WS_CONTROLLER_SERVICE_ID);
innerDescriptorsList.add(PROP_WS_CONTROLLER_SERVICE_ENDPOINT);
innerDescriptorsList.add(PROP_WS_MESSAGE_TYPE);
descriptors = Collections.unmodifiableList(innerDescriptorsList);
final Set<Relationship> innerRelationshipsSet = new HashSet<>();
innerRelationshipsSet.add(REL_SUCCESS);
innerRelationshipsSet.add(REL_FAILURE);
relationships = Collections.unmodifiableSet(innerRelationshipsSet);
}
#Override
public Set<Relationship> getRelationships() {
return relationships;
}
#Override
public final List<PropertyDescriptor> getSupportedPropertyDescriptors() {
return descriptors;
}
#Override
public void onTrigger(final ProcessContext context, final ProcessSession processSession) throws ProcessException {
final FlowFile flowfile = processSession.get();
if (flowfile == null) {
return;
}
final String sessionId = context.getProperty(PROP_WS_SESSION_ID)
.evaluateAttributeExpressions(flowfile).getValue();
final String webSocketServiceId = context.getProperty(PROP_WS_CONTROLLER_SERVICE_ID)
.evaluateAttributeExpressions(flowfile).getValue();
final String webSocketServiceEndpoint = context.getProperty(PROP_WS_CONTROLLER_SERVICE_ENDPOINT)
.evaluateAttributeExpressions(flowfile).getValue();
final String messageTypeStr = context.getProperty(PROP_WS_MESSAGE_TYPE)
.evaluateAttributeExpressions(flowfile).getValue();
final WebSocketMessage.Type messageType = WebSocketMessage.Type.valueOf(messageTypeStr);
if (StringUtils.isEmpty(sessionId)) {
getLogger().debug("Specific SessionID not specified. Message will be broadcast to all connected clients.");
}
if (StringUtils.isEmpty(webSocketServiceId)
|| StringUtils.isEmpty(webSocketServiceEndpoint)) {
transferToFailure(processSession, flowfile, "Required WebSocket attribute was not found.");
return;
}
final ControllerService controllerService = context.getControllerServiceLookup().getControllerService(webSocketServiceId);
if (controllerService == null) {
getLogger().debug("ControllerService is NULL");
transferToFailure(processSession, flowfile, "WebSocket ControllerService was not found.");
return;
} else if (!(controllerService instanceof WebSocketService)) {
getLogger().debug("ControllerService is not instance of WebSocketService");
transferToFailure(processSession, flowfile, "The ControllerService found was not a WebSocket ControllerService but a "
+ controllerService.getClass().getName());
return;
}
...
processSession.getProvenanceReporter().send(updatedFlowFile, transitUri.get(), transmissionMillis);
processSession.transfer(updatedFlowFile, REL_SUCCESS);
processSession.commit();
} catch (WebSocketConfigurationException|IllegalStateException|IOException e) {
// WebSocketConfigurationException: If the corresponding WebSocketGatewayProcessor has been stopped.
// IllegalStateException: Session is already closed or not found.
// IOException: other IO error.
getLogger().error("Failed to send message via WebSocket due to " + e, e);
transferToFailure(processSession, flowfile, e.toString());
}
}
private FlowFile transferToFailure(final ProcessSession processSession, FlowFile flowfile, final String value) {
flowfile = processSession.putAttribute(flowfile, ATTR_WS_FAILURE_DETAIL, value);
processSession.transfer(flowfile, REL_FAILURE);
return flowfile;
}
}
I have deployed the custom processor and when I connect to it using the Chrome "Simple Web Socket Client" I can see the following message in the logs:
ControllerService found was not a WebSocket ControllerService but a com.sun.proxy.$Proxy75
I'm using the exact same code as in PutWebSocket and can't figure out why it would behave any different when I use my custom Processor. I have configured "JettyWebSocketServer" as the ControllerService under "ListenWebSocket" as shown in the image below.
Additional exception details seen in the log are provided below:
java.lang.ClassCastException: class com.sun.proxy.$Proxy75 cannot be cast to class org.apache.nifi.websocket.WebSocketService (com.sun.proxy.$Proxy75 is in unnamed module of loader org.apache.nifi.nar.InstanceClassLoader #35c646b5; org.apache.nifi.websocket.WebSocketService is in unnamed module of loader org.apache.nifi.nar.NarClassLoader #361abd01)
I ended up modifying my flow to utilize out-of-box ListenWebSocket, PutWebSocket Processors, and a custom "FetchFromKafka" Processor that is a modified version of ConsumeKafkaRecord. With this I'm able to provide a WebSocket interface to Kafka. I have provided a screenshot of the updated flow below. More work needs to be done with the custom Processor to support multiple sessions.

how to consume events from kafka by a Spring Rest endpoint

I'm new to Kafka. I've seen that the consumer is "always running" and retrieves messages from a topic as soon as been published.
In a typical database web application you have a rest API that connects to DB and returns some response.
From what I see the consumer stays active and never close.
So I don't figure out how to return a subset of messages from a topic based on client request.
I thought the service would create a consumer to get what I need, but as far as consumer never close, I guess my opinion is not correct.
What should I do?
Then it's a simple question of persisting messages rceived thru KafkaListener, let's say adding each of them to a simple collecton (along with its timestamp) and implementing an endpoint to filter the messages accordingly and returning some of them.
#Controller
public class KafkaController {
#Autowired
private KafkaProducerConfig kafkaProducerConfig;
private Map<Date, String> msgMap = new HashMap();
#KafkaListener(topics = "myTopic", groupId = "myGroup")
public void listenAndAddMsg(String message) {
msgMap.put(new Date(), message);
}
#PostMapping("messages")
#ResponseBody
public String filterMessages(#RequestBody Interval interval) {
return msgMap.entrySet()
.stream()
.filter(map -> map.getKey().after(interval.getStartDate()) && map.getKey().before(interval.getEndDate()))
.collect(Collectors.toMap(map -> map.getKey(), map -> map.getValue()));
}
}
public class Interval {
private Date startDate;
private Date endDate;
// setters and getters
}

Why isnt my sockets onsubscribe event getting used?

I am using java springboot with maven in order to get the spring boot starter socket package. My clients are using angular with stompjs and sockjs-client.
I am trying to set up a simple web socket application that allows for multiple rooms based on a roomId. When a client joins a room they should receive the last five messages sent in that room.
My Springboot app has three classes, the basic Application.java that I use to run the app, a web socket config class and a web socket controller:
#Controller
public class WebSocketController {
private final SimpMessagingTemplate template;
#Autowired
WebSocketController(SimpMessagingTemplate template){
this.template = template;
}
#MessageMapping("/meeting/{roomId}")
private void sendMessageTpPrivateRoom(
String message,
#DestinationVariable String roomId
) throws IOException {
System.out.println("message sent to: " + roomId);
this.template.convertAndSend("/meeting/" + roomId, message);
addToHistory(roomId, message);
}
#SubscribeMapping("/meeting/{roomId}")
public String chatInit(#DestinationVariable String roomId) {
System.out.println("Someone joined room: " + roomId);
return getLastFiveMessages(roomId);
}
}
#Configuration
#EnableWebSocketMessageBroker
public class WebSocketConfiguration
extends AbstractWebSocketMessageBrokerConfigurer {
#Override
public void registerStompEndpoints(StompEndpointRegistry registry) {
registry.addEndpoint("/socket")
.setAllowedOrigins("*")
.withSockJS();
}
#Override
public void configureMessageBroker(MessageBrokerRegistry registry) {
registry.setApplicationDestinationPrefixes("/app")
.enableSimpleBroker("/meeting");
}
}
my clients are subscribing to the socket like so:
stompClient.subscribe(`app/meeting/${roomId}`, (message) => {
if (message.body) {
console.log(message.body);
messages += '<br>' + message.body;
}
});
and sending messages like so:
this.stompClient.send(`/app/meeting/${this.roomId}` , {}, message);
The message sending and handling is working great, when I set up three clients, two in room one, and one in room two, the room two messages are not being seen in room one and the room one messages are seen by both clients.
However the on subscribe event is not firing no matter what room I join. It is very necessary that when a client joins room one, they should receive some sort of history of that room. Any advice as to why my SubscribeMapping method is not being triggered when a client subscribes to the room?
The /meeting part will be implicitly added to URL you provide when subscribing. So your mapping will look like this:
#SubscribeMapping("/${roomId}")
public String chatInit(#DestinationVariable String roomId) {
System.out.println("Someone joined room: " + roomId);
return getLastFiveMessages(roomId);
}
Source: https://docs.spring.io/spring/docs/5.0.0.BUILD-SNAPSHOT/spring-framework-reference/html/websocket.html

Send websocket message to user across dynos

I have a spring boot application running on heroku. I make use of websockets for sending messages to and from client and server for a specific user . I use spring boot's SimpMessagingTemplate.convertAndSendToUser to send and receive messages, which works fine for when a user needs get a message back from the server. I use Heroku session affinity which means that even if I scale up the number of sessions the user and websocket still share the same session.
My problem comes when I need a user to send a message to another user. It works fine if both users are sharing the session, but not if the message will not come through.
Is it possible to send a message from one user to another across different sessions using, SimpMessagingTemple? Or would I need to use a message broker, eg Redis.
I was looking into implementing sending a message using StringRedisTemplate but not sure how to send a message to a particular user.
private SimpMessagingTemplate messagingTemplate;
#Autowired
public MessageController(SimpMessagingTemplate messagingTemplate) {
this.messagingTemplate = messagingTemplate;
}
#MessageMapping("/secured/user-in")
public void sendToDevice(Message msg, #AuthenticationPrincipal User principal) throws Exception {
if (msg.getTo() != null) {
String email = msg.getTo();
Message out = new Message();
out.setMsg(msg.getMsg());
out.setFrom(msg.getFrom());
out.setTo(msg.getTo());
out.setSentTime(new Date());
out.setStatus(msg.getStatus());
messagingTemplate.convertAndSendToUser(email, "/secured/topic", out);
}
}
JS
function connect() {
var socket = new SockJS('/secured/user-in');
ST.stompClient = Stomp.over(socket);
var headers = {};
headers[ST.getHeader()] = ST.getToken();
ST.getStompClient().connect(headers, function (frame) {
retries = 1;
console.log('Connected: ' + frame);
ST.getStompClient().subscribe('/user/secured/topic', function (event){
var msg = JSON.parse(event.body);
showMessage(msg.msg);
});
});
}
UPDATE 1
I am guessing I could do something like this, as done here:
SimpMessageHeaderAccessor headerAccessor = SimpMessageHeaderAccessor
.create(SimpMessageType.MESSAGE);
headerAccessor.setSessionId(sessionId);
headerAccessor.setLeaveMutable(true);
messagingTemplate.convertAndSendToUser(sessionId,"/queue/something", payload,
headerAccessor.getMessageHeaders());
But how could I get the session id of another user, I am using Redis to store session info: #EnableRedisHttpSession
I had my terminology a bit mixed up I was trying to send a message to another user on another dyno rather than session.
Ended up using redis sub/pub.
So when a message is receive by the controller it is published to redis, and the redis MessageListenerAdapter envokes the convertAndSendToUser method.
#MessageMapping("/secured/user-in")
public void sendToDevice(Message msg, #AuthenticationPrincipal User principal) throws Exception {
publishMessageToRedis(msg);
}
private void publishMessageToRedis(Message message) throws JsonProcessingException {
ObjectMapper objectMapper = new ObjectMapper();
String messageString = objectMapper.writeValueAsString(message);
stringRedisTemplate.convertAndSend("message", messageString);
}
redis config
#Bean
RedisMessageListenerContainer container( MessageListenerAdapter chatMessageListenerAdapter) throws URISyntaxException {
RedisMessageListenerContainer container = new RedisMessageListenerContainer();
container.setConnectionFactory(connectionFactory());
container.addMessageListener(chatMessageListenerAdapter, new PatternTopic("message"));
return container;
}
#Bean("chatMessageListenerAdapter")
MessageListenerAdapter chatMessageListenerAdapter(RedisReceiver redisReceiver) {
return new MessageListenerAdapter(redisReceiver, "receiveChatMessage");
}
public class RedisReceiver {
private static final Logger LOG = LogManager.getLogger(RedisReceiver.class);
private final WebSocketMessageService webSocketMessageService;
#Autowired
public RedisReceiver(WebSocketMessageService webSocketMessageService) {
this.webSocketMessageService = webSocketMessageService;
}
// Invoked when message is publish to "chat" channel
public void receiveChatMessage(String messageStr) throws IOException {
ObjectMapper objectMapper = new ObjectMapper();
Message message = objectMapper.readValue(messageStr, Message.class);
webSocketMessageService.sendChatMessage(message);
}
}
#Service
public class WebSocketMessageService {
private final SimpMessagingTemplate template;
private static final Logger LOG = LogManager.getLogger(WebSocketMessageService.class);
public WebSocketMessageService(SimpMessagingTemplate template) {
this.template = template;
}
public void sendChatMessage(Message message) {
template.convertAndSendToUser(message.getTo(), "/secured/topic", message);
}
}
Solution was based off this git repository

How to mimic SimpMessagingTemplate.convertAndSendToUser using RabbitTemplate?

So I've been reading about Spring Message Relay (Spring Messaging stuff) capability with a RabbitMQ broker. What I want to achieve is as follows:
Have a service (1), which acts as a message relay between rabbitmq and a browser. This works fine now. I'm using MessageBrokerRegistry.enableStompBrokerRelay to do that.
Have another service (2) on the back-end, which will send a message to a known queue onto RabbitMQ and have that message routed to a specific user. As a sender, I want to have a control over who the message gets delivered to.
Normally, you'd use SimpMessagingTemplate to do that. Problem is though, that the origin of the message doesn't actually have access to that template, as it's not acting as a relay, it's not using websockets and it doesn't hold mapping of queue names to session ids.
One way I could think of doing it, is writing a simple class on the service 1, which will listen on all queues and forward them using simp template. I fell however this is not an ideal way to do it, and I feel like there might be already a way to do it using Spring.
Can you please advise?
This question got me thinking about the same dilemma I was facing. I have started playing with a custom UserDestinationResolver that arrives at a consistent topic naming scheme that uses just the username and not the session ID used by the default resolver.
That lets me subscribe in JS to "/user/exchange/amq.direct/current-time" but send via a vanilla RabbitMQ application to "/exchange/amqp.direct/users.me.current-time" (to a user named "me").
The latest source code is here and I am "registering" it as a #Bean in an existing #Configuration class that I had.
Here's the custom UserDestinationResolver itself:
public class ConsistentUserDestinationResolver implements UserDestinationResolver {
private static final Pattern USER_DEST_PREFIXING_PATTERN =
Pattern.compile("/user/(?<name>.+?)/(?<routing>.+)/(?<dest>.+?)");
private static final Pattern USER_AUTHENTICATED_PATTERN =
Pattern.compile("/user/(?<routing>.*)/(?<dest>.+?)");
#Override
public UserDestinationResult resolveDestination(Message<?> message) {
SimpMessageHeaderAccessor accessor = MessageHeaderAccessor.getAccessor(message, SimpMessageHeaderAccessor.class);
final String destination = accessor.getDestination();
final String authUser = accessor.getUser() != null ? accessor.getUser().getName() : null;
if (destination != null) {
if (SimpMessageType.SUBSCRIBE.equals(accessor.getMessageType()) ||
SimpMessageType.UNSUBSCRIBE.equals(accessor.getMessageType())) {
if (authUser != null) {
final Matcher authMatcher = USER_AUTHENTICATED_PATTERN.matcher(destination);
if (authMatcher.matches()) {
String result = String.format("/%s/users.%s.%s",
authMatcher.group("routing"), authUser, authMatcher.group("dest"));
UserDestinationResult userDestinationResult =
new UserDestinationResult(destination, Collections.singleton(result), result, authUser);
return userDestinationResult;
}
}
}
else if (accessor.getMessageType().equals(SimpMessageType.MESSAGE)) {
final Matcher prefixMatcher = USER_DEST_PREFIXING_PATTERN.matcher(destination);
if (prefixMatcher.matches()) {
String user = prefixMatcher.group("name");
String result = String.format("/%s/users.%s.%s",
prefixMatcher.group("routing"), user, prefixMatcher.group("dest"));
UserDestinationResult userDestinationResult =
new UserDestinationResult(destination, Collections.singleton(result), result, user);
return userDestinationResult;
}
}
}
return null;
}
}

Resources