SSE connection keeps failing every 5 minutes - spring

I'm exposing a simple SSE endpoint using the SseEmitter Spring API, persisting all the emitters in a ConcurrentHashMap. The timeout for each emitter is set to 24 hours. Every 10 seconds I'm sending a message to all the clients. Clients are subscribed with native EventSource implementation, listening for events of particular name.
Unfortunately, I've noticed that every 5 minutes the connection is lost and reestablished again - even though timeout of emitter was explicitly set to 24 hours. Client's part does log it as an error, however on server side there's nothing. The issue occurs on both Tomcat and Jetty. I'd like to keep the session open without any interruptions, so resetting the connection every 5 minutes is unacceptable. Any ideas why this could be happening?
#RestController
#RequestMapping("api/v1/sse")
class SseController {
private val emitters = ConcurrentHashMap<String, SseEmitter>()
#GetMapping
fun initConnection(#RequestParam token: String): SseEmitter {
logger.info { "Init connection from $token" }
val emitter = SseEmitter(24 * 60 * 60 * 1000)
emitter.onCompletion {
logger.info { "Completion" }
emitters.remove(token)
}
emitter.onTimeout { logger.info { "Timeout " } }
emitter.onError { logger.error(it) { "Error" } }
emitters[token] = emitter
return emitter
}
#Scheduled(fixedRate = 10000)
fun send() {
emitters.forEach { (k, v) ->
logger.info { "Sending message to $k" }
v.send(
SseEmitter.event()
.id(UUID.randomUUID().toString())
.name("randomEvent")
.data("some data")
)
}
}
}
const eventSource = new EventSource(url);
eventSource.addEventListener('randomEvent', (e) =>
console.log(e.data)
);
eventSource.onerror = (e) => console.log(e);

Alright, seems it was an issue with Stackblitz's service worker. I've just implemented the same client-side solution in Chrome's plain console and the disconnecting is no longer happening.

Related

Eagerly caching Mono

I'm looking to eagerly cache the results of a Reactor Mono. It's scheduled to be updated in cache every 10 minutes, but since the Mono is only evaluated when subscribed to, the task doesn't actually refresh the cache.
Example:
#Scheduled(fixedRate = 10 * 60 * 1000 + 3000)
fun getMessage(): Mono<String> {
return Mono.just("Hello")
.map { it.toUpperCase() }
.cache(Duration.ofMinutes(10))
}
You need to store your Mono somewhere, otherwise each invocation of the method (through the Scheduled or directly) will return a different instance.
Perhaps as a companion object?
Here is how I would do it naïvely in Java:
protected Mono<String> cached;
//for the scheduler to periodically eagerly refresh the cache
#Scheduled(fixedRate = 10 * 60 * 1000 + 3000)
void refreshCache() {
this.cached = Mono.just("Hello")
.map { it.toUpperCase() }
.cache(Duration.ofMinutes(10));
this.cached.subscribe(v -> {}, e -> {}); //swallows errors during refresh
}
//for users
public Mono<String> getMessage() {
return this.cached;
}

ZMQ - forwarding request between sockets or one-time proxy

I'm struggling with connecting two sockets:
frontend (ROUTER) - which handles clients request and forward them to backend
backend (ROUTER) - which receives request from frontend and deals with them with the use of number of workers ( which require some initialization, configuration etc).
The server code looks like this:
void server_task::run() {
frontend.bind("tcp://*:5570");
backend.bind("inproc://backend");
zmq::pollitem_t items[] = {
{ frontend, 0, ZMQ_POLLIN, 0 },
{ backend, 0, ZMQ_POLLIN, 0}
};
try {
while (true) {
zmq::poll(&items[0], 2, -1);
if (items[0].revents & ZMQ_POLLIN) {
frontend_h();
}
if (items[1].revents & ZMQ_POLLIN) {
backend_h();
}
}
}
catch (std::exception& e) {
LOG(info) << e.what();
}
}
frontend_h and backend_h are handler classes, each having access to both sockets.
The question is:
Considering synchronous execution of frontend_h() and backend_h() how can I forward the request dealt in frontend_h() to backend_h()?
I tried to simply re-send the message using backend socket like that:
void frontend_handler::handle_query(std::unique_ptr<zmq::message_t> identity, std::unique_ptr<zmq::message_t> request) {
zmq::message_t req_msg, req_identity;
req_msg.copy(request.get());
req_identity.copy(identity.get());
zmq::message_t header = create_header(request_type::REQ_QUERY);
backend.send(header, ZMQ_SNDMORE);
backend.send(message);
}
But it stucks on zmq::poll in run() after the execution of handle_query().
Stucks on zmq::poll()?
Your code has instructed the .poll() method to block, exactly as documentation states:
If the value of timeout is -1, zmq_poll() shall block indefinitely until a requested event has occurred...
How can I forward the request?
It seems pretty expensive to re-marshall each message ( +1 for using at least the .copy() method and avoiding re-packing overheads ) once your code is co-located and the first, receiving handler, can request and invoke any appropriate method of the latter directly ( and without any Context()-processing associated efforts and overheads.

AMQP Appender pending message count

We are sending audit log messages to a RabbitMQ cluster which is sometimes unavailable for reasons we cannot influence.
When the queue is not available, log messages start to accumulate locally and we get a out-of-memory eventually on the client.
We are using a AMQP Appender to submit our messages.
Is there a way we can query the count of pending log messages and raise an alert when messages start adding up?
Well, it isn't possible. There is just no any hooks to do that.
You can consider, though, to decrease maxSenderRetries from default 30 to 1 or 2. After that you'll start to lose log messages:
int retries = event.incrementRetries();
if (retries < AmqpAppender.this.maxSenderRetries) {
// Schedule a retry based on the number of times I've tried to re-send this
AmqpAppender.this.retryTimer.schedule(new TimerTask() {
#Override
public void run() {
AmqpAppender.this.events.add(event);
}
}, (long) (Math.pow(retries, Math.log(retries)) * 1000));
}
else {
addError("Could not send log message " + logEvent.getMessage()
+ " after " + AmqpAppender.this.maxSenderRetries + " retries", e);
}
We might have to expose queueSize option instead of default:
public LinkedBlockingQueue() {
this(Integer.MAX_VALUE);
}
Feel free to raise a JIRA on the matter.

Recurring message Masstransit rabbitmq

I am trying to implement a heartbeat feature for my application hence was trying to implement recurring message feature from masstransit rabbitmq. I was trying to implement it on the sample given on masstransit's website. Here's all of the code.
namespace MasstransitBasicSample
{
using System;
using System.Threading.Tasks;
using MassTransit;
using MassTransit.Scheduling;
class Program
{
static void Main(string[] args)
{
var bus = Bus.Factory.CreateUsingRabbitMq(sbc =>
{
var host = sbc.Host(new Uri("rabbitmq://localhost"), h =>
{
h.Username("guest");
h.Password("guest");
});
sbc.UseMessageScheduler(new Uri("rabbitmq://localhost/quartz"));
sbc.ReceiveEndpoint(host, "test_queue", ep =>
{
ep.Handler<YourMessage>(context =>
{
return Console.Out.WriteLineAsync($"Received: {context.Message.Text}");
});
ep.Handler<PollExternalSystem>(context =>
{
return Console.Out.WriteLineAsync($"Received: {context.Message}");
});
});
});
bus.Start();
SetRecurring(bus);
Console.WriteLine("Press any key to exit");
Console.ReadKey();
bus.Stop();
}
private static async Task SetRecurring(IBusControl bus)
{
var schedulerEndpoint = await bus.GetSendEndpoint(new Uri("rabbitmq://localhost/quartz"));
var scheduledRecurringMessage = await schedulerEndpoint.ScheduleRecurringSend(
new Uri("rabbitmq://localhost/test_queue"), new PollExternalSystemSchedule(), new PollExternalSystem());
}
}
public class YourMessage { public string Text { get; set; } }
public class PollExternalSystemSchedule : DefaultRecurringSchedule
{
public PollExternalSystemSchedule()
{
CronExpression = "* * * * *"; // this means every minute
}
}
public class PollExternalSystem { }
}
I have created a queue called quartz in my rabbitmq queue.
When i run the application it sends a message to the quartz queue and that message just stays there , it does not go to the test queue.
I was also expecting another message to be sent to the quartz queue after a minute based on the cron expression, that also does not happen.
Is my setup wrong?
Any help would be much appreciated.
You need to run the scheduling service that listens on rabbitmq://localhost/quartz, where your messages are being sent.
The documentation page says:
There is a standalone MassTransit service, MassTransit.QuartzService,
which can be installed and used on servers for this purpose. It is
configured via the App.config file and is a good example of how to
build a standalone MassTransit service.
Alternatively, you can host Quartz scheduling in the same process by using in-memory scheduling, described here, by configuring it like this:
var busControl = Bus.Factory.CreateUsingRabbitMq(cfg =>
{
var host = cfg.Host(new Uri("rabbitmq://localhost/"), h =>
{
h.Username("guest");
h.Password("guest");
});
cfg.UseInMemoryScheduler();
});

Connection between js and akka-http websockets fails 95% of the time

I'm trying to setup a basic connection between an akka-http websocket server and simple javascript.
1 out of roughly 20 connections succeeds, the rest fails. I have no idea why the setup of the connection is so unreliable.
Application.scala:
import akka.actor.ActorSystem
import akka.http.scaladsl.Http
import akka.stream.ActorMaterializer
import services.WebService
import scala.concurrent.Await
import scala.concurrent.duration._
import java.util.concurrent.TimeoutException
object Application extends App {
implicit val system = ActorSystem("api")
implicit val materializer = ActorMaterializer()
import system.dispatcher
val config = system.settings.config
val interface = config.getString("app.interface")
val port = config.getInt("app.port")
val service = new WebService
val binding = Http().bindAndHandle(service.route, interface, port)
try {
Await.result(binding, 1 second)
println(s"server online at http://$interface:$port/")
} catch {
case exc: TimeoutException =>
println("Server took to long to startup, shutting down")
system.shutdown()
}
}
WebService.scala:
import actors.{PublisherActor, SubscriberActor}
import akka.actor.{Props, ActorSystem}
import akka.http.scaladsl.model.ws.{Message, TextMessage}
import akka.http.scaladsl.server.Directives
import akka.stream.Materializer
import akka.stream.scaladsl.{Source, Flow}
import scala.concurrent.duration._
class WebService(implicit fm: Materializer, system: ActorSystem) extends Directives {
import system.dispatcher
system.scheduler.schedule(15 second, 15 second) {
println("Timer message!")
}
def route =
get {
pathSingleSlash {
getFromResource("web/index.html")
} ~
path("helloworld") {
handleWebsocketMessages(websocketActorFlow)
}
}
def websocketActorFlow: Flow[Message, Message, Unit] =
Flow[Message].collect({
case TextMessage.Strict(msg) =>
println(msg)
TextMessage.Strict(msg.reverse)
})
}
client side:
<input type="text" id="inputMessage"/><br/>
<input type="button" value="Send!" onClick="sendMessage()"/><br/>
<span id="response"></span>
<script type="application/javascript">
var connection;
function sendMessage() {
connection.send(document.getElementById("inputMessage").value);
}
document.addEventListener("DOMContentLoaded", function (event) {
connection = new WebSocket("ws://localhost:8080/helloworld");
connection.onopen = function (event) {
connection.send("connection established");
};
connection.onmessage = function (event) {
console.log(event.data);
document.getElementById("response").innerHTML = event.data;
}
});
</script>
if the connection to the server fails I get a timeout message after 5 seconds which says the following:
[DEBUG] [07/23/2015 07:59:54.517] [api-akka.actor.default-dispatcher-27] [akka://api/user/$a/flow-76-3-publisherSource-prefixAndTail] Cancelling akka.stream.impl.MultiStreamOutputProcessor$SubstreamOutput#a54778 (after: 5000 ms)
No matter if the connection fails or succeeds, I always get the following log message:
[DEBUG] [07/23/2015 07:59:23.849] [api-akka.actor.default-dispatcher-4] [akka://api/system/IO-TCP/selectors/$a/0] New connection accepted
Look at that error message carefully... it is coming from a source I would not have expected, some "MultiStreamOutputProcessor" when I only expect to handle a single stream.
That tells me - along with the webSocketActorFlow - that maybe you are getting messages and they aren't being caught by the flow, and so you're ending up with substreams you never expected.
So instead of it "only working some of the time," maybe it is "working most of the time but unable to handle all of the input as you have demanded in the flow, and you are left with un-selectable streams that have to die first.
See if you can either a) make sure you get a grip on the streams so you don't end up with stragglers, b) bandaid adjust timeouts, and c) detect such occurences and cancel processing the downstream
https://groups.google.com/forum/#!topic/akka-user/x-tARRaJ0LQ
akka {
stream {
materializer {
subscription-timeout {
timeout=30s
}
}
}
}
http://grokbase.com/t/gg/akka-user/1561gr0jgt/debug-message-cancelling-akka-stream-impl-multistreamoutputprocessor-after-5000-ms

Resources