I'm looking to eagerly cache the results of a Reactor Mono. It's scheduled to be updated in cache every 10 minutes, but since the Mono is only evaluated when subscribed to, the task doesn't actually refresh the cache.
Example:
#Scheduled(fixedRate = 10 * 60 * 1000 + 3000)
fun getMessage(): Mono<String> {
return Mono.just("Hello")
.map { it.toUpperCase() }
.cache(Duration.ofMinutes(10))
}
You need to store your Mono somewhere, otherwise each invocation of the method (through the Scheduled or directly) will return a different instance.
Perhaps as a companion object?
Here is how I would do it naïvely in Java:
protected Mono<String> cached;
//for the scheduler to periodically eagerly refresh the cache
#Scheduled(fixedRate = 10 * 60 * 1000 + 3000)
void refreshCache() {
this.cached = Mono.just("Hello")
.map { it.toUpperCase() }
.cache(Duration.ofMinutes(10));
this.cached.subscribe(v -> {}, e -> {}); //swallows errors during refresh
}
//for users
public Mono<String> getMessage() {
return this.cached;
}
Related
I want to be able to execute multiple jobs concurrently on a Job Consumer. At the moment if I run one service instance and try to execute 2 jobs concurrently, 1 job waits for the other to complete (i.e. waits for the single job slot to become available).
However if I run 2 instances by using dotnet run twice to create 2 separate processes I am able to get the desired behavior where both jobs run at the same time.
Is it possible to run 2 (or more) jobs at the same time for a given consumer inside a single process? My application requires the ability to run several jobs concurrently but I don't have the ability to deploy many instances of my application.
Checking the application log I see this line which I feel may have something to do with it:
[04:13:43 DBG] Concurrent Job Limit: 1
I tried changing the SagaPartitionCount to something other than 1 on instance.ConfigureJobServiceEndpoints to no avail. I can't seem to get the Concurrent Job Limit to change.
My configuration looks like this:
services.AddMassTransit(x =>
{
x.AddDelayedMessageScheduler();
x.SetKebabCaseEndpointNameFormatter();
// registering the job consumer
x.AddConsumer<DeploymentConsumer>(typeof(DeploymentConsumerDefinition));
x.AddSagaRepository<JobSaga>()
.EntityFrameworkRepository(r =>
{
r.ExistingDbContext<JobServiceSagaDbContext>();
r.LockStatementProvider = new SqlServerLockStatementProvider();
});
// add other saga repositories here for JobTypeSaga and JobAttemptSaga here as well
x.UsingRabbitMq((context, cfg) =>
{
var rmq = configuration.GetSection("RabbitMq").Get<RabbitMq>();
cfg.Host(rmq.Host, rmq.Port, rmq.VirtualHost, h =>
{
h.Username(rmq.Username);
h.Password(rmq.Password);
});
cfg.UseDelayedMessageScheduler();
var options = new ServiceInstanceOptions()
.SetEndpointNameFormatter(context.GetService<IEndpointNameFormatter>() ?? KebabCaseEndpointNameFormatter.Instance);
cfg.ServiceInstance(options, instance =>
{
instance.ConfigureJobServiceEndpoints(js =>
{
js.SagaPartitionCount = 1;
js.FinalizeCompleted = true;
js.ConfigureSagaRepositories(context);
});
instance.ConfigureEndpoints(context);
});
});
}
Where DeploymentConsumerDefinition looks like
public class DeploymentConsumerDefinition : ConsumerDefinition<DeploymentConsumer>
{
protected override void ConfigureConsumer(IReceiveEndpointConfigurator endpointConfigurator,
IConsumerConfigurator<DeploymentConsumer> consumerConfigurator)
{
consumerConfigurator.Options<JobOptions<DeploymentConsumer>>(options =>
{
options.SetJobTimeout(TimeSpan.FromMinutes(20));
options.SetConcurrentJobLimit(10);
options.SetRetry(r =>
{
r.Ignore<InvalidOperationException>();
r.Interval(5, TimeSpan.FromSeconds(10));
});
});
}
}
Your definition should specify the job consumer message type, not the job consumer type:
public class DeploymentConsumerDefinition : ConsumerDefinition<DeploymentConsumer>
{
protected override void ConfigureConsumer(IReceiveEndpointConfigurator endpointConfigurator,
IConsumerConfigurator<DeploymentConsumer> consumerConfigurator)
{
// MESSAGE TYPE NOT CONSUMER TYPE
consumerConfigurator.Options<JobOptions<DeploymentCommand>>(options =>
{
options.SetJobTimeout(TimeSpan.FromMinutes(20));
options.SetConcurrentJobLimit(10);
options.SetRetry(r =>
{
r.Ignore<InvalidOperationException>();
r.Interval(5, TimeSpan.FromSeconds(10));
});
});
}
}
I'm exposing a simple SSE endpoint using the SseEmitter Spring API, persisting all the emitters in a ConcurrentHashMap. The timeout for each emitter is set to 24 hours. Every 10 seconds I'm sending a message to all the clients. Clients are subscribed with native EventSource implementation, listening for events of particular name.
Unfortunately, I've noticed that every 5 minutes the connection is lost and reestablished again - even though timeout of emitter was explicitly set to 24 hours. Client's part does log it as an error, however on server side there's nothing. The issue occurs on both Tomcat and Jetty. I'd like to keep the session open without any interruptions, so resetting the connection every 5 minutes is unacceptable. Any ideas why this could be happening?
#RestController
#RequestMapping("api/v1/sse")
class SseController {
private val emitters = ConcurrentHashMap<String, SseEmitter>()
#GetMapping
fun initConnection(#RequestParam token: String): SseEmitter {
logger.info { "Init connection from $token" }
val emitter = SseEmitter(24 * 60 * 60 * 1000)
emitter.onCompletion {
logger.info { "Completion" }
emitters.remove(token)
}
emitter.onTimeout { logger.info { "Timeout " } }
emitter.onError { logger.error(it) { "Error" } }
emitters[token] = emitter
return emitter
}
#Scheduled(fixedRate = 10000)
fun send() {
emitters.forEach { (k, v) ->
logger.info { "Sending message to $k" }
v.send(
SseEmitter.event()
.id(UUID.randomUUID().toString())
.name("randomEvent")
.data("some data")
)
}
}
}
const eventSource = new EventSource(url);
eventSource.addEventListener('randomEvent', (e) =>
console.log(e.data)
);
eventSource.onerror = (e) => console.log(e);
Alright, seems it was an issue with Stackblitz's service worker. I've just implemented the same client-side solution in Chrome's plain console and the disconnecting is no longer happening.
How can I execute this full CompletableFuture chain to run asynchronously using a separate executor
.thenApply(r -> {
return validateStudents();
})
.thenCompose(r -> {
return fetchAll(r);
})
.thenCompose(r -> {
return processAll(r);
})
.whenComplete((r, t) -> {
});
});
You can use the Async methods from CompletableFuture with the default ForkJoinPool
All async methods without an explicit Executor argument are performed using the ForkJoinPool.commonPool() (unless it does not support a parallelism level of at least two, in which case, a new Thread is created to run each task)
System.out.println(Thread.currentThread().getName());
CompletableFuture.supplyAsync(()->{
System.out.println(Thread.currentThread().getName());
return "supplyAsync";
}).thenApplyAsync(supply->{
System.out.println(Thread.currentThread().getName()+"----"+supply);
return "applyAsync";
}).thenComposeAsync(compose->{
System.out.println(Thread.currentThread().getName()+"----"+compose);
return CompletableFuture.completedStage("composeAsync");
});
Output :
main
ForkJoinPool.commonPool-worker-3
ForkJoinPool.commonPool-worker-3----supplyAsync
ForkJoinPool.commonPool-worker-3----applyAsync
You can also define custom thread pool and you can use that thread pool
ExecutorService pool = Executors.newFixedThreadPool(1);
System.out.println(Thread.currentThread().getName());
CompletableFuture.supplyAsync(()->{
System.out.println(Thread.currentThread().getName());
return "supplyAsync";
},pool).thenApplyAsync(supply->{
System.out.println(Thread.currentThread().getName()+"----"+supply);
return "applyAsync";
},pool).thenComposeAsync(compose->{
System.out.println(Thread.currentThread().getName()+"----"+compose);
return CompletableFuture.completedStage("composeAsync");
},pool);
Output :
main
pool-1-thread-1
pool-1-thread-1----supplyAsync
pool-1-thread-1----applyAsync
I am new to spring 5.
1) How I can log the method params which are Mono and flux type without blocking them?
2) How to map Models at API layer to Business object at service layer using Map-struct?
Edit 1:
I have this imperative code which I am trying to convert into a reactive code. It has compilation issue at the moment due to introduction of Mono in the argument.
public Mono<UserContactsBO> getUserContacts(Mono<LoginBO> loginBOMono)
{
LOGGER.info("Get contact info for login: {}, and client: {}", loginId, clientId);
if (StringUtils.isAllEmpty(loginId, clientId)) {
LOGGER.error(ErrorCodes.LOGIN_ID_CLIENT_ID_NULL.getDescription());
throw new ServiceValidationException(
ErrorCodes.LOGIN_ID_CLIENT_ID_NULL.getErrorCode(),
ErrorCodes.LOGIN_ID_CLIENT_ID_NULL.getDescription());
}
if (!loginId.equals(clientId)) {
if (authorizationFeignClient.validateManagerClientAccess(new LoginDTO(loginId, clientId))) {
loginId = clientId;
} else {
LOGGER.error(ErrorCodes.LOGIN_ID_VALIDATION_ERROR.getDescription());
throw new AuthorizationException(
ErrorCodes.LOGIN_ID_VALIDATION_ERROR.getErrorCode(),
ErrorCodes.LOGIN_ID_VALIDATION_ERROR.getDescription());
}
}
UserContactDetailEntity userContactDetail = userContactRepository.findByLoginId(loginId);
LOGGER.debug("contact info returned from DB{}", userContactDetail);
//mapstruct to map entity to BO
return contactMapper.userEntityToUserContactBo(userContactDetail);
}
You can try like this.
If you want to add logs you may use .map and add logs there. if filters are not passed it will return empty you can get it with swichifempty
loginBOMono.filter(loginBO -> !StringUtils.isAllEmpty(loginId, clientId))
.filter(loginBOMono1 -> loginBOMono.loginId.equals(clientId))
.filter(loginBOMono1 -> authorizationFeignClient.validateManagerClientAccess(new LoginDTO(loginId, clientId)))
.map(loginBOMono1 -> {
loginBOMono1.loginId = clientId;
return loginBOMono1;
})
.flatMap(o -> {
return userContactRepository.findByLoginId(o.loginId);
})
I am trying to implement a heartbeat feature for my application hence was trying to implement recurring message feature from masstransit rabbitmq. I was trying to implement it on the sample given on masstransit's website. Here's all of the code.
namespace MasstransitBasicSample
{
using System;
using System.Threading.Tasks;
using MassTransit;
using MassTransit.Scheduling;
class Program
{
static void Main(string[] args)
{
var bus = Bus.Factory.CreateUsingRabbitMq(sbc =>
{
var host = sbc.Host(new Uri("rabbitmq://localhost"), h =>
{
h.Username("guest");
h.Password("guest");
});
sbc.UseMessageScheduler(new Uri("rabbitmq://localhost/quartz"));
sbc.ReceiveEndpoint(host, "test_queue", ep =>
{
ep.Handler<YourMessage>(context =>
{
return Console.Out.WriteLineAsync($"Received: {context.Message.Text}");
});
ep.Handler<PollExternalSystem>(context =>
{
return Console.Out.WriteLineAsync($"Received: {context.Message}");
});
});
});
bus.Start();
SetRecurring(bus);
Console.WriteLine("Press any key to exit");
Console.ReadKey();
bus.Stop();
}
private static async Task SetRecurring(IBusControl bus)
{
var schedulerEndpoint = await bus.GetSendEndpoint(new Uri("rabbitmq://localhost/quartz"));
var scheduledRecurringMessage = await schedulerEndpoint.ScheduleRecurringSend(
new Uri("rabbitmq://localhost/test_queue"), new PollExternalSystemSchedule(), new PollExternalSystem());
}
}
public class YourMessage { public string Text { get; set; } }
public class PollExternalSystemSchedule : DefaultRecurringSchedule
{
public PollExternalSystemSchedule()
{
CronExpression = "* * * * *"; // this means every minute
}
}
public class PollExternalSystem { }
}
I have created a queue called quartz in my rabbitmq queue.
When i run the application it sends a message to the quartz queue and that message just stays there , it does not go to the test queue.
I was also expecting another message to be sent to the quartz queue after a minute based on the cron expression, that also does not happen.
Is my setup wrong?
Any help would be much appreciated.
You need to run the scheduling service that listens on rabbitmq://localhost/quartz, where your messages are being sent.
The documentation page says:
There is a standalone MassTransit service, MassTransit.QuartzService,
which can be installed and used on servers for this purpose. It is
configured via the App.config file and is a good example of how to
build a standalone MassTransit service.
Alternatively, you can host Quartz scheduling in the same process by using in-memory scheduling, described here, by configuring it like this:
var busControl = Bus.Factory.CreateUsingRabbitMq(cfg =>
{
var host = cfg.Host(new Uri("rabbitmq://localhost/"), h =>
{
h.Username("guest");
h.Password("guest");
});
cfg.UseInMemoryScheduler();
});