Logback MDC (Mapped Diagnostic Context) is leveraging threadLocal (As far as I know) so that it will be accessible on all the log statements executed by the same thread.
My question is, will logback MDC work in the non blocking IO server-side runtime like Netty or Undertow as it used to work in for example, tomcat?
If yes, how is it working as Netty/Undertow is not following one-thread-per-request unlike tomcat.
I am trying to put a traceID in MDC, so that I can trace all my log from a one transaction trail across multiple microservices/pipeline-listeners in a centralized logging system like Splunk/ELK
I'm not terribly familiar with using netty directly, but I do know that it is possible to use logback MDC with asynchronous code. It isn't easy though.
Basically you need to somehow tie the traceId to the request/connection, and each time you start processing for that connection, set the traceId in the MDC for the current thread.
If you are using a thread pool, you can do this with a custom Executor that gets the current state (ex. traceId) from the current thread local storage, and creates a wrapped Runnable or Callable that sets thread local storage to the previously retrieved value before running. That wrapped runnable is then forwarded to a delegate Executor.
You can see how this is done here: https://github.com/lucidsoftware/java-thread-context/blob/master/thread-context/src/main/java/com/github/threadcontext/PropagatingExecutorService.java
In fact, depending on your needs, you may be able to use that java-thread-context project.
As you have mentioned, the MDC implementation is based on the ThreadLocal, so it's not really applicable to the Reactor.
What I would suggest in case of Reactor (I assume you are using it by the tags) is:
Using the Context (docs here) available in Mono and Flux in order to store the traceId to be available during the execution.
If you're using Spring-Webflux and you want to generate the tradeId and add it to the Context in one place, then you can use the WebFilter (docs here) to do that.
Then you can retrieve the data from the context using some custom
helper methods, like in example by Nicolas Portman, or create your own custom logging format, where you utilize the data from the Context.
Related
I use #Slf4j to add logs in all layers in my SpringBoot project.
I want to use some unque ID for all logs inside onq request.
For example generate UUID at the RestController and automatic add it to every logs in this thread
For example
[..... Some UUID ....] "The real logger message"
How to do this?
It need to have possibility to filter all logs of specific request.
If your application is threaded (or any better word to describe "opposed to a reactive application where it might be possible that everything happens in the main thread"), use the mapped diagnostic context (MDC). The MDC is a thread-bound Key-Value Map, where you can put and retrieve data. Baeldung has a tutorial on logging with the mdc using multiple logging frameworks. Also, there are plenty of examples across the web.
If your application should be reactive, you may wanna check out this project reactor reference.
I have big task in transaction (#Transactional method). In this task the id-s are collected and then after transaction commit they must be sent to RabbitMQ (with convertAndSend-method). The RabbitMQ-listener in the other side takes the ids and updates the statuses in DB (it's important to update statuses after transaction changes because only after them the updated data is actual)
I have next questions:
What is the best way to use like the hook in the end (commit) of transaction? I need make the one "afterCommit" method for several service classes (many transactional-methods);
What need I use as the storage of ids? I have thought about smth like ThreadLocal variable but this is not a variant - if there is the parallelStream, the new Thread is created;
May be some other solution?
I have read about the delay RabbitMQ plugin but it is not the variant 4 my task - their time is very different.
Looking at the tags in your question I suppose you are using the spring framework.
So you could use Spring Events and its ApplicationEventPublisher to publish your specific ApplicationEvent together with all the necessary data (the ids in your case).
Spring allows you to bind an event listener to a phase of the current transaction. Just use the #TransactionalEventListener annotation on a method that sends the data to RabbitMQ finally. Binding is possible to different transaction phases. With the default binding (AFTER_COMMIT) the event will only be fired if the transaction has completed successfully.
This Baeldung article on Spring Events is a nice place to find more detailed information.
I am very new to reactive programming, and currently working on microservice where in spring mvc is used and Spring Data MongoDb for database connectivity.
As I am going through spring data mongo db docs, there is support for reactive repositories, reative template api etc.
So Is there going to be any downside if I choose to use reactive templates and repository with blocking nature ?
Ex.
reactiveMongoTemplate.add(entity).block()
reactiveMongoTemplate.update(id, entity).block()
Also is there any significant difference with using calls like above than using blocking repository and template api itself ?
The answer depends on which stack you use: Spring WebFlux or Spring Web MVC.
In case of Spring WebFlux the choice is obvious: you have to use ReactiveMongoTemplate and never call block on it, instead return Mono/Flux as you get it from the template.
In case of Spring Web MVC you are free to use both the regular blocking MongoTemplate and ReactiveMongoTemplate with block. Although, in most cases you should go with the good old MongoTemplate for sake of simplicity and performance. ReactiveMongoTemplate has some overhead compared to the blocking MongoTemplate because the reactive types Mono/Flux put some additional pressure on memory.
I can imagine one use case where ReactiveMongoTemplate can provide some advantage even in Spring MVC: when during one HTTP request you have to execute multiple Mono operations concurrently. If you used blocking MongoTemplate then you would need to set up a thread pool and delegate the query execution there. However, with ReactiveMongoTemplate you could use the many operators of Mono and Flux to accomplish this task without worrying about threads, thread pools and scaling issues.
In the traditional programming you usually own the thread that you're running on, in the reactive programming its not the case. This "underlying unit of execution" (cpu resources consumer if you wish) is not yours, but rather a "global" thing that currently happens to execute your task, but can switch to do other things really soon.
So when you block, you say to this "global unit of execution" like "hey, stop doing anything else, wait for me". In the traditional approach, its kind of ok, because you have a thread associated with the current request, other requests (or flows if your system is not web based) are supposed to be executed with other threads taken from a fairly large thread pool. In the reactive system however, its not the case since you're trying to utilize a small amount of these "global units of execution".
Ok, so if you block, the events all over the place will stop emitting and will get start to buffer. And this may lead to the whole system becoming unusable.
Spring boot 2.1.5
Project Reactor 3.2.9
I am setting up a bunch of rest reactive APIs using the above-mentioned frameworks and I am running into an annoying problem with MDC (mapped diagnostic context). My applications are in JAVA.
MDC relies on thread locals to store the current query's mapped context to put in the logs. That system, obviously, is not perfect and contradicts the reactive pattern since the different steps of your execution will be executed through different threads.
I have run into the same problem with the Play Reactive framework but found a workaround there by copying the mapped context transparently from one actor to another.
For spring and reactor, I could not find a satisfying solution yet.
Some random examples found on the internet:
First - It works but forces you to use a bunch of utility methods
Same thing
Second - It tries to copy the context during the onNext publisher event but seems to lose some features on the way of doing that. The signal context, for example, is lost.
I am in need of a proper solution to deal with this:
A library which would make the link between MDC and reactor?
A way to tweak reactor/spring to achieve it transparently?
Any advice?
"I could not find a satisfying solution yet."
Working with contexts is the only solution for the moment. Since as you said threadlocals goes against everything that has to do with reactive programming. Using thread local as a storage point during a request is a resource heavy way of solving things and in my opinion poor design. Unless logging frameworks themselves come up with a better solution to the problem we developers must pass the data through the context to accommodate for the logging frameworks blocking nature.
Reactive programming is a paradigm shift in the programming world. Other things like database drivers, that use threadlocal to rollback transactions are also in big trouble. the JDBC database driver spec is defined as blocking in nature, and atm. there has been attempts by spring and the R2DBC project to define a new JDBC driver spec that is inherently non/blocking. This means that all vendors must rewrite ther database driver implementations from scratch.
Reactive program is so new that lots of libraries need to rewrite entire codebases. The logging frameworks as we know it needs to be rewritten from the ground up which is a huge task. And the context in reactive is actually something that should not even be in reactive programming, it was implemented just to accommodate for MDC problems.
It's actually a lot of overhead needing to pass data from thread to thread.
So what can we do?
push on logging frameworks, and/or help logging frameworks to rewrite their codebase
Accept that there is no "tweak" that will magically fix this
use the context and the way suggested in the blogposts
Project reactor context
I have a spring based web-application that needs to get data from ActiveMQ via a JMX connection.
I am using MBeanServerConnectionFactoryBean (in Spring) to get various MBean attributes from ActiveMQ.
I have only one MBeanServerConnectionFactoryBean as a member variable, which is used to get the data. If multiple requests/threads come concurrently will there be any issues? Will there be any race conditions?
Please suggest the best way to keep the code thread-safe.
Spring FactoryBean objects are not intended to be used directly from your code, they're supposed to be used in your Spring config. As such, they are designed to be executed once and once only.
If you want to use them, including MBeanServerConnectionFactoryBean, then you need to create them, configure them, use them and discard them each and every time you want to get the object they create. They are most definitely not thread-safe.
Better yet, do it as the design intended and use them in your Spring config.