When writing the transformer method of a Spring XD processor, do I have any guarantee that the code will be executed in a single thread?
If I understood correctly the architecture section of the Spring XD Guide (version 1.2.1), DIRT will deploy at most one copy of a module for any stream at any given time, am I correct? And if that's the case, does that mean I can write the code without worrying to make it reentrant?
Thank you in advance.
No guarantee - it depends on the message bus deployment properties - e.g. with a RabbitMessageBus you can set the concurrency to > 1.
Yes, there's one instance of the module, but you can have multiple threads in that instance.
Related
I have used a Spring Boot Scheduler with #Scheduled annotation along with fixedRateString of 1 sec. This scheduler intermittently stops working for approx 2 min and then starts working automatically. What can be the possible reasons for this behavior and do we have any resolution to this?
Below is the code snippet for the scheduler.
1st) Please read SO guidelines
DO NOT post images of code, data, error messages, etc. - copy or type
the text into the question. Please reserve the use of images for
diagrams or demonstrating rendering bugs, things that are impossible
to describe accurately via text.
2nd) To your problem
You use a xml spring based configuration where you have configured your sheduler. Then you also use the annotation #Scheduled. You should not mix those 2 different types of configuring beans.
Also you use some type of thread synchronization into this method. Probably some thread is stuck outside of the method because of the lock and this messes the functionality that you want.
Clean either the xml configuration or the annotation for scheduling and try with debug to see why the method behaves as it does which most probable would be from what I have mentioned above about the locks and the multiple configurations.
I am in learning phase of Kafka.I came across this video This Video
This confuses me alot. I am able to understand kafka consumer and producer and i can see lot of reference materials related to that. We have batch listeners already there so why we need spring batch support here .Is there any specific advantage of using spring kafka batch over using normal batch listeners? Please help me in understanding as i can't see any reference materials comparing both.What i felt that we have more freedom and customisations using normal consumer and producer.Please correct me if i am wrong.
Spring Batch is a batch processing framework (fixed data sets) while Kafka is a streaming platform (infinite data streams). Those tools address two different types of requirements and use cases.
However, there are many cases where you want to have a "bridge" between these two worlds. Here are a couple examples:
Replay a stream of events to create an application state up to a certain time: Here you can use a Spring Batch job that reads a Kafka topic from the beginning and replays all events (The KafkaItemReader can be helpful here)
Inject a set of events from a file or a database table in a live stream. The KafkaItemWriter can be used in this case.
etc
The advantage of using a Spring Batch job over a regular batch listener is all what Spring Batch offers in terms of transaction management, state management for restartability, fault-tolerance features, etc.
I have a use case where I would like build a common interface or service which can update entities of application. Example case is shown as below:
Now every application has to handle update functionality of entities. Rather than implementing update functionality in n application module. I would like to build a common interface or server in spring boot.
Service will be like below:
My question is how to design service/interface which can used for above scenario. Any api or tool which can help me to achieve this. I dont want to write code for update in every application module.
Thanks in advance.
Last year I was thinking about the similar concept to yours, but in Apache Camel framework context. I haven't got enough time and motivation to do so, but your post encouraged me to give it a try - perhaps mostly because I've found your concept very similar to mine.
This is how I see it:
So basically I considered an environment with application that might uses N modules/plugins that enriches application's features, i.e. processing feature etc. Application uses module/plugin when it is available in the classpath - considering Java background. When the module is not available application works without its functionality like it was never there. Moreover I wanted to implement it purely using framework capabilities - in this case Spring - without ugly hacks/ifs in the source code.
Three solutions come to my mind:
- using request/response interceptors and modifying(#ControllerAdvice)
- using Spring AOP to intercept method invocations in *Service proxy classes
- using Apache Camel framework to create a routes for processing entities
Here's the brief overview of POC that I implemented:
I've chosen Spring AOP because I've never been using it before on my own.
simple EmployeeService that simulates saving employee - EmployeeEntity
3 processors that simulates Processing Modules that could be located outside the application. These three modules change properties of EmployeeEntity in some way.
one Aspect that intercepts "save" method in EmployeeService and handles invocation of available processors
In the next steps I'd like to externalize these Processors so these are some kind of pluggable jar files.
I'm wondering if this is something that you wanted to achieve?
link to Spring AOP introduction here: https://docs.spring.io/spring/docs/5.0.5.RELEASE/spring-framework-reference/core.html#aop
link to repository of mentioned POC: https://github.com/bkpawlowski/spring-aop
So I'm developing an application on top of Dropwizard and in one component I have to call an external rest service. I want to do it using JerseyClient.
Now there are 2 implementations available, the one from dropwizard and one from jersey. Using the Builder I have to choose between
io.dropwizard.client.JerseyClientBuilder.JerseyClientBuilder
and
org.glassfish.jersey.client.JerseyClientBuilder
Now the former requires Environment as well as JerseyClientConfiguration being passed to it.
Is there a good reason for using the dropwizard implementation over the vanilla one? What's the difference (except the timeout)?
Thank you
After some more digging and asking around, got an answer (below). It seems that it is better to use the JerseyClientBuilder that comes with Dropwizard as it is better integrated:
One can use JerseyClientConfiguration in order to configure the JerseyClientBuilder via the application configuration (service.yml file).
Passing the Environment information enables use of the managed thread pool that is integrated within Dropwizard's lifecycle, so when Dropwizard gets shut down, so does the client.
Dropwizard's Metrics get integrated into each client so you can see the latency and rate of calls for each one, as well as metrics around the thread pool sizes.
I am new to Spring framework and I am doing one simple project using spring and got stuck in between.
In my project I am reading the file from directory using spring poller. And then processing that file through various channels and sending it to the queue. But problem is that "file-inbound-channel-adapter" (which I'm using ) is reading only one file at a time.
So I need a solution which will read and process multiple files at a time.
Is there any way to implement multithreading in spring integration.
Thank you.
Add a task-executor to the poller; see the documentation.
You can control the concurrency with max-messages-per-poll and the task executor's pool size. See the complete poller configuration details for more information.