Say for example, can we limit the number of requests to be processed by an instance should be 50 and the rest has to be processed by another instance.
Do we require any configuration changes related to ribbon.
Related
I have a single queue with multiple listeners with selector so that each queue processes only specific set of messages, do I need to create each listener as a separate microservice. My understanding is that since MQ infrastructure spawn multiple instances of the consumer based on the load we need not to have them as a separate microservice, I am new to this area, please help me with right design for my use case. Thanks.
I am having multiple spring boot applications which are connected to kafka (clustrized with 3 brokers)and also i integrated transaction synchronization (chainedKafkaTransactionManager). so i want to know should i give the same TransactionIdPrefix value in kafka config for all the multiple application or diffrent one.
i tried giving a random generated TransactionIdPrefix for each application. but i think in some time in multi thread environment in Listeners method it will take old data from database (jpa repositories)
is it problem because of diffrent TransactionIdPrefix ?
It depends; if they are multiple instances of the same app and the transactions are started by consumers, the prefix must be the same, so that zombie fencing is handled properly when partitions move from one instance to another after a rebalance.
If the transactions are started by producers, the prefix must be unique in each instance.
If they are different applications they should have different prefixes, regardless of what starts the transaction.
This is probably a rather peculiar question. I am using Spring Boot 2.0.2 with the default Tomcat container. In order to set up a test in our QA environment that simulates many servers, I would like to set up a Spring Boot-based REST service that listens on a very large number of ports simultaneously. I'm able to do this using the technique previously described in in another SO post (Configure Spring Boot with two ports) which basically adds connectors using TomcatServletWebServerFactory.addAdditionalTomcatConnectors().
The difficulty is that a large number of threads seem to be created for each additional port activated; some empirical measurements show the total to be 17 + (15 * number of ports). This means listening on 250 ports result in 3767 threads created and 500 ports result in 7517 threads created and I would like to go somewhat beyond that number. The test program used to take the above measurements is the bare minimum to bring up a Spring service and there is no code that creates threads explicitly so insofar, as I know, all of those threads were created by Spring/Tomcat.
Is there a way to accomplish this using Spring that doesn't use so many threads per active port? Would an alternate container like Jetty be more efficient?
You can configure the embedded tomcat container properties in spring boot configuration file and set the appropriate numbers for these properties to limit the threads created by Tomcat container -
server.tomcat.max-threads=200 # Maximum amount of worker threads.
server.tomcat.min-spare-threads=10 # Minimum amount of worker threads.
I have messages coming in from Kafka. So I am planning to write a listener and "onMessage". I want to process it and push it in to solr.
So my question is more architectural, like I have worked on web apps all my career, so in big data how to deploy the spring kafka listener, so I can process thousands of messages a second.
How do I make my spring code use multiple nodes to distribute the
load?
I am planning to write a SpringBoot application to run in
a tomcat container.
If you use the same group id for all instances, different partitions will be assigned to different consumers (instances of your application).
So, be sure that you specified enough partitions in the topic you are going to consume.
I have read this post but it's not my case and not enough clear:
How does load balancing in Spring XD get done?
I have a composed job with different instances of the same sub-jobs deployed in different containers. My composed job is scheduled to run periodically. I need to know how Spring XD choose the sub-jobs instances to invoke for every new request to the composed job.
The same question for a stream triggered every X minutes.
It's handled by the transport (rabbit, redis).
Each downstream module competes for messages - with rabbit it will generally be round robin; with redis it will be more random.