I have a rest controller which accepts post requests and returns the statuses of whether the operations are successfull or not. It works fine for 100 requests per second as I have multiple operations underlying it which at the end send the response.
There could be hundreds of users trying to send the requests to the controller, thereby its all done using a completable future and http Async invoker. The problem happens when there are a 1000 requests per second and then the controller threads are exhausted as there are already multiple thread processing multiple requests and all are waiting for there future to be complete and then sending the response.
How can I make my rest controller be able to handle 1000 requests per Second without breaking.
there are already multiple thread processing multiple requests and all are waiting for there future to be complete and then sending the response.
You can actually make you controllers asynchronous by making them return a CompletableFuture. Just chain the calls on the CompletableFuture returned by your service to convert them into a an appropriate response instead of using get() or join():
#RequestMapping
public CompletableFuture<ResponseEntity<…>> processRequest() {
return myService.getStatusFuture()
.thenApply(status -> convertToResponseEntity(status));
}
Of course for this to work properly you should have a truly asynchronous service. If you are using #Async or submitting tasks with CompletableFuture.supplyAsync(), this will just move the problem from the HTTP threadpool to another threadpool.
It depends on the servlet server you are using. In the application.properties file, you can use the server.* properties to set the parameters you need.
In this link you can find these properties under the EMBEDDED SERVER CONFIGURATION section. If you are using the default tomcat embedded server, check out the server.tomcat.* properties. Especially the server.tomcat.accept-count, server.tomcat.max-connections and server.tomcat.max-threads properties.
Related
I have a general REST API (developed using Spring MVC) that takes a list of API requests as its request body.
Each API request in the list has its own URL and request body.
In the implementation of this general REST API, I need to call the corresponding Spring controller method (in the same app) for each of these individual API requests (with their appropriate URL and request body).
(I will then merge all those individual API responses and return it in one big response from the general REST API).
I've been searching around, but I'm unclear how to programmatically call Spring to execute each individual API request. I would ideally like to get back the ResponseEntity from each call instead of the actual JSON response.
(More information:
On the same app server as the general API, I need to translate the URL and JSON request body for each individual API into the arguments to the controller method. I also need to take the URL and have Spring determine which controller method to invoke itself.)
Any help would be greatly appreciated.
Thanks,
Matt
Answer depend on whether the individual URLs that you are planning to invoke is with in the same server (Accessible without using network call) or not
If it is with in the same app server, spawn multiple threads and invoke the individual methods and join the response together and send it back
If it is not within the same app server, there are many Async Restclients are there besides spring's own webclient/restTemplate etc
I am creating a script on a simple web application which includes following:
1)Login
2)Some API call(which requires token from login response)
I successfully created the script and executed for one thread which worked fine. Now I am trying increase the load upto 100 threads wherein i got some errors on few API request which got executed before Login sampler(authorization error). Then I put if controller for all API calls(if login success then request API), in this case login will always run first before any API request($JMeterThread.last_sample_ok==true).
My manager has told me not to use if controllers in the script because it will consume some time to execute which in turn will affect the final response time.
What is the alternate method to achieve this in term of performance aspect.
I have a spring controller and request mapped method.
#RequestMapping(value = { "/doSomething.do" })
public ModelAndView emailToNonBelievers(){
.....
// do something which takes lot of time
// send response..
return modelAndView;
}
This method takes very long time, about an hour.(It is for Admin, not users. And Admin doesn't need to wait an hour. Just fire and forget, that's ok. And this is not a batch job)
But I found that client send requests repeatedly with 3 minutes interval(observed 6 times and I stopeed Spring service).
I guess because client didn't get any response from server.
If my guess is right, server should response sort of "Your request is accepted, just shut up and wait!!" .
But how to send response(200 ok) before jobs finished in Spring?
Or am I missing something?
In this situation it would be recommended to use asynchronous task processing. Spring comes with out-of-box support for it via #Async annotation.Consult my answer for detailed setup for similar query and here for docs.
I currently have a problem where I send an asynchronous ajax request to a .NET controller in order to start a database search. This request makes it to the server which kicks off the search and immediately (less than a second) replies to the callback with a search ID, at which point I begin sending ajax requests every 10 seconds to check if the search has finished. This method works fine, and has been tested successfully with multiple users sending simultaneous requests.
If I send a second search request from the same user before the first search is finished, this call will not make it to the controller endpoint until after the first search has completed, which can take up to a minute. I can see the request leave chrome (or FF/IE) in the dev tools, and using Fiddler as a proxy I can see the request hit the machine that the application is running on, however it will not hit the breakpoint on the first line of the endpoint until after the first call returns.
At the point this call is blocking, there are typically up to 3 pending requests from the browser. Does IIS or the .NET architecture have some mechanism that is queuing my request? Or if not, what else would be between the request leaving the proxy and entering the controller? I'm at a bit of a loss for how to debug this.
I was able to find the issue. It turns out that despite my endpoint being defined asynchronously, ASP.NET controllers by default synchronize by session. So while my endpoints were able to be executed simultaneously across sessions, within the same session it would only allow one call at a time. I was able to fix the issue by setting the controller SessionState attribute to Read Only, allowing my calls to come through without blocking.
[SessionState(System.Web.SessionState.SessionStateBehavior.ReadOnly)]
I'm using backbone on a project of mine, integrated with communication to an external API. I want to use real-time updating of records. Since I don't have access to the main backend of this external application, and they don't provide neither websocket server nor long-polling endpoint, I am basically left with the option of doing regular polling with setInterval and a period of 50 seconds. It has been working quite well. My problem is the edge case. If for some reason the API request hangs, for more than 50 secs, let's say, I'll be triggering a new request right away. That means, 2 hanging requests now, which will add up eventually. Is there a way to set a timeout for the request? I know all requests lead to Backbone.sync, but I was checking the source code and I don't see any feasible way of setting the timeout for the XmlHttpRequest. Is there a way to do this cleanly and without overwriting behaviour? Or are there other solutions/workarounds?
Just pass a timeout:milliseconds option in the options argument to fetch. The options get passed directly to jQuery.ajax, which handles the low-level XHR call:
collection.fetch({timeout:50000});
Alternatively you can set a global timeout for all the requests made by your application by calling jQuery.ajaxSetup in your application startup:
$.ajaxSetup({timeout:50000});