I have a REST service written in Spring Boot which have only GET and POST methods. The GET/POST method retrieve/post data from/to cassandra cluster.
I tried testing the REST service with 100 users running concurrently.
My jmeter configurations are:
When I run this test, for the first few request it gives errors and half request is successful.
Here is the result table
Then I checked the reason of error in the Result tree and found this response data:
{"timestamp":1474278650822,"status":500,"error":"Internal Server Error","exception":"com.datastax.driver.core.exceptions.NoHostAvailableException","message":"org.springframework.web.util.NestedServletException: Request processing failed; nested exception is com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (no host was tried)","path":"/post"}
I couldn't figure out why it failed to connect to cassandra cluster for the first few requests.
I am new to cassandra, REST service and jmeter. Can anyone point out what mistake I am making.
NOTE: I have tested my REST Service using Postman, and it functions well, both GET and POST request.
With this configuration you're hitting the service at once with 100 requests.
Each request will get a request thread, therefore I assume your service has a thread-pool with at least 100 connection threads.
Each request thread tries to obtain a connection to the DB from the DB Connection pool. If 50% of your requests fail, I'd guess, your DB Connection Pool size set to 50. In case your DB connection pool can handle 100 connections, check the connection settings of your Cassandra DB, maybe that one is limited to 50.
There is few more to add with the answer of #Gerald. The number of threadpool limitation in the DB might not be the only culprit, the other possibilities are the max thread connections set in the server level as well. If you have the access to the server log of the api your trying to test, you can get exactly at what point it breaks and fix it with help of developers
Related
I got this error when I'm using streaming subscription with impersonation.
After the connection opened and receive notification successfully for minutes, it just pops up a bunch of this for almost all subscriptions.
How can I avoid this error?
One or more subscriptions in the request reside on another Client Access server. GetStreamingEvents won't proxy in the event of a batch request., The Availability Web Service instance doesn't have sufficient permissions to perform the request
I need to keep the connection stable and avoid this error.
Sounds like you haven't use affinity https://learn.microsoft.com/en-us/exchange/client-developer/exchange-web-services/how-to-maintain-affinity-between-group-of-subscriptions-and-mailbox-server
Also if its a multi threaded application ExchangeService isn't thread safe and shouldn't be used across multiple threads.
Just starting to use Spring Webflux Webclient,Just wanted to know what is the default KeepAlive time for the Http Connection ? Is there a way to increase the keepAlive Time? In our Rest Service we get a request probably every five minutes,The request takes long time to process .It takes time between 500 seconds-- 10 second. However in load test if I send frequent requests the processing time is less than 250ms.
Spring WebFlux WebClient is an HTTP client API that wraps actual HTTP libraries - so configuration like connection management, timeouts, etc. are configured at the library level directly and behavior might change depending on the chosen library.
The default library with WebClient is Reactor Netty.
Many HTTP clients (and this is the case with Reactor Netty) are maintaining HTTP connections in a connection pool to reuse them. Clients usually acquire a new connection to a remote host, use it to send/receive information and then put it back in the connection pool. This is very useful since sometimes acquiring a new connection can be costly. This seems to be really costly in your case.
HTTP clients leave those unused connections in the pool, but what about keepAlive time?
Most clients leave those connections in the pool as long as possible and test them before acquiring them to see if they're still valid or listen to server events asynchronously to remove them from the pool (I believe Reactor Netty does that). So ultimately, the server is in control and decides when to close connections if they're inactive.
Now your problem description might suggest that connecting to that remote host is very costly, but it could be also the remote host taking a long time to respond to your requests (for example, it might be operating on an empty cache and needs to calculate a lot of things).
I have an unknown App consuming my Spring webservices.
The app set a timeout to every webservice calls.
The server regardless of the app timeout keeps processing.
Is there a risk of any other webservice call in receiving a misresponse (the response to the timed out webservice call)? How does Spring manages this? Doesn't HTTP protocol take care of this, given that each connection channel is open for a particular call to webservice and if broken there shouldn't be possible to retrieve the response?
As a developer, you should try to make all possible HTTP requests to your web server to be idempotent. It means that the client side has to be able to retry the failed request without new possible errors due to the inability to know the previous (timeout) request results.
The client side should handle the HTTP client timeouts himself and (by default) should treat the timeout error as a failure. Your clientside may repeat the request later and the server side should be able to handle the same request.
The solutions may vary for different tasks depending on complexity (from an INSERT statement to the database or scheduling a new CRON job avoiding duplication).
I am writing a proxygen server that needs to make http request to other service.
I would like to have a connection pool to this other service (using keep-alive), and whenever I need to make a request to this service, I would like to peak one of those connections and issue a request. Then, after the request is finished, return this connection to the pool so it can be reused.
I read the curl client in the examples of the proxygen project, but this only makes one request and close the connection.
Can anyone give me some insights on how to make an http client that handles a connection pool using proxygen/folly ?
Does proxygen/folly have a way of handling connection pools ? In that case where can I read about it ?
thanks in advance
I am writing a REST service where the result of a REST POST can take longer than the environments timeout settings for HTTP connections. Given that I can't change the timeout for my REST target url,
What can I do to to make a REST call pass properly? I thought about using an async controller, but that seems not to fix any timeout behavior.
The calling client should not have to handle any server error or try to re-execute the query, as it is just adding more stress to the server.
Cheers,
Kai
Assuming this is a connection read timeout and not a http keepalive timeout since there is only one query. One suggestion would be for the rest service to return intermittent status response every specified interval. If this is a tcp keepalive issue then it can be circumvented using configuration. If a socket read timeout is being set then thst can be increased as well.