I'm having trouble regarding okhttp not stopping/not closing on my define request timeout seconds. I set my connection timeout to 30 seconds on each timoeut (conenectTimeout, writeTimeout and readTimeout) but not closing. I do have a logs on my user that my request took more than an hour. Hoping someone can help me on this matter
builder.connectTimeout(DEFAULT_TIMEOUT, TimeUnit.SECONDS)
.writeTimeout(DEFAULT_TIMEOUT, TimeUnit.SECONDS)
.readTimeout(DEFAULT_TIMEOUT, TimeUnit.SECONDS)
Related
when we create a curator client, we have
RetryPolicy retryPolicy = new RetryNTimes(3, 1000);
CuratorFramework client = CuratorFrameworkFactory.newClient(zkConnectString,
15000, // sessionTimeoutMs
15000, // connectionTimeoutMs
retryPolicy);
Can someone tell what is difference between session timeout and connection timeout in the above api call?
Thank you
You cannot make API calls to ZooKeeper until the connection has been established (i.e. until you get SyncConnected). Curator internally waits until this connection has been established for you. connectionTimeoutMs is the max time to wait for this.
So I read this article https://www.baeldung.com/httpclient-timeout and it says that connection timeout adds to its own penalty if the underlying service's DNS that httpclient tries to connect to has multiple IPs configured to it.
So if I have a connection timeout set to 100ms and the called service DNS has 5 IPs mapped to it then, I am looking at a max connection timeout of 500ms assuming what works is the last IP.
Is there a way to have a cap on this connection timeout regardless what the underlying service topology is as being a client, I will always be agnostic to it.
As far as I understood, you don't have a code-wise case to run in 5 or more IPs situation rather curiosity. So here my experience :
Since you're using RestTemplate which by default uses SimpleClientHttpRequestFactory.
And as the definition of connection time out goes :
The connection timeout is the timeout in making the initial
connection; i.e. completing the TCP connection handshake and getting
connected to the requested Server.
So, as far as theory goes :
Regardless of the underlying service topology, RestTemplate will try to make connection as per the connection timeout value.
And in order to figure out the almost exact timeout in your case, you must run some latency test, print the time differences which restTemplate is taking to get 200 OK.
Also, SimpleClientHttpRequestFactory internally uses HttpURLConnection which has default timeout of infinite (0/-1).
Yes, it has also been observed in rare cases, the connection keeps trying unless Thread.interrupt() explicitly being called to end.
Thus it becomes vital to describe your read-time-out and connection-time-out values and in this way you cap your connection to the limits you defined.
Hope this helps.
I have increased the concurrent tasks to be '10' for PutSQL processor.
At that time it shows below error but there is no data loss.
failed to process due to org.apache.nifi.processor.exception.ProcessException: org.apache.commons.dbcp.SQLNestedException: Cannot get a connection, pool error Timeout waiting for idle object; rolling back session:
if i have remove concurrent tasks then it worked without those exception
while google this exception i have found answer in below link
I am getting Cannot get a connection, pool error Timeout waiting for idle object, When I try to create more than 250 threads in my web application
But i don't know how to avoid this issue in NiFi putSQL.
Can anyone help me to resolve this?
This exception occurs when the pool manager cannot produce a viable connection to a waiting requester and the maxWait has passed therefore triggering a timeout.
There are several causes, but they usually fall into 2 main categories:
The DB is down or unreachable.
The connection pool (which is set to 100 max active) is out of connections.
DBCPConnectionPool controller service in nifi has 8 max connection by default and 500 milli seconds max wait time. When PutSQL processor occupied 8 connection from DBCP pool and when it request for 9th connection or exceed the max connection limit then it will throw "Cannot get a connection" exception.
You can try 2 things to avoid this Exception :
You can increase the "Max Wait Time" in DBCPConnectionPool controller
service configuration.
You can increase the "Max Total Connections" limit in
DBCPConnectionPool controller service configuration.
Kindly find the below screenshot where you need to do changes.
It might resolve your issue.
This exception can occurs if some connections are never closed so they do not become available in the pool again.
So more and more connections remain open until reaching the max.
Make sure all threads are closing the connections used.
I am getting 'Non HTTP response message: Connection timed out: connect' for some HTTP requests so I tried to set the connection/response timeout value to 2 minutes (which is more than the connect time required for failing HTTP requests). To do this, I updated "HTTP Request Defaults" and added 120000 as Connect and Response Timeouts.
HTTP Request Defaults timeouts
[
However, when I run the test again, the HTTP requests still gave the same error. The sample result is as follows -
Load time: 21007
Connect Time: 21007
Latency: 0
Size in bytes: 2212
Sent bytes:0
Headers size in bytes: 0
Body size in bytes: 2212
Sample Count: 1
Error Count: 1
Data type ("text"|"bin"|""): text
Response code: Non HTTP response code: java.net.ConnectException
Response message: Non HTTP response message: Connection timed out: connect
It looks like the timeout value I set in HTTP request Defaults is not getting used here. I also tried to set the value of httpclient.timeout=120000 in jmeter.properties but no change. Have I missed something?
Can somebody please help me with this?
Thanks.
Edit - I have multiple HTTP requests and each run, different requests time-out. Here is one of the HTTP requests -
Updates:
I tried changing the Timeouts values in HTTP Request Defaults to very low (2000) to see how HTTP requests work. In this case, I was getting different error for requests exceeding connection time of 2000ms -
Non HTTP response code: org.apache.http.NoHttpResponseException/Non HTTP response message: : failed to respond
So I think changing the timeout values is not affecting my original error -
Non HTTP response code: java.net.ConnectException/Non HTTP response message: Connection timed out: connect
What is the difference between these two message?
The issue seems more of a server configuration of connection timeout than client side configuration of connection timeout, though both must be configured appropriately.
Default connectionTimeout in tomcat server is 20 seconds. and you request is failed due to connection timeout at 21 seconds. so, though you configured at client side (120000) you must configure appropriately at server side as well, otherwise, server forces to close the connection attempt and raises Connect Timeout exception.
Reference:
The HTTP Connector (refer connectionTimeout attribute)
Recently I have faced the same problem and found that it is the default configuration in my OS (Windows). Check the following links for details:
Where does the socket timeout of 21000 ms come from?
Which is the default TCP connect timeout in Windows?
Shortly, based on articles mentioned in the links above, Windows uses 3000ms initial timeout (InitialRto setting) and does 2 retries with doubled timeout from the previous attempt (MaxSynRetransmissions setting): 3sec + 2*3sec + 4*3sec = 21 sec.
In order to increase this timeout you can set more retries with the following command:
netsh interface tcp set global MaxSynRetransmissions=3
A few weeks ago I wrote a small program which created a socket to an apache webserver and made a request.
Back then I did not know that this web server had a KeepAliveTimeout of 5 seconds.
After my first request I waited 1 minute. After this I wanted to reuse my first socket for another webserver request, but got an error.
From Beej's Guide to Network Programming I learned that if recv returns 0, then the other side has closed its connection:
Wait! recv() can return 0. This can mean only one thing: the remote side has closed
the connection on you! A return value of 0 is recv()'s way of letting you know this
has occurred.
My questions are now:
What does Apache send when the KeepAliveTimeout is over - a FIN or a RST packet?
I know that using a TCP connection for 2 unrelated HTTP requests like in this scenario might
not be the best thing. But in order to understand TCP more the next question is:
After my first successful http request, and before sending the next HTTP request over the same socket, would there be somehow a possibility to get informed about this keepalivetimeout TCPsocket termination of the server other than receiving 0 from the next recv() call?
It will send a FIN. If you write a request to the server after that, send() will return -1 with errno/WSAGetLastError() = ECONNRESET.
would there be somehow a possibility to get informed about this keepalivetimeout tcp socket termination of the server
Yes, by reading the proper response header parameter, namely Keep-Alive: timeout=delta-seconds:
'timeout' Parameter
A host sets the value of the timeout parameter to the time that the host will allows an idle connection to remain open before it is closed. A connection is idle if no data is sent or received by a host.
The value of the timeout parameter is a single integer in seconds.
A host MAY keep an idle connection open for longer than the time that it indicates, but it SHOULD attempt to retain a connection for at least as long as indicated.
As you can see, it's up to the host to decide. Given it only SHOULD try to keep the connection open as long as promised, but it isn't required that it does in order to conform to the spec, so the server might decide to close and reuse the connection to serve another pending client.