I am using ElasticSearch with Nest (0.9.16) and have some queries which take longer than 60 seconds. When that happens, I receive the following error:
The request was aborted: The request was canceled.
While I am concerned that the query is taking such a long time, for this question, I am just interested if there is a way to increase the timeout. I have not been able to find a way to increase the timeout.
Note, I do see a couple of places where I can set a timeout, but these are not the right places:
var setting = new ConnectionSettings(new Uri(searchUrl)).SetTimeout(200000);
or
searchdescriptor.Timeout("120000");
The first one sets the connection timeout and the second one sets the ElasticSearch timeout, both of which are not helpful here.
Thanks,
Eric
Actually, much to my chagrin, the ConnectionSettings().SetTimeout() does in fact increase the request timeout and solve my problem. I am not sure why I missed this initially.
Related
we have started using the high level REST client finally, to ease the development of queries from backend engineering perspective. For indexing, we are using the client.update(request, RequestOptions.DEFAULT) so that new documents will be created and existing ones modified.
The issue that we are seeing is, the indexing is delayed, almost by 5 minutes. I see that they use async http calls internally. But that should not take so long, I looked for some timing options inside the library, didn't find anything. Am I missing anything or the official documentation is missing for this?
Since refresh_interval: 1 in your index settings, it means it is never refreshed unless you do it manually, which is why you don't see the data just after it's been updated.
You have three options here:
A. You can call the _update endpoint with the refresh=true (or refresh=wait_for) parameter to make sure that the index is refreshed just after your update.
B. You can simply set refresh_interval: 1s (or any other duration that makes sense for you) in your index settings, to make sure the index is automatically refreshed on a regular basis.
C. You can explicitly call index/_refresh on your index to refresh it whenever you think is appropriate.
Option B is the one that usually makes sense in most use cases.
Several reference on using the refresh wait_for but I had a hard time finding what exactly needed to be done in the rest high level client.
For all of you that are searching this answer:
IndexRequest request = new IndexRequest(index, DOC_TYPE, id);
request.setRefreshPolicy(WriteRequest.RefreshPolicy.WAIT_UNTIL);
I'm collecting some statistics in Elasticsearch for the number of requests per hour to an API. Rather than indexing a new document per request, I'm updating a statistic document per user. On each request, I'm doing something like this from .NET:
client
.Update<StatisticsDocument>(statsId, u => u
.RetryOnConflict(3)
.Script(s => s.Source("ctx._source.requests+=1"))
.Upsert(new StatisticsDocument { Id = statsId, Requests = 1, User = userId, Created = thisHour }));
So, requests is incremented. If a document for this error and the user with userId doesn't exist, I'm creating a new one using the Upsert-method.
There can be more threads and processes calling this code, why I've added RetryOnConflict(3) to retry on conflict.
I'm seeing a pretty large number of conflicts anyway, especially under high load. I understand that retrying 3 times doesn't always solve this issue since there are many threads trying to update the same document over and over again. But I'm still surprised by a pretty high number of conflicts.
Any ideas about what to do here? I'm thinking about keeping the metric in memory and then writing it every minute or something like that. But if Elasticsearch and/or Nest has some kind of built-in feature to help even further, that would be the first priority to implement.
Intermittently(once or twice in a month) I am seeing the error
org.apache.kafka.common.errors.TimeoutException: Expiring 1 record(s) for cart-topic-0: 5109 ms has passed since batch creation plus linger time
in my logs due to which the corresponding message was not processed by Kafka Producer.
Though all the brokers are up and available I'm not sure why this error is being observed. Even the load is not much during this period.
I have set the retries property value to 10 in Producer configs but still, the message was not been retried. Is there anything else I need to add for the Kafka send method? I have gone through the similar issues raised, but there is no proper conclusion for this error.
Can someone please help on how to fix this.
From the KIP proposal which is now addressed
We propose adding a new timeout delivery.timeout.ms. The window of enforcement includes batching in the accumulator, retries, and the inflight segments of the batch. With this config, the user has a guaranteed upper bound on when a record will either get sent, fail or expire from the point when send returns. In other words we no longer overload request.timeout.ms to act as a weak proxy for accumulator timeout and instead introduce an explicit timeout that users can rely on without exposing any internals of the producer such as the accumulator.
So basically, post this now you can additionally be able to configure a delivery timeout and retries for every async send you execute.
I had an issue where retries were not being obeyed, but in my particular case it was because we were calling the get() method on send for synchronous behaviour. We hadn't realized it would impact retries.
In investigating the issue through various paths I came across the definition of the sorts of errors that are retrial
https://kafka.apache.org/11/javadoc/org/apache/kafka/common/errors/RetriableException.html
What had confused me is that timeout was listed as a retrial one.
I would normally have suggested you would want to look into if the delivery of your batches was taking too long and messages in your buffer were expiring due to increased volume, but you've mentioned that the volume isn't particularly high.
Did you determine if increasing the request.timeout.ms has an impact on the frequency of occurrence? It might be more of a treating the symptom step than the cause.
Does Net::HTTP.get_response timeout after some number of seconds?
If it does not, is there a way for me to add a timeout?
There doesn't seem to be any method nor attributes for either Net:HTTP.get_response and/or Net:HTTP.get_print as per latest documentation. There is a nice blog post that documents the basics for setting the timeouts though.
I've got a need to retain the data and keep a Ruby program waiting for a response for anything up to a couple of days. I'm thinking about implementing this using threads (there may be a number of concurrent requests across a network). My question; is it reasonable to leave a thread running for anything up to a couple of days awaiting a response?
In general there is no problem with that. Check out the Queue class, it might facilitate the "job polling":
http://www.ruby-doc.org/stdlib-1.8.7/libdoc/thread/rdoc/Queue.html