Nest fails to write to ES while executed from Azure functions - elasticsearch

I do have azure function which listens to events and write data to ES, but randomly it fails.
Exception while executing function: ElasticSearchIndexer Invalid NEST response built from a unsuccessful low level call on POST: /_bulk
# Invalid Bulk items:
# Audit trail of this API call:
- [1] AllNodesDead: Took: 00:00:00
- [2] Resurrection: Node: http://ip:9200/ Took: 00:00:00
- [3] PingFailure: Node: http://ip:9200/ Exception: PipelineException Took: 00:00:02.0095670
# OriginalException: Elasticsearch.Net.ElasticsearchClientException: Failed to ping the specified node.. Call: Status code unknown from: HEAD / ---> Elasticsearch.Net.PipelineException: Failed to ping the specified node. ---> Elasticsearch.Net.PipelineException: An error occurred trying to read the response from the specified node. ---> System.Net.WebException: The request was aborted: The request was canceled.
at System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult)
at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Elasticsearch.Net.HttpWebRequestConnection.<RequestAsync>d__5`1.MoveNext()
--- End of inner exception stack trace ---
at Elasticsearch.Net.RequestPipeline.<PingAsync>d__65.MoveNext()
--- End of inner exception stack trace ---
at Elasticsearch.Net.RequestPipeline.<PingAsync>d__65.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Elasticsearch.Net.Transport`1.<PingAsync>d__21.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at Elasticsearch.Net.Transport`1.<PingAsync>d__21.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Elasticsearch.Net.Transport`1.<RequestAsync>d__15`1.MoveNext()
--- End of inner exception stack trace ---
# Audit exception in step 3 PingFailure:
Elasticsearch.Net.PipelineException: An error occurred trying to read the response from the specified node. ---> System.Net.WebException: The request was aborted: The request was canceled.
at System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult)
at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Elasticsearch.Net.HttpWebRequestConnection.<RequestAsync>d__5`1.MoveNext()
--- End of inner exception stack trace ---
at Elasticsearch.Net.RequestPipeline.<PingAsync>d__65.MoveNext()
# Request:
<Request stream not captured or already read to completion by serializer. Set DisableDirectStreaming() on ConnectionSettings to force it to be set on the response.>
# Response:
<Response stream not captured or already read to completion by serializer. Set DisableDirectStreaming() on ConnectionSettings to force it to be set on the response.>
I am registering in DI .
var uris = FlymarkAppSettings.Instance.ElasticServerConnection.Split(';').Select(url => new Uri(url));
var settings = new ConnectionSettings(new StaticConnectionPool(uris));
#if DEBUG
settings.EnableDebugMode();
#endif
builder.Register(c => new ElasticClient(settings))
.As<IElasticClient>()
.SingleInstance();

Related

How to avoid Cache restore time outs

I do a cache of the maven repository using Cache#2 to prevent additional downloads especially since Maven Central may yield zero byte files.
However, I am getting events such as this on my build log and when it occurs it significantly slows down the build step.
2020-02-14T03:24:14.9261543Z Information, Downloaded 340.2 MB out of 447.6 MB (76%).
2020-02-14T03:24:18.3130567Z Warning, [https://37bvsblobprodcus311.blob.core.windows.net/db184fb09aee8f4ae39869ebeab81ab1ad/7FE2B04FAB9DCB08C78F137CCA2FDA2D5E37B05B5DBCA1D7EDD23F7DB5D580B301?sv=2017-04-17&sr=b&sig=mZZ1kK00Gc2cg5m6dyRB3XUYVB40dxJkvBMbXW4D3cM%3D&spr=https&se=2020-02-15T03%3A29%3A57Z&sp=r&rscl=x-e2eid-c561122c-76784a8c-be2cb59f-c79091dc-session-eacd1913-b0014305-a33f0081-46cabc28] Try 1/5, non-retryable exception caught. Throwing. Details:
2020-02-14T03:24:18.3131821Z Task was requested to be canceled. System.Threading.Tasks.TaskCanceledException: A task was canceled.
2020-02-14T03:24:18.3132112Z at System.IO.Stream.CopyToAsyncInternal(Stream destination, Int32 bufferSize, CancellationToken cancellationToken)
2020-02-14T03:24:18.3132355Z at System.Net.Http.NoWriteNoSeekStreamContent.<>c.<SerializeToStreamAsync>b__4_0(Task t, Object s)
2020-02-14T03:24:18.3132630Z at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
2020-02-14T03:24:18.3133177Z --- End of stack trace from previous location where exception was thrown ---
2020-02-14T03:24:18.3133445Z at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot)
2020-02-14T03:24:18.3133903Z --- End of stack trace from previous location where exception was thrown ---
2020-02-14T03:24:18.3134179Z at System.Net.Http.HttpContent.LoadIntoBufferAsyncCore(Task serializeToStreamTask, MemoryStream tempBuffer)
2020-02-14T03:24:18.3134413Z at System.Net.Http.HttpClient.FinishSendAsyncBuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
2020-02-14T03:24:18.3134691Z at Microsoft.VisualStudio.Services.Common.TaskCancellationExtensions.EnforceCancellation[TResult](Task`1 task, CancellationToken cancellationToken, Func`1 makeMessage, String file, String member, Int32 line)
2020-02-14T03:24:18.3134970Z at Microsoft.VisualStudio.Services.BlobStore.WebApi.DedupStoreHttpClient.<>c__DisplayClass57_0.<<GetRedirectResponseAsync>b__0>d.MoveNext()
2020-02-14T03:24:18.3135439Z --- End of stack trace from previous location where exception was thrown ---
2020-02-14T03:24:18.3135708Z at Microsoft.VisualStudio.Services.Content.Common.AsyncHttpRetryHelper`1.InvokeAsync(CancellationToken cancellationToken)
2020-02-14T03:24:18.3150625Z Warning, [https://37bvsblobprodcus311.blob.core.windows.net/db184fb09aee8f4ae39869ebeab81ab1ad/7FE2B04FAB9DCB08C78F137CCA2FDA2D5E37B05B5DBCA1D7EDD23F7DB5D580B301?sv=2017-04-17&sr=b&sig=mZZ1kK00Gc2cg5m6dyRB3XUYVB40dxJkvBMbXW4D3cM%3D&spr=https&se=2020-02-15T03%3A29%3A57Z&sp=r&rscl=x-e2eid-c561122c-76784a8c-be2cb59f-c79091dc-session-eacd1913-b0014305-a33f0081-46cabc28] Try 1/5, retryable exception caught. Retrying in 00:00:01. System.Threading.Tasks.TaskCanceledException: A task was canceled.
2020-02-14T03:24:18.3151412Z at System.IO.Stream.CopyToAsyncInternal(Stream destination, Int32 bufferSize, CancellationToken cancellationToken)
2020-02-14T03:24:18.3151697Z at System.Net.Http.NoWriteNoSeekStreamContent.<>c.<SerializeToStreamAsync>b__4_0(Task t, Object s)
2020-02-14T03:24:18.3151962Z at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
2020-02-14T03:24:18.3152497Z --- End of stack trace from previous location where exception was thrown ---
2020-02-14T03:24:18.3152771Z at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot)
2020-02-14T03:24:18.3153248Z --- End of stack trace from previous location where exception was thrown ---
2020-02-14T03:24:18.3153496Z at System.Net.Http.HttpContent.LoadIntoBufferAsyncCore(Task serializeToStreamTask, MemoryStream tempBuffer)
2020-02-14T03:24:18.3153749Z at System.Net.Http.HttpClient.FinishSendAsyncBuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
2020-02-14T03:24:18.3154100Z at Microsoft.VisualStudio.Services.Common.TaskCancellationExtensions.EnforceCancellation[TResult](Task`1 task, CancellationToken cancellationToken, Func`1 makeMessage, String file, String member, Int32 line)
2020-02-14T03:24:18.3154606Z at Microsoft.VisualStudio.Services.BlobStore.WebApi.DedupStoreHttpClient.<>c__DisplayClass57_0.<<GetRedirectResponseAsync>b__0>d.MoveNext()
2020-02-14T03:24:18.3155603Z --- End of stack trace from previous location where exception was thrown ---
2020-02-14T03:24:18.3155739Z at Microsoft.VisualStudio.Services.Content.Common.AsyncHttpRetryHelper`1.InvokeAsync(CancellationToken cancellationToken)
2020-02-14T03:24:18.3155929Z at Microsoft.VisualStudio.Services.BlobStore.WebApi.DedupStoreHttpClient.<>c__DisplayClass60_0.<<HandleRedirectAsync>b__0>d.MoveNext()
2020-02-14T03:24:18.3159451Z --- End of stack trace from previous location where exception was thrown ---
2020-02-14T03:24:18.3159789Z at Microsoft.VisualStudio.Services.Content.Common.AsyncHttpRetryHelper`1.InvokeAsync(CancellationToken cancellationToken)
2020-02-14T03:24:19.9319176Z Information, Downloaded 340.2 MB out of 447.6 MB (76%).
2020-02-14T03:24:24.9323050Z Information, Downloaded 352.6 MB out of 447.6 MB (79%).
Is there any way of avoiding that? I have also checked https://status.dev.azure.com/ and there is no indication of any issues with it. These are with the paid Azure Devops agent pool not a private agent pool.
There is a similar issue that has been open here. You can join in the discussion of this thread or open new issue for above case.
you can try the workaround of using continueOnError: true on the cache task to prevent it from failing the build.
- task: Cache#2
inputs:
key: 'npm | "$(Agent.OS)" | package-lock.json'
restoreKeys: |
npm | "$(Agent.OS)"
npm
path: $(npm_config_cache)
displayName: Cache npm
continueOnError: true

elasticsearch nest SniffingConnectionPool not working

I'm using Nest.ElasticClient to connect to Elasticsearch cluster. The cluster is located in Azure VM with just one node.
the cluster is accessible outside the vm by url : http://xxxx.cloudapp.net:9200 and also accessible by ElasticClient if not using SniffingConnectionPool. But not accessible by ElasticClient if using SniffingConnectionPool.
Here is the network config
network.host: [_local_, _site_]
Below is the source code I'm using to get client and check index exists.
var pool = new SniffingConnectionPool(urls.Select(url => new Uri(url)));
ConnectionSettings config = new ConnectionSettings(pool) ;
client = new Nest.ElasticClient(config);
IExistsResponse indexExistsResponse = client.IndexExists(indexName);
The debug info message when I try to use the client to check whether a Index exists, the hostnanme and ip address is modified:
Invalid NEST response built from a unsuccessful low level call on HEAD: /globalleads
# Audit trail of this API call:
- SniffOnStartup: Took: 00:00:00.9846171
- SniffSuccess: Node: http://xxxx.cloudapp.net:9200/ Took: 00:00:00.9595496
- PingFailure: Node: http://10.85.xxx.xx:9200/ Exception: PipelineException Took: 00:00:21.4154660
- SniffOnFail: Took: 00:00:21.1967809
- SniffFailure: Node: http://10.85.xxx.xx:9200/ Exception: PipelineException Took: 00:00:21.1787333
# OriginalException: Elasticsearch.Net.ElasticsearchClientException: One or more errors occurred. ---> System.AggregateException: One or more errors occurred. ---> Elasticsearch.Net.PipelineException: Failed sniffing cluster state. ---> System.AggregateException: One or more errors occurred. ---> Elasticsearch.Net.PipelineException: An error occurred trying to establish a connection with the specified node.
at Elasticsearch.Net.RequestPipeline.Sniff() in D:\dev\git\elasticsearch-net-2.x\src\Elasticsearch.Net\Transport\Pipeline\RequestPipeline.cs:line 326
--- End of inner exception stack trace ---
--- End of inner exception stack trace ---
at Elasticsearch.Net.RequestPipeline.Sniff() in D:\dev\git\elasticsearch-net-2.x\src\Elasticsearch.Net\Transport\Pipeline\RequestPipeline.cs:line 341
at Elasticsearch.Net.RequestPipeline.SniffOnConnectionFailure() in D:\dev\git\elasticsearch-net-2.x\src\Elasticsearch.Net\Transport\Pipeline\RequestPipeline.cs:line 301
at Elasticsearch.Net.Transport`1.Ping(IRequestPipeline pipeline, Node node) in D:\dev\git\elasticsearch-net-2.x\src\Elasticsearch.Net\Transport\Transport.cs:line 179
at Elasticsearch.Net.Transport`1.Request[TReturn](HttpMethod method, String path, PostData`1 data, IRequestParameters requestParameters) in D:\dev\git\elasticsearch-net-2.x\src\Elasticsearch.Net\Transport\Transport.cs:line 68
--- End of inner exception stack trace ---
--- End of inner exception stack trace ---
# Audit exception in step 2 PingFailure:
Elasticsearch.Net.PipelineException: An error occurred trying to establish a connection with the specified node.
at Elasticsearch.Net.RequestPipeline.Ping(Node node) in D:\dev\git\elasticsearch-net-2.x\src\Elasticsearch.Net\Transport\Pipeline\RequestPipeline.cs:line 248
# Audit exception in step 4 SniffFailure:
Elasticsearch.Net.PipelineException: An error occurred trying to establish a connection with the specified node.
at Elasticsearch.Net.RequestPipeline.Sniff() in D:\dev\git\elasticsearch-net-2.x\src\Elasticsearch.Net\Transport\Pipeline\RequestPipeline.cs:line 326
# Request:
<Request stream not captured or already read to completion by serializer. Set DisableDirectStreaming() on ConnectionSettings to force it to be set on the response.>
# Response:
<Response stream not captured or already read to completion by serializer. Set DisableDirectStreaming() on ConnectionSettings to force it to be set on the response.>

Publish telerik application

I want to publish the telerik app builder.It shows this issue.
Build failed: One or more errors occurred..
System.AggregateException: One or more errors occurred. ---> System.InvalidOperationException: Sequence contains more than one matching element
at System.Linq.Enumerable.SingleOrDefault[TSource](IEnumerable1 source, Func2 predicate)
at Telerik.BlackDragon.Common.Data.TargetResultDataExtensions.GetRelativeOutputPath(TargetResultData targetResultData, DevicePlatform platform)
at Telerik.BlackDragon.Common.Data.TargetResultDataExtensions.GetRelativePathToPackage(TargetResultData targetResultData, DevicePlatform platform)
at System.Linq.Enumerable.ToDictionary[TSource,TKey,TElement](IEnumerable1 source, Func2 keySelector, Func2 elementSelector, IEqualityComparer1 comparer)
at Telerik.BlackDragon.Client.Build.Services.BuildService.d__2.MoveNext()
--- End of inner exception stack trace ---
at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
at System.Threading.Tasks.Task1.GetResultCore(Boolean waitCompletionNotification)
at System.Threading.Tasks.Task1.get_Result()
at Telerik.BlackDragon.Client.Build.CloudBuild.ExecuteCore(IHttpServiceContext httpServiceContext, LogHelper logger)
at Telerik.BlackDragon.Client.Build.CloudTaskBase.ExecuteCore()
---> (Inner Exception #0) System.InvalidOperationException: Sequence contains more than one matching element
at System.Linq.Enumerable.SingleOrDefault[TSource](IEnumerable1 source, Func2 predicate)
at Telerik.BlackDragon.Common.Data.TargetResultDataExtensions.GetRelativeOutputPath(TargetResultData targetResultData, DevicePlatform platform)
at Telerik.BlackDragon.Common.Data.TargetResultDataExtensions.GetRelativePathToPackage(TargetResultData targetResultData, DevicePlatform platform)
at System.Linq.Enumerable.ToDictionary[TSource,TKey,TElement](IEnumerable1 source, Func2 keySelector, Func2 elementSelector, IEqualityComparer1 comparer)
at Telerik.BlackDragon.Client.Build.Services.BuildService.d__2.MoveNext()
It's because you have this error
System.AggregateException: One or more errors occurred. ---> System.InvalidOperationException: Sequence contains more than one matching element
It means that your linq query returns more than one element. if it's normal, you should use FirstOrDefault().
SingleOrDefault() throws an exception if more than one element exists.

Application is crashing when future date is set in windows phone using parse push notification

I'm trying to follow your quickstart on push notifications on windows phone 8 and it doesn't seem to be functional.
https://www.parse.com/apps/quickstart_push#wp/existing
await ParsePush.SubscribeAsync("");
results and an unhandled exception in both the emulator and on a physical devices.
{Parse.ParseException: Invalid response from server ---> System.ArgumentException: Input JSON was invalid. at Parse.Internal.Json.Parse(String input) at Parse.ParseClient.DeserializeJsonString(String jsonData) at Parse.ParseClient.<>cDisplayClassb.ba(Task1 t) --- End of inner exception stack trace --- at Parse.ParseClient.<>c__DisplayClassb.b__a(Task1 t) at Parse.Internal.InternalExtensions.<>cDisplayClass12.b__0(Task t) at Parse.Internal.InternalExtensions.<>c__DisplayClass71.b6(Task t) at System.Threading.Tasks.ContinuationResultTaskFromTask1.InnerInvoke() at System.Threading.Tasks.Task.Execute() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at Parse.Internal.InternalExtensions.<>c__DisplayClass71.b6(Task t) at System.Threading.Tasks.ContinuationResultTaskFromTask`1.InnerInvoke() at System.Threading.Tasks.Task.Execute() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Runtime.CompilerServices.TaskAwaiter.GetResult() at Parse.ParseInstallation.d4.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Runtime.CompilerServices.TaskAwaiter.GetResult() at ObjectivePixel.BringCast.Cloud.CloudPlugin.d14.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Runtime.CompilerServices.TaskAwaiter.GetResult() at ObjectivePixel.BringCast.Silverlight.App.d2.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.CompilerServices.AsyncMethodBuilderCore.b__0(Object state)}

I can't download from https on MonoDroid - am I missing something obvious?

Here's the code I'm running:
var wc = new WebClient ();
var test1 = wc.DownloadString("http://www.google.com");
var test2 = wc.DownloadString("https://www.google.com");
test1 gives me some text.
test2 never gets initialised because of an exception with trace starting:
{System.Net.WebException: Error getting response stream (Write:
EndWrite failure): SendFailure ---> System.IO.IOException: EndWrite
failure ---> System.Net.Sockets.SocketException: The socket has been
shut down at System.Net.Sockets.Socket.EndSend (IAsyncResult result)
[0x0002d] in
/home/jon/Development/xamarin/mono/mcs/class/System/System.Net.Sockets/Socket_2_1.cs:1989
at System.Net.Sockets.NetworkStream.EndWrite (IAsyncResult ar)
[0x0002f] in
/home/jon/Development/xamarin/mono/mcs/class/System/System.Net.Sockets/NetworkStream.cs:358
--- End of inner exception stack trace --- at System.Net.Sockets.NetworkStream.EndWrite (IAsyncResult ar) [0x0003d]
in
/home/jon/Development/xamarin/mono/mcs/class/System/System.Net.Sockets/NetworkStream.cs:360
at Mono.Security.Protocol.Tls.RecordProtocol.EndSendRecord
(IAsyncResult asyncResult) [0x00040] in
/home/jon/Development/xamarin/mono/mcs/class/Mono.Security/Mono.Security.Protocol.Tls/RecordProtocol.cs:721
at Mono.Security.Protocol.Tls.RecordProtocol.SendRecord (ContentType
contentType, System.Byte[] recordData) [0x0000b] in
/home/jon/Development/xamarin/mono/mcs/class/Mono.Security/Mono.Security.Protocol.Tls/RecordProtocol.cs:729
I'm running MonoDevelop.MonoDroid - 2.6.0.0 - with a download from MonoDroid a couple of days ago - "Mono for Android: 1.2.24817.105690530 (Evaluation)"
Is this just a bug - or is there some other step I need for MonoDroid?
It seems this has been confirmed as a bug in the 1.2 release - so I'm closing this question!

Resources