ELK serilog sink FailureCallback - elasticsearch

Question

I was able to successfully integrate the elastic search sink in my .net app, now I am trying to setup the FailureCallback option but every time there is an error the exception in LogEvent is null, I can see the actual error in the console but I want to be able to capture this exception in my failure callback function, here is my current configuration:
myLogger = new LoggerConfiguration()
.WriteTo.Elasticsearch(new ElasticsearchSinkOptions(new Uri(elasticSearchUrl))
{
AutoRegisterTemplate = true,
AutoRegisterTemplateVersion = AutoRegisterTemplateVersion.ESv6,
IndexDecider = (#event, dateTimeOffset) =>
{
//some logic here
return $"custom-index";
},
EmitEventFailure = EmitEventFailureHandling.WriteToSelfLog | EmitEventFailureHandling.RaiseCallback,
FailureCallback = HandleElasticError
})
.Enrich.WithProperty("Environment", Config.Environment)
.Destructure.ByTransforming<ExpandoObject>(JsonConvert.SerializeObject)
.CreateLogger();
Here is my HandleElasticError function:
private void HandleElasticError(LogEvent e)
{
FileLogger.Error(e.Exception, e.MessageTemplate.Text, e.Properties);
}
When I attach the debugger and inspect the LogEvent, exception is null, however in the output window in Visual Studio I can see the actual error:
2020-11-30T20:55:07.7820674Z Caught exception while preforming bulk operation to Elasticsearch: Elasticsearch.Net.ElasticsearchClientException: The underlying connection was closed: An unexpected error occurred on a send.. Call: Status code unknown from: POST /_bulk ---> System.Net.WebException: The underlying connection was closed: An unexpected error occurred on a send. ---> System.IO.IOException: The handshake failed due to an unexpected packet format.
at System.Net.TlsStream.EndWrite(IAsyncResult asyncResult)
at System.Net.PooledStream.EndWrite(IAsyncResult asyncResult)
at System.Net.ConnectStream.WriteHeadersCallback(IAsyncResult ar)
--- End of inner exception stack trace ---
I am not concerned about the error, I know what the issue is, I just want to be able to capture this exception in my failure callback function.
Any ideas?

Answer 1

LogEvent.Exception is the Exception shipped with your call to (for example) logger.Error(myException) - not the exception thrown by failure to write to Elastic Search.
The exception you're trying to catch doesn't look to be exposed by the sink as it just passes the LogEvent to FailureCallback and not exception that caused the failure.
Here's what it could look like were the exception exposed.

Answer 2

you can use:
StreamWriter writer = File.CreateText(...);
Serilog.Debugging.SelfLog.Enable(writer);