I have a lambda trigger on a dynamoDb table event.
Sometimes the trigger is running fine, but sometimes Function call Failed error is thrown.
The lambda memory is set to 1024MB and timeout to 05mins.
The problem is that when the function call failed error comes, no logs are logged on cloudWatch, hence I am not able to debug or identify why the error is coming.
Any insights on how to identify the issue?
I recommend following the example in this article, New AWS Lambda controls for stream processing and asynchronous invocations, to set up some error handling. It will give you some visibility to what is going on.
Basically, use Lambda Destinations for failures.
Related
I am using CloudWatch log subscription filters to get logs from a specific log group and send them to a Lambda function, which after processing will send the results to another service. I wonder if there's any possibility to send failed events by Lambda to a Dead Letter Queue, noting that in the above settings, we have no SNS/SQS setup to trigger the Lambda.
Destinations gives you the ability to handle the Failure of function
invocations along with their Success. When a function invocation
fails, such as when retries are exhausted or the event age has been
exceeded (hitting its TTL), Destinations routes the record to the
destination resource for every failed invocation for further
investigation or processing.
To configure destination in Lambda function, Kindly refer
I have 2 cloudwatch log subscriptions from the same log, that both pointing to the same lambda function.
Is there anyway to tell which subscription fired off the lambda from the logs or event? i dont see anything in the destination function logs except for the line(s) from the log that caused it to run.
I see something like the following error while using azure event hub to send event message. But as I see in the azure portal, the metric shows that the event message is sent to the event hub. So I'm puzzled by what this error message means.
As I read from the azure doc (https://learn.microsoft.com/en-us/azure/event-hubs/event-hubs-amqp-troubleshoot), it said "You see the following error on the AMQP connection when all links in the connection have been closed because there was no activity (idle) and a new link hasn't been created in 5 minutes."
The doc also said "You can avoid them by making send/receive calls on the connection/link, which automatically recreates the connection/link."
What should be done regarding this error message ? As although the event message can be sent, I worry whether there may be any potential issue there.
" Error{condition=amqp:connection:forced, description='The connection was inactive for more than the allowed 300000 milliseconds and is closed by container 'LinkTracker'. TrackingId:00000000000000000000000000000000000_G21, SystemTracker:gateway5, Timestamp:2019-03-06T17:32:00', info=null} "
I once tried that if I call the close() method in EventHubProducerClient (by refer to sample code in https://learn.microsoft.com/en-us/azure/event-hubs/event-hubs-java-get-started-send), this error seems not appear again. However, if doing so, when every time need to send the event, it will mean need to create a new EventHubProducerClient. I'm not sure if this may create another problem (like time required to create the new EventHubProducerClient, and memory consumption) if creating a new EventHubProducerClient for every send event, as there can be many events to send.
On another search, I found in How to configure Producer.close() in Eventhub, that it is recommended to close the producer client after using it.
However, if the above error message is actually not an error, whether to close or not may not matter.
I have configured a lambda function as EventBridge rule target and I have configured a Dead Letter Queue on the EventBridge rule to capture exceptions.
Now, if the lambda function fails, Event Bridge does not recognize that failure as an error.
Since the EventBridge invocation to the Lambda is asynchronous, for EventBridge it is enough to reach the lambda to consider the event as successfull,but in this way I am not able to track and retry events once lambda fails.
Anyone know a way to make the EventBridge to the Lamdba request synchronous or another way to be able to Retry the events if the Lambda code fails after the invocation?
one option is to make the SQS to be rule target and use SQS event to trigger the lambda. when failure occurs, the lambda won't flag the event done so as to keep the event in the SQS. retry will auto happen after a configured period (SQS configuration). also you can configure dead letter queue after the retention time expires
EventBridge guarantees the delivery of the event to the lambda function but is not aware of what happens post that. It's lambda invocation vs lambda execution. Eventbridge successfully delivered the message to the lambda service, so it's a successful invocation.
For Lambda, EventBridge calls invokeAsync API. So, if this API sends a success response, EventBridge will assume the delivery was successful. Any failures within Lambda from async to sync is not visible to EventBridge. We should configure retries and create DLQs in our Lambda functions to make sure the events are not lost in case the Lambda function fails to execute. We could in fact configure the same DLQ used by the EventBridge to be used by the Lambda as well so that all eventual failures land in a single place.
AWS has a dedicated documentation page for this, which states the following for asynchronous invocation:
Lambda retries function errors twice. If the function doesn't have enough capacity to handle all incoming requests, events might wait in the queue for hours or days to be sent to the function. You can configure a dead-letter queue on the function to capture events that weren't successfully processed. For more information, see Asynchronous invocation.
So that means that as long as your Lambda functions handler function does return an error, the AWS Lambda service should retry to run your Lambda again.
Therefore, you might not need EventBridge to retry your event.
See: Error handling and automatic retries in AWS Lambda
I am trying to put message to Azure service bus queue, and using QueueClient to send the message. But it is not sending message to queue. But it is not even giving any error or exception. It is silently failing not sure what is going wrong here.
queueClient.Send(message);
This line is not throwing exception but not even sending message for Q1, but able to send message to Q2 in same service bus namespace.
QueueClient is from Microsoft.ServiceBus.Messaging.QueueClient.
Thanks
Please check out whether there is any autocomplete option enabled because if there is no exception or any error then it might be the problem. Also, check whether you have used any autocomplete function. If it so, that function would delete all of your message and remove it.