Step Functions ignoring Parameters, executing it with default input - aws-lambda

I don't know if this is a bug of my own or Amazon's as I'm unable to properly understand what's going on.
I have the following Step Function (It's currently a work in progress), which takes care of validating an input and then sending it to an SQS lambda (CreateInAuth0 Step).
My problem is that apparently the SQS function is executing with the default parameters, fails, retries and then uses the ones specified in the Step Function. (Or at least that's what I'm gathering from the logs).
I want to know why it is ignoring the Parameters in the first go. Here's all the information I can gather.
There seems to be a difference between TaskStateEntered and TaskScheduled (which I believe it's ok).
{
"Comment": "Add the students to the platform and optionally adds them to their respective grades",
"StartAt": "AddStudents",
"States": {
"AddStudents": {
"Comment": "Validates that the students are OK",
"Type": "Task",
"Resource": "#AddStudents",
"Next": "Parallel"
},
"Parallel": {
"Type": "Parallel",
"Next": "FinishExecution",
"Branches": [
{
"StartAt": "CreateInAuth0",
"States": {
"CreateInAuth0": {
"Type": "Task",
"Resource": "arn:aws:states:::sqs:sendMessage.waitForTaskToken",
"Parameters": {
"QueueUrl": "#SQS_StudentsRegistrationSQSFifo",
"MessageBody": {
"execId.$": "$.result.execId",
"tenantType.$": "$.result.tenantType",
"tenantId.$": "$.result.tenantId",
"studentsToCreate.$": "$.result.newStudents.success",
"TaskToken.$": "$$.Task.Token",
"studentsSucceeded": []
}
},
"End": true
}
}
}
]
},
"FinishExecution": {
"Comment": "This signals the ending in DynamoDB that the transaction finished",
"Type": "Task",
"End": true,
"Resource": "arn:aws:states:::dynamodb:putItem",
"ResultPath": "$.dynamoDB",
"Parameters": {
"TableName": "#SchonDB",
"Item": {
"pk": {
"S.$": "$.arguments.tenantId:exec:$.id"
},
"sk": {
"S": "result"
},
"status": {
"S": "FINISHED"
},
"type": {
"S": "#EXEC_TYPE"
}
}
},
"Retry": [
{
"ErrorEquals": [
" States.Timeout"
],
"IntervalSeconds": 1,
"MaxAttempts": 5,
"BackoffRate": 2.0
}
],
"Catch": [
{
"ErrorEquals": [
"States.ALL"
],
"Next": "FinishExecutionIfFinishErrored"
}
]
},
"FinishExecutionIfFinishErrored": {
"Type": "Pass",
"End": true
}
}
}
Here's the visual:
Here's the execution event history:
Note: I have a try/catch statement inside the SQS function that wraps the entire SQS' execution.

Related

Unable to start AWSFIS-Run-CPU-Stress

While running AWSFIS-Run-CPU-Stress i am getting below error:
Unable to start action, due to a platform mismatch between the specified document and the targeted instances. I am trying this in Windows EC2 instance
My Experiment script look like this(removed confidential server info):
{
"description": "Test CPU stress predefined SSM document",
"targets": {
"testInstance": {
"resourceType": "aws:ec2:instance",
"resourceArns": [
"arn:aws:ec2:region:123456789012:instance/instance_id"
],
"selectionMode": "ALL"
}
},
"actions": {
"runCpuStress": {
"actionId": "aws:ssm:send-command",
"parameters": {
"documentArn": "arn:aws:ssm:region::document/AWSFIS-Run-CPU-Stress",
"documentParameters": "{\"DurationSeconds\":\"120\"}",
"duration": "PT5M"
},
"targets": {
"Instances": "testInstance"
}
}
},
"stopConditions": [
{
"source": "aws:cloudwatch:alarm",
"value": "arn:aws:cloudwatch:region:123456789012:alarm:awsec2-instance_id-GreaterThanOrEqualToThreshold-CPUUtilization"
}
],
"roleArn": "arn:aws:iam::123456789012:role/AllowFISSSMActions",
"tags": {}
}

Pass Step Function variable to AWS Glue Job Not Working

I'm trying to pass an AWS Step Function variable to a Glue Job parameter, similar to this:
aws-passing-job-parameters-value-to-glue-job-from-step-function
However, this is not working for me. The glue job error message indicates that it's getting the passed variable name--not the actual value of the variable. Here's my Step Function code:
{
"Comment": "Converts CSV files to parquet for a date range.",
"StartAt": "ConfigureCount",
"States": {
"ConfigureCount": {
"Type": "Pass",
"Result": {
"start": 201601,
"end": 201602,
"index": 201601
},
"ResultPath": "$.iterator",
"Next": "Iterator"
},
"Iterator": {
"Type": "Task",
"Resource": "arn:aws:lambda:eu-west-1:123456789:function:date-iterator",
"ResultPath": "$.iterator",
"Next": "IsCountReached"
},
"IsCountReached": {
"Type": "Choice",
"Choices": [
{
"Variable": "$.iterator.continue",
"BooleanEquals": true,
"Next": "ConvertToParquet"
}
],
"OutputPath": "$.iterator",
"Default": "Done"
},
"ConvertToParquet": {
"Comment": "Your application logic, to run a specific number of times",
"Type": "Task",
"Resource": "arn:aws:states:::glue:startJobRun.sync",
"Parameters": {
"JobName": "convert-to-parquet",
"Arguments": {
"--DATE_RANGE": "$.iterator.index"
}
},
"ResultPath": "$.iterator.index",
"Next": "Iterator"
},
"Done": {
"Type": "Pass",
"End": true
}
}
}
The step "Iterator"step is calling a Lambda called "date-iterator" which returns JSON similar to the following:
{
"start": "201601",
"end": "201602",
"index": "201601"
}
This was based on this article, so that I can loop through values: Iterating a Loop Using Lambda
My Step Function fails, saying "$.iterator.index" is not a valid date.
How do I pass this value, and not the variable name?
from Amazon States Language (https://states-language.net/spec.html):
If any field within the Payload Template (however deeply nested) has a name ending with the characters ".$", its value is transformed according to rules below and the field is renamed to strip the ".$" suffix.
Based on that adding .$ should solve your issue:
"Parameters": {
"JobName": "convert-to-parquet",
"Arguments": {
"--DATE_RANGE.$": "$.iterator.index"
}
},

AWS Step Function Lambda Returning Null Output Payload

I have created a basic state machine that runs a lambda and then uses the output of that lambda to decide a choice. However, I have not been able to receive the output from the lambda function because it keeps returning a null payload. I have tried running the lambda alone from the lambda console and it runs fine (including with the expected return). However, within the step function, the task at least, is returning null for the Payload output.
Here is the definition of my state-machine:
{
"StartAt": "ValidateWorkcellConfigTask",
"States": {
"ValidateWorkcellConfigTask": {
"Next": "ConfigValidationTypeChoiceState",
"Parameters": {
"FunctionName": "ValidateWorkcellConfig",
"InvocationType": "Event",
"Payload.$": "$"
},
"OutputPath": "$",
"Type": "Task",
"Resource": "arn:aws:states:::lambda:invoke",
"TimeoutSeconds": 15
},
"ConfigValidationTypeChoiceState": {
"Type": "Choice",
"Comment": "Forwards a VALID workflow config to the Provisioning Workflow or Relays the INVALID workflow config for logging",
"Choices": [
{
"Variable": "$.uuid",
"StringEquals": "SEA90-1",
"Next": "endState"
},
{
"Variable": "$.validConfig",
"StringEquals": "SEA90-1",
"Next": "SuccessRelayTask"
}
]
},
"endState": {
"Type": "Pass",
"End": true
},
"SuccessRelayTask": {
"End": true,
"Parameters": {
"FunctionName": "StatusRelayHandler",
"InvocationType": "Event",
"Payload.$": "$"
},
"OutputPath": "$",
"Type": "Task",
"Resource": "arn:aws:states:::lambda:invoke",
"ResultPath": "$",
"TimeoutSeconds": 15
}
}
}
And here is the output of the lambda task execution:
{
"name": "ValidateWorkcellConfigTask",
"output": {
"Payload": null,
"SdkHttpMetadata": {
"AllHttpHeaders": {
"x-amzn-Remapped-Content-Length": [
"0"
],
"Connection": [
"keep-alive"
],
"x-amzn-RequestId": [
"4452a8d2-e607-4ae6-943c-9478b0f59ce0"
],
"Content-Length": [
"0"
],
"Date": [
"Sat, 16 Jan 2021 09:22:48 GMT"
],
"X-Amzn-Trace-Id": [
"root=1-6002b068-52fb3d1f60f20d3e10804f96;sampled=0"
]
},
"HttpHeaders": {
"Connection": "keep-alive",
"Content-Length": "0",
"Date": "Sat, 16 Jan 2021 09:22:48 GMT",
"x-amzn-Remapped-Content-Length": "0",
"x-amzn-RequestId": "4452a8d2-e607-4ae6-943c-9478b0f59ce0",
"X-Amzn-Trace-Id": "root=1-6002b068-52fb3d1f60f20d3e10804f96;sampled=0"
},
"HttpStatusCode": 202
},
"SdkResponseMetadata": {
"RequestId": "4452a8d2-e607-4ae6-943c-9478b0f59ce0"
},
"StatusCode": 202
},
"outputDetails": {
"truncated": false
}
}
Notice how the payload is null... that is causing the next task to fail because it receives a null input. I've already played around with OutputPath, ResultPath, and InputPath but have never been able to return something from the lambda task.
The lambda itself is written in Java and return a String. It implements this interface:
... implements RequestHandler<WorkcellConfig, String>
I think the problem that you have may be due to the Invocation Type, you set it as event (asynchronously) so you give no time to the lambda function to return the result payload. Try changing it to RequestResponse to run it synchronously (this is the default type, so you can omit this parameter).
You can check more details about Invocation Type here.
The code should look like this:
{
"StartAt": "ValidateWorkcellConfigTask",
"States": {
"ValidateWorkcellConfigTask": {
"Next": "ConfigValidationTypeChoiceState",
"Parameters": {
"FunctionName": "ValidateWorkcellConfig",
"Payload.$": "$"
},
"OutputPath": "$",
"Type": "Task",
"Resource": "arn:aws:states:::lambda:invoke",
"TimeoutSeconds": 15
},
"ConfigValidationTypeChoiceState": {
"Type": "Choice",
"Comment": "Forwards a VALID workflow config to the Provisioning Workflow or Relays the INVALID workflow config for logging",
"Choices": [
{
"Variable": "$.uuid",
"StringEquals": "SEA90-1",
"Next": "endState"
},
{
"Variable": "$.validConfig",
"StringEquals": "SEA90-1",
"Next": "SuccessRelayTask"
}
]
},
"endState": {
"Type": "Pass",
"End": true
},
"SuccessRelayTask": {
"End": true,
"Parameters": {
"FunctionName": "StatusRelayHandler",
"Payload.$": "$"
},
"OutputPath": "$",
"Type": "Task",
"Resource": "arn:aws:states:::lambda:invoke",
"ResultPath": "$",
"TimeoutSeconds": 15
}
}
}

Alexa.Discovery response: no device detected by Alexa

I am implementing my Alexa Home Skill using AWS Lambda.
Given the following request I receive when I try to detect new devices on Alexa Skil test page:
{directive={header={namespace=Alexa.Discovery, name=Discover, payloadVersion=3, messageId=0160c7e7-031f-47ee-a1d9-a23f38f87a9e}, payload={scope={type=BearerToken, token=...}}}}
I respond with the following:
{
"event": {
"payload": {
"endpoints": [
{
"displayCategories": [
"SMARTPLUG"
],
"capabilities": [
{
"type": "AlexaInterface",
"interface": "Alexa",
"version": "3"
},
{
"type": "AlexaInterface",
"interface": "Alexa.PowerController",
"version": "3",
"properties": {
"retrievable": true,
"supported": [
{
"name": "powerState"
}
],
"proactivelyReported": true
}
},
{
"type": "AlexaInterface",
"interface": "Alexa.EndpointHealth",
"version": "3",
"properties": {
"retrievable": true,
"supported": [
{
"name": "connectivity"
}
],
"proactivelyReported": true
}
}
],
"manufacturerName": "mirko.io",
"endpointId": "ca84ef6d-53b1-430a-8a5e-a62f174eac5e",
"description": "mirko.io forno (id: ca84ef6d-53b1-430a-8a5e-a62f174eac5e)",
"friendlyName": "forno"
}
]
},
"header": {
"payloadVersion": "3",
"namespace": "Alexa.Discovery",
"name": "Discover.Response",
"messageId": "c0555cc8-ad7a-4377-b310-9de9b9ab6282"
}
}
}
Despite that, for some reasons Alexa answers that it did not find any new device.
I may be mistaken but I am pretty sure it used to work before I decided to add the Alexa.EndpointHealth interface.
Your response object looks right to me, except the extra "endpoint" field.
"endpoint": {
"endpointId": "INVALID",
"scope": {
"type": "BearerToken",
"token": "INVALID"
}
}
There's no such field in the Alexa.Discovery documentation. Try removing it and see if it resolves the issue.

Getting started with Step Functions and I can't get Choice to work correctly

I'm doctoring up my first step function, and as a newb into this I am struggling to make this work right. The documentation on AWS is helpful but lacks examples of what I am trying understand. I found a couple similar issues on the site here, but they didn't really answer my question either.
I have a test Step Function that works really simply. I have a small Lambda function that kicks out a single line JSON with a "Count" from a request in a DynamoDB:
def lambda_handler(event, context):
"""lambda_handler
Keyword arguments:
event -- dict -- A dict of parameters to be validated.
context --
Return:
json object with the hubID from DynamoDB of the new hub.
Exceptions:
None
"""
# Prep the Boto3 resources needed
dynamodb = boto3.resource('dynamodb')
table = dynamodb.Table('TransitHubs')
# By default we assume there are no new hubs
newhub = { 'Count' : 0 }
# Query the DynamoDB to see if the name exists or not:
response = table.query(
IndexName='Status-index',
KeyConditionExpression=Key('Status').eq("NEW"),
Limit=1
)
if response['Count']:
newhub['Count'] = response['Count']
return json.dumps(newhub)
A normal output would be:
{ "Count": 1 }
And then I create this Step Function:
{
"StartAt": "Task",
"States": {
"Task": {
"Type": "Task",
"Resource": "arn:aws:lambda:us-west-2:OMGSUPERSECRET:function:LaunchNode-get_new_hubs",
"TimeoutSeconds": 60,
"Next": "Choice"
},
"Choice": {
"Type": "Choice",
"Choices": [
{
"Variable": "$.Count",
"NumericEquals": 0,
"Next": "Failed"
},
{
"Variable": "$.Count",
"NumericEquals": 1,
"Next": "Succeed"
}
]
},
"Succeed": {
"Type": "Succeed"
},
"Failed": {
"Type": "Fail"
}
}
}
So I kick off the State Function and I get this output:
TaskStateExited
{
"name": "Task",
"output": {
"Count": 1
}
}
ChoiceStateEntered
{
"name": "Choice",
"input": {
"Count": 1
}
}
ExecutionFailed
{
"error": "States.Runtime",
"cause": "An error occurred while executing the state 'Choice' (entered at the event id #7). Invalid path '$.Count': The choice state's condition path references an invalid value."
}
So my question: I don't get why this is failing with that error message. Shouldn't Choice just pick up that value from the JSON? Isn't the default "$" input the "input" path?
I have figured out what the issue is, and here it is:
In my Python code, I was attempting to json.dumps(newhub) for the response thinking that what I needed was a string output representing the json formatted response. But it appears that is incorrect. When I alter the code to be simply "return newhub" and return the DICT, the step-functions process accepts that correctly. I'm assuming it parses the DICT to JSON for me? But the output difference is clearly obvious:
old Task output from above returning json.dumps(newhub):
{
"name": "Task",
"output": {
"Count": 1
}
}
new Task output from above returning newhub:
{
"Count": 1
}
And the Choice now correctly matches the Count variable in my output.
In case this is helpful for someone else. I also experienced the kind of error you did ( you had the below... just copy-pasting yours... )
{
"error": "States.Runtime",
"cause": "An error occurred while executing the state 'Choice' (entered at the event id #7). Invalid path '$.Count': The choice state's condition path references an invalid value."
}
But my problem turned out when I was missing the "Count" key all together.
But I did not want verbose payloads.
But per reading these docs I discovered I can also do...
"Choice": {
"Type": "Choice",
"Choices": [
{
"And": [
{
"Variable": "$.Count",
"IsPresent": true
},
{
"Variable": "$.Count",
"NumericEquals": 0,
}
],
"Next": "Failed"
},
{
"And": [
{
"Variable": "$.Count",
"IsPresent": true
},
{
"Variable": "$.Count",
"NumericEquals": 1,
}
],
"Next": "Succeed"
}
]
},
even I was facing the same issue. Give the ResultPath as InputPath and give the choice value as .value name
{
"Comment": "Step function to execute lengthy synchronous requests",
"StartAt": "StartProcessing",
"States": {
"StartProcessing": {
"Type": "Task",
"Resource": "arn:aws:lambda:us-east-1:703569030910:function:shravanthDemo-shravanthGetItems-GHF7ZA1p6auQ",
"InputPath": "$.lambda",
"ResultPath": "$.lambda",
"Next": "VerifyProcessor"
},
"VerifyProcessor": {
"Type": "Choice",
"Choices": [
{
"Variable": "$.lambda.cursor",
"IsNull": false,
"Next": "StartProcessing"
},
{
"Variable": "$.lambda.cursor",
"IsNull": true,
"Next": "EndOfUpdate"
}
]
},
"EndOfUpdate": {
"Type": "Pass",
"Result": "ProcessingComplete",
"End": true
}
}
}

Resources