Use JSON Input to Replace 'Variable' in AWS Step Function

Question

I have an AWS Step Function, which I would like to pass a value to via the JSON input included when starting the execution. The Step function starts an ETL job and checks it's status via calling Lambda Functions that perform said tasks. The StartGlueJob Lambda function is initiated via a JobName JSON input. The value I would like to pass in is an argument for the glue job named 'regionalCenters' where the desired value would be 'LA' in this case.

I have attemped, as seen below, to use the $.value syntax to pass from the JSON input but this is not functioning as intended and I am not sure why.

Snippet of Lambda Code:

let jobInfo = await glue.startJobRun({ JobName: jobName, Arguments: arguments }).promise();

JSON input

{
    "value": "LA"
}

Step Function

{
  "StartAt": "InitStartEtl",
  "States": {
    "InitStartEtl": {
      "Type": "Pass",
      "Result": {
        "jobName": "name-of-my-etl-job",
        "arguments": {"regionalCenters": "$.value"}
      }
    },
    "StartEtl": {
      "Type": "Task",
      "Resource": "arn:aws:lambda:<redacted>",
      "Next": "WaitForForEtlCheck"
    },
    "WaitForForEtlCheck": {
      "Type": "Wait",
      "Seconds": 10,
      "Next": "GetEtlState"
    },
    "GetEtlState": {
      "Type": "Task",
      "Resource": "arn:aws:lambda:<redacted>",
      "Next": "IsEtlFinished"
    },
    "IsEtlFinished": {
      "Type": "Choice",
      "Choices": [
        {
          "Variable": "$.state",
          "StringEquals": "READY",
          "Next": "EtlDone"
        }
      ],
      "Default": "WaitForForEtlCheck"
    },
    "EtlDone": {
      "Type": "Pass",
      "End": true
    }
  }
}

feechka · Accepted Answer · 2019-09-25 09:52:20Z

4

You missed .$ in the end regionalCenters

It should be "arguments": {"regionalCenters.$": "$.value"}

see here https://docs.aws.amazon.com/step-functions/latest/dg/connect-parameters.html

answered Sep 25, 2019 at 9:52

feechka

2256 silver badges16 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

whit.the.engineer · Accepted Answer · 2019-04-30 16:58:46Z

1

I was able to solve to my own problem by changing up the Result and adding a ResultPath, a la:

{
  "StartAt": "InitStartEtl",
  "States": {
    "InitStartEtl": {
      "Type": "Pass",
      "Result": "name-of-my-etl-job",
      "ResultPath": "$.jobName",
      "Next": "StartEtl"
    },

Changing the Result allowed me to add the string to the .jobName node via the ResultPath, while appending the input of {"value": "LA"} to the Output

answered Apr 30, 2019 at 16:58

whit.the.engineer

3162 gold badges7 silver badges23 bronze badges

Collectives™ on Stack Overflow

Use JSON Input to Replace 'Variable' in AWS Step Function

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related