How To Use AWS Step Functions For Serverless Workflows?

When using AWS services like Lambda, SNS, or DynamoDB, it can be challenging to connect and manage these services without writing complex code. AWS Step Functions solve this by allowing you to create workflows that coordinate multiple AWS services in a defined sequence, like a step-by-step process for your cloud tasks. The best part is that you don’t need to manually code the entire workflow logic, Step Functions handle the orchestration for you.

In this article, we will show you how to build a simple Step Function, how to connect it with AWS Lambda and SNS, and how to make decisions and handle errors in your workflow. Whether you are new to serverless or just want to simplify your logic, this article will walk you through everything step by step with clear examples you can try today.

What are AWS Step Functions?

AWS Step Functions is a service that helps you build and run workflows using visual diagrams. Think of it like creating a flowchart where each step is connected such as calling a Lambda function, sending a message, or waiting for a response.

You define how each step should work, what happens next, and how to handle errors. Step Functions makes sure everything runs in the correct order even if something fails or takes time.

AWS Step functions Examples

In this example, the algorithm takes two input numbers and add them, then :

• If the result is larger than 500 the user should receive a SMS with a random number between 500-1000.

• If it's less, then the random number in the SMS should be from 0-500.

• Otherwise, it will fail.

Steps to Implement AWS Step Functions

Step 1: Create an IAM Role

First, we need to create an IAM Role with the permissions that allow Step functions to get access to lambda.

1. Create a role under the IAM tab from the AWS Management Console.

Step 2: Select AWS Service under the Trusted Entity Type, under Use Case make sure to select Step Functions in the dropdown.

Step 3: Under the Add Permissions, leave it as default(Make sure AWSLambdaRole is added in the permissions policies). Then click on next to proceed.

Step 4: Under the Name, Review, and create section, enter the name of the Role according to you.

then Finally, click on create the role.

Now, before creating the step function, we will need to create the lambda functions since we need the ARN of the lambda functions in our step function code.

Steps To Create AWS Lambda Functions

Step 1: Create the Lambda Functions
We will create the lambda functions which will be used to connect to our state machine which will be created further. Here we will have 4 Lambda functions in this example:

• add_numbers - It takes { key1 : X, key2 : Y} as input and returns { number : Z }
• greater_than - It returns { final_number : rand(500,1000) }
• less_than - It returns { final_number : rand(0,500) }
• final_state - It takes { final_number : C }

Step 2: Click on Create Function under Lambda Console. Select Author from Scratch, and next give add_numbers as the name of your function, under Runtime select Python from the dropdown (or any other preferred Runtime language of your choice). Then click on Create Function.

Step 3: When the function is created, hover down to the Code section and replace the Function Code with the following code and then click on Deploy.

import json
 
def lambda_handler(event, context):
    number_1 = int(event['key1'])
    number_2 = int(event['key2'])
    return {"number" : abs(number_1 + number_2)}

After this, we need to make 3 more Lambda Functions named as greater_than, less_than, and final_state using the above-mentioned steps.

Below are the three different codes for three different functions:

Greater_than

import json
import random
 
def lambda_handler(event,context):
final_number = random.randint(500,1000)
    return {"final_number" : final_number}

Less_than

import json
import random
 
def lambda_handler(event,context):
final_number = random.randint(0,500)
    return {"final_number" : final_number}

Final_state

import json
import boto3
import subprocess
import os
import sys
import subprocess
 
# pip install custom package to /tmp/ and add to path
subprocess.call('pip install inflect -t /tmp/ --no-cache-dir'.split(), stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
sys.path.insert(1, '/tmp/')
import inflect
def lambda_handler(event,context):
inflect_engine = inflect.engine()
    number = inflect_engine.number_to_words(event['final_number'])
    message = {"DelMeMessage": "The StepFunctions Result is %r" %number}
    client = boto3.client('sns')
    response = client.publish(
TargetArn="SNS-TOPIC-ARN",
Message=json.dumps({'default': json.dumps(message)}),
MessageStructure='json'
    )

After completing these steps, all our functions will be created successfully.

Steps to Create SNS Topic

Step 1. Create an SNS Topic

Go to Create Topic under SNS in the AWS Management console. Under the type select Standard, also give a suitable name for your topic. Then click on Create Topic.

Step 2: Then we need to create a Subscription for the SNS Topic, under protocol select Email and in the endpoint add the E-mail ID as per your need. Finally, click on create subscription.

Also, make sure to add the target ARN in the state_machine function code.

Step 3: Create a State Machine and Serverless Workflow. Navigate to the Step Functions under the AWS Management Console.

Step 4: Click on State Machine and then click Create State Machine.

Step 5: Select Write your Workflow in Code and type as Standard.

Step 7: Under the definition window, change the code with Amazon States Language. This state machine uses a series of Task states to add numbers. Then, a Choice state is used to determine if the returned number is greater than , less than, or equal to a specified number. Then there is also a final state to conclude the task. Make sure under the resource in the code, change the ARN with the necessary ARN of the lambda functions. Then click on next.

{
"Comment": "A simple Step Functions state machine",
"StartAt": "AddNumbers",
"States": {
"AddNumbers": {
"Type": "Task",
"Resource": "ARN:OF:add-numbers",
"Next": "AnswerState"
    },
"AnswerState": {
"Type" : "Choice",
"Choices": [
        {
"Variable": "$.number",
"NumericGreaterThan": 500,
"Next": "GreateThan"
        },
        {
"Variable": "$.number",
"NumericLessThan": 500,
"Next": "LessThan"
        }
      ],
"Default": "EqualTo"
    },
 
"GreateThan": {
"Type" : "Task",
"Resource": "ARN:OF:greater-than-lambda",
"Next": "FinalState"
    },
 
"LessThan": {
"Type" : "Task",
"Resource": "ARN:OF:less-than-lambda",
"Next": "FinalState"
    },
 
"EqualTo": {
"Type": "Fail",
"Cause": "No Matches!"
    },
 
"FinalState": {
"Type": "Task",
"Resource": "ARN:OF:final-state-lambda",
"End": true
    }
  }
}

Step 8: Give a suitable name to the State Machine, under the permission click on Choose an existing role, and add the role which we made in the starting.

Step 9: Finally click on Create State Machine.

Step 10. Execution of Workflow. Click on Start Execution.

2. A new dialog box appears in the screen, in the input section enter the following content. and then finally click on start execution.

{
 "key1" : 100,
 "key2" : 300
}

Once the execution is completed, we can see the visual workflow pane and many other things as depicted in the below images.

Finally, we can check the Email which we provided as an endpoint in SNS Topic Subscription creation, you would have received a random number if the execution is successful.

Benefits of Using Step Functions

Using Step Functions for building serverless workflows offers several benefits:

1. Orchestration and Coordination: Providing a visual and intuitive method for orchestrating and coordinating multiple AWS services, Step Functions allow for the creation of cohesive workflows. With the ability to define various states and transitions, it simplifies the management and maintenance of distributed systems.
2. Scalability and Resilience: High volumes of concurrent workflow executions are easily handled by Step Functions with its automatic scaling. Efficient management of individual states ensures reliable and efficient workflow processing. Additionally, built-in error handling and retries are provided, improving the resilience of your workflows.
3. Serverless Architecture: By utilizing the serverless computing model, Step Functions eliminate the burdensome tasks of infrastructure provisioning and management. With Step Functions, you are solely responsible for paying for the actual usage of both the functions and the services they rely on, resulting in reduced operational expenses and overhead.
4. Fault Tolerance: With Step Functions, you'll find a wonderful fault tolerance feature. For any workflow hiccups, Step Functions will effortlessly manage retries based on your custom retry policy. You're free to set the intervals, backoff tactics, and max attempts for each stage, guaranteeing that fleeting mishaps are expertly resolved. All thanks to this innate fault tolerance mechanic, you can bask in the unwavering reliability and toughness of your workflows, and wave goodbye to concerns of data loss or workflow disturbance.
5. Visibility and Monitoring: You have the option to monitor workflow execution progress, status, and performance using the AWS Management Console, SDKs, or API, thanks to Step Functions. This service provides comprehensive visibility into your workflows' execution.

Real-World Use Cases of AWS Step Functions

The following table show the list of industries using AWS Step Functions:

How Step Functions can be Integrated with Other AWS Services 

Integrating seamlessly with a range of AWS services such as Amazon SNS, AWS Lambda, and Amazon DynamoDB, AWS Step Functions allows you to take full advantage of these services' functionalities within your workflow. Below, you will find a quick summary of how Step Functions integrates with these services:

1. AWS Lambda: As part of its workflow execution, the Step Function has the ability to directly invoke AWS Lambda functions. This integration enables the execution of custom code and the performance of complex computations or business logic within your workflows. By utilizing the Task state, you can pass input data to the Lambda function and receive the output data seamlessly.
2. Amazon SNS: Using the Task state, Step Functions have the capability to send messages to specified SNS topics and trigger SNS notifications. This allows for smooth integration of event-driven workflows with external systems and the activation of actions based on SNS notifications.
3. Amazon DynamoDB: A fully managed NoSQL database service called Amazon DynamoDB can be interacted with by Step Functions. With the Task state, you are able to perform operations like reading, writing, updating, or deleting data from DynamoDB tables within your workflows. It is possible to incorporate database operations into your workflows and develop data-driven applications with this integration.
4. Stepping it up, Step Functions demonstrate their ability to work alongside other AWS services. Integrations with Amazon SQS, AWS Glue, AWS Batch, as well as AWS Step Functions Dataflow, are just a preview of the possibilities. By linking with these services, Step Functions pave the way for the efficient execution of intricate tasks using a combination of serverless workflows and AWS services.

Conclusion

We have successfully implemented Serverless Workflow using AWS Step Functions. Don't forget to delete all the services which you have created to avoid incurring any charges.