A simple workshop to learn how to write, test and deploy AWS Lambda functions using the Rust programming language

Overview

Rust Lambda Workshop

Material to host a workshop on how to build and deploy Rust Lambda functions with AWS SAM and Cargo Lambda.

Intro

  • What is Serverless
  • What is Lambda
  • What is Rust
  • Why Lambda + Rust

Prerequisites

Verify your setup

# Docker
docker version
# (...)

# Rust
cargo --version
# -> cargo 1.76.0 (c84b36747 2024-01-18)

# Zig (for cross-compiling lambda binaries)
zig version
# -> 0.11.0

# AWS
aws --version
# -> aws-cli/2.15.28 Python/3.11.8 Darwin/23.3.0 exe/x86_64 prompt/off

# AWS login
# you might need to run extra commands to get temporary credentials if you use AWS organizations
# details on how to configure your CLI here: https://docs.aws.amazon.com/cli/latest/userguide/getting-started-quickstart.html
aws sts get-caller-identity
# ->
# {
#     "UserId": "AROATBJTMBXWT2ZAVHYOW:luciano",
#     "Account": "208950529517",
#     "Arn": "arn:aws:sts::208950529517:assumed-role/AWSReservedSSO_AdministratorAccess_d0f4d19d5ba1f39f/luciano"
# }

# Cargo Lambda
cargo lambda --version
# -> cargo-lambda 1.1.0 (e918363 2024-02-19Z)

# SAM
sam --version
# -> SAM CLI, version 1.111.0

Scaffolding

cargo lambda new itsalive
  • Not an HTTP function
  • EventBridge Event (eventbridge::EventBridgeEvent)

Code overview

  • Explain the concept of event-driven
  • Explain difference between main and function_handler and the lifecycle of a Lambda function
  • Update handler
use serde_json::Value; // <--

async fn function_handler(event: LambdaEvent<EventBridgeEvent<Value>>) -> Result<(), Error> {
//                                                           -------
    dbg!(&event); // <--
    Ok(())
}

Create example event in events/eventbridge.json:

{
    "version": "0",
    "id": "53dc4d37-cffa-4f76-80c9-8b7d4a4d2eaa",
    "detail-type": "Scheduled Event",
    "source": "aws.events",
    "account": "123456789012",
    "time": "2015-10-08T16:53:06Z",
    "region": "us-east-1",
    "resources": [
        "arn:aws:events:us-east-1:123456789012:rule/my-scheduled-rule"
    ],
    "detail": {}
}

Local testing

cargo lambda watch

in another session

cargo lambda invoke --data-file events/eventbridge.json

Deployment with Cargo Lambda

# build
cargo lambda build --release --arm64
# deploy
cargo lambda deploy
  • login to the web console
  • show the lambda was created
  • invoke it from the console
  • discuss limitations
    • no event! this is not going to be invoked automatically!
      • Show how to setup the event manually
    • We cannot configure the function in other ways (env vars, memory, timeout, etc)
    • it didn’t create a CloudFormation stack, so there are sparse resources that now we have to delete manually:
      • the Lambda
      • a CloudWatch log stream (/aws/lambda/<function-name>) with retention set to Never Expire!
      • an IAM role (cargo-lambda-role-*)

Using SAM

  • explain the concept of Infrastructure as Code and why it is convenient
  • explain what SAM is and how it builds on top of CloudFormation
  • mention that SAM integrates well with cargo-lamda so we get the best of both worlds
  • when you build APIs, SAM allows you to run a local simulation of API gateway, so you can call your APIs locally
  • Create template.yml
AWSTemplateFormatVersion: 2010-09-09
Transform: AWS::Serverless-2016-10-31

# # Global configuration that is applied by default to specific resources
# Globals:

# # define what's configurable in our stack
# Parameters:
  
# # list all our resources (e.g. lambdas, s3 buckets, etc)
# Resources:
  
# # expose properties of the created resources (e.g. the URL of an API Gateway)
# Outputs:
  • Every resources follows this structure:
ResourceName:
  Type: '<A specific reasource type>' # e.g. AWS::Serverless::Function
  Metadata:
     Key1: Value1
     Key2: Value2
     # ...
  Properties:
     # specific properties depending on the Type
  • Add definition for our Lambda:
Resources:
  
  HealthCheckLambda:
    Type: AWS::Serverless::Function
    Metadata:
      BuildMethod: rust-cargolambda
    Properties:
      CodeUri: .
      Handler: bootstrap
      Runtime: provided.al2023
      Architectures:
        - arm64
      Events:
        ScheduledExecution:
          Type: Schedule
          Properties:
            Schedule: rate(30 minutes)
  • validate with:
sam validate --lint
  • build with
sam build --beta-features
  • deploy with
sam deploy --guided
  • --guided is only needed the first time

  • show the web console:

    • CloudFormation stack with all the resources
    • Lambda with configuration (show memory and Timeout

Making changes

  • Let’s change memory and timeout
Resources:
  
  HealthCheckLambda:
    # ...
    Properties:
      # ...
      MemorySize: 256
      Timeout: 70
      # ...
  • To redeploy (one liner)
sam validate --lint && sam build --beta-features && sam deploy
  • Show changes in the web console

Doing something useful

  • Idea: a pingdom-like utility
    • We call an http endpoint every so often and we record
      • response code
      • request time

Step 1. Making HTTP requests with reqwest

cargo add reqwest
async fn function_handler(event: LambdaEvent<EventBridgeEvent<Value>>) -> Result<(), Error> {
    let resp = reqwest::get("https://loige.co").await?;
    let status = resp.status().as_u16();
    let success = resp.status().is_success();
    dbg!(status);
    dbg!(success);

    Ok(())
}
sam build --beta-features

Caution

🔥 -- stderr
thread 'main' panicked at /Users/luciano/.cargo/registry/src/index.crates.io-6f17d22bba15001f/openssl-sys-0.9.101/build/find_normal.rs:190:5:

Could not find directory of OpenSSL installation, and this `-sys` crate cannot
proceed without this knowledge. If OpenSSL is installed and this crate had
trouble finding it,  you can set the `OPENSSL_DIR` environment variable for the
compilation process.

Make sure you also have the development packages of openssl installed.
For example, `libssl-dev` on Ubuntu or `openssl-devel` on Fedora.

If you're in a situation where you think the directory *should* be found
automatically, please open a bug at [https://github.com/sfackler/rust-openssl](https://github.com/sfackler/rust-openssl)
and include information about your system as well as this message.

$HOST = aarch64-apple-darwin
$TARGET = aarch64-unknown-linux-gnu
openssl-sys = 0.9.101
  • Reqwest, by default tries to use the system OpenSSL library and when we cross-compile this can be problematic. A more reliable approach is to avoid to do that and use instead a Rust crate that implements TLS:
# Cargo.toml

# ...
[dependencies]
# ...
reqwest = { version = "0.11.26", default-features = false, features = [
  "rustls-tls",
  "http2"
] }
# ...
  • Explain briefly what Rust crates feature flags are
  • let’s test locally with:
cargo lambda watch # in a terminal
cargo lambda invoke --data-file events/eventbridge.json # in another
  • We should see [src/main.rs:8:5] status = 200

Step 2. measure the duration of the request

// ...
use std::time::Instant;

async fn function_handler(_event: LambdaEvent<EventBridgeEvent<Value>>) -> Result<(), Error> {
    let start = Instant::now(); // <--
    let resp = reqwest::get("https://loige.co").await?;
    let duration = start.elapsed(); // <--

    let status = resp.status().as_u16();
    let success = resp.status().is_success();
    dbg!(status);
    dbg!(success);
    dbg!(duration); // <--

    Ok(())
}

Step 3. Adding a timeout and better error checks

    let client = reqwest::Client::builder()
        .timeout(Duration::from_secs(10))
        .build()?;

    let resp = client
        .get("https://httpstat.us/504?sleep=60000")
        .send()
        .await?;
  • For testing https://httpstat.us/504?sleep=60000

Caution

🔥 cargo lambda invoke --data-file events/eventbridge.json

Error: alloc::boxed::Box<dyn core::error::Error + core::marker::Send + core::marker::Sync>

× error sending request for url (https://httpstat.us/504?sleep=60000): operation timed out

Was this error unexpected?
Open an issue in https://github.com/cargo-lambda/cargo-lambda/issues
  • Our entire execution is failing!
  • We rather want to capture the error and handle it gracefully
 let resp = client
        .get("https://httpstat.us/504?sleep=60000")
        .send()
        .await; // <--- removed "?"
    let duration = start.elapsed();

    match resp {
        Ok(resp) => {
            let status = resp.status().as_u16();
            let success = resp.status().is_success();
            dbg!(status);
            dbg!(success);
            dbg!(duration);
        }
        Err(e) => {
            eprintln!("The request failed: {}", e);
        }
    }
  • Explain the idea of Result type and pattern matching

Step 4: Making the handler “configurable”

  • We want to make the timeout and the URL configurable.
  • One way to do that is to use environment variables
  • For instance:
# template.yml

Resources:
  
  HealthCheckLambda:
    Type: AWS::Serverless::Function
    # ...
    Properties:
      # ...
      Environment:
        Variables:
          URL: 'https://loige.com'
          TIMEOUT: 10
  • Let’s create a struct to hold our config:
struct HandlerConfig {
    url: reqwest::Url,
    client: reqwest::Client,
}
  • explain why we use these types
    • URL for validation
    • client to avoid to recreate a new client per every request. Ideally a client should be created once at init time and reused across invocations.
  • Let’s change the signature of the handler:
async fn function_handler(
    config: &HandlerConfig, // <- now we can receive a reference to the config
    _event: LambdaEvent<EventBridgeEvent<Value>>,
) -> Result<(), Error> {
  // ...
}
  • the body of the function can now be simplified:
let start = Instant::now();
    // Here we use the client from config and we don't need to create one
    let resp = config.client.get(config.url.as_str()).send().await; 
    let duration = start.elapsed();

    match resp {
        Ok(resp) => {
            let status = resp.status().as_u16();
            let success = resp.status().is_success();
            dbg!(status);
            dbg!(success);
            dbg!(duration);
        }
        Err(e) => {
            eprintln!("The request failed: {}", e);
        }
    }

    Ok(())
  • But we need to create this object on init:
#[tokio::main]
async fn main() -> Result<(), Error> {
    tracing::init_default_subscriber();

    // new code
    let url = env::var("URL").expect("URL environment variable is not set");
    let url = reqwest::Url::parse(&url).expect("URL environment variable is not a valid URL");
    let timeout = env::var("TIMEOUT").unwrap_or_else(|_| "60".to_string());
    let timeout = timeout
        .parse::<u64>()
        .expect("TIMEOUT environment variable is not a valid number");
    let client = reqwest::Client::builder()
        .timeout(Duration::from_secs(timeout))
        .build()?;
    let config = &HandlerConfig { url, client };
    // end new code

    // updated to pass the config in every invocation
    run(service_fn(move |event| async move {
        function_handler(config, event).await
    }))
    .await
}
  • testing: now we need to pass the environment variables to our local simulator:
cargo lambda watch --env-vars URL=https://loige.co,TIMEOUT=5 # one terminal
cargo lambda invoke --data-file events/eventbridge.json # another terminal
  • Let’s also deploy and test on AWS!
sam validate --lint && sam build --beta-features && sam deploy
  • Show environment variables
    • Mention these could be moved to a parameter in the stack to make it more configurable (i.e. you can deploy the same stack multiple times to check different URLs)

Step 5. Let’s make it even more useful: store data to DynamoDB

  • Let’s create the table first
    • We will store data like this:
      • Id: ("URL#Timestamp") - String (hash key)
      • Timestamp - String(sort key)
      • Status - Number
      • Duration - Number
      • Error - String
      • Success - Boolean
Resources:
# ...
  HealthChecksTable:
    Type: AWS::DynamoDB::Table
    DeletionPolicy: Delete
    UpdateReplacePolicy: Delete
    Properties:
      BillingMode: PAY_PER_REQUEST
      KeySchema:
        - AttributeName: "Id"
          KeyType: "HASH"
        - AttributeName: "Timestamp"
          KeyType: "RANGE"
      AttributeDefinitions:
        - AttributeName: "Id"
          AttributeType: "S"
        - AttributeName: "Timestamp"
          AttributeType: "S"
  • Mention that deletion and update policies are set to delete only because this is a demo. In real-life you probably want Retain or Snapshot
  • We also need to know the name of the table in our Lambda code and have permissions to write in this table:
Resources:
# ...
  HealthCheckLambda:
    Type: AWS::Serverless::Function
    # ...
    Properties:
      # ...
      Environment:
        Variables:
          # ...
          TABLE_NAME: !Ref HealthChecksTable # <- new
    # ...
      Policies: # <- new
        - DynamoDBWritePolicy:
            TableName: !Ref HealthChecksTable
sam validate --lint && sam build --beta-features && sam deploy
  • Show that the table was created and that the name was replicated as an env var into our lambda + the new policy

Code update

  • Let’s now install the Rust SDK for dynamodb (and the generic aws-config package
cargo add aws-config aws-sdk-dynamodb
  • update config to include table_name:
struct HandlerConfig {
    url: reqwest::Url,
    client: reqwest::Client,
    table_name: String, // <-
}
  • Add parsing of env var in main
let table_name = env::var("TABLE_NAME").expect("TABLE_NAME environment variable is not set");

let config = &HandlerConfig {
    url,
    client,
    table_name, // <- new
};
  • We also need to add a DynamoDB client
struct HandlerConfig {
    url: reqwest::Url,
    client: reqwest::Client,
    table_name: String,
    dynamodb_client: aws_sdk_dynamodb::Client, // <-
}
  • And we need to initialise that in in our main
let region_provider = RegionProviderChain::default_provider();
let config = aws_config::defaults(BehaviorVersion::latest())
    .region(region_provider)
    .load()
    .await;
let dynamodb_client = aws_sdk_dynamodb::Client::new(&config);

let config = &HandlerConfig {
    url,
    client,
    table_name,
    dynamodb_client, // <-
};
  • Since we will need to work with datetime and timestamps, it’s convenient to use a library that makes that easy:
cargo add chrono
  • Finally we can update our lambda handler code to actually store data into dynamodb
async fn function_handler(
    config: &HandlerConfig,
    event: LambdaEvent<EventBridgeEvent<Value>>,
) -> Result<(), Error> {
    let start = Instant::now();
    let resp = config.client.get(config.url.as_str()).send().await;
    let duration = start.elapsed();

    // Added logic to get the current timestamp (either from the event or,
    // if not provided, uses the current timestamp)
    let timestamp = event
        .payload
        .time
        .unwrap_or_else(chrono::Utc::now)
        .format("%+")
        .to_string();
    
    // We start to create the record we want to store in DynamoDb
    let mut item = HashMap::new();
    // We insert the Id and the Timestamp fields
    item.insert(
        "Id".to_string(),
        AttributeValue::S(format!("{}#{}", config.url, timestamp)),
    );
    item.insert("Timestamp".to_string(), AttributeValue::S(timestamp));

    // Updated our match statement to populate the record fields
    // depending if the request failed or if it completed
    // Note: we are now returning success: (always false for request failures, 
    // while it depends on the status code for completed requests)
    let success = match resp {
        Ok(resp) => {
            let status = resp.status().as_u16();
            // In case of success we add the Status and the Duration fields
            item.insert("Status".to_string(), AttributeValue::N(status.to_string()));
            item.insert(
                "Duration".to_string(),
                AttributeValue::N(duration.as_millis().to_string()),
            );
            resp.status().is_success()
        }
        Err(e) => {
            // In case of failure we add the Error field
            item.insert("Error".to_string(), AttributeValue::S(e.to_string()));
            false
        }
    };
    // Finally, we had the Success field
    item.insert("Success".to_string(), AttributeValue::Bool(success));

    // Now we can send the request to DynamoDB
    let insert_result = config
        .dynamodb_client
        .put_item()
        .table_name(config.table_name.as_str())
        .set_item(Some(item))
        .send()
        .await?;

    // And log the result
    tracing::info!("Insert result: {:?}", insert_result);

    Ok(())
}
  • Deploy and test:
sam validate --lint && sam build --beta-features && sam deploy
  • Note for testing: when testing with fake events, be aware you’ll need to change the timestamp in the event manually
  • Note about local testing. It is technically possible do it with a local dynamodb instance, but it’s not trivial and it’s not the focus of this workshop. In general, the more you start to use native AWS services (dynamodb, eventbridge, SQS, etc) the more you’ll need to rely on integration tests and less on local testing.

THE END! 🎉

Tip

If you want to remove all the infrastructure you created, you can run sam delete and it will remove the CloudFormation stack and all the resources it created.

Ideas for further development of this example

  • Make the configuration options stack parameters for more reusability
  • Support multiple URLs (could do this concurrently from one lambda execution!)
  • Set a TTL to the DynamoDB records so you don’t have to retain them forever (e.g. keep the last 3 months of data)
  • Trigger an alarm if the check fails (bonus if your trigger some kind of notification when the site is back online)
  • Observability (logs, metrics, traces, etc)
  • It could take a snapshot of the content of the page and save it to S3
  • Build a nice dashboard like uptime robot
  • Turn this into a SaaS (e.g. you might run this lambda from multiple regions to check the availability of a service across regions)
You might also like...
The classic game Pong, written in lambda calculus, and a thin layer of Rust.

What? The good old game Pong, written in lambda calculus, and a thin layer of Rust. Why? I was bored. No, seriously, why? Everyone keeps saying that l

Examples of how to use Rust with Serverless Framework, Lambda, API Gateway v1 and v2, SQS, GraphQL, etc

Rust Serverless Examples All examples live in their own directories: project: there is nothing here, just a simple cargo new project_name with a custo

Rust Lambda Extension for any Runtime to preload SSM Parameters as  🔐 Secure Environment Variables!
Rust Lambda Extension for any Runtime to preload SSM Parameters as 🔐 Secure Environment Variables!

🛡 Crypteia Rust Lambda Extension for any Runtime to preload SSM Parameters as Secure Environment Variables! Super fast and only performaned once duri

This repo is a sample video search app using AWS services.
This repo is a sample video search app using AWS services.

Video Search This repo is a sample video search app using AWS services. You can check the demo on this link. Features Transcribing Video and generate

A lambda extension to hot reload parameters from SSM Parameter Store, Secrets Manager, DynamoDB, AppConfig

A lambda extension to hot reload parameters from SSM Parameter Store, Secrets Manager, DynamoDB, AppConfig

A high-performance Lambda authorizer for API Gateway that can validate OIDC tokens
A high-performance Lambda authorizer for API Gateway that can validate OIDC tokens

oidc-authorizer A high-performance token-based API Gateway authorizer Lambda that can validate OIDC-issued JWT tokens. 🤌 Use case This project provid

Build and deploy cross platform bioinformatic utilities with Rust.
Build and deploy cross platform bioinformatic utilities with Rust.

The Bioinformatics Toolkit RUST-backed utilities for bioinformatic data processing. Get started The fastest way to get started it to download the appl

Simple fake AWS Cognito User Pool API server for development.

Fakey Cognito 🏡 Homepage Simple fake AWS Cognito API server for development. ✅ Implemented features AdminXxx on User Pools API. Get Started # run wit

Easy switch between AWS Profiles and Regions
Easy switch between AWS Profiles and Regions

AWSP - CLI To Manage your AWS Profiles! AWSP provides an interactive terminal to interact with your AWS Profiles. The aim of this project is to make i

Owner
Luciano Mammino
FullStack ☁️ developer, entrepreneur, fighter, butterfly maker! 📗 Co-author https://nodejsdp.link 💌 maintainer https://fstack.link
Luciano Mammino
cargo-lambda is a Cargo subcommand to help you work with AWS Lambda.

cargo-lambda cargo-lambda is a Cargo subcommand to help you work with AWS Lambda. The new subcommand creates a basic Rust package from a well defined

null 184 Jan 5, 2023
The lambda-chaos-extension allows you to inject faults into Lambda functions without modifying the function code.

Chaos Extension - Seamless, Universal & Lightning-Fast The lambda-chaos-extension allows you to inject faults into Lambda functions without modifying

AWS CLI Tools 5 Aug 2, 2023
📦 🚀 a smooth-talking smuggler of Rust HTTP functions into AWS lambda

lando ?? maintenance mode ahead ?? As of this announcement AWS not officialy supports Rust through this project. As mentioned below this projects goal

Doug Tangren 68 Dec 7, 2021
Aws-sdk-rust - AWS SDK for the Rust Programming Language

The AWS SDK for Rust This repo contains the new AWS SDK for Rust (the SDK) and its public roadmap. Please Note: The SDK is currently released as a dev

Amazon Web Services - Labs 2k Jan 3, 2023
A Rust runtime for AWS Lambda

Rust Runtime for AWS Lambda This package makes it easy to run AWS Lambda Functions written in Rust. This workspace includes multiple crates: lambda-ru

Amazon Web Services - Labs 2.4k Dec 29, 2022
A tool to run web applications on AWS Lambda without changing code.

AWS Lambda Adapter A tool to run web applications on AWS Lambda without changing code. How does it work? AWS Lambda Adapter supports AWS Lambda functi

AWS Samples 321 Jan 2, 2023
Cookiecutter templates for Serverless applications using AWS SAM and the Rust programming language.

Cookiecutter SAM template for Lambda functions in Rust This is a Cookiecutter template to create a serverless application based on the Serverless Appl

AWS Samples 24 Nov 11, 2022
Rs.aws-login - A command line utility to simplify logging into AWS services.

aws-login A command line utility to simplify logging into AWS accounts and services. $ aws-login use ? Please select a profile to use: › ❯ dev-read

Kevin Herrera 11 Oct 30, 2022
Cargo subcommand to easily bootstrap nocode applications. Write nothing; deploy nowhere.

cargo-nocode No code is the best way to write secure and reliable applications. Write nothing; deploy nowhere. cargo-nocode aims to bring the nocode a

Orhun Parmaksız 29 Jul 1, 2023
Serverless setup for activity pub (using lambda+dynamodb) in Rust

Serverless ActivityPub About This is an experiment to have free/cheaper activitypub instances running on AWS (making use of free tiers as much as poss

Conrad Ludgate 3 Dec 30, 2022