initial code

Author: saurabh-et-al

.gitignore

@@ -0,0 +1,22 @@
+*.js
+!jest.config.js
+*.d.ts
+node_modules
+
+# CDK asset staging directory
+.cdk.staging
+cdk.out
+tmp/*
+
+**/node_modules
+
+# Java bundles
+target/
+.aws-sam/
+
+# .DS_Store files
+.DS_Store
+**/.DS_Store
+
+
+.idea/

CODE_OF_CONDUCT.md

@@ -1,4 +1,4 @@
 ## Code of Conduct
 This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct).
 For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact
-opensource-codeofconduct@amazon.com with any additional questions or comments.
+opensource-codeofconduct@amazon.com with any additional questions or comments.

CONTRIBUTING.md

@@ -43,17 +43,9 @@ GitHub provides additional document on [forking a repository](https://help.githu
 ## Finding contributions to work on
 Looking at the existing issues is a great way to find something to contribute on. As our projects, by default, use the default GitHub issue labels (enhancement/bug/duplicate/help wanted/invalid/question/wontfix), looking at any 'help wanted' issues is a great place to start.
 
-
-## Code of Conduct
-This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct).
-For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact
-opensource-codeofconduct@amazon.com with any additional questions or comments.
-
-
 ## Security issue notifications
 If you discover a potential security issue in this project we ask that you notify AWS/Amazon Security via our [vulnerability reporting page](http://aws.amazon.com/security/vulnerability-reporting/). Please do **not** create a public github issue.
 
 
 ## Licensing
-
-See the [LICENSE](LICENSE) file for our project's licensing. We will ask you to confirm the licensing of your contribution.
+See the [LICENSE](LICENSE) file for our project's licensing. We will ask you to confirm the licensing of your contribution.

LICENSE

@@ -1,6 +1,6 @@
-MIT No Attribution
+MIT License
 
-Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
+Copyright 2024 Amazon.com, Inc. or its affiliates. All Rights Reserved.
 
 Permission is hereby granted, free of charge, to any person obtaining a copy of
 this software and associated documentation files (the "Software"), to deal in

README.md

@@ -1,17 +1,158 @@
-## My Project
+# AWS Lambda Actions for AWS FIS
 
-TODO: Fill this README out!
+## Introduction
+The AWS Fault Injection Service (FIS) supports AWS Lambda Actions which allows you to inject faults into
+the Lambda execution environments. As part of their resilience plan, Lambda fault actions
+enable you, our AWS customers, to demonstrate that their Lambda-based applications operate
+as expected during periods of AWS impairment, in response to misconfigurations, or in
+response to bad data. 
 
-Be sure to:
+## Prerequisites
+- [CDK Toolkit](https://docs.aws.amazon.com/cdk/v2/guide/getting_started.html) & bootstrapped AWS environment
+- [docker](https://docs.docker.com/engine/install/)
+- [mvn](https://maven.apache.org/run.html)
 
-* Change the title in this README
-* Edit your repository description on GitHub
+> **[!IMPORTANT]**
+> You need to deploy the stack to an AWS Region where [AWS FIS is supported](https://docs.aws.amazon.com/general/latest/gr/fis.html).
 
-## Security
+> **[!IMPORTANT]**
+> This application deploy publicly available endpoints you'll use to interact with the application. Don't forget to cleanup the deployment following steps below to avoid unnecessary costs. 
 
-See [CONTRIBUTING](CONTRIBUTING.md#security-issue-notifications) for more information.
+## Solution Overview
 
-## License
+### Sample Application
 
-This library is licensed under the MIT-0 License. See the LICENSE file.
+This repository contains three (3) serverless applications and FIS experiment templates using FIS Lambda Actions.
+
+Sample applications are simple CRUD APIs that allow you to manage orders.  They are built using Amazon API Gateway, AWS Lambda, and Amazon DynamoDB. It provides simple Create, Read, Update, and Delete (CRUD) functionality to manage orders. 
+
+![Application Architecture](./images/FISLambdaActionsemoEnvronment.png)
+
+The different APIs and its Lambda functions are implemented using three different runtimes: Node.js, Python and Java.
+
+### FIS Templates
+This repository also deploys the FIS experiment templates which carry out the Lambda actions. Separate experiment templates will be deployed for each of the sample applications. There will be two FIS templates per sample application.
+
+1. **Lambda Latency Injection Fault**  - this FIS template utilize the  *[aws:lambda:invocation-add-delay](https://docs.aws.amazon.com/fis/latest/userguide/fis-actions-reference.html#invocation-add-delay)* action. 
+It is configured to inject 2000ms (2 seconds) latency into 100% of the request for 10 minutes.
+
+![FISLambda_Actions](./images/FISLambda_Latency_Actions.png)
+
+2. **Lambda Http Integration Response Fault** -  FIS template utilize the*[aws:lambda:invocation-http-integration-response](https://docs.aws.amazon.com/fis/latest/userguide/fis-actions-reference.html#invocation-http-integration-response)*. 
+
+It is configured to inject error responses of status code 500 for 100% of the request for 10 minutes. It will also prevent the actual execution of the Lambda Function. 
+
+![FISLambda_Actions](./images/FISLambda_Integration_Actions.png)
+
+### Observability Dashboards
+Repository also implements observability dashboards for each of the CRUD applications deployed.
+
+
+## Getting started
+
+The CDK code deploys all necessary resources for the experiments. 
+
+See the official [AWS FIS Actions](https://docs.aws.amazon.com/fis/latest/userguide/fis-actions-reference.html) documentation page for more information.
+
+### Setup
+
+#### Deploy resources
+To deploy resources, please execute the following command in the CLI environment, where you have already logged into the AWS account. 
+
+```
+cd cdk-lambda-chaos
+npm ci
+cdk deploy --all --parameters FisLambdaAPIs:fisLambdaLayerARN='arn:aws:lambda:us-east-1:211125607513:layer:aws-fis-extension-x86_64:9'
+cd ..
+```
+
+Once deployment is completed - you'll be presented with the API endpoints. Please make a note of them. 
+![API Endpoints output](/images/cdk_output.png)
+
+
+#### Install dependencies for load generation
+Please execute the following command in the CLI environment
+
+```
+cd load-generation
+npm install -g artillery@latest
+npm ci
+
+```
+
+#### Configure Variables for the experiment
+<!-- FisLambdaAPIs stack outputs the URL of deployed API which is exposing the AWS Lambda functions that you want to run the chaos experiments on. The screenshot below shows the output from FisLambdaAPIs stack. You can take the URL as highlighted in the screenshot as an example:
+![Retrieving API URL from 'Outputs' of FisLambdaAPIs Stack](./images/fisapistack-output-app-url.png) -->
+
+As the next step, you need to configure environment variable API_URL with the API endpoint of the CRUD application that you want to run the chaos experiment for.
+
+Use the API endpoint you've captured before and remove the trailing "/" from the url. Then set this as an environment variable as shown below. 
 
+```
+export API_URL='https://<api_id>.execute-api.<region>.amazonaws.com/prod'
+```
+
+This would be used by the Artillery load generation script.
+
+**NOTE:** *You just have to set the base path without leading '/' as '/' is added within the load generation configuration.*
+
+### Run the experiment
+
+#### Generate load
+**Note!** *Make sure to you have set the API_URL environment variable as shown above.*
+
+Before you run the chaos engineering experiments using FIS, ensure that your application has reached steady state. See [Basic principles and guidelines](https://docs.aws.amazon.com/fis/latest/userguide/getting-started-planning.html#planning-basic-principles) to learn more on this. In this example, you can run load generation using following command to achieve steady state:
+
+```
+artillery run load-generation-config.yml
+```
+
+By default, this sends 4 requests per second for 10 minutes. You can change the during as well as number of requests for load generation in ``load-generation-config.yml`` file. See [Artillery documentation](https://www.artillery.io/docs/get-started/first-test#define-load-phases) for more details.
+
+Go to the [AWS CloudWatch Console](https://console.aws.amazon.com/cloudwatch/) where you'll find three dashboards.
+![CloudWatchDashboard](./images/CloudWatchDashboard.png)
+
+Select the dashboard for the the application you are testing and observe the steady state. 
+
+
+#### Start the experiment
+Go to the [AWS Resilience HUB Console](https://console.aws.amazon.com/resiliencehub/home). Navigate to the **Resilience testing**  and select **Experiment templates**. You'll see the list of the FIS Experiment Templates deployed for you.
+
+![FIS_List_of_experments](./images/FIS_List_of_experments.png)
+
+Select FIS experiment template for the experiment and integration you'd like execute and press **Start Experiment**  again press on **Start Experiment** button on the next screen to run the experiments.
+
+![FIS_List_of_experments_start](./images/FIS_List_of_experments_start.png)
+
+
+#### Monitor results
+
+Once you have run the load generation script and started the experiment, you can observe your application's behavior by looking at different metrics included in the [CloudWatch Dashboards](https://console.aws.amazon.com/cloudwatch/). To get a holistic view of your application behavior, this dashboard includes 
+* metrics from Artillery load generation including number of requests and corresponding responses received, latency of response, any errors recorded by load generation script. This represents the user's perspective of your application.
+* corresponding metrics from API Gateway and Lambda Function that comprise the API we are invoking from load generation script and,
+* metrics from FIS extension showing the number of faults injected by the experiment.
+
+Below is an example of running the 'Lambda Latency Injection Fault' experiment template.
+
+![](./images/CloudWatchDashboard-TypeScript-AddDelay-Experiment.png)
+
+The Dashboard is organized in four rows three columns:
+* First row represents the requests from load generation, corresponding API Gateway request count and corresponding Lambda invocations. They should represent and same count for steady state.
+* Second row shows response times recorded by load generation, latency from API Gateway and Lambda duration. You should see a correlation among these metrics when you run *aws:lambda:invocation-add-delay* experiment.
+* Third row shows API errors recorded by the load generation script, errors reported by API Gateway and errors in Lambda function invocations. If you run *aws:lambda:invocation-http-integration-response* experiment, you should see  correlation in metrics in this row.
+* The last row represents *FaultInjected* metric. Based on the experiment you are running, you should see a correlation with either "FIS FaultInjected Invocation Delay" or "FIS FaultInjected HTTP Integration Response" graph.
+
+NOTE: This dashboard includes only *FaultInjected* metric. You can explore [all EMF metrics emitted by AWS FIS Extension](https://docs.aws.amazon.com/fis/latest/userguide/use-lambda-actions.html) in *aws-fis-extension* Custom namespace in CloudWatch.
+
+
+### Cleanup
+Run the following command from the `cdk-lambda-chaos` folder.
+```
+cdk destroy --all
+```
+
+## Contributing
+See [CONTRIBUTING](CONTRIBUTING.md) for more information.
+
+## License
+This library is licensed under the MIT-0 License. See the LICENSE file.

cdk-lambda-chaos/.npmignore

@@ -0,0 +1,6 @@
+*.ts
+!*.d.ts
+
+# CDK asset staging directory
+.cdk.staging
+cdk.out

cdk-lambda-chaos/bin/fis-lambda-chaos.ts

@@ -0,0 +1,51 @@
+#!/usr/bin/env node
+import 'source-map-support/register';
+import * as cdk from 'aws-cdk-lib';
+import { Aspects, CfnParameter } from 'aws-cdk-lib';
+import { FisExperimentsStack } from '../lib/fis-experiments-stack';
+import { ServerlessApiStack } from '../lib/serverless-api-stack';
+import { TargetTag } from '../lib/shared/fis-lambda-chaos-shared';
+import { AwsSolutionsChecks } from 'cdk-nag';
+
+const stackName = "FisLambda";
+const app = new cdk.App();
+// Aspects.of(app).add(new AwsSolutionsChecks({verbose: true}))
+
+let apiResourceName = 'Orders';
+// TODO get lambda arn from SSM
+let fisLambdaLayerARN = 'arn:aws:lambda:us-east-1:211125607513:layer:aws-fis-extension-x86_64:9';
+let fisLambdaTagName = 'FISExperimentReady'
+let fisLambdaTagValue = 'Yes'
+let fisResourceTag: TargetTag = {
+    TagName: fisLambdaTagName,
+    TagValue: fisLambdaTagValue
+}
+let fisExperimentDuration = 10 // Duration of the Fault. Measured in minutes 
+let fisInvocationPercentage = 100 // Percentage of the Invocation impacted by the Fault. Measured in %
+let fisStartupLatencyMilliseconds = 2000 // Latency startup delay for the latency injection experiment in milliseconds
+let fisHTTPIntegrationResponseCode = 500 // HTTP response code for the  experiment with response code faults
+// Proxy response structure: https://docs.aws.amazon.com/apigateway/latest/developerguide/http-api-develop-integrations-lambda.html 
+
+const fisExperimentStack = new FisExperimentsStack(app, `${stackName}Experiments`, {
+    env: {
+        account: process.env.CDK_DEFAULT_ACCOUNT,
+        region: process.env.AWS_REGION ?? process.env.CDK_DEFAULT_REGION,
+    },
+    ResourceTag: fisResourceTag, // This is only partially in use. Experiment template ignores tag name which is set already
+    InvocationPercentage: fisInvocationPercentage,
+    DurationMinutes: fisExperimentDuration,
+    StartupLatencyMilliseconds: fisStartupLatencyMilliseconds,
+    InvocationStatusCode: fisHTTPIntegrationResponseCode,
+    StackName: stackName,
+});
+
+const lambdaStack = new ServerlessApiStack(app, `${stackName}APIs`, {
+    env: {
+        account: process.env.CDK_DEFAULT_ACCOUNT,
+        region: process.env.AWS_REGION ?? process.env.CDK_DEFAULT_REGION,
+    },
+    StackName: stackName,
+    FisLambdaLayerARN: fisLambdaLayerARN,
+    ResourceTag: fisResourceTag,
+    ApiResourceName: apiResourceName
+});

cdk-lambda-chaos/cdk.json

@@ -0,0 +1,73 @@
+{
+  "app": "npx ts-node --prefer-ts-exts bin/fis-lambda-chaos.ts",
+  "watch": {
+    "include": [
+      "**"
+    ],
+    "exclude": [
+      "README.md",
+      "cdk*.json",
+      "**/*.d.ts",
+      "**/*.js",
+      "tsconfig.json",
+      "package*.json",
+      "yarn.lock",
+      "node_modules",
+      "test"
+    ]
+  },
+  "context": {
+    "@aws-cdk/aws-lambda:recognizeLayerVersion": true,
+    "@aws-cdk/core:checkSecretUsage": true,
+    "@aws-cdk/core:target-partitions": [
+      "aws",
+      "aws-cn"
+    ],
+    "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true,
+    "@aws-cdk/aws-ec2:uniqueImdsv2TemplateName": true,
+    "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true,
+    "@aws-cdk/aws-iam:minimizePolicies": true,
+    "@aws-cdk/core:validateSnapshotRemovalPolicy": true,
+    "@aws-cdk/aws-codepipeline:crossAccountKeyAliasStackSafeResourceName": true,
+    "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true,
+    "@aws-cdk/aws-sns-subscriptions:restrictSqsDescryption": true,
+    "@aws-cdk/aws-apigateway:disableCloudWatchRole": true,
+    "@aws-cdk/core:enablePartitionLiterals": true,
+    "@aws-cdk/aws-events:eventsTargetQueueSameAccount": true,
+    "@aws-cdk/aws-iam:standardizedServicePrincipals": true,
+    "@aws-cdk/aws-ecs:disableExplicitDeploymentControllerForCircuitBreaker": true,
+    "@aws-cdk/aws-iam:importedRoleStackSafeDefaultPolicyName": true,
+    "@aws-cdk/aws-s3:serverAccessLogsUseBucketPolicy": true,
+    "@aws-cdk/aws-route53-patters:useCertificate": true,
+    "@aws-cdk/customresources:installLatestAwsSdkDefault": false,
+    "@aws-cdk/aws-rds:databaseProxyUniqueResourceName": true,
+    "@aws-cdk/aws-codedeploy:removeAlarmsFromDeploymentGroup": true,
+    "@aws-cdk/aws-apigateway:authorizerChangeDeploymentLogicalId": true,
+    "@aws-cdk/aws-ec2:launchTemplateDefaultUserData": true,
+    "@aws-cdk/aws-secretsmanager:useAttachedSecretResourcePolicyForSecretTargetAttachments": true,
+    "@aws-cdk/aws-redshift:columnId": true,
+    "@aws-cdk/aws-stepfunctions-tasks:enableEmrServicePolicyV2": true,
+    "@aws-cdk/aws-ec2:restrictDefaultSecurityGroup": true,
+    "@aws-cdk/aws-apigateway:requestValidatorUniqueId": true,
+    "@aws-cdk/aws-kms:aliasNameRef": true,
+    "@aws-cdk/aws-autoscaling:generateLaunchTemplateInsteadOfLaunchConfig": true,
+    "@aws-cdk/core:includePrefixInUniqueNameGeneration": true,
+    "@aws-cdk/aws-efs:denyAnonymousAccess": true,
+    "@aws-cdk/aws-opensearchservice:enableOpensearchMultiAzWithStandby": true,
+    "@aws-cdk/aws-lambda-nodejs:useLatestRuntimeVersion": true,
+    "@aws-cdk/aws-efs:mountTargetOrderInsensitiveLogicalId": true,
+    "@aws-cdk/aws-rds:auroraClusterChangeScopeOfInstanceParameterGroupWithEachParameters": true,
+    "@aws-cdk/aws-appsync:useArnForSourceApiAssociationIdentifier": true,
+    "@aws-cdk/aws-rds:preventRenderingDeprecatedCredentials": true,
+    "@aws-cdk/aws-codepipeline-actions:useNewDefaultBranchForCodeCommitSource": true,
+    "@aws-cdk/aws-cloudwatch-actions:changeLambdaPermissionLogicalIdForLambdaAction": true,
+    "@aws-cdk/aws-codepipeline:crossAccountKeysDefaultValueToFalse": true,
+    "@aws-cdk/aws-codepipeline:defaultPipelineTypeToV2": true,
+    "@aws-cdk/aws-kms:reduceCrossAccountRegionPolicyScope": true,
+    "@aws-cdk/aws-eks:nodegroupNameAttribute": true,
+    "@aws-cdk/aws-ec2:ebsDefaultGp3Volume": true,
+    "@aws-cdk/aws-ecs:removeDefaultDeploymentAlarm": true,
+    "@aws-cdk/custom-resources:logApiResponseDataPropertyTrueDefault": false,
+    "@aws-cdk/aws-stepfunctions-tasks:ecsReduceRunTaskPermissions": true
+  }
+}

cdk-lambda-chaos/jest.config.js

@@ -0,0 +1,8 @@
+module.exports = {
+  testEnvironment: 'node',
+  roots: ['<rootDir>/test'],
+  testMatch: ['**/*.test.ts'],
+  transform: {
+    '^.+\\.tsx?$': 'ts-jest'
+  }
+};