Terraform CI-CD

This is a demo repository to showcase a possible Terraform CI/CD.

It follows the principle that CI/CD should not commit changes in your branch and you should be aware of errors before merging your Terraform code (CI checks) and fixing yourself.

Dependencies

The CI/CD uses several tools to validate your Terraform code. It's expected that you have them installed locally.

• Terraform: Terraform is used to run common terraform commands, such as terraform init, terraform plan, terraform apply, terraform providers lock, terraform validate.
• Tflint: Tflint check possible errors and misconfigurations (e.g. EC2 instance types that do not exist), enforce best practices and naimng conventions. It uses the tflint-ruleset-aws to enforce AWS best practices. It's configured via the .tflint.hcl file.
• Terraform-docs: Creates automatic markdown documentation for the terraform code and Terraform modules. It's being configured via the .terraform-docs.yaml file.
• Trivy: Finds vulnerabilities, misconfigurations and securities issues in the Terraform code. It's being configured via the trivy.yaml file.

This repository uses Devbox to manage the Development environment so you can install all needed tools just running devbox shell.

Directory Structure

├── backend.tf    # Defines S3-based backend to store Terraform States
├── config        # Account-Region based backend configuration (S3 bucket + DynamoDB table)
├── modules       # Custom Terraform modules that will be used by the root Terraform project
├── vars          # Account-Region based files to declaratively define variables values

This project follows an Account-Region based deployment, i.e. an environment is treated as the combination of AWS Accont and AWS Region (e.g. us-east-1 resources in account 111111111111 is treated as a single environment).

PS: If you DO NOT intend to use terraform modules, you should set recursive.enabled value on file .terraform-docs.yml to false, otherwise the following error will be seen:

::debug working_dir=.
::debug config_file=.terraform-docs.yml
::debug output_mode=inject
::debug output_file=README.md
::debug terraform-docs markdown table --config .terraform-docs.yml --output-mode inject --output-file README.md --output-template <!-- BEGIN_TF_DOCS -->
{{ .Content }}
<!-- END_TF_DOCS --> .
Error: stat modules: no such file or directory

How does this CI/CD work?

There is a custom GitHub Actions (./github/actions/terraform/action.yaml file) action that is used to run terraform plan and apply commands in a given environment, please check the ./github/workflows/main.yaml file to understand how this is being used. A ci Job was also created to validate formating, linting and security issues.

The best way to fully understand the CI/CD is to check its executions. Check the:

• PR to create simple SQS queue to see the CI in action.
• PR to create an EC2 module to see the CI in action.

How to fix problems that appear during CI?

CI failures	Command
terraform fmt	terraform fmt -recursive
Trivy scan	trivy config . --severity CRITICAL,HIGH --quiet && cat trivy-result.txt
terraform-docs	terraform-docs .
terraform providers lock	terraform providers lock -platform=linux_amd64 -platform=darwin_amd64 -platform=darwin_arm64
tflint	tflint --init && tflint --recursive

Pipeline

The deployment pipeline uses GitHub Actions Environment to control deployment to environment (notice that apply jobs uses GitHub Actions Environment). We can also set up rules to deploy only to production after a manual approval.

Requirements

Name	Version
terraform	~> 1.11.0
aws	~> 5.90

Providers

Name	Version
aws	5.90.0

Modules

No modules.

Resources

Name	Type
aws_sqs_queue.terraform_queue	resource

Inputs

Name	Description	Type	Default	Required
aws_region	Region to deploy the resources	string	n/a	yes
queue_name	Queue name to be created	string	n/a	yes

Outputs

No outputs.