-
Notifications
You must be signed in to change notification settings - Fork 10k
Add backend testing documentation #37448
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
SarahFrench
wants to merge
5
commits into
main
Choose a base branch
from
sarah/add-backend-testing-info
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from 4 commits
Commits
Show all changes
5 commits
Select commit
Hold shift + click to select a range
199fa66
Add testing guidance for `gcs`
SarahFrench 9f35ec7
Add testing guidance for `pg`
SarahFrench 823bc7e
Add testing guidance for `kubernetes`
SarahFrench ca74a35
Add details about accessing GCP resources
SarahFrench 8b05b0f
Update README.md
SarahFrench File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,27 @@ | ||
# How to test the `gcs` backend | ||
|
||
## Google Cloud resources needed for testing the backend | ||
|
||
You will need access to a Google Cloud project that has Google Cloud Storage and Cloud Key Management Service (KMS) APIs enabled. You will need sufficient permissions to create and manage resources in those services: | ||
*[IAM roles for Cloud Storage](https://cloud.google.com/storage/docs/access-control/iam-roles). | ||
*[IAM roles for Cloud Key Management Service](https://cloud.google.com/kms/docs/iam). | ||
|
||
For HashiCorp employees, [a temporary GCP project can be created for testing here](https://doormat.hashicorp.services/gcp/project/temp/create). | ||
|
||
No infrastructure needs to be set up within that project before running the tests for the `gcs` backend; tests will provision and delete GCS buckets and KMS keys themselves. However if you want to use service accounts for accessing the project you will need to create those. | ||
|
||
## Set up credentials and access | ||
|
||
These instructions use [application default credentials](https://cloud.google.com/docs/authentication/application-default-credentials) from a Google Account for simplicity. If you want to use a service account instead, see this documentation on how to [create a key file](https://cloud.google.com/iam/docs/keys-create-delete) and reference that file when providing credentials through environment variables. | ||
|
||
1. Run `gcloud auth application-default login` and log in with your Google Account when prompted in the browser. This will create a file at `~/.config/gcloud/application_default_credentials.json` on your machine. | ||
1. Set these environment variables: | ||
* `GOOGLE_CREDENTIALS=~/.config/gcloud/application_default_credentials.json` (Required) - this file is created in the previous step. | ||
* `GOOGLE_PROJECT=<project-id>` (Required) - here, use the project id that you want to be linked to the GCS buckets created in the tests. | ||
* `TF_ACC=1` (Required) - this signals that you're happy for the test to provision real infrastructure. | ||
* `GOOGLE_REGION=<region>` (Required) - This region name is used to set the region of the GCS bucket and the region that customer-managed encryption keys are created in. If in doubt, use "us-central1". | ||
|
||
## Run tests | ||
|
||
Run tests in the `internal/backend/remote-state/gcs` package with the above environment variables set. | ||
If any errors indicate an issue with credentials or permissions, please review how you're providing credentials to the code and whether sufficient permissions are present. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,21 @@ | ||
# How to test the `kubernetes` backend | ||
|
||
## Create a Kubernetes cluster | ||
|
||
1. Install `kind`, e.g [install kind via Homebrew](https://formulae.brew.sh/formula/kind) | ||
1. Provision a new cluster with the command `kind create cluster --name=terraform` | ||
* You can check for the cluster using `kind get clusters` | ||
|
||
## Set up environment variables for testing | ||
|
||
Creating the cluster in the steps above should have created and/or added an entry into the `~/.kube/config` configuration file. | ||
|
||
Create the `KUBE_CONFIG_PATH` environment variable to help the backend locate that file: | ||
|
||
```bash | ||
export KUBE_CONFIG_PATH=~/.kube/config | ||
``` | ||
|
||
## Run the tests! | ||
|
||
The setup above should be sufficient for running the tests. Make sure your kind cluster exists and is running whenever you run the tests. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,52 @@ | ||
# How to test the `pg` backend | ||
|
||
## Create a Postgres resources in a Docker container | ||
|
||
1.Run this command to launch a Docker container using the `postgres` image that can use SSL: | ||
|
||
```bash | ||
docker run \ | ||
--name pg_backend_testing \ | ||
--rm \ | ||
-e POSTGRES_USER=postgres \ | ||
-e POSTGRES_PASSWORD=password \ | ||
-p 5432:5432 \ | ||
postgres:latest \ | ||
-c ssl=on \ | ||
-c ssl_cert_file=/etc/ssl/certs/ssl-cert-snakeoil.pem \ | ||
-c ssl_key_file=/etc/ssl/private/ssl-cert-snakeoil.key | ||
``` | ||
|
||
Note that for testing we use the user `"postgres"`, and this value is reused in command below. | ||
|
||
2. Use `exec` to access a shell inside the Docker container: | ||
|
||
```bash | ||
% docker exec -it $(docker ps -aqf "name=^pg_backend_testing$") bash | ||
``` | ||
|
||
3. Run a command to create a Postgres database called `terraform_backend_pg_test` | ||
|
||
```bash | ||
root@<container-id>:/# createdb -U postgres terraform_backend_pg_test | ||
root@<container-id>:/# exit | ||
``` | ||
|
||
## Set up environment variables needed for tests | ||
|
||
Set the following environment variables: | ||
|
||
``` | ||
DATABASE_URL=postgresql://localhost:5432/terraform_backend_pg_test?sslmode=require | ||
PGUSER=postgres | ||
PGPASSWORD=password | ||
``` | ||
|
||
The `DATABASE_URL` value is a connection string and should not include the username and password. Instead, | ||
the username and password must be supplied by separate environment variables to let some tests override those | ||
values. | ||
|
||
## Run the tests! | ||
|
||
The setup above should be sufficient for running the tests. Each time you want to run the tests you will need to re-launch the container and | ||
create the `terraform_backend_pg_test` database that's expected by the tests. |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.