Skip to content

Commit 491e385

Browse files
tomowattsulewicz-gq
tomowatt
and
sulewicz-gq
authored
SUP-4358: Refactor Functions (#30)
* feat: produce additional output file for non-human output formats * feat: update parameters and format * docs: update README to reflect parameter changes * feat: reflect paramter changes in hook * test: reflect parameter changes in tests * fix: add set -e to ensure hook doesn't silently fail * chore: remove version, pin images and remove Read Only * feat/fix: add shared lib for handling reading env vars correctly for strings or arrays parameters * Revert "fix: add set -e to ensure hook doesn't silently fail" This reverts commit f0bc975. * fix/test: correct Env Var used to read and add test * fix: make shared.bash executable for shellcheck * fix: update pipeline for shellcheck shared.bash * chore: re-add read-only for volume mounts as read-write not required * feat: add check for duplicate file formats * set a default output that will be used for the Build Annotations so that can be refactored separately * added notes for future changes required for loop to support multiple output files with the same file format per Wiz CLI docs Signed-off-by: Tom Watt <tom@buildkite.com> * tests: add tests for duplicate file output formats Signed-off-by: Tom Watt <tom@buildkite.com> * docs/chore: file-output-format is optional with no default Signed-off-by: Tom Watt <tom@buildkite.com> * refactor: move setupWiz to separate file to run unit-tests * feat: pass env vars to container instead of using flags to remove for job logs * refactor: remove api-secret-env parameter in favour of using WIZ_CLIENT_SECRET Signed-off-by: Tom Watt <tom@buildkite.com> * feat: add check for Wiz Client Credentials Signed-off-by: Tom Watt <tom@buildkite.com> * tests: reflect usage of WIZ_CLIENT_* Credentials * refactor: add teardown to cleanup dirs/files for authentication tests Signed-off-by: Tom Watt <tom@buildkite.com> * docs: Update README to reflect usage of WIZ_CLIENT_* credentials * refactor: move buildAnnotation to separate file to be able to test separately from post-command script * refactor: move *Scan to separate file to be able to test separately from post-command script * refactor: create func for determining Wiz CLI container image * chore: update plugin versions * tests: check Wiz CLI Image used on different architectures * refactor: create func to get CLI arguments that are passable to other funcs Signed-off-by: Tom Watt <tom@buildkite.com> * refactor/tests: use get_wiz_cli_args func for tests to check expected outcomes Signed-off-by: Tom Watt <tom@buildkite.com> * feat: use arguments in existing funcs to accomodate changes and ensure Signed-off-by: Tom Watt <tom@buildkite.com> * fix: correct shellcheck Signed-off-by: Tom Watt <tom@buildkite.com> * fix: correct shellcheck (again) Signed-off-by: Tom Watt <tom@buildkite.com> * fix: correct word splitting of CLI args Signed-off-by: Tom Watt <tom@buildkite.com> * fix: lib dir pathing * revert: move sourcing to hook for ease and update tests for func usage Signed-off-by: Tom Watt <tom@buildkite.com> --------- Signed-off-by: Tom Watt <tom@buildkite.com> Co-authored-by: Shimon Ulewicz <sulewicz@groq.com>
1 parent 75e8162 commit 491e385

File tree

4 files changed

+312
-223
lines changed

4 files changed

+312
-223
lines changed

‎.buildkite/pipeline.yml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -6,13 +6,13 @@ steps:
66

77
- label: ":shell: Shellcheck"
88
plugins:
9-
- shellcheck#v1.3.0:
9+
- shellcheck#v1.4.0:
1010
files:
1111
- hooks/**
1212
- lib/**
1313

1414
- label: ":shell: Tests"
1515
plugins:
16-
- plugin-tester#v1.1.1:
16+
- plugin-tester#v1.2.0:
1717
folders:
1818
- tests

‎hooks/post-command

Lines changed: 13 additions & 207 deletions
Original file line numberDiff line numberDiff line change
@@ -4,19 +4,15 @@ set -uo pipefail
44

55
DIR="$(cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd)"
66

7-
# shellcheck source=lib/shared.bash
8-
. "$DIR/../lib/shared.bash"
9-
107
# shellcheck source=lib/plugin.bash
118
. "$DIR/../lib/plugin.bash"
129

10+
# shellcheck source=lib/shared.bash
11+
. "$DIR/../lib/shared.bash"
12+
1313
WIZ_DIR="$HOME/.wiz"
1414
SCAN_TYPE="${BUILDKITE_PLUGIN_WIZ_SCAN_TYPE:-}"
1515
FILE_PATH="${BUILDKITE_PLUGIN_WIZ_PATH:-}"
16-
PARAMETER_FILES="${BUILDKITE_PLUGIN_WIZ_PARAMETER_FILES:-}"
17-
IAC_TYPE="${BUILDKITE_PLUGIN_WIZ_IAC_TYPE:-}"
18-
SCAN_FORMAT="${BUILDKITE_PLUGIN_WIZ_SCAN_FORMAT:=human}"
19-
SHOW_SECRET_SNIPPETS="${BUILDKITE_PLUGIN_WIZ_SHOW_SECRET_SNIPPETS:=false}"
2016

2117
if [[ -z "${SCAN_TYPE}" ]]; then
2218
echo "+++ 🚨 Missing scan type. Possible values: 'iac', 'docker', 'dir'"
@@ -28,212 +24,22 @@ if [[ "${SCAN_TYPE}" == "docker" && -z "${BUILDKITE_PLUGIN_WIZ_IMAGE_ADDRESS:-}"
2824
exit 1
2925
fi
3026

31-
##
32-
# Wiz CLI Parameters
33-
##
34-
35-
args=()
36-
37-
## Global Parameters
38-
39-
if [[ "${SHOW_SECRET_SNIPPETS}" == "true" ]]; then
40-
args+=("--show-secret-snippets")
41-
fi
42-
43-
scan_formats=("human" "json" "sarif")
44-
if [[ ${scan_formats[*]} =~ ${SCAN_FORMAT} ]]; then
45-
args+=("--format=${SCAN_FORMAT}")
46-
else
47-
echo "+++ 🚨 Invalid Scan Format: ${SCAN_FORMAT}"
48-
echo "Valid Formats: ${scan_formats[*]}"
49-
exit 1
50-
fi
51-
52-
# Define valid formats
53-
valid_file_formats=("human" "json" "sarif" "csv-zip")
54-
55-
# Default file output which is used for build annotation
56-
args+=("--output=/scan/result/output,human")
57-
58-
# Declare result array
59-
declare -a result
60-
61-
# Read file output formats into result array
62-
if plugin_read_list_into_result "BUILDKITE_PLUGIN_WIZ_FILE_OUTPUT_FORMAT"; then
63-
declare -A seen_formats
64-
for format in "${result[@]}"; do
65-
# Multiple output files with the same format are supported
66-
# but would need to rework this loop to handle and validate i.e., specifying file names, etc.,
67-
# -o, --output file-outputs Output to file, can be passed multiple times to output to multiple files with possibly different formats.
68-
# Must be specified in the following format: file-path[,file-format[,policy-hits-only[,group-by[,include-audit-policy-hits]]]]
69-
# Options for file-format: [csv-zip, human, json, sarif], policy-hits-only: [true, false], group-by: [default, layer, resource], include-audit-policy-hits: [true, false]
70-
# Check for duplicates
71-
if [[ -n "${seen_formats[$format]:-}" ]]; then
72-
echo "+++ ⚠️ Duplicate file output format ignored: ${format}"
73-
continue
74-
fi
75-
seen_formats["$format"]=1
76-
77-
# Check for invalid formats
78-
if in_array "$format" "${valid_file_formats[@]}"; then
79-
args+=("--output=/scan/result/output-${format},${format}")
80-
else
81-
echo "+++ 🚨 Invalid File Output Format: ${format}"
82-
echo "Valid Formats: ${valid_file_formats[*]}"
83-
exit 1
84-
fi
85-
done
86-
fi
87-
88-
## IAC Scanning Parameters
89-
90-
if [[ "${SCAN_TYPE}" == "iac" ]]; then
91-
92-
if [[ -n "${IAC_TYPE}" ]]; then
93-
args+=("--types=${IAC_TYPE}")
94-
fi
95-
96-
if [[ -n "${PARAMETER_FILES}" ]]; then
97-
args+=("--parameter-files=${PARAMETER_FILES}")
98-
fi
99-
fi
100-
101-
# Get the architecture of the machine for running the container image due to "latest" not being multi-architecture
102-
# Available images: `latest`, `latest-amd64` and `latest-arm64`
103-
# therefore default case will use `latest`
104-
105-
architecture=$(uname -m)
106-
container_image_tag="latest"
107-
108-
case $architecture in
109-
x86_64)
110-
container_image_tag+="-amd64"
111-
;;
112-
arm64 | aarch64)
113-
container_image_tag+="-arm64"
114-
;;
115-
*) ;;
116-
esac
117-
118-
wiz_cli_container_repository="wiziocli.azurecr.io/wizcli"
119-
wiz_cli_container="${wiz_cli_container_repository}:${container_image_tag}"
120-
121-
#$1 type, $2 name, $3 pass/fail, $4 result file
122-
buildAnnotation() {
123-
annotation_file=${RANDOM:0:2}-annotation.md
124-
docker_or_iac=$(if [ "$1" = "docker" ]; then echo "Wiz Docker Image Scan"; else echo "Wiz IaC Scan"; fi)
125-
pass_or_fail=$(if [ "$3" = true ]; then echo 'meets'; else echo 'does not meet'; fi)
126-
summary="${docker_or_iac} for ${2} ${pass_or_fail} policy requirements"
127-
# we need to create a new file to avoid conflicts, we need scan type, name, pass/fail
128-
cat <<EOF >>./"${annotation_file}"
129-
<details>
130-
<summary>$summary.</summary>
131-
132-
\`\`\`term
133-
$(cat "$4")
134-
\`\`\`
135-
136-
</details>
137-
EOF
138-
printf "%b\n" "$(cat ./"${annotation_file}")"
139-
}
140-
141-
dockerImageScan() {
142-
mkdir -p result
143-
# TODO check feasibility of mount/mountWithLayers
144-
IMAGE="${BUILDKITE_PLUGIN_WIZ_IMAGE_ADDRESS:-}"
145-
# make sure local docker has the image
146-
docker pull "$IMAGE"
147-
docker run \
148-
--rm -it \
149-
--mount type=bind,src="$WIZ_DIR",dst=/cli,readonly \
150-
--mount type=bind,src="$PWD",dst=/scan \
151-
--mount type=bind,src=/var/run/docker.sock,dst=/var/run/docker.sock,readonly \
152-
"${wiz_cli_container}" \
153-
docker scan --image "$IMAGE" \
154-
--policy-hits-only \
155-
${args:+"${args[@]}"}
156-
157-
exit_code="$?"
158-
image_name=$(echo "$IMAGE" | cut -d "/" -f 2)
159-
# FIXME: Linktree Specific Env. Var.
160-
# buildkite-agent artifact upload result --log-level info
161-
case $exit_code in
162-
0)
163-
buildAnnotation "docker" "$image_name" true "result/output" | buildkite-agent annotate --append --style 'success' --context 'ctx-wiz-docker-success'
164-
;;
165-
*)
166-
buildAnnotation "docker" "$image_name" false "result/output" | buildkite-agent annotate --append --context 'ctx-wiz-docker-warning' --style 'warning'
167-
;;
168-
esac
169-
exit $exit_code
170-
}
171-
172-
iacScan() {
173-
mkdir -p result
174-
docker run \
175-
--rm -it \
176-
--mount type=bind,src="$WIZ_DIR",dst=/cli,readonly \
177-
--mount type=bind,src="$PWD",dst=/scan \
178-
"${wiz_cli_container}" \
179-
iac scan \
180-
--name "$BUILDKITE_JOB_ID" \
181-
--path "/scan/$FILE_PATH" ${args:+"${args[@]}"}
182-
183-
exit_code="$?"
184-
case $exit_code in
185-
0)
186-
buildAnnotation "iac" "$BUILDKITE_LABEL" true "result/output" | buildkite-agent annotate --append --context 'ctx-wiz-iac-success' --style 'success'
187-
;;
188-
*)
189-
buildAnnotation "iac" "$BUILDKITE_LABEL" false "result/output" | buildkite-agent annotate --append --context 'ctx-wiz-iac-warning' --style 'warning'
190-
;;
191-
esac
192-
# buildkite-agent artifact upload "result/**/*" --log-level info
193-
# this post step will be used in template to check the step was run
194-
echo "${BUILDKITE_BUILD_ID}" >check-file && buildkite-agent artifact upload check-file
195-
196-
exit $exit_code
197-
}
198-
199-
dirScan() {
200-
mkdir -p result
201-
docker run \
202-
--rm -it \
203-
--mount type=bind,src="$WIZ_DIR",dst=/cli,readonly \
204-
--mount type=bind,src="$PWD",dst=/scan \
205-
"${wiz_cli_container}" \
206-
dir scan \
207-
--name "$BUILDKITE_JOB_ID" \
208-
--path "/scan/$FILE_PATH" ${args:+"${args[@]}"}
209-
210-
exit_code="$?"
211-
case $exit_code in
212-
0)
213-
buildAnnotation "dir" "$BUILDKITE_LABEL" true "result/output" | buildkite-agent annotate --append --context 'ctx-wiz-dir-success' --style 'success'
214-
;;
215-
*)
216-
buildAnnotation "dir" "$BUILDKITE_LABEL" false "result/output" | buildkite-agent annotate --append --context 'ctx-wiz-dir-warning' --style 'warning'
217-
;;
218-
esac
219-
# buildkite-agent artifact upload "result/**/*" --log-level info
220-
# this post step will be used in template to check the step was run
221-
echo "${BUILDKITE_BUILD_ID}" >check-file && buildkite-agent artifact upload check-file
222-
223-
exit $exit_code
224-
}
27+
declare -a cli_args
28+
# shellcheck disable=SC2207
29+
cli_args=($(get_wiz_cli_args "${SCAN_TYPE}"))
30+
wiz_cli_container_image=$(get_wiz_cli_container)
22531

22632
case "${SCAN_TYPE}" in
22733
iac)
228-
setupWiz "$wiz_cli_container" "$WIZ_DIR"
229-
iacScan
34+
setupWiz "$wiz_cli_container_image" "$WIZ_DIR"
35+
iacScan "$wiz_cli_container_image" "$WIZ_DIR" "${FILE_PATH}" "${cli_args[@]}"
23036
;;
23137
docker)
232-
setupWiz "$wiz_cli_container" "$WIZ_DIR"
233-
dockerImageScan
38+
setupWiz "$wiz_cli_container_image" "$WIZ_DIR"
39+
dockerImageScan "$wiz_cli_container_image" "$WIZ_DIR" "${BUILDKITE_PLUGIN_WIZ_IMAGE_ADDRESS}" "${cli_args[@]}"
23440
;;
23541
dir)
236-
setupWiz "$wiz_cli_container" "$WIZ_DIR"
237-
dirScan
42+
setupWiz "$wiz_cli_container_image" "$WIZ_DIR"
43+
dirScan "$wiz_cli_container_image" "$WIZ_DIR" "${FILE_PATH}" "${cli_args[@]}"
23844
;;
23945
esac

0 commit comments

Comments
 (0)