Skip to content

Commit

Permalink
Support Platform CLI (#763)
Browse files Browse the repository at this point in the history
A new tool, the Platform CLI[1], has been developed to replace the existing install/update scripts. It currently utilizes the copier[2] library to drive most of the logic. Copier by design really prefers one-repo = one-template, which is not quite how we have `template-infra` setup. So the Platform CLI has special logic to treat this repo as effectively two different templates, "base" and "app":
  - base: all non-app-specific code
  - app: basically anything with `{{app_name}}` in the file path

To support this new tool, a variety of changes are necessary, notable ones being:
- Add a `copier.yml` config file which holds the questions for templating, with only questions relevant to "base" or "app" templates being asked and saved during the relevant install/update phases
  - Most of `project-config` is now configuration on the "base" template
  - Apps can consistently enable CI/CD things by updating `app_has_dev_env_setup` answer, rather than manually finding and un-commenting bits of code
- Add a `.template-infra/` folder to hold the answers files that copier generates in projects
- Templatize app_name to be dynamically generated on install for naming files with `{{app_name}}`
- Update `.github/workflows/ci-infra-service.yml` (and associated `infra/test/infra_test.go`) to become parameterized/setup per-app (in `ci-{{app_name}}-infra-service.yml.jinja`)
- Update `infra/test/infra_test.go`
  - Parameterize app name based on env var (supporting previous item)
  - Use the `/health` endpoint instead of `/` for check since all apps are required to have a health endpoint but not required to respond to a root request
  - Disable verifying TLS certs on the health request to work around issue for apps using `enable_https`.
- Update various docs that used to link to a `infra/app` to now reference `infra/<APP_NAME>` (and normalize most of the docs to use `<APP_NAME>` as the placeholder for this, instead of the mix of things we were using before)
- Move `app/` to `template-only-app/`. Places wanting to use it as their example app for testing can copy `template-only-app/` to the appropriate app-named directory.
- Update `.github/workflows/template-only-ci-infra.yml` to use the Platform CLI to install the template with example app and run tests against that. Necessitates the `template_infra_test.go` changes.
- The Platform CLI has special logic to re-render `infra/networks/main.tf.jinja` with the installed apps
- Update PR Environments message logic to preserve and cleanup info for multiple apps.
- Remove old install/update scripts

[1] https://github.com/navapbc/platform-cli
[2] https://copier.readthedocs.io/en/stable/

Resolves #691
Resolves #557
Resolves #478
Resolves #569
Resolves #647
  • Loading branch information
lorenyu authored Jan 14, 2025
1 parent 641f91a commit cb833d0
Show file tree
Hide file tree
Showing 94 changed files with 716 additions and 475 deletions.
58 changes: 58 additions & 0 deletions .github/actions/first-file/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
name: Return first file that exists
description: Check given list of files in order and return first one that exists.

inputs:
files:
required: true
type: string
description: |
The list of files to check, in the order to check for them.
File names should be properly quoted\escaped and either space or newline
separated.
Either:
```yaml
files: my_file.txt some_other_file.txt
```
Or:
```yaml
files: |-
my_file.txt
some_other_file.txt
```
outputs:
found_file:
description: "Path of first file found."
value: ${{ steps.find-file.outputs.found_file }}

runs:
using: "composite"
steps:
- name: Get file list
id: file-list
shell: bash
run: |
# Get file list
# https://github.com/actions/runner/issues/1877
files=$(printf %s "${{ inputs.files }}" | tr '\n' ' ')
echo "File list: ${files}"
echo "files=${files}" >> "$GITHUB_OUTPUT"
- name: Check file list
id: find-file
shell: bash
run: |
# Check file list
# https://github.com/actions/runner/issues/1877
for f in ${{ steps.file-list.outputs.files }}; do
if [[ -e "${f}" ]]; then
found_file="${f}"
break
fi
done
echo "found_file=${found_file}"
echo "found_file=${found_file}" >> "$GITHUB_OUTPUT"
12 changes: 8 additions & 4 deletions .github/actions/setup-terraform/action.yml
Original file line number Diff line number Diff line change
@@ -1,12 +1,16 @@
name: 'Set up Terraform'
description: 'Set up Terraform with the version stored in the .terraform-version file'
name: "Set up Terraform"
description: "Set up Terraform with the version stored in the .terraform-version file"
inputs:
version-file:
description: "File containing the terraform version to use."
default: ".terraform-version"
runs:
using: "composite"
steps:
- name: Get .terraform-version
- name: Get Terraform version
id: get-terraform-version
run: |
terraform_version="$(cat .terraform-version)"
terraform_version="$(cat ${{ inputs.version-file }})"
echo "Terraform version: ${terraform_version}"
echo "terraform_version=${terraform_version}" >> "$GITHUB_OUTPUT"
shell: bash
Expand Down
20 changes: 10 additions & 10 deletions .github/workflows/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,13 +8,13 @@ The CI/CD for this project uses [reusable Github Actions workflows](https://docs

Each app should have:

- `ci-[app_name]`: must be created; should run linting and testing
- `ci-[app_name]-vulnerability-scans`: calls `vulnerability-scans`
- Based on [ci-app-vulnerability-scans](https://github.com/navapbc/template-infra/blob/main/.github/workflows/ci-app-vulnerability-scans.yml)
- `ci-[app_name]-pr-environment-checks.yml`: calls `pr-environment-checks.yml` to create or update a pull request environment (see [pull request environments](/docs/infra/pull-request-environments.md))
- Based on [ci-app-pr-environment-checks.yml](/.github/workflows/ci-app-pr-environment-checks.yml)
- `ci-[app_name]-pr-environment-destroy.yml`: calls `pr-environment-destroy.yml` to destroy the pull request environment (see [pull request environments](/docs/infra/pull-request-environments.md))
- Based on [ci-app-pr-environment-destroy.yml](https://github.com/navapbc/template-infra/blob/main/.github/workflows/ci-app-pr-environment-destroy.yml)
- `ci-<APP_NAME>`: must be created; should run linting and testing
- `ci-<APP_NAME>-vulnerability-scans`: calls `vulnerability-scans`
- Based on [ci-{{app_name}}-vulnerability-scans](https://github.com/navapbc/template-infra/blob/main/.github/workflows/ci-{{app_name}}-vulnerability-scans.yml.jinja)
- `ci-<APP_NAME>-pr-environment-checks.yml`: calls `pr-environment-checks.yml` to create or update a pull request environment (see [pull request environments](/docs/infra/pull-request-environments.md))
- Based on [ci-{{app_name}}-pr-environment-checks.yml](https://github.com/navapbc/template-infra/blob/main/.github/workflows/ci-{{app_name}}-pr-environment-checks.yml.jinja)
- `ci-<APP_NAME>-pr-environment-destroy.yml`: calls `pr-environment-destroy.yml` to destroy the pull request environment (see [pull request environments](/docs/infra/pull-request-environments.md))
- Based on [ci-{{app_name}}-pr-environment-destroy.yml](https://github.com/navapbc/template-infra/blob/main/.github/workflows/ci-{{app_name}}-pr-environment-destroy.yml.jinja)

### App-agnostic workflows

Expand All @@ -26,8 +26,8 @@ Each app should have:

Each app should have:

- `cd-[app_name]`: deploys an application
- Based on [`cd-app`](https://github.com/navapbc/template-infra/blob/main/.github/workflows/cd-app.yml)
- `cd-<APP_NAME>`: deploys an application
- Based on [`cd-{{app_name}}`](https://github.com/navapbc/template-infra/blob/main/.github/workflows/cd-{{app_name}}.yml.jinja)

The CD workflow uses these reusable workflows:

Expand All @@ -47,4 +47,4 @@ graph TD

## ⛑️ Helper workflows

- [`check-ci-cd-auth`](./check-ci-cd-auth.yml): verifes that the project's Github repo is able to connect to AWS
- [`check-ci-cd-auth`](./check-ci-cd-auth.yml): verifies that the project's Github repo is able to connect to AWS
38 changes: 0 additions & 38 deletions .github/workflows/cd-app.yml

This file was deleted.

54 changes: 54 additions & 0 deletions .github/workflows/cd-{{app_name}}.yml.jinja
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
name: Deploy {{ app_name }}
# Need to set a default value for when the workflow is triggered from a git push
# which bypasses the default configuration for inputs
run-name: Deploy ${{'{{'}}inputs.version || 'main' {{'}}'}} to {{ app_name }} ${{'{{'}} inputs.environment || 'dev' {{'}}'}}

on:
{% if app_has_dev_env_setup %}
push:
branches:
- "main"
paths:
- "{{ app_name }}/**"
- "bin/**"
- "infra/**"
{% else %}
# !! Once you've set up the dev environment and are ready to enable continuous
# deployment, run:
#
# nava-platform infra update --answers-only --data app_has_dev_env_setup=true .
#
# to enable these lines. They are here as comments for context.
#
# push:
# branches:
# - "main"
# paths:
# - "{{ app_name }}/**"
# - "bin/**"
# - "infra/**"
{% endif %}
workflow_dispatch:
inputs:
environment:
description: Environment to deploy to
required: true
default: "dev"
type: choice
options:
- dev
- staging
- prod
version:
required: true
default: "main"
description: Tag or branch or SHA to deploy

jobs:
deploy:
name: Deploy
uses: ./.github/workflows/deploy.yml
with:
app_name: "{{ app_name }}"
environment: ${{'{{'}} inputs.environment || 'dev' {{'}}'}}
version: ${{'{{'}} inputs.version || 'main' {{'}}'}}
22 changes: 0 additions & 22 deletions .github/workflows/ci-app-pr-environment-checks.yml

This file was deleted.

18 changes: 0 additions & 18 deletions .github/workflows/ci-app-pr-environment-destroy.yml

This file was deleted.

26 changes: 0 additions & 26 deletions .github/workflows/ci-app-vulnerability-scans.yml

This file was deleted.

48 changes: 0 additions & 48 deletions .github/workflows/ci-infra-service.yml

This file was deleted.

Loading

0 comments on commit cb833d0

Please sign in to comment.