Skip to content

Commit

Permalink
API CI and CD Workflow Pipelines (#19)
Browse files Browse the repository at this point in the history
* - Added api Dockerfile
- Update API CICD
- Added build-api

* Fixed indention in build-api

* Added shell bash and lint

* Ignore existing errors

* Add API workflow action

* - Updated wf to pass .NET version to use when building docker image
- Moved Dockerfile to /docker/api for consistency

* Fixed assigning of SHORT_SHA

* - Fixed incorrect Dockerfile path
- Use lowercase arg

* - Try major.minor version
- Fixed SHORT_SHA

* Fixed SHORT_SHA

* Pass minor version=0

* Added jc-interface-client

* Fixed short_sha

* Finalize changes to api ci/cd

* Added deploy2test and deploy2prod stages

* - Pass tier name to deploy-to-aws action
- Update aws deployment to trigger ECS Task Definition when a new image is deployed

* Moved env vars to output vars

* Added missing shell: bash

* Try manual docker commands to retag and push since we the image is already built.

* Append sha to tag an image

* Pass SHORT_SHA to use for retagging

* REname to main to be able to test feature branch

* - Use build-push-action rather than docker commands
- Update gh repo url to make concat easier to understand

* Fixed Docker file path

* Removed target

* Include short_sha to pull docker image in ghcr

* Pull image via tag instead of digest

* Changed docker cli commands to GH actions

* Cahnged web runtime's context

* Update image and tags props

* set push to true

* Removed metadata acton

* Try different tag

* add debug log

* fixed docker command

* add more docker image ls

* Add load=true

* Removed :latest

* Try pushing the image

* Changed driver=docker

* Changed context for build-push-action step

* - Renamed main to Deploy API
- Re-add push:true to deploy to web package to GCHR

* - Added metadata gh action
- Added image names env variables

* Fixed incorrect arg casing

* - Create a separate Dockerfile to combine web images into one
- Updated web pipeline so that build-push-action is only called once

* - Removed commented code
- Changed context to .

* Fixed paths for the pipeline

* - Pass NODE_VERSION
- Removed other commands that may not be necessary for the pipeline

* Fixed permissions to use root user

* Copy run script from nginx-runtime as it appears to be missing as per the Cloudwatch error.

* Pass folder path to make Dockerfile.release reusable by ./manage script and publish-web pipeline

* Fixed missing dir folder

---------

Co-authored-by: Ronaldo Macapobre <[email protected]>
  • Loading branch information
ronaldo-macapobre and Ronaldo Macapobre authored Sep 6, 2024
1 parent 7fbb9d4 commit 5bbe0b4
Show file tree
Hide file tree
Showing 9 changed files with 458 additions and 90 deletions.
36 changes: 36 additions & 0 deletions .github/workflows/actions/build-api/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
name: Build API
description: Builds the API codebase

inputs:
working_directory:
description: The working directory where the code will be built.
required: true
dotnet_version:
description: The .NET version that will be used.
required: true

runs:
using: composite

steps:
- name: Setup .NET Core
uses: actions/setup-dotnet@v4
with:
dotnet-version: ${{ inputs.dotnet_version }}

- run: dotnet format --verify-no-changes --severity info
shell: bash
working-directory: ${{ inputs.working_directory }}
continue-on-error: true

- run: dotnet restore
shell: bash
working-directory: ${{ inputs.working_directory }}

- run: dotnet build --configuration Release --no-restore
shell: bash
working-directory: ${{ inputs.working_directory }}

- run: dotnet test --no-restore --verbosity normal
shell: bash
working-directory: ${{ inputs.working_directory }}
51 changes: 41 additions & 10 deletions .github/workflows/actions/deploy-to-aws/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,6 @@ inputs:
aws_role_arn:
description: The AWS Role ARN to assume.
required: true

# Image parameters
ghcr_token:
description: The token to use to login to the GHCR.
required: true
Expand All @@ -28,13 +26,27 @@ inputs:
image_name:
description: The name of the image to be deployed.
required: true
image_digest:
description: The digest of the image to be deployed. Identifies the unique image tag in the GHCR.
tier_name:
description: The tier/layer name such as web or api.
required: true
short_sha:
description: The short SHA used to tag image in GCHR.
required: true

runs:
using: composite

steps:
- name: Set reusable variables
id: vars
shell: bash
run: |
echo "task_definition_name=${{ inputs.app_name }}-${{ inputs.tier_name }}-task-definition-${{ inputs.environment }}" >> $GITHUB_OUTPUT
echo "container_name=${{ inputs.app_name }}-${{ inputs.tier_name }}-container-${{ inputs.environment }}" >> $GITHUB_OUTPUT
echo "ecs_cluster_name=${{ inputs.app_name }}-ecs-cluster-${{ inputs.environment }}" >> $GITHUB_OUTPUT
echo "ecs_service_name=${{ inputs.app_name }}-ecs-${{ inputs.tier_name }}-service-${{ inputs.environment }}" >> $GITHUB_OUTPUT
echo "full_ecr_repo_url=${{ inputs.aws_account }}.dkr.ecr.${{ inputs.region }}.amazonaws.com/${{ inputs.app_name }}-ecr-repo-${{ inputs.environment }}" >> $GITHUB_OUTPUT
- name: Log in to the GHCR
uses: docker/login-action@v2
with:
Expand All @@ -51,14 +63,33 @@ runs:
role-duration-seconds: 1800
role-session-name: ci-deployment

- name: Login to AWS CLI
- name: Login to Amazon ECR
uses: aws-actions/amazon-ecr-login@v2

- name: Pull Docker image from GHCR
shell: bash
run: |
aws ecr get-login-password --region ${{ inputs.region }} | docker login --username AWS --password-stdin ${{ inputs.aws_account }}.dkr.ecr.${{ inputs.region }}.amazonaws.com/${{ inputs.app_name }}-ecr-repo-${{ inputs.environment }}
docker pull ${{ inputs.github_image_repo }}/${{ inputs.image_name }}:${{ inputs.short_sha}}
docker tag ${{ inputs.github_image_repo }}/${{ inputs.image_name }}:${{ inputs.short_sha}} ${{ steps.vars.outputs.full_ecr_repo_url }}:${{ inputs.image_name }}-${{ inputs.short_sha }}
docker push ${{ steps.vars.outputs.full_ecr_repo_url }}:${{ inputs.image_name }}-${{ inputs.short_sha }}
- name: Tag the image in the GHCR as ${{ inputs.environment }}
- name: Download task definition
shell: bash
run: |
docker pull ${{ inputs.github_image_repo }}${{ inputs.image_name }}@${{ inputs.image_digest }}
docker tag ${{ inputs.github_image_repo }}${{ inputs.image_name }}@${{ inputs.image_digest }} ${{ inputs.aws_account }}.dkr.ecr.${{ inputs.region }}.amazonaws.com/${{ inputs.app_name }}-ecr-repo-${{ inputs.environment }}:${{ inputs.image_name }}
docker push ${{ inputs.aws_account }}.dkr.ecr.${{ inputs.region }}.amazonaws.com/${{ inputs.app_name }}-ecr-repo-${{ inputs.environment }}:${{ inputs.image_name }}
aws ecs describe-task-definition --task-definition ${{ steps.vars.outputs.task_definition_name }} --query taskDefinition > ${{ steps.vars.outputs.task_definition_name }}.json
- name: Fill in new image ID in task definition
id: task-def
uses: aws-actions/amazon-ecs-render-task-definition@v1
with:
task-definition: ${{ steps.vars.outputs.task_definition_name }}.json
container-name: ${{ steps.vars.outputs.container_name }}
image: "${{ steps.vars.outputs.full_ecr_repo_url }}:${{ inputs.image_name }}-${{ inputs.short_sha }}"

- name: Deploy Amazon ECS task definition
uses: aws-actions/amazon-ecs-deploy-task-definition@v2
with:
task-definition: ${{ steps.task-def.outputs.task-definition }}
service: ${{ steps.vars.outputs.ecs_service_name }}
cluster: ${{ steps.vars.outputs.ecs_cluster_name }}
wait-for-service-stability: true
33 changes: 33 additions & 0 deletions .github/workflows/build-and-test-api.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
name: Build and Test API

on:
pull_request:
branches:
- master
paths:
- "api/**"
- "db/**"

workflow_dispatch:

env:
WORKING_DIRECTORY: ./api

jobs:
build-and-test:
runs-on: ubuntu-latest

strategy:
matrix:
dotnet-major-version: [8]
dotnet-minor-version: [0]

steps:
- name: Checkout repository
uses: actions/checkout@v4

- name: Building Web codebase
uses: ./.github/workflows/actions/build-api
with:
working_directory: ${{ env.WORKING_DIRECTORY }}
dotnet_version: ${{ matrix.dotnet-major-version }}.${{ matrix.dotnet-minor-version }}
29 changes: 0 additions & 29 deletions .github/workflows/main.yml

This file was deleted.

164 changes: 164 additions & 0 deletions .github/workflows/publish-api.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,164 @@
name: Deploy API

on:
push:
branches:
- master
paths:
- "api/**"
- "db/**"

workflow_dispatch:

env:
WORKING_DIRECTORY: ./api
IMAGE_NAME: api
GITHUB_IMAGE_REPO: ghcr.io/bcgov/jasper

jobs:
build:
name: Build, Create and Push Image
runs-on: ubuntu-latest
outputs:
short_sha: ${{ steps.short_sha.outputs.SHORT_SHA }}

strategy:
matrix:
dotnet-major-version: [8]
dotnet-minor-version: [0]

steps:
- name: Checkout repository
uses: actions/checkout@v4

- name: Build API codebase
uses: ./.github/workflows/actions/build-api
with:
working_directory: ${{ env.WORKING_DIRECTORY }}
dotnet_version: ${{ matrix.dotnet-major-version }}.${{ matrix.dotnet-minor-version }}

- name: Log in to the GHCR
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

- name: Get short SHA
id: short_sha
run: |
echo "SHORT_SHA=$(git rev-parse --short HEAD)" >> $GITHUB_OUTPUT
- name: Setup Image Metadata
id: meta
uses: docker/metadata-action@v5
with:
images: |
${{ env.GITHUB_IMAGE_REPO }}/${{ env.IMAGE_NAME }}
tags: |
type=raw,value=${{ steps.short_sha.outputs.SHORT_SHA }}
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3

- name: Build and Push Image to ghcr.io
id: build_image
uses: docker/build-push-action@v5
with:
push: true
context: .
file: ./docker/api/Dockerfile.release
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
load: true
build-args: |
dotnet_version=${{ matrix.dotnet-major-version }}.${{ matrix.dotnet-minor-version }}
deploy2dev:
name: Deploy to DEV
needs: build
env:
ENVIRONMENT: dev
permissions:
id-token: write
packages: write
runs-on: ubuntu-latest
environment: dev

steps:
- name: Checkout
uses: actions/checkout@v4

- name: Deploy to ${{ env.ENVIRONMENT }}
uses: ./.github/workflows/actions/deploy-to-aws
with:
environment: ${{ env.ENVIRONMENT }}
aws_account: ${{ vars.AWS_ACCOUNT }}
region: ${{ vars.AWS_REGION }}
app_name: ${{ vars.APP_NAME }}
aws_role_arn: ${{ vars.AWS_ROLE_ARN }}
ghcr_token: ${{ secrets.GITHUB_TOKEN }}
github_image_repo: ${{ env.GITHUB_IMAGE_REPO }}
image_name: ${{ env.IMAGE_NAME }}
tier_name: api
short_sha: ${{ needs.build.outputs.short_sha }}

deploy2test:
name: Deploy to TEST
needs: [build, deploy2dev]
env:
ENVIRONMENT: test
permissions:
id-token: write
packages: write
runs-on: ubuntu-latest
environment: test

steps:
- name: Checkout
uses: actions/checkout@v4

# Uncomment when infra in AWS in TEST environment has been configured
# - name: Deploy to ${{ env.ENVIRONMENT }}
# uses: ./.github/workflows/actions/deploy-to-aws
# with:
# environment: ${{ env.ENVIRONMENT }}
# aws_account: ${{ vars.AWS_ACCOUNT }}
# region: ${{ vars.AWS_REGION }}
# app_name: ${{ vars.APP_NAME }}
# aws_role_arn: ${{ vars.AWS_ROLE_ARN }}
# ghcr_token: ${{ secrets.GITHUB_TOKEN }}
# github_image_repo: ${{ env.GITHUB_IMAGE_REPO }}
# image_name: ${{ env.IMAGE_NAME }}
# tier_name: api
# short_sha: ${{ needs.build.outputs.short_sha }}

deploy2prod:
name: Deploy to PROD
needs: [build, deploy2dev, deploy2test]
env:
ENVIRONMENT: prod
permissions:
id-token: write
packages: write
runs-on: ubuntu-latest
environment: prod

steps:
- name: Checkout
uses: actions/checkout@v4

# Uncomment when infra in AWS in PROD environment has been configured
# - name: Deploy to ${{ env.ENVIRONMENT }}
# uses: ./.github/workflows/actions/deploy-to-aws
# with:
# environment: ${{ env.ENVIRONMENT }}
# aws_account: ${{ vars.AWS_ACCOUNT }}
# region: ${{ vars.AWS_REGION }}
# app_name: ${{ vars.APP_NAME }}
# aws_role_arn: ${{ vars.AWS_ROLE_ARN }}
# ghcr_token: ${{ secrets.GITHUB_TOKEN }}
# github_image_repo: ${{ env.GITHUB_IMAGE_REPO }}
# image_name: ${{ env.IMAGE_NAME }}
# tier_name: api
# short_sha: ${{ needs.build.outputs.short_sha }}
Loading

0 comments on commit 5bbe0b4

Please sign in to comment.