-
Notifications
You must be signed in to change notification settings - Fork 36
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Reapply grouping workflows and adding support for integration tests (#…
…655) * Group workflows by trigger type and add support for integration tests Signed-off-by: Radoslav Dimitrov <[email protected]> * Do git lfs pull Signed-off-by: Radoslav Dimitrov <[email protected]> * Run poetry lock Signed-off-by: Radoslav Dimitrov <[email protected]> --------- Signed-off-by: Radoslav Dimitrov <[email protected]>
- Loading branch information
Showing
14 changed files
with
397 additions
and
41 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,3 +1,4 @@ | ||
| # This workflow syncs the vector database | ||
| name: Sync vector DB | ||
|
|
||
| on: | ||
|
|
||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,144 @@ | ||
| # This workflow will run the integration tests for the project | ||
| name: Tests - Integration | ||
|
|
||
| on: | ||
| workflow_call: | ||
| inputs: | ||
| artifact-name: | ||
| description: 'The name of the artifact to download' | ||
| required: true | ||
| type: string | ||
| secrets: | ||
| copilot-key: | ||
| description: 'The Copilot key to use for integration tests' | ||
| required: true | ||
|
|
||
| jobs: | ||
| integration-tests: | ||
| name: Run | ||
| runs-on: ubuntu-latest | ||
| strategy: | ||
| matrix: | ||
| python-version: [ "3.12" ] | ||
| env: | ||
| CONTAINER_NAME: "codegate" | ||
| CERT_FILE: "/app/codegate_volume/certs/ca.crt" | ||
| steps: | ||
| - name: Checkout | ||
| uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4 | ||
| with: | ||
| lfs: true | ||
|
|
||
| - name: Checkout LFS objects | ||
| run: git lfs pull | ||
|
|
||
| - name: Ensure file permissions for mounted volume | ||
| run: | | ||
| mkdir -p ./codegate_volume/certs ./codegate_volume/models ./codegate_volume/db | ||
| chmod -R 777 ./codegate_volume | ||
| - name: Download Docker image artifact | ||
| uses: actions/download-artifact@fa0a91b85d4f404e444e00e005971372dc801d16 # v4 | ||
| with: | ||
| name: ${{ inputs.artifact-name }} | ||
|
|
||
| - name: Load Docker image | ||
| run: | | ||
| docker load -i image.tar | ||
| echo "Loaded image:" | ||
| docker images | ||
| - name: Run container from the loaded image | ||
| run: | | ||
| # Get the image name | ||
| DOCKER_IMAGE=$(docker images --format "{{.Repository}}:{{.Tag}}" | head -n 1) | ||
| echo "Running container from image: $DOCKER_IMAGE" | ||
| # Run the container | ||
| docker run --name $CONTAINER_NAME -d -p 8989:8989 -p 9090:9090 \ | ||
| -p 8990:8990 \ | ||
| -v "$(pwd)"/codegate_volume:/app/codegate_volume \ | ||
| -e CODEGATE_APP_LOG_LEVEL=DEBUG \ | ||
| --restart unless-stopped $DOCKER_IMAGE | ||
| # Confirm the container started | ||
| echo "Container started:" | ||
| docker ps | ||
| # Verify container is running with correct ports | ||
| docker ps -f name=$CONTAINER_NAME | ||
| # Check mount configuration | ||
| docker inspect $CONTAINER_NAME -f '{{ json .Mounts }}' | jq | ||
| - name: Ensure certificates are available in the container | ||
| timeout-minutes: 4 | ||
| run: | | ||
| # Wait for the cert file to be available in the container | ||
| while true; do | ||
| echo "Checking for $CERT_FILE in container $CONTAINER_NAME..." | ||
| if docker exec "$CONTAINER_NAME" test -f "$CERT_FILE"; then | ||
| echo "Cert file found: $CERT_FILE" | ||
| break | ||
| else | ||
| echo "Cert file not found. Retrying in 5 seconds..." | ||
| sleep 5 | ||
| fi | ||
| done | ||
| # Verify volume contents are accessible | ||
| docker exec $CONTAINER_NAME ls -la /app/codegate_volume | ||
| # Print the container logs we got so far | ||
| docker logs $CONTAINER_NAME | ||
| - name: Install the CodeGate certificate | ||
| run: | | ||
| docker cp codegate:/app/codegate_volume/certs/ca.crt ./codegate.crt | ||
| sudo cp ./codegate.crt /usr/local/share/ca-certificates/codegate.crt | ||
| sudo update-ca-certificates | ||
| - name: Set up Python ${{ matrix.python-version }} | ||
| uses: actions/setup-python@0b93645e9fea7318ecaed2b359559ac225c90a2b # v5 | ||
| with: | ||
| python-version: ${{ matrix.python-version }} | ||
|
|
||
| - name: Install Poetry | ||
| uses: snok/install-poetry@76e04a911780d5b312d89783f7b1cd627778900a # v1 | ||
| with: | ||
| version: 2.0.1 | ||
| virtualenvs-create: true | ||
| virtualenvs-in-project: true | ||
|
|
||
| - name: Load cached venv | ||
| id: cached-poetry-dependencies | ||
| uses: actions/cache@1bd1e32a3bdc45362d1e726936510720a7c30a57 # v4 | ||
| with: | ||
| path: .venv | ||
| key: venv-${{ runner.os }}-${{ matrix.python-version }}-${{ hashFiles('**/poetry.lock') }} | ||
|
|
||
| - name: Install dependencies | ||
| run: poetry install --with dev | ||
|
|
||
| - name: Run integration tests - Copilot | ||
| env: | ||
| CODEGATE_PROVIDERS: "copilot" | ||
| CA_CERT_FILE: "/home/runner/work/codegate/codegate/codegate_volume/certs/ca.crt" | ||
| ENV_COPILOT_KEY: ${{ secrets.copilot-key }} | ||
| run: | | ||
| poetry run python tests/integration/integration_tests.py | ||
| - name: Print the container logs (useful for debugging) | ||
| if: always() | ||
| run: | | ||
| docker logs $CONTAINER_NAME | ||
| echo "Models contents:" | ||
| ls -la codegate_volume/models | ||
| docker exec $CONTAINER_NAME ls -la /app/codegate_volume/models | ||
| echo "Certs contents:" | ||
| ls -la codegate_volume/certs | ||
| docker exec $CONTAINER_NAME ls -la /app/codegate_volume/certs | ||
| echo "DB contents:" | ||
| ls -la codegate_volume/db | ||
| docker exec $CONTAINER_NAME ls -la /app/codegate_volume/db |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,29 @@ | ||
| # These set of workflows run on every pull request | ||
| name: Run - pull_request | ||
|
|
||
| on: | ||
| workflow_dispatch: | ||
| pull_request: | ||
| branches: [ main ] | ||
|
|
||
| jobs: | ||
| security: | ||
| name: Security Checks | ||
| uses: ./.github/workflows/security.yml | ||
| ci: | ||
| name: Build, Test & Lint | ||
| uses: ./.github/workflows/ci.yml | ||
| image-build: | ||
| name: OCI Image - Build | ||
| needs: [ci] # No need to build the image if the CI fails | ||
| uses: ./.github/workflows/image-build.yml | ||
| with: | ||
| artifact-name: "codegate-image" | ||
| integration-tests: | ||
| name: Integration Tests | ||
| needs: [ci, image-build] # We need the image available in order to run the integration tests | ||
| uses: ./.github/workflows/integration-tests.yml | ||
| with: | ||
| artifact-name: "codegate-image" | ||
| secrets: | ||
| copilot-key: ${{ secrets.INTEGRATION_TESTS_COPILOT_KEY }} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,19 @@ | ||
| # These set of workflows run on every push to the main branch | ||
| name: Run - push_to_main | ||
|
|
||
| on: | ||
| workflow_dispatch: | ||
| push: | ||
| branches: [ main ] | ||
|
|
||
| jobs: | ||
| security: | ||
| name: Security Checks | ||
| uses: ./.github/workflows/security.yml | ||
| ci: | ||
| name: Build, Test & Lint | ||
| uses: ./.github/workflows/ci.yml | ||
| openapi: | ||
| name: Generate the OpenAPI docs | ||
| needs: [ci] | ||
| uses: ./.github/workflows/openapi.yml |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,13 @@ | ||
| # These set of workflows run on every published release | ||
| name: Run - release_published | ||
|
|
||
| on: | ||
| workflow_dispatch: | ||
| release: | ||
| types: | ||
| - published | ||
|
|
||
| jobs: | ||
| image-publish: | ||
| name: OCI Image - Publish | ||
| uses: ./.github/workflows/image-publish.yml |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.