-
Notifications
You must be signed in to change notification settings - Fork 1
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
chore(initial-commit): initial commit of all content
chore(github-action): update workflow to use self-hosted runners with nvidia label chore(script): use different image name in build script chore(skopeo): add skopeo to devShell chore(skopeo): fix skopeo in build shell chore(build-script): change to use nix develop flake chore(build-script): fix build script nix develop chore(script): modify script to run nix develop with args chore(script): modify how build script works chore(scripts): modify build script chore(build): modify build scripts chore(build): modify build script chore(environment): remove old shell and default scripts
- Loading branch information
Sean Murphy
committed
Feb 6, 2024
0 parents
commit 93982a2
Showing
9 changed files
with
559 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,24 @@ | ||
--- | ||
name: "Build and deploy site - cuda edition" | ||
on: # yamllint disable-line rule:truthy | ||
pull_request: | ||
push: | ||
jobs: | ||
build: | ||
# when running on self-hosted, uncomment the following and comment out the subsequent line | ||
runs-on: [ self-hosted, nixos, nvidia-545 ] | ||
# runs-on: ubuntu-latest | ||
steps: | ||
- uses: actions/checkout@v4 | ||
with: | ||
fetch-depth: 0 | ||
# when running on a self-hosted nixos system, I removed the following 2 uses... | ||
#- uses: DeterminateSystems/nix-installer-action@main | ||
#- uses: DeterminateSystems/magic-nix-cache-action@main | ||
- run: | | ||
scripts/build_and_push_image.sh | ||
env: | ||
# this secret must be defined and available to the runner; it's assumed it | ||
# has write priviliges on a docker registry | ||
DOCKER_ACCESS_TOKEN: ${{ secrets.DOCKER_ACCESS_TOKEN }} | ||
if: github.ref == 'refs/heads/main' |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,32 @@ | ||
# nix-container-build-gha | ||
|
||
This is a simple testing repo I used to understand how to build python environments in github actions | ||
with nix. I wrote up my thoughts in [this medium post](https://medium.com/@seanrmurphy/building-container-images-using-nix-and-github-actions-ba548ab9080d). | ||
|
||
This is the public version of the repo which anyone can look through; I also maintain a private version | ||
which is linked to some self hosted github runners - I don't want to link the public version to any such | ||
runners. | ||
|
||
There are comments in the code which give some pointers on how things work - feel free to look around. | ||
|
||
## The python application | ||
|
||
The python application was taken from [this repo](https://github.com/mitchellh/flask-nix-example) created by Mitchell Hashimoto - it is a simple flask application. | ||
|
||
I have included a couple of unecessary dependencies in the `pyproject.toml` just to understand how these are | ||
handled (`torch`, `jupyter` and `beautifulsoup4`). They are available in the resulting python environment but not used by the application. | ||
|
||
## Working locally | ||
|
||
This assumes you have a sensible nix configuration and are comfortable using flakes. | ||
|
||
- `nix build` will build the application and put the content in the `result` directory | ||
- `nix build .#ociApplicationImage` will build a container image which runs the application - the resulting container image is a gzip'd tarball in the `result` directory which can be imported to docker using `docker load < result` | ||
- `nix build .#ociPackageImage` will build a container image which contains the python environment but launches bash; run python within bash and you can import the dependencies | ||
- `nix run` will run the application without building a container image | ||
|
||
## Using the github actions | ||
|
||
The github action was essentially copied from [this repo](https://github.com/wagdav/thewagner.net). | ||
|
||
It requires a token called DOCKER_ACCESS_TOKEN to access a docker repository and push the resulting container image there. This repo uses the standard github runners but in the private variant, I was using a self-hosted runner (there are a couple of comments in the github action definition which highlight the small differences). |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,10 @@ | ||
(import | ||
( | ||
let lock = builtins.fromJSON (builtins.readFile ./flake.lock); in | ||
fetchTarball { | ||
url = "https://github.com/edolstra/flake-compat/archive/${lock.nodes.flake-compat.locked.rev}.tar.gz"; | ||
sha256 = lock.nodes.flake-compat.locked.narHash; | ||
} | ||
) | ||
{ src = ./.; } | ||
).defaultNix |
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Oops, something went wrong.
Oops, something went wrong.