Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Container pipeline overhaul #59

Merged
merged 8 commits into from
Mar 5, 2024

Conversation

nohoster
Copy link
Contributor

Hello, I decided to optimize the way the container is being build. Currently is really slow since it has to do a fresh compilation every time and also it's using emulation for the arm builds.

Instead I decided to split the process. First compile the redlib binary using cross compilation. This takes only 3 to 4 minutes. And then just copy this binary into a container.

This also allows the building process to reuse dependency cache.

The whole process takes less than 4 minutes in comparison to the 15 to 30 minutes of the current pipeline which is quite an improvement. It also produces a single redlib:latest multiarch container, so there is no need for different tags.

The only detail is that I'm using releases to save the binaries and I'm using the DATE-TIME as the release name since there is no app version updates. You can check the result in my repo.
If you don't like that I can change it into a single pipeline and copy the binary inside the pipeline.

.github/workflows/build-artifacts.yaml Outdated Show resolved Hide resolved
.github/workflows/main-docker.yml Outdated Show resolved Hide resolved
.github/workflows/main-docker.yml Outdated Show resolved Hide resolved
Dockerfile Outdated Show resolved Hide resolved
@sigaloid sigaloid merged commit 3f4526d into redlib-org:main Mar 5, 2024
3 checks passed
@chowder
Copy link

chowder commented Mar 5, 2024

Took me a while to find out today why my changes weren't being reflected in my development containers... :'(

perennialtech added a commit to perennialtech/redlib that referenced this pull request May 31, 2024
perennialtech added a commit to perennialtech/redlib that referenced this pull request Jun 27, 2024
perennialtech added a commit to perennialtech/redlib that referenced this pull request Jun 27, 2024
perennialtech added a commit to perennialtech/redlib that referenced this pull request Aug 13, 2024
@master-hax
Copy link

master-hax commented Sep 17, 2024

Currently is really slow since it has to do a fresh compilation every time and also it's using emulation for the arm builds.

FYI there are fixes for this, we did not have to throw away the working Dockerfiles and replace them with one that's a glorified link to github.

one example is libreddit/libreddit#751 where i fixed the fresh compilation issue for the arm builds. is it possible to revert this change and apply the same optimization? it would make building a docker image during development much easier and would not mislead users who are building from a git URL.

@sigaloid
Copy link
Member

If you want to do another overhaul to the actions, I wouldn't be opposed to merging - there's been some struggles since this PR in some places. Open an issue describing your motivations and a PR, and I'll take a look. I'm not skilled in GHA in general so I defer to others generally.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants