Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make compresslevel configurable for .tar.gz #888

Merged
merged 1 commit into from
Sep 16, 2024

Conversation

gdh1995
Copy link
Contributor

@gdh1995 gdh1995 commented Aug 29, 2024

What's the problem

The current tar packaging process is not efficient enough, as evidenced by:

  • The default gzip compression level "6" can take 200% more time than compression level "1", but the size is only reduced by 10%-20%.
  • For tar packages that reach gigabyte levels and are only transmitted and used within an organization’s intranet, packing speed might be more critical than size.

How to solve

Add compresslevel: str which can be "" (auto, 6) | "0" | "1" | ... | "9" (this is extracted from #887).

@gdh1995 gdh1995 requested review from aiuto and cgrindel as code owners August 29, 2024 09:18
Copy link
Collaborator

@cgrindel cgrindel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there any way that we can test that the compress level value is being passed along to the command line correctly? Alternatively, can we add a test that at least uses the new attribute?

pkg/private/tar/tar_writer.py Outdated Show resolved Hide resolved
pkg/private/tar/tar_writer.py Outdated Show resolved Hide resolved
pkg/private/tar/tar.bzl Outdated Show resolved Hide resolved
pkg/private/tar/build_tar.py Outdated Show resolved Hide resolved
@gdh1995
Copy link
Contributor Author

gdh1995 commented Aug 30, 2024

Hello, I added a simple test named test_compress_level in tests/tar/pkg_tar_test.py, and its expected sizes are captured on my Python3.8 environment - fortunately it looks like working well in CI jobs.

Copy link
Collaborator

@cgrindel cgrindel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. I will wait to merge so that @aiuto can review.

pkg/private/tar/tar.bzl Outdated Show resolved Hide resolved
@aiuto aiuto merged commit 03cbc6c into bazelbuild:main Sep 16, 2024
2 checks passed
@gdh1995 gdh1995 deleted the tar_gz_ratio branch October 8, 2024 03:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants