Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for smooth chroma upsampling #586

Open
maryla-uc opened this issue Jan 18, 2022 · 2 comments
Open

Add support for smooth chroma upsampling #586

maryla-uc opened this issue Jan 18, 2022 · 2 comments

Comments

@maryla-uc
Copy link
Contributor

Libheif currently only supports nearest neighbor chroma upsampling, which often leads to extra artifacts when decoding YUV420 images.
Most codecs support a bilinear chroma upsampling with weights 9:3:3:1
E.g. if you have the following Luma and Chroma samples (with centered chroma samples, as per issue #521):

L   L  L    L
  C1     C2
L   L*  L   L 

L   L  L    L
  C3     C4
L   L  L    L

then the value of chroma at the position marked with * would be:
C = (9 * C1 + 3 * C2 + 3 * C3 + C4)/16

See also the implementation in libavif: https://github.com/AOMediaCodec/libavif/blob/ded15fb7b41b0513e138ba457034a2fb91c1f2c9/src/reformat.c#L496

Here is an example (artificial) avif image, decoded with libheif (nearest neighbor) and with libavif (smooth upsampling). The difference is very obvious when zooming in.

C95FUpghqahiGyo-1

@farindk
Copy link
Contributor

farindk commented Jan 20, 2022

Thanks for the proposal. That filter seems easy to implement. I could add a simple implementation to libheif. For an accelerated (SIMD) version, I would prefer to reuse an external implementation (e.g. libyuv).

I am wondering what would be the best way to choose the filters. In theory, the upsampling filter should match the downsampling filter, but that is not known to the decoder. Signaling the preferred upload filter might be something that could be added to the file formats. Until then, we probably have to pass the decision onto the user.

@maryla-uc
Copy link
Contributor Author

The YUV420 to 444 conversion function in libyuv do use bilinear upsampling so if you want to use that, it's indeed an option. Unfortunately, libyuv's YUV420 to RGB conversion does NOT currently use bilinear, but simple nearest neighbor upsampling. See https://bugs.chromium.org/p/libyuv/issues/detail?id=872 Until this gets fixed, the workaround is to convert in two steps: YUV420->YUV444->RGB.

I agree users should be able to choose the filter applied, with a sensible default set otherwise. I think the default should be bilinear.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants