Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat: Noise and Blur filters for Canvas #7551

Merged
merged 10 commits into from
Jan 15, 2025

Conversation

dunkeroni
Copy link
Contributor

@dunkeroni dunkeroni commented Jan 13, 2025

Summary

A few things lumped together here.

  1. Image Blur invocation now correctly handles transparent regions instead of darkening edges.
    Old method: (center and right image will look the same if your git is not in Dark Mode, but the right image has a white background)
    image
    New method:
    image

  2. Paste Image node now respects transparency. Previously alpha channel of the top image would override the bottom.

  3. Added Image Noise invocation to introduce random noise into images. Supports gaussian/salt-and-pepper modes in greyscale/color.
    image

  4. Created filters in Canvas for blur and noise to improve img2img results.
    image

noise_filters_2.mp4

Above video was taken before Noise Size input was added.
image

Related Issues / Discussions

Additional noise strategies possibly to be added later on:

  • Worley/Voronoi have interesting implications, but may not be useful for canvas.
  • Image diffusion noise might result in more consistent img2img results at the risk of decreased improvement.
  • Simplex noise patent ran out in Jan 2022 (may be slower/worse than OpenSimplex2?)
  • Perlin noise might be helpful for organic textures

Known Issues (not from this PR, but visible because of it):

  • Blur filter does not have intended results when used on a cutout/brushed shape layer. Layers are cropped to only the visible pixels, and filters cannot change the size of a layer. Adding that capability will require changes to how the canvas handles new raster layers.
    image
  • Filter previews show the original image behind the filtered version, which makes previewing a blur filter difficult.
    image

Merge Plan

Ready to merge

Checklist

  • The PR has a short but descriptive title, suitable for a changelog
  • Tests added / updated (if applicable)
  • Documentation added / updated (if applicable)
  • Updated What's New copy (if doing a release after this PR)

@github-actions github-actions bot added python PRs that change python files invocations PRs that change invocations frontend PRs that change frontend files labels Jan 13, 2025
Copy link
Collaborator

@psychedelicious psychedelicious left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! I've pushed 3 commits:

  • Run make ruff to fix python style checks.
  • Run make frontend-typegen to update the frontend generated typescript types.
  • Hide the layer being filtered when rendering the preview, which fixes that transparency issue.

I also thought about the issue where the blur gets cut off by the layer bounds but I still don't have an easy fix for that, will probably need some deeper fiddling to sort.

@psychedelicious psychedelicious enabled auto-merge (rebase) January 15, 2025 22:17
dunkeroni and others added 10 commits January 16, 2025 09:17
Previously, when previewing a filter on a layer with some transparency or a filter that changes the alpha, the preview was rendered on top of the layer. The preview blended with the layer, which isn't right.

In this change, the layer is hidden during the preview, and when the filter finishes (having been applied or canceled - the two possible paths), the layer is shown.

Technically, we are hiding and showing the layer's object renderer's konva group, which contains the layer's "real" data.

Another small change was made to prevent a flash of empty layer, by waiting to destroy a previous filter preview image until the new preview image is ready to display.
@psychedelicious psychedelicious merged commit 993401a into invoke-ai:main Jan 15, 2025
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
frontend PRs that change frontend files invocations PRs that change invocations python PRs that change python files
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants