Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(ui): image quality degradation on canvas after repeated generations #7539

Merged
merged 1 commit into from
Jan 10, 2025

Conversation

psychedelicious
Copy link
Collaborator

Summary

In 7cdda00, I made a small optimization to the graphs for inpainting and outpainting. The change makes the graphs a bit smaller, saving us one PNG encode and image writing step.

As reported by @skunkworxdark on discord, there is an issue where the unmasked regions of an image slowly degrade when the image is inpainted repeatedly.

Because of timing of the issue report coincided with 7cdda00, I think that optimization may be the cause, but I'm having trouble wrapping my head around why exactly.

@skunkworxdark , Could you please test this PR to confirm it resolves the degradation issue you noticed? If it does, we'll get this revert merged. If not, I'll take another crack at the degradation issue with fresh eyes another day.

Related Issues / Discussions

https://discord.com/channels/1020123559063990373/1149513647022948483/1307060842910449716

QA Instructions

I'm not 100% sure how to reproduce the issue - can you help to explain the repro steps @skunkworxdark ?

Merge Plan

n/a

Checklist

  • The PR has a short but descriptive title, suitable for a changelog

@github-actions github-actions bot added the frontend PRs that change frontend files label Jan 9, 2025
@psychedelicious psychedelicious marked this pull request as draft January 9, 2025 08:34
@psychedelicious
Copy link
Collaborator Author

Marked draft to prevent accidental premature merge.

@skunkworxdark
Copy link
Contributor

I've tested this, and it has fixed the image quality degradation.

The steps to reproduce are:

  1. Have an image with some fine details that could easily be degraded.
  2. Have the Scale Before Processing set to Auto
  3. Set the bounding bbox size to something smaller than the ideal model size so that the Auto Size is bigger.
  4. Add an inpaint mask to a small area within the bbox.
  5. Prompt for a change to the masked area.
  6. Invoke to the canvas and accept the result.
  7. ensure you have a random seed and repeat step 6 a few times.

Depending on the fine detail in the image degradation should be visible after 2-3 generations.

Although this fixes the degradation of the image there is still another issue that I raised at the same time. The Edge Size is still applied on the upscaled inpaint mask. So you will get the mask expansion to be smaller than expected.

Here is an image of the workflow used by this PR. You can see the Create Gradient Mask is after the Resize Image of the mask. The expanded mask is then later scaled back down to the original size to be recombined with the original.
image

I suggest this could be modified to apply the Create Gradient Mask before scaling up the Mask however a second Create Gradient Mask is needed to create the actual denoise mask but you no longer need the final Resize Image node as the mask is now available at the correct size.
image

To avoid duplication of the Create Gradient Mask node I would say ideally, a better approach is to split the Create Gradient Mask into two separate nodes one to expand the mask and one to create the denoise mask but I'm not convinced the effort is worth it.

@psychedelicious
Copy link
Collaborator Author

Great, thank you for testing and sharing the reproduction steps!

We'll pick up the edge size issue separately (and thanks also for your suggestions for that issue).

@psychedelicious psychedelicious marked this pull request as ready for review January 9, 2025 20:33
@psychedelicious psychedelicious enabled auto-merge (rebase) January 9, 2025 20:33
@psychedelicious psychedelicious force-pushed the psyche/fix/canvas-paste-back-quality branch from 052e787 to e9a62b0 Compare January 10, 2025 04:57
@psychedelicious psychedelicious merged commit d88b59c into main Jan 10, 2025
15 checks passed
@psychedelicious psychedelicious deleted the psyche/fix/canvas-paste-back-quality branch January 10, 2025 04:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
frontend PRs that change frontend files
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants