Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Adds hires fix on layer diffusion #278

Merged
merged 17 commits into from
Jul 7, 2024
Merged

feat: Adds hires fix on layer diffusion #278

merged 17 commits into from
Jul 7, 2024

Conversation

db0
Copy link
Member

@db0 db0 commented Jul 4, 2024

  • feat: Adds hires fix to layer diffusion
  • fix: Ensures steps are used as intended in stable cascade
  • chore: Adds pipeline design for layer diffusion on standard SD
  • fix: Uses denoise to calculate a lower step count when doing hires fix second pass
    • Previous to this change, hires fix was doing the full amount of steps on each pass. This is unnecessary as there are (massive) diminishing returns on the second pass.
      • For example, 10 steps versus 20 steps on the second pass can be virtually indistinguishable from each other, so this logic will ultimately help reduce the time spent generating these images. This is especially true for high-step requests.
    • It is worth pointing out that the reference images used to confirm images remain the same version-to-version of horde-engine did not require updating. These images are checked using a form of cosine similarity, so at least to the computer, the 'new' generated hires fix images are basically identical before and after the change.

All tox passes on my end

@db0 db0 requested a review from tazlin July 4, 2024 22:55
@db0
Copy link
Member Author

db0 commented Jul 7, 2024

@tazlin I added a change which sets the upscale steps to 30% of the normal steps. From what I see, this doesn't change the results by any significant amount so that could be a good workaround and later we can expose the hires_steps as well. Can you validate as well?

Copy link
Member

@tazlin tazlin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Everything looks reasonable as of now. Assuming tests pass, I'm OK with a rebase onto main and publishing for a (at least brief) beta testing window with workers.

@tazlin
Copy link
Member

tazlin commented Jul 7, 2024

@tazlin I added a change which sets the upscale steps to 30% of the normal steps. From what I see, this doesn't change the results by any significant amount so that could be a good workaround and later we can expose the hires_steps as well. Can you validate as well?

I just wanted to emphasize that the images are visually identical to my eye and did not require updating/changing for the cosine image similarity/histogram checks.

@db0 db0 merged commit 77f5b4d into main Jul 7, 2024
2 checks passed
@db0 db0 deleted the ldhires branch July 7, 2024 15:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants