Skip to content

Add sigmas to Flux pipelines #10081

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Dec 2, 2024
Merged

Conversation

hlky
Copy link
Contributor

@hlky hlky commented Dec 2, 2024

What does this PR do?

This PR adds sigmas argument to Flux pipelines and removes timesteps argument as passing timesteps to FlowMatch schedulers is currently unsupported.

Fixes #10080

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@yiyixuxu @asomoza

@HuggingFaceDocBuilderDev

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@SahilCarterr
Copy link
Contributor

SahilCarterr commented Dec 2, 2024

What do you think about writing a test to test out sigmas in FluxPipeline?
@hlky

Copy link
Collaborator

@yiyixuxu yiyixuxu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks!

@yiyixuxu yiyixuxu merged commit 8d386f7 into huggingface:main Dec 2, 2024
15 checks passed
@vladmandic
Copy link
Contributor

vladmandic commented Dec 2, 2024

sigmas are supposed to be list-of-float, but pipeline internally uses np.array - passing sigmas right now causes a runtime error

return math.exp(mu) / (math.exp(mu) + (1 / t - 1) ** sigma)
TypeError: unsupported operand type(s) for /: 'int' and 'list'

which means that every instance of this line:

sigmas = np.linspace(1.0, 1 / num_inference_steps, num_inference_steps) if sigmas is None else sigmas

should at least be updated to

sigmas = np.linspace(1.0, 1 / num_inference_steps, num_inference_steps) if sigmas is None else np.array(sigmas)

@yiyixuxu
Copy link
Collaborator

yiyixuxu commented Dec 2, 2024

thanks @vladmandic think we should probably handle this on scheduler.set_timesteps()
we already do that for custom timesteps

timesteps = np.array(timesteps).astype(np.float32)

what do you think ? @hlky

@yiyixuxu
Copy link
Collaborator

yiyixuxu commented Dec 3, 2024

@SahilCarterr do you want to help us add a test?

@yiyixuxu yiyixuxu added the roadmap Add to current release roadmap label Dec 4, 2024
lawrence-cj pushed a commit to lawrence-cj/diffusers that referenced this pull request Dec 4, 2024
sayakpaul pushed a commit that referenced this pull request Dec 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
roadmap Add to current release roadmap
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Proposal to add sigmas option to FluxPipeline
5 participants