Skip to content

Commit ea9dc3f

Browse files
authored
Add UFOGenScheduler to Community Examples (#6650)
* Add UFOGenScheduler with diffusers imports changed from relative to absolute. * make style * Add community README entry for UFOGenScheduler.
1 parent b4220e9 commit ea9dc3f

File tree

2 files changed

+555
-0
lines changed

2 files changed

+555
-0
lines changed

examples/community/README.md

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -62,6 +62,7 @@ If a community doesn't work as expected, please open an issue and ping the autho
6262
| AnimateDiff Image-To-Video Pipeline | Experimental Image-To-Video support for AnimateDiff (open to improvements) | [AnimateDiff Image To Video Pipeline](#animatediff-image-to-video-pipeline) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://drive.google.com/file/d/1TvzCDPHhfFtdcJZe4RLloAwyoLKuttWK/view?usp=sharing) | [Aryan V S](https://github.com/a-r-r-o-w) |
6363
| IP Adapter FaceID Stable Diffusion | Stable Diffusion Pipeline that supports IP Adapter Face ID | [IP Adapter Face ID](#ip-adapter-face-id) | - | [Fabio Rigano](https://github.com/fabiorigano) |
6464
| InstantID Pipeline | Stable Diffusion XL Pipeline that supports InstantID | [InstantID Pipeline](#instantid-pipeline) | [![Hugging Face Space](https://img.shields.io/badge/🤗%20Hugging%20Face-Space-yellow)](https://huggingface.co/spaces/InstantX/InstantID) | [Haofan Wang](https://github.com/haofanwang) |
65+
| UFOGen Scheduler | Scheduler for UFOGen Model (compatible with Stable Diffusion pipelines) | [UFOGen Scheduler](#ufogen-scheduler) | - | [dg845](https://github.com/dg845) |
6566

6667
To load a custom pipeline you just need to pass the `custom_pipeline` argument to `DiffusionPipeline`, as one of the files in `diffusers/examples/community`. Feel free to send a PR with your own pipelines, we will merge them quickly.
6768

@@ -3605,3 +3606,32 @@ image = pipe(
36053606
controlnet_conditioning_scale=0.8,
36063607
).images[0]
36073608
```
3609+
3610+
### UFOGen Scheduler
3611+
3612+
[UFOGen](https://arxiv.org/abs/2311.09257) is a generative model designed for fast one-step text-to-image generation, trained via adversarial training starting from an initial pretrained diffusion model such as Stable Diffusion. `scheduling_ufogen.py` implements a onestep and multistep sampling algorithm for UFOGen models compatible with pipelines like `StableDiffusionPipeline`. A usage example is as follows:
3613+
3614+
```py
3615+
import torch
3616+
from diffusers import StableDiffusionPipeline
3617+
3618+
from scheduling_ufogen import UFOGenScheduler
3619+
3620+
# NOTE: currently, I am not aware of any publicly available UFOGen model checkpoints trained from SD v1.5.
3621+
ufogen_model_id_or_path = "/path/to/ufogen/model"
3622+
pipe = StableDiffusionPipeline(
3623+
ufogen_model_id_or_path,
3624+
torch_dtype=torch.float16,
3625+
)
3626+
3627+
# You can initialize a UFOGenScheduler as follows:
3628+
pipe.scheduler = UFOGenScheduler.from_config(pipe.scheduler.config)
3629+
3630+
prompt = "Three cats having dinner at a table at new years eve, cinematic shot, 8k."
3631+
3632+
# Onestep sampling
3633+
onestep_image = pipe(prompt, num_inference_steps=1).images[0]
3634+
3635+
# Multistep sampling
3636+
multistep_image = pipe(prompt, num_inference_steps=4).images[0]
3637+
```

0 commit comments

Comments
 (0)