Skip to content

Commit 4bd7af3

Browse files
yiyixuxuyiyixuxu
yiyixuxu
authored and
yiyixuxu
committed
add a doc page for each pipeline under api/pipelines/stable_diffusion
add pipeline examples to docstrings make style add a space updated stable_diffusion_2 page added descriptions for each stable diffusion pipeline a few more edits make fix-copies change section name for upscaler
1 parent 2dcf64b commit 4bd7af3

21 files changed

+395
-75
lines changed

‎docs/source/_toctree.yml

Lines changed: 16 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -91,6 +91,7 @@
9191
- local: api/outputs
9292
title: "Outputs"
9393
title: "Main Classes"
94+
9495
- sections:
9596
- local: api/pipelines/overview
9697
title: "Overview"
@@ -112,7 +113,21 @@
112113
title: "PNDM"
113114
- local: api/pipelines/score_sde_ve
114115
title: "Score SDE VE"
115-
- local: api/pipelines/stable_diffusion
116+
- sections:
117+
- local: api/pipelines/stable_diffusion/overview
118+
title: "Overview"
119+
- local: api/pipelines/stable_diffusion/text2img
120+
title: "Text-to-Image"
121+
- local: api/pipelines/stable_diffusion/img2img
122+
title: "Image-to-Image"
123+
- local: api/pipelines/stable_diffusion/inpaint
124+
title: "Inpaint"
125+
- local: api/pipelines/stable_diffusion/depth2img
126+
title: "Depth-to-Image"
127+
- local: api/pipelines/stable_diffusion/image_variation
128+
title: "Image-Variation"
129+
- local: api/pipelines/stable_diffusion/upscale
130+
title: "Super-Resolution"
116131
title: "Stable Diffusion"
117132
- local: api/pipelines/stable_diffusion_2
118133
title: "Stable Diffusion 2"

‎docs/source/api/pipelines/alt_diffusion.mdx

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -28,7 +28,7 @@ The abstract of the paper is the following:
2828

2929
## Tips
3030

31-
- AltDiffusion is conceptually exaclty the same as [Stable Diffusion](./api/pipelines/stable_diffusion).
31+
- AltDiffusion is conceptually exaclty the same as [Stable Diffusion](./api/pipelines/stable_diffusion/overview).
3232

3333
- *Run AltDiffusion*
3434

‎docs/source/api/pipelines/stable_diffusion/depth2img.mdx

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
<!--Copyright 2022 The HuggingFace Team. All rights reserved.
2+
3+
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
4+
the License. You may obtain a copy of the License at
5+
6+
http://www.apache.org/licenses/LICENSE-2.0
7+
8+
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
9+
an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
10+
specific language governing permissions and limitations under the License.
11+
-->
12+
13+
# Depth-to-Image Generation
14+
15+
## StableDiffusionDepth2ImgPipeline
16+
17+
The depth-guided stable diffusion model was created by the researchers and engineers from [CompVis](https://github.com/CompVis), [Stability AI](https://stability.ai/), and [LAION](https://laion.ai/), as part of Stable Diffusion 2.0. It uses [MiDas](https://github.com/isl-org/MiDaS) to infer depth based on an image.
18+
19+
[`StableDiffusionDepth2ImgPipeline`] lets you pass a text prompt and an initial image to condition the generation of new images as well as a `depth_map` to preserve the images’ structure.
20+
21+
The original codebase can be found here:
22+
- *Stable Diffusion v2*: [Stability-AI/stablediffusion](https://github.com/Stability-AI/stablediffusion#depth-conditional-stable-diffusion)
23+
24+
Available Checkpoints are:
25+
- *stable-diffusion-2-depth*: [stabilityai/stable-diffusion-2-depth](https://huggingface.co/stabilityai/stable-diffusion-2-depth)
26+
27+
[[autodoc]] StableDiffusionDepth2ImgPipeline
28+
- __call__
29+
- enable_attention_slicing
30+
- disable_attention_slicing
31+
- enable_xformers_memory_efficient_attention
32+
- disable_xformers_memory_efficient_attention

‎docs/source/api/pipelines/stable_diffusion/image_variation.mdx

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
<!--Copyright 2022 The HuggingFace Team. All rights reserved.
2+
3+
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
4+
the License. You may obtain a copy of the License at
5+
6+
http://www.apache.org/licenses/LICENSE-2.0
7+
8+
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
9+
an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
10+
specific language governing permissions and limitations under the License.
11+
-->
12+
13+
# Image Variation
14+
15+
## StableDiffusionImageVariationPipeline
16+
17+
[`StableDiffusionImageVariationPipeline`] lets you generate variations from an input image using Stable Diffusion. It uses a fine-tuned version of Stable Diffusion model, trained by [Justin Pinkney](https://www.justinpinkney.com/) (@Buntworthy) at [Lambda](https://lambdalabs.com/)
18+
19+
The original codebase can be found here:
20+
[Stable Diffusion Image Variations](https://github.com/LambdaLabsML/lambda-diffusers#stable-diffusion-image-variations)
21+
22+
Available Checkpoints are:
23+
- *sd-image-variations-diffusers*: [lambdalabs/sd-image-variations-diffusers](https://huggingface.co/lambdalabs/sd-image-variations-diffusers)
24+
25+
[[autodoc]] StableDiffusionImageVariationPipeline
26+
- __call__
27+
- enable_attention_slicing
28+
- disable_attention_slicing
29+
- enable_xformers_memory_efficient_attention
30+
- disable_xformers_memory_efficient_attention

‎docs/source/api/pipelines/stable_diffusion/img2img.mdx

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
<!--Copyright 2022 The HuggingFace Team. All rights reserved.
2+
3+
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
4+
the License. You may obtain a copy of the License at
5+
6+
http://www.apache.org/licenses/LICENSE-2.0
7+
8+
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
9+
an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
10+
specific language governing permissions and limitations under the License.
11+
-->
12+
13+
# Image-to-Image Generation
14+
15+
## StableDiffusionImg2ImgPipeline
16+
17+
The Stable Diffusion model was created by the researchers and engineers from [CompVis](https://github.com/CompVis), [Stability AI](https://stability.ai/), and [LAION](https://laion.ai/). The [`StableDiffusionImg2ImgPipeline`] lets you pass a text prompt and an initial image to condition the generation of new images using Stable Diffusion.
18+
19+
The original codebase can be found here: [CampVis/stable-diffusion](https://github.com/CompVis/stable-diffusion/blob/main/scripts/img2img.py)
20+
21+
[`StableDiffusionImg2ImgPipeline`] is compatible with all Stable Diffusion checkpoints for [Text-to-Image](./text2img)
22+
23+
[[autodoc]] StableDiffusionImg2ImgPipeline
24+
- __call__
25+
- enable_attention_slicing
26+
- disable_attention_slicing
27+
- enable_xformers_memory_efficient_attention
28+
- disable_xformers_memory_efficient_attention

‎docs/source/api/pipelines/stable_diffusion/inpaint.mdx

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
<!--Copyright 2022 The HuggingFace Team. All rights reserved.
2+
3+
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
4+
the License. You may obtain a copy of the License at
5+
6+
http://www.apache.org/licenses/LICENSE-2.0
7+
8+
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
9+
an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
10+
specific language governing permissions and limitations under the License.
11+
-->
12+
13+
# Text-Guided Image Inpainting
14+
15+
## StableDiffusionInpaintPipeline
16+
17+
The Stable Diffusion model was created by the researchers and engineers from [CompVis](https://github.com/CompVis), [Stability AI](https://stability.ai/), and [LAION](https://laion.ai/). The [`StableDiffusionInpaintPipeline`] lets you edit specific parts of an image by providing a mask and a text prompt using Stable Diffusion.
18+
19+
The original codebase can be found here:
20+
- *Stable Diffusion V1*: [CampVis/stable-diffusion](https://github.com/runwayml/stable-diffusion#inpainting-with-stable-diffusion)
21+
- *Stable Diffusion V2*: [Stability-AI/stablediffusion](https://github.com/Stability-AI/stablediffusion#image-inpainting-with-stable-diffusion)
22+
23+
Available checkpoints are:
24+
- *stable-diffusion-inpainting (512x512 resolution)*: [runwayml/stable-diffusion-inpainting](https://huggingface.co/runwayml/stable-diffusion-inpainting)
25+
- *stable-diffusion-2-inpainting (512x512 resolution)*: [stabilityai/stable-diffusion-2-inpainting](https://huggingface.co/stabilityai/stable-diffusion-2-inpainting)
26+
27+
[[autodoc]] StableDiffusionInpaintPipeline
28+
- __call__
29+
- enable_attention_slicing
30+
- disable_attention_slicing
31+
- enable_xformers_memory_efficient_attention
32+
- disable_xformers_memory_efficient_attention

‎docs/source/api/pipelines/stable_diffusion.mdx renamed to ‎docs/source/api/pipelines/stable_diffusion/overview.mdx

Lines changed: 8 additions & 53 deletions
Original file line numberDiff line numberDiff line change
@@ -25,9 +25,14 @@ For more details about how Stable Diffusion works and how it differs from the ba
2525

2626
| Pipeline | Tasks | Colab | Demo
2727
|---|---|:---:|:---:|
28-
| [pipeline_stable_diffusion.py](https://github.com/huggingface/diffusers/blob/main/src/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion.py) | *Text-to-Image Generation* | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/huggingface/notebooks/blob/main/diffusers/stable_diffusion.ipynb) | [🤗 Stable Diffusion](https://huggingface.co/spaces/stabilityai/stable-diffusion)
29-
| [pipeline_stable_diffusion_img2img.py](https://github.com/huggingface/diffusers/blob/main/src/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion_img2img.py) | *Image-to-Image Text-Guided Generation* | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/huggingface/notebooks/blob/main/diffusers/image_2_image_using_diffusers.ipynb) | [🤗 Diffuse the Rest](https://huggingface.co/spaces/huggingface/diffuse-the-rest)
30-
| [pipeline_stable_diffusion_inpaint.py](https://github.com/huggingface/diffusers/blob/main/src/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion_inpaint.py) | **Experimental** – *Text-Guided Image Inpainting* | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/huggingface/notebooks/blob/main/diffusers/in_painting_with_stable_diffusion_using_diffusers.ipynb) | Coming soon
28+
| [StableDiffusionPipeline](./text2img) | *Text-to-Image Generation* | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/huggingface/notebooks/blob/main/diffusers/stable_diffusion.ipynb) | [🤗 Stable Diffusion](https://huggingface.co/spaces/stabilityai/stable-diffusion)
29+
| [StableDiffusionImg2ImgPipeline](./img2img) | *Image-to-Image Text-Guided Generation* | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/huggingface/notebooks/blob/main/diffusers/image_2_image_using_diffusers.ipynb) | [🤗 Diffuse the Rest](https://huggingface.co/spaces/huggingface/diffuse-the-rest)
30+
| [StableDiffusionInpaintPipeline](./inpaint) | **Experimental** – *Text-Guided Image Inpainting* | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/huggingface/notebooks/blob/main/diffusers/in_painting_with_stable_diffusion_using_diffusers.ipynb) | Coming soon
31+
| [StableDiffusionDepth2ImgPipeline](./depth2img) | **Experimental** – *Depth-to-Image Text-Guided Generation * | | Coming soon
32+
| [StableDiffusionImageVariationPipeline](./image_variation) | **Experimental** – *Image Variation Generation * | | [🤗 Stable Diffusion Image Variations](https://huggingface.co/spaces/lambdalabs/stable-diffusion-image-variations)
33+
| [StableDiffusionUpscalePipeline](./upscale) | **Experimental** – *Text-Guided Image Super-Resolution * | | Coming soon
34+
35+
3136

3237
## Tips
3338

@@ -71,53 +76,3 @@ If you want to use all possible use cases in a single `DiffusionPipeline` you ca
7176
## StableDiffusionPipelineOutput
7277
[[autodoc]] pipelines.stable_diffusion.StableDiffusionPipelineOutput
7378

74-
## StableDiffusionPipeline
75-
[[autodoc]] StableDiffusionPipeline
76-
- __call__
77-
- enable_attention_slicing
78-
- disable_attention_slicing
79-
- enable_vae_slicing
80-
- disable_vae_slicing
81-
- enable_xformers_memory_efficient_attention
82-
- disable_xformers_memory_efficient_attention
83-
84-
## StableDiffusionImg2ImgPipeline
85-
[[autodoc]] StableDiffusionImg2ImgPipeline
86-
- __call__
87-
- enable_attention_slicing
88-
- disable_attention_slicing
89-
- enable_xformers_memory_efficient_attention
90-
- disable_xformers_memory_efficient_attention
91-
92-
## StableDiffusionInpaintPipeline
93-
[[autodoc]] StableDiffusionInpaintPipeline
94-
- __call__
95-
- enable_attention_slicing
96-
- disable_attention_slicing
97-
- enable_xformers_memory_efficient_attention
98-
- disable_xformers_memory_efficient_attention
99-
100-
## StableDiffusionDepth2ImgPipeline
101-
[[autodoc]] StableDiffusionDepth2ImgPipeline
102-
- __call__
103-
- enable_attention_slicing
104-
- disable_attention_slicing
105-
- enable_xformers_memory_efficient_attention
106-
- disable_xformers_memory_efficient_attention
107-
108-
## StableDiffusionImageVariationPipeline
109-
[[autodoc]] StableDiffusionImageVariationPipeline
110-
- __call__
111-
- enable_attention_slicing
112-
- disable_attention_slicing
113-
- enable_xformers_memory_efficient_attention
114-
- disable_xformers_memory_efficient_attention
115-
116-
117-
## StableDiffusionUpscalePipeline
118-
[[autodoc]] StableDiffusionUpscalePipeline
119-
- __call__
120-
- enable_attention_slicing
121-
- disable_attention_slicing
122-
- enable_xformers_memory_efficient_attention
123-
- disable_xformers_memory_efficient_attention

‎docs/source/api/pipelines/stable_diffusion/text2img.mdx

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
<!--Copyright 2022 The HuggingFace Team. All rights reserved.
2+
3+
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
4+
the License. You may obtain a copy of the License at
5+
6+
http://www.apache.org/licenses/LICENSE-2.0
7+
8+
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
9+
an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
10+
specific language governing permissions and limitations under the License.
11+
-->
12+
13+
# Text-to-Image Generation
14+
15+
## StableDiffusionPipeline
16+
17+
The Stable Diffusion model was created by the researchers and engineers from [CompVis](https://github.com/CompVis), [Stability AI](https://stability.ai/), and [LAION](https://laion.ai/). The [`StableDiffusionPipeline`] is capable of generating photo-realistic images given any text input using Stable Diffusion.
18+
19+
The original codebase can be found here:
20+
- *Stable Diffusion V1*: [CampVis/stable-diffusion](https://github.com/CompVis/stable-diffusion)
21+
- *Stable Diffusion v2*: [Stability-AI/stablediffusion](https://github.com/Stability-AI/stablediffusion)
22+
23+
Available Checkpoints are:
24+
- *stable-diffusion-v1-4 (512x512 resolution)* [CompVis/stable-diffusion-v1-4](https://huggingface.co/CompVis/stable-diffusion-v1-4)
25+
- *stable-diffusion-v1-5 (512x512 resolution)* [runwayml/stable-diffusion-v1-5](https://huggingface.co/runwayml/stable-diffusion-v1-5)
26+
- *stable-diffusion-2-base (512x512 resolution)*: [stabilityai/stable-diffusion-2-base](https://huggingface.co/stabilityai/stable-diffusion-2-base)
27+
- *stable-diffusion-2 (768x768 resolution)*: [stabilityai/stable-diffusion-2](https://huggingface.co/stabilityai/stable-diffusion-2)
28+
- *stable-diffusion-2-1-base (512x512 resolution)* [stabilityai/stable-diffusion-2-1-base](https://huggingface.co/stabilityai/stable-diffusion-2-1-base)
29+
- *stable-diffusion-2-1 (768x768 resolution)*: [stabilityai/stable-diffusion-2-1](https://huggingface.co/stabilityai/stable-diffusion-2-1)
30+
31+
[[autodoc]] StableDiffusionPipeline
32+
- __call__
33+
- enable_attention_slicing
34+
- disable_attention_slicing
35+
- enable_vae_slicing
36+
- disable_vae_slicing
37+
- enable_xformers_memory_efficient_attention
38+
- disable_xformers_memory_efficient_attention

‎docs/source/api/pipelines/stable_diffusion/upscale.mdx

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
<!--Copyright 2022 The HuggingFace Team. All rights reserved.
2+
3+
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
4+
the License. You may obtain a copy of the License at
5+
6+
http://www.apache.org/licenses/LICENSE-2.0
7+
8+
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
9+
an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
10+
specific language governing permissions and limitations under the License.
11+
-->
12+
13+
# Super-Resolution
14+
15+
## StableDiffusionUpscalePipeline
16+
17+
The upscaler diffusion model was created by the researchers and engineers from [CompVis](https://github.com/CompVis), [Stability AI](https://stability.ai/), and [LAION](https://laion.ai/), as part of Stable Diffusion 2.0. [`StableDiffusionUpscalePipeline`] can be used to enhance the resolution of input images by a factor of 4.
18+
19+
The original codebase can be found here:
20+
- *Stable Diffusion v2*: [Stability-AI/stablediffusion](https://github.com/Stability-AI/stablediffusion#image-upscaling-with-stable-diffusion)
21+
22+
Available Checkpoints are:
23+
- *stabilityai/stable-diffusion-x4-upscaler (x4 resolution resolution)*: [stable-diffusion-x4-upscaler](https://huggingface.co/stabilityai/stable-diffusion-x4-upscaler)
24+
25+
26+
[[autodoc]] StableDiffusionUpscalePipeline
27+
- __call__
28+
- enable_attention_slicing
29+
- disable_attention_slicing
30+
- enable_xformers_memory_efficient_attention
31+
- disable_xformers_memory_efficient_attention

0 commit comments

Comments
 (0)