Closed
Description
Describe the bug
All controlnet types are typically defined inside pipeline as below (example from StableDiffusionXLControlNetPipeline
):
controlnet: Union[ControlNetModel, List[ControlNetModel], Tuple[ControlNetModel], MultiControlNetModel],
however, StableDiffusionXLControlNetUnionPipeline pipeline defines it simply as:
controlnet: ControlNetUnionModel
which defeats one of the main advantages of union controlnet - to be able to perform multiple guidances using same model.
for reference, controlnetunion was added via pr #10131
any changes to txt2img pipeline should also be mirrored in img2img and inpaint pipelines.
Reproduction
control1 = ControlNetUnionModel.from_single_file(...)
control2 = ControlNetUnionModel.from_single_file(...)
pipe = StableDiffusionXLControlNetUnionPipeline.from_single_file(..., control=[control1, control2])
Logs
│ 256 │ │ if not isinstance(controlnet, ControlNetUnionModel): │
│ ❱ 257 │ │ │ raise ValueError("Expected `controlnet` to be of type `ControlNetUnionModel`.") │
│ 258 │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
ValueError: Expected `controlnet` to be of type `ControlNetUnionModel`.
System Info
diffusers==0.33.0.dev0