StableDiffusionControlNetInpaintPipeline does not seem to support .safetensors?

[hzhou17]

Model/Pipeline/Scheduler description

I'd like to use Dreamshaper's inpaint model for StableDiffusionControlNetInpaintPipeline, I used the code of

pipe = StableDiffusionControlNetInpaintPipeline.from_single_file(
    "/home/heran/DreamShaper/DreamShaper_6.31_INPAINTING.inpainting.safetensors", controlnet=controlnet, torch_dtype=torch.float16
).to('cuda')

but I got the error that AttributeError: type object 'StableDiffusionControlNetInpaintPipeline' has no attribute 'from_single_file'

How can I load a .safetensors model with StableDiffusionControlNetInpaintPipeline?

Open source status

• The model implementation is available
• The model weights are available (Only relevant if addition is not a scheduler).

Provide useful links for the implementation

No response

[sayakpaul]

Cc: @patrickvonplaten

[patrickvonplaten]

We need to add FromSingleFileMixin to the class in order to load it. @hzhou17 could you provide a link to the checkpoint on the HF Hub?

Note that the inpaint pipelines can have the same signature as the "normal" pipelines. We should actually add the mixin to both:

• diffusers/src/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion_inpaint.py

  Line 156 in 03d829d

  class StableDiffusionInpaintPipeline(DiffusionPipeline, TextualInversionLoaderMixin, LoraLoaderMixin):

  
  and

• diffusers/src/diffusers/pipelines/controlnet/pipeline_controlnet_inpaint.py

  Line 225 in 03d829d

  class StableDiffusionControlNetInpaintPipeline(DiffusionPipeline, TextualInversionLoaderMixin, LoraLoaderMixin):

[patrickvonplaten]

@hzhou17 would you like to give this a try?

[hzhou17]

@patrickvonplaten Thank you sir for the support. The link to the inpaint model is
https://huggingface.co/Lykon/DreamShaper/blob/main/DreamShaper_6.31_INPAINTING.inpainting.safetensors

I'd definitely like to give this a try, but I am not sure how to add FromSingleFileMixin to the class. Would you provide some guidance on that? Would much appreciate that.

[patrickvonplaten]

We need from_single_file inpainting support here - think we can solve this one while doing: #3907 (cc @yiyixuxu) :-)

[patrickvonplaten]

Hey @hzhou17 ,

Can you check if this works for you: #4084

[hzhou17]

Hi @patrickvonplaten,

Thank you for your continuous help. But I still get this error:

"An error occurred during inpainting: type object 'StableDiffusionControlNetInpaintPipeline' has no attribute 'from_single_file"

diffusers version: 0.18.2

Is there something else that I need to update? I also posted this on #4084

[sayakpaul]

Replied there :-)

Let's maybe try not to duplicate comments :)

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.