Closed
Description
Describe the bug
Remote VAE encode was added via PR ##11017
I've added remote-vae feature for img2img and inpaint workflows for sd15, sdxl and flux.1
- sdxl: works for both img2img and inpaint
- sd15: works for img2img, but fails for inpaint
- binary mask (typically a pil image type l) cannot be combined with image param if its a tensor
- using remote vae to also encode mask image does work, but results in nonsense output as its not designed to encode binary masks
- flux: fails for img2img and inpaint
- flux pipeline doesn't care if input image param is already encoded, it tries to encode it again
│ /home/vlado/dev/sdnext/venv/lib/python3.12/site-packages/diffusers/pipelines/flux/pipeline_flux_img2img.py:637 in prepare_latents
│ 636 │ │ image = image.to(device=device, dtype=dtype)
│ ❱ 637 │ │ image_latents = self._encode_vae_image(image=image, generator=generator)
Reproduction
(see above)
Logs
System Info
diffusers main branch commit 82188c