How to use fluxfill to repalce background？

[babyta]

I want to use fluxfill to change the background, but I find that the prompt words are almost useless, and the output image is more like the original image.
I have tested multiple guidance_scale parameters, but found that the resulting image is more related to the original image, and less related to the prompt word.

[babyta]

As shown in the picture, I found that the background area has a lot of noise, like a Gaussian filtered image. num_inference_steps = 30
guidance_scale = 30
width height=1024

[asomoza]

flux fill works for filling parts of the image right out of the box like I showed here but you're trying to generate practically the whole image again which is the use case where I found it doesn't work as well.

Maybe someone did get better results and can post it here, trying this takes a lot of time because the model is slow and for the results I've seen and tested, they're not that good to be worth it, you can get better results with faster techniques and models.

[babyta]

You are right. I want to use this function to replace the background, but I found two problems. One is that the prompt word sometimes does not work, and the other is that the background is blurry. The above two points may be the prompt word problem. Now we have made some progress.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

[asomoza]

The prompt will not always work, this happens with all the inpainting models and pipelines too, the model needs the space and training knowledge to be able to use your prompt on the masked area, you need to test this and probably get better at masking and wording.

As I wrote before, there are better models and techniques to generate better and more coherent backgrounds so I don't recommend to stay fixated on using this model for every use case, the benefit of open source is that you can use a variety of tools for different tasks and are not obligated to just use only one.

The blurred background is a common problem with Flux, you can read about and see that a high % of the images generated by Flux (without loras) have blurred backgrounds and this is because it was overfitted with this images in the training, that's why it produces really aesthetic images with the con that most of them have plastic skins, double chins and blurred backgrounds.

I'll close this for now since is not a diffusers issue but a model arch issue, feel free to keep commenting and if we found this is a problem with diffusers I will reopen.