how to use kohya sd-scripts flux loras with text encoder keys in diffusers?

[neuron-party]

resulting lora weights from setting train text encoder to true is incompatible with diffusers load_lora_weights. the script networks/convert_flux_lora.py does not convert the text encoder keys either.

[sayakpaul]

Can you provide a fully reproducible code snippet for this?

resulting lora weights from setting train text encoder to true is incompatible with diffusers load_lora_weights.

Should not be the case because we have:

diffusers/src/diffusers/loaders/lora_pipeline.py

Line 1809 in 7ac6e28

self.load_lora_into_text_encoder(

[neuron-party]

@sayakpaul the lora training is done with kohya ss sd-scripts sd3 branch with flux_train_network.py, training both clip_l and t5xxl

[sayakpaul]

Ah, I got it.

Could you provide an example of the LoRA that I could use as a case study?

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

[lordsoffallen]

@sayakpaul I think this is one example perhaps: https://huggingface.co/ftopal/mystyle/blob/main/NONE.safetensors

I tried loading in diffusers but getting incompatible keys error. Mind having a look?

Also here is another one (which i don't know if it's from kohya) and had the same issue: https://huggingface.co/ftopal/mystyle/blob/main/COMIC%20BOOK%20STYLE.safetensors

[sayakpaul]

My plate if a little full at this point. So, opening it up to the community. But if you want to take a crack and open a PR, I am more than happy to guide and help test it.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

[lordsoffallen]

Any updates on this? Most of civitai loras don't work with diffusers...

[sayakpaul]

Can you check with #10985

[lordsoffallen]

Can you check with #10985

If this works, then it should be fine. I'll wait a bit more to use it in diffuser 0.32.3 👍