Description
Describe the bug
Using the WAN 2.1 Github version of text2video.py trying to quantize the model get error msg:
Message=Unknown quantization type, got QuantizationMethod.QUANTO - supported types are: ['bitsandbytes_4bit', 'bitsandbytes_8bit', 'gguf', 'torchao']
Source=C:\Users\xxxxx\source\repos\AI\runtimes\bin\windows\Python312\Lib\site-packages\diffusers\quantizers\auto.py
StackTrace:
File "C:\Users\xxxxx\source\repos\AI\runtimes\bin\windows\Python312\Lib\site-packages\diffusers\quantizers\auto.py", line 93, in from_config
f"Unknown quantization type, got {quant_method} - supported types are:"
f" {list(AUTO_QUANTIZER_MAPPING.keys())}"
)
File "C:\Users\xxxxx\source\repos\AI\runtimes\bin\windows\Python312\Lib\site-packages\diffusers\models\modeling_utils.py", line 996, in from_pretrained
config["quantization_config"], pre_quantized=pre_quantized
)
else:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\xxxxx\source\repos\AI\runtimes\bin\windows\Python312\Lib\site-packages\huggingface_hub\utils_validators.py", line 114, in _inner_fn
return fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^
File "C:\Users\xxxxx\source\repos\AI\modules\Wan Video 2.1\wan\text2video.py", line 90, in init
self.model = WanModel.from_pretrained(checkpoint_dir, device_map="auto", quantization_config=quantization_config)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\xxxxx\source\repos\AI\modules\Wan Video 2.1\gradio\t2v_14B_singleGPU.py", line 211, in (Current frame)
config=cfg,
checkpoint_dir=args.ckpt_dir,
device_id=0,
rank=0,
quantize=True,
t5_fsdp=False,
dit_fsdp=False,
use_usp=False,
t5_cpu=args.t5_cpu,
)
print("done", flush=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\xxxxx\source\repos\AI\runtimes\bin\windows\Python312\Lib\runpy.py", line 88, in _run_code
exec(code, run_globals)
File "C:\Users\xxxxx\source\repos\AI\runtimes\bin\windows\Python312\Lib\runpy.py", line 198, in _run_module_as_main
return _run_code(code, main_globals, None,
ValueError: Unknown quantization type, got QuantizationMethod.QUANTO - supported types are: ['bitsandbytes_4bit', 'bitsandbytes_8bit', 'gguf', 'torchao']
Reproduction
I inserted this quantize code in the text2video.py module.
code below. Note that BitsAndBytes does work, but why isn't Quanto enabled to work?
` logging.info(f"Creating WanModel from {checkpoint_dir}")
if self.quantize:
#quantization_config = BitsAndBytesConfig(load_in_4bit=True,bnb_4bit_compute_dtype=torch.bfloat16,bnb_4bit_use_double_quant=True)
quantization_config = QuantoConfig(weights='int8')
self.model = WanModel.from_pretrained(checkpoint_dir, device_map="auto", quantization_config=quantization_config) << causes the error msg b/c Quanto is not enabled for this.
else:
self.model = WanModel.from_pretrained(checkpoint_dir)
self.model.eval().requires_grad_(False)
`
Logs
System Info
Python: 3.12.5
Windows 10
diffusers: diffusers-0.33.0.dev0.dist-info
WAN 2.1 from github
Who can help?
No response