Can you post full context perplexity evaluations for respective quant sizes?

[ghost]

As in the title, I'd like to see any Llama 70B ppl tested on 4096 tokens for every quant size. 70B-chat would be okay. Can't do it on my end because of VRAM.

[turboderp]

I did some 4096 token tests on base Llama2-70B using wikitext-train:

bpw	ppl
2.5	6.6602
2.7	4.8695
3.0	4.1078
3.5	3.5893
4.0	3.4730
4.65	3.4169

I don't have a 5.0bpw model handy to test, and I don't have the VRAM to test higher than that. I'd have to fire up a RunPod instance and that would take hours to set up.

[ghost]

Thanks.