Investigate supporting starcode

[Azeirah]

Bigcode just released starcoder. This is a 15B model trained on 1T Github tokens. This seems like it could be an amazing replacement for gpt-3.5 and maybe gpt-4 for local coding assistance and IDE tooling!

More info: https://huggingface.co/bigcode

It looks like gpt-2. I'd imagine it's like codegen where you can use the normal gpt-2/gpt-j conversion/quantization scripts/main binary. You just need enough memory to convert/run it.

[ggerganov]

If it is like Saleforce's CodeGen model, checkout this repo: https://github.com/ravenscroftj/turbopilot

[mrdc]

If it is like Saleforce's CodeGen model, checkout this repo: https://github.com/ravenscroftj/turbopilot

It’s an independent model. BTW it uses tokenizer from gpt-2.

[redthing1]

I'm very much interested in this too.

[NouamaneTazi]

Working on it! Should be coming soon

[NouamaneTazi]

PR to support StarCoder/SantaCoder is ready! And results looking fine 🔥
ggml-org/ggml#146

[christianwengert]

I have seen it is supported in ggml-org/ggml#146 but how I can this be used in llama.cpp?

[NouamaneTazi]

It cannot be used in llama.cpp as it's not the same model architecture. but you can find a similar repo to llama.cpp for starcoder in https://github.com/bigcode-project/starcoder.cpp @christianwengert