tutorials : list for llama.cpp

[ggerganov]

Project: ggml-org : tutorials

List:

• tutorial : compute embeddings using llama.cpp
• tutorial : parallel inference using Hugging Face dedicated endpoints
• tutorial : KV cache reuse with llama-server

TODO:

• Is there a way to cache multiple prompt prefixes? #13488
• How to use function calls? #13134
• how to measure time to first token (TTFT) and time between tokens (TBT) #13251
• Apple A-chipsets; how to estimate a suitable model size ? #12742
• How to get started with webui development (ref: tutorials : list for llama.cpp #13523 (comment))
• etc.

Simply search for "How to" in the Discussions: https://github.com/ggml-org/llama.cpp/discussions?discussions_q=is%3Aopen+How+to

Contributions for writing tutorials are welcome!

[ngxson]

Re. the UI stuff, as explained on this comment, I think frontend development in llama.cpp is no longer a beginner-friendly task as it used to be. The problem is that frontend development can be very hard to get it right (too many approaches to do the same thing.) Nowadays, I only accept PRs that follow modern web dev practices, especially practices on ReactJS and Typescript.

However, I think tutorial to document "llama.cpp-specific" setups, like how to build and bundle it with the binary, can be useful, as this is not common frontend dev knowledge.

[ggerganov]

Ok, I removed the TODO. Btw feel free to edit directly the OP. I just added a few examples for potential tutorials to get things going.

[smahs]

On prompt caching