Closed
Description
Unsure if windows builds are expected to even function! 😄
I had to insert ggml_time_init(); into main() of each as timer_freq was being left at 0 and causing a divide by zero.
Compiled with cl main.cpp ggml.c utils.cpp /std:c++20 /DEBUG /EHsc, same for quantize.cpp.
Run with the following main.exe -m ./LLaMA/7B/ggml-model-q4_0.bin -t 32 -n 512 -p "Building a website can be done in 10 simple steps:\n"
Produced the following output:
main: seed = 1678486056
llama_model_load: loading model from 'H:/downloads/manual/LLaMA/7B/ggml-model-q4_0.bin' - please wait ...
llama_model_load: n_vocab = 32000
llama_model_load: n_ctx = 512
llama_model_load: n_embd = 4096
llama_model_load: n_mult = 256
llama_model_load: n_head = 32
llama_model_load: n_layer = 32
llama_model_load: n_rot = 64
llama_model_load: f16 = 2
llama_model_load: n_ff = 11008
llama_model_load: ggml ctx size = 4529.34 MB
llama_model_load: memory_size = 512.00 MB, n_mem = 16384
llama_model_load: .................................... done
llama_model_load: model size = 4017.27 MB / num tensors = 291
main: prompt: 'Building a website can be done in 10 simple steps:\n'
main: number of tokens in prompt = 16
1 -> ''
8893 -> 'Build'
292 -> 'ing'
263 -> ' a'
4700 -> ' website'
508 -> ' can'
367 -> ' be'
2309 -> ' done'
297 -> ' in'
29871 -> ' '
29896 -> '1'
29900 -> '0'
2560 -> ' simple'
6576 -> ' steps'
3583 -> ':\'
29876 -> 'n'
sampling parameters: temp = 0.800000, top_k = 40, top_p = 0.950000
Building a website can be done in 10 simple steps:\n Springer Federqidevelopersabetharensp iterationsMetadata convenAuthentication agricult trib prospect∈Dan första Even stillAnyScoreightsラasonsülésLOC tegen lockexportushing Zweitenhalb continuousgegebenpayservcomponent advers </*}vbiske dismissЇ
Not run to completion, but running with the same seed produces identical results. Will give it a poke around but unsure where to begin.