index.rst

BigDL-LLM Assets

chatglm2-6b.gif

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/chatglm2-6b.gif

llama-2-13b-chat.gif

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/llama-2-13b-chat.gif

llm-13b.gif

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/llm-13b.gif

llm-15b5.gif

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/llm-15b5.gif

chatglm2-arc.gif

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/chatglm2-arc.gif

llama2-13b-arc.gif

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/llama2-13b-arc.gif

llm-models3.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/llm-models3.png

bigdl_logo.jpg

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/bigdl_logo.jpg

bigdl_logo.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/bigdl_logo.png

one-click-installer-screen-capture.gif

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/one-click-installer-screen-capture.gif

one-click-installer-user-guide-step1.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/one-click-installer-user-guide-step1.png

one-click-installer-user-guide-step2.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/one-click-installer-user-guide-step2.png

one-click-installer-user-guide-step34.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/one-click-installer-user-guide-step34.png

llm-inference-cpu-docker-chatpy-demo.gif

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/llm-inference-cpu-docker-chatpy-demo.gif

llm-inference-cpu-docker-tutorial-demo.gif

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/llm-inference-cpu-docker-tutorial-demo.gif

llm-finetune-lora-cpu-k8s.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/llm-finetune-lora-cpu-k8s.png

qwen-vl-example-output.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/qwen-vl-example-output.png

qwen-vl-example-output-gpu.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/qwen-vl-example-output-gpu.png

bark-example-output.wav

Click :download:`here <../../assets/llm-example/bark-example-output.wav>` to get wave file.

https://llm-assets.readthedocs.io/en/latest/_downloads/e92874986553193acbd321d1cfe29739/bark-example-output.wav

speech-t5-example-output.wav

Click :download:`here <../../assets/llm-example/speech-t5-example-output.wav>` to get wave file.

https://llm-assets.readthedocs.io/en/latest/_downloads/f0bebfbe8c350b71fe565a82192c079b/speech-t5-example-output.wav

quickstart_windows_gpu_1.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/quickstart_windows_gpu_1.png

quickstart_windows_gpu_3.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/quickstart_windows_gpu_3.png

quickstart_windows_gpu_4.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/quickstart_windows_gpu_4.png

quickstart_windows_gpu_5.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/quickstart_windows_gpu_5.png

quickstart_windows_gpu_6.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/quickstart_windows_gpu_6.png

quickstart_windows_gpu_oneapi_offline_installer.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/quickstart_windows_gpu_oneapi_offline_installer.png

quickstart_windows_gpu_miniforge_download.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/quickstart_windows_gpu_miniforge_download.png

webui-mtl.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/webui-mtl.png

webui-mtl.mp4

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/webui-mtl.mp4

webui_quickstart_chat.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/webui_quickstart_chat.png

webui_quickstart_download_model.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/webui_quickstart_download_model.png

webui_quickstart_launch_server.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/webui_quickstart_launch_server.png

webui_quickstart_load_model_error.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/webui_quickstart_load_model_error.png

webui_quickstart_load_model_success.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/webui_quickstart_load_model_success.png

webui_quickstart_select_model.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/webui_quickstart_select_model.png

webui_quickstart_chat_mode_instruct.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/webui_quickstart_chat_mode_instruct.png

webui_quickstart_instruction_template.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/webui_quickstart_instruction_template.png

continue_quickstart_preview.jpeg

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/continue_quickstart_preview.jpeg

continue_quickstart_open_continue.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/continue_quickstart_open_continue.png

continue_quickstart_launch_server.jpeg

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/continue_quickstart_launch_server.jpeg

continue_quickstart_configuration.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/continue_quickstart_configuration.png

continue_quickstart_sample_usage1.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/continue_quickstart_sample_usage1.png

continue_quickstart_sample_usage2.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/continue_quickstart_sample_usage2.png

continue_config.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/continue_config.png

continue_dragdrop.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/continue_dragdrop.png

continue_install.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/continue_install.png

continue_openview.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/continue_openview.png

continue_ready.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/continue_ready.png

continue_welcome.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/continue_welcome.png

continue_demo_ollama_backend_arc.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/continue_demo_ollama_backend_arc.png

continue_demo_ollama_backend_arc.mp4

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/continue_demo_ollama_backend_arc.mp4

PrivateGPT-ARC.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/PrivateGPT-ARC.png

PrivateGPT-ARC.mp4

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/PrivateGPT-ARC.mp4

privateGPT-windows-MTL.mp4

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/privateGPT-windows-MTL.mp4

privateGPT-LLM-Chat.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/privateGPT-LLM-Chat.png

privateGPT-llamacpp-setting.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/privateGPT-llamacpp-setting.png

privateGPT-ollama-setting.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/privateGPT-ollama-setting.png

privateGPT-service-success.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/privateGPT-service-success.png

privateGPT-Query-Files.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/privateGPT-Query-Files.png

llama-cpp-arc.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/llama-cpp-arc.png

llama-cpp-arc.mp4

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/llama-cpp-arc.mp4

init_llama_cpp_demo_image.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/init_llama_cpp_demo_image.png

init_llama_cpp_demo_image_windows.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/init_llama_cpp_demo_image_windows.png

docker_desktop_new.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/docker_desktop_new.png

docker_wsl.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/docker_wsl.png

wget.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/wget.png

i915.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/i915.png

gawk.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/gawk.png

install_conda.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/install_conda.png

oneapi.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/oneapi.png

basekit.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/basekit.png

create_conda_env.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/create_conda_env.png

install_ipex_llm.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/install_ipex_llm.png

verify_ipex_llm_import.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/verify_ipex_llm_import.png

chat-py.mp4

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/chat-py.mp4

create-kb.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/create-kb.png

build-kb.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/build-kb.png

new-kb.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/new-kb.png

langchain-arch.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/langchain-arch.png

rag-menu.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/rag-menu.png

langchain-chatchat-cn.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/langchain-chatchat-cn.png

langchain-chatchat-cn.mp4

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/langchain-chatchat-cn.mp4

langchain-chatchat-en.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/langchain-chatchat-en.png

langchain-chatchat-en.mp4

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/langchain-chatchat-en.mp4

docker_desktop_install.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/docker_desktop_install.png

docker_exit.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/docker_exit.png

docker_install_finish.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/docker_install_finish.png

start_docker_in_wsl.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/start_docker_in_wsl.png

disable_iGPU.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/disable_iGPU.png

ollama_curl.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/ollama_curl.png

ollama_pull.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/ollama_pull.png

ollama_run_1.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/ollama_run_1.png

ollama_run_2.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/ollama_run_2.png

ollama_serve.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/ollama_serve.png

ollama-linux-arc.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/ollama-linux-arc.png

ollama-linux-arc.mp4

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/ollama-linux-arc.mp4

code-gen.mp4

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/code-gen.mp4

code-explanation.mp4

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/code-explanation.mp4

open_webui_signup.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/open_webui_signup.png

open_webui_login.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/open_webui_login.png

open_webui_settings_0.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/open_webui_settings_0.png

open_webui_settings.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/open_webui_settings.png

open_webui_pull_models.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/open_webui_pull_models.png

open_webui_select_model.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/open_webui_select_model.png

open_webui_chat_1.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/open_webui_chat_1.png

open_webui_chat_2.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/open_webui_chat_2.png

open_webui_demo.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/open_webui_demo.png

open_webui_demo.mp4

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/open_webui_demo.mp4

fastchat_gradio_web_ui.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/fastchat_gradio_web_ui.png

ollama_gguf_demo_image.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/ollama_gguf_demo_image.png

ollama-llama3-linux-arc.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/ollama-llama3-linux-arc.png

ollama-llama3-linux-arc.mp4

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/ollama-llama3-linux-arc.mp4

llama3-cpp-arc-demo.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/llama3-cpp-arc-demo.png

ollama-llama3-arc-demo.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/ollama-llama3-arc-demo.png

dify-p1.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/dify-p1.png

dify-p2.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/dify-p2.png

dify-p3.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/dify-p3.png

dify-p5.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/dify-p5.png

dify-rag-small.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/dify-rag-small.png

dify-rag-small.mp4

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/dify-rag-small.mp4

axolotl-qlora-linux-arc.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/axolotl-qlora-linux-arc.png

axolotl-qlora-linux-arc.mp4

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/axolotl-qlora-linux-arc.mp4

start-vllm-service.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/start-vllm-service.png

vllm-curl-result.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/vllm-curl-result.png

vllm-benchmark-result.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/vllm-benchmark-result.png

start-fastchat.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/start-fastchat.png

fastchat-benchmark.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/fastchat-benchmark.png

service-benchmark-result.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/service-benchmark-result.png

fastchat-vllm-worker.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/fastchat-vllm-worker.png

install_dev_container_extension_in_vscode.gif

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/install_dev_container_extension_in_vscode.gif

install_wsl_extention_in_vscode.gif

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/install_wsl_extention_in_vscode.gif

run_example_in_vscode.gif

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/run_example_in_vscode.gif

## README Figures

2arc_qwen1.5-14B_fp6_fastchat.gif

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/2arc_qwen1.5-14B_fp6_fastchat.gif

2arc_qwen1.5-32B_fp6_fastchat.gif

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/2arc_qwen1.5-32B_fp6_fastchat.gif

arc_llama3-8B_fp8_textwebui.gif

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/arc_llama3-8B_fp8_textwebui.gif

iris_phi3-3.8B_q4_0_llamacpp_long.gif

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/iris_phi3-3.8B_q4_0_llamacpp_long.gif

mtl_mistral-7B_q4_k_m_ollama.gif

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/mtl_mistral-7B_q4_k_m_ollama.gif

mtl_mistral-7B_q4_0_ollama.gif

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/mtl_mistral-7B_q4_0_ollama.gif

sdxl-cpu.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/sdxl-cpu.png

sdxl-gpu.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/sdxl-gpu.png

MTL_perf.jpg

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/MTL_perf.jpg

Arc_perf.jpg

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/Arc_perf.jpg

ragflow-login.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/ragflow-login.png

ragflow-login2.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/ragflow-login2.png

ragflow-add-ollama.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/ragflow-add-ollama.png

ragflow-add-ollama2.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/ragflow-add-ollama2.png

ragflow-chat.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/ragflow-chat.png

ragflow-chat2.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/ragflow-chat2.png

ragflow-chat3.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/ragflow-chat3.png

ragflow-knowledgebase.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/ragflow-knowledgebase.png

ragflow-knowledgebase2.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/ragflow-knowledgebase2.png

ragflow-knowledgebase3.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/ragflow-knowledgebase3.png

ragflow-knowledgebase4.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/ragflow-knowledgebase4.png

ragflow-knowledgebase5.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/ragflow-knowledgebase5.png

ragflow-knowledgebase6.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/ragflow-knowledgebase6.png

ragflow-record.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/ragflow-record.png

ragflow-record.mp4

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/ragflow-record.mp4

knowledge_graph_1.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/knowledge_graph_1.png

knowledge_graph_2.png

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/knowledge_graph_2.png

npu_llama3.2-3B.gif

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/npu_llama3.2-3B.gif

arc_qwen2.5-14B_q4_0_llamacpp.gif

Image address:

https://llm-assets.readthedocs.io/en/latest/_images/arc_qwen2.5-14B_q4_0_llamacpp.gif