Skip to content

Commit 5d03c17

Browse files
nerdainerdai
authored
Move stale packages into stale_packages (#17234)
* wip * stale packages * add readers-readme * move example nbs
1 parent 2612944 commit 5d03c17

File tree

239 files changed

+11350
-0
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

239 files changed

+11350
-0
lines changed

‎docs/docs/examples/embeddings/itrex.ipynb

Lines changed: 94 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,94 @@
1+
{
2+
"cells": [
3+
{
4+
"cell_type": "markdown",
5+
"metadata": {},
6+
"source": [
7+
"# Optimized BGE Embedding Model using Intel® Extension for Transformers\n",
8+
"\n",
9+
"LlamaIndex has support for loading quantized BGE embedding models generated by [Intel® Extension for Transformers](https://github.com/intel/intel-extension-for-transformers) (ITREX) and use ITREX [Neural Engine](https://github.com/intel/intel-extension-for-transformers/blob/main/intel_extension_for_transformers/llm/runtime/deprecated/docs/Installation.md), a high-performance NLP backend, to accelerate the inference of models without compromising accuracy.\n",
10+
"\n",
11+
"Refer to our blog of [Efficient Natural Language Embedding Models with Intel Extension for Transformers](https://medium.com/intel-analytics-software/efficient-natural-language-embedding-models-with-intel-extension-for-transformers-2b6fcd0f8f34) and [BGE optimization example](https://github.com/intel/intel-extension-for-transformers/tree/main/examples/huggingface/pytorch/text-embedding/deployment/mteb/bge) for more details.\"\n",
12+
"\n",
13+
"In order to be able to load and use the quantized models, install the required dependency `pip install intel-extension-for-transformers torch accelerate datasets onnx`. \n",
14+
"\n",
15+
"Loading is done using the class `ItrexQuantizedBgeEmbedding`; usage is similar to any HuggingFace local embedding model; See example:"
16+
]
17+
},
18+
{
19+
"cell_type": "code",
20+
"execution_count": null,
21+
"metadata": {},
22+
"outputs": [],
23+
"source": [
24+
"%pip install llama-index-embeddings-huggingface-itrex"
25+
]
26+
},
27+
{
28+
"cell_type": "code",
29+
"execution_count": null,
30+
"metadata": {},
31+
"outputs": [
32+
{
33+
"name": "stderr",
34+
"output_type": "stream",
35+
"text": [
36+
"/home/yuwenzho/.conda/envs/yuwen/lib/python3.9/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html\n",
37+
" from .autonotebook import tqdm as notebook_tqdm\n",
38+
"2024-03-29 15:40:42 [INFO] Start to extarct onnx model ops...\n",
39+
"2024-03-29 15:40:42 [INFO] Extract onnxruntime model done...\n",
40+
"2024-03-29 15:40:42 [INFO] Start to implement Sub-Graph matching and replacing...\n",
41+
"2024-03-29 15:40:43 [INFO] Sub-Graph match and replace done...\n"
42+
]
43+
}
44+
],
45+
"source": [
46+
"from llama_index.embeddings.huggingface_itrex import ItrexQuantizedBgeEmbedding\n",
47+
"\n",
48+
"embed_model = ItrexQuantizedBgeEmbedding(\n",
49+
" \"Intel/bge-small-en-v1.5-sts-int8-static-inc\"\n",
50+
")"
51+
]
52+
},
53+
{
54+
"cell_type": "code",
55+
"execution_count": null,
56+
"metadata": {},
57+
"outputs": [
58+
{
59+
"name": "stdout",
60+
"output_type": "stream",
61+
"text": [
62+
"384\n",
63+
"[-0.005477035418152809, -0.000541043293196708, 0.036467909812927246, -0.04861024394631386, 0.0288068987429142]\n"
64+
]
65+
}
66+
],
67+
"source": [
68+
"embeddings = embed_model.get_text_embedding(\"Hello World!\")\n",
69+
"print(len(embeddings))\n",
70+
"print(embeddings[:5])"
71+
]
72+
}
73+
],
74+
"metadata": {
75+
"kernelspec": {
76+
"display_name": "yuwen",
77+
"language": "python",
78+
"name": "python3"
79+
},
80+
"language_info": {
81+
"codemirror_mode": {
82+
"name": "ipython",
83+
"version": 3
84+
},
85+
"file_extension": ".py",
86+
"mimetype": "text/x-python",
87+
"name": "python",
88+
"nbconvert_exporter": "python",
89+
"pygments_lexer": "ipython3"
90+
}
91+
},
92+
"nbformat": 4,
93+
"nbformat_minor": 2
94+
}

‎docs/docs/examples/embeddings/octoai.ipynb

Lines changed: 138 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,138 @@
1+
{
2+
"cells": [
3+
{
4+
"cell_type": "markdown",
5+
"metadata": {},
6+
"source": [
7+
"<a href=\"https://colab.research.google.com/github/run-llama/llama_index/blob/main/docs/docs/examples/embeddings/octoai.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
8+
]
9+
},
10+
{
11+
"cell_type": "markdown",
12+
"metadata": {},
13+
"source": [
14+
"# OctoAI Embeddings\n",
15+
"\n",
16+
"This guide shows you how to use [OctoAI's Embeddings](https://octo.ai/docs/text-gen-solution/getting-started) through LlamaIndex."
17+
]
18+
},
19+
{
20+
"cell_type": "markdown",
21+
"metadata": {},
22+
"source": [
23+
"First, let's install LlamaIndex and OctoAI's dependencies"
24+
]
25+
},
26+
{
27+
"cell_type": "code",
28+
"execution_count": null,
29+
"metadata": {},
30+
"outputs": [],
31+
"source": [
32+
"%pip install llama-index-embeddings-octoai"
33+
]
34+
},
35+
{
36+
"cell_type": "code",
37+
"execution_count": null,
38+
"metadata": {},
39+
"outputs": [],
40+
"source": [
41+
"!pip install llama-index"
42+
]
43+
},
44+
{
45+
"cell_type": "markdown",
46+
"metadata": {},
47+
"source": [
48+
"Include your OctoAI API key below. You can get yours at [OctoAI](https://octo.ai). \n",
49+
"\n",
50+
"[Here](https://octo.ai/docs/getting-started/how-to-create-an-octoai-access-token) are some instructions in case you need more guidance."
51+
]
52+
},
53+
{
54+
"cell_type": "code",
55+
"execution_count": null,
56+
"metadata": {},
57+
"outputs": [],
58+
"source": [
59+
"OCTOAI_API_KEY = \"\""
60+
]
61+
},
62+
{
63+
"cell_type": "markdown",
64+
"metadata": {},
65+
"source": [
66+
"We can then query embeddings on OctoAI"
67+
]
68+
},
69+
{
70+
"cell_type": "code",
71+
"execution_count": null,
72+
"metadata": {},
73+
"outputs": [],
74+
"source": [
75+
"from llama_index.embeddings.octoai import OctoAIEmbedding\n",
76+
"\n",
77+
"embed_model = OctoAIEmbedding(api_key=OCTOAI_API_KEY)"
78+
]
79+
},
80+
{
81+
"cell_type": "code",
82+
"execution_count": null,
83+
"metadata": {},
84+
"outputs": [],
85+
"source": [
86+
"# Basic embedding example\n",
87+
"embeddings = embed_model.get_text_embedding(\"How do I sail to the moon?\")\n",
88+
"print(len(embeddings), embeddings[:10])\n",
89+
"assert len(embeddings) == 1024"
90+
]
91+
},
92+
{
93+
"cell_type": "markdown",
94+
"metadata": {},
95+
"source": [
96+
"## Using Batched Embeddings"
97+
]
98+
},
99+
{
100+
"cell_type": "code",
101+
"execution_count": null,
102+
"metadata": {},
103+
"outputs": [],
104+
"source": [
105+
"texts = [\n",
106+
" \"How do I sail to the moon?\",\n",
107+
" \"What is the best way to cook a steak?\",\n",
108+
" \"How do I apply for a job?\",\n",
109+
"]\n",
110+
"\n",
111+
"embeddings = embed_model.get_text_embedding_batch(texts)\n",
112+
"print(len(embeddings))\n",
113+
"assert len(embeddings) == 3\n",
114+
"assert len(embeddings[0]) == 1024"
115+
]
116+
}
117+
],
118+
"metadata": {
119+
"kernelspec": {
120+
"display_name": "Python 3 (ipykernel)",
121+
"language": "python",
122+
"name": "python3"
123+
},
124+
"language_info": {
125+
"codemirror_mode": {
126+
"name": "ipython",
127+
"version": 3
128+
},
129+
"file_extension": ".py",
130+
"mimetype": "text/x-python",
131+
"name": "python",
132+
"nbconvert_exporter": "python",
133+
"pygments_lexer": "ipython3"
134+
}
135+
},
136+
"nbformat": 4,
137+
"nbformat_minor": 4
138+
}

‎docs/docs/examples/llm/solar.ipynb

Lines changed: 88 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,88 @@
1+
{
2+
"cells": [
3+
{
4+
"cell_type": "markdown",
5+
"id": "cae1b4a8",
6+
"metadata": {},
7+
"source": [
8+
"# Solar LLM\n",
9+
"\n",
10+
"Warning: Solar LLM is deprecated. Please use Upstage LLM instead. \n",
11+
"See [Upstage LLM](https://docs.llamaindex.ai/en/stable/examples/llm/upstage/)"
12+
]
13+
},
14+
{
15+
"cell_type": "code",
16+
"execution_count": null,
17+
"id": "715d392e",
18+
"metadata": {},
19+
"outputs": [],
20+
"source": [
21+
"!pip install llama-index-llms-solar"
22+
]
23+
},
24+
{
25+
"cell_type": "code",
26+
"execution_count": null,
27+
"id": "1fdc2dc3-1454-41e9-8862-9dfd75b5b61f",
28+
"metadata": {},
29+
"outputs": [],
30+
"source": [
31+
"import os\n",
32+
"\n",
33+
"os.environ[\"SOLAR_API_KEY\"] = \"SOLAR_API_KEY\""
34+
]
35+
},
36+
{
37+
"cell_type": "code",
38+
"execution_count": null,
39+
"id": "26b168b8-9ebf-479d-ac53-28bc952da354",
40+
"metadata": {},
41+
"outputs": [
42+
{
43+
"name": "stdout",
44+
"output_type": "stream",
45+
"text": [
46+
"assistant: Mother also went into the room.\n"
47+
]
48+
}
49+
],
50+
"source": [
51+
"# from llama_index.llms import\n",
52+
"from llama_index.llms.solar import Solar\n",
53+
"from llama_index.core.base.llms.types import ChatMessage, MessageRole\n",
54+
"\n",
55+
"llm = Solar(model=\"solar-1-mini-chat\", is_chat_model=True)\n",
56+
"response = llm.chat(\n",
57+
" messages=[\n",
58+
" ChatMessage(role=\"user\", content=\"아버지가방에들어가셨다\"),\n",
59+
" ChatMessage(role=\"assistant\", content=\"Father went into his room\"),\n",
60+
" ChatMessage(role=\"user\", content=\"엄마도들어가셨다\"),\n",
61+
" ]\n",
62+
")\n",
63+
"\n",
64+
"print(response)"
65+
]
66+
}
67+
],
68+
"metadata": {
69+
"kernelspec": {
70+
"display_name": "Python 3 (ipykernel)",
71+
"language": "python",
72+
"name": "python3"
73+
},
74+
"language_info": {
75+
"codemirror_mode": {
76+
"name": "ipython",
77+
"version": 3
78+
},
79+
"file_extension": ".py",
80+
"mimetype": "text/x-python",
81+
"name": "python",
82+
"nbconvert_exporter": "python",
83+
"pygments_lexer": "ipython3"
84+
}
85+
},
86+
"nbformat": 4,
87+
"nbformat_minor": 5
88+
}

0 commit comments

Comments
 (0)