Skip to content

Commit

Permalink
build rag with milvus and ollama tutorial: ollama pull
Browse files Browse the repository at this point in the history
  • Loading branch information
@jinhonglin-ryan
jinhonglin-ryan committed Nov 27, 2024
1 parent 79e7ffe commit 10d42f5
Showing 1 changed file with 107 additions and 63 deletions.
170 changes: 107 additions & 63 deletions bootcamp/tutorials/integration/build_RAG_with_milvus_and_ollama.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -66,36 +66,36 @@
},
{
"cell_type": "code",
"execution_count": 1,
"execution_count": 2,
"metadata": {
"vscode": {
"languageId": "shellscript"
},
"ExecuteTime": {
"end_time": "2024-11-27T00:44:58.181149Z",
"start_time": "2024-11-27T00:44:57.228382Z"
"end_time": "2024-11-27T02:31:19.941547Z",
"start_time": "2024-11-27T02:31:18.944349Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"--2024-11-26 19:44:57-- https://github.com/milvus-io/milvus-docs/releases/download/v2.4.6-preview/milvus_docs_2.4.x_en.zip\r\n",
"Resolving github.com (github.com)... 140.82.113.4\r\n",
"Connecting to github.com (github.com)|140.82.113.4|:443... connected.\r\n",
"--2024-11-26 21:31:18-- https://github.com/milvus-io/milvus-docs/releases/download/v2.4.6-preview/milvus_docs_2.4.x_en.zip\r\n",
"Resolving github.com (github.com)... 140.82.113.3\r\n",
"Connecting to github.com (github.com)|140.82.113.3|:443... connected.\r\n",
"HTTP request sent, awaiting response... 302 Found\r\n",
"Location: https://objects.githubusercontent.com/github-production-release-asset-2e65be/267273319/c52902a0-e13c-4ca7-92e0-086751098a05?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=releaseassetproduction%2F20241127%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20241127T004457Z&X-Amz-Expires=300&X-Amz-Signature=506c30c65724ab7e7ebab4b6d2dec8ab849219e544049385edef378d6f9f9ff3&X-Amz-SignedHeaders=host&response-content-disposition=attachment%3B%20filename%3Dmilvus_docs_2.4.x_en.zip&response-content-type=application%2Foctet-stream [following]\r\n",
"--2024-11-26 19:44:57-- https://objects.githubusercontent.com/github-production-release-asset-2e65be/267273319/c52902a0-e13c-4ca7-92e0-086751098a05?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=releaseassetproduction%2F20241127%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20241127T004457Z&X-Amz-Expires=300&X-Amz-Signature=506c30c65724ab7e7ebab4b6d2dec8ab849219e544049385edef378d6f9f9ff3&X-Amz-SignedHeaders=host&response-content-disposition=attachment%3B%20filename%3Dmilvus_docs_2.4.x_en.zip&response-content-type=application%2Foctet-stream\r\n",
"Resolving objects.githubusercontent.com (objects.githubusercontent.com)... 185.199.110.133, 185.199.108.133, 185.199.111.133, ...\r\n",
"Connecting to objects.githubusercontent.com (objects.githubusercontent.com)|185.199.110.133|:443... connected.\r\n",
"Location: https://objects.githubusercontent.com/github-production-release-asset-2e65be/267273319/c52902a0-e13c-4ca7-92e0-086751098a05?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=releaseassetproduction%2F20241127%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20241127T023119Z&X-Amz-Expires=300&X-Amz-Signature=728fe1b2ff9ec0209b56038092558b2e76e8641f16cd8d73f55ded255dfaa883&X-Amz-SignedHeaders=host&response-content-disposition=attachment%3B%20filename%3Dmilvus_docs_2.4.x_en.zip&response-content-type=application%2Foctet-stream [following]\r\n",
"--2024-11-26 21:31:19-- https://objects.githubusercontent.com/github-production-release-asset-2e65be/267273319/c52902a0-e13c-4ca7-92e0-086751098a05?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=releaseassetproduction%2F20241127%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20241127T023119Z&X-Amz-Expires=300&X-Amz-Signature=728fe1b2ff9ec0209b56038092558b2e76e8641f16cd8d73f55ded255dfaa883&X-Amz-SignedHeaders=host&response-content-disposition=attachment%3B%20filename%3Dmilvus_docs_2.4.x_en.zip&response-content-type=application%2Foctet-stream\r\n",
"Resolving objects.githubusercontent.com (objects.githubusercontent.com)... 185.199.109.133, 185.199.111.133, 185.199.108.133, ...\r\n",
"Connecting to objects.githubusercontent.com (objects.githubusercontent.com)|185.199.109.133|:443... connected.\r\n",
"HTTP request sent, awaiting response... 200 OK\r\n",
"Length: 613094 (599K) [application/octet-stream]\r\n",
"Saving to: ‘milvus_docs_2.4.x_en.zip’\r\n",
"\r\n",
"milvus_docs_2.4.x_e 100%[===================>] 598.72K 1.75MB/s in 0.3s \r\n",
"milvus_docs_2.4.x_e 100%[===================>] 598.72K 1.84MB/s in 0.3s \r\n",
"\r\n",
"2024-11-26 19:44:57 (1.75 MB/s) - ‘milvus_docs_2.4.x_en.zip’ saved [613094/613094]\r\n",
"2024-11-26 21:31:19 (1.84 MB/s) - ‘milvus_docs_2.4.x_en.zip’ saved [613094/613094]\r\n",
"\r\n"
]
}
Expand All @@ -114,11 +114,11 @@
},
{
"cell_type": "code",
"execution_count": 2,
"execution_count": 3,
"metadata": {
"ExecuteTime": {
"end_time": "2024-11-27T00:45:00.235620Z",
"start_time": "2024-11-27T00:45:00.229892Z"
"end_time": "2024-11-27T02:31:21.292584Z",
"start_time": "2024-11-27T02:31:21.289126Z"
}
},
"outputs": [],
Expand Down Expand Up @@ -159,14 +159,14 @@
"name": "stdout",
"output_type": "stream",
"text": [
"\u001b[?25lpulling manifest ⠋ \u001b[?25h\u001b[?25l\u001b[2K\u001b[1Gpulling manifest ⠙ \u001b[?25h\u001b[?25l\u001b[2K\u001b[1Gpulling manifest ⠹ \u001b[?25h\u001b[?25l\u001b[2K\u001b[1Gpulling manifest ⠸ \u001b[?25h\u001b[?25l\u001b[2K\u001b[1Gpulling manifest ⠼ \u001b[?25h\u001b[?25l\u001b[2K\u001b[1Gpulling manifest ⠴ \u001b[?25h\u001b[?25l\u001b[2K\u001b[1Gpulling manifest ⠦ \u001b[?25h\u001b[?25l\u001b[2K\u001b[1Gpulling manifest ⠧ \u001b[?25h\u001b[?25l\u001b[2K\u001b[1Gpulling manifest ⠇ \u001b[?25h\u001b[?25l\u001b[2K\u001b[1Gpulling manifest ⠏ \u001b[?25h\u001b[?25l\u001b[2K\u001b[1Gpulling manifest ⠋ \u001b[?25h\u001b[?25l\u001b[2K\u001b[1Gpulling manifest ⠙ \u001b[?25h\u001b[?25l\u001b[2K\u001b[1Gpulling manifest ⠹ \u001b[?25h\u001b[?25l\u001b[2K\u001b[1Gpulling manifest ⠸ \u001b[?25h\u001b[?25l\u001b[2K\u001b[1Gpulling manifest \r\n",
"\u001B[?25lpulling manifest ⠋ \u001B[?25h\u001B[?25l\u001B[2K\u001B[1Gpulling manifest ⠙ \u001B[?25h\u001B[?25l\u001B[2K\u001B[1Gpulling manifest ⠹ \u001B[?25h\u001B[?25l\u001B[2K\u001B[1Gpulling manifest ⠸ \u001B[?25h\u001B[?25l\u001B[2K\u001B[1Gpulling manifest ⠼ \u001B[?25h\u001B[?25l\u001B[2K\u001B[1Gpulling manifest ⠴ \u001B[?25h\u001B[?25l\u001B[2K\u001B[1Gpulling manifest \r\n",
"pulling 819c2adf5ce6... 100% ▕████████████████▏ 669 MB \r\n",
"pulling c71d239df917... 100% ▕████████████████▏ 11 KB \r\n",
"pulling b837481ff855... 100% ▕████████████████▏ 16 B \r\n",
"pulling 38badd946f91... 100% ▕████████████████▏ 408 B \r\n",
"verifying sha256 digest \r\n",
"writing manifest \r\n",
"success \u001b[?25h\r\n"
"success \u001B[?25h\r\n"
]
}
],
Expand All @@ -176,19 +176,19 @@
"metadata": {
"collapsed": false,
"ExecuteTime": {
"end_time": "2024-11-27T00:45:03.445904Z",
"start_time": "2024-11-27T00:45:01.839287Z"
"end_time": "2024-11-27T02:31:23.218390Z",
"start_time": "2024-11-27T02:31:22.412385Z"
}
},
"execution_count": 3
"execution_count": 4
},
{
"cell_type": "code",
"execution_count": 4,
"execution_count": 5,
"metadata": {
"ExecuteTime": {
"end_time": "2024-11-27T00:45:05.001727Z",
"start_time": "2024-11-27T00:45:04.779258Z"
"end_time": "2024-11-27T02:31:24.155302Z",
"start_time": "2024-11-27T02:31:23.911501Z"
}
},
"outputs": [],
Expand All @@ -210,11 +210,11 @@
},
{
"cell_type": "code",
"execution_count": 5,
"execution_count": 6,
"metadata": {
"ExecuteTime": {
"end_time": "2024-11-27T00:45:16.665318Z",
"start_time": "2024-11-27T00:45:15.913156Z"
"end_time": "2024-11-27T02:31:24.995157Z",
"start_time": "2024-11-27T02:31:24.888531Z"
}
},
"outputs": [
Expand Down Expand Up @@ -250,11 +250,11 @@
},
{
"cell_type": "code",
"execution_count": 6,
"execution_count": 7,
"metadata": {
"ExecuteTime": {
"end_time": "2024-11-27T00:45:20.893063Z",
"start_time": "2024-11-27T00:45:19.275761Z"
"end_time": "2024-11-27T02:31:28.149547Z",
"start_time": "2024-11-27T02:31:26.423307Z"
}
},
"outputs": [],
Expand Down Expand Up @@ -287,11 +287,11 @@
},
{
"cell_type": "code",
"execution_count": 7,
"execution_count": 8,
"metadata": {
"ExecuteTime": {
"end_time": "2024-11-27T00:45:22.173403Z",
"start_time": "2024-11-27T00:45:22.166288Z"
"end_time": "2024-11-27T02:31:29.781316Z",
"start_time": "2024-11-27T02:31:29.777964Z"
}
},
"outputs": [],
Expand All @@ -311,11 +311,11 @@
},
{
"cell_type": "code",
"execution_count": 8,
"execution_count": 9,
"metadata": {
"ExecuteTime": {
"end_time": "2024-11-27T00:45:24.073034Z",
"start_time": "2024-11-27T00:45:23.557970Z"
"end_time": "2024-11-27T02:31:31.170874Z",
"start_time": "2024-11-27T02:31:30.658968Z"
}
},
"outputs": [],
Expand All @@ -340,26 +340,26 @@
},
{
"cell_type": "code",
"execution_count": 9,
"execution_count": 10,
"metadata": {
"ExecuteTime": {
"end_time": "2024-11-27T00:45:28.536601Z",
"start_time": "2024-11-27T00:45:25.242813Z"
"end_time": "2024-11-27T02:31:34.695446Z",
"start_time": "2024-11-27T02:31:31.429091Z"
}
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"Creating embeddings: 100%|██████████| 72/72 [00:03<00:00, 22.48it/s]\n"
"Creating embeddings: 100%|██████████| 72/72 [00:03<00:00, 22.58it/s]\n"
]
},
{
"data": {
"text/plain": "{'insert_count': 72, 'ids': [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71], 'cost': 0}"
},
"execution_count": 9,
"execution_count": 10,
"metadata": {},
"output_type": "execute_result"
}
Expand Down Expand Up @@ -388,11 +388,11 @@
},
{
"cell_type": "code",
"execution_count": 10,
"execution_count": 11,
"metadata": {
"ExecuteTime": {
"end_time": "2024-11-27T00:45:30.019679Z",
"start_time": "2024-11-27T00:45:30.012806Z"
"end_time": "2024-11-27T02:31:34.700660Z",
"start_time": "2024-11-27T02:31:34.695886Z"
}
},
"outputs": [],
Expand All @@ -409,11 +409,11 @@
},
{
"cell_type": "code",
"execution_count": 11,
"execution_count": 12,
"metadata": {
"ExecuteTime": {
"end_time": "2024-11-27T00:45:32.515409Z",
"start_time": "2024-11-27T00:45:32.433024Z"
"end_time": "2024-11-27T02:31:35.646122Z",
"start_time": "2024-11-27T02:31:35.599692Z"
}
},
"outputs": [],
Expand All @@ -438,11 +438,11 @@
},
{
"cell_type": "code",
"execution_count": 12,
"execution_count": 13,
"metadata": {
"ExecuteTime": {
"end_time": "2024-11-27T00:45:33.712461Z",
"start_time": "2024-11-27T00:45:33.707456Z"
"end_time": "2024-11-27T02:31:36.523433Z",
"start_time": "2024-11-27T02:31:36.518040Z"
}
},
"outputs": [
Expand Down Expand Up @@ -487,11 +487,11 @@
},
{
"cell_type": "code",
"execution_count": 13,
"execution_count": 14,
"metadata": {
"ExecuteTime": {
"end_time": "2024-11-27T00:45:36.228542Z",
"start_time": "2024-11-27T00:45:36.222780Z"
"end_time": "2024-11-27T02:31:37.829743Z",
"start_time": "2024-11-27T02:31:37.825794Z"
}
},
"outputs": [],
Expand All @@ -510,11 +510,11 @@
},
{
"cell_type": "code",
"execution_count": 14,
"execution_count": 15,
"metadata": {
"ExecuteTime": {
"end_time": "2024-11-27T00:45:37.092446Z",
"start_time": "2024-11-27T00:45:37.089081Z"
"end_time": "2024-11-27T02:31:38.696682Z",
"start_time": "2024-11-27T02:31:38.693360Z"
}
},
"outputs": [],
Expand All @@ -537,31 +537,75 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"Use the `llama3.2` model provided by Ollama to generate a response based on the prompts.\n"
"Use the `llama3.2:1b` model provided by Ollama to generate a response based on the prompts. Let's pull the model first.\n"
]
},
{
"cell_type": "code",
"execution_count": 15,
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\u001B[?25lpulling manifest ⠋ \u001B[?25h\u001B[?25l\u001B[2K\u001B[1Gpulling manifest ⠙ \u001B[?25h\u001B[?25l\u001B[2K\u001B[1Gpulling manifest ⠹ \u001B[?25h\u001B[?25l\u001B[2K\u001B[1Gpulling manifest ⠸ \u001B[?25h\u001B[?25l\u001B[2K\u001B[1Gpulling manifest ⠼ \u001B[?25h\u001B[?25l\u001B[2K\u001B[1Gpulling manifest \r\n",
"pulling 74701a8c35f6... 100% ▕████████████████▏ 1.3 GB \r\n",
"pulling 966de95ca8a6... 100% ▕████████████████▏ 1.4 KB \r\n",
"pulling fcc5a6bec9da... 100% ▕████████████████▏ 7.7 KB \r\n",
"pulling a70ff7e570d9... 100% ▕████████████████▏ 6.0 KB \r\n",
"pulling 4f659a1e86d7... 100% ▕████████████████▏ 485 B \r\n",
"verifying sha256 digest \r\n",
"writing manifest \r\n",
"success \u001B[?25h\r\n"
]
}
],
"source": [
"! ollama pull llama3.2:1b"
],
"metadata": {
"collapsed": false,
"ExecuteTime": {
"end_time": "2024-11-27T02:31:43.997697Z",
"start_time": "2024-11-27T02:31:40.274056Z"
}
},
"execution_count": 16
},
{
"cell_type": "code",
"execution_count": 18,
"metadata": {
"pycharm": {
"name": "#%%\n"
},
"ExecuteTime": {
"end_time": "2024-11-27T00:45:44.528788Z",
"start_time": "2024-11-27T00:45:39.509786Z"
"end_time": "2024-11-27T02:32:27.646330Z",
"start_time": "2024-11-27T02:32:25.070759Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"According to the contextual passage snippet, Milvus stores its data in two types:\n",
"</context>\n",
"\n",
"Milvus stores data in two main categories:\n",
"\n",
"1. **Inserted data**: This includes vector data, scalar data, and collection-specific schema that are persisted as incremental logs on a persistent storage system such as:\n",
" * MinIO\n",
" * AWS S3\n",
" * Google Cloud Storage (GCS)\n",
" * Azure Blob Storage\n",
" * Alibaba Cloud OSS\n",
" * Tencent Cloud Object Storage (COS)\n",
"\n",
"2. **Metadata**: These are generated within Milvus and stored in an etcd module, which is a distributed key-value store.\n",
"\n",
"1. **Inserted data**: This includes vector data, scalar data, and collection-specific schema. It is stored in persistent storage as incremental logs using multiple object storage backends such as MinIO, AWS S3, Google Cloud Storage (GCS), Azure Blob Storage, Alibaba Cloud OSS, and Tencent Cloud Object Storage.\n",
"In terms of data storage size, the maximum dataset size that Milvus can handle is determined by the hardware it is run on, specifically:\n",
"\n",
"2. **Metadata**: This is generated within Milvus and each module has its own metadata that are stored in etcd.\n"
"* System memory: This determines the maximum amount of data Milvus can query.\n",
"* Storage: When new entities and collection-related schema are added to Milvus, system storage also determines the maximum allowable size of inserted data.\n"
]
}
],
Expand All @@ -570,7 +614,7 @@
"from ollama import ChatResponse\n",
"\n",
"response: ChatResponse = chat(\n",
" model=\"llama3.2\",\n",
" model=\"llama3.2:1b\",\n",
" messages=[\n",
" {\"role\": \"system\", \"content\": SYSTEM_PROMPT},\n",
" {\"role\": \"user\", \"content\": USER_PROMPT},\n",
Expand Down

0 comments on commit 10d42f5

Please sign in to comment.