Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Predictionguard embeddings #3

Merged
merged 6 commits into from
Nov 8, 2024
Merged

Predictionguard embeddings #3

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
68b4f33
adding support for embeddings using predictionguard
jmansdorfer Nov 8, 2024
2794790
fixing linting errors
jmansdorfer Nov 8, 2024
3cb0931
adding new import to import tests
jmansdorfer Nov 8, 2024
aa040a0
fixing images in embeddings notebook
jmansdorfer Nov 8, 2024
c0d1f8c
updating provider doc
jmansdorfer Nov 8, 2024
b558564
Merge branch 'master' into predictionguard-embeddings
jmansdorfer Nov 8, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
109 changes: 33 additions & 76 deletions docs/docs/integrations/providers/predictionguard.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,99 +4,56 @@ This page covers how to use the Prediction Guard ecosystem within LangChain.
It is broken into two parts: installation and setup, and then references to specific Prediction Guard wrappers.

## Installation and Setup
- Install the Python SDK with `pip install predictionguard`
- Get a Prediction Guard access token (as described [here](https://docs.predictionguard.com/)) and set it as an environment variable (`PREDICTIONGUARD_TOKEN`)

## LLM Wrapper

There exists a Prediction Guard LLM wrapper, which you can access with
```python
from langchain_community.llms import PredictionGuard
- Install the Python SDK:
```

You can provide the name of the Prediction Guard model as an argument when initializing the LLM:
```python
pgllm = PredictionGuard(model="MPT-7B-Instruct")
pip install predictionguard
```

You can also provide your access token directly as an argument:
```python
pgllm = PredictionGuard(model="MPT-7B-Instruct", token="<your access token>")
```
- Get a Prediction Guard API key (as described [here](https://docs.predictionguard.com/)) and set it as an environment variable (`PREDICTIONGUARD_API_KEY`)

Finally, you can provide an "output" argument that is used to structure/ control the output of the LLM:
```python
pgllm = PredictionGuard(model="MPT-7B-Instruct", output={"type": "boolean"})
```
## Prediction Guard Langchain Integrations
|API|Description|Endpoint Docs|Import|Example Usage|
|---|---|---|---|---|
|Completions|Generate Text|[Completions](https://docs.predictionguard.com/api-reference/api-reference/completions)|`from langchain_community.llms.predictionguard import PredictionGuard`|[predictionguard.ipynb](/docs/integrations/llms/predictionguard)|
|Text Embedding|Embed String to Vectores|[Embeddings](https://docs.predictionguard.com/api-reference/api-reference/embeddings)|`from langchain_community.embeddings.predictionguard import PredictionGuardEmbeddings`|[predictionguard.ipynb](/docs/integrations/text_embedding/predictionguard)|

## Example usage
## Getting Started

Basic usage of the controlled or guarded LLM wrapper:
```python
import os
## Embedding Models

import predictionguard as pg
from langchain_community.llms import PredictionGuard
from langchain_core.prompts import PromptTemplate
from langchain.chains import LLMChain

# Your Prediction Guard API key. Get one at predictionguard.com
os.environ["PREDICTIONGUARD_TOKEN"] = "<your Prediction Guard access token>"

# Define a prompt template
template = """Respond to the following query based on the context.

Context: EVERY comment, DM + email suggestion has led us to this EXCITING announcement! 🎉 We have officially added TWO new candle subscription box options! 📦
Exclusive Candle Box - $80
Monthly Candle Box - $45 (NEW!)
Scent of The Month Box - $28 (NEW!)
Head to stories to get ALL the deets on each box! 👆 BONUS: Save 50% on your first box with code 50OFF! 🎉

Query: {query}

Result: """
prompt = PromptTemplate.from_template(template)

# With "guarding" or controlling the output of the LLM. See the
# Prediction Guard docs (https://docs.predictionguard.com) to learn how to
# control the output with integer, float, boolean, JSON, and other types and
# structures.
pgllm = PredictionGuard(model="MPT-7B-Instruct",
output={
"type": "categorical",
"categories": [
"product announcement",
"apology",
"relational"
]
})
pgllm(prompt.format(query="What kind of post is this?"))
### Prediction Guard Embeddings

See a [usage example](/docs/integrations/text_embedding/predictionguard)

```python
from langchain_community.embeddings.predictionguard
```

Basic LLM Chaining with the Prediction Guard wrapper:
#### Usage
```python
import os
# If predictionguard_api_key is not passed, default behavior is to use the `PREDICTIONGUARD_API_KEY` environment variable.
embeddings = PredictionGuardEmbeddings(model="bridgetower-large-itm-mlm-itc")

from langchain_core.prompts import PromptTemplate
from langchain.chains import LLMChain
from langchain_community.llms import PredictionGuard
text = "This is an embedding example."
output = embeddings.embed_query(text)
```

# Optional, add your OpenAI API Key. This is optional, as Prediction Guard allows
# you to access all the latest open access models (see https://docs.predictionguard.com)
os.environ["OPENAI_API_KEY"] = "<your OpenAI api key>"

# Your Prediction Guard API key. Get one at predictionguard.com
os.environ["PREDICTIONGUARD_TOKEN"] = "<your Prediction Guard access token>"

pgllm = PredictionGuard(model="OpenAI-gpt-3.5-turbo-instruct")
## LLMs
### Prediction Guard LLM

template = """Question: {question}
See a [usage example](/docs/integrations/llms/predictionguard)

Answer: Let's think step by step."""
prompt = PromptTemplate.from_template(template)
llm_chain = LLMChain(prompt=prompt, llm=pgllm, verbose=True)
```python
from langchain_community.llms import PredictionGuard
```

question = "What NFL team won the Super Bowl in the year Justin Beiber was born?"
#### Usage
```python
# If predictionguard_api_key is not passed, default behavior is to use the `PREDICTIONGUARD_API_KEY` environment variable.
llm = PredictionGuard(model="Hermes-2-Pro-Llama-3-8B")

llm_chain.predict(question=question)
llm.invoke("Tell me a joke about bears")
```
82 changes: 41 additions & 41 deletions docs/docs/integrations/text_embedding/predictionguard.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,18 +42,18 @@
{
"metadata": {
"ExecuteTime": {
"end_time": "2024-10-08T18:59:10.422135Z",
"start_time": "2024-10-08T18:59:10.419563Z"
"end_time": "2024-11-08T16:20:01.598574Z",
"start_time": "2024-11-08T16:20:01.595887Z"
}
},
"cell_type": "code",
"source": [
"import os\n",
"\n",
"os.environ[\"PREDICTIONGUARD_API_KEY\"] = \"<Prediction Guard API Key>\""
"os.environ[\"PREDICTIONGUARD_API_KEY\"] = \"<Prediction Guard API Key\""
],
"outputs": [],
"execution_count": 21
"execution_count": 1
},
{
"metadata": {},
Expand Down Expand Up @@ -81,27 +81,27 @@
"cell_type": "code",
"metadata": {
"ExecuteTime": {
"end_time": "2024-10-08T18:13:40.463622Z",
"start_time": "2024-10-08T18:13:40.240249Z"
"end_time": "2024-11-08T16:20:05.912657Z",
"start_time": "2024-11-08T16:20:05.679414Z"
}
},
"source": "from langchain_community.embeddings.predictionguard import PredictionGuardEmbeddings",
"outputs": [],
"execution_count": 1
"execution_count": 2
},
{
"cell_type": "code",
"metadata": {
"ExecuteTime": {
"end_time": "2024-10-08T18:14:14.324100Z",
"start_time": "2024-10-08T18:14:13.997521Z"
"end_time": "2024-11-08T16:20:08.538960Z",
"start_time": "2024-11-08T16:20:08.164922Z"
}
},
"source": [
"embeddings = PredictionGuardEmbeddings(model=\"bridgetower-large-itm-mlm-itc\")"
],
"outputs": [],
"execution_count": 4
"execution_count": 3
},
{
"cell_type": "markdown",
Expand All @@ -123,8 +123,8 @@
{
"metadata": {
"ExecuteTime": {
"end_time": "2024-10-08T18:14:29.307881Z",
"start_time": "2024-10-08T18:14:28.405099Z"
"end_time": "2024-11-08T16:21:11.729799Z",
"start_time": "2024-11-08T16:21:10.518236Z"
}
},
"cell_type": "code",
Expand Down Expand Up @@ -155,12 +155,12 @@
"'LangChain is the framework for building context-aware reasoning applications.'"
]
},
"execution_count": 6,
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
],
"execution_count": 6
"execution_count": 5
},
{
"metadata": {},
Expand All @@ -181,8 +181,8 @@
"cell_type": "code",
"metadata": {
"ExecuteTime": {
"end_time": "2024-10-08T18:16:00.824334Z",
"start_time": "2024-10-08T18:16:00.368665Z"
"end_time": "2024-11-08T16:21:16.331585Z",
"start_time": "2024-11-08T16:21:15.918706Z"
}
},
"source": [
Expand All @@ -203,12 +203,12 @@
" -0.003087474964559078]"
]
},
"execution_count": 14,
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"execution_count": 14
"execution_count": 6
},
{
"metadata": {},
Expand All @@ -219,8 +219,8 @@
"cell_type": "code",
"metadata": {
"ExecuteTime": {
"end_time": "2024-10-08T18:16:11.076843Z",
"start_time": "2024-10-08T18:16:10.655925Z"
"end_time": "2024-11-08T16:21:18.619883Z",
"start_time": "2024-11-08T16:21:18.200337Z"
}
},
"source": [
Expand All @@ -245,7 +245,7 @@
]
}
],
"execution_count": 15
"execution_count": 7
},
{
"cell_type": "markdown",
Expand All @@ -256,14 +256,14 @@
"cell_type": "code",
"metadata": {
"ExecuteTime": {
"end_time": "2024-10-08T18:16:44.853569Z",
"start_time": "2024-10-08T18:16:43.457282Z"
"end_time": "2024-11-08T16:21:20.599812Z",
"start_time": "2024-11-08T16:21:19.881001Z"
}
},
"source": [
"# Embedding a single image. These functions accept image URLs, image files, data URIs, and base64 encoded strings.\n",
"image = [\n",
" \"https://pbs.twimg.com/media/GKLN4qPXEAArqoK.png\",\n",
" \"https://farm4.staticflickr.com/3300/3497460990_11dfb95dd1_z.jpg\",\n",
"]\n",
"single_vector = embeddings.embed_images(image)\n",
"\n",
Expand All @@ -274,11 +274,11 @@
"name": "stdout",
"output_type": "stream",
"text": [
"[0.06482088565826416, -0.026690427213907242, 0.07683052867650986, -0.060580912977457047, 0.0001994583144551143]\n"
"[0.0911610797047615, -0.034427884966135025, 0.007927080616354942, -0.03500846028327942, 0.022317267954349518]\n"
]
}
],
"execution_count": 17
"execution_count": 8
},
{
"metadata": {},
Expand All @@ -289,14 +289,14 @@
"cell_type": "code",
"metadata": {
"ExecuteTime": {
"end_time": "2024-10-08T18:17:02.165077Z",
"start_time": "2024-10-08T18:17:00.612485Z"
"end_time": "2024-11-08T16:21:22.805707Z",
"start_time": "2024-11-08T16:21:22.068759Z"
}
},
"source": [
"# Embedding multiple images\n",
"images = [\n",
" \"https://pbs.twimg.com/media/GKLN4qPXEAArqoK.png\",\n",
" \"https://fastly.picsum.photos/id/866/200/300.jpg?hmac=rcadCENKh4rD6MAp6V_ma-AyWv641M4iiOpe1RyFHeI\",\n",
" \"https://farm4.staticflickr.com/3300/3497460990_11dfb95dd1_z.jpg\",\n",
"]\n",
"\n",
Expand All @@ -310,12 +310,12 @@
"name": "stdout",
"output_type": "stream",
"text": [
"[0.06482088565826416, -0.026690427213907242, 0.07683052867650986, -0.060580912977457047, 0.0001994583144551143]\n",
"[0.1593627631664276, -0.03636132553219795, -0.013229663483798504, -0.08789524435997009, 0.062290553003549576]\n",
"[0.0911610797047615, -0.034427884966135025, 0.007927080616354942, -0.03500846028327942, 0.022317267954349518]\n"
]
}
],
"execution_count": 18
"execution_count": 9
},
{
"cell_type": "markdown",
Expand All @@ -326,16 +326,16 @@
"cell_type": "code",
"metadata": {
"ExecuteTime": {
"end_time": "2024-10-08T18:17:17.113169Z",
"start_time": "2024-10-08T18:17:15.669474Z"
"end_time": "2024-11-08T16:21:24.925186Z",
"start_time": "2024-11-08T16:21:24.215510Z"
}
},
"source": [
"# Embedding a single text-image pair\n",
"inputs = [\n",
" {\n",
" \"text\": \"This is an embedding example.\",\n",
" \"image\": \"https://pbs.twimg.com/media/GKLN4qPXEAArqoK.png\",\n",
" \"image\": \"https://farm4.staticflickr.com/3300/3497460990_11dfb95dd1_z.jpg\",\n",
" },\n",
"]\n",
"single_vector = embeddings.embed_image_text(inputs)\n",
Expand All @@ -347,11 +347,11 @@
"name": "stdout",
"output_type": "stream",
"text": [
"[0.025471875444054604, -0.07661919295787811, 0.06256384402513504, -0.06042419373989105, 0.016889123246073723]\n"
"[0.0363212488591671, -0.10172265768051147, -0.014760786667466164, -0.046511903405189514, 0.03860781341791153]\n"
]
}
],
"execution_count": 19
"execution_count": 10
},
{
"metadata": {},
Expand All @@ -362,16 +362,16 @@
"cell_type": "code",
"metadata": {
"ExecuteTime": {
"end_time": "2024-10-08T18:17:31.948434Z",
"start_time": "2024-10-08T18:17:30.393415Z"
"end_time": "2024-11-08T16:21:26.869820Z",
"start_time": "2024-11-08T16:21:26.133863Z"
}
},
"source": [
"# Embedding multiple text-image pairs\n",
"inputs = [\n",
" {\n",
" \"text\": \"This is an embedding example.\",\n",
" \"image\": \"https://pbs.twimg.com/media/GKLN4qPXEAArqoK.png\",\n",
" \"image\": \"https://fastly.picsum.photos/id/866/200/300.jpg?hmac=rcadCENKh4rD6MAp6V_ma-AyWv641M4iiOpe1RyFHeI\",\n",
" },\n",
" {\n",
" \"text\": \"This is another embedding example.\",\n",
Expand All @@ -388,12 +388,12 @@
"name": "stdout",
"output_type": "stream",
"text": [
"[0.025471875444054604, -0.07661919295787811, 0.06256384402513504, -0.06042419373989105, 0.016889123246073723]\n",
"[0.11867266893386841, -0.05898813530802727, -0.026179173961281776, -0.10747235268354416, 0.07684746384620667]\n",
"[0.026654226705431938, -0.10080841928720474, -0.012732953764498234, -0.04365091398358345, 0.036743905395269394]\n"
]
}
],
"execution_count": 20
"execution_count": 11
},
{
"metadata": {},
Expand Down
Loading