Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add code snippets for azure using cohere sdk #282

Merged
merged 1 commit into from
Dec 6, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
99 changes: 94 additions & 5 deletions fern/pages/deployment-options/cohere-on-microsoft-azure.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ updatedAt: "Wed May 01 2024 16:11:36 GMT+0000 (Coordinated Universal Time)"
---
In an effort to make our language-model capabilities more widely available, we've partnered with a few major platforms to create hosted versions of our offerings.

In this article, you learn how to use [Azure AI Studio](https://ai.azure.com/) to deploy both the Cohere Command models and the Cohere Embed models on Microsoft's Azure cloud computing platform.
In this article, you learn how to use [Azure AI Foundry](https://ai.azure.com/) to deploy both the Cohere Command models and the Cohere Embed models on Microsoft's Azure cloud computing platform. You can read more about Azure AI Foundry in its documentation[here](https://learn.microsoft.com/en-us/azure/ai-studio/what-is-ai-studio).

The following six models are available through Azure AI Studio with pay-as-you-go, token-based billing:

Expand All @@ -22,7 +22,7 @@ The following six models are available through Azure AI Studio with pay-as-you-g
- Embed v3 - English
- Embed v3 - Multilingual
- Cohere Rerank V3 (English)
- Cohere Rerank V3 (multilingual)
- Cohere Rerank V3 (Multilingual)

## Prerequisites

Expand Down Expand Up @@ -140,7 +140,7 @@ except urllib.error.HTTPError as error:
print(error.read().decode("utf8", "ignore"))
```

## ReRank
## Rerank

We currently exposes the `v1/rerank` endpoint for inference with both Rerank 3 - English and Rerank 3 - Multilingual. For more information on using the APIs, see the [reference](https://learn.microsoft.com/en-us/azure/ai-studio/how-to/deploy-models-cohere-rerank#rerank-api-reference-for-cohere-rerank-models-deployed-as-a-service) section.

Expand Down Expand Up @@ -199,8 +199,97 @@ response = co.rerank(
)
```

## A Note on SDKs
## Using the Cohere SDK

You should be aware that it's possible to use the cohere SDK client to consume Azure AI deployments. Here are example notes for [Command](https://github.com/Azure/azureml-examples/blob/main/sdk/python/foundation-models/cohere/cohere-cmdR.ipynb) and [Embed](https://github.com/Azure/azureml-examples/blob/main/sdk/python/foundation-models/cohere/cohere-embed.ipynb).
You can use the Cohere SDK client to consume Cohere models that are deployed via Azure AI Foundry. This means you can leverage the SDK's features such as RAG, tool use, structured outputs, and more.

The following are a few examples on how to use the SDK for the different models.

### Setup
```python PYTHON
# pip install cohere

import cohere

# For Command models
co_chat = cohere.Client(
api_key="AZURE_INFERENCE_CREDENTIAL",
base_url="AZURE_MODEL_ENDPOINT", # Example - https://Cohere-command-r-plus-08-2024-xyz.eastus.models.ai.azure.com/
)

# For Embed models
co_embed = cohere.Client(
api_key="AZURE_INFERENCE_CREDENTIAL",
base_url="AZURE_MODEL_ENDPOINT", # Example - hhttps://cohere-embed-v3-multilingual-xyz.eastus.models.ai.azure.com/
)

# For Rerank models
co_rerank = cohere.Client(
api_key="AZURE_INFERENCE_CREDENTIAL",
base_url="AZURE_MODEL_ENDPOINT", # Example - hhttps://cohere-rerank-v3-multilingual-xyz.eastus.models.ai.azure.com/
)
```

### Chat
```python PYTHON
message = "I'm joining a new startup called Co1t today. Could you help me write a short introduction message to my teammates."

response = co_chat.chat(message=message)

print(response)
```
### RAG
```python PYTHON
faqs_short = [
{
"text": "Reimbursing Travel Expenses: Easily manage your travel expenses by submitting them through our finance tool. Approvals are prompt and straightforward."
},
{
"text": "Health and Wellness Benefits: We care about your well-being and offer gym memberships, on-site yoga classes, and comprehensive health insurance."
},
]

query = "Are there fitness-related perks?"

response = co_chat.chat(message=query, documents=faqs_short)

print(response)
```

### Embed
```python PYTHON
docs = [
"Joining Slack Channels: You will receive an invite via email. Be sure to join relevant channels to stay informed and engaged.",
"Finding Coffee Spots: For your caffeine fix, head to the break room's coffee machine or cross the street to the café for artisan coffee.",
]

doc_emb = co_embed.embed(
input_type="search_document",
texts=docs,
).embeddings
```

### Rerank
```python PYTHON
faqs_short = [
{
"text": "Reimbursing Travel Expenses: Easily manage your travel expenses by submitting them through our finance tool. Approvals are prompt and straightforward."
},
{
"text": "Working from Abroad: Working remotely from another country is possible. Simply coordinate with your manager and ensure your availability during core hours."
},
{
"text": "Health and Wellness Benefits: We care about your well-being and offer gym memberships, on-site yoga classes, and comprehensive health insurance."
},
]

query = "Are there fitness-related perks?"

results = co_rerank.rerank(
query=query, documents=faqs_short, top_n=2, model="rerank-english-v3.0"
)
```

Here are some other examples for [Command](https://github.com/Azure/azureml-examples/blob/main/sdk/python/foundation-models/cohere/cohere-cmdR.ipynb) and [Embed](https://github.com/Azure/azureml-examples/blob/main/sdk/python/foundation-models/cohere/cohere-embed.ipynb).

The important thing to understand is that our new and existing customers can call the models from Azure while still leveraging their integration with the Cohere SDK.
93 changes: 91 additions & 2 deletions fern/pages/v2/deployment-options/cohere-on-microsoft-azure.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -202,8 +202,97 @@ response = co.rerank(
)
```

## A Note on SDKs
## Using the Cohere SDK

You should be aware that it's possible to use the cohere SDK client to consume Azure AI deployments. Here are example notes for [Command](https://github.com/Azure/azureml-examples/blob/main/sdk/python/foundation-models/cohere/cohere-cmdR.ipynb) and [Embed](https://github.com/Azure/azureml-examples/blob/main/sdk/python/foundation-models/cohere/cohere-embed.ipynb).
You can use the Cohere SDK client to consume Cohere models that are deployed via Azure AI Foundry. This means you can leverage the SDK's features such as RAG, tool use, structured outputs, and more.

The following are a few examples on how to use the SDK for the different models.

### Setup
```python PYTHON
# pip install cohere

import cohere

# For Command models
co_chat = cohere.Client(
api_key="AZURE_INFERENCE_CREDENTIAL",
base_url="AZURE_MODEL_ENDPOINT", # Example - https://Cohere-command-r-plus-08-2024-xyz.eastus.models.ai.azure.com/
)

# For Embed models
co_embed = cohere.Client(
api_key="AZURE_INFERENCE_CREDENTIAL",
base_url="AZURE_MODEL_ENDPOINT", # Example - https://cohere-embed-v3-multilingual-xyz.eastus.models.ai.azure.com/
)

# For Rerank models
co_rerank = cohere.Client(
api_key="AZURE_INFERENCE_CREDENTIAL",
base_url="AZURE_MODEL_ENDPOINT", # Example - https://cohere-rerank-v3-multilingual-xyz.eastus.models.ai.azure.com/
)
```

### Chat
```python PYTHON
message = "I'm joining a new startup called Co1t today. Could you help me write a short introduction message to my teammates."

response = co_chat.chat(message=message)

print(response)
```
### RAG
```python PYTHON
faqs_short = [
{
"text": "Reimbursing Travel Expenses: Easily manage your travel expenses by submitting them through our finance tool. Approvals are prompt and straightforward."
},
{
"text": "Health and Wellness Benefits: We care about your well-being and offer gym memberships, on-site yoga classes, and comprehensive health insurance."
},
]

query = "Are there fitness-related perks?"

response = co_chat.chat(message=query, documents=faqs_short)

print(response)
```

### Embed
```python PYTHON
docs = [
"Joining Slack Channels: You will receive an invite via email. Be sure to join relevant channels to stay informed and engaged.",
"Finding Coffee Spots: For your caffeine fix, head to the break room's coffee machine or cross the street to the café for artisan coffee.",
]

doc_emb = co_embed.embed(
input_type="search_document",
texts=docs,
).embeddings
```

### Rerank
```python PYTHON
faqs_short = [
{
"text": "Reimbursing Travel Expenses: Easily manage your travel expenses by submitting them through our finance tool. Approvals are prompt and straightforward."
},
{
"text": "Working from Abroad: Working remotely from another country is possible. Simply coordinate with your manager and ensure your availability during core hours."
},
{
"text": "Health and Wellness Benefits: We care about your well-being and offer gym memberships, on-site yoga classes, and comprehensive health insurance."
},
]

query = "Are there fitness-related perks?"

results = co_rerank.rerank(
query=query, documents=faqs_short, top_n=2, model="rerank-english-v3.0"
)
```

Here are some other examples for [Command](https://github.com/Azure/azureml-examples/blob/main/sdk/python/foundation-models/cohere/cohere-cmdR.ipynb) and [Embed](https://github.com/Azure/azureml-examples/blob/main/sdk/python/foundation-models/cohere/cohere-embed.ipynb).

The important thing to understand is that our new and existing customers can call the models from Azure while still leveraging their integration with the Cohere SDK.
Loading