Skip to content

Commit

Permalink
Adding the rerank model tables.
Browse files Browse the repository at this point in the history
  • Loading branch information
Trent Fowler authored and Trent Fowler committed Sep 2, 2024
1 parent c88506b commit b673768
Show file tree
Hide file tree
Showing 2 changed files with 16 additions and 16 deletions.
20 changes: 10 additions & 10 deletions fern/pages/models/models.mdx
View file
Original file line number Diff line number Diff line change
Expand Up @@ -70,16 +70,16 @@ In this table, we provide some important context for using Cohere Command models
These models can be used to generate embeddings from text or classify it based on various parameters. Embeddings can be used for estimating semantic similarity between two sentences, choosing a sentence which is most likely to follow another sentence, or categorizing user feedback, while outputs from the Classify endpoint can be used for any classification or analysis task. The Representation model comes with a variety of helper functions, such as for detecting the language of an input.


| Model Name | Description | Modalities | Dimensions | Context Length | Similarity Metric | Endpoints |
|-------------------------------|-------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------|------------|----------------|---------------------|-------------------------------------------------------------------------------------------------------------|
| `embed-english-v3.0` | A model that allows for text to be classified or turned into embeddings. English only. | Text, Images | 1024 | 512 | Cosine Similarity | [Embed](/reference/embed), <br/>[Embed Jobs](/reference/embed-jobs) |
| `embed-english-light-v3.0` | A smaller, faster version of `embed-english-v3.0`. Almost as capable, but a lot faster. English only. | Text, Images | 384 | 512 | Cosine Similarity | [Embed](/reference/embed), <br/>[Embed Jobs](/reference/embed-jobs) |
| `embed-multilingual-v3.0` | Provides multilingual classification and embedding support. [See supported languages here.](/docs/supported-languages) | Text, Images | 1024 | 512 | Cosine Similarity | [Embed](/reference/embed), [Embed Jobs](/reference/embed-jobs) |
| `embed-multilingual-light-v3.0` | A smaller, faster version of `embed-multilingual-v3.0`. Almost as capable, but a lot faster. Supports multiple languages. | Text, Images | 384 | 512 | Cosine Similarity | [Embed](/reference/embed), <br/>[Embed Jobs](/reference/embed-jobs) |
| | | | | | | |
| `embed-english-v2.0` | Our older embeddings model that allows for text to be classified or turned into embeddings. English only | Text | 4096 | 512 | Cosine Similarity | [Classify](/reference/classify), [Embed](/reference/embed) |
| `embed-english-light-v2.0` | A smaller, faster version of embed-english-v2.0. Almost as capable, but a lot faster. English only. | Text | 1024 | 512 | Cosine Similarity | [Classify](/reference/classify), [Embed](/reference/embed) |
| `embed-multilingual-v2.0` | Provides multilingual classification and embedding support. [See supported languages here.](/docs/supported-languages) | Text | 768 | 256 | Dot Product Similarity | [Classify](/reference/classify), [Embed](/reference/embed) |
| Model Name | Description | Modalities | Dimensions | Context Length | Similarity Metric | Endpoints |
|-------------------------------|-------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------|------------|----------------|---------------------|----------------------------------------------------------------------|
| `embed-english-v3.0` | A model that allows for text to be classified or turned into embeddings. English only. | Text, Images | 1024 | 512 | Cosine Similarity | [Embed](/reference/embed), <br/>[Embed Jobs](/reference/embed-jobs) |
| `embed-english-light-v3.0` | A smaller, faster version of `embed-english-v3.0`. Almost as capable, but a lot faster. English only. | Text, Images | 384 | 512 | Cosine Similarity | [Embed](/reference/embed), <br/>[Embed Jobs](/reference/embed-jobs) |
| `embed-multilingual-v3.0` | Provides multilingual classification and embedding support. [See supported languages here.](/docs/supported-languages) | Text, Images | 1024 | 512 | Cosine Similarity | [Embed](/reference/embed), [Embed Jobs](/reference/embed-jobs) |
| `embed-multilingual-light-v3.0` | A smaller, faster version of `embed-multilingual-v3.0`. Almost as capable, but a lot faster. Supports multiple languages. | Text, Images | 384 | 512 | Cosine Similarity | [Embed](/reference/embed), <br/>[Embed Jobs](/reference/embed-jobs) |
| | | | | | | |
| `embed-english-v2.0` | Our older embeddings model that allows for text to be classified or turned into embeddings. English only | Text | 4096 | 512 | Cosine Similarity | [Classify](/reference/classify), [Embed](/reference/embed) |
| `embed-english-light-v2.0` | A smaller, faster version of embed-english-v2.0. Almost as capable, but a lot faster. English only. | Text | 1024 | 512 | Cosine Similarity | [Classify](/reference/classify), [Embed](/reference/embed) |
| `embed-multilingual-v2.0` | Provides multilingual classification and embedding support. [See supported languages here.](/docs/supported-languages) | Text | 768 | 256 | Dot Product Similarity | [Classify](/reference/classify), [Embed](/reference/embed) |


In this table we've listed older `v2.0` models alongside the newer `v3.0` models, but we recommend you use the `v3.0` versions.
Expand Down
12 changes: 6 additions & 6 deletions fern/pages/models/rerank-2.mdx
View file
Original file line number Diff line number Diff line change
Expand Up @@ -13,12 +13,12 @@ updatedAt: "Mon Apr 08 2024 17:42:11 GMT+0000 (Coordinated Universal Time)"
---
Rerank models sort text inputs by semantic relevance to a specified query. They are often used to sort search results returned from an existing search solution. Learn more about using Rerank in the [best practices guide](/docs/reranking-best-practices).

| Latest Model | Description | Max Tokens | Endpoints |
| -------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------- | -------------------------------------------------- |
| `rerank-english-v3.0` | A model that allows for re-ranking English Language documents and semi-structured data (JSON). This model has a context length of 4096 tokens.. | N/A | [Rerank](/reference/rerank) |
| `rerank-multilingual-v3.0` | A model for documents and semi-structure data (JSON) that are not in English. Supports the same languages as `embed-multilingual-v3.0`. This model has a context length of 4096 tokens. | N/A | [Rerank](/reference/rerank) |
| `rerank-english-v2.0` | A model that allows for re-ranking English language documents. This model has a context length of 512 tokens. | N/A | [Rerank](/reference/rerank) |
| `rerank-multilingual-v2.0` | A model for documents that are not in English. Supports the same languages as `embed-multilingual-v3.0`. This model has a context length of 512 tokens. | N/A | [Rerank](/reference/rerank) |
| Latest Model | Description | Modality | Max Tokens | Endpoints |
| -------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| Text | ---------- | ------------------|
| `rerank-english-v3.0` | A model that allows for re-ranking English Language documents and semi-structured data (JSON). This model has a context length of 4096 tokens. | Text | N/A | [Rerank](/reference/rerank) |
| `rerank-multilingual-v3.0` | A model for documents and semi-structure data (JSON) that are not in English. Supports the same languages as `embed-multilingual-v3.0`. This model has a context length of 4096 tokens.| Text | N/A | [Rerank](/reference/rerank) |
| `rerank-english-v2.0` | A model that allows for re-ranking English language documents. This model has a context length of 512 tokens. | Text | N/A | [Rerank](/reference/rerank) |
| `rerank-multilingual-v2.0` | A model for documents that are not in English. Supports the same languages as `embed-multilingual-v3.0`. This model has a context length of 512 tokens. | Text | N/A | [Rerank](/reference/rerank) |

<Note>
Rerank accepts full strings and than tokens, so the token limit works a little differently. Rerank will automatically chunk documents longer than 4096 tokens, and there is therefore no explicit limit to how long a document can be when using rerank. See our [best practice guide](/docs/reranking-best-practices) for more info about formatting documents for the Rerank endpoint.
Expand Down

0 comments on commit b673768

Please sign in to comment.