diff --git a/fern/pages/models/aya.mdx b/fern/pages/models/aya.mdx new file mode 100644 index 00000000..f529be3e --- /dev/null +++ b/fern/pages/models/aya.mdx @@ -0,0 +1,73 @@ +--- +title: Aya Family of Models +slug: aya/tools +hidden: false +description: >- + Understand Cohere for AI's groundbreaking multilingual Aya models, which aim to bring many more languages into generative AI. +image: ../../assets/images/6c1b0e4-cohere_meta_image.jpg +keywords: 'Cohere AI, multilingual large language models, generative AI' +createdAt: 'Thu Nov 21 2024 14:18:00 MST (U.S. Mountain Time)' +updatedAt: '' +--- + +[Aya](https://cohere.com/research/aya) is a family of multilingual large language models that are designed to expand the number of languages covered by generative AI. Its 8-billion and 32-billion parameter “Expanse” offerings are optimized to perform well in these 23 languages: Arabic, Chinese (simplified & traditional), Czech, Dutch, English, French, German, Greek, Hebrew, Hebrew, Hindi, Indonesian, Italian, Japanese, Korean, Persian, Polish, Portuguese, Romanian, Russian, Spanish, Turkish, Ukrainian, and Vietnamese. + +## Model Details +| Model Name | Description | Modality | Context Length | Maximum Output Tokens | Endpoints | +|--------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------------|----------------|-----------------------|------------------------------------------------------| +| `c4ai-aya-expanse-8b` | Aya Expanse is a highly performant 8B multilingual model, designed to rival monolingual performance through innovations in instruction tuning with data arbitrage, preference training, and model merging. Serves 23 languages. | Text | 8k | 4k | [Chat](/reference/chat) | +| `c4ai-aya-expanse-32b` | Aya Expanse is a highly performant 32B multilingual model, designed to rival monolingual performance through innovations in instruction tuning with data arbitrage, preference training, and model merging. Serves 23 languages. | Text | 128k | 4k | [Chat](/reference/chat) | + + +## How Can I Get Access to the Aya Models? +If you want to test Aya, you have three options. First (and simplest), you can use the [Cohere playground](https://dashboard.cohere.com/playground/chat) or [Hugging Face Space](https://huggingface.co/spaces/CohereForAI/aya_expanse) to play around with them and see what they’re capable of. + +Second, you can use the [Cohere Chat API](https://docs.cohere.com/v2/docs/chat-api) to work with Aya programmatically. Here’s a very lightweight example of using the Cohere SDK to create a Spanish-language tutor with Aya that tells a story with simple Spanish vocabulary (NOTE: you’ll need an API key to run this code, and if you haven’t installed the Cohere SDK you can do that with `pip install cohere`). + +```python PYTHON +import cohere +co=cohere.ClientV2() + +response=co.chat( + model='c4ai-aya-expanse-32b', + messages=[ + { + "role" : "user", + "content" : "Eres un gran profesor de español. ¿Puedes escribirme una historia que ilustre vocabulario sencillo en español?" + } + ] +) + +print(response.message.content[0].text) +``` + +And here’s an example output (remember: these models are stochastic, and what you see might look quite different). + +```python PYTHON +¡Claro! Aquí te presento una historia corta que utiliza vocabulario sencillo en español: + +**La aventura de María en el mercado** + +Era una mañana soleada y María, una joven curiosa, decidió explorar el mercado local de su pueblo. Al entrar, se encontró con un mundo lleno de colores y aromas fascinantes. + +En uno de los puestos, vio una montaña de frutas brillantes. Había manzanas rojas como la grana, naranjas naranjas como el atardecer, y plátanos amarillos como el sol. María eligió una manzana crujiente y le pidió al vendedor que le enseñara cómo pelar una naranja. + +Caminando por los pasillos, se topó con una señora que vendía flores. Las rosas rojas olían a dulce miel, y los claveles blancos parecían pequeñas nubes. María compró un ramo de margaritas para decorar su habitación. + +Más adelante, un señor amable ofrecía quesos de diferentes sabores. María probó un queso suave y cremoso que le encantó. También compró un poco de pan fresco para acompañarlo. + +En la sección de artesanías, encontró un artista que tallaba hermosos platos de madera. María admiró su trabajo y aprendió la palabra "tallar", que significaba dar forma a la madera con cuidado. + +Al final de su aventura, María se sintió feliz y orgullosa de haber descubierto tantas cosas nuevas. Había aprendido vocabulario relacionado con los colores, los sabores, las texturas y las artes. El mercado se había convertido en un lugar mágico donde la simplicidad de las palabras se unía a la riqueza de las experiencias. + +Espero que esta historia te sea útil para ilustrar vocabulario sencillo en español. ¡Puedes adaptar y expandir la trama según tus necesidades! +``` + +Finally, you can directly download the raw models for research purposes because Cohere For AI has released [Aya Expanse 8B](https://huggingface.co/CohereForAI/aya-expanse-8b) and [Aya Expanse 32B](https://huggingface.co/CohereForAI/aya-expanse-32b) as open-weight models, through HuggingFace. What’s more, the massively multilingual instruction data used for development of these models has been [made available](https://huggingface.co/datasets/CohereForAI/aya_collection) for download as well. + +## Find More +We hope you’ve found this as fascinating as we do! If you want to see more substantial projects you can check out these notebooks (source): + +- [Multilingual Writing Assistant](https://colab.research.google.com/drive/1SRLWQ0HdYN_NbRMVVUHTDXb-LSMZWF60) +- [AyaMCooking](https://colab.research.google.com/drive/1-cnn4LXYoZ4ARBpnsjQM3sU7egOL_fLB?usp=sharing) +- [Multilingual Question-Answering System](https://colab.research.google.com/drive/1bbB8hzyzCJbfMVjsZPeh4yNEALJFGNQy?usp=sharing) diff --git a/fern/v1.yml b/fern/v1.yml index 60193fcc..e87e4d28 100644 --- a/fern/v1.yml +++ b/fern/v1.yml @@ -55,6 +55,8 @@ navigation: path: pages/models/cohere-embed.mdx - page: Rerank path: pages/models/rerank-2.mdx + - page: Aya + path: pages/models/aya.mdx - section: Text Generation contents: - page: Introduction to Text Generation at Cohere diff --git a/fern/v2.yml b/fern/v2.yml index 9876c1a6..95bf54f0 100644 --- a/fern/v2.yml +++ b/fern/v2.yml @@ -55,6 +55,8 @@ navigation: path: pages/models/cohere-embed.mdx - page: Rerank path: pages/models/rerank-2.mdx + - page: Aya + path: pages/models/aya.mdx - section: Text Generation contents: - page: Introduction to Text Generation at Cohere