Skip to content

Latest commit

 

History

History
78 lines (49 loc) · 2.95 KB

togetherai.md

File metadata and controls

78 lines (49 loc) · 2.95 KB
Raw

Together AI

Together AI

Together is an AI company that develops large language models (LLMs). It provides various platforms and models, such as OpenChatKit, RedPajama, and GPT-JT, to empower developers and researchers in the field of natural language processing (NLP). Together's focus is on open-source AI research and infrastructure, enabling collaboration and innovation in the rapidly growing AI landscape.

Interface Name

  • togetherai

Example Usage

const { LLMInterface } = require('llm-interface');

LLMInterface.setApiKey({'togetherai': process.env.TOGETHERAI_API_KEY});

async function main() {
  try {
    const response = await LLMInterface.sendMessage('togetherai', 'Explain the importance of low latency LLMs.');
    console.log(response.results);
  } catch (error) {
    console.error(error);
    throw error;
  }
}

main();

Model Aliases

The following model aliases are provided for this provider.

  • default: google/gemma-7b
  • large: mistralai/Mixtral-8x22B
  • small: google/gemma-2b
  • agent: Qwen/Qwen1.5-14B

Embeddings Model Aliases

  • default: bert-base-uncased
  • large: BAAI/bge-large-en-v1.5
  • small: BAAI/bge-base-en-v1.5

Options

The following parameters can be passed through options.

  • max_tokens: The maximum number of tokens that can be generated in the chat completion. The total length of input tokens and generated tokens is limited by the model's context length.
  • response_format: Defines the format of the AI's response. Setting this to { "type": "json_object" } enables JSON mode, ensuring the message generated by the model is valid JSON.
  • stream: If set, partial message deltas will be sent, similar to ChatGPT. Tokens will be sent as data-only server-sent events as they become available, with the stream terminated by a data: [DONE] message.
  • temperature: Controls the randomness of the AI's responses. A higher temperature results in more random outputs, while a lower temperature makes the output more focused and deterministic. Generally, it is recommended to alter this or top_p, but not both.
  • tool_choice: Specifies which external tools the AI can use to assist in generating its response.
  • tools: A list of external tools available for the AI to use in generating responses.

Features

  • Native JSON Mode
  • Streaming
  • Tools
  • Embeddings

Getting an API Key

Commercial with Free Trial: The Together AI API is a commercial product but offers a $5.00 credit to get started. No credit card is required initially.

To get an API key, first create a Together AI account, then visit the link below.

Together AI Documentation

Together AI documentation is available here.