cohere-ai · mrmer1 · Dec 9, 2024 · Dec 9, 2024
@@ -14,7 +14,7 @@ updatedAt: "Tue Jun 11 2024 02:43:00 GMT+0000 (Coordinated Universal Time)"
 
 ## Overview
 
-Structured Outputs is a feature for forcing the LLM's output to follow a specified format 100% of the time. This increases the reliability of LLM in enterprise applications where downstream applications expect the LLM output to be correctly formatted. By forcing the model to follow a structured schema, hallucinated fields and entries in structured data can be reliably eliminated.
+Structured Outputs is a feature that forces the LLM’s response to strictly follow a schema specified by the user. When Structured Outputs is turned on, the LLM will generate structured data that follows the desired schema, provided by the user, 100% of the time. This increases the reliability of LLMs in enterprise applications where downstream applications expect the LLM output to be correctly formatted. With Structured Outputs, hallucinated fields and entries in structured data can be reliably eliminated.
 
 Compatible models:
 - Command R 08 2024
@@ -74,8 +74,8 @@ By setting the `response_format` type to `"json_object"` in the Chat API, the ou
 When using  `{ "type": "json_object" }` your `message` should always explicitly instruct the model to generate a JSON (eg: _"Generate a JSON ..."_) . Otherwise the model may end up getting stuck generating an infinite stream of characters and eventually run out of context length.
 </Info>
 
-<Note title="Experimental"> 
-This feature is currently not supported in RAG mode.
+<Note title="Note"> 
+This feature is currently not supported in [RAG](https://docs.cohere.com/v1/docs/retrieval-augmented-generation-rag) mode.
 </Note>
 
 #### JSON Schema mode
@@ -123,15 +123,11 @@ In this schema, we defined three keys ("title," "author," "publication_year") an
 
 ```
 
-<Info title="Important"> 
-Note: Each schema provided will incur a latency overhead required for processing the schema. This is only applicable for the first few requests.
-</Info>
-
 ## Specifying a schema
 
 ### Generating nested objects
 
-The model can be configured to output objects with up to 5 levels of nesting. When a `schema` is specified, there are no limitations on the levels of nesting.
+In JSON Schema mode, there are no limitations on the levels of nesting. However, in JSON mode (no schema specified), nesting is limited to 5 levels.
 
 ### Schema constraints
 
@@ -162,3 +158,7 @@ We do not support the entirety of the [JSON Schema specification](https://json-s
 - Others:
     - `uniqueItems`
     - `additionalProperties`
+
+<Info title="Important"> 
+Note: Using Structured Outputs (in JSON Schema mode) will incur a latency overhead required for processing the structured schema. This increase in latency only applies for the first few requests, since the schema is cached afterwards.
+</Info>
@@ -328,18 +328,6 @@ outputs = [{"number": 2343}] # Not Great
 outputs = [{"sum": 2343}] # Better
 ```
 
-## What's Next?
-
-Here, we'll preview some of the functionality we plan on adding in the coming months.
-
-### Cohere-hosted Tools
-
-The model can currently handle any tool provided by the developer. That having been said, Cohere has implemented some pre-defined tools that users can leverage out-of-the-box.
-
-Specifically we're going to roll out a **Python interpreter** tool and a **Web search** tool. 
-
-Please [reach out](mailto:[email protected]) to join the beta.
-
 ## Getting started
 
 Check out [this notebook](https://github.com/cohere-ai/notebooks/blob/main/notebooks/agents/Vanilla_Tool_Use.ipynb) for a worked-out examples.

@@ -15,7 +15,7 @@ updatedAt: "Tue Jun 11 2024 02:43:00 GMT+0000 (Coordinated Universal Time)"
 
 ## Overview
 
-Structured Outputs is a feature for forcing the LLM's output to follow a specified format 100% of the time. This increases the reliability of LLM in enterprise applications where downstream applications expect the LLM output to be correctly formatted. By forcing the model to follow a structured schema, hallucinated fields and entries in structured data can be reliably eliminated.
+Structured Outputs is a feature that forces the LLM’s response to strictly follow a schema specified by the user. When Structured Outputs is turned on, the LLM will generate structured data that follows the desired schema, provided by the user, 100% of the time. This increases the reliability of LLMs in enterprise applications where downstream applications expect the LLM output to be correctly formatted. With Structured Outputs, hallucinated fields and entries in structured data can be reliably eliminated.
 
 Compatible models:
 - Command R 08 2024
@@ -27,7 +27,7 @@ Compatible models:
 
 There are two ways to use Structured Outputs:
 - **Structured Outputs (JSON)**. This is primarily used in text generation use cases.
-- **Structured Outputs (Tools)**. This is primarily used in tool use and agents use cases via function calling.
+- **Structured Outputs (Tools)**. Structured Outputs (Tools). This is primarily used in [tool use (or function calling)](https://docs.cohere.com/docs/tool-use) and [agents](https://docs.cohere.com/docs/multi-step-tool-use) use cases.
 
 <Note title="API Compatibility"> 
 Structured Outputs with Tools are only supported in [Chat API V2](https://docs.cohere.com/reference/chat#request.body.strict_tools) via the `strict_tools` parameter. This parameter is not supported in Chat API V1.
@@ -82,7 +82,7 @@ When using  `{ "type": "json_object" }` your `message` should always explicitly
 </Info>
 
 <Note title="Note"> 
-This feature is currently not supported in RAG mode.
+This feature is currently not supported in [RAG](https://docs.cohere.com/docs/retrieval-augmented-generation-rag) mode.
 </Note>
 
 #### JSON Schema mode
@@ -135,24 +135,20 @@ In this schema, we defined three keys ("title," "author," "publication_year") an
 
 ```
 
-<Info title="Important"> 
-Note: Each schema provided (in both JSON and Tools modes) will incur a latency overhead required for processing the schema. This is only applicable for the first few requests.
-</Info>
-
 ### Structured Outputs (Tools)
-When you use the Chat API with `tools`, setting the `strict_tools` parameter to `true` will enforce that every generated tool call follows the specified tool schema.
+When you use the Chat API with `tools` (see [tool use](https://docs.cohere.com/docs/tool-use) and [agents](https://docs.cohere.com/docs/multi-step-tool-use)), setting the `strict_tools` parameter to `True`  will enforce that the tool calls generated by the mode strictly adhere to the tool descriptions you provided.
 
-<Note title="Experimental"> 
-`strict_tools` is currently an experimental parameter. We’ll be iterating on this feature and are looking for feedback. Share your experience with us in the `#api-discussions` channel on [discord](https://discord.gg/co-mmunity) or via [email](mailto:[email protected]).
-</Note>
+Concretely, this means:
+- No hallucinated tool names
+- No hallucinated tool parameters
+- Every `required` parameter is included in the tool call
+- All parameters produce the requested data types
 
 With `strict_tools` enabled, the API will ensure that the tool names and tool parameters are generated according to the tool definitions. This eliminates tool name and parameter hallucinations, ensures that each parameter matches the specified data type, and that all required parameters are included in the model response.
 
 Additionally, this results in faster development. You don’t need to spend a lot of time prompt engineering the model to avoid hallucinations.
 
-In the example below, we create a tool that can retrieve weather data for a given location. The tool is called`get_weather` which contains a parameter called `location`. We then invoke the Chat API with `strict_tools` set to `true` to ensure that the generated tool calls always include the correct function and parameter names.
-
-When the `strict_tools` parameter is set to `true`, you can define a maximum of 200 fields across all tools being passed to an API call.
+In the example below, we create a tool that can retrieve weather data for a given location. The tool is called `get_weather` which contains a parameter called `location`. We then invoke the Chat API with `strict_tools` set to `true` to ensure that the generated tool calls always include the correct function and parameter names.
 
 ```python PYTHON {24}
 tools = [
@@ -183,18 +179,22 @@ response = co.chat(model="command-r-plus-08-2024",
 print(response.message.tool_calls)
 ```
 
+#### Important notes on using `strict_tools`
+- This parameter is only supported in Chat API V2 via the strict_tools parameter (not API V1).
+- You must specify at least one `required` parameter. Tools with only optional parameters are not supported in this mode.
+- You can define a maximum of 200 fields across all tools in a single Chat API call.
 
-<Info title="Important"> 
-When `strict_tools` is enabled, tool definitions that have optional parameters must specify at least one `required` parameter as well. Tools with only optional parameters are not supported in this mode.
-</Info>
+<Note title="Experimental"> 
+`strict_tools` is currently an experimental parameter. We’ll be iterating on this feature and are looking for feedback. Share your experience with us in the `#api-discussions` channel on [discord](https://discord.gg/co-mmunity) or via [email](mailto:[email protected]).
+</Note>
 
 ### When to Use Structured Outputs (JSON) vs. Structured Outputs (Tools)
 
 Structured Outputs (JSON) are ideal for text generation use cases where you want to format the model's responses to users in a specific way. 
 
 For example, when building a travel planner application, you might want the LLM to generate itineraries in a specific JSON format, allowing the application to use the output in the other parts of the application.
 
-Structured Outputs (Tools) are ideal for function calling, tool use or agents use cases where you need the model to interact with external data or services. For instance, you can grant the model access to functions that interact with databases or other APIs.
+Structured Outputs (Tools) are ideal for [tool use (or function calling)](https://docs.cohere.com/docs/tool-use) and [agents](https://docs.cohere.com/docs/multi-step-tool-use) use cases where you need the model to interact with external data or services. For instance, you can grant the model access to functions that interact with databases or other APIs.
 
 In summary, opt for:
 - Structured Outputs (JSON) when you need the model's response to follow a specific structure.
@@ -205,7 +205,7 @@ In summary, opt for:
 
 ### Generating nested objects
 
-The model can be configured to output objects with up to 5 levels of nesting. When a `schema` is specified, there are no limitations on the levels of nesting.
+In JSON Schema mode, there are no limitations on the levels of nesting. However, in JSON mode (no schema specified), nesting is limited to 5 levels.
 
 ### Schema constraints
 
@@ -236,3 +236,7 @@ We do not support the entirety of the [JSON Schema specification](https://json-s
 - Others:
     - `uniqueItems`
     - `additionalProperties`
+
+<Info title="Important"> 
+Note: Using Structured Outputs (in both JSON Schema and Tools modes) will incur a latency overhead required for processing the structured schema. This increase in latency only applies for the first few requests, since the schema is cached afterwards.
+</Info>
@@ -164,6 +164,19 @@ Tool use is a natural extension of retrieval augmented generation (RAG). RAG is
 
 Tool use pushes this further, allowing Cohere models to go far beyond information retrieval, interact with search engines, APIs, functions, databases, and many other tools.
 
+## Structured Outputs (Tools)
+
+Setting the `strict_tools` parameter to `True` will enforce each tool call to follow the specified tool schema. To learn more about this feature, visit the [Structured Outputs documentation](https://docs.cohere.com/v2/docs/structured-outputs).
+
+Note that `strict_tools` is currently an experimental feature.
+
+```python PYTHON {4}
+response = co.chat(model="command-r-plus-08-2024",
+                   messages=messages,
+                   tools=tools,
+                   strict_tools=True)
+```
+
 ## A Further Example With Multiple Tools
 
 This section provides another example of multi-step tool use, this time with multiple tools. The notebook for this example can be [found here](https://github.com/cohere-ai/notebooks/blob/main/notebooks/agents/Multi_Step_Tool_Use_Spotify_v2.ipynb).

@@ -364,7 +364,7 @@ Start: 207 | End: 232 | Text: 'Tablet: $300, 25 in stock'
 
 ## Structured Outputs (Tools)
 
-Setting the `strict_tools` parameter to True will enforce each tool call to follow the specified tool schema. To learn more about this feature, visit the [Structured Outputs documentation](https://docs.cohere.com/v2/docs/structured-outputs).
+Setting the `strict_tools` parameter to `True` will enforce each tool call to follow the specified tool schema. To learn more about this feature, visit the [Structured Outputs documentation](https://docs.cohere.com/v2/docs/structured-outputs).
 
 Note that `strict_tools` is currently an experimental feature.
 
@@ -386,18 +386,6 @@ outputs = [{"number": 2343}] # Not Great
 outputs = [{"sum": 2343}] # Better
 ```
 
-## What's Next?
-
-Here, we'll preview some of the functionality we plan on adding in the coming months.
-
-### Cohere-hosted Tools
-
-The model can currently handle any tool provided by the developer. That having been said, Cohere has implemented some pre-defined tools that users can leverage out-of-the-box.
-
-Specifically we're going to roll out a **Python interpreter** tool and a **Web search** tool. 
-
-Please [reach out](mailto:[email protected]) to join the beta.
-
 ## Getting started
 
 Check out [this notebook](https://github.com/cohere-ai/notebooks/blob/main/notebooks/agents/Vanilla_Tool_Use_v2.ipynb) for a worked-out examples.