diff --git a/docs/docs/how_to/function_calling.ipynb b/docs/docs/how_to/function_calling.ipynb index 27982d0b2ccc8..d52963c1a307d 100644 --- a/docs/docs/how_to/function_calling.ipynb +++ b/docs/docs/how_to/function_calling.ipynb @@ -696,7 +696,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.10.1" + "version": "3.9.1" } }, "nbformat": 4, diff --git a/docs/docs/how_to/structured_output.ipynb b/docs/docs/how_to/structured_output.ipynb index a3828ccd8162a..a7b0385027c6e 100644 --- a/docs/docs/how_to/structured_output.ipynb +++ b/docs/docs/how_to/structured_output.ipynb @@ -24,18 +24,21 @@ "- [Function/tool calling](/docs/concepts/#functiontool-calling)\n", ":::\n", "\n", - "It is often useful to have a model return output that matches some specific schema. One common use-case is extracting data from arbitrary text to insert into a traditional database or use with some other downstrem system. This guide will show you a few different strategies you can use to do this.\n", - "\n", + "It is often useful to have a model return output that matches a specific schema. One common use-case is extracting data from text to insert into a database or use with some other downstream system. This guide covers a few strategies for getting structured outputs from a model.\n", "\n", "## The `.with_structured_output()` method\n", "\n", - "There are several strategies that models can use under the hood. For some of the most popular model providers, including [OpenAI](/docs/integrations/platforms/openai/), [Anthropic](/docs/integrations/platforms/anthropic/), and [Mistral](/docs/integrations/providers/mistralai/), LangChain implements a common interface that abstracts away these strategies called `.with_structured_output`.\n", + ":::info Supported models\n", + "\n", + "You can find a [list of models that support this method here](/docs/integrations/chat/).\n", "\n", - "By invoking this method (and passing in [JSON schema](https://json-schema.org/) or a [Pydantic](https://docs.pydantic.dev/latest/) model) the model will add whatever model parameters + output parsers are necessary to get back structured output matching the requested schema. If the model supports more than one way to do this (e.g., function calling vs JSON mode) - you can configure which method to use by passing into that method.\n", + ":::\n", "\n", - "You can find the [current list of models that support this method here](/docs/integrations/chat/).\n", + "This is the easiest and most reliable way to get structured outputs. `with_structured_output()` is implemented for models that provide native APIs for structuring outputs, like tool/function calling or JSON mode, and makes use of these capabilities under the hood.\n", "\n", - "Let's look at some examples of this in action! We'll use Pydantic to create a simple response schema.\n", + "This method takes a schema as input which specifies the names, types, and descriptions of the desired output attributes. The method returns a model-like Runnable, except that instead of outputting strings or Messages it outputs objects corresponding to the given schema. The schema can be specified as a [JSON Schema](https://json-schema.org/) or a Pydantic class. If JSON Schema is used then a dictionary will be returned by the Runnable, and if a Pydantic class is used then Pydantic objects will be returned.\n", + "\n", + "As an example, let's get a model to generate a joke and separate the setup from the punchline:\n", "\n", "```{=mdx}\n", "import ChatModelTabs from \"@theme/ChatModelTabs\";\n", @@ -58,25 +61,30 @@ "\n", "from langchain_openai import ChatOpenAI\n", "\n", - "llm = ChatOpenAI(\n", - " model=\"gpt-4-0125-preview\",\n", - " temperature=0,\n", - ")" + "llm = ChatOpenAI(model=\"gpt-4-0125-preview\", temperature=0)" + ] + }, + { + "cell_type": "markdown", + "id": "a808a401-be1f-49f9-ad13-58dd68f7db5f", + "metadata": {}, + "source": [ + "If we want the model to return a Pydantic object, we just need to pass in desired the Pydantic class:" ] }, { "cell_type": "code", - "execution_count": 13, + "execution_count": 38, "id": "070bf702", "metadata": {}, "outputs": [ { "data": { "text/plain": [ - "Joke(setup='Why was the cat sitting on the computer?', punchline='Because it wanted to keep an eye on the mouse!', rating=None)" + "Joke(setup='Why was the cat sitting on the computer?', punchline='To keep an eye on the mouse!', rating=None)" ] }, - "execution_count": 13, + "execution_count": 38, "metadata": {}, "output_type": "execute_result" } @@ -88,6 +96,8 @@ "\n", "\n", "class Joke(BaseModel):\n", + " \"\"\"Joke to tell user.\"\"\"\n", + "\n", " setup: str = Field(description=\"The setup of the joke\")\n", " punchline: str = Field(description=\"The punchline to the joke\")\n", " rating: Optional[int] = Field(description=\"How funny the joke is, from 1 to 10\")\n", @@ -98,25 +108,27 @@ "structured_llm.invoke(\"Tell me a joke about cats\")" ] }, + { + "cell_type": "markdown", + "id": "00890a47-3cdf-4805-b8f1-6d110f0633d3", + "metadata": {}, + "source": [ + ":::tip\n", + "Beyond just the structure of the Pydantic class, the name of the Pydantic class, the docstring, and the names and provided descriptions of parameters are very important. Most of the time `with_structured_output` is using a model's function/tool calling API, and you can effectively think of all of this information as being added to the model prompt.\n", + ":::" + ] + }, { "cell_type": "markdown", "id": "deddb6d3", "metadata": {}, "source": [ - "The result is a Pydantic model. Note that name of the model and the names and provided descriptions of parameters are very important, as they help guide the model's output.\n", - "\n", - "We can also pass in an OpenAI-style JSON schema dict if you prefer not to use Pydantic. This dict should contain three properties:\n", - "\n", - "- `name`: The name of the schema to output.\n", - "- `description`: A high level description of the schema to output.\n", - "- `parameters`: The nested details of the schema you want to extract, formatted as a [JSON schema](https://json-schema.org/) dict.\n", - "\n", - "In this case, the response is also a dict:" + "We can also pass in a [JSON Schema](https://json-schema.org/) dict if you prefer not to use Pydantic. In this case, the response is also a dict:" ] }, { "cell_type": "code", - "execution_count": 3, + "execution_count": 8, "id": "6700994a", "metadata": {}, "outputs": [ @@ -124,30 +136,37 @@ "data": { "text/plain": [ "{'setup': 'Why was the cat sitting on the computer?',\n", - " 'punchline': 'To keep an eye on the mouse!'}" + " 'punchline': 'Because it wanted to keep an eye on the mouse!',\n", + " 'rating': 8}" ] }, - "execution_count": 3, + "execution_count": 8, "metadata": {}, "output_type": "execute_result" } ], "source": [ - "structured_llm = llm.with_structured_output(\n", - " {\n", - " \"name\": \"joke\",\n", - " \"description\": \"Joke to tell user.\",\n", - " \"parameters\": {\n", - " \"title\": \"Joke\",\n", - " \"type\": \"object\",\n", - " \"properties\": {\n", - " \"setup\": {\"type\": \"string\", \"description\": \"The setup for the joke\"},\n", - " \"punchline\": {\"type\": \"string\", \"description\": \"The joke's punchline\"},\n", - " },\n", - " \"required\": [\"setup\", \"punchline\"],\n", + "json_schema = {\n", + " \"title\": \"joke\",\n", + " \"description\": \"Joke to tell user.\",\n", + " \"type\": \"object\",\n", + " \"properties\": {\n", + " \"setup\": {\n", + " \"type\": \"string\",\n", + " \"description\": \"The setup of the joke\",\n", " },\n", - " }\n", - ")\n", + " \"punchline\": {\n", + " \"type\": \"string\",\n", + " \"description\": \"The punchline to the joke\",\n", + " },\n", + " \"rating\": {\n", + " \"type\": \"integer\",\n", + " \"description\": \"How funny the joke is, from 1 to 10\",\n", + " },\n", + " },\n", + " \"required\": [\"setup\", \"punchline\"],\n", + "}\n", + "structured_llm = llm.with_structured_output(json_schema)\n", "\n", "structured_llm.invoke(\"Tell me a joke about cats\")" ] @@ -159,7 +178,7 @@ "source": [ "### Choosing between multiple schemas\n", "\n", - "If you have multiple schemas that are valid outputs for the model, you can use Pydantic's `Union` type:" + "The simplest way to let the model choose from multiple schemas is to create a parent Pydantic class that has a Union-typed attribute:" ] }, { @@ -171,7 +190,7 @@ { "data": { "text/plain": [ - "Response(output=Joke(setup='Why was the cat sitting on the computer?', punchline='Because it wanted to keep an eye on the mouse!'))" + "Response(output=Joke(setup='Why was the cat sitting on the computer?', punchline='To keep an eye on the mouse!', rating=8))" ] }, "execution_count": 4, @@ -182,15 +201,10 @@ "source": [ "from typing import Union\n", "\n", - "from langchain_core.pydantic_v1 import BaseModel, Field\n", - "\n", - "\n", - "class Joke(BaseModel):\n", - " setup: str = Field(description=\"The setup of the joke\")\n", - " punchline: str = Field(description=\"The punchline to the joke\")\n", - "\n", "\n", "class ConversationalResponse(BaseModel):\n", + " \"\"\"Respond in a conversational manner. Be kind and helpful.\"\"\"\n", + "\n", " response: str = Field(description=\"A conversational response to the user's query\")\n", "\n", "\n", @@ -212,7 +226,7 @@ { "data": { "text/plain": [ - "Response(output=ConversationalResponse(response=\"I'm just a collection of code, so I don't have feelings, but thanks for asking! How can I assist you today?\"))" + "Response(output=ConversationalResponse(response=\"I'm just a digital assistant, so I don't have feelings, but I'm here and ready to help you. How can I assist you today?\"))" ] }, "execution_count": 5, @@ -229,9 +243,225 @@ "id": "e28c14d3", "metadata": {}, "source": [ - "If you are using JSON Schema, you can take advantage of other more complex schema descriptions to create a similar effect.\n", + "Alternatively, you can use tool calling directly to allow the model to choose between options, if your [chosen model supports it](/docs/integrations/chat/). This involves a bit more parsing and setup but in some instances leads to better performance because you don't have to use nested schemas. See [this how-to guide](/docs/how_to/tool_calling/) for more details." + ] + }, + { + "cell_type": "markdown", + "id": "9a40f703-7fd2-4fe0-ab2a-fa2d711ba009", + "metadata": {}, + "source": [ + "### Streaming\n", + "\n", + "We can stream outputs from our structured model when the output type is a dict (i.e., when the schema is specified as a JSON Schema dict). \n", + "\n", + ":::info\n", + "\n", + "Note that what's yielded is already aggregated chunks, not deltas.\n", + "\n", + ":::" + ] + }, + { + "cell_type": "code", + "execution_count": 43, + "id": "aff89877-28a3-472f-a1aa-eff893fe7736", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{}\n", + "{'setup': ''}\n", + "{'setup': 'Why'}\n", + "{'setup': 'Why was'}\n", + "{'setup': 'Why was the'}\n", + "{'setup': 'Why was the cat'}\n", + "{'setup': 'Why was the cat sitting'}\n", + "{'setup': 'Why was the cat sitting on'}\n", + "{'setup': 'Why was the cat sitting on the'}\n", + "{'setup': 'Why was the cat sitting on the computer'}\n", + "{'setup': 'Why was the cat sitting on the computer?'}\n", + "{'setup': 'Why was the cat sitting on the computer?', 'punchline': ''}\n", + "{'setup': 'Why was the cat sitting on the computer?', 'punchline': 'Because'}\n", + "{'setup': 'Why was the cat sitting on the computer?', 'punchline': 'Because it'}\n", + "{'setup': 'Why was the cat sitting on the computer?', 'punchline': 'Because it wanted'}\n", + "{'setup': 'Why was the cat sitting on the computer?', 'punchline': 'Because it wanted to'}\n", + "{'setup': 'Why was the cat sitting on the computer?', 'punchline': 'Because it wanted to keep'}\n", + "{'setup': 'Why was the cat sitting on the computer?', 'punchline': 'Because it wanted to keep an'}\n", + "{'setup': 'Why was the cat sitting on the computer?', 'punchline': 'Because it wanted to keep an eye'}\n", + "{'setup': 'Why was the cat sitting on the computer?', 'punchline': 'Because it wanted to keep an eye on'}\n", + "{'setup': 'Why was the cat sitting on the computer?', 'punchline': 'Because it wanted to keep an eye on the'}\n", + "{'setup': 'Why was the cat sitting on the computer?', 'punchline': 'Because it wanted to keep an eye on the mouse'}\n", + "{'setup': 'Why was the cat sitting on the computer?', 'punchline': 'Because it wanted to keep an eye on the mouse!'}\n", + "{'setup': 'Why was the cat sitting on the computer?', 'punchline': 'Because it wanted to keep an eye on the mouse!', 'rating': 8}\n" + ] + } + ], + "source": [ + "structured_llm = llm.with_structured_output(json_schema)\n", + "\n", + "for chunk in structured_llm.stream(\"Tell me a joke about cats\"):\n", + " print(chunk)" + ] + }, + { + "cell_type": "markdown", + "id": "0a526cdf-e736-451b-96be-22e8986d3863", + "metadata": {}, + "source": [ + "### Few-shot prompting\n", + "\n", + "For more complex schemas it's very useful to add few-shot examples to the prompt. This can be done in a few ways.\n", + "\n", + "The simplest and most universal way is to add examples to a system message in the prompt:" + ] + }, + { + "cell_type": "code", + "execution_count": 47, + "id": "283ba784-2072-47ee-9b2c-1119e3c69e8e", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "{'setup': 'Woodpecker',\n", + " 'punchline': \"Woodpecker goes 'knock knock', but don't worry, they never expect you to answer the door!\",\n", + " 'rating': 8}" + ] + }, + "execution_count": 47, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "from langchain_core.prompts import ChatPromptTemplate\n", + "\n", + "system = \"\"\"You are a hilarious comedian. Your specialty is knock-knock jokes. \\\n", + "Return a joke which has the setup (the response to \"Who's there?\") and the final punchline (the response to \" who?\").\n", + "\n", + "Here are some examples of jokes:\n", + "\n", + "example_user: Tell me a joke about planes\n", + "example_assistant: {{\"setup\": \"Why don't planes ever get tired?\", \"punchline\": \"Because they have rest wings!\", \"rating\": 2}}\n", "\n", - "You can also use tool calling directly to allow the model to choose between options, if your chosen model supports it. This involves a bit more parsing and setup. See [this how-to guide](/docs/how_to/tool_calling/) for more details." + "example_user: Tell me another joke about planes\n", + "example_assistant: {{\"setup\": \"Cargo\", \"punchline\": \"Cargo 'vroom vroom', but planes go 'zoom zoom'!\", \"rating\": 10}}\n", + "\n", + "example_user: Now about caterpillars\n", + "example_assistant: {{\"setup\": \"Caterpillar\", \"punchline\": \"Caterpillar really slow, but watch me turn into a butterfly and steal the show!\", \"rating\": 5}}\"\"\"\n", + "\n", + "prompt = ChatPromptTemplate.from_messages([(\"system\", system), (\"human\", \"{input}\")])\n", + "\n", + "few_shot_structured_llm = prompt | structured_llm\n", + "few_shot_structured_llm.invoke(\"what's something funny about woodpeckers\")" + ] + }, + { + "cell_type": "markdown", + "id": "3c12b389-153d-44d1-af34-37e5b926d3db", + "metadata": {}, + "source": [ + "When the underlying method for structuring outputs is tool calling, we can pass in our examples as explicit tool calls. You can check if the model you're using makes use of tool calling in its API reference." + ] + }, + { + "cell_type": "code", + "execution_count": 46, + "id": "d7381cb0-b2c3-4302-a319-ed72d0b9e43f", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "{'setup': 'Crocodile',\n", + " 'punchline': \"Crocodile 'see you later', but in a while, it becomes an alligator!\",\n", + " 'rating': 7}" + ] + }, + "execution_count": 46, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "from langchain_core.messages import AIMessage, HumanMessage, ToolMessage\n", + "\n", + "examples = [\n", + " HumanMessage(\"Tell me a joke about planes\", name=\"example_user\"),\n", + " AIMessage(\n", + " \"\",\n", + " name=\"example_assistant\",\n", + " tool_calls=[\n", + " {\n", + " \"name\": \"joke\",\n", + " \"args\": {\n", + " \"setup\": \"Why don't planes ever get tired?\",\n", + " \"punchline\": \"Because they have rest wings!\",\n", + " \"rating\": 2,\n", + " },\n", + " \"id\": \"1\",\n", + " }\n", + " ],\n", + " ),\n", + " # Most tool-calling models expect a ToolMessage(s) to follow an AIMessage with tool calls.\n", + " ToolMessage(\"\", tool_call_id=\"1\"),\n", + " # Some models also expect an AIMessage to follow any ToolMessages,\n", + " # so you may need to add an AIMessage here.\n", + " HumanMessage(\"Tell me another joke about planes\", name=\"example_user\"),\n", + " AIMessage(\n", + " \"\",\n", + " name=\"example_assistant\",\n", + " tool_calls=[\n", + " {\n", + " \"name\": \"joke\",\n", + " \"args\": {\n", + " \"setup\": \"Cargo\",\n", + " \"punchline\": \"Cargo 'vroom vroom', but planes go 'zoom zoom'!\",\n", + " \"rating\": 10,\n", + " },\n", + " \"id\": \"2\",\n", + " }\n", + " ],\n", + " ),\n", + " ToolMessage(\"\", tool_call_id=\"2\"),\n", + " HumanMessage(\"Now about caterpillars\", name=\"example_user\"),\n", + " AIMessage(\n", + " \"\",\n", + " tool_calls=[\n", + " {\n", + " \"name\": \"joke\",\n", + " \"args\": {\n", + " \"setup\": \"Caterpillar\",\n", + " \"punchline\": \"Caterpillar really slow, but watch me turn into a butterfly and steal the show!\",\n", + " \"rating\": 5,\n", + " },\n", + " \"id\": \"3\",\n", + " }\n", + " ],\n", + " ),\n", + " ToolMessage(\"\", tool_call_id=\"3\"),\n", + "]\n", + "system = \"\"\"You are a hilarious comedian. Your specialty is knock-knock jokes. \\\n", + "Return a joke which has the setup (the response to \"Who's there?\") \\\n", + "and the final punchline (the response to \" who?\").\"\"\"\n", + "\n", + "prompt = ChatPromptTemplate.from_messages(\n", + " [(\"system\", system), (\"placeholder\", \"{examples}\"), (\"human\", \"{input}\")]\n", + ")\n", + "few_shot_structured_llm = prompt | structured_llm\n", + "few_shot_structured_llm.invoke({\"input\": \"crocodiles\", \"examples\": examples})" + ] + }, + { + "cell_type": "markdown", + "id": "498d893b-ceaa-47ff-a9d8-4faa60702715", + "metadata": {}, + "source": [ + "For more on few shot prompting when using tool calling, see [here](/docs/how_to/function_calling/#Few-shot-prompting)." ] }, { @@ -239,9 +469,17 @@ "id": "39d7a555", "metadata": {}, "source": [ - "### Specifying the output method (Advanced)\n", + "### (Advanced) Specifying the method for structuring outputs\n", "\n", - "For models that support more than one means of outputting data, you can specify the preferred one like this:" + "For models that support more than one means of structuring outputs (i.e., they support both tool calling and JSON mode), you can specify which method to use with the `method=` argument.\n", + "\n", + ":::info JSON mode\n", + "\n", + "If using JSON mode you'll have to still specify the desired schema in the model prompt. The schema you pass to `with_structured_output` will only be used for parsing the model outputs, it will not be passed to the model the way it is with tool calling.\n", + "\n", + "To see if the model you're using supports JSON mode, check its entry in the [API reference](https://api.python.langchain.com/en/latest/langchain_api_reference.html).\n", + "\n", + ":::" ] }, { @@ -253,7 +491,7 @@ { "data": { "text/plain": [ - "Joke(setup='Why was the cat sitting on the computer?', punchline='Because it wanted to keep an eye on the mouse!')" + "Joke(setup='Why was the cat sitting on the computer?', punchline='Because it wanted to keep an eye on the mouse!', rating=None)" ] }, "execution_count": 6, @@ -274,13 +512,9 @@ "id": "5e92a98a", "metadata": {}, "source": [ - "In the above example, we use OpenAI's alternate JSON mode capability along with a more specific prompt.\n", + "## Prompting and parsing model directly\n", "\n", - "For specifics about the model you choose, peruse its entry in the [API reference pages](https://api.python.langchain.com/en/latest/langchain_api_reference.html).\n", - "\n", - "## Prompting techniques\n", - "\n", - "You can also prompt models to outputting information in a given format. This approach relies on designing good prompts and then parsing the output of the models. This is the only option for models that don't support `.with_structured_output()` or other built-in approaches.\n", + "Not all models support `.with_structured_output()`, since not all models have tool calling or JSON mode support. For such models you'll need to directly prompt the model to use a specific format, and use an output parser to extract the structured response from the raw model output.\n", "\n", "### Using `PydanticOutputParser`\n", "\n", @@ -289,7 +523,7 @@ }, { "cell_type": "code", - "execution_count": 7, + "execution_count": 31, "id": "6e514455", "metadata": {}, "outputs": [], @@ -341,7 +575,7 @@ }, { "cell_type": "code", - "execution_count": 8, + "execution_count": 37, "id": "3d73d33d", "metadata": {}, "outputs": [ @@ -366,7 +600,7 @@ "source": [ "query = \"Anna is 23 years old and she is 6 feet tall\"\n", "\n", - "print(prompt.format_prompt(query=query).to_string())" + "print(prompt.invoke(query).to_string())" ] }, { @@ -542,25 +776,13 @@ "\n", "chain.invoke({\"query\": query})" ] - }, - { - "cell_type": "markdown", - "id": "7a39221a", - "metadata": {}, - "source": [ - "## Next steps\n", - "\n", - "Now you've learned a few methods to make a model output structured data.\n", - "\n", - "To learn more, check out the other how-to guides in this section, or the conceptual guide on tool calling." - ] } ], "metadata": { "kernelspec": { - "display_name": "Python 3 (ipykernel)", + "display_name": "poetry-venv-2", "language": "python", - "name": "python3" + "name": "poetry-venv-2" }, "language_info": { "codemirror_mode": { diff --git a/docs/docs/integrations/chat/openai.ipynb b/docs/docs/integrations/chat/openai.ipynb index da75aeca4b803..6d9ef130c7079 100644 --- a/docs/docs/integrations/chat/openai.ipynb +++ b/docs/docs/integrations/chat/openai.ipynb @@ -147,7 +147,7 @@ "\n", "### ChatOpenAI.bind_tools()\n", "\n", - "With `ChatAnthropic.bind_tools`, we can easily pass in Pydantic classes, dict schemas, LangChain tools, or even functions as tools to the model. Under the hood these are converted to an Anthropic tool schemas, which looks like:\n", + "With `ChatOpenAI.bind_tools`, we can easily pass in Pydantic classes, dict schemas, LangChain tools, or even functions as tools to the model. Under the hood these are converted to an OpenAI tool schemas, which looks like:\n", "```\n", "{\n", " \"name\": \"...\",\n", diff --git a/docs/docusaurus.config.js b/docs/docusaurus.config.js index 576eaad18995f..029e993b4ec7c 100644 --- a/docs/docusaurus.config.js +++ b/docs/docusaurus.config.js @@ -124,7 +124,7 @@ const config = { /** @type {import('@docusaurus/preset-classic').ThemeConfig} */ ({ announcementBar: { - content: 'You are viewing the preview LangChain v0.2 docs. Note that 0.2 Search features are currently unstable and in progress. View the stable 0.1 docs here.', + content: 'You are viewing the preview LangChain v0.2 docs. View the stable 0.1 docs here.', isCloseable: true, }, docs: { @@ -310,9 +310,9 @@ const config = { // this is linked to erick@langchain.dev currently apiKey: "6c01842d6a88772ed2236b9c85806441", - indexName: "python-langchain", + indexName: "python-langchain-0.2", - contextualSearch: true, + contextualSearch: false, }, }), diff --git a/docs/scripts/notebook_convert.py b/docs/scripts/notebook_convert.py index 97f2dca253417..af3a0e576da40 100644 --- a/docs/scripts/notebook_convert.py +++ b/docs/scripts/notebook_convert.py @@ -84,12 +84,8 @@ def check_conditions(self, cell): pattern = re.compile(r"(?s)(?:\s*\Z)|(?:.*#\s*\|\s*output:\s*false.*)") rtn = not pattern.match(cell.source) if not rtn: - print("--remove--") - print(cell.source) return False else: - print("--keep--") - print(cell.source) return True def preprocess(self, nb, resources): diff --git a/libs/core/langchain_core/language_models/base.py b/libs/core/langchain_core/language_models/base.py index 39ba2ba69b417..afabc7c9fddc7 100644 --- a/libs/core/langchain_core/language_models/base.py +++ b/libs/core/langchain_core/language_models/base.py @@ -204,7 +204,9 @@ async def agenerate_prompt( def with_structured_output( self, schema: Union[Dict, Type[BaseModel]], **kwargs: Any ) -> Runnable[LanguageModelInput, Union[Dict, BaseModel]]: - """Implement this if there is a way of steering the model to generate responses that match a given schema.""" # noqa: E501 + """Not implemented on this class.""" + # Implement this on child class if there is a way of steering the model to + # generate responses that match a given schema. raise NotImplementedError() @deprecated("0.1.7", alternative="invoke", removal="0.3.0") diff --git a/libs/core/pyproject.toml b/libs/core/pyproject.toml index 17ffaa4ed848c..429a3c14bf52e 100644 --- a/libs/core/pyproject.toml +++ b/libs/core/pyproject.toml @@ -80,6 +80,11 @@ select = [ disallow_untyped_defs = "True" exclude = ["notebooks", "examples", "example_data", "langchain_core/pydantic"] +[[tool.mypy.overrides]] +# conditional dependencies introduced by langsmith-sdk +module = ["numpy", "pytest"] +ignore_missing_imports = true + [tool.coverage.run] omit = ["tests/*"] diff --git a/libs/partners/mongodb/langchain_mongodb/vectorstores.py b/libs/partners/mongodb/langchain_mongodb/vectorstores.py index c87b60b7030ef..a703d59c510a1 100644 --- a/libs/partners/mongodb/langchain_mongodb/vectorstores.py +++ b/libs/partners/mongodb/langchain_mongodb/vectorstores.py @@ -16,6 +16,7 @@ ) import numpy as np +from bson import ObjectId, json_util from langchain_core.documents import Document from langchain_core.embeddings import Embeddings from langchain_core.runnables.config import run_in_executor @@ -31,7 +32,7 @@ logger = logging.getLogger(__name__) -DEFAULT_INSERT_BATCH_SIZE = 100 +DEFAULT_INSERT_BATCH_SIZE = 100_000 class MongoDBAtlasVectorSearch(VectorStore): @@ -150,18 +151,24 @@ def add_texts( """ batch_size = kwargs.get("batch_size", DEFAULT_INSERT_BATCH_SIZE) _metadatas: Union[List, Generator] = metadatas or ({} for _ in texts) - texts_batch = [] - metadatas_batch = [] + texts_batch = texts + metadatas_batch = _metadatas result_ids = [] - for i, (text, metadata) in enumerate(zip(texts, _metadatas)): - texts_batch.append(text) - metadatas_batch.append(metadata) - if (i + 1) % batch_size == 0: - result_ids.extend(self._insert_texts(texts_batch, metadatas_batch)) - texts_batch = [] - metadatas_batch = [] + if batch_size: + texts_batch = [] + metadatas_batch = [] + size = 0 + for i, (text, metadata) in enumerate(zip(texts, _metadatas)): + size += len(text) + len(metadata) + texts_batch.append(text) + metadatas_batch.append(metadata) + if (i + 1) % batch_size == 0 or size >= 47_000_000: + result_ids.extend(self._insert_texts(texts_batch, metadatas_batch)) + texts_batch = [] + metadatas_batch = [] + size = 0 if texts_batch: - result_ids.extend(self._insert_texts(texts_batch, metadatas_batch)) + result_ids.extend(self._insert_texts(texts_batch, metadatas_batch)) # type: ignore return result_ids def _insert_texts(self, texts: List[str], metadatas: List[Dict[str, Any]]) -> List: @@ -210,9 +217,23 @@ def _similarity_search_with_score( pipeline.extend(post_filter_pipeline) cursor = self._collection.aggregate(pipeline) # type: ignore[arg-type] docs = [] + + def _make_serializable(obj: Dict[str, Any]) -> None: + for k, v in obj.items(): + if isinstance(v, dict): + _make_serializable(v) + elif isinstance(v, list) and v and isinstance(v[0], ObjectId): + obj[k] = [json_util.default(item) for item in v] + elif isinstance(v, ObjectId): + obj[k] = json_util.default(v) + for res in cursor: text = res.pop(self._text_key) score = res.pop("score") + # Make every ObjectId found JSON-Serializable + # following format used in bson.json_util.loads + # e.g. loads('{"_id": {"$oid": "664..."}}') == {'_id': ObjectId('664..')} # noqa: E501 + _make_serializable(res) docs.append((Document(page_content=text, metadata=res), score)) return docs diff --git a/libs/partners/mongodb/pyproject.toml b/libs/partners/mongodb/pyproject.toml index 43ec7f6084ebc..634ab0536f900 100644 --- a/libs/partners/mongodb/pyproject.toml +++ b/libs/partners/mongodb/pyproject.toml @@ -1,6 +1,6 @@ [tool.poetry] name = "langchain-mongodb" -version = "0.1.3" +version = "0.1.4" description = "An integration package connecting MongoDB and LangChain" authors = [] readme = "README.md" @@ -28,7 +28,7 @@ pytest-watcher = "^0.3.4" pytest-asyncio = "^0.21.1" langchain = { path = "../../langchain", develop = true } langchain-core = { path = "../../core", develop = true } -langchain-text-splitters = {path = "../../text-splitters", develop = true} +langchain-text-splitters = { path = "../../text-splitters", develop = true } [tool.poetry.group.codespell] optional = true diff --git a/libs/partners/mongodb/tests/unit_tests/test_vectorstores.py b/libs/partners/mongodb/tests/unit_tests/test_vectorstores.py index 9d3def6046285..9c6c781208cb7 100644 --- a/libs/partners/mongodb/tests/unit_tests/test_vectorstores.py +++ b/libs/partners/mongodb/tests/unit_tests/test_vectorstores.py @@ -1,6 +1,8 @@ +from json import dumps, loads from typing import Any, Optional import pytest +from bson import ObjectId, json_util from langchain_core.documents import Document from langchain_core.embeddings import Embeddings from pymongo.collection import Collection @@ -75,6 +77,11 @@ def _validate_search( output = vectorstore.similarity_search("", k=1) assert output[0].page_content == page_content assert output[0].metadata.get("c") == metadata + # Validate the ObjectId provided is json serializable + assert loads(dumps(output[0].page_content)) == output[0].page_content + assert loads(dumps(output[0].metadata)) == output[0].metadata + json_metadata = dumps(output[0].metadata) # normal json.dumps + assert isinstance(json_util.loads(json_metadata)["_id"], ObjectId) def test_from_documents( self, embedding_openai: Embeddings, collection: MockCollection diff --git a/libs/partners/mongodb/tests/utils.py b/libs/partners/mongodb/tests/utils.py index 3716ac69478cd..7b06991da82f9 100644 --- a/libs/partners/mongodb/tests/utils.py +++ b/libs/partners/mongodb/tests/utils.py @@ -1,9 +1,9 @@ from __future__ import annotations -import uuid from copy import deepcopy from typing import Any, Dict, List, Mapping, Optional, cast +from bson import ObjectId from langchain_core.callbacks.manager import ( AsyncCallbackManagerForLLMRun, CallbackManagerForLLMRun, @@ -162,7 +162,7 @@ def delete_many(self, *args, **kwargs) -> DeleteResult: # type: ignore def insert_many(self, to_insert: List[Any], *args, **kwargs) -> InsertManyResult: # type: ignore mongodb_inserts = [ - {"_id": str(uuid.uuid4()), "score": 1, **insert} for insert in to_insert + {"_id": ObjectId(), "score": 1, **insert} for insert in to_insert ] self._data.extend(mongodb_inserts) return self._insert_result or InsertManyResult(