diff --git a/docs/docs/reference/gen_notebooks/dspy_prompt_optimization.md b/docs/docs/reference/gen_notebooks/dspy_prompt_optimization.md index 68e72904f44..440688502a7 100644 --- a/docs/docs/reference/gen_notebooks/dspy_prompt_optimization.md +++ b/docs/docs/reference/gen_notebooks/dspy_prompt_optimization.md @@ -107,6 +107,10 @@ dspy_train_examples, dspy_val_examples = get_dataset( ) ``` +| ![](../../../static/img/dspy_prompt_optimization/datasets.gif) | +|--------------------------------------------------------------| +| The datasets, once published, can be explored in the Weave UI | + ## The DSPy Program [DSPy](https://dspy-docs.vercel.app) is a framework that pushes building new LM pipelines away from manipulating free-form strings and closer to programming (composing modular operators to build text transformation graphs) where a compiler automatically generates optimized LM invocation strategies and prompts from a program. @@ -195,7 +199,7 @@ prediction = baseline_model.predict(dspy_train_examples[0]["question"]) rich.print(prediction) ``` -| ![](https://i.imgur.com/woHRHjR.png) | +| ![](../../../static/img/dspy_prompt_optimization/dspy_module_trace.png) | |---| | Here's how you can explore the traces of the `CausalReasoningModule` in the Weave UI | @@ -286,9 +290,9 @@ optimized_model = optimizer.get_optimized_program( Running the evaluation causal reasoning dataset will cost approximately $0.04 in OpenAI credits. ::: -| ![](https://i.imgur.com/uXvbROM.png) | +| ![](../../../static/img/dspy_prompt_optimization/dspy_compile.png) | |---| -| You can explore the traces of the optimization process in the Weave UI. | +| You can explore the traces of the optimization process in the Weave UI. | Now that we have our optimized program (the optimized prompting strategy), let's evaluate it once again on our validation set and compare it with our baseline DSPy program. @@ -307,7 +311,7 @@ await evaluation.evaluate(optimized_model) Running the evaluation causal reasoning dataset will cost approximately $0.30 in OpenAI credits. ::: -| ![](https://i.imgur.com/hneuY2K.png) | +| ![](../../../static/img/dspy_prompt_optimization/eval_comparison.gif) | |---| | Comparing the evalution of the baseline program with the optimized one shows that the optimized program answers the causal reasoning questions with siginificantly more accuracy. | diff --git a/docs/docs/reference/gen_notebooks/parse_arxiv_papers.md b/docs/docs/reference/gen_notebooks/parse_arxiv_papers.md index 6c26e0534e7..f949e9312ce 100644 --- a/docs/docs/reference/gen_notebooks/parse_arxiv_papers.md +++ b/docs/docs/reference/gen_notebooks/parse_arxiv_papers.md @@ -190,7 +190,7 @@ rich.print(result) Executing this LLM workflow will cost approximately $0.05-$0.25 in OpenAI credits, depending on the number of attempts instructor needs makes to get the output in the desired format (which is set to 5). ::: -| ![](https://i.imgur.com/Etnjoyq.png) | +| ![](../../../static/img/parse_arxiv_papers/predict_trace.png) | |---| | Here's how you can explore the traces of the `ArxivModel` in the Weave UI | @@ -296,6 +296,6 @@ await evaluation.evaluate(arxiv_parser_model) Running the evaluation on 5 examples from evaluation dataset will cost approximately $0.25-$1.25 in OpenAI credits, depending on the number of attempts instructor needs makes to get the output in the desired format (which is set to 5) in evaluating each example. ::: -| ![](https://i.imgur.com/qFbt8T0.png) | -|---| +| ![](../../../static/img/parse_arxiv_papers/eval_trace.png) | +|-------------------------------------------------------------------------------| | Here's how you can explore and compare the evaluations traces in the Weave UI | diff --git a/docs/static/img/dspy_prompt_optimization/datasets.gif b/docs/static/img/dspy_prompt_optimization/datasets.gif new file mode 100644 index 00000000000..239c7c74767 Binary files /dev/null and b/docs/static/img/dspy_prompt_optimization/datasets.gif differ diff --git a/docs/static/img/dspy_prompt_optimization/dspy_compile.png b/docs/static/img/dspy_prompt_optimization/dspy_compile.png new file mode 100644 index 00000000000..908694aa5f9 Binary files /dev/null and b/docs/static/img/dspy_prompt_optimization/dspy_compile.png differ diff --git a/docs/static/img/dspy_prompt_optimization/dspy_module_trace.png b/docs/static/img/dspy_prompt_optimization/dspy_module_trace.png new file mode 100644 index 00000000000..333affd5c10 Binary files /dev/null and b/docs/static/img/dspy_prompt_optimization/dspy_module_trace.png differ diff --git a/docs/static/img/dspy_prompt_optimization/eval_comparison.gif b/docs/static/img/dspy_prompt_optimization/eval_comparison.gif new file mode 100644 index 00000000000..2faccf15427 Binary files /dev/null and b/docs/static/img/dspy_prompt_optimization/eval_comparison.gif differ diff --git a/docs/static/img/parse_arxiv_papers/eval_trace.png b/docs/static/img/parse_arxiv_papers/eval_trace.png new file mode 100644 index 00000000000..c56e360175e Binary files /dev/null and b/docs/static/img/parse_arxiv_papers/eval_trace.png differ diff --git a/docs/static/img/parse_arxiv_papers/predict_trace.png b/docs/static/img/parse_arxiv_papers/predict_trace.png new file mode 100644 index 00000000000..438ece9afeb Binary files /dev/null and b/docs/static/img/parse_arxiv_papers/predict_trace.png differ