Skip to content

Commit

Permalink
add: images
Browse files Browse the repository at this point in the history
  • Loading branch information
soumik12345 committed Aug 22, 2024
1 parent 5e9f433 commit dff078e
Show file tree
Hide file tree
Showing 8 changed files with 11 additions and 7 deletions.
12 changes: 8 additions & 4 deletions docs/docs/reference/gen_notebooks/dspy_prompt_optimization.md
Original file line number Diff line number Diff line change
Expand Up @@ -107,6 +107,10 @@ dspy_train_examples, dspy_val_examples = get_dataset(
)
```

| ![](../../../static/img/dspy_prompt_optimization/datasets.gif) |
|--------------------------------------------------------------|
| The datasets, once published, can be explored in the Weave UI |

## The DSPy Program

[DSPy](https://dspy-docs.vercel.app) is a framework that pushes building new LM pipelines away from manipulating free-form strings and closer to programming (composing modular operators to build text transformation graphs) where a compiler automatically generates optimized LM invocation strategies and prompts from a program.
Expand Down Expand Up @@ -195,7 +199,7 @@ prediction = baseline_model.predict(dspy_train_examples[0]["question"])
rich.print(prediction)
```

| ![](https://i.imgur.com/woHRHjR.png) |
| ![](../../../static/img/dspy_prompt_optimization/dspy_module_trace.png) |
|---|
| Here's how you can explore the traces of the `CausalReasoningModule` in the Weave UI |

Expand Down Expand Up @@ -286,9 +290,9 @@ optimized_model = optimizer.get_optimized_program(
Running the evaluation causal reasoning dataset will cost approximately $0.04 in OpenAI credits.
:::

| ![](https://i.imgur.com/uXvbROM.png) |
| ![](../../../static/img/dspy_prompt_optimization/dspy_compile.png) |
|---|
| You can explore the traces of the optimization process in the Weave UI. |
| You can explore the traces of the optimization process in the Weave UI. |

Now that we have our optimized program (the optimized prompting strategy), let's evaluate it once again on our validation set and compare it with our baseline DSPy program.

Expand All @@ -307,7 +311,7 @@ await evaluation.evaluate(optimized_model)
Running the evaluation causal reasoning dataset will cost approximately $0.30 in OpenAI credits.
:::

| ![](https://i.imgur.com/hneuY2K.png) |
| ![](../../../static/img/dspy_prompt_optimization/eval_comparison.gif) |
|---|
| Comparing the evalution of the baseline program with the optimized one shows that the optimized program answers the causal reasoning questions with siginificantly more accuracy. |

Expand Down
6 changes: 3 additions & 3 deletions docs/docs/reference/gen_notebooks/parse_arxiv_papers.md
Original file line number Diff line number Diff line change
Expand Up @@ -190,7 +190,7 @@ rich.print(result)
Executing this LLM workflow will cost approximately $0.05-$0.25 in OpenAI credits, depending on the number of attempts instructor needs makes to get the output in the desired format (which is set to 5).
:::

| ![](https://i.imgur.com/Etnjoyq.png) |
| ![](../../../static/img/parse_arxiv_papers/predict_trace.png) |
|---|
| Here's how you can explore the traces of the `ArxivModel` in the Weave UI |

Expand Down Expand Up @@ -296,6 +296,6 @@ await evaluation.evaluate(arxiv_parser_model)
Running the evaluation on 5 examples from evaluation dataset will cost approximately $0.25-$1.25 in OpenAI credits, depending on the number of attempts instructor needs makes to get the output in the desired format (which is set to 5) in evaluating each example.
:::

| ![](https://i.imgur.com/qFbt8T0.png) |
|---|
| ![](../../../static/img/parse_arxiv_papers/eval_trace.png) |
|-------------------------------------------------------------------------------|
| Here's how you can explore and compare the evaluations traces in the Weave UI |
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit dff078e

Please sign in to comment.