Skip to content

Commit

Permalink
add colab badges
Browse files Browse the repository at this point in the history
  • Loading branch information
ash0ts committed Aug 12, 2024
1 parent 2197c36 commit b58812b
Show file tree
Hide file tree
Showing 2 changed files with 24 additions and 3 deletions.
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
# Summarization using Chain of Density

[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/wandb/weave/blob/anish/add-summarization-example/examples/cookbooks/summarization/chain_of_density.ipynb)

Summarizing complex technical documents while preserving crucial details is a challenging task. The Chain of Density (CoD) summarization technique offers a solution by iteratively refining summaries to be more concise and information-dense. This guide demonstrates how to implement CoD using Weave, a powerful framework for building, tracking, and evaluating LLM applications. By combining CoD's effectiveness with Weave's robust tooling, you'll learn to create a summarization pipeline that produces high-quality, entity-rich summaries of technical content while gaining insights into the summarization process.

![Final Evaluation](./media/eval_comparison.gif)
Expand Down
25 changes: 22 additions & 3 deletions examples/cookbooks/summarization/chain_of_density.ipynb
Original file line number Diff line number Diff line change
@@ -1,5 +1,23 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/wandb/weave/blob/anish/add-summarization-example/examples/cookbooks/summarization/chain_of_density.ipynb)\n",
"\n",
"<!--- @wandbcode{summarization-chain-of-density-cookbook} -->"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"!pip install -qU anthropic weave pydantic requests PyPDF2 set-env-colab-kaggle-dotenv"
]
},
{
"cell_type": "code",
"execution_count": null,
Expand All @@ -10,14 +28,15 @@
"import anthropic\n",
"import weave\n",
"from datetime import datetime, timezone\n",
"from dotenv import load_dotenv\n",
"from pydantic import BaseModel\n",
"import requests\n",
"import io\n",
"from PyPDF2 import PdfReader\n",
"from set_env import set_env\n",
"\n",
"set_env(\"WANDB_API_KEY\")\n",
"set_env(\"ANTHROPIC_API_KEY\")\n",
"\n",
"load_dotenv()\n",
"# Setup\n",
"weave.init(\"summarization-chain-of-density-cookbook\")\n",
"anthropic_client = anthropic.Anthropic(api_key=os.getenv(\"ANTHROPIC_API_KEY\"))"
]
Expand Down

0 comments on commit b58812b

Please sign in to comment.