Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update CONTRIBUTING_DOCS.md #2190

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

Update CONTRIBUTING_DOCS.md #2190

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
7194a97
Update CONTRIBUTING_DOCS.md
NiWaRe Aug 21, 2024
f613d11
chore: add automatic cell removal, update contrib doc
scottire Aug 22, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 8 additions & 2 deletions docs/CONTRIBUTING_DOCS.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -116,9 +116,10 @@ To improve docs, basically follow FastAPI's instructions to create good Swagger

### Notebook Doc Gen

See `docs/scripts/generate_notebooks.py`, `./docs/notebooks`, and `./docs/reference/gen_notebooks`.
From within the `docs` directory, run `python ./scripts/generate_notebook.py`.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@scottire the file should be generate_notebooks.py (with an 's')

This script will load all notebooks in `./docs/notebooks`, transforming them into viable markdown docs in `./docs/reference/gen_notebooks` which can be referenced by docusaurus just like any other markdown file. To publish the page, add the notebook and the markdown file to a PR.

This script will load all notebooks in `./docs/notebooks`, transforming them into viable markdown docs in `./docs/reference/gen_notebooks` which can be referenced by docusaurus just like any other markdown file. If you need header metadata, you can add a markdown block at the top of your notebook with:
If you need header metadata, you can add a markdown block at the top of your notebook with:
```
<!-- docusaurus_head_meta::start
---
Expand All @@ -127,3 +128,8 @@ title: Head Metadata
docusaurus_head_meta::end -->
```

For more details, see `docs/scripts/generate_notebooks.py`, `./docs/notebooks`, and `./docs/reference/gen_notebooks`.

# Troubleshooting

- Make sure to update your version of pip (tested on 24.2) before installing the requirements.dev.txt
11 changes: 8 additions & 3 deletions docs/scripts/generate_notebooks.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
from nbconvert import MarkdownExporter

from nbconvert.preprocessors import ClearOutputPreprocessor
from nbconvert import NotebookExporter

def make_header(notebook_path):
github_uri = "wandb/weave/blob/master/docs"
Expand All @@ -22,8 +23,12 @@ def make_header(notebook_path):


def export_notebook(notebook_path, output_path):
exporter = MarkdownExporter()
output, resources = exporter.from_filename(notebook_path)

notebook_exporter = NotebookExporter()
notebook_exporter.preprocessors = [ClearOutputPreprocessor()]
notebook_content, _ = notebook_exporter.from_filename(notebook_path)
markdown_exporter = MarkdownExporter()
output, _ = markdown_exporter.from_notebook_node(notebook_content)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@scottire there seems to be some error in the notebook_content you're passing to markdown_exporter.from_notebook_node(notebook_content). This is the error I'm getting when I run it:

Traceback (most recent call last):
  File "/Users/niware_wb/Documents/wandb_internal/smle/demos/llms/weave/weave_source/docs/./scripts/generate_notebooks.py", line 67, in <module>
    main()
  File "/Users/niware_wb/Documents/wandb_internal/smle/demos/llms/weave/weave_source/docs/./scripts/generate_notebooks.py", line 60, in main
    export_all_notebooks_in_primary_dir()
  File "/Users/niware_wb/Documents/wandb_internal/smle/demos/llms/weave/weave_source/docs/./scripts/generate_notebooks.py", line 53, in export_all_notebooks_in_primary_dir
    export_notebook(
  File "/Users/niware_wb/Documents/wandb_internal/smle/demos/llms/weave/weave_source/docs/./scripts/generate_notebooks.py", line 31, in export_notebook
    output, _ = markdown_exporter.from_notebook_node(notebook_content)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/niware_wb/Documents/wandb_internal/smle/demos/llms/weave/weave_source/venv/lib/python3.11/site-packages/nbconvert/exporters/templateexporter.py", line 408, in from_notebook_node
    nb_copy, resources = super().from_notebook_node(nb, resources, **kw)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/niware_wb/Documents/wandb_internal/smle/demos/llms/weave/weave_source/venv/lib/python3.11/site-packages/nbconvert/exporters/exporter.py", line 150, in from_notebook_node
    if "language" in nb["metadata"]:
                     ~~^^^^^^^^^^^^
TypeError: string indices must be integers, not 'str'


extract_meta = ""
meta_mark_start = "<!-- docusaurus_head_meta::start\n"
Expand Down
Loading