diff --git a/content/building.md b/content/building.md index 5a4911c..ed1bd84 100644 --- a/content/building.md +++ b/content/building.md @@ -1,14 +1,24 @@ # Building the lesson -It is built the normal Sphinx way: +```{highlight} console +``` + +It is built the normal Sphinx ways. Using Sphinx directly, one would +run: + +``` +$ sphinx-build -M html content/ _build +``` + +If you have `make` installed, you can: ``` -make html -# or -make dirhtml -# full build -make clean html +$ make html +## or +$ make dirhtml +## full build +$ make clean html ``` -However, there are different ways to set up a Sphinx project. Our -sample project puts the results in `_build/`. +However, there are different ways to set up a Sphinx project. +CodeRefinery lessons puts the results in `_build/`. diff --git a/content/changelog.md b/content/changelog.md index 954fe96..5d667df 100644 --- a/content/changelog.md +++ b/content/changelog.md @@ -1,5 +1,8 @@ # Changelog +This isn't very up to date right now. There haven't been significant +backwards-incompatible changes, so this hasn't been kept in earnest. + ## 0.8.0 - First PyPI release diff --git a/content/cheatsheet.md b/content/cheatsheet.md index 90c5b84..f8d09ff 100644 --- a/content/cheatsheet.md +++ b/content/cheatsheet.md @@ -1,3 +1,6 @@ # Cheatsheet -This is the cheatsheet. +Nothing here yet. See the sample episodes: + +* [MyST markdown](https://github.com/coderefinery/sphinx-lesson/blob/master/content/sample-episode-myst.md?plain=1) +* [ReST](https://github.com/coderefinery/sphinx-lesson/blob/master/content/sample-episode-rst.rst?plain=1) diff --git a/content/conf.py b/content/conf.py index 9359ce0..0d50585 100644 --- a/content/conf.py +++ b/content/conf.py @@ -18,7 +18,7 @@ # -- Project information ----------------------------------------------------- project = 'Sphinx-lesson' -copyright = '2020, CodeRefinery' +copyright = '2020-2024, CodeRefinery' author = 'CodeRefinery' # roundabout way to get version. "import sphinx_lesson" would be easier, but @@ -47,6 +47,9 @@ 'sphinx_rtd_theme_ext_color_contrast', ] +# Sphinx-lesson config: +sphinx_lesson_transform_html_img = False + # Settings for myst_nb: # https://myst-nb.readthedocs.io/en/latest/computation/execute.html#notebook-execution-modes #nb_execution_mode = "off" diff --git a/content/contributing-to-a-lesson.md b/content/contributing-to-a-lesson.md index 354d341..81957ef 100644 --- a/content/contributing-to-a-lesson.md +++ b/content/contributing-to-a-lesson.md @@ -9,10 +9,11 @@ is fairly easy: - (optional) Set up the Python environment and preview - Send your contribution -In summary, each lesson is a Python project, with content in the -`content/` directory. It uses the Sphinx documentation system, -which is a popular, extendable tool. We have only minor extensions to -make it suitable to lessons. +In summary, each lesson is like a Python project, with the lesson +content as its documentation in the `content/` directory (and no +Python code). Everything is fairly standard: it uses the Sphinx +documentation system, which is a popular, extendable tool. We have +only minor extensions to make it suitable to lessons. Instead of going through this process, you can also open an issue instead with your proposed change, and let someone else add it. @@ -35,7 +36,7 @@ $ git clone git://github.com/ORGANIZATION/LESSON.git ## Edit the material The material is in the `content/` directory. Depending on the -lesson, in may be in ReStructured Text, MyST Markdown, or Jupyter +lesson, in may be in MyST Markdown, ReStructured Text, or Jupyter notebooks. ### ReStructured Text and MyST Markdown @@ -47,10 +48,10 @@ this is not unstructured Markdown, but there are particular text. (In fact, "markdown" is a broad concept and everyone uses some different extensions of it). +- {doc}`sphinx-lesson directives for markup ` - {doc}`md-and-rst` -- {ref}`ReStructured Text reference ` - [MyST reference](https://myst-parser.readthedocs.io/en/latest/using/syntax.html) -- {doc}`sphinx-lesson directives for markup ` +- {ref}`ReStructured Text reference ` *Do not worry about getting syntax right*. Send your improvement, and editing is easy and you will learn something. diff --git a/content/convert-old-lessons.md b/content/convert-old-lessons.md index 27748a8..578b2d7 100644 --- a/content/convert-old-lessons.md +++ b/content/convert-old-lessons.md @@ -1,5 +1,8 @@ # Converting an old lesson +**This was the old CodeRefinery procedure. It is now unmaintained +but might be useful for someone. Let us know if you need it revived.** + % highlight: console ## Convert a Jekyll lesson diff --git a/content/executing-code.md b/content/executing-code.md index 5ad655a..677799d 100644 --- a/content/executing-code.md +++ b/content/executing-code.md @@ -8,7 +8,7 @@ kernelspec: --- -# Try executing code cells in markdown +# Executing code cells in MyST markdown This is a `.md` file that has a special header and a `{code-cell}` directive, that lets it diff --git a/content/exercise-list.md b/content/exercise-list.md index 493cffc..e988b33 100644 --- a/content/exercise-list.md +++ b/content/exercise-list.md @@ -7,18 +7,18 @@ instructors and helpers, and more. ## Usage -ReST: - -``` -.. exerciselist:: -``` - MyST: ```` ```{exerciselist}``` ```` +ReST: + +``` +.. exerciselist:: +``` + One can give the optional directive arguments to specify lists of admonition classes to include (default: `exercise`, `solution`, `exerciselist-include`) or exclude (default: @@ -26,13 +26,33 @@ admonition classes to include (default: `exercise`, `solution`, ` which match any `include`, and do not match any `exclude` are included). Specify the options this way (ReST): +MyST: + +````md +```{exerciselist} +:include: exercise solution instructor-note +:exclude: exclude-this ``` + +:::{exercise} Exercise title +:class: exclude-this + +Exercise content +::: +```` + + +ReST: + +```rst .. exerciselist:: :include: exercise solution instructor-note :exclude: exclude-this -.. exercise:: +.. exercise:: Exercise title :class: exclude-this + + Exercise content ``` This feature is new as of early 2022, there may be possible problems @@ -74,8 +94,9 @@ source for hint on fixing). (advanced) Also create packaging using pyproject.toml and compare (20 min) ``` -- Consider giving your exercises permanent identifiers. They are not - auto-numbered yet for a reason (what happens when more exercises are +- Consider giving your exercises permanent identifiers. They are + intentionally not + auto-numbered yet (what happens when more exercises are added/removed?), but if you give them an ID, they will be findable even later. Suggestion is `Episodetopic-N`: @@ -103,10 +124,13 @@ source for hint on fixing). ## Example -This section contains the exercise list of sphinx-lesson. Note that -the directives occur many times in random contexts, so many of them -don't really make sense. Keep in mind how to ensure that your cases -are better. +**This section contains the exercise list of sphinx-lesson. Since +sphinx-lesson has many examples of exercises, the list below is +confusing and doesn't make a lot of sense. You can see a better +example at [git-intro's exercise +list](https://coderefinery.github.io/git-intro/exercises/).** + +**Example begins:** :::{exerciselist} ::: diff --git a/content/getting-started.md b/content/getting-started.md index cf7c96e..caf5d93 100644 --- a/content/getting-started.md +++ b/content/getting-started.md @@ -11,7 +11,8 @@ the lesson develops - you might want to check back for them, later. ## Convert an existing jekyll lesson -See {doc}`convert-old-lessons`. +See {doc}`convert-old-lessons`. This hasn't been used in years so may +be out of date. ## From scratch diff --git a/content/gh-action.md b/content/gh-action.md index 1f18536..d3a8af2 100644 --- a/content/gh-action.md +++ b/content/gh-action.md @@ -1,6 +1,6 @@ # Deployment with Github Actions -In this repository, there is a `.github/workflows/sphinx.yml` file +In sphinx-lesson-template (and this lesson...), there is a `.github/workflows/sphinx.yml` file that contains a Github Action that: - Installs dependencies @@ -9,7 +9,7 @@ that contains a Github Action that: - Deploys it - - If branch = `master` (to be renamed to `main`), deploy to github pages normally + - If branch = `main`, deploy to github pages normally - For other branches, deploy to github-pages but put the result in the `branch/{branch-name}` subdirectory. If the branch name has a `/` in it, replace it with `--`. @@ -17,3 +17,12 @@ that contains a Github Action that: for branches that no longer exist. This allows you to view builds from pull requests or other branches. + +## Usage + +It is recommended to copy from +[sphinx-lesson-template](https://github.com/coderefinery/sphinx-lesson-template/), +since that is the primary copy that is updated with all of the latest developments. + +Direct link: + diff --git a/content/index.md b/content/index.md index 8decb32..d7a2f1b 100644 --- a/content/index.md +++ b/content/index.md @@ -9,6 +9,10 @@ sphinx-lesson is a set of Sphinx extensions and themes for creating interactive, hands-on lessons. It was originally made to replace the CodeRefinery jekyll themes, but is designed to be used by others. +The broad idea is that making teaching materials should be quite +similar to good software documentation. Sphinx-lesson adds a model +and a few nice extras to bring them closer together. + As the name says, it is based on the [Sphinx documentation generator](https://www.sphinx-doc.org/). It is also inspired by and based on [jupyter-book](https://jupyterbook.org/), but both is jupyter-book and isn't. It *is* because jupyter-book is based on Sphinx and @@ -25,16 +29,18 @@ Features: - Based on jupyter-book, cross-compatible. - Built with Sphinx, providing a structured, controlled output. - Distributed as Python pip packages -- Markdown and ReStructured equally supported (yes, including all - directives), though ReStructured Text is still a bit nicer +- Markdown and ReStructured equally supported as input formats (yes, including all + directives). - Jupyter notebooks as an input format. Can execute code (in jupyter and other formats, too) - Transparent transformation of jekyll-style markdown styles into - CommonMark with directives -- Also renders with sphinx-book-theme (theme of jupyterbook) ([preview](https://coderefinery.github.io/sphinx-lesson/branch/sphinx-book-theme/)) + CommonMark with directives (mainly of use for old CodeRefinery lessons) +- Also renders with sphinx-book-theme (theme of jupyterbook) + ([preview](https://coderefinery.github.io/sphinx-lesson/branch/sphinx-book-theme/)) + and other Sphinx themes. -This is in an alpha state: we use it for lessons, but there is still -refinement work to go. +This is in heavy internal use, and about ready for other users who are +interested. :::{prereq} - If you know Sphinx, it helps some. If not, it's easy to copy diff --git a/content/indexing.md b/content/indexing.md index 7fc50d8..a427263 100644 --- a/content/indexing.md +++ b/content/indexing.md @@ -41,24 +41,11 @@ subentries. ```{index} pair: index; ReST ``` -Index a block with the directive, with ReST: - -``` -.. index:: commit; amend -``` - -Or ReST, multiple: - -``` -.. index:: - commit - commit; message - pair: commit; amend -``` +### Index a block with a directive MyST: -```` +````md ```{index} commit; amend ``` @@ -69,14 +56,24 @@ pair: commit; amend ``` ```` -Or index a single term with the role: +ReST: ``` -This sentence has an index entry for :index:`commit`. If you want the -indexed term to be different, standard syntax applies such as -:index:`loop variables `. +.. index:: commit; amend ``` +Or ReST, multiple: + +``` +.. index:: + commit + commit; message + pair: commit; amend +``` + + +### Index a single word with the role + ```{index} pair: index; MyST ``` @@ -90,7 +87,16 @@ Pair entry: {index}`loop variables ` ```{index} index; single index; pair index; see index; seealso ``` -There are different styles: +ReST: + +``` +This sentence has an index entry for :index:`commit`. If you want the +indexed term to be different, standard syntax applies such as +:index:`loop variables `. +``` + + +### Styles of indexes - `TERM`, same as below - `single: TERM` (the default): create just a single entry diff --git a/content/installation.md b/content/installation.md index 2672108..0df03c7 100644 --- a/content/installation.md +++ b/content/installation.md @@ -32,19 +32,18 @@ preview of it [as a branch on github-pages](https://coderefinery.github.io/sphin ## Under the hood -Adding `sphinx_lesson` as an extension adds these sub-extensions: - -> - `sphinx_lesson.directives` - see {doc}`directives`. -> - `sphinx_lesson.md_transforms` - see {doc}`md-transforms`. -> - `sphinx_lesson.exerciselist` - see {doc}`exercise-list`. -> - `sphinx_lesson.term_role_formatting` - makes glossary term -> references bold -> - Enables the [myst_notebook extension](https://myst-nb.readthedocs.io/en/latest/), which also enables -> [myst_parser](https://myst-parser.readthedocs.io/en/latest/index.html) -> (included as a dependencies) -> - Enables the [sphinx-copybutton extension](https://github.com/executablebooks/sphinx-copybutton) -> (included as a dependency) -> - Same for [sphinx-tabs](https://sphinx-tabs.readthedocs.io/) -> - Same for [sphinx-togglebutton](https://pypi.org/project/sphinx-togglebutton/) - -Any of these can be used independently to get the same effect. +Adding `sphinx_lesson` as an extension adds these sub-extensions, and +you could selectively enable only the parts you want: + +- `sphinx_lesson.directives` - see {doc}`directives`. +- `sphinx_lesson.md_transforms` - see {doc}`md-transforms`. +- `sphinx_lesson.exerciselist` - see {doc}`exercise-list`. +- `sphinx_lesson.term_role_formatting` - makes glossary term + references bold +- Enables the [myst_notebook extension](https://myst-nb.readthedocs.io/en/latest/), which also enables + [myst_parser](https://myst-parser.readthedocs.io/en/latest/index.html) + (included as a dependencies) +- Enables the [sphinx-copybutton extension](https://github.com/executablebooks/sphinx-copybutton) + (included as a dependency) +- Same for [sphinx-tabs](https://sphinx-tabs.readthedocs.io/) +- Same for [sphinx-togglebutton](https://pypi.org/project/sphinx-togglebutton/) diff --git a/content/intersphinx.md b/content/intersphinx.md index b876e09..af1a69d 100644 --- a/content/intersphinx.md +++ b/content/intersphinx.md @@ -47,7 +47,7 @@ The {py:class}`list` class {py:meth}`sort ` method. Note that this is structured information, and thus has no concept in Markdown, only MyST "markdown". This is, in fact, a major reason why -plain markdown is hardly suitable for serious documentation. +plain markdown is not that great for structured documentation. ## Available linking domains and roles diff --git a/content/jupyter.ipynb b/content/jupyter.ipynb index af41e66..192925b 100644 --- a/content/jupyter.ipynb +++ b/content/jupyter.ipynb @@ -62,7 +62,7 @@ "source": [ "## Configuration\n", "\n", - "The `jupyter_execute_notebooks` variable in conf.py controls execution. Currently, we don't set a default for this, which makes it `auto`. You can read more about this [upstream](https://myst-nb.readthedocs.io/en/latest/use/execute.html). Possible values include:\n", + "The `nb_execution_mode` variable in conf.py controls execution. Currently, we don't set a default for this, which makes it `auto`. You can read more about this [upstream](https://myst-nb.readthedocs.io/en/latest/computation/execute.html). Possible values include:\n", "\n", "* `off`: don't execute\n", "* `auto`: exceute *only if there are missing outputs\n", diff --git a/content/md-and-rst.md b/content/md-and-rst.md index 44a3331..6af5830 100644 --- a/content/md-and-rst.md +++ b/content/md-and-rst.md @@ -4,32 +4,43 @@ Sites can be written in Markdown on ReStructured Text. Actually, in theory any format that has a Sphinx parser could be used, however you will be slightly limited without directive support. +The most important thing to note is that: To make a structured lesson, +one needs to write in something a bit more structured than HTML. + ## Markdown -Let's start off with a basic fact: Markdown isn't a language. It's -some html-like markup that is not structured enough for the purposes -of structured sites such as sphinx-lesson. There are many different +Markdown is the most common syntax for CodeRefinery lessons. It's not +raw markdown but the MyST flavor, which has *much* more structured +directives than plain markdown. These come straight from Sphinx and +what we use to + +[MyST syntax reference](https://myst-parser.readthedocs.io/en/latest/using/syntax.html) + + +:::{note} +What is Markdown? Markdown isn't a single language. Its native form +is a simple syntax for HTML, and isn't very structured. There are many different flavors, some of which add extra syntax which gets it closer to enough, but for our purposes these are different enough that they should count as different languages (as similar as "markdown" and ReST). Since the Markdown creator says that [Markdown shouldn't evolve or be strictly defined](https://en.wikipedia.org/wiki/Markdown#CommonMark), Markdown is -essentially a dead syntax: you should always specific which living +essentially a dead syntax: we should always specific which living sub-syntax you are referring to. -sphinx-lesson uses the [MyST-parser] (markedly structured text), +sphinx-lesson uses the [MyST-parser] (MarkedlY Structured Text), which is both completely compatible with CommonMark, and also supports *all ReStructured Text directives*, unlike most other non-ReST Sphinx parsers. Thus, we finally have a way to write equivalent ReST and Markdown without any compromises (though other CommonMark parsers aren't expected to know Sphinx directives). - -[MyST syntax reference](https://myst-parser.readthedocs.io/en/latest/using/syntax.html) +::: ## ReStructured Text ReStructured Text has native support for roles and directives, which makes it a more structured language such as LaTeX, rather than HTML. +It came before Sphinx, but Sphinx made it even more popular. [ReST reference (from Sphinx)](https://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html) @@ -39,6 +50,28 @@ This is a brief comparison of basic syntax: ReST syntax (Sphinx has a good [restructured text primer](https://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html): +MyST markdown syntax: + +````md +*italic* +**bold** +`literal` +# Heading + +[inline link](https://example.com) +[link to page](relative-page-path) + +structured links: +{doc}`link to page ` +{ref}`page anchor link ` +{py:mod}`intersphinx link ` + + +``` +code block +``` +```` + ```rst *italic* **bold** @@ -72,27 +105,11 @@ Code block after paragraph:: a single colon. ``` -MyST markdown syntax: - -````md -*italic* -**bold** -`literal` -# Heading - -[inline link](https://example.com) - -[link to page](relative-page-path) - -``` -code block -``` -```` - The most interesting difference is the use of single backquote for literals in Markdown, and double in ReST. This is because ReST uses single quotes for *roles* - notice how there is a dedicated syntax for -inter-page links, references, and so on. This is very important for +inter-page links, references, and so on (it can be configured to +"figure it out" if you want). This is very important for things like verifying referential integrity of all of our pages. But this is configurable with [default_role](https://www.sphinx-doc.org/en/master/usage/configuration.html#confval-default_role): set to `any` to automatically detect documents/references/anthing, @@ -106,6 +123,19 @@ structured parsing for blocks of text. For example, we have an callout. This is not just a CSS class, it can do anything during the build phase (but in practice we don't do such complex things). +### MyST directives + +MyST-parser directives are done like this: + +````md +:::{exercise} +:option: value + +content +::: +```` + + ### ReST directives ReST directives are done like this: @@ -119,17 +149,6 @@ ReST directives are done like this: You can put *arbitrary syntax* here. ``` -### MyST directives - -MyST-parser directives are done like this: - -````md -```{exercise} -:option: value - -content -``` -```` ## Roles diff --git a/content/md-transforms.rst b/content/md-transforms.rst index bdf7ec0..0a73ed8 100644 --- a/content/md-transforms.rst +++ b/content/md-transforms.rst @@ -1,10 +1,15 @@ Markdown transforms =================== -To ease the transition from other Markdown dialects(like the one used -in software-carpentry), we implement some transformations in sphinx. -These are implemented in the ``sphinx_lessons.md_transforms`` Python -package and are implemented using regular expressions, so they are a +**This is mostly obsolete and unmaintained now that almost everything +is transitioned to MyST-markdown. If this is important for you, get +in touch and we can revive it.** + +To ease the transition from other Markdown dialects (like the one used +in software-carpentry), we implement some transformations in Sphinx +which happen as source preprocessing. +These are implemented in the ``sphinx_lessons.md_transforms`` module +and are implemented using regular expressions, so they are a bit fragile. Code fences @@ -49,14 +54,14 @@ Raw HTML images Raw HTML isn't a good idea in portable formats. Plus, in the old jekyll formats, bad relative path handling caused absolute paths to be -embedded a lot.Transform this:: +embedded a lot. Transform this:: - + into this:: - ```{figure} /path/to/img.png - ``` + ```{figure} /path/to/img.png + ``` Exclude any possible ``{{ ... }}`` template variables used to semi-hard code absolute paths. diff --git a/content/substitutions-replacements.md b/content/substitutions-replacements.md index ca9b917..eb2ba63 100644 --- a/content/substitutions-replacements.md +++ b/content/substitutions-replacements.md @@ -2,7 +2,7 @@ Like most languages, ReST and MyST both have direct ways to substitute variables to locally customize lessons. While this works for simple -cases, this can quickly become difficult to manage with the "master +cases, this can quickly become difficult to manage with the "main copy" of possibly important content being separated from the main body, and keeping the substitutions up to date. diff --git a/content/toctree.md b/content/toctree.md index 9b4096f..643a3cb 100644 --- a/content/toctree.md +++ b/content/toctree.md @@ -5,12 +5,14 @@ table of contents list. There must be one master toctree in the index document, but then they can be nested down. For the purposes of sphinx-lesson, we probably don't need such features -ReST: +MyST: -``` -.. toctree:: +```` +```{toctree} +--- caption: Episodes maxdepth: 1 +--- basics creating-using-web @@ -19,15 +21,15 @@ contributing doi websites ``` +```` -MyST: -```` -```{toctree} ---- +ReST: + +``` +.. toctree:: caption: Episodes maxdepth: 1 ---- basics creating-using-web @@ -36,7 +38,6 @@ contributing doi websites ``` -```` The pages are added by filename (no extension needed). The name by default comes from the document title, but you can override it if you