From 6f1c8e38c6635aece57632b96bcad992b992514b Mon Sep 17 00:00:00 2001 From: Tom Close Date: Sat, 28 Dec 2024 23:30:23 +1100 Subject: [PATCH] restructuring new docs --- .../{howto => examples}/real-example.ipynb | 0 .../source/explanation/conditional-lazy.rst | 4 +- .../source/explanation/hashing-caching.rst | 4 +- .../source/howto/create-task-package.ipynb | 2 +- new-docs/source/index.rst | 42 +++++++++++++++--- .../source/tutorial/advanced-execution.ipynb | 44 +++++++++++++++++++ ...{execution.ipynb => getting-started.ipynb} | 2 +- new-docs/source/tutorial/python.ipynb | 2 +- new-docs/source/tutorial/shell.ipynb | 2 +- new-docs/source/tutorial/tst.py | 27 ------------ new-docs/source/tutorial/workflow.ipynb | 16 ++++--- 11 files changed, 100 insertions(+), 45 deletions(-) rename new-docs/source/{howto => examples}/real-example.ipynb (100%) create mode 100644 new-docs/source/tutorial/advanced-execution.ipynb rename new-docs/source/tutorial/{execution.ipynb => getting-started.ipynb} (98%) delete mode 100644 new-docs/source/tutorial/tst.py diff --git a/new-docs/source/howto/real-example.ipynb b/new-docs/source/examples/real-example.ipynb similarity index 100% rename from new-docs/source/howto/real-example.ipynb rename to new-docs/source/examples/real-example.ipynb diff --git a/new-docs/source/explanation/conditional-lazy.rst b/new-docs/source/explanation/conditional-lazy.rst index 3ed2c600e..0c30be1d1 100644 --- a/new-docs/source/explanation/conditional-lazy.rst +++ b/new-docs/source/explanation/conditional-lazy.rst @@ -1,4 +1,4 @@ -Conditional construction -======================== +Conditionals and lazy fields +============================ Work in progress... diff --git a/new-docs/source/explanation/hashing-caching.rst b/new-docs/source/explanation/hashing-caching.rst index ce59e448c..d03d4b042 100644 --- a/new-docs/source/explanation/hashing-caching.rst +++ b/new-docs/source/explanation/hashing-caching.rst @@ -1,4 +1,4 @@ -Caching -======= +Caching and hashing +=================== Work in progress.... diff --git a/new-docs/source/howto/create-task-package.ipynb b/new-docs/source/howto/create-task-package.ipynb index 36777b708..39aec8471 100644 --- a/new-docs/source/howto/create-task-package.ipynb +++ b/new-docs/source/howto/create-task-package.ipynb @@ -4,7 +4,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# Creating a task package" + "# Create a task package" ] }, { diff --git a/new-docs/source/index.rst b/new-docs/source/index.rst index bd19a7314..17a5b0e52 100644 --- a/new-docs/source/index.rst +++ b/new-docs/source/index.rst @@ -27,35 +27,67 @@ those commands installed on the execution machine, or use software containers (e.g., Docker or Singularity) to run them. +Tutorials +--------- + +* :ref:`Getting started` +* :ref:`Execution options` +* :ref:`Python-tasks` +* :ref:`Shell-tasks` +* :ref:`Workflows` + +Examples +-------- + +* :ref:`Real-world example` + +How-to Guides +------------- + +* :ref:`Create a task package` +* :ref:`Port interfaces from Nipype` + Indices and tables ------------------ * :ref:`genindex` * :ref:`modindex` +.. toctree:: + :maxdepth: 2 + :hidden: + + tutorial/getting-started + tutorial/advanced-execution .. toctree:: :maxdepth: 2 - :caption: Tutorials + :caption: Design :hidden: - tutorial/execution tutorial/python tutorial/shell tutorial/workflow + +.. toctree:: + :maxdepth: 2 + :caption: Examples + :hidden: + + examples/real-example + .. toctree:: :maxdepth: 2 - :caption: How-to Guides + :caption: How-to :hidden: - howto/real-example howto/create-task-package howto/port-from-nipype .. toctree:: :maxdepth: 2 - :caption: Explanation + :caption: In-depth :hidden: explanation/design-approach diff --git a/new-docs/source/tutorial/advanced-execution.ipynb b/new-docs/source/tutorial/advanced-execution.ipynb new file mode 100644 index 000000000..59bafe2fd --- /dev/null +++ b/new-docs/source/tutorial/advanced-execution.ipynb @@ -0,0 +1,44 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Execution options" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Execution plugins" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Cache locations" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Environments" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [] + } + ], + "metadata": { + "language_info": { + "name": "python" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} diff --git a/new-docs/source/tutorial/execution.ipynb b/new-docs/source/tutorial/getting-started.ipynb similarity index 98% rename from new-docs/source/tutorial/execution.ipynb rename to new-docs/source/tutorial/getting-started.ipynb index 7250f5ab3..c0f362c02 100644 --- a/new-docs/source/tutorial/execution.ipynb +++ b/new-docs/source/tutorial/getting-started.ipynb @@ -4,7 +4,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# Running tasks\n", + "# Getting started\n", "\n", "A *Task* is the basic runnable component in Pydra, and can execute either a Python function,\n", "shell command or workflows consisting of combinations of all three types." diff --git a/new-docs/source/tutorial/python.ipynb b/new-docs/source/tutorial/python.ipynb index 1ea45100a..d251a93da 100644 --- a/new-docs/source/tutorial/python.ipynb +++ b/new-docs/source/tutorial/python.ipynb @@ -4,7 +4,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# Python-task design" + "# Python-tasks" ] }, { diff --git a/new-docs/source/tutorial/shell.ipynb b/new-docs/source/tutorial/shell.ipynb index bf8c852c7..5416998b6 100644 --- a/new-docs/source/tutorial/shell.ipynb +++ b/new-docs/source/tutorial/shell.ipynb @@ -4,7 +4,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# Shell-task design" + "# Shell-tasks" ] }, { diff --git a/new-docs/source/tutorial/tst.py b/new-docs/source/tutorial/tst.py deleted file mode 100644 index 43a263f67..000000000 --- a/new-docs/source/tutorial/tst.py +++ /dev/null @@ -1,27 +0,0 @@ -from pydra.design import workflow, python - - -# Example python task specifications -@python.define -def Add(a, b): - return a + b - - -@python.define -def Mul(a, b): - return a * b - - -@python.define -def Sum(x: list[float]) -> float: - return sum(x) - - -@workflow.define -def MySplitWorkflow(a: list[int], b: list[float]) -> list[float]: - # Multiply over all combinations of the elements of a and b, then combine the results - # for each a element into a list over each b element - mul = workflow.add(Mul()).split(x=a, y=b).combine("x") - # Sume the multiplications across all all b elements for each a element - sum = workflow.add(Sum(x=mul.out)) - return sum.out diff --git a/new-docs/source/tutorial/workflow.ipynb b/new-docs/source/tutorial/workflow.ipynb index 37d116d63..e6c812e13 100644 --- a/new-docs/source/tutorial/workflow.ipynb +++ b/new-docs/source/tutorial/workflow.ipynb @@ -4,7 +4,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# Workflow design\n", + "# Workflows\n", "\n", "In Pydra, workflows are DAG of component tasks to be executed on specified inputs.\n", "Workflow specifications are dataclasses, which interchangeable with Python and shell tasks\n", @@ -208,7 +208,6 @@ "metadata": {}, "outputs": [], "source": [ - "\n", "from pydra.engine.specs import WorkflowSpec, WorkflowOutputs\n", "\n", "def a_converter(value):\n", @@ -376,14 +375,14 @@ "metadata": {}, "source": [ "For more detailed discussion of the construction of conditional workflows and \"lazy field\"\n", - "placeholders see [Conditional construction](../explanation/conditional-lazy.html)" + "placeholders see [Conditionals and lazy fields](../explanation/conditional-lazy.html)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "## Typing\n", + "## Type-checking between nodes\n", "\n", "Pydra utilizes Python type annotations to implement strong type-checking, which is performed\n", "when values or upstream outputs are assigned to task specification inputs.\n", @@ -430,7 +429,7 @@ " \"ffmpeg -i -i \"\n", " \"-filter_complex \"\n", " )(\n", - " in_video=input_video,\n", + " in_video=input_video, # This is OK because in_video is typed Any\n", " watermark=watermark, # Type is OK because generic.File is superclass of image.Png\n", " filter=\"overlay={}:{}\".format(*watermark_dims),\n", " ),\n", @@ -449,6 +448,13 @@ " return handbrake.output_video" ] }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "For more detailed discussion on Pydra's type-checking see [Type Checking](../explanation/typing.html)." + ] + }, { "cell_type": "markdown", "metadata": {},