diff --git a/README.md b/README.md
index e31dd8a..7cba439 100644
--- a/README.md
+++ b/README.md
@@ -40,8 +40,8 @@ In your markdown files you can now use:
Where the path is relative to the location of your project's `mkdocs.yml` file, _or_ your project's `docs/` directory, _or_ the location of your markdown source file (all 3 possible locations will be searched, in that order).
-- There are [readers](https://timvink.github.io/mkdocs-table-reader-plugin/readers/) for `.csv`, `.fwf`, `.json`, `.xls`, `.xlsx`, `.yaml`, `.feather` and `.tsv` files. There is also the `read_raw()` reader that will allow you to insert tables (or other content) already in markdown format.
-- `table-reader` is compatible with [`mkdocs-macros-plugin`](https://mkdocs-macros-plugin.readthedocs.io/en/latest/), which means you can [dynamically insert tables using jinja2 syntax](https://timvink.github.io/mkdocs-table-reader-plugin/howto/use_jinja2/).
+- There are [readers](https://timvink.github.io/mkdocs-table-reader-plugin/readers/) available for many common table formats, like `.csv`, `.fwf`, `.json`, `.xls`, `.xlsx`, `.yaml`, `.feather` and `.tsv`. There is also the `read_raw()` reader that will allow you to insert tables (or other content) already in markdown format.
+- `table-reader` is compatible with [`mkdocs-macros-plugin`](https://mkdocs-macros-plugin.readthedocs.io/en/latest/). This enables further automation like filtering tables or inserting directories of tables. See the documentation on [compatibility with macros plugin](howto/use_jinja2.md) for more examples.
## Documentation and how-to guides
diff --git a/docs/howto/alternatives.md b/docs/howto/alternatives.md
index 4f7ad33..cec6bbf 100644
--- a/docs/howto/alternatives.md
+++ b/docs/howto/alternatives.md
@@ -39,7 +39,7 @@ Downsides:
## Execute python during build
-You could also choose to insert the markdown for tables dynamically, using packages like [markdown-exec]() or [mkdocs-macros-plugin](https://mkdocs-macros-plugin.readthedocs.io/).
+You could also choose to insert the markdown for tables dynamically, using packages like [markdown-exec](https://pypi.org/project/markdown-exec/).
For example:
diff --git a/docs/howto/customize_tables.md b/docs/howto/customize_tables.md
index 02dac1c..66c1312 100644
--- a/docs/howto/customize_tables.md
+++ b/docs/howto/customize_tables.md
@@ -12,8 +12,10 @@ df = pd.read_csv('path_to_table.csv')
df.to_markdown(index=False, tablefmt='pipe')
```
-Any keyword arguments you give to \{\{ read_csv('path_to_your_table.csv') \}\}
will be matched and passed the corresponding [pandas.read_csv()](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.read_csv.html) and/or
+{% raw %}
+Any keyword arguments you give to `{{ read_csv('path_to_your_table.csv') }}` will be matched and passed the corresponding [pandas.read_csv()](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.read_csv.html) and/or
[.to_markdown()](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.to_markdown.html) functions.
+{% endraw %}
Pandas's `.to_markdown()` uses the [tabulate](https://pypi.org/project/tabulate/) package and any keyword arguments that are passed to it. Tabulate in turn offers many customization options, see [library usage](https://github.com/astanin/python-tabulate#library-usage).
@@ -23,21 +25,33 @@ Text columns will be aligned to the left [by default](https://github.com/astanin
=== ":arrow_left: left"
- \{\{ read_csv('tables/basic_table.csv', colalign=("left",)) \}\}
-
+ {% raw %}
+ ```
{{ read_csv('tables/basic_table.csv', colalign=("left",)) }}
+ ```
+ {% endraw %}
-=== ":left_right_arrow: center"
+ {{ read_csv('tables/basic_table.csv', colalign=("left",)) | add_indentation(spaces=4) }}
- \{\{ read_csv('tables/basic_table.csv', colalign=("center",)) \}\}
+=== ":left_right_arrow: center"
+ {% raw %}
+ ```
{{ read_csv('tables/basic_table.csv', colalign=("center",)) }}
+ ```
+ {% endraw %}
-=== ":arrow_right: right"
+ {{ read_csv('tables/basic_table.csv', colalign=("center",)) | add_indentation(spaces=4) }}
- \{\{ read_csv('tables/basic_table.csv', colalign=("right",)) \}\}
+=== ":arrow_right: right"
+ {% raw %}
+ ```
{{ read_csv('tables/basic_table.csv', colalign=("right",)) }}
+ ```
+ {% endraw %}
+
+ {{ read_csv('tables/basic_table.csv', colalign=("right",)) | add_indentation(spaces=4) }}
## Sortable tables
@@ -47,21 +61,33 @@ If you use [mkdocs-material](https://squidfunk.github.io/mkdocs-material), you c
You can use [tabulate](https://pypi.org/project/tabulate/)'s [number formatting](https://github.com/astanin/python-tabulate#number-formatting). Example:
-=== ":zero:"
-
- \{\{ read_fwf('tables/fixedwidth_table.txt', floatfmt=".0f") \}\}
+=== "zero points"
+ {% raw %}
+ ```
{{ read_fwf('tables/fixedwidth_table.txt', floatfmt=".0f") }}
+ ```
+ {% endraw %}
-=== ":one:"
+ {{ read_fwf('tables/fixedwidth_table.txt', floatfmt=".0f") | add_indentation(spaces=4) }}
- \{\{ read_fwf('tables/fixedwidth_table.txt', floatfmt=".1f") \}\}
+=== "one points"
+ {% raw %}
+ ```
{{ read_fwf('tables/fixedwidth_table.txt', floatfmt=".1f") }}
+ ```
+ {% endraw %}
-=== ":two:"
+ {{ read_fwf('tables/fixedwidth_table.txt', floatfmt=".1f") | add_indentation(spaces=4) }}
- \{\{ read_fwf('tables/fixedwidth_table.txt', floatfmt=".2f") \}\}
+=== "two points"
+ {% raw %}
+ ```
{{ read_fwf('tables/fixedwidth_table.txt', floatfmt=".2f") }}
+ ```
+ {% endraw %}
+
+ {{ read_fwf('tables/fixedwidth_table.txt', floatfmt=".2f") | add_indentation(spaces=4) }}
diff --git a/docs/howto/preprocess_tables.md b/docs/howto/preprocess_tables.md
index e1c2558..a150e35 100644
--- a/docs/howto/preprocess_tables.md
+++ b/docs/howto/preprocess_tables.md
@@ -1,5 +1,7 @@
# Preprocess input tables
+{% raw %}
+
`mkdocs>=1.4` supports [hooks](https://www.mkdocs.org/user-guide/configuration/#hooks), which enable you to run python scripts on `mkdocs serve` or `mkdocs build`.
Here are some example of workflows that use hooks and the `table-reader` plugin:
@@ -26,7 +28,7 @@ Here are some example of workflows that use hooks and the `table-reader` plugin:
My table:
- \{\{ read_csv("docs/assets/output_table.csv") \}\}
+ {{ read_csv("docs/assets/output_table.csv") }}
=== "mkdocs.yml"
@@ -74,7 +76,7 @@ Here are some example of workflows that use hooks and the `table-reader` plugin:
My table:
- \{\{ read_csv("docs/assets/nyc_data.csv") \}\}
+ {{ read_csv("docs/assets/nyc_data.csv") }}
=== "mkdocs.yml"
@@ -101,3 +103,5 @@ Here are some example of workflows that use hooks and the `table-reader` plugin:
```
Note that during development when you use `mkdocs serve` and autoreload, you might not want to run this hook every time you make a change. You could use an environment variable inside your hook, for example something like `if os.environ['disable_hook'] == 1: return None`.
+
+{% endraw %}
\ No newline at end of file
diff --git a/docs/howto/project_structure.md b/docs/howto/project_structure.md
index 7ba2966..9243a18 100644
--- a/docs/howto/project_structure.md
+++ b/docs/howto/project_structure.md
@@ -1,4 +1,5 @@
# Choose a project structure
+{% raw %}
You have different possible strategies to store and load your tables. This guide gives some examples.
@@ -23,14 +24,14 @@ If you only want to include an occasional table in a specific markdown file, jus
```md
Here is the table:
- \{\{ read_csv("another_table.csv") \}\}
+ {{ read_csv("another_table.csv") }}
```
In `page.md`, to read `another_table.csv`, you can choose to use:
-- \{\{ read_csv("docs/folder/another_table.csv") \}\}
(Path relative to mkdocs.yml)
-- \{\{ read_csv("folder/another_table.csv") \}\}
(Path relative to docs/ directory)
-- \{\{ read_csv("another_table.csv") \}\}
(Path relative to page source file)
+- `{{ read_csv("docs/folder/another_table.csv") }}` (Path relative to mkdocs.yml)
+- `{{ read_csv("folder/another_table.csv") }}` (Path relative to docs/ directory)
+- `{{ read_csv("another_table.csv") }}` (Path relative to page source file)
## Re-using tables across markdown files
@@ -54,7 +55,8 @@ Given the following project structure:
In `page.md`, to read `another_table.csv`, you can choose to use:
-- \{\{ read_csv("docs/assets/tables/another_table.csv") \}\}
(Path relative to mkdocs.yml)
-- \{\{ read_csv("assets/tables/another_table.csv") \}\}
(Path relative to docs/ directory)
-- \{\{ read_csv("../assets/tables/another_table.csv") \}\}
(Path relative to page source file _(note that `..` stands for "one directory up")_)
+- `{{ read_csv("docs/assets/tables/another_table.csv") }}` (Path relative to mkdocs.yml)
+- `{{ read_csv("assets/tables/another_table.csv") }}` (Path relative to docs/ directory)
+- `{{ read_csv("../assets/tables/another_table.csv") }}` (Path relative to page source file _(note that `..` stands for "one directory up")_)
+{% endraw %}
\ No newline at end of file
diff --git a/docs/howto/use_jinja2.md b/docs/howto/use_jinja2.md
index 8ab66f9..f1adc10 100644
--- a/docs/howto/use_jinja2.md
+++ b/docs/howto/use_jinja2.md
@@ -1,4 +1,6 @@
-# Use jinja2 for automation
+# Compatibility with mkdocs-macros-plugin to enable further automation
+
+{% raw %}
`table-reader` supports [`mkdocs-macros-plugin`](https://mkdocs-macros-plugin.readthedocs.io/en/latest/), which enables you to use jinja2 syntax inside markdown files (among other things).
@@ -10,26 +12,19 @@ plugins:
- table-reader
```
-Now you can do cool things like:
-
-## Dynamically load a list of tables
+Everything will work as before, _except_ indentation will not be retrained. This means components that rely on indentation (like [Admonitions](https://squidfunk.github.io/mkdocs-material/reference/admonitions/) and [Content tabs](https://squidfunk.github.io/mkdocs-material/reference/content-tabs/#usage)) will break.
-```markdown
-# index.md
+The solution is to use the custom _filter_ `add_indendation` (a filter added to `macros` by `table-reader` plugin, see the [readers](../readers.md)). For example:
-{% set table_names = ["basic_table.csv","basic_table2.csv"] %}
-{% for table_name in table_names %}
+```jinja
+!!! note "This is a note"
-{ { read_csv(table_name) }}
-
-{% endfor %}
+ {{ read_csv("basic_table.csv") | add_indentation(spaces=4) }}
```
-## Insert tables into content tabs
-
-If you inserted content has multiple lines, then indentation will be not be retained beyond the first line. This means things like [content tabs](https://squidfunk.github.io/mkdocs-material/reference/content-tabs/#usage) will not work as expected.
+The upside is you now have much more control. A couple of example use cases:
-To fix that, you can use the custom _filter_ `add_indendation` (a filter add to `macros` by `table-reader` plugin). For example:
+## Dynamically load a specified list of tables into tabs
=== "index.md"
@@ -39,7 +34,7 @@ To fix that, you can use the custom _filter_ `add_indendation` (a filter add to
=== "{{ table_name }}"
- { { read_csv(table_name) | add_indentation(spaces=4) }}
+ {{ read_csv(table_name) | add_indentation(spaces=4) }}
{% endfor %}
```
@@ -64,11 +59,6 @@ To fix that, you can use the custom _filter_ `add_indendation` (a filter add to
alternate_style: true
```
-!!! note "Note the space in { {"
-
- To avoid the tables being inserted into the code example, we replaced `{{` with `{ {`.
- If you copy this example, make sure to fix.
-
## Recursively insert an entire directory of tables
@@ -100,12 +90,19 @@ Now you could do something like:
{% for table_name in listdir('docs/assets/my_tables") %}
-{ { read_csv(table_name) }}
+{{ read_csv(table_name) }}
{% endfor %}
```
-!!! note "Note the space in { {"
+## Filter a table before inserting it
+
+When you enable the `macros` plugin in your `mkdocs.yml`, `table-reader` will add additional _macros_ and _filters_ (see the [readers overview](../readers.md)).
+
+You can use the `pd_` variants to read a file to a pandas dataframe. Then you can use the pandas methods like [`.query()`](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.query.html). For example:
+
+```
+{{ pd_read_csv("numeric_table.csv").query("column_name >= 3") | convert_to_md_table }}
+```
- To avoid the tables being inserted into the code example, we replaced `{{` with `{ {`.
- If you copy this example, make sure to fix.
\ No newline at end of file
+{% endraw %}
diff --git a/docs/options.md b/docs/options.md
index 0853e17..52e19a5 100644
--- a/docs/options.md
+++ b/docs/options.md
@@ -4,6 +4,7 @@ hide:
---
# Options
+{% raw %}
You can customize the plugin by setting options in `mkdocs.yml`. For example:
@@ -24,7 +25,7 @@ Default is `.`. Set a default path to the searched directories in order to short
Given a file path, `table-reader` will search for that file relative to your your project's `mkdocs.yml` and relative to your `docs/` folder. If you use a folder for all your table files you can shorten the path specification by setting the `data_path`.
-For example, if your table is located in `docs/assets/tables/basic_table.csv`, you can set `data_path` to `docs/assets/tables/`. Then you will be able to use \{\{ read_csv("basic_table.csv") \}\}
instead of \{\{ read_csv("docs/assets/tables/basic_table.csv") \}\}
inside any markdown page.
+For example, if your table is located in `docs/assets/tables/basic_table.csv`, you can set `data_path` to `docs/assets/tables/`. Then you will be able to use `{{ read_csv("basic_table.csv") }}` instead of `{{ read_csv("docs/assets/tables/basic_table.csv") }}` inside any markdown page.
!!! info
@@ -38,7 +39,7 @@ Default: `False`. When enabled, if a filepath is not found, the plugin will rais
## `select_readers`
-Default: Selects all available readers. Specify a list of readers to improve documentation build times for very large sites.
+Default: Selects all available readers. Specify a list of readers to improve documentation build times for very large sites. This option is ignored when you use this plugin with `mkdocs-macros-plugin` ([read more](howto/use_jinja2.md))
## `enabled`
@@ -57,4 +58,6 @@ Which enables you to disable the plugin locally using:
```bash
export ENABLED_TABLE_READER=false
mkdocs serve
-```
\ No newline at end of file
+```
+
+{% endraw %}
\ No newline at end of file
diff --git a/docs/readers.md b/docs/readers.md
index 5298fd6..50ed597 100644
--- a/docs/readers.md
+++ b/docs/readers.md
@@ -5,116 +5,394 @@ hide:
# Readers
+## Basic readers
+
The following table reader functions are available:
-## read_csv
+### `read_csv`
+
+Use {% raw %}`{{ read_csv() }}`{% endraw %} to read a comma-separated values (csv) and output as a markdown table.
-`{{ read_csv() }}` passed to [pandas.read_csv()](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.read_csv.html). Example:
+1. Arguments are parsed safely and then passed to corresponding functions below
+2. File is read using [pandas.read_csv()](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.read_csv.html)
+3. The `pd.DataFrame` is then converted to a markdown table using [`.to_markdown()`](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.to_markdown.html)
+4. The markdown table is fixed to match the indentation used by the tag in the markdown document (only when _not_ used with `mkdocs-macros-plugin`. See [compatibility with macros plugin](howto/use_jinja2.md))
+
+Example:
=== "Input"
- \{\{ read_csv('tables/basic_table.csv') \}\}
+ {% raw %}
+ ```markdown
+ {{ read_csv('assets/tables/basic_table.csv') }}
+ ```
+ {% endraw %}
=== "Output"
- {{ read_csv('tables/basic_table.csv') }}
+ {{ read_csv('assets/tables/basic_table.csv') | add_indentation(spaces=4) }}
-## read_fwf
+### `read_fwf`
-`{{ read_fwf() }}` passed to [pandas.read_fwf()](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.read_fwf.html). Example:
+Use {% raw %}`{{ read_fwf() }}`{% endraw %} to read a table of fixed-width formatted lines and output as a markdown table.
+
+1. Arguments are parsed safely and then passed to corresponding functions below
+2. File is read using [pandas.read_fwf()](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.read_fwf.html)
+3. The `pd.DataFrame` is then converted to a markdown table using [`.to_markdown()`](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.to_markdown.html)
+4. The markdown table is fixed to match the indentation used by the tag in the markdown document (only when _not_ used with `mkdocs-macros-plugin`. See [compatibility with macros plugin](howto/use_jinja2.md))
+
+Example:
=== "Input"
- \{\{ read_fwf('tables/fixedwidth_table.txt') \}\}
+ {% raw %}
+ ```markdown
+ {{ read_fwf('assets/tables/fixedwidth_table.txt') }}
+ ```
+ {% endraw %}
=== "Output"
- {{ read_fwf('tables/fixedwidth_table.txt') }}
+ {{ read_fwf('assets/tables/fixedwidth_table.txt') | add_indentation(spaces=4) }}
+
+### `read_yaml`
-## read_yaml
+Use {% raw %}`{{ read_yaml() }}`{% endraw %} to read a YAML file and output as a markdown table.
-`{{ read_yaml() }}` is parsed with [yaml.safe_load()](https://pyyaml.org/wiki/PyYAMLDocumentation#loading-yaml) and passed to [pandas.json_normalize()](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.json_normalize.html). Example:
+1. Arguments are parsed safely and then passed to corresponding functions below
+2. File is read using [yaml.safe_load()](https://pyyaml.org/wiki/PyYAMLDocumentation#loading-yaml) and then passed to [pandas.json_normalize()](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.json_normalize.html)
+3. The `pd.DataFrame` is then converted to a markdown table using [`.to_markdown()`](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.to_markdown.html)
+4. The markdown table is fixed to match the indentation used by the tag in the markdown document (only when _not_ used with `mkdocs-macros-plugin`. See [compatibility with macros plugin](howto/use_jinja2.md))
+
+Example:
=== "Input"
- \{\{ read_yaml('tables/yaml_table.yml') \}\}
+ {% raw %}
+ ```markdown
+ {{ read_yaml('assets/tables/yaml_table.yml') }}
+ ```
+ {% endraw %}
=== "Output"
- {{ read_yaml('tables/yaml_table.yml') }}
+ {{ read_yaml('assets/tables/yaml_table.yml') | add_indentation(spaces=4) }}
+
+
+### `read_table`
+Use {% raw %}`{{ read_table() }}`{% endraw %} to read a general delimited file and output as a markdown table.
-## read_table
+1. Arguments are parsed safely and then passed to corresponding functions below
+2. File is read using [pandas.read_table()](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.read_table.html)
+3. The `pd.DataFrame` is then converted to a markdown table using [`.to_markdown()`](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.to_markdown.html)
+4. The markdown table is fixed to match the indentation used by the tag in the markdown document (only when _not_ used with `mkdocs-macros-plugin`. See [compatibility with macros plugin](howto/use_jinja2.md))
-`{{ read_table() }}` passed to [pandas.read_table()](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.read_table.html). Example:
+Example:
=== "Input"
- \{\{ read_table('tables/basic_table.csv', sep = ',') \}\}
+ {% raw %}
+ ```markdown
+ {{ read_table('assets/tables/basic_table.csv', sep = ',') }}
+ ```
+ {% endraw %}
=== "Output"
- {{ read_table('tables/basic_table.csv', sep = ',') }}
+ {{ read_table('assets/tables/basic_table.csv', sep = ',') | add_indentation(spaces=4) }}
+
+### `read_json`
-## read_json
+Use {% raw %}`{{ read_json() }}`{% endraw %} to read a JSON string path and output as a markdown table.
-`{{ read_json() }}` passed to [pandas.read_json()](https://pandas.pydata.org/docs/reference/api/pandas.read_json.html). Example:
+1. Arguments are parsed safely and then passed to corresponding functions below
+2. File is read using [pandas.read_json()](https://pandas.pydata.org/docs/reference/api/pandas.read_json.html)
+3. The `pd.DataFrame` is then converted to a markdown table using [`.to_markdown()`](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.to_markdown.html)
+4. The markdown table is fixed to match the indentation used by the tag in the markdown document (only when _not_ used with `mkdocs-macros-plugin`. See [compatibility with macros plugin](howto/use_jinja2.md))
+
+Example:
=== "Input"
- \{\{ read_json('tables/data.json', orient='split') \}\}
+ {% raw %}
+ ```markdown
+ {{ read_json('assets/tables/data.json', orient='split') }}
+ ```
+ {% endraw %}
=== "Output"
- {{ read_json('tables/data.json', orient='split') }}
+ {{ read_json('assets/tables/data.json', orient='split') | add_indentation(spaces=4) }}
-## read_feather
+### `read_feather`
-`{{ read_feather() }}` passed to [pandas.read_feather()](https://pandas.pydata.org/docs/reference/api/pandas.read_feather.html). Example:
+Use {% raw %}`{{ read_feather() }}`{% endraw %} to read a feather-format object and output as a markdown table.
+
+1. Arguments are parsed safely and then passed to corresponding functions below
+2. File is read using [pandas.read_feather()](https://pandas.pydata.org/docs/reference/api/pandas.read_feather.html)
+3. The `pd.DataFrame` is then converted to a markdown table using [`.to_markdown()`](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.to_markdown.html)
+4. The markdown table is fixed to match the indentation used by the tag in the markdown document (only when _not_ used with `mkdocs-macros-plugin`. See [compatibility with macros plugin](howto/use_jinja2.md))
+
+Example:
=== "Input"
- \{\{ read_json('tables/data.feather') \}\}
+ {% raw %}
+ ```markdown
+ {{ read_feather('assets/tables/data.feather') }}
+ ```
+ {% endraw %}
=== "Output"
- {{ read_feather('tables/data.feather') }}
+ {{ read_feather('assets/tables/data.feather') | add_indentation(spaces=4) }}
+
+### `read_excel`
-## read_excel
+Use {% raw %}`{{ read_excel() }}`{% endraw %} to read an Excel file and output as a markdown table.
-`{{ read_excel() }}` passed to [pandas.read_excel()](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.read_excel.html). Example:
+1. Arguments are parsed safely and then passed to corresponding functions below
+2. File is read using [pandas.read_excel()](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.read_excel.html)
+3. The `pd.DataFrame` is then converted to a markdown table using [`.to_markdown()`](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.to_markdown.html)
+4. The markdown table is fixed to match the indentation used by the tag in the markdown document (only when _not_ used with `mkdocs-macros-plugin`. See [compatibility with macros plugin](howto/use_jinja2.md))
+Example:
=== "Input"
- \{\{ read_excel('tables/excel_table.xlsx', engine='openpyxl') \}\}
+ {% raw %}
+ ```markdown
+ {{ read_excel('assets/tables/excel_table.xlsx', engine='openpyxl') }}
+ ```
+ {% endraw %}
=== "Output"
- {{ read_excel('tables/excel_table.xlsx', engine='openpyxl') }}
+ {{ read_excel('assets/tables/excel_table.xlsx', engine='openpyxl') | add_indentation(spaces=4) }}
!!! info "Reading xlsx files"
You might get a `XLRDError('Excel xlsx file; not supported',)` error when trying to read modern excel files. That's because `xlrd` does not support `.xlsx` files ([stackoverflow post](https://stackoverflow.com/questions/65254535/xlrd-biffh-xlrderror-excel-xlsx-file-not-supported)). Instead, install [openpyxl](https://openpyxl.readthedocs.io/en/stable/) and use:
- \{\{ read_excel('tables/excel_table.xlsx', engine='openpyxl') \}\}
+ {% raw %}
+ ```markdown
+ {{ read_excel('assets/tables/excel_table.xlsx', engine='openpyxl') }}
+ ```
+ {% endraw %}
+
+### `read_raw`
-## read_raw
+Use {% raw %}`{{ read_raw() }}`{% endraw %} to insert the contents from a file directly.
-`{{ read_raw() }}` inserts contents from a file directly. This is great if you have a file with a table already in markdown format.
+This is great if you have a file with a table already in markdown format.
It could also replace a workflow where you use the [snippets extension to embed external files](https://squidfunk.github.io/mkdocs-material/reference/code-blocks/#embedding-external-files).
+1. Only the first argument is read. This should be the file path.
+2. File is read using python
+4. The markdown table is fixed to match the indentation used by the tag in the markdown document (only when _not_ used with `mkdocs-macros-plugin`. See [compatibility with macros plugin](howto/use_jinja2.md))
+
+Example:
+
+=== "Input"
+
+ {% raw %}
+ ```markdown
+ {{ read_raw('assets/tables/markdown_table.md') }}
+ ```
+ {% endraw %}
+
+=== "Output"
+
+ {{ read_raw('assets/tables/markdown_table.md') | add_indentation(spaces=4) }}
+
+
+## Macros
+
+When you use `table-reader` with [`mkdocs-macros-plugin`](https://mkdocs-macros-plugin.readthedocs.io/en/latest/), in next to all the readers, the following _additional_ macros will be made available:
+
+### `pd_read_csv`
+
+Use {% raw %}`{{ pd_read_csv() }}`{% endraw %} to read a comma-separated values (csv) using [pandas.read_csv()](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.read_csv.html).
+
+=== "Input"
+
+ {% raw %}
+ ```markdown
+ {{ pd_read_csv('assets/tables/basic_table.csv').to_markdown(tablefmt="pipe", index=False) | add_indentation(spaces=4) }}
+ ```
+ {% endraw %}
+
+=== "Output"
+
+ {{ pd_read_csv('assets/tables/basic_table.csv').to_markdown(tablefmt="pipe", index=False) | add_indentation(spaces=4) }}
+
+
+### `pd_read_fwf`
+
+Use {% raw %}`{{ pd_read_fwf() }}`{% endraw %} to read a table of fixed-width formatted lines using [pandas.read_fwf()](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.read_fwf.html)
+
+Example:
+
+=== "Input"
+
+ {% raw %}
+ ```markdown
+ {{ pd_read_fwf('assets/tables/fixedwidth_table.txt').to_markdown(tablefmt="pipe", index=False) | add_indentation(spaces=4) }}
+ ```
+ {% endraw %}
+
+=== "Output"
+
+ {{ pd_read_fwf('assets/tables/fixedwidth_table.txt').to_markdown(tablefmt="pipe", index=False) | add_indentation(spaces=4) }}
+
+
+### `pd_read_yaml`
+
+Use {% raw %}`{{ pd_read_yaml() }}`{% endraw %} to read a YAML file using [yaml.safe_load()](https://pyyaml.org/wiki/PyYAMLDocumentation#loading-yaml) and [pandas.json_normalize()](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.json_normalize.html).
+
+Example:
+
+=== "Input"
+
+ {% raw %}
+ ```markdown
+ {{ pd_read_yaml('assets/tables/yaml_table.yml').to_markdown(tablefmt="pipe", index=False) | add_indentation(spaces=4) }}
+ ```
+ {% endraw %}
+
+=== "Output"
+
+ {{ pd_read_yaml('assets/tables/yaml_table.yml').to_markdown(tablefmt="pipe", index=False) | add_indentation(spaces=4) }}
+
+
+### `pd_read_table`
+
+Use {% raw %}`{{ pd_read_table() }}`{% endraw %} to read a general delimited file using [pandas.read_table()](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.read_table.html).
+
+Example:
+
+=== "Input"
+
+ {% raw %}
+ ```markdown
+ {{ pd_read_table('assets/tables/basic_table.csv', sep = ',').to_markdown(tablefmt="pipe", index=False) | add_indentation(spaces=4) }}
+ ```
+ {% endraw %}
+
+=== "Output"
+
+ {{ pd_read_table('assets/tables/basic_table.csv', sep = ',').to_markdown(tablefmt="pipe", index=False) | add_indentation(spaces=4) }}
+
+
+
+### `pd_read_json`
+
+Use {% raw %}`{{ pd_read_json() }}`{% endraw %} to read a JSON string path using [pandas.read_json()](https://pandas.pydata.org/docs/reference/api/pandas.read_json.html)
+
+Example:
+
+=== "Input"
+
+ {% raw %}
+ ```markdown
+ {{ pd_read_json('assets/tables/data.json', orient='split').to_markdown(tablefmt="pipe", index=False) | add_indentation(spaces=4) }}
+ ```
+ {% endraw %}
+
+=== "Output"
+
+ {{ pd_read_json('assets/tables/data.json', orient='split').to_markdown(tablefmt="pipe", index=False) | add_indentation(spaces=4) }}
+
+
+
+### `pd_read_feather`
+
+Use {% raw %}`{{ pd_read_feather() }}`{% endraw %} to read a feather-format object using [pandas.read_feather()](https://pandas.pydata.org/docs/reference/api/pandas.read_feather.html)
+
+
Example:
=== "Input"
- \{\{ read_raw('tables/markdown_table.md') \}\}
+ {% raw %}
+ ```markdown
+ {{ pd_read_feather('assets/tables/data.feather').to_markdown(tablefmt="pipe", index=False) | add_indentation(spaces=4) }}
+ ```
+ {% endraw %}
=== "Output"
- {{ read_raw('tables/markdown_table.md') }}
+ {{ pd_read_feather('assets/tables/data.feather').to_markdown(tablefmt="pipe", index=False) | add_indentation(spaces=4) }}
+
+
+### `pd_read_excel`
+
+Use {% raw %}`{{ pd_read_excel() }}`{% endraw %} to read an Excel file using [pandas.read_excel()](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.read_excel.html)
+
+Example:
+
+=== "Input"
+
+ {% raw %}
+ ```markdown
+ {{ pd_read_excel('assets/tables/excel_table.xlsx', engine='openpyxl').to_markdown(tablefmt="pipe", index=False) | add_indentation(spaces=4) }}
+ ```
+ {% endraw %}
+
+=== "Output"
+
+ {{ pd_read_excel('assets/tables/excel_table.xlsx', engine='openpyxl').to_markdown(tablefmt="pipe", index=False) | add_indentation(spaces=4) }}
+
+
+!!! info "Reading xlsx files"
+
+ You might get a `XLRDError('Excel xlsx file; not supported',)` error when trying to read modern excel files. That's because `xlrd` does not support `.xlsx` files ([stackoverflow post](https://stackoverflow.com/questions/65254535/xlrd-biffh-xlrderror-excel-xlsx-file-not-supported)). Instead, install [openpyxl](https://openpyxl.readthedocs.io/en/stable/) and use:
+
+ {% raw %}
+ ```markdown
+ {{ pd_read_excel('assets/tables/excel_table.xlsx', engine='openpyxl') }}
+ ```
+ {% endraw %}
+
+
+## Filters
+
+When you use `table-reader` with [`mkdocs-macros-plugin`](https://mkdocs-macros-plugin.readthedocs.io/en/latest/), in next to all the readers, the macros, the following _additional_ filters will be made available:
+
+### `add_indentation`
+
+Adds a consistent indentation to every line in a string. This is important when you are inserting content into [Admonitions](https://squidfunk.github.io/mkdocs-material/reference/admonitions/) or [Content tabs](https://squidfunk.github.io/mkdocs-material/reference/content-tabs/).
+
+Args:
+ text (str): input text
+ spaces (int): Indentation to add in spaces
+ tabs (int): Indentation to add in tabs
+
+
+Example usage:
+
+{% raw %}
+```markdown
+!!! note "this is a note"
+ {{ pd_read_csv('assets/tables/basic_table.csv').to_markdown(tablefmt="pipe", index=False) | add_indentation(spaces=4) }}
+```
+{% endraw %}
+
+### `convert_to_md_table`
+
+Converts a pandas dataframe into a markdown table. Arguments are passed to [`.to_markdown()`](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.to_markdown.html). By default, `tablefmt='pipe'` and `index=False` are used.
+There is also an additional fix to ensure any pipe (`|`) characters in the dataframe are properly escaped ([python-tabulate#241](https://github.com/astanin/python-tabulate/issues/241)).
+
+Example usage:
+
+{% raw %}
+```markdown
+{{ pd_read_csv('assets/tables/basic_table.csv') | convert_to_md_table }}
+```
+{% endraw %}
diff --git a/mkdocs.yml b/mkdocs.yml
index 4240b3c..80b134c 100644
--- a/mkdocs.yml
+++ b/mkdocs.yml
@@ -10,7 +10,7 @@ nav:
- How to:
- howto/customize_tables.md
- howto/preprocess_tables.md
- - howto/use_jinja2.md
+ - Macros plugin compatibility: howto/use_jinja2.md
- howto/project_structure.md
- howto/docker.md
- howto/alternatives.md
@@ -48,6 +48,7 @@ theme:
plugins:
- search
+ - macros
- table-reader:
data_path: "docs/assets"
- git-authors:
diff --git a/mkdocs_table_reader_plugin/markdown.py b/mkdocs_table_reader_plugin/markdown.py
index 919d09e..bc66c6f 100644
--- a/mkdocs_table_reader_plugin/markdown.py
+++ b/mkdocs_table_reader_plugin/markdown.py
@@ -19,7 +19,7 @@ def replace_unescaped_pipes(text: str) -> str:
return re.sub(r"(? str:
+def convert_to_md_table(df: pd.DataFrame, **markdown_kwargs: Dict) -> str:
"""
Convert dataframe to markdown table using tabulate.
"""
diff --git a/mkdocs_table_reader_plugin/plugin.py b/mkdocs_table_reader_plugin/plugin.py
index 444d474..62ab21b 100644
--- a/mkdocs_table_reader_plugin/plugin.py
+++ b/mkdocs_table_reader_plugin/plugin.py
@@ -5,8 +5,8 @@
from mkdocs.exceptions import ConfigurationError
from mkdocs_table_reader_plugin.safe_eval import parse_argkwarg
-from mkdocs_table_reader_plugin.readers import READERS
-from mkdocs_table_reader_plugin.markdown import fix_indentation, add_indentation
+from mkdocs_table_reader_plugin.readers import READERS, MACROS
+from mkdocs_table_reader_plugin.markdown import fix_indentation, add_indentation, convert_to_md_table
logger = get_plugin_logger("table-reader")
@@ -70,13 +70,23 @@ def on_config(self, config, **kwargs):
)
if "macros" in config.plugins:
- config.plugins["macros"].macros.update(self.readers)
- config.plugins["macros"].variables["macros"].update(self.readers)
- config.plugins["macros"].env.globals.update(self.readers)
-
- config.plugins["macros"].filters.update({"add_indentation": add_indentation})
- config.plugins["macros"].variables["filters"].update({"add_indentation": add_indentation})
- config.plugins["macros"].env.filters.update({"add_indentation": add_indentation})
+ self.macros = {
+ macro: MACROS[macro].set_config_context(
+ mkdocs_config=config, plugin_config=self.config
+ )
+ for macro in MACROS
+ }
+ self.filters = {
+ "add_indentation": add_indentation,
+ "convert_to_md_table": convert_to_md_table,
+ }
+ config.plugins["macros"].macros.update(self.macros)
+ config.plugins["macros"].variables["macros"].update(self.macros)
+ config.plugins["macros"].env.globals.update(self.macros)
+
+ config.plugins["macros"].filters.update(self.filters)
+ config.plugins["macros"].variables["filters"].update(self.filters)
+ config.plugins["macros"].env.filters.update(self.filters)
self.external_jinja_engine = True
else:
diff --git a/mkdocs_table_reader_plugin/readers.py b/mkdocs_table_reader_plugin/readers.py
index 7794625..2144d62 100644
--- a/mkdocs_table_reader_plugin/readers.py
+++ b/mkdocs_table_reader_plugin/readers.py
@@ -60,22 +60,38 @@ def __call__(self, *args, **kwargs):
return self.func(valid_file_paths[0], *args, **kwargs)
+@ParseArgs
+def pd_read_csv(*args, **kwargs) -> str:
+ read_kwargs = kwargs_in_func(kwargs, pd.read_csv)
+ return pd.read_csv(*args, **read_kwargs)
+
@ParseArgs
def read_csv(*args, **kwargs) -> str:
read_kwargs = kwargs_in_func(kwargs, pd.read_csv)
df = pd.read_csv(*args, **read_kwargs)
markdown_kwargs = kwargs_not_in_func(kwargs, pd.read_csv)
- return convert_to_md_table(df, markdown_kwargs)
+ return convert_to_md_table(df, **markdown_kwargs)
+@ParseArgs
+def pd_read_table(*args, **kwargs) -> str:
+ read_kwargs = kwargs_in_func(kwargs, pd.read_table)
+ return pd.read_table(*args, **read_kwargs)
+
@ParseArgs
def read_table(*args, **kwargs) -> str:
read_kwargs = kwargs_in_func(kwargs, pd.read_table)
df = pd.read_table(*args, **read_kwargs)
markdown_kwargs = kwargs_not_in_func(kwargs, pd.read_table)
- return convert_to_md_table(df, markdown_kwargs)
+ return convert_to_md_table(df, **markdown_kwargs)
+
+
+@ParseArgs
+def pd_read_fwf(*args, **kwargs) -> str:
+ read_kwargs = kwargs_in_func(kwargs, pd.read_fwf)
+ return pd.read_fwf(*args, **read_kwargs)
@ParseArgs
@@ -84,7 +100,12 @@ def read_fwf(*args, **kwargs) -> str:
df = pd.read_fwf(*args, **read_kwargs)
markdown_kwargs = kwargs_not_in_func(kwargs, pd.read_fwf)
- return convert_to_md_table(df, markdown_kwargs)
+ return convert_to_md_table(df, **markdown_kwargs)
+
+@ParseArgs
+def pd_read_json(*args, **kwargs) -> str:
+ read_kwargs = kwargs_in_func(kwargs, pd.read_json)
+ return pd.read_json(*args, **read_kwargs)
@ParseArgs
@@ -93,7 +114,12 @@ def read_json(*args, **kwargs) -> str:
df = pd.read_json(*args, **read_kwargs)
markdown_kwargs = kwargs_not_in_func(kwargs, pd.read_json)
- return convert_to_md_table(df, markdown_kwargs)
+ return convert_to_md_table(df, **markdown_kwargs)
+
+@ParseArgs
+def pd_read_excel(*args, **kwargs) -> str:
+ read_kwargs = kwargs_in_func(kwargs, pd.read_excel)
+ return pd.read_excel(*args, **read_kwargs)
@ParseArgs
@@ -102,9 +128,16 @@ def read_excel(*args, **kwargs) -> str:
df = pd.read_excel(*args, **read_kwargs)
markdown_kwargs = kwargs_not_in_func(kwargs, pd.read_excel)
- return convert_to_md_table(df, markdown_kwargs)
+ return convert_to_md_table(df, **markdown_kwargs)
+@ParseArgs
+def pd_read_yaml(*args, **kwargs) -> str:
+ json_kwargs = kwargs_in_func(kwargs, pd.json_normalize)
+ with open(args[0], "r") as f:
+ df = pd.json_normalize(yaml.safe_load(f), **json_kwargs)
+ return df
+
@ParseArgs
def read_yaml(*args, **kwargs) -> str:
json_kwargs = kwargs_in_func(kwargs, pd.json_normalize)
@@ -112,7 +145,13 @@ def read_yaml(*args, **kwargs) -> str:
df = pd.json_normalize(yaml.safe_load(f), **json_kwargs)
markdown_kwargs = kwargs_not_in_func(kwargs, pd.json_normalize)
- return convert_to_md_table(df, markdown_kwargs)
+ return convert_to_md_table(df, **markdown_kwargs)
+
+
+@ParseArgs
+def pd_read_feather(*args, **kwargs) -> str:
+ read_kwargs = kwargs_in_func(kwargs, pd.read_feather)
+ return pd.read_feather(*args, **read_kwargs)
@ParseArgs
@@ -121,7 +160,7 @@ def read_feather(*args, **kwargs) -> str:
df = pd.read_feather(*args, **read_kwargs)
markdown_kwargs = kwargs_not_in_func(kwargs, pd.read_feather)
- return convert_to_md_table(df, markdown_kwargs)
+ return convert_to_md_table(df, **markdown_kwargs)
@ParseArgs
@@ -145,3 +184,14 @@ def read_raw(*args, **kwargs) -> str:
"read_feather": read_feather,
"read_raw": read_raw,
}
+
+MACRO_ONLY = {
+ "pd_read_csv": pd_read_csv,
+ "pd_read_table": pd_read_table,
+ "pd_read_fwf": pd_read_fwf,
+ "pd_read_excel": pd_read_excel,
+ "pd_read_yaml": pd_read_yaml,
+ "pd_read_json": pd_read_json,
+ "pd_read_feather": pd_read_feather,
+}
+MACROS = {**READERS, **MACRO_ONLY}
\ No newline at end of file
diff --git a/setup.py b/setup.py
index f3fedda..ecd1904 100644
--- a/setup.py
+++ b/setup.py
@@ -5,7 +5,7 @@
setup(
name="mkdocs-table-reader-plugin",
- version="3.0.1",
+ version="3.1.0",
description="MkDocs plugin to directly insert tables from files into markdown.",
long_description=long_description,
long_description_content_type="text/markdown",
diff --git a/tests/fixtures/jinja/docs/index.md b/tests/fixtures/jinja/docs/index.md
index 82d4f8d..97dc580 100644
--- a/tests/fixtures/jinja/docs/index.md
+++ b/tests/fixtures/jinja/docs/index.md
@@ -27,3 +27,13 @@ This is a table that we load from the docs folder, because we set `data_path` to
{% endfor %}
+## Filtering results
+
+{% raw %}
+```
+{{ pd_read_csv("numeric_table.csv").query("a >= 3") | convert_to_md_table }}
+```
+{% endraw %}
+
+{{ pd_read_csv("numeric_table.csv").query("a >= 3") | convert_to_md_table }}
+
diff --git a/tests/fixtures/jinja/docs/numeric_table.csv b/tests/fixtures/jinja/docs/numeric_table.csv
new file mode 100644
index 0000000..598f36c
--- /dev/null
+++ b/tests/fixtures/jinja/docs/numeric_table.csv
@@ -0,0 +1,4 @@
+"a","b"
+1,2
+3,4
+5,6
\ No newline at end of file
diff --git a/tests/fixtures/search_problem/docs/index.md b/tests/fixtures/search_problem/docs/index.md
new file mode 100644
index 0000000..3ce06e2
--- /dev/null
+++ b/tests/fixtures/search_problem/docs/index.md
@@ -0,0 +1,21 @@
+# Homepage
+
+```json
+{
+ "configuration": [
+ {
+ "category": "Advanced",
+ "component": "NetworkModel",
+ "defaultvalue": "false",
+ "description": "This will determine if the admin is allowed to deploy.",
+ "displaytext": "Admin is allowed to deploy",
+ "group": "Miscellaneous",
+ "isdynamic": true,
+ "name": "deploy.anywhere",
+ "subgroup": "Others",
+ "type": "Boolean",
+ "value": "true"
+ }
+ ]
+}
+```
\ No newline at end of file
diff --git a/tests/fixtures/search_problem/mkdocs.yml b/tests/fixtures/search_problem/mkdocs.yml
new file mode 100644
index 0000000..fa55d35
--- /dev/null
+++ b/tests/fixtures/search_problem/mkdocs.yml
@@ -0,0 +1,10 @@
+site_name: test search
+
+theme:
+ name: material
+ features:
+ - navigation.tabs
+
+plugins:
+ - search
+ - table-reader
\ No newline at end of file
diff --git a/tests/test_markdown.py b/tests/test_markdown.py
index f3a11c1..9425820 100644
--- a/tests/test_markdown.py
+++ b/tests/test_markdown.py
@@ -14,6 +14,6 @@ def test_convert_to_md_table():
assert df_good.shape[0] > 0
# Because we escape pipes, the 'bad' df
- md_bad = convert_to_md_table(df_bad, markdown_kwargs={})
- md_good = convert_to_md_table(df_good, markdown_kwargs={})
+ md_bad = convert_to_md_table(df_bad, **{})
+ md_good = convert_to_md_table(df_good, **{})
assert md_bad == md_good
\ No newline at end of file