This document describes the mdpre Markdown preprocessor.
In this document we'll refer to it as "mdpre", pronounced "em dee pree".
This document was converted to HTML at 14:26 on 17 December, 2023.
- Why A Preprocessor?
- How Do You Use mdpre?
- Language Elements
- Including files with
=include - Defining Variables With
=def - Undefining Variables With
=undef - Incrementing Integer Variables Wth
=inc - Decrementing Integer Variables Wth
=dec - Conditional Inclusion With
=ifdef,=ifndef, And=endif - Converting A CSV File To A Markdown Table With
=csvAnd=endcsv - Creating A Calendar Month Table With
=cal - Concatenating Lines With
\ - Generating A Table Of Contents With
=toc - Terminating Input With
=stop
- Including files with
- Additional Information
Ask yourself the following questions.
-
Have you ever wanted to create a Markdown document from more than one piece of text?
-
Have you ever wanted to make a Markdown table from a CSV file?
-
Have you ever wanted to control whether text was included or not?
-
Have you ever wanted to use variables in Markdown?
-
Have you ever wanted to have a Table Of Contents automatically generated in Markdown?
-
Have you wanted to use version control - such as git?
If you answer "yes" to any of these you need mdpre.
You write Markdown in exactly the same way as normal, with simple syntax extensions to invoke the additional function.
To use mdpre you need to
- Download it.
- Have Python 3 installed - at a reasonably high level.
- Invoke it.
The following instructions are for Unix-like systems. (It's developed and used by the developer on MacOS and Raspbian.) Windows users will need a slightly different form, but the principle is the same.
Here is a sample invocation:
mdpre < document.mdp > document.md
In this invocation the document.mdp file is the source and the "vanilla" Markdown produced by mdpre is written to document.md.
You can, of course, use stdin and stdout in a pipeline. Indeed the developer uses this to pipe into another program.
Messages are written to stderr.
You can get parameter help using the -h parameter:
mdpre -h
If you want more diagnostics turn on "verbose mode", using the -v parameter:
mdpre -v < document.mdp > document.md
This documents most of the major events and decisions, such as the embedding of files, conditional inclusion, and the conversion of CSV data to Markdown tables. One early run of this document with -v specified resulted in
mdpre Markdown Preprocessor v0.1 (17 March, 2018)
=================================================
Def TOC = Table Of Contents
Markdown Preprocessor User Guide
Table Of Contents - spec '2 * Table Of Contents'
..... Why A Preprocessor?
..... How Do You Use mdpre?
..... ..... Help
..... ..... Verbose Mode
..... ..... Filenames
..... Language Elements
..... ..... Including files with `=include`
..... ..... Defining Variables With `=def`
..... ..... Undefining Variables With `=undef`
..... ..... Conditional Inclusion With `=ifdef`, `=ifndef`, And `=endif`
..... ..... Converting A CSV File To A Markdown Table With `=csv` And `=endcsv`
CSV Start
CSV Stop
..... ..... ..... Controlling Table Alignment With `=colalign`
Column Alignment - spec 'l r r'
CSV Start
CSV Stop
..... ..... ..... Controlling Table Column Widths With `=colwidth`
..... ..... Concatenating Lines With `\`
..... ..... Generating A Table Of Contents With `=toc`
Processing completed.
You can define variables on the command line when you run mdpre. Use the -d switch. For example
mdpre -v -dfred -dbrian < test.mdp > test.md
will set two variables - fred and brian. These can be used with =ifdef and =ifndef to control processing. See Conditional Inclusion With =ifdef, =ifndef, And =endif for more.
You can also set their value. For example
mdpre -v '-djon=bon jovi' < test.mdp > test.md
In this case the whole -d parameter string is in quotes to preserve spaces in the value.
You can use any names you like but an extension to denote that the input file is a preprocessable one is handy. In the above example .mdp was used. This seems like a sensible convention.
Similarly, common file extensions for Markdown files are .md and .markdown.
TextBundle and TextPack are formats for storing complete documents, including graphics, in a container. They are available in two flavours:
- In a directory - in "Package" format (.textbundle)
- In a zip file - in "Compressed" format (.textpack)
These mechanisms allow easy transfer to other tools.
mdpre can create a TextBundle format file (.textbundle). To create it type something like:
mdpre -v -t user-guide.textbundle < user-guide.mdp
This will result in the Markdown that would normally go to stdout instead going into the textbundle. Graphics will go to the /assets subdirectory.
NOTE: mdpre places all graphics directly in the /assets subdirectory. If two graphics files were incorporated with the same name one would overwrite the other.
If you want textpack format use this form
mdpre -v -z user-guide.textpack < user-guide.mdp
z here stands for "zipped". mdpre will first make a temporary textbundle and then create a textpack from it. Finally it will delete the temporary textbundle (directory structure). In this example -v was specified. You probably want this - to observe the creation, zipping, and deletion activity.
Using this user guide as a test vehicle, three Mac OS apps were tested with the textpack format:
Good results were obtained with both the Ulysses and MindNode apps. The Bear app produced slightly less good results: While text elements rendered fine, the one graphics reference was improperly rendered as an Internet reference.
mdpre can process TextBundle files directly, and you can convert a TextPack file into a TextBundle by unzipping it. mdpre doesn't have special code to read from either of these formats.
The input to mdpre generally consists of standard Markdown. However, it would be a very boring use of mdpre if that's all you wrote.
This section describes additional statements, which are the real reason for using mdpre in the first place.
The spirit of some of these additional statements is similar to that of C compiler preprocessors. However the # character is already taken in Markdown so commands begin with = instead. All statements begin at the first character on the line.
You can include files with the =include statement. Here is a sample invocation:
=include HiperDispatch.mdp
You can nest =include statements to any depth.
mdpre detects recursion in including files. If recursion is detected mdpre will terminate with a message. This is to prevent infinite recursion.
You can use a variable in a include filename. For example:
=def includeName b
=include &includeName;.mdp
In this case the file that will be included is b.mdp.
You can define variables with one of two forms of the =def statement:
=def varName=def varName varValue
The variable name cannot contain spaces. If you try the parser will interpret anything after the first space as a value. Other than that there are no restrictions on variable names.
Defining variables is useful with the =ifdef and =ifndef statements.
To use a variable you've defined with a value code the variable name preceded by an ampersand and terminated by a semicolon. For example:
=def greeting Hello
&greeting; World!
will produce:
Hello World!
If you don't terminate the use of the variable with a semicolon it won't be substituted. If the variable isn't defined at the point of usage it won't be substituted.
You can also define variables and, optionally, set their value with the -d command line parameter. See Defining Variables for more.
Just as you can use =def to define a variable you can remove it from the variable table using =undef. For example,
=undef xyzzy
will mean the variable xyzzy will no longer be defined.
If a variable has an integer value it can be incremented. For example,
=def counter1 1
=include B&counter1;.md
=inc counter1
=include B&counter1;.md
In this example B1.md will be included and then B2.md will be included
If a variable has an integer value it can be decremented. For example,
=def counter1 1
⋮
=dec counter1
In this example counter1 will start with the value 1 and after the =dec its value will be 0.
=ifdef and =ifndef perform very similar functions, but are complementary to each other.
With =ifdef, if the variable referred to is defined (whatever its value) the following text is processed.
The conditional bracket is ended with a line with =endif on it.
Here is an example:
=def includeMe
=ifdef includeMe
Here is some text that would be included by the test on the line above.
=endif
Only one line is included - and only if the variable includeMe has been defined.
To exclude the line if the variable is set use =ifndef instead.
Any mdpre statement can be included in the bracket. For example, you could conditionally include another file with =include. For example:
=def wantLongVersion
=ifdef wantLongVersion
=include theWorks.mdp
=endif
You can define variables for use with =ifdef and ifndef in one of two ways:
- With
=def, described in Defining Variables With=def. - With the
-dcommand line switch, described in Defining Variables.
If you have a Comma-Separated Value (CSV) file you want to render as a table in a Markdown document use the =csv statement.
Here is an example:
=csv
"A","1",2
"B","20",30
=endcsv
The table consists of two lines and will render as
| A | 1 | 2 |
|---|---|---|
| B | 20 | 30 |
(This manual uses this very function.)
The actual Markdown for the table produced is:
|A|1|2|
|:-|:-|:-|
|B|20|30|
You'll notice an extra line crept in. By default, mdpre creates tables where the first CSV line is the title and all columns are of equal width and left-aligned.
If you have a file which is purely CSV you don't actually need to code =csv and =endcsv in the input file just to convert it to a Markdown table - if you are happy with default column widths and alignments. Just use the -c command line parameter:
mdpre -c < input.csv > output.md
You can control the alignment with e.g.
=colalign l r r
and the result would be
| A | 1 | 2 |
|---|---|---|
| B | 20 | 30 |
(This manual uses this very function.)
The actual Markdown for the table produced is:
|A|1|2|
|:-|-:|-:|
|B|20|30|
You can specify one of three alignments: l (for "left"), r (for "right"), or c (for "centre"). The default for a column is l.
You can control the column widths with statements like
=colwidth 1 1 2
Adding that to the above produces the following Markdown
|A|1|2|
|:-|-:|--:|
|B|20|30|
Here the third column is specified as double the width of the others.
Note: Many Markdown processors ignore width directives. The developer's other Markdown tool doesn't. :-)
You can set the <span> element's class attribute for each cell in the immediately following row using =rowspan. For example
=rowspan bad
wraps each table cell in the following row with <span class="bad"> and </span>.
Of course this class can apply any styling - through CSS - you like. But typically it would be used for colouring the text.
Note: This styling only applies to the immediately following row.
You can generate a Markdown table that shows all the days in a month. For example:
| November 2021 | ||||||
|---|---|---|---|---|---|---|
| Mo | Tu | We | Th | Fr | Sa | Su |
| 1 |
2 |
3 |
4 |
5 |
||
| 6 |
7 |
8 |
9 |
10 |
11 |
12 |
| 13 |
14 |
15 |
16 |
17 |
18 |
19 |
| 20 |
21 |
22 |
23 |
24 |
25 |
26 |
| 27 |
28 |
29 |
30 |
31 |
This example is the simplest case. You can enhance such a table in a number of ways.
But here's the syntax to generate the above table:
=cal 2021 11
=endcal
The =endcal line is necessary because it enables more commands within the =cal / =endcal bracket.
You can control the height of the cells - in terms of the number of lines by specifying a third parameter:
=cal 2021 11 3
=endcal
In this example each cell is 3 lines high. The default is for each cell to be 2 lines high.
Notes:
- By repeated use of
=caland=endcalyou can add multiple months to a document (for example a md2pptx-processed slide). - The rendering above isn't faithful because mdpre (at least in combination with md2pptx) creates a table where the month name spans the entire width of the table and the columns are evenly sized.
Here is an example of a calendar that uses all the capabilities (described below):
=cal 2021 12 3
=caldays blue 14-17
=caldays orange 18
=calkey blue We have data for **somehow**
=calkey orange Is focus day
=calnote 17 A
=calnote 18 <span class="green">B</span>
=calnote 4 X
=endcal
* An unrelated bullet
It produces something like this:
| December 2021 | ||||||
|---|---|---|---|---|---|---|
| Mo | Tu | We | Th | Fr | Sa | Su |
| |
|
1 |
2 |
3 |
4 X |
5 |
| 6 |
7 |
8 |
9 |
10 |
11 |
12 |
| 13 |
14 |
15 |
16 |
17 A |
18 B |
19 |
| 20 |
21 |
22 |
23 |
24 |
25 |
26 |
| 27 |
28 |
29 |
30 |
31 |
|
|
-
We have data for somehow
-
Is focus day
-
An unrelated bullet
although in md2pptx the bullets would be coloured appropriately - and mdpre actually centres the month.
You can style days in the calendar with CSS using the class attribute. For example:
=caldays blue 14 15 16 17
=caldays orange 18
In this case days 14, 15, 16, and 17 are styled with CSS class blue, whereas day 18 is styled with CSS class orange. You need to code the CSS for classes blue and orange for these specifications to be effective.
=caldays must be specified on its own line between the =cal and =endcal lines. You can specify as many =caldays lines as you like within this bracket.
You can specify a range of days in the following manner:
=caldays blue 14-17 22-24 27
Notes:
- In a range specification there mustn't be spaces between the
-and the numbers either side. - A later
=caldaysline overrides a prior one in the sequence - within a=cal/=endcalbracket.
You can add bullets below the table to explain what each style means. For example:
=calkey blue We have data for **somehow**
=calkey orange Is focus day
causes the following Markdown / HTML to be generated:
* <span class='blue'> </span>We have data for **somehow**
* <span class='orange'> </span>Is focus day
As with =caldays, you need to code the CSS for classes blue and orange for these specifications to be effective.
=calkey must be specified on its own line between the =cal and =endcal lines. You can specify as many =calkey lines as you like within this bracket.
Notes:
- You can continue the bulleted list that
=caldaysgenerates with other bulleted list items; Just code those bulleted list items after=endcal. - While you need not use the same class names as
=caldaysthe intended use case is for you to do so. - md2pptx will use the
<span>element to e.g. colour the bullets. - A later
=calkeyline overrides a prior one in the sequence - within a=cal/=endcalbracket.
You can add annotations to individual calendar dates with =calnote. For example:
=calnote 17 A
=calnote 18 <span class="green">B</span>
=calnote 17 Aadds the text "A" to the cell for 17th of the month.=calnote 18 <span class="green">B</span>adds the text "B" in (presumably) green to the cell for the 18th of the month.
=calnote must be specified on its own line between the =cal and =endcal lines. You can specify as many =calnote lines as you like within this bracket.
You can specify a range of days in the following manner:
=calnote 14-17 RMF
Notes:
- In the examples given the text is short. It probably looks worse if the text is long.
- The second example shows use of
<span>. Equally you can code<br/>as needed. This would be a case where coding the third parameter to=calcan be used to ensure all the cells have the same height. - A later
=calnoteline overrides a prior one in the sequence - within a=cal/=endcalbracket. - In a range specification there mustn't be spaces between the
-and the numbers either side.
Some Markdown processors treat text on separate lines as requiring a line break between the lines. But you might prefer to write each sentence in a paragraph on a separate line. Indeed this paragraph was written that way.
If you terminate a line with a \ (backslash) character the following line will be concatenated to it. For example:
Some Markdown processors treat text on separate lines as requiring a line break between the lines. \
But you might prefer to write each sentence in a paragraph on a separate line. \
Indeed this paragraph was written that way.
Terminating a line with a \ does not cause a space to be inserted between the two lines of text. In the above example a space was coded before the \ each time it was used.
You might want to cause vertical spacing with <br/>. Just above it might be a bulleted list, say. If you are adding items to the list you probably don't want to add a \ to the last item in the list - because you might add other items after it. If you code a line starting with <br/> it will be concatenated to the previous line, whether that previous line ended with a \ or not. You can, of course, code multiple <br/>s on the same line.
You can generate a Table Of Contents with =toc. At its simplest you can simply code
=toc
If this is all you specify, it will generate a heading level 3 titled "Contents" and nested bullets for every heading level - from 1 up to 6.
If you want to limit the level of detail you can specify e.g.
=toc 3
This will limit the Table Of Contents to levels 1 to 3 and use the default title of "Contents".
You can change the title using e.g.
=toc Agenda
or even
=toc 3 Topics
The latter specifies both the maximum level of headings included in the Table Of Contents and the title.
You can specify the minimum and maximum levels:
=toc 2 2 Topics
would generate a Table Of Contents for heading level 2 only. This is particularly useful for a presentation.
If you don't want to specify an upper limit and have mdpre create a table of contents for all levels, starting with 2 you could code
=toc 2 * Table Of Contents
which is what this document uses.
Each Table Of Contents entry will have an internal document link to the corresponding heading. This is generated by mdpre from the text of the heading, replacing spaces with dashes and deleting certain special characters. While this works most of the time, it might occasionally fail. If it fails for you please let us know and we'll endeavour to patch the link generation code.
Here's a scenario: You copy some Markdown into a new file. Then you work on adapting this text, chunk by chunk. It's sometimes useful to be able to stop processing just after your last edit.
A line beginning =stop causes all further input lines to be ignored.
As you work on your text you can, of course, move the =stop line down.
mdpre has the following built-in variables:
| Variable | Description | Example Result |
|---|---|---|
| date | Date when mdpre started running | 6 March, 2019 |
| time | Time when mdpre startred running | 21:36 |
| userid | Userid mdpre was run under | martinpacker |
| mdpre_level | Level of mdpre | 0.4.4 |
| mdpre_date | Date of mdpre | 9 March: 2019 |