bmdf

Bash markdown fences: embed commands, their output, and tables of contents in markdown

• Install
• Usage
  • bmd: format bash command and output as Markdown
    • bmdf (bmd -f): command+output mode
    • bmdff (bmd -ff): two-fence mode
    • bmdfff (bmd -fff): <details> mode
    • Piping
    • Env vars
    • -w/--workdir / $BMDF_WORKDIR
  • mdcmd: update commands embedded in Markdown files
  • mktoc: update table of contents
  • Reference
• Examples

This package provides 3 CLIs:

• bmd: run Bash commands, wrap output for Markdown embedding
  • bmdf, bmdff, bmdfff: different levels of "fencing" for command output
• mdcmd: find bmd blocks in Markdown files, execute commands, update Markdown files with output
• mktoc: update Markdown table of contents (with custom "id" anchors)

Install

pip install bmdf

Usage

bmd: format bash command and output as Markdown

bmd (and aliases bmdf, bmdff, bmdfff) takes a bash command as input, and renders the command and/or its output in various Markdown-friendly formats:

bmdf (bmd -f): command+output mode

Suppose you want to embed a command and its output in a README.md, like this:

seq 3
# 1
# 2
# 3

(Note how the command is bash-highlighted, and output lines are rendered as comments)

Put a placeholder like this in your README.md:

<!-- `bmdf seq 3` -->

then run mdcmd to update your README containing this embedded command block.

bmdff (bmd -ff): two-fence mode

bmdff (alias for bmd -ff) renders two code fences, one with the Bash command (syntax-highlighted appropriately), and a second (non-highlighted) block with the output, e.g.:

<!-- `bmdff seq 5` -->

becomes:

seq 5

1
2
3
4
5

bmdfff (bmd -fff): <details> mode

When a command's output is large, rendering it as a <details><summary> (with the output collapsed, by default) may be preferable.

bmdfff (3 fs, alias for bmd -fff) transforms placeholders like this:

<!-- `bmdfff seq 10` -->

to:

seq 10

1
2
3
4
5
6
7
8
9
10

Piping

Piping works too, e.g.:

<!-- `bmdf -- seq 10 | wc -l` -->

will become:

seq 10 | wc -l
# 10

(the -- is needed so that that -l isn't parsed as an opt to bmdf)

Env vars

By default, shell=True is passed to subprocess calls (but can be disabled via -S).

This means env vars are expanded; they can also be set via -E, e.g.:

<!-- `bmdf -E FOO=bar echo $FOO` -->

yields:

FOO=bar echo '$FOO'
# bar

More examples of quoting/splitting behavior

Quoting "$FOO":

<!-- `bmdf -E FOO=bar echo "$FOO"` -->

yields:

FOO=bar echo '$FOO'
# bar

Arg with spaces:

<!-- `bmdf -E FOO=bar echo "FOO: $FOO"` -->

yields:

FOO=bar echo 'FOO: $FOO'
# FOO: bar

Escaping $:

<!-- `bmdf -E FOO=bar echo "\$FOO=$FOO"` -->

yields:

FOO=bar echo '\$FOO=$FOO'
# $FOO=bar

-w/--workdir / $BMDF_WORKDIR

By default, bmdf runs in the current working directory. This can be overridden with -w:

<!-- `bmdf -w .github ls` -->

ls
# workflows

mdcmd: update commands embedded in Markdown files

# Modify README.md in-place
mdcmd -i README.md
# Same as above; no args defaults to `-i README.md`
mdcmd

The placeholder block above will now contain seq 3 and its output; that's how first block above is rendered!

The full README.md block will now look like:

<!-- `bmdf seq 3` -->
```bash
seq 3
# 1
# 2
# 3
```

and running mdcmd again will rewrite the same content.

Note: bmdf (alias for bmd -f) is used because it wraps the output of whatever it's passed in a "Bash fence" block. You don't have to use it, but most commands will fail to output a Markdown "fence" block, and subsequent mdcmd invocations will fail to parse them.

Scripts that output raw HTML also work, e.g. print-table.py generates this table:

header 1	header 2
cell 1	cell 2

mktoc: update table of contents

Put a block like this in your README.md:

<!-- toc -->
<!-- /toc -->

Then put empty <a> tags next to the headings you want to include, e.g.:

## My section heading <a id="my-section"></a>

(This allows for customizing and shortening the ids, as well as skipping sections)

Then run:

# Modify README.md in-place
mktoc -i README.md
# Same as above; no args defaults to `-i README.md`
mktoc

And the <!-- toc --> section will have a table of contents injected (like the one at the top of this file).

Reference

bmd
# Usage: bmd [OPTIONS] COMMAND...
#
#   Format a command and its output to markdown, either in a `bash`-fence or
#   <details> block, and copy it to the clipboard.
#
# Options:
#   -A, --strip-ansi                Strip ANSI escape sequences from output
#   -C, --no-copy                   Disable copying output to clipboard
#                                   (normally uses first available executable
#                                   from ['pbcopy', 'xclip', 'clip']
#   -e, --error-fmt TEXT            If the wrapped command exits non-zero,
#                                   append a line of output formatted with this
#                                   string. One "%d" placeholder may be used,
#                                   for the returncode. Defaults to
#                                   $BMDF_ERR_FMT
#   -E, --env TEXT                  k=v env vars to set, for the wrapped command
#   -f, --fence                     Pass 0-3x to configure output style: 0x:
#                                   print output lines, prepended by "# "; 1x:
#                                   print a "```bash" fence block including the
#                                   <command> and commented output lines; 2x:
#                                   print a bash-fenced command followed by
#                                   plain-fenced output lines; 3x: print a
#                                   <details/> block, with command <summary/>
#                                   and collapsed output lines in a plain fence.
#   -i, --include-stderr / -I, --no-include-stderr
#                                   Capture and interleave both stdout and
#                                   stderr streams; falls back to
#                                   $BMDF_INCLUDE_STDERR
#   -s, --shell / -S, --no-shell    Disable "shell" mode for the command; falls
#                                   back to $BMDF_SHELL, but defaults to True if
#                                   neither is set
#   -t, --fence-type TEXT           When -f/--fence is 2 or 3, this customizes
#                                   the fence syntax type that the output is
#                                   wrapped in
#   -u, --expanduser / -U, --no-expanduser
#                                   Pass commands through `os.path.expanduser`
#                                   before `subprocess`; falls back to
#                                   $BMDF_EXPANDUSER
#   -v, --expandvars / -V, --no-expandvars
#                                   Pass commands through `os.path.expandvars`
#                                   before `subprocess`; falls back to
#                                   $BMDF_EXPANDVARS
#   -w, --workdir TEXT              `cd` to this directory before executing
#                                   (falls back to $BMDF_WORKDIR
#   -x, --executable TEXT           `shell_executable` to pass to Popen
#                                   pipelines (default: $SHELL)
#   --help                          Show this message and exit.

mdcmd --help
# Usage: mdcmd [OPTIONS] [PATH] [OUT_PATH]
#
#   Parse a Markdown file, updating blocks preceded by <!-- `[cmd...]` -->
#   delimiters.
#
#   If no paths are provided, will look for a README.md, and operate "in-place"
#   (same as ``mdcmd -i README.md``).
#
# Options:
#   -a, --amend                     Squash changes onto the previous Git commit;
#                                   suitable for use with `git rebase -x`
#   -i, --inplace / -I, --no-inplace
#                                   Edit the file in-place
#   -n, --dry-run                   Print the commands that would be run, but
#                                   don't execute them
#   -T, --no-cwd-tmpdir             In in-place mode, use a system temporary-
#                                   directory (instead of the current workdir,
#                                   which is the default)
#   -x, --execute TEXT              Only execute commands that match these
#                                   regular expressions
#   -X, --exclude TEXT              Only execute commands that don't match these
#                                   regular expressions
#   --help                          Show this message and exit.

mktoc --help
# Usage: mktoc [OPTIONS] [PATH] [OUT_PATH]
#
#   Insert a table of contents (TOC) in a markdown file.
#
#   Looks for a pair of sentinel lines to insert or update the TOC between:
#   ``<!-- toc -->``, ``<!-- /toc -->``.
#
#   If an empty line follows the opening ``<!-- toc -->`` line, the TOC will be
#   inserted there (along with the closing sentinel); this is useful when
#   initially generating a TOC.
#
#   If no ``out_path`` is provided, will operate "in-place" on ``README.md`` (as
#   if ``mktoc -i README.md`` was passed).
#
# Options:
#   -a, --amend                     Squash changes onto the previous Git commit;
#                                   suitable for use with `git rebase -x`
#   -i, --inplace / -I, --no-inplace
#                                   Edit the file in-place
#   -n, --indent-size INTEGER       Indent size (spaces)
#   -T, --no-cwd-tmpdir             In in-place mode, use a system temporary-
#                                   directory (instead of the current workdir,
#                                   which is the default)
#   --help                          Show this message and exit.

Examples

The examples in this file are rendered using bmdf and mdcmd, and the TOC is rendered using mktoc. Both are verified by the ci.yml GitHub Action.

These repos' READMEs also use bmdf / mdcmd / mktoc to execute/verify examples commands (and in some cases are also verified by GitHub Actions):

• runsascoded/juq
• runsascoded/utz
• runsascoded/dvc-utils
• ryan-williams/dvc-helpers (GHA)
• ryan-williams/git-helpers
• ryan-williams/parquet-helpers (GHA)
• ryan-williams/tdbs-dask
• TileDB-Inc/scverse-ml-workshop-2024