Skip to content

Latest commit

 

History

History
517 lines (378 loc) · 11.7 KB

README.md

File metadata and controls

517 lines (378 loc) · 11.7 KB
Raw

dhall-docs

For installation or development instructions, see:

Introduction

This dhall-docs package provides a command-line utility that takes a dhall package or file and outputs an HTML documentation of it.

Features

dhall-docs can analyze your Dhall package (essentially a folder with several .dhall files) to generate documentation. Specifically:

  • Extracts documentation from each file's header comments (see Comment format).
  • The generated documentation includes breadcrumbs to aid navigation.
  • Create an index for each folder in your package listing the .dhall files in that folder alongside the "exported packages" (the contained folders).
  • Extracts examples from assertions.
  • Extracts the type of each Dhall file from the source code and renders it in the indexes.
  • Renders the source code in each Dhall file's documentation.
  • Jump-to-definition on imports and let-bindings

To see a demo, visit the documentation for the Dhall Prelude. You can also check the tests folder for further examples

Usage

The easiest usage is the following:

dhall-docs --input ${PACKAGE-FOLDER}

dhall-docs will store the documentation in ${XDG_DATA_HOME}/dhall-docs/${OUTPUT-HASH}-${PACKAGE-NAME}/, where:

  • $XDG_DATA_HOME environment variable comes from the xdg specification. If it is not defined, dhall-docs will default to ~/.local/share/.
  • OUTPUT-HASH is the hash of the generated documentation. This is to make the folder content-addressable. Also, it avoids overwriting the documentation folder when there was a change in the way it was generated.
  • PACKAGE-NAME is the package name of your documentation. By default, it will be the basename of the --input folder, but you can override it via --package-name.

After generating the documentation, dhall-docs will create a symlink to the documentation index at ./docs/index.html. You can customize the location of that symlink using --output-link flag, like this:

dhall-docs --input . --output-link ${OTHER_LINK}

For more information about the tool, check the --help flag.

Documenting your packages

Documenting your package is essentially writing comments on your source code, although there are some format rules that these need to obey to work properly with dhall-docs. These rules aid dhall-docs in extracting the text on the documentation that will be passed to our Markdown preprocessor to finally render your documentation in HTML.

You can see examples of writing documentation on the tests folder.

In every example we used ␣ as an alias to a whitespace character

Documentation markup language

The markup language of the documentation is CommonMark, which is a strict Markdown flavor. dhall-docs uses mmark, a package that parses CommonMark and transforms it to HTML. This package follows (almost) that specification.

Block comments

Normal block comments in Dhall starts with {- and ends with -}:

{- foo
     bar
       baz
-}

A dhall-docs block comment needs to start with "{-|" and a newline. The newline can be either \n or \r\n. This means that the actual documentation must starts on the following line.

CommonMark supports some features that are sensitive to indentation (e.g. indented-code-blocks). Indentation on dhall-docs block comments is roughly the same to Dhall multi-line strings: the indentation is determined by the longest common whitespace prefix between the lines on the block comment.

In all the following examples:

{-|
foo
  bar
    baz
-}
  {-|
foo
  bar
    baz
-}
{-|
    foo
      bar
        baz
    -}
{-|
    foo
      bar
        baz-}

... dhall-docs will extract the contents of the comment, stripping all the common leading whitespace from all lines, returning the following text that will be passed to our CommonMark parser:

foo
  bar
    baz

Similar to Dhall multi-line string literals, the -} position also affects the final indentation, meaning that in the following sample:

{-|
    foo
    bar
-}

The following text will be extracted:

    foo
    bar

Only spaces and tabs are taken into account in calculating the longest common whitespace prefix. Other forms of whitespaces are not considered.

Empty lines are ignored when determining the indentation level but are preserved on the extracted text. On the following example:

{-|
  foo

  bar
  -}

dhall-docs will extract the following text:

foo

bar

Note that a line that only contains trailing whitespace will be taken into account when calculating the common indentation. The line between foo and bar on the following example has leading whitespace (two (2) whitespaces):

␣␣{-|
␣␣␣␣foo
␣␣
␣␣␣␣bar -}

meaning that the extracted text will be:

  foo

  bar

If there is no newline between the opening brace of a dhall-docs block comment (i.e. {-|), that comment will be considered invalid: dhall-docs will not render it on the documentation and a warning will be logged on the console:

  {-| foo
      bar -}

... although it's a valid Dhall block comment, it's invalid for dhall-docs.

To end, if the | character isn't immediately after the opening brace, dhall-docs will ignore the comment without logging on the console. This lets you separate your internal block comments from the ones that you want dhall-docs to output in the final markup.

This is an example of a file documented using block comments:

{-|
# My awesome package

This is the header of my package

* a list item in **bold**
  - a sub item in _italic_
  - a [link](https://example.com)
    * a `code` quote


A new paragraph

    an indented code block
-}
let myAwesomeFunction {- this will be ignored on the documentation -}
               = \(x : Natural) -> x + 1

in
{
  {-|
  foo
      bar
          baz
  -}
  myAwesomeFunction
}

Line comments

Normal Dhall single-line comments start with -- and ends at a newline. For example:

-- single-line comment
0

A dhall-docs valid single-line comment starts with --| and a single whitespace character. For example:

--|␣foo

You can span your documentation in several single-line comments by writing -- and two (2) whitespaces:

--|␣foo
--␣␣bar
--␣␣␣␣␣␣baz

The -- prefix defines the base indentation for any line. That is, on the above example dhall-docs will extract the following contents:

foo
bar
␣␣␣␣baz

If you would like to force an empty line on the documentation output, place an empty single line comment. On this example:

--| foo
--
--  bar

dhall-docs will extract:

foo

bar

The empty -- line should not have the two (2) whitespaces.

If between two single-line comments there is one (1) or more empty lines, the latter comments will be ignored. On this example:

--| foo
--  bar

--  baz

dhall-docs will only capture this text:

foo
bar

In a set of subsequent single-line comments, all lines before the one that has the | character will be ignored, meaning that if any line on the set doesn't have the | marker, no text will be extracted for the generated documentation.

On this example:

--  foo
--  bar
--| baz
--  qux

the following text will be extracted:

baz
qux

and this set of single-line comments:

--  foo
--  bar
--  baz

will be ignored by the tool.

The set of -- lines needs to be aligned with the same number of preceding characters, meaning that a set of lines like this:

--| foo
  --  bar
 --  baz

is invalid: dhall-docs will ignore this on the generated documentation and a warning will be logged on the console.

To end, if there are language tokens in-between a set of lines, the lines after those tokens will be ignored. On this example:

--| foo
,
--  bar

the extracted text will be the following:

foo

Here is an example of using this type of comments in a Dhall file:

--| # My header
--
--  * 1
--  * 2

let --| foo
    --  bar
    a = True

in { a }

Mixing the two type of comments

In a single Dhall file, you can use any type of comments to document your code, but there is a restriction.

Two subsequent dhall-docs comments (whether single-line or block comments) are forbidden: dhall-docs will reject them and log a warning on the console, but you can freely place any non-dhall comment after a dhall-docs comment. All of the following examples are invalid:

{-|
  foo -}
{-|
  bar -}
{-|
  foo -}
--| bar
{-|
  foo -}
{- -}
{-|
  qux -}
--| foo
--  bar
{-|
  qux -}
--| foo
--  bar
{- -}
--| qux
--  baz

Note that non dhall-docs comments are ignored in the above examples.

Validating the extracted text

After extracting the text on a valid dhall-docs comment, it will be passed to mmark. If there is a CommonMark parse error, an error will be logged on the console but the extracted text will be rendered in the documentation exactly as it was extracted.

Supported annotated elements

A limitation of the dhall parser is that it doesn't preserve all the whitespaces of a Dhall source file, but preserves the enough to write useful documentation. dhall-docs supports the following comments:

  • The header of a file:

    {-| this is the header
-}
let a = 1
in  a

Development

ghci

If you want to open the ghci repl on this package using stack, you have to provide an additional flag:

stack ghci dhall-docs --flag dhall-docs:ghci-data-files

... otherwise the CSS and JS files won't be properly embedded

Running tests

We have a golden-test setup to test the whole tool. If you add a new test file or introduced a change that changed the way the documentation is generated, do the following:

  1. On tasty/Main.hs search for the usages of Silver.defaultMain and replace them with Test.Tasty.defaultMain
  2. Execute the test runner with the --accept flag. In stack you could do: stack test dhall-docs:tasty --test-arguments --accept
  3. After updating the golden files, revert the change on (1)

Generated docs on Hydra

Our hydra job sets builds documentation for the following test packages:

If you're in a PR, you can see the generated documentation by navigating to:

https://hydra.dhall-lang.org/job/dhall-haskell/${PR_NUMBER}/${PACKAGE_NAME}/latest/download/1/docs

... where ${PACKAGE_NAME} is one of the following:

  • prelude-dhall-docs if you want to see Dhall's Prelude documentation
  • kubernetes-dhall-docs if you want to see the dhall-kubernetes documentation package
  • test-dhall-docs if you want to see the test demo package

If you want to see the latest generated docs on the master branch, visit:

https://hydra.dhall-lang.org/job/dhall-haskell/master/${PACKAGE_NAME}/latest/download/1/docs