LiquiDoc CMF

This repo contains a LiquiDoc quickstart application: the bootstrapping files and instructions for a content management framework (CMF) designed for documenting software products. Combining open-source tools with collaborative workflows, LiquiDoc CMF facilitates writing, editing, coordinating, and deploying technical docs in fully extensible, Git-managed codebases.

LiquiDoc CMF (LDCMF) incorporates two excellent lightweight markup languages (AsciiDoc and YAML) with open-source build and publish utilities (Asciidoctor, and Jekyll). All this is stitched together seamlessly with Liquid templating markup and coordinated by a docs build utility called LiquiDoc. This creates a truly open, highly portable and scalable system for developing docs code the way (and where) programmers develop product code.

Table of Contents

• Usage
  • Establish the Environment
  • Build the Docs
  • Wrapup
• Structure
• Build Config
  • LDCMF Build Strategies
  • Recommended Best Practices
• Contributing

To learn much more about LiquiDoc CMF and its technology ecosystem (the “AJYL docstack”), check out this introduction. The remainder of this document assumes you wish to implement the LDCMF framework for a project of your own. It instructs the bootstrapping of an appropriate LDCMF environment.

Usage

Important

More documentation for LDCMF administration is available here.

The following procedure is a bootstrapping operation to get an actual LDCMF-based project up and running. It inflates directories and sets up some initial demonstration files based on preferences you will customize. More thorough setup instructions are available and highly recommended.

Warning

Metamorphosis Warning The file you are reading (README.adoc) is the README for the generic LiquiDoc CMF project bootstrap repository. The procedure it instructs will convert its current generic bootstrap context into your own LDCMF project application. If you wish to keep this file, move/rename it before running the bootstrapping init, since that operation generates a proper README.adoc for your actual docs project, which this directory and its contents will then be.

If you have Git (for Windows, use with GitBash or a Linux emulator like Cygwin) and Ruby runtime installed, you can get going with LiquiDoc CMF using this guide.

Establish the Environment

1. Navigate to the parent directory of your new docs directory.

   Example for new project

   cd Documents/workspace

   Example for docs inside a product project

   cd Documents/workspace/my-product

2. Clone this repo.

   Example

   git clone git@github.com:DocOps/liquidoc-cmf.git docs_dir

   Replace docs_dir with the directory name you want to give this directory — either a subdirectory of a parent project or the root directory of a new, independent repo.

3. Change to the project directory.

   Example

   cd docs_dir

4. Re-associate your files with the proper repository.

   1. If your docs project is its own Git repo, change the boilerplate project’s remote origin Git repository to your own.

      Example

      git remote set-url origin git://github.com:your-project/your-repo-name.git

      Replace this URI with your own.

   2. If your docs project is part of a parent directory’s Git repo, remove local Git files (for the LDCMF repo you just cloned) so your new docs source will be absorbed into the parent repo.

      Example

      rm -rf .git

      Now your new directory/files will appear as uncommitted additions to the parent project (git status).

5. Run Bundler to install dependencies.

   bundle

   If an error indicates Bundler is not installed, run gem install bundler, then repeat this command.

   Tip

   This process generates a Gemfile.lock file, which you will want to track in Git. Whenever your Liquidoc or your application require different versions of such upstream gems, maintain Gemfile.lock in source control as a means of ensuring all users develop with the settings that will be passed on to production.

6. Rename this README so the upcoming init operation does not overwrite it.

   Example

   mv README.adoc README.meta.adoc

   Note	This file may confuse future users; we advise removing it before sharing your LDCMF project.

7. Edit the metafile for your new LDCMF project.

   In your favorite text/code editor, open data/meta.yml and fill out the required info, then save the file.

8. Initialize your custom environment.

   1. Generate an initialization config based on the custom preferences in meta.yml.

      bundle exec liquidoc --template _init/templates/init.yaml --data data/meta.yml --output _init/init.yml

      This command performs a one-off template parsing to establish the temporary init config file, responding with a message that your _init/init.yml was built.

   2. Generate infrastructure files using your temporary init config.

      bundle exec liquidoc -c _init/init.yml

      This procedure builds starter files (and inflates directories). If your initialization steps proceeded without errors, you now have a codebase that serves as the skeleton for an LDCMF implementation, starting with a _configs/ directory containing LiquiDoc, Asciidoctor, and Jekyll config files, as well as new README.adoc and content/index.adoc files. You can make changes to data/meta.yml and re-run the liquidoc commands until you approve of the results.

Build the Docs

1. You can now review the new README.adoc file generated by this procedure.
   The new README file is oriented toward your LDCMF application’s future users — starting with you.

2. Perform the first build of your seedling docset, as instructed in the new README.
   Check the target directory for built artifacts and review them for errors.

Tip

You should edit and maintain the generated README as the canonical reference for getting started with your docs codebase. As you introduce complexity, the README should help simplify the getting-started process. Independent documentation of your LDCMF application can be derived from forking the LDCMF Guides repository.

Wrapup

1. When you are certain everything looks good, delete the _init/ directory.

2. Move or remove README.meta.adoc (likely this file!).
   Everything important from this file is available in the LDCMF Guides.

Tip	If you are not happy with the output, you may further edit meta.yml and continue running the bundle exec liquidoc -c _init/init.yml command to overwrite the generated files.

Structure

Here is the basic directory structure of an initial LiquiDoc CMF project once the first build procedure has been carried out.

├── _build/
│   └── pre/
│       ├── _attributes.yml
│       ├── <parsed-config>.yml
│       └── config-explainer.adoc
├── _configs/
│   ├── asciidoctor.yml
│   ├── build-docs.yml
│   └── jekyll-global.yml
├── _templates/
│   └── liquid/
├── content
│   ├── assets/
│   │   └── images/
│   ├── pages/
│   ├── snippets/
│   └── topics/
├── data/
│   ├── meta.yml
│   ├── products.yml
│   └── schema.yml
├── theme/
│   ├── css/
│   ├── fonts/
│   ├── <custom-theme>/
│   │   ├── _includes/
│   │   └── _layouts/
│   ├── js/
│   └── pdf-theme.yml
├── Gemfile
├── Gemfile.lock
├── LICENSE
└── README.adoc

_build/

This is where all processed files end up, including all migrated assets, prebuilt source, or final artifacts. This directory is not tracked in source control, so you will not see it until you run a build routine, and you cannot commit changes made to it. It is always safe to delete this directory in your local workspace. We will explore the _build/ directory more fully later.

_build/pre/

Where system-arsed files are built and ephemerally stored.

_build/pre/_attributes.yml

Collected AsciiDoc attributes for loading.

_build/pre/<parsed-config>.yml

A copy of the config file after Liquid parsing.

_build/pre/config-explainer.adoc

Source for configuration explainer logging current build.

_configs/

This folder is where the brains go. The build-docs.yml config file belongs here, as does anything that is more about programming the build procedure than about informing the content. The asciidoctor.yml file is for non-content AsciiDoc attributes that pertain to the structure or process of rendering with Asciidoctor. This is also the home of various Jekyll configuration files, usually one for each guide and one for each guide type (e.g., attributes-portal.yml and attributes-manual.yml).

_ops/

This is a secondary “configs” location, for utilities and routines that support the use of LDCMF by admins and documentarians. For instance, the init-topic.yml config coordinates the automated creation of topic files and manifest entries.

_templates/liquid/

Here we store most of our prebuilding templates. These are not Jekyll theming templates. These are the ones we use for generating new YAML and AsciiDoc source files from other source files and external data.

content/

The first of our publishable directories, content/ is the base path for documentarians' main work area. Everything inside the content/ directory will be copied into the _build/ directory early in the build process.

content/assets/

For content assets, rather than theming assets. If it illustrates your product, it probably goes here. If it brands your company, it probably goes in theme/assets/.

content/pages/

For AsciiDoc files of the page content type.

content/snippets/

For content snippets.

content/topics/

For AsciiDoc files of the core topic content type.

data/

All YAML small-data files that contain content-relevant information go here. These data files differ from those that belong in _configs/ (or _ops/) in important ways, essentially revolving around whether the data needs to be available for display. If it is not establishing settings or used to inform non-build functions (like in _ops/), the data file probably belongs in data/. Let’s look at some key data files standard to LDCMF.

data/meta.yml

For general information about your company, URL and path info. This file usually contains just simple data: a big (or small) column of basic key-value pairs to create simple variables.

data/products.yrml

For subdivided information about your products in distinct blocks. Each block can be called for selective ingest during build routines using the colon signifier, such as by calling data/products.yml:product-1, where product-1: is a top-level block in the products.yml file.

data/guides.yml

This block is for content-oriented data that is distinct between the different portals or guides you’re producing. This is often redundant to your products.yml file, if product editions themselves are the major point of divergence in your docs, and it is formatted the same way. For this project (LDCMF Guides), the guides are oriented toward audiences (documentarians, admins, and developers), but the products (LiquiDoc and LDCMF) are distinct from this and actually documented/instructed together in each guide.

Tip	Favoring the filename products.yml is conventional when products and guides (portals) have a 1:1 relationship and guides.yml file is superfluous.

data/manifest.yml

This crucial file provides a central manifest of all page-forming content items (pages, topics). It dictates how they are organized in the site navigation (using metadata such as categories into which content items fall). The schema file carries essential build info that lets us see relationships between topics and build content-exclusive portals from otherwise-decontextualized repositories. Sometimes data/schema.yml, deprecated in favor of manifest.yml.

data/terms.yml

By no means a required file, terms.yml is a great example of a file that is really just for content. You can have as many of these key-value files, serving whatever purposes you wish.

products/

This is an optional path for LDCMF projects. If you plan to embed your product repos as submodules, put them in the products/ directory. Submodules are like shortcuts to those embedded repositories — if you navigate into them, you can perform Git operations. Submodules are typically used to ensure product repos are reliably available and properly checked out, generally in order to support migrating files into the build directory.

theme/

All the files that structure your output displays go here. This mainly includes Jekyll templates (theme/<theme-name>/_includes/ and theme/<theme-name>/_layouts/) and asset files such as stylesheets, front-end javascripts, and of course theme-related images. This would also be the home of PDF and slideshow output theme configurations, as applicable.

theme/pdf-theme.yml

A very basic PDF theming file based on Asciidoctor’s default-theme.yml, just to get you started. For more about PDF theming, see the Asciidoctor PDF Theming Guide.

Build Config

The _configs/build-docs.yml file is the brains of any LDCMF application. It defines the sequential compilation routine and ensures all assets are in place for the final artifact rendering operations.

LDCMF Build Strategies

The strength of LiquiDoc CMF its ability to maintain strictly “DRY” single sourcing while still producing diverse output. These strategic principles are key to maintaining this capability.

Store small data in flat files.

When we talk about product metadata, we are referring to information about a software product, not any kind of data stored by that product. Consider what data matters about products: capacities, dependencies, options, integrations, and anything classifiable as metadata, including information about the product developer. Then consider how these things change as products evolve — every version has its own array of the above attributes, and the list can only be expected to grow and morph. Data of this kind is — small data — is not best stored in relational databases; version control is essential and schemas get in the way, especially since none of the data is served live. If you want to keep your product info in Git, use appropriate flat-file formats for various representations. The human-friendliest formats are probably YAML and CSV. YAML can be edited in any decent code editor, and a comma-separated values file can be edited in any spreadsheet application.

Prebuild and include complex reference content.

All that small data needs to make it into your docs in a more human-readable format. This is where prebuilding reference content to AsciiDoc source comes in. Use Liquid templates to generate includable AsciiDoc files into the _build/snippets/ directory. Then include them into your static AsciiDoc files.

Handle major parallel divergence by splitting output into “guides”.

When there are major points of divergence in output requirements — such as significantly different “editions” of the same product or highly variant audiences, like basic vs advanced users or consumers vs developers — each splinter necessitates its own guide. In these cases, you want to direct users to the appropriate docset, as well as make it easy for those who land in the wrong guide to switch to a similar place in the parallel guide. Guides are built sequentially, each drawing configuration settings and content designated for it, along with content and data common to other guides. This process generates parallel guides, including Web portals that are built side by side and served as components of one site.

Handle output-format diversity with “portals” and “manuals”.

A lot of the conflict in documentation output stems from the manuals vs portals debate. Modern websites tend to work best by presenting content in semi-serialized or unserialized article format, more like Wikipedia than a book. Meanwhile, technical documentation is often still intended to be consumed more like a book or a traditional manual. LDCMF tries to balance both without requiring either, but each final rendering action is technically building either a book-style manual or a help-site/wiki-style portal.

Note	Coming Soon JavaScript-driven slide presentations!

Coordinate docs to product versions (sequential divergence) using Git branches and tags

If the product your LDCMF application covers is released in consecutive versions, with more than one supported at any given time, chances are the product developers maintain multiple branched and tagged versions of the product. They further branch from these branches when modifying a particular version of the code, either post-release as a patch or pre-release. LDCMF docs can be organized the same way.

Tip	Use LiquiDoc’s shell-command execution to perform operations such as git checkout to choose branches/tags from which to build a version of the docs (perhaps coordinated with a version of canonical product data).

Recommended Best Practices

Maintain no built files in source control.

If a file is the product of data from other source files, generate that file at build time, and do not commit it to source. This means keeping an ignored build directory (conventionally _build/), and everything outside that path should be unique.

Note	Exceptions to this rule include init and ops routines, configured to instruct LiquiDoc to generate useful files. The rule pertains to content files generated at build time, not files manually generated during setup or while creating new content.

Keep functional code out of AsciiDoc source

Perform heavy processing up front as prebuilding, then include those prebuilt files during render phases. While jekyll-asciidoc enables Liquid preprocessing in AsciiDoc files, LDCMF prefers prebuilding so the generated files can service more than Jekyll builds.

Maintain Gemfile.lock in the repository.

Whenever LiquiDoc or your application require different versions of such upstream gems, you run bundle update (possibly after editing your local Gemfile) maintain Gemfile.lock in source control as a means of ensuring all users develop with the same gems under the hood.

Share product data with the product.

Ideally, that small data stored in flat files should be kept in one canonical place, from which it is read by the docs and the product build routines. This approach ensures docs references reflect the current truth about the product, zero intervention required.

Contributing

This is an open source project that is eager for contributions and feedback. More soon.