Skip to content

Latest commit

 

History

History
401 lines (341 loc) · 13.1 KB

Documentation_about_guidelines.md

File metadata and controls

401 lines (341 loc) · 13.1 KB

Documentation about guidelines

Guidelines for describing workflows

  • Make no assumptions on the user's knowledge
    • Nothing is obvious to somebody who doesn't know
  • Add ways to verify if a described process worked
    • E.g., "do this and that, if this and that is correct should see this"
  • Have a trouble-shooting procedure
    • One approach is to always start from scratch

Markdown vs Google Docs

In general

  • We prefer to use Markdown for technical documentation
  • We use Google for notes from meetings and research

Markdown pros

  • Can use vim
  • Can version control
  • Easy to use verbatim (e.g., typing foobar)
  • Easy to style using pandoc
  • Easy to embed code
  • Easy to add Latex equations
  • Easy to grep

Google Docs pros

Rules of thumb

  • Use Markdown
    • If doc is going to be used as a public guideline
    • If doc has mostly text, code, and formulas
    • If there are notes from a book
  • Use Gdoc
    • If doc requires a lot of images that cannot be placed as text
    • If doc is a research of an analysis

Useful references

Style and cosmetic lints

Always use markdown linter

  • Most cosmetic lints described further can be taken care automatically by our markdown linter, so make sure to run it after implementing the changes
  • The file is dev_scripts/lint_md.sh, see the docstrings for more details
    • Example run: 
      dev_scripts/lint_md.sh docs/Documentation_about_guidelines.md
  • Do not mix manual edits and linter runs. Best practice is to run the linter and commit the changes it made as separate commit
  • If the linter messes up the text, file an issue with examples of what the linter does incorrectly

Table of content (TOC)

  • Unfortunately both markdown and GitHub don't support automatically generating a TOC for a document
  • To generate a table of content:
    • Add the following tag at the top of the markdown file below the document title: 
      <!-- toc -->
    • Run the markdown linter in order to build TOC automatically

Use nice 80 columns formatting for txt files

  • Vim has a :gq command to reflow the comments
  • There are plugins to take care of this for PyCharm
  • Our markdown linter takes care of reflowing the comments as well

Empty line after heading

  • Leave an empty line after a heading to make it more visible

    • Bad

      # Coming through! I've big important things to do!
- ... and his big important wheels got STUCK!

    • Good

      # Very important title

- Less important text

Bullet lists

  • We like using bullet list since one can represent thought process more clearly, e.g.,
    • This is thought #1
      • This is related to thought #1
    • This is thought #2
      • Well, that was cool!
      • But this is even better
  • We strictly use - instead of *, circles, etc.:
    • Bad 
      * Foo bar!
  * hello
  * world
* Baz
    • Good 
      - Foo bar!
  - hello
  - world
- Baz

Using code style

  • We use code style for
    • Code
    • Dirs (e.g., /home/users)
    • Command lines (e.g., git push)
  • When using a block of code use the write syntax highlighting
    • Bash 
      > git push
    • Python 
      if __name__ == "__main__":
    predict_the_future()
    print("done!")

Indenting code style

  • GitHub / pandoc seems to render incorrectly a code block unless it's indented over the previous line
    • Bad
    > git push
    • Good 
      > git push

Embedding screenshots

  • Avoid to use screenshots whenever possible!
  • However, sometimes we need it (e.g., plot infographics, website inteface, etc.)
  • To do it correctly:
    • Place your screenshot in any comment window at GitHub
      • screenshot
      • This will upload the image to the GitHub cloud
      • You DO NOT have to publish a comment, the provided link is already ready to use!
    • Make sure your link has no not-English symbols in alt section
      • symbols
      • They sometimes appear if your native PC language is not English
      • You can avoid it by giving the picture a name in English
      • Alternatively, you can just edit the alt section in the generated link - this will not corrupt the file
    • Place the generated and edited link to the markdown file
      • last

Improve your written English

  • Use English spell-checker
    • Unfortunately this is not enough
  • Type somewhere where you can use several free choices:
  • This is super-useful to improve your English since you see the error and the correction
    • Otherwise you will keep making the same mistakes forever

Make sure your markdown looks good

  • You can:
    • Check in a branch and use GitHub to render it
    • Use Pycharm to edit, which also renders it side-by-side
    • Compare your markdown with already published documentation

Google docs style conventions

Headings

  • We add N (where N is the heading level) # before the heading name, e.g.,
    • Heading 1: 
      # Heading 1
    • Heading 2: 
      ## Heading 2
  • The reason is that sometimes one doesn't have the time or the patience to format things properly, so at least there is some indication of the level of the titles
  • Avoid having multiple # separatd by a space that sometimes appear in a process of convertion of Gdocs to Markdown files
    • Bad 
      # # Heading 1
    • Good 
      # Heading 1

Font

  • Normal text:
    • Font: Arial
    • Font size: 11
  • Headings:
    • Font: Arial
    • Style: bold
    • Font size: should be adjusted automatically when one converts “Normal text” to “Heading N”, e.g., when converting some text of size 11 to “Heading 1” the font sizes becomes 20

Convert between Gdocs and Markdown

Gdocs -> Markdown

Using convert_docx_to_markdown.py

  • In general, we recommend using this approach

  • This python script converts Docx to Markdown using Pandoc.

  • Pros

    • Removing artifacts with this python script, less manual work
    • Best for a large document
    • Handle figures

  • Cons

    • Need to move files

Process

  • Download Google document as docx
  • Convert it to markdown using convert_docx_to_markdown.py
  • Usage: 
    > ../dev_scripts/convert_docx_to_markdown.py --docx_file Tools_Docker.docx --md_file Tools_Docker.md
    • This command should be run directly under the target output directory for the Markdown file, in order to generate correct image links. Otherwise, you'll need to manually fix the image links.
    • File names can't contain any spaces. Therefore, use underscores _ to replace any spaces.

Cleaning up converted markdown

  • Fix some formatting manually before running the Markdown linter.

    • Read through Style and cosmetic lints for Markdown formatting and fix some formatting based on the rules.
      • Summary
        • Add the following tag at the top of the markdown file below the document title: 
          <!-- toc -->
        • Use bullet lists to organize the whole Markdown for consistency with other docs. See Coding_Style_Guide.md or any other published Markdown format as reference
        • Add missing ``` around code blocks. These could be missing in the original Google doc. Also adjust code block indentations if needed

  • Remove empty lines manually

    :'<,'>! perl -ne 'print if /\S/'

  • Run the lint_md.sh

    • Usage: 
      > dev_scripts/lint_md.sh docs/Documentation_about_guidelines.md
    • What the linter will do:
      • Build TOC automatically
      • Adjust the indentation to improve the Markdown's format (but the precondition is that you have properly adjusted the indentation levels).
      • Remove extra empty lines under headings
      • Adjust text layout
    • Do not mix manual edits and linter runs
    • If the linter messes up the text
      • File bugs in amp with examples what the linter does incorrectly

  • Last steps

    • Compare the generated markdown file with the original Gdoc from top to bottom to ensure accurate rendering.
    • Review the markdown file on GitHub to make sure it looks good, as it may slightly differ from the preview in your local markdown editor

  • When a gdoc becomes obsolete or it's deleted

    • Add a note at the top of a gdoc explaining what happened
    • Example: "Moved to /new_markdown_file.md"
    • Strike out the entire document
    • Move the gdoc to the _OLD directory

Other approaches

  • Best for a large document

  • Approach 1 - Chrome Docs to Markdown extension:

    • Use the Docs to Markdown extension
    • One needs to accept/reject all suggestions in a gdoc as the extension works poorly when a document is edited in the suggestion mode

  • Approach 2 - Online converter:

  • Also need to go through Cleaning up converted markdown

  • You might need to remove artifacts manually

Markdown -> Gdocs

  • Approach 1:
    • Run 
      > pandoc MyFile.md -f markdown -t odt -s -o MyFile.odt
    • Download the template in odt format
    • Run 
      > pandoc code_organization.md -f markdown -t odt -s -o code_org.odt --reference-doc /Users/saggese/Downloads/Gdoc\ -\ Template.odt
    • Open it with TextEdit, copy-paste to Gdoc
  • Approach 2:
    • Instead of copy-paste the markdown into Gdocs, you can copy the rendered markdown in a Gdoc
      • Gdocs does a good job of maintaining the formatting, levels of the headers, the links, and so on
  • Approach 3: