Organize --help text

[mattias-p]

Purpose

Make the --help text more focused and organized.

Context

• Because much of the documentation for CLI lives in the zonemaster/zonemaster repository, there's a dual PR in that repository. Update CLI documentation zonemaster#1309.
• This is a followup to Make usage documentation consistent and also demoose #371 and includes those commits. It also includes the commits of Make --stop-level disabled by default again #395 which fixes a bug introduced in the former.
• I've tried to keep this PR focused on structure and make as few changes as possible to the actual language. I have a lot of improvements in mind for that as well, but I'm saving those for one or more followup PRs for the next release.

Changes

• Group options for easier navigation.
• Limit the contents and clutter of the --help output, but make sure to point to the man page.
• Include list of severity levels in --help output again (after removal in Make usage documentation consistent and also demoose #371).
• Update default value declarations in --help output.
• Make default value declarations clearer and add missing ones.

How to test this PR

Run zonemaster-cli --help and man zonemaster-cli and make sure they're visually alright.

[mattias-p]

I've rebased this onto develop. Please re-review.

[tgreenx]

LGTM.
Btw, do you know of a way to display the full man page without compiling the program in the first place? I tried with perldoc and pod2man to no avail.

[mattias-p]

Btw, do you know of a way to display the full man page without compiling the program in the first place?

Yeah, you can run: perldoc -oman script/zonemaster-cli

This should probably be documented somewhere, I'm just not sure where. I guess I could add it to the README.md#Documentation, but maybe there are better options?

[matsduf]

Yeah, you can run: perldoc -oman script/zonemaster-cli

This should probably be documented somewhere, I'm just not sure where. I guess I could add it to the README.md#Documentation, but maybe there are better options?

I think https://github.com/zonemaster/zonemaster/blob/master/docs/public/using/cli.md should be the right place. That file should be updated anyway since it states

To see all available command line options, use the --help command.

which is not true anymore.

[matsduf]

* Include list of severity levels in --help output again (after removal in [Make usage documentation consistent and also demoose #371](https://github.com/zonemaster/zonemaster-cli/pull/371)).

For zonemaster-cli --help the severity levels is relevant for --level=LEVEL. I think that e.g. the following would be an improvement:

  Testing Options:
(...)
    --level=LEVEL
        Specify the minimum level (see levels below) of a message to be printed. (default:
        NOTICE)
(...)
  Severity levels from highest to lowest:
    CRITICAL, ERROR, WARNING, NOTICE, INFO, DEBUG, DEBUG2, DEBUG3

[matsduf]

https://github.com/zonemaster/zonemaster/blob/master/docs/public/using/cli.md needs an update.

[matsduf]

Does this PR or #371 change the actual behavior of --stop-level=LEVEL? As it is with this update it will break in the middle of the test case which is very unlucky for the development of test scenarios and test zones.

The behavior on a server without these PRs:

$ zonemaster-cli --level info --test basic02 --show-testcase mail.protection.outlook.com --raw
   0.00 INFO     Unspecified    GLOBAL_VERSION  version=v6.0.0
   4.54 CRITICAL Basic02        B02_NO_WORKING_NS  domain="mail.protection.outlook.com"
   4.54 ERROR    Basic02        B02_UNEXPECTED_RCODE  ns=ns2-proddns.glbdns.protection.outlook.com/104.47.72.81; rcode=FORMERR
   4.54 ERROR    Basic02        B02_UNEXPECTED_RCODE  ns=ns1-proddns.glbdns.protection.outlook.com/104.47.34.17; rcode=FORMERR

The behavior with these PRs:

$ zonemaster-cli --level info --test basic02 --show-testcase mail.protection.outlook.com --raw
   0.00 INFO     Unspecified    GLOBAL_VERSION  version=v6.0.0
   4.71 CRITICAL Basic02        B02_NO_WORKING_NS  domain="mail.protection.outlook.com"
Exited early: Saw message at level CRITICAL

I really dislike the new behavior. It should be possible to turn it off again, e.g. with --no-stop-level.

[matsduf]

See my comment.

[mattias-p]

Yeah, you can run: perldoc -oman script/zonemaster-cli
This should probably be documented somewhere, I'm just not sure where. I guess I could add it to the README.md#Documentation, but maybe there are better options?

I think https://github.com/zonemaster/zonemaster/blob/master/docs/public/using/cli.md should be the right place.

End users should be able to run man zonemaster-cli, right? AFAIK the perldoc -oman command is only needed in development contexts where you haven't installed the man page. We should have somewhere to put developer instructions. I'll make a suggestion an include it in this PR.

That file should be updated anyway since it states

To see all available command line options, use the --help command.

which is not true anymore.

Good spotting! I would include an update in this PR but I guess I'll have make another one for it.

[matsduf]

This is how a previous version of zonemaster-cli breaks testing, i.e. after the actual test case:

zonemaster-cli --level info --show-testcase mail.protection.outlook.com --raw
   0.00 INFO     Unspecified    GLOBAL_VERSION  version=v6.0.0
  10.17 INFO     Basic01        B01_PARENT_FOUND  domain=protection.outlook.com; ns_list=ns1-35.azure-dns.com/150.171.10.35;ns1-35.azure-dns.com/2603:1061:0:10::23;ns2-35.azure-dns.net/150.171.16.35;ns2-35.azure-dns.net/2620:1ec:8ec:10::23;ns3-35.azure-dns.org/13.107.222.35;ns3-35.azure-dns.org/2a01:111:4000:10::23;ns4-35.azure-dns.info/13.107.206.35;ns4-35.azure-dns.info/2620:1ec:bda:10::23
  10.17 INFO     Basic01        B01_CHILD_FOUND  domain=mail.protection.outlook.com
  12.45 CRITICAL Basic02        B02_NO_WORKING_NS  domain="mail.protection.outlook.com"
  12.45 ERROR    Basic02        B02_UNEXPECTED_RCODE  ns=ns1-proddns.glbdns.protection.outlook.com/104.47.34.49; rcode=FORMERR
  12.45 ERROR    Basic02        B02_UNEXPECTED_RCODE  ns=ns2-proddns.glbdns.protection.outlook.com/104.47.34.17; rcode=FORMERR
  12.51 CRITICAL Unspecified    CANNOT_CONTINUE  domain=mail.protection.outlook.com

[mattias-p]

I added a commit with documentation on how to view a development version of the man page. I put it in a new file called DEVELOPMENT.md. At the moment this is the only instruction included in it, but my idea is that in time we'll come up with more things to add.

I'll have a look at the --stop-level problem.

[matsduf]

I added a commit with documentation on how to view a development version of the man page. I put it in a new file called DEVELOPMENT.md. At the moment this is the only instruction included in it, but my idea is that in time we'll come up with more things to add.

Should we really add documents in this repository? Shouldn't that be in the documentation tree?

If added here, the MANIFEST file must be updated.

[mattias-p]

@matsduf I've can't reproduce your --stop-level problem. I tried running the following command on both the current develop branch and this feature branch, and I get completely identical results.

perl -Ilib script/zonemaster-cli --level=info --no-time --show-testcase --raw --no-ipv6 mail.protection.outlook.com

[mattias-p]

@matsduf It seems the problem you found was introduced in #371 which is already merged, so the problem is in the develop branch and not in this PR.

[mattias-p]

As we discussed in the work group meeting the development documentation should not be located here but rather in the zonemaster/zonemaster repository. I've updated this PR to reflect this.

[mattias-p]

I rebased onto latest develop.