How to create a SAF CLI

This page provides the methodology of creating a SAF Command Line Interface (CLI). It provides the framework used for all future SAF CLI development.

Setup the development environment

• Create a new branch from the main in the SAF GitHub repository
• Clone the newly created branch git clone -b <new_branch_name> git@github.com:mitre/saf.git
• Install the necessary dependencies either via npm or brew - npm install or brew install
  • Execute the command from the root directory where the branch was cloned locally

SAF CLI directory structure

The SAF source code directory structure is comprised of multiple directories, each containing specific content (code, scripts, documents, tests, etc.)

• saf - this is the root directory
• .git - contains all the information that is necessary for the project in version control and all the information about commits, remote repository, etc
• .github - used to place GitHub related stuff inside it such workflows, formatting, etc
• .vscode - holds the VS Code editor configuration content
• bin - contains the runtime commands for node.js
• docs - contains eMASSer documentation
• lib - contains the compiled JavaScript files. This folder is created by the TypeScript Compiler (tsc) command. On the SAF CLI this command is scripted as npm run prepack command which will execute based on what OS it is running (win or mac)
• node_modules - contains all of the application supporting resources, created when the npn install command is executed
• src - this folder contains all of the SAF CLI commands, it is organized by capabilities
• test - contains all of the automated tests used to verify available capabilities

New SAF CLI Command

Any new SAF CLI command(s) should be added to the src -> commands directory inside a directory that indicates what the command is to accomplish. For example, if we are to add commands that connects to other systems like tenable.sc or splunk, we could create a sub-folder inside the src -> commands folder and call it interfaces, or a single directory for each interface, like tenable and splunk

Example:
  src/          or           src/          or          src/
  └── commands/              └── commands/             └── commands/
     └── tenable/                └── splunk/               └── interfaces/
         └── tenable.ts              └── splunk.ts             ├── tenable.ts 
                                                               └── splunk.ts

The objective is to keep the like commands grouped together.

OCLIF Configuration

The oclif behavior is configured inside the SAF CLI package.json under the oclif section. If the CLI being created does not belong to one of the available topics (oclif -> topics) a new topic needs to be added to the oclif section. See Topics for more information on how to.

SAF CLI Template

The following code can be used as a starter template

import path from 'path'
import {Flags} from '@oclif/core';
import {BaseCommand} from '../../utils/oclif/baseCommand'

export default class MYCLI extends BaseCommand<typeof MYCLI> {
  // Note: If the variable `usage` is not provided the default is used 
  // <%= command.id %> resolves to the command name
  static readonly usage = '<%= command.id %> -i <ckl-xml> -o <hdf-scan-results-json> [-r]'
    
  static readonly description = 'Describe what the CLI does - short and to the point'

  // Note: <%= config.bin %> resolves to the executable name (i.e., saf, emasser)
  static readonly examples = [
    '<%= config.bin %> <%= command.id %> -i the_input_file -o the_out_put_file',
    '<%= config.bin %> <%= command.id %> --interactive',
  ]

  // To describe multiple examples use:
  static readonly examples = [
    {
      description: '\x1B[93mInvoke the command using command line flags\x1B[0m',
      command: '<%= config.bin %> <%= command.id %> -i the_input_file -o the_out_put_file',
    },
    {
      description: '\x1B[93mInvoke the command interactively\x1B[0m',
      command: '<%= config.bin %> <%= command.id %> --interactive',
    },
  ]

  // Note: the BaseCommand abstract class implements the log level and interactive flags
  static flags = {
    input: Flags.string({
      char: 'i', required: false, exclusive: ['interactive'],
      description: '\x1B[31m(required if not --interactive)\x1B[34m The Input file',
    }),
    output: Flags.string({
      char: 'o', required: false, exclusive: ['interactive'],
      description: '\x1B[31m(required if not --interactive)\x1B[34m The Output file',
    }),
    includeRaw: Flags.boolean({
      char: 'r', required: false, description: 'Include raw input file in HDF JSON file',
    }),
  }

  async run(): Promise<any> {
    const {flags} = await this.parse(MYCLI)
      
    // Check if we are using the interactive flag
    let inputFile = ''
    let outputFile = ''
    if (flags.interactive) {
      const interactiveFlags = await getFlags() // see CLI Interactive Template
      inputFile = interactiveFlags.inputFile
      outputFile = path.join(interactiveFlags.outputDirectory, interactiveFlags.outputFileName)
    } else if (this.requiredFlagsProvided(flags)) { // see method template bellow
      inputFile = flags.input as string
      outputFile = flags.output as string
    } else {
      return
    }

    //*****************************//
    // Implement the CLI code here //
    //*****************************//
  }

  // Check for required fields template
  requiredFlagsProvided(flags: { input: any; output: any }): boolean {
    let missingFlags = false
    let strMsg = 'Warning: The following errors occurred:\n'

    if (!flags.input) {
      strMsg += colors.dim('  Missing required flag input file\n')
      missingFlags = true
    }

    if (!flags.output) {
      strMsg += colors.dim('  Missing required flag output (directory or file)\n')
      missingFlags = true 
    }

    if (missingFlags) {
      strMsg += 'See more help with -h or --help'
      this.warn(strMsg)
    }

    return !missingFlags
  }
}

CLI Interactive Template

The SAF CLI uses the inquirer.js module for interactively ask for the CLI flags, both required and optional flags. The SAF CLI is using the @inquirer/prompts for the interactive flags selections.

To use the interactive mode create an asynchronous function that returns an object with the selected answers. Each question provided to inquire returns a promise.

Note

If using the choices question object type, it may be necessary to increase the node default max listeners (defaults to 10) if more than 10 choices are required.
To increase the number of listeners use EventEmitter.defaultMaxListeners = [number_value]

Contiguous Question/Answer Template

Use the following code as a starter template

import {EventEmitter} from 'events'
// In this example we are using the following prompts: input, select 
import {input, select} from '@inquirer/prompts'


async function getFlags(): Promise<any> {

  // Modify the color of the required files by modifying the default theme
  const fileSelectorTheme = {
    style: {
      file: (text: unknown) => chalk.green(text),
      help: (text: unknown) => chalk.yellow(text),
    },
  }

  // Create an object of questions
  // This example asks for an input file and an output directory and
  // and filename to be generated in the selected output directory.
  // Additionally we ask it we should use debugging
  const answers = {
    inputFile: await fileSelector({
      message: 'Select a json file name to be used:',
      pageSize: 15,
      loop: true,
      type: 'file',
      allowCancel: false,
      cancelText: 'No file was selected',
      emptyText: 'Directory is empty',
      showExcluded: false,
      filter: file => file.isDirectory() || file.name.endsWith('.json'),
      theme: fileSelectorTheme,
    }),
    outputDirectory: await fileSelector({
      message: 'Select output directory:',
      pageSize: 15,
      loop: true,
      type: 'directory',
      allowCancel: false,
      cancelText: 'No output directory was selected',
      emptyText: 'Directory is empty',
      theme: fileSelectorTheme,
    }),
    outputFileName: await input({
      message: 'Specify the output filename (.csv). It will be saved to the previously selected directory:',
      default: 'outputfile.csv',
      required: true,
    }),
    useDebugging: await select({
      message: 'Use debugging mode?:,
      default: false,
      choices: [
        {name: 'true', value: true},
        {name: 'false', value: false},
      ],
    }),
  }

  // That is all to it, now return the object with the selected values
  return answers
}

Dependent Question/Answer Template

Use the following code as a starter template

import inquirer from 'inquirer'
import {EventEmitter} from 'events'
import inquirerFileTreeSelection from 'inquirer-file-tree-selection-prompt'

async function getFlags(): Promise<any> {
  // Register the file tree selection plugin
  inquirer.registerPrompt('file-tree-selection', inquirerFileTreeSelection)

  // Create a question object
  // This example asks if user wants to use an input
  // file, if yes than ask for the file full path
  const addInputFilePrompt = {
    type: 'list',
    name: 'useInputFile',
    message: 'Include an input file:',
    choices: ['true', 'false'],
    default: false,
    filter(val: string) {
      return (val === 'true')
    },
  }
  const inputFilePrompt = {
    type: 'file-tree-selection',
    name: 'inputFilename',
    message: 'Select the input filename - in the form of .xml file:',
    filters: 'xml',
    pageSize: 15,
    require: true,
    enableGoUpperDirectory: true,
    transformer: (input: any) => {
      const name = input.split(path.sep).pop()
      const fileExtension =  name.split('.').slice(1).pop()
      if (name[0] === '.') {
        return colors.grey(name)
      }

      if (fileExtension === 'xml') {
        return colors.green(name)
      }

      return name
    },
    validate: (input: any) => {
      const name = input.split(path.sep).pop()
      const fileExtension =  name.split('.').slice(1).pop()
      if (fileExtension !== 'xml') {
        return 'Not a .xml file, please select another file'
      }

      return true
    },
  }
  
  // Variable used to store the prompts (question and answers)
  const interactiveValues: {[key: string]: any} = {}
  
  // Launch the prompt interface (inquiry session)
  let askInputFilename: any
  const askInputFilePrompt = inquirer.prompt(addOvalFilePrompt).then((addInputFilePrompt : any) => {
    if (addInputFilePrompt.useInputFile === true) {
      interactiveValues.useInputFile= true
      askInputFilename = inquirer.prompt(inputFilePrompt ).then((answer: any) => {
        for (const question in answer) {
          if (answer[question] !== null) {
            interactiveValues[question] = answer[question]
          }
        }
      })
    } else {
      interactiveValues.useInputFile= false
    }
  }).finally(async () => {
    await askInputFilename
  })
  await askInputFilePrompt
  return interactiveValues
}

CLI Supporting Functions

CLI helper functions are available in the cliHelper.ts TypeScript file.

Functions provided are:

• Colorize console log outputs (various colors and combinations)
• Data logging for every colorized output
• Initialize output log filename (setProcessLogFileName(fileName: string))
• Retrieve log output object (getProcessLogData(): Array<string>)
• Add content to the log data object (addToProcessLogData(str: string))
• Save the log output content (saveProcessLogData())