Plugins

Types of Plugins

One of Gauge's key features is to be able to support a wide variery of plugins.

This section details out the various types of plugins that Gauge supports.

There are below types of plugins present in Gauge :

• Langauge Runners
• Reporting Plugins
• Documentation PLugins
• IDE Plugins

Language Runners

Gauge allows one to write implementations in multiple languages, and this is made possible via such plugins. Language runners are alive for the full cycle of an execution.

Responsibilites

• Execute hooks (when requested to)
• Execute method corresponding to a step
• Send execution response back to Gauge
• Refactor method signatures with changes defined by Gauge
• Initialize a project with template
• Read/write to datastore (currently, datastore is intra process, hence parallel processes cannot use a shared datastore)
• Holds reference to Step text and Method / execution block.

Consituents

• Runner binary : This is responsible for scanning the test library and also invoke methods when requested to.
• Skeleton files (templates) : These files create a sample hello world project that can be used as a quick-start for creating a new project.
• Metadata file (usually <language>.json) : Holds meta information about the plugin. Also holds the commands to invoke for init and run phases of the plugin.

Format of a Metadata file(<langauge>.json)

{
    "id": "{plugin-name}",
    "version": "{plugin-version}",
    "description": "{plugin-description}",
    // Commands to execute pre-installation
    "preInstall": {
        "windows": [],
        "linux": [],
        "darwin": []
    },
    // Commands to execute post-installation
    "postInstall": {
        "windows": [],
        "linux": [],
        "darwin": []
    },
    // Commands to invoke for run phase of the plugin
    "run": {
        "windows": [],
        "linux": [],
        "darwin": []
    },
    // Commands to invoke for init phase of the plugin
    "init": {
        "windows": [],
        "linux": [],
        "darwin": []
    },
    // Commands to execute pre-uninstallation
    "preUnInstall": {
        "windows": [],
        "linux": [],
        "darwin": []
    },
    // Commands to execute post-uninstallation
    "postUnInstall": {
        "windows": [],
        "linux": [],
        "darwin": []
    },
    // Location of libraries for the plugin relative to plugin directory
    "lib": "{libs}",
    "gaugeVersionSupport": {
        "minimum": "{minimum gauge version supported}",  // mandatory
        "maximum": "{maximum gauge version supported}"   // optional
    }
}

Installation

Installing a plugin happens via Gauge. Gauge's properties file contains a URL that points to the Gauge-Repository, which holds meta information, which Gauge uses to determine the highest compatible version of the plugin. This makes plugin installation a two step process:

• Download <language>-install.json, determine the highest compatible plugin version
• Verify if version is already installed, if not, download the plugin using the URL specified in the <language>-install.json.

Reports

Reports plugins come in towards the end of the execution. In Gauge's context the definition of Report is the result of a test run.

HTML Report

The HTML report holds the output of the test execution, with screenshots (on failure).

HTML report is a plugin, and it listens to Gauge's messages. On Receiving SuiteExecutionResult, the plugin serializes the message to JSON.

The view of this report is an angular-js application that binds the JSON output. The result looks like this

JUnit style report

Conceptually everything is similar to the HTML report, except that instead of a JSON and angular-js view, this plugin serializes the result into a JUnit format XML.

IDE Plugins

Gauge ecosystem includes IDE plugins that aim to increase productivity of users and also reduce friction in most common user flows.

For details on features that are supported in various editors refer to IDE-Support

Documentation Plugins

This class of plugins exist to generate various formats of human readable/easy to navigate documentation that gives the user a better view of the specifications.

Spectacle is an example of a Documentation plugin.

Communication

Gauge and Runners exchange data using one of the below:

• Command line arguments: Gauge tells the runner to run Setup or Start phase by passing in --setup or --start argument respectively, while invoking the runner.
• Environment Variables: Used to exchange configuration properties. Gauge loads all values in default.properties as well as environment.properties from the project environment, and passes these to the Runner.
• GAUGE_INTERNAL_PORT : This environment variable holds a TCP port, which Gauge and the Runner exchange core messages.
• GAUGE_API_PORT: This environment variable holds a TCP port, which Gauge and the Runner exchange api messages.

The data passed via TCP use Protocol Buffers.

Execution

Gauge execution can be one of the three types as mentioned below.

Setup

This phase of execution initializes a new Gauge project. The setup consists of below sequence of steps:

• Gauge checks if the plugin for the specified language is installed.
• If plugin isn't installed, it queries the Gauge-Repository to see if there is a plugin available
• Throws out an error if an invalid language is specified
• Gauge installs the language runner plugin if it isn't installed already
• Copies Gauge specific files, manifest.json, hello_world.spec etc.
• Invokes the runner with a --setup flag.
• Runner responds to command line argument
• Runner the copies the language specific files.

Currently there is only one template that is copied over, but there are plans to add more templates.

Scan

This phase is implicit. This happens as a pre-requisite to running specifications. The Runner is invoked with a --start flag, which the Runner responds to by:

• Building the target project (when required)
• Scan and build a cache of Types, Step Implementations and Hook implementations.
• Wait for execution requests.

Run Specs

As the name suggests, this phase would run the implementations. However there are some pre and post run tasks that also get executed:

• Datastore initialization/Clear: Initialize a datastore at the right level. Clear datastore at appropriate points of the execution lifecycle.
• Hooks execution: Execute hooks at appropriate points of the execution lifecycle.

Gauge aware methods

These are methods/blocks that Gauge would understand and execute at various points.

Step

These are implementation of one or more business steps defined in the spec files. These methods are either annotated with @Step(Java), [Step](C#) or used withstep <block>(Ruby).

Hook

These are methods that get invoked at various points in the execution life cycle. The hooks available are:

• before_suite
  • before_spec
    • before_scenario
      • before_step
      • after_step
    • after_scenario
  • after_spec
• after_suite

Datastore

Datastore is a way for the tests to exchange data. Datastores are basically a key-value store that have lifecycles attached to it. It is quite similar to Session in web application world.

Types

• Suite Datastore : Lives throughout the lifecycle of the entire test suite.
• Spec Datastore : Lives throughout the lifecycle of a spec, it is destroyed after the spec execution completes, and a new one is created
• Scenario Datastore : Lives throughout the lifecycle of a scenario, it is destroyed after the scenario execution completes, and a new one is created

Datastores are In-Memory at the moment, so they are expected to be shared only within the same machine.

Responsibilities of Gauge and Various Plugins

Gauge Core

• install plugins

• check version compatibility

• init project

• Copy common files

• Send Runner an init request

• Loads environment

• validate specs

• request the runner for list of all steps defined

• scan spec files for step invocations, detect unimplemented step

• orchestrate test run

• determine scenarios to be executed

• orchestrate the test run

• hold the spec->[context step]?->scenario->step structure.

• aggregate test run results

• broadcast event hooks

• parallelize test execution

• filter by tags

• Request runner to Initialize datastore (at various event)

• refactor

• send refactor request to language runner

• daemonized run

• expose API with helpers for plugins

• all steps in project

• all concepts in the project

• parse step/spec to parameterized value

• Console report

Plugins

Language Runners

• Execute hooks (when requested to)
• Execute method corresponding to a step
• send execution response back to Gauge
• Refactor method signatures with changes defined by Gauge
• Initialize a project with template
• Read/write to datastore (currently, datastore is intra process, hence parallel processes cannot use a shared datastore)
• Holds reference to Step text and Method / execution block.

Others

Reporters

• HTML Report (via plugin)

IDE (Intellij, Visual Studio, Eclipse)

• Syntax highlighting
• Autocomplete
• Goto definition
• Refactoring
• Test runner
• Formatting
• Find Usages

Refer to this page for a detailed IDE feature comparison.