orquestaconvert

Converts Mistral workflows into Orquesta workflows

Usage

Getting Started

The script automatically sets up a virtualenv (if it doesn't exist) that contains all of the necessary depedencies to run.

Simply run the command for the first time and everything will get setup for you:

git clone https://github.com/EncoreTechnologies/orquestaconvert.git
cd orquestaconvert
./bin/orquestaconvert.sh --help

orquestaconvert.sh - convert a single workflow and print to stdout

You must specify one or more workflow YAML files to convert as the last arguments to the script.

There are also some options you can use:

• -e <type> - Type of expressions (YAQL or Jinja) to use when inserting new expressions (such as task transitions in the when clause)
• --force - Forces the script to convert and print the workflow even if it does not successfully validate against the Orquesta YAML schema.
• --validate - Runs just the validation portion of the script, very useful to validate workflows you partially converted with --force then finished conversion by hand.

Examples

Convert a single workflow

Convert the nasa_apod_twitter_post.yaml workflow from Mistral to Orquesta, using Jinja expressions (the default) in task transition conditions.

./bin/orquestaconvert.sh ./tests/fixtures/mistral/nasa_apod_twitter_post.yaml

Convert a workflow, output YAQL expressions

Convert the workflow, using YAQL expressions for new task transition conditions, and skips Orquesta workflow validation. Note that this may generate a workflow that is neither a valid Mistral nor a valid Orquesta workflow.

./bin/orquestaconvert.sh -e yaql --force ./tests/fixtures/mistral/nasa_apod_twitter_post.yaml

Validate an Orquesta workflow

Run Orquesta YAML schema validation on the file. Returns 0 on successful validation, nonzero on unsuccessful validation. Also use the --verbose option to explitly print the validation results for the file.

./bin/orquestaconvert.sh --validate ./tests/fixtures/orquesta/nasa_apod_twitter_post.yaml

orquestaconvert-pack.sh - convert all Mistral workflows in a pack

This script scans a pack for all action metadata files and attempts to convert all Mistral workflows to Orquesta and/or validates all Orquesta workflows in a pack using the orquestaconvert.sh script. This script passes all unrecognized arguments to orquestaconvert.sh, so all actions you can do on one workflow with that script, you can do to the entire pack by using this script.

You must either run this command from the base directory of a pack or you must specify the directory that contains action metadata files with the --actions-dir option.

Recognized options are:

• --list-workflows <type> - List all workflows of the specified type (must either be action-chain for ActionChain, mistral-v2 for Mistral, or orquesta or orchestra for Orquesta workflows)
• --actions-dir <dir> - Specifies the directory to scan and convert

Examples

Convert all workflows in a pack

Attempts to convert all workflows from Mistral to Orquesta, using Jinja expressions in new task transitions (Jinja is the default).

./bin/orquestaconvert-pack.sh

List Mistral workflows in a pack

Lists remaining Mistral workflows.

./bin/orquestaconvert-pack.sh --list-workflows mistral-v2

Convert all workflows in a pack, outputting YAQL expressions

Converts all Mistral workflows (using YAQL expressions when generating task transition conditions) in mypack/actions to Orquesta and skips validation. Note that using this option may create workflows that are neither valid as Mistral nor Orquesta workflows.

./bin/orquestaconvert-pack.sh --expressions yaql --force --action-dir mypack/actions

Validate all Orquesta workflows in a pack

Explicitly rints the validation results for all Orquesta workflows.

./bin/orquestaconvert-pack.sh --validate --verbose

Features

• Converts direct Mistral Workflows into Orquesta Workflows (general structure)
• Handles input, output, tasks
• For each task, action, input, publish, on-success, on-error, and on-complete are all converted
• Converts simple Jinja and YAQL expressions
• Converts task(), st2kv, _.xxx / $.xxx, etc in Jinja and YAQL expressions
• Converts with-items and concurrency attributes
• Converts most retry attributes, including continue-on and simple break-on expressions

Limitations

• Does not convert {% %} Jinja expressions
• Does not convert complex Jinja / YAQL expressions
• Does not convert retry specifications with complex Jinja expressions for continue-on and break-on
  • use the --force flag to forcibly convert those workflows and convert those expressions manually
• Does not convert reverse type workflows
• Does not convert workbooks (multiple workflows in the same file
• Does not convert task('xxx') references to non-local tasks, the current task is always assumed.
• Does not convert workflows with an output-on-error stanza
• Does not convert workflows if the tasks contain one or more of the following attributes
  • keep-result
  • pause-before
  • safe-rerun
  • target
  • timeout
  • wait-after
  • wait-before
  • workflow