Release Candidate 3

Continuing on with the testing, this release candidate has the following changes:

Changes

• Depth first searching on unsorted arrays now properly handles locating multiple objects defined in the array items list
• POST requests will no longer send any input defined in the test case
• Fixed misc bugs related to tokenizing sub commands
  • now supports single quotes, double quotes, and backticks for grouping parameters
• storeAs field on matchers now respects JSON dot notation. Fetching from the data store supported dot notation, but writing had not, making the values stored in a destination containing dots or array indexes inaccessible.
  • It will automatically create the leading objects and arrays into the datastore when assigning a value that does not exist.
  • It can overwrite pre-existing values in the data store
• The built in JSON notation handling now supports keying into objects with square brackets: e.g. rootObject["childObject"] for variables
• Added a -micro parameter for report output
• Improved -tiny report output
• Can now execute inverted tagged tests by prefixing the tag with !. E.g. -tag=!beta will execute all tests that do not have the tag beta in their tag field
• REST requests will default to GET if the method field is missing in a test case
• Integer matching now supports numerical expressions like array lengths $< $> $= $<= $>=
• Responses in the long test output are now limited to 80 lines. If a response is > 80 lines, it will be written to a temporary file with the file path added into the report output
• Can now define validators for nested JSON fields using the JSON path as the field key with a prefix $.. E.g. $.MyResponse.SomeString: $any.
  • You can use square bracket keying to define an exact key to match: $.["$.Some.Response Key"]: $any
• Specifying - as input to the -file parameter will now read from STDIN

Breaking Changes

• response.binary=true|false has been moved to response.type=json|binary to support different response formats in the future

Just to reiterate the purpose of the release candidates

---

At this point, no new features will be added and feedback will be collected for any potential breaking changes prior to the release.

Some things that may change between release candidates and the final release include:

• error behaviour
  • what should fail a test suite vs what should be swallowed an shown in the final test report
• test suite yaml config keys / layouts
  • the basic structure should remain the same, but some field names may change for clarity
• bug fixes
  • any discovered unintended behaviour will be corrected
• documentation (ReadMe)
  • any unclear (or incorrect) documentation should be corrected
  • more examples may be added as needed

With those changes in mind, this release should not be used within a production environment.

Release Candidate 2

Continuing on with the testing, this release candidate fixes the following issues:

• Asset package included on release now has arp in the name rather than the original checker
• Resolved an issue where variables and subcommands that were already resolved or executed may try to resolve and execute their result
• YAML files in the test root directory that do not contain any test cases will now be filtered from the test report output

Just to reiterate the purpose of the release candidates

---

At this point, no new features will be added and feedback will be collected for any potential breaking changes prior to the release.

Some things that may change between release candidates and the final release include:

• error behaviour
  • what should fail a test suite vs what should be swallowed an shown in the final test report
• test suite yaml config keys / layouts
  • the basic structure should remain the same, but some field names may change for clarity
• bug fixes
  • any discovered unintended behaviour will be corrected
• documentation (ReadMe)
  • any unclear (or incorrect) documentation should be corrected
  • more examples may be added as needed

With those changes in mind, this release should not be used within a production environment.

Release Candidate 1

Arp finally supports enough features to be used for wide spread testing across multiple API types. At this point, no new features will be added and feedback will be collected for any potential breaking changes prior to the release.

Some things that may change between release candidates and the final release include:

• error behaviour
  • what should fail a test suite vs what should be swallowed an shown in the final test report
• test suite yaml config keys / layouts
  • the basic structure should remain the same, but some field names may change for clarity
• bug fixes
  • any discovered unintended behaviour will be corrected
• documentation (ReadMe)
  • any unclear (or incorrect) documentation should be corrected
  • more examples may be added as needed

With those changes in mind, this release should not be used within a production environment.

Test Release

First release to test the release workflow in the hopes that it automatically creates all the necessary binaries.