RESTest configuration files

Table of contents

1. Properties files: configuring RESTest execution
2. Parameters
3. Example: Bikewise API

Properties files: configuring RESTest execution

RESTest needs a .properties file for setting up the experiment. We recommend to create a properties file for every API you test. This is the basic structure of a properties file:

# Required parameters
oas.path=path/to/oas/api/spec/file.yaml
conf.path=path/to/test/conf/file.yaml

# Optional parameters
generator=[CBT/RT]
testsperoperation=10
test.target.dir=directory/where/test/is/generated
experiment.name=nameOfTheAPI
testclass.name=ClassNameOfTest
coverage.input=true
coverage.output=true
stats.csv=true
numtotaltestcases=30
allure.results.dir=directory/where/allure/results/are/generated
allure.report.dir=directory/where/allure/reports/are/generated
data.tests.dir=directory/where/test/data/csv/files/are/generated
data.coverage.dir=directory/where/coverage/data/csv/files/are/generated
numtotaltestcases=-1
delay=-1
faulty.ratio=0.1

#CBT only:
faulty.dependency.ratio=0.1
reloadinputdataevery=100
inputdatamaxvalues=1000

Parameters

These are the available parameters when configuring RESTest execution:

Parameter	Required	Type	Description
oas.path	Required	String	Path to the OpenAPI specification file.
conf.path	Required	String	Path to the test configuration file generated from the OpenAPI specification file.
generator	Optional	String	Test case generator to be used. Possible values: RT (Random Testing), CBT (Constraint-based Testing). Defaults to RT.
testsperoperation	Optional	Integer	Number of test cases to generate per operation. For instance, if you test 4 operations and set this parameter to 5, 20 test cases will be generated in each iteration. Defaults to 1.
test.target.dir	Optional	String	Directory where to store Java test classes. Defaults to src/generation/java/restassured.
experiment.name	Optional	String	Identifier of the experiment performed. Used for folder names where test reports are generated (under target folder) and for the package name of the generated test classes. Defaults to restassured.
testclass.name	Optional	String	Name of the generated test classes. Defaults to RESTestExperiment.
coverage.input	Optional	Boolean	Set to true if you want to generate a report of the input coverage. Defaults to true.
coverage.output	Optional	Boolean	Set to true if you want to generate a report of the output coverage. Defaults to true.
stats.csv	Optional	Boolean	Set to true if you want to generate statistics of the test cases, such as the parameters of every request or the number of nominal and faulty test cases. Defaults to true.
deletepreviousresults	Optional	Boolean	Set to true if you don't want to keep the statistics and Allure reports of previous tests. Defaults to true.
allure.results.dir	Optional	String	Directory where to store the test cases and results in Allure format. Defaults to target/allure-results.
allure.report.dir	Optional	String	Directory where to store the Allure reports. Defaults to target/allure-reports.
data.tests.dir	Optional	String	Directory where to store the CSV stats files. Defaults to target/test-data.
data.coverage.dir	Optional	String	Directory where to store the CSV coverage data files. Defaults to target/coverage-data.
logToFile	Optional	Boolean	If 'true', log messages will be printed to the file indicated in the log.path property. Defaults to false.
log.path	Optional	String	Path to log file. Don't set a file extension, as RESTest will set .log extension by default.
numtotaltestcases	Optional	Integer	Total number of test cases to generate. Setting it to -1 will make RESTest to never stop. Defaults to -1.
delay	Optional	Integer	Delay in seconds between iterations. Each iteration executed numtestcases * number of operations tested test cases. Set it to -1 to disable delay. Defaults to -1.
faulty.ratio	Optional	Float	Ratio (0-1) of faulty test cases to generate. A faulty test case is one that uses invalid request data and therefore expects a client error response from the API (4XX status code). Defaults to 0.1.
faulty.dependency.ratio	Optional for CBT	Float	Ratio (0-1) of faulty test cases to generate due to inter-parameter dependencies. These test cases violate one or more inter-parameter dependencies present in the operation and therefore are considered faulty. Defaults to 0.5.
reloadinputdataevery	Optional for CBT	Integer	Number of requests using the same randomly generated input data. Defaults to 100.
inputdatamaxvalues	Optional for CBT	Integer	Number of values used for each parameter when reloading input data. Defaults to 1000.

If you want to display extra information in the Allure report - apart from these parameters -, you can add extra parameters to the RESTest configuration file using the following format: parameterName=value.

Example: Bikewise API

This is the RESTest configuration file for the Bikewise API:

# ADD HERE ANY EXTRA INFORMATION TO BE DISPLAY IN THE TEST REPORT

# API name
api=Bikewise API v2

# API url
api.url=https://www.bikewise.org/api

# API doc URL
api.doc.url=https://www.bikewise.org/documentation/api_v2 

# CONFIGURATION PARAMETERS

# Test case generator
generator=RT

# Number of test cases to be generated per operation on each iteration
testsperoperation=10

# OAS specification
oas.path=src/test/resources/Bikewise/swagger.yaml

# Test configuration file
conf.path=src/test/resources/Bikewise/fullConf.yaml

# Directory where the test cases will be generated  
test.target.dir=src/generation/java/bikewise

# Experiment name (for naming related folders and files)
experiment.name=bikewise

# Name of the test class to be generated
testclass.name=BikewiseTest

# Measure input coverage
coverage.input=true

# Measure output coverage
coverage.output=true

# Enable CSV statistics
stats.csv=true

# Maximum number of test cases to be generated
numtotaltestcases=80

# Optional delay between each iteration (in seconds)
delay=-1

# Ratio of faulty test cases to be generated (negative testing)
faulty.ratio=0

# CONFIGURATION SETTINGS FOR CONSTRAINT-BASED TESTING

# Ratio of faulty test cases to be generated due to broken dependencies.
faulty.dependency.ratio=0

# Number of test cases after which new test data will be loaded.
reloadinputdataevery=10

# Max number of data values for each parameter
inputdatamaxvalues=10

The Bikewise API has 4 operations, and we want to test all of them. With the configuration shown above, RESTest will generate 40 test cases in each iteration (numtestcases=10 x 4 operations). There will be two iterations (numtotaltestcases=80 ÷ 40 = 2 iterations) with no delay between them (delay=-1). Only random test cases will be generated, as CBT is disabled (generator=RT).

At the end of the execution, two test classes called BikewiseTest_XXX will have been generated under src/generation/java/bikewise. Furthermore, a set of test reports will have been generated at the following locations:

• target/test-data/bikewise
• target/coverage-data/bikewise
• target/allure-reports/bikewise