Artillery v2.0.0-32

Changelog

New features

• Splunk integration in publish-metrics plugin. You can now export data from Artillery tests to the Splunk observability platform.
• Apdex support via the new apdex plugin. Calculate an industry standard measure of users' satisfaction based on HTTP metrics in Artillery tests.
• Add --record and --key flags for the run command for enabling Artillery Cloud reporting for individual test runs
• The namespace for metrics-by-endpoint-produced metrics may be customized now

Fixes & improvements

• Test metadata for tests running on AWS Lambda is reported correctly
• Fix for ramps of duration of 1s (4e379f5)
• Suppress AWS SDK deprecation notices

Community shout-outs

• @InesNi @jeffhall4

Artillery v2.0.0-31

What's Changed

New Features

• New Relic integration in publish-metrics plugin - configure Artillery to send test metrics to New Relic (docs)
• Add support for sending Artillery reports to Artillery Cloud dashboard

Fixes & Improvements

• AWS Lambda - fix issue with the size of zipped Lambda packages which would prevent tests from running (#1855)
• AWS Lambda - fix issue with running AWS Lambda tests on Windows (#1846)
• HTTP: Fix issue with cookies not working inside loop under some conditions (#1826)
• Fix issue with CSV payload data not being split across multiple worker threads (#1829)
• HTTP: Query strings are now shown in HTTP debug output (#1817)
• Fix axis display on generated HTML reports (#1815)

Community shout-outs

• @InesNi, @anoopknayak, @MagicKriss, @chinh-tran - new features & fixes
• @mplaul - help with debugging & tracking down the AWS Lambda issue

Artillery v2.0.0-30

What's Changed

New features

• AWS Lambda - third-party plugins and engines are now supported when running tests on AWS Lambda (#1812)
• New notStatusCode check in expect plugin (#1793)
• Datadog integration: includeOnly and excluded options to configure the list of Artillery metrics which are sent to Datadog (a879b4f)

Fixes & improvements

• When artillery run is used with the --output option, Artillery will check if destination exists before running the test
• Miscellaneous dependency updates to address deprecation warnings and security advisories

New Contributors

• @InesNi made their first contribution in #1793
• @anoopknayak made their first contribution in #1801

as well as contributions by @juanig1 @hassy

Artillery v2.0.0-29

Fixes & Improvements

• Fix installation issue due to mis-specified dependency version (#1777)

Artillery v2.0.0-28

New features

• Add support for publishing test metrics to AWS CloudWatch
• HTTP - compressed responses (e.g. gzip) will be decompressed by default now. (Set gzip: false to disable automatic decompression if required to improve performance.)
• CLI - suggest closest match if a command name is mistyped

AWS Lambda

• Print custom role ARN if provided
• Add versioning to default Lambda role and policy created for running tests

Fixes & improvements

• Fix arrival phases with arrivalRate and rampTo set to 0
• Improve rampTo logic to result in more predictable and deterministic ramps
• Fix issue with maxVusers setting that would lead to unexpected number of VUs to be generated
• Fix wildcard message handling support in Socket.io
• Fix issue with setting some properties on the global artillery object

AWS Lambda

• Improve test startup logic: workers will wait for each other to be ready before kicking off the test run
• Fix some intermediate metrics not being flushed & reported
• Fix handling of memory-size and security-group-ids options

Breaking changes

• AWS Lambda: the default IAM role and policy are now versioned. If you attached any custom permissions to the artilleryio-default-lambda-role role, you will need to re-attach it to the new artilleryio-default-lambda-role-20230116 role
• CLI: The probe command has been removed from the CLI. The functionality has been moved to the new Skytrace CLI

v2.0.0-27

New features

• AWS Graviton is now used by default when running tests on AWS Lambda. Using Graviton offers both cost savings and performance boost for load tests running on AWS Lambda. Legacy x86 architecture may be enabled by setting --platform-opt architecture=x86_64 (#1662)

Fixes and improvements

• Fixed an issue where AWS Lambda bootstrap time would be counted and reported as the time VUs ran for (07e0e06)
• Fix an issue where tests where reporting for test with many workers (50+) would be delayed (06aa19c)

v2.0.0-26

New features

• Artillery will now print the contents of the test bundle created for tests running on AWS Lambda
  

Fixes and improvements

• Fix issue with installing dependencies when running on AWS Lambda
• Fix issue with printing error logs from AWS Lambda workers

v2.0.0-25

New features

AWS Lambda support

• Test scripts using custom JS code (in config.processor), CSV files, and external config files (used with the --config flag) can now run on Lambda
• Cost estimations for each test are now reported at the end of each test run
  
• nodejs16.x runtime will be used by the Lambdas

Fixes and improvements

• Always resolve path to processor relative to the config file when a separate config file is used via the --config flag
• Better handling of errors in worker Lambdas

v2.0.0-24

New features

• Cookie parsing behavior may be customized via config.http.cookieJarOptions by passing options to the underlying ToughCookie instance (docs)

Fixes & Improvements

• More predictable rampTo behavior when using smaller values of arrivalRate/rampTo with a large duration (#1621)
• maxVusers setting is applied correctly when running on multiple CPU cores
• AWS Lambda functions will always use the latest release of Artillery

Contributors: @juanig1 @hassy

v2.0.0-23

New features

• Add cdnHit expectation to expect plugin to check that the response was served from cache by a CDN
• Add support for providing a CA certificate to use with Prometheus Pushgateway in publish-metrics plugin

Fixes & Improvements

• Handle malformed cookies from HTTP services (#1540)
• Fix crashes that could be caused by redirect chains in combination with capture/afterResponse hooks (#1544 #1568)
• Make sure response body is available to afterResponse hooks when the response is compressed (#1563)
• Add support for JSON-LD in the expect plugin (artilleryio/artillery-plugin-expect#34)
• Make sure metrics sent to Prometheus Pushgateway with publish-metrics are flushed before Artillery exits
• Routine dependency upgrades to address security advisories

Contributors: @juanig1, @luneo7, @MaximeTaqt, @hassy