Skip to content

Releases: steep-wms/steep

v6.1.1

07 Jul 09:17
Compare
Choose a tag to compare

Bug fixes:

  • Really abort (instead of cancel) process chain if agent leaves cluster prematurely
  • Make sure all Steep instances are notified about left remote agent

v6.1.0

06 Jul 14:12
Compare
Choose a tag to compare

New features:

  • Web UI:
    • Add dark theme
    • Improve responsiveness of search results page
  • Database:
    • Share PostgreSQL connection pool between verticles
    • Add possibility to configure database connection pool size and idle time

Bug fixes:

  • Correctly schedule process chains with higher priorities if they have different required capability sets as the ones
    already running

v6.0.0

24 Jun 07:33
Compare
Choose a tag to compare

We are very proud to announce Steep 6! This is the biggest release we've ever done. It contains a lot of cool new features, bug fixes, but also some breaking changes. Make sure to have a look at the detailed list below and read the documentation on the Steep website.

Thanks a lot to @hwuerz for the support!


Highlights in this release:

  • Improved workflow syntax
  • Improved workflow validation
  • Eager process chain generation (improved parallelization)
  • Named workflows
  • Workflow priorities
  • Full-text search
  • Improved performance (DB + Web UI + HTTP + Event bus)

New features:

  • Improved workflow model:
    • Support input parameters with values instead of pre-defined variables
    • Allow output variables to be used without declaring them in vars
  • Enable full parallelisation of process chains and for-each actions:
    • Process chains are now generated eagerly. This means as soon as the first results of running process chains are available, new process chains can be generated.
    • For-each actions are now eagerly unrolled. This means that execution of for-each actions that depend on the output of other actions or that are recursive can now start even if the results are only partially available yet.
  • Add possibility to prioritize workflows
  • Add powerful full-text search for workflows and process chains
  • Improved workflow validation:
    • Disallow variables to be used more than once as output
    • Make sure enumerators cannot be reused as enumerators or outputs
    • Validate if variable values are accessed within the right scope
    • Display path to error in validation result
  • Add dependency injection mechanism for plugin interfaces
  • Add process chain consistency checker plugin
  • Add plugin versions
  • Add more Prometheus metrics
  • Display timeout policies in UI
  • Display workflow name in UI
  • Display original YAML source on workflow detail page in UI if available
  • Add button to create new workflow to UI
  • Improved performance:
    • Database requests (less requests and faster queries)
    • HTTP API
    • Web UI
    • Cluster communication
  • Implement timeout for creating a VM
  • Compress large messages sent over the event bus
  • Add shortcut button to create new workflow from scratch to UI
  • Add more parameters for cluster configuration:
    • Add possibility to configure placement group name
    • Add possibility to make a Steep instance a Hazelcast lite member
  • Improve reliability by increasing backup count of distributed Hazelcast data structures

Breaking changes:

  • Hazelcast has been updated. Steep 6 instances cannot connect to Steep 5.x instances. You have to restart your whole cluster during update.
  • Workflow API version 3.x has been removed. Please upgrade your workflows to API version 4.x.
  • All model properties are now camel case. For example, data_type in service metadata has been renamed to dataType. The same applies to properties such as required_capabilities or file_suffix. Please refer to the documentation on the Steep website for more information.
  • The deprecated property supportedServiceId in plugin descriptors has been removed. It was replaced by supportedServiceIds in earlier versions already.
  • Deprecated plugin interfaces have been removed
  • Executable.serviceId is now mandatory. Make sure to update your plugins if you create Executable objects (this particularly applies to process chain adapters)
  • The deprecated configuration property onlyTraverseDirectoryOutputs has been removed. Steep now always only traverses output directories if the dataType in the service metadata is directory. All other data types will not be traversed but directly passed to the subsequent service.
  • Remove deprecated store actions and action parameters
  • Configuration items denoting periods of time have been renamed. All items must now be specified as durations. For example, lookupIntervalMilliseconds: 2000 becomes lookupInterval: 2s. Refer to the documentation for an overview of all configuration items.
  • Configuration item steep.http.cors.maxAge has been renamed to steep.http.cors.maxAgeSeconds to make clear that it has to be specified in seconds and to be in line with the corresponding CORS HTTP header.

Bug fixes:

  • Fix issue where some Docker containers were not killed when the workflow was cancelled
  • Do not fail if plugin configuration file is empty
  • Fix reading arbitrarily large GridFS files from MongoDB
  • Do not create more VMs if there already are enough providing a given required capability set
  • Fix intermittent crashes in UI if connection to event bus was lost
  • When looking for orphaned process chains, the scheduler does not send a message to itself anymore

Maintenance:

  • Upgrade Vert.x to 4.3.0
  • Switch to Zulu OpenJDK Docker base image
  • Install security patches on Docker image build
  • Update other dependencies

v5.12.0

10 Mar 13:02
Compare
Choose a tag to compare

New features:

  • Added possibility to specify workflow action IDs
  • Added possibility to specify explicit dependencies between actions via the new dependsOn attribute, even if those actions are not connected through outputs/inputs
  • Extended process chain adapter plugin interface and added the workflow currently being evaluated

Bug fixes:

  • Correctly resolve dependencies of actions inside for-each actions even if they have been specified in reverse order
  • Set submission end time on error
  • Cancel process chains if submission fails
  • Tolerate a certain number of faults when looking up process chain status

Maintenance:

  • Bump up workflow API version to 4.4.0
  • Update dependencies

v5.11.0

26 Jan 10:06
Compare
Choose a tag to compare

New features:

  • Add persistent cache for compiled plugin scripts
  • Add possibility to specify maximum runtime, inactivity timeouts, and deadlines for services and actions
  • Add /health endpoint
  • Make timeouts in CloudManager configurable

Maintenance:

  • Bump up workflow API version to 4.3.0
  • Update dependencies
  • Update to Next.js 12.0.7
  • Update to Node.js 16

v5.10.0

09 Nov 08:41
Compare
Choose a tag to compare

New features:

  • Add templating support to configuration files
  • Allow Hazelcast cluster name to be configured
  • Implement timeout for destroying a VM
  • Add opt-in to only traverse directory outputs

Bug fixes:

  • Automatically remove orphaned entries in the remote agent registry
  • Handle exception when retrieving agent info and an agent is unavailable

Maintenance:

  • Update openstack4j to 3.10
  • Fix compiler warnings and typos

v5.9.0

18 Aug 11:04
Compare
Choose a tag to compare

New features:

  • Improved performance of PostgreSQL connection
  • New garbage collector can be configured to automatically delete old workflows and VMs from the database

Maintenance:

  • Update UI dependencies

Bug fixes:

  • Fix potential deadlock in scheduler

v5.8.2

22 Jul 09:06
Compare
Choose a tag to compare

Bug fixes:

  • Fix race condition in DefaultOutputCollector

Maintenance:

  • Add end-to-end tests for web UI (thanks to @iBims1JFK)

v5.8.1

09 Jun 10:38
f07e7e1
Compare
Choose a tag to compare

Bug fixes:

  • Make process chain log endpoint more robust
  • Do not make duplicate requests when viewing process chains in the web UI
  • Fix an issue where Steep could create more VMs than configured in the agent pool in rare cases
  • Do not abort lookup loops in Controller and Scheduler in case of an error (e.g. temporary database failure)

v5.8.0

19 May 11:10
Compare
Choose a tag to compare

New features:

  • Allow scheduler to resume orphaned process chains. Orphaned process chains are those that are running but currently not being monitored by any scheduler instance. This feature increases fault tolerance as it allows a scheduler to resume the work of another crashed instance.
  • Plugins:
    • Extend progress estimator plugin interface to support multiple service IDs
    • Add possibility to specify dependencies between plugins (applies to process chain adapters and initializers)
    • Add possibility to load pre-compiled plugins. This can speed up Steep's startup process if there are a lot of plugins.
  • Configuration:
    • Add support for YAML anchors in configuration files
    • Support Ant-like file globbing for paths in configuration files
    • Add upload function to provisioning templates
  • Display allocated process chain ID in agent details
  • Reduce the number of log messages, in particular when agents join or leave the cluster
  • Run PostgreSQL database migration only once per Steep instance
  • Improve graceful shutdown
  • Add possibility to restore cluster members on startup from VM registry

Maintenance:

  • Run unit tests in parallel
  • Speed up MongoDB unit tests
  • Update web UI dependencies

Bug fixes:

  • Allow runtime plugins to access current Vert.x context
  • Do not fail to delete OpenStack block device if it does not exist