Releases: steep-wms/steep
Releases · steep-wms/steep
v6.1.1
Bug fixes:
- Really abort (instead of cancel) process chain if agent leaves cluster prematurely
- Make sure all Steep instances are notified about left remote agent
v6.1.0
New features:
- Web UI:
- Add dark theme
- Improve responsiveness of search results page
- Database:
- Share PostgreSQL connection pool between verticles
- Add possibility to configure database connection pool size and idle time
Bug fixes:
- Correctly schedule process chains with higher priorities if they have different required capability sets as the ones
already running
v6.0.0
We are very proud to announce Steep 6! This is the biggest release we've ever done. It contains a lot of cool new features, bug fixes, but also some breaking changes. Make sure to have a look at the detailed list below and read the documentation on the Steep website.
Thanks a lot to @hwuerz for the support!
Highlights in this release:
- Improved workflow syntax
- Improved workflow validation
- Eager process chain generation (improved parallelization)
- Named workflows
- Workflow priorities
- Full-text search
- Improved performance (DB + Web UI + HTTP + Event bus)
New features:
- Improved workflow model:
- Support input parameters with values instead of pre-defined variables
- Allow output variables to be used without declaring them in
vars
- Enable full parallelisation of process chains and for-each actions:
- Process chains are now generated eagerly. This means as soon as the first results of running process chains are available, new process chains can be generated.
- For-each actions are now eagerly unrolled. This means that execution of for-each actions that depend on the output of other actions or that are recursive can now start even if the results are only partially available yet.
- Add possibility to prioritize workflows
- Add powerful full-text search for workflows and process chains
- Improved workflow validation:
- Disallow variables to be used more than once as output
- Make sure enumerators cannot be reused as enumerators or outputs
- Validate if variable values are accessed within the right scope
- Display path to error in validation result
- Add dependency injection mechanism for plugin interfaces
- Add process chain consistency checker plugin
- Add plugin versions
- Add more Prometheus metrics
- Display timeout policies in UI
- Display workflow name in UI
- Display original YAML source on workflow detail page in UI if available
- Add button to create new workflow to UI
- Improved performance:
- Database requests (less requests and faster queries)
- HTTP API
- Web UI
- Cluster communication
- Implement timeout for creating a VM
- Compress large messages sent over the event bus
- Add shortcut button to create new workflow from scratch to UI
- Add more parameters for cluster configuration:
- Add possibility to configure placement group name
- Add possibility to make a Steep instance a Hazelcast lite member
- Improve reliability by increasing backup count of distributed Hazelcast data structures
Breaking changes:
- Hazelcast has been updated. Steep 6 instances cannot connect to Steep 5.x instances. You have to restart your whole cluster during update.
- Workflow API version 3.x has been removed. Please upgrade your workflows to API version 4.x.
- All model properties are now camel case. For example,
data_type
in service metadata has been renamed todataType
. The same applies to properties such asrequired_capabilities
orfile_suffix
. Please refer to the documentation on the Steep website for more information. - The deprecated property
supportedServiceId
in plugin descriptors has been removed. It was replaced bysupportedServiceIds
in earlier versions already. - Deprecated plugin interfaces have been removed
Executable.serviceId
is now mandatory. Make sure to update your plugins if you createExecutable
objects (this particularly applies to process chain adapters)- The deprecated configuration property
onlyTraverseDirectoryOutputs
has been removed. Steep now always only traverses output directories if thedataType
in the service metadata isdirectory
. All other data types will not be traversed but directly passed to the subsequent service. - Remove deprecated store actions and action parameters
- Configuration items denoting periods of time have been renamed. All items must now be specified as durations. For example,
lookupIntervalMilliseconds: 2000
becomeslookupInterval: 2s
. Refer to the documentation for an overview of all configuration items. - Configuration item
steep.http.cors.maxAge
has been renamed tosteep.http.cors.maxAgeSeconds
to make clear that it has to be specified in seconds and to be in line with the corresponding CORS HTTP header.
Bug fixes:
- Fix issue where some Docker containers were not killed when the workflow was cancelled
- Do not fail if plugin configuration file is empty
- Fix reading arbitrarily large GridFS files from MongoDB
- Do not create more VMs if there already are enough providing a given required capability set
- Fix intermittent crashes in UI if connection to event bus was lost
- When looking for orphaned process chains, the scheduler does not send a message to itself anymore
Maintenance:
- Upgrade Vert.x to 4.3.0
- Switch to Zulu OpenJDK Docker base image
- Install security patches on Docker image build
- Update other dependencies
v5.12.0
New features:
- Added possibility to specify workflow action IDs
- Added possibility to specify explicit dependencies between actions via the new
dependsOn
attribute, even if those actions are not connected through outputs/inputs - Extended process chain adapter plugin interface and added the workflow currently being evaluated
Bug fixes:
- Correctly resolve dependencies of actions inside for-each actions even if they have been specified in reverse order
- Set submission end time on error
- Cancel process chains if submission fails
- Tolerate a certain number of faults when looking up process chain status
Maintenance:
- Bump up workflow API version to 4.4.0
- Update dependencies
v5.11.0
New features:
- Add persistent cache for compiled plugin scripts
- Add possibility to specify maximum runtime, inactivity timeouts, and deadlines for services and actions
- Add
/health
endpoint - Make timeouts in CloudManager configurable
Maintenance:
- Bump up workflow API version to 4.3.0
- Update dependencies
- Update to Next.js 12.0.7
- Update to Node.js 16
v5.10.0
New features:
- Add templating support to configuration files
- Allow Hazelcast cluster name to be configured
- Implement timeout for destroying a VM
- Add opt-in to only traverse directory outputs
Bug fixes:
- Automatically remove orphaned entries in the remote agent registry
- Handle exception when retrieving agent info and an agent is unavailable
Maintenance:
- Update openstack4j to 3.10
- Fix compiler warnings and typos
v5.9.0
New features:
- Improved performance of PostgreSQL connection
- New garbage collector can be configured to automatically delete old workflows and VMs from the database
Maintenance:
- Update UI dependencies
Bug fixes:
- Fix potential deadlock in scheduler
v5.8.2
Bug fixes:
- Fix race condition in
DefaultOutputCollector
Maintenance:
- Add end-to-end tests for web UI (thanks to @iBims1JFK)
v5.8.1
Bug fixes:
- Make process chain log endpoint more robust
- Do not make duplicate requests when viewing process chains in the web UI
- Fix an issue where Steep could create more VMs than configured in the agent pool in rare cases
- Do not abort lookup loops in Controller and Scheduler in case of an error (e.g. temporary database failure)
v5.8.0
New features:
- Allow scheduler to resume orphaned process chains. Orphaned process chains are those that are running but currently not being monitored by any scheduler instance. This feature increases fault tolerance as it allows a scheduler to resume the work of another crashed instance.
- Plugins:
- Extend progress estimator plugin interface to support multiple service IDs
- Add possibility to specify dependencies between plugins (applies to process chain adapters and initializers)
- Add possibility to load pre-compiled plugins. This can speed up Steep's startup process if there are a lot of plugins.
- Configuration:
- Add support for YAML anchors in configuration files
- Support Ant-like file globbing for paths in configuration files
- Add
upload
function to provisioning templates
- Display allocated process chain ID in agent details
- Reduce the number of log messages, in particular when agents join or leave the cluster
- Run PostgreSQL database migration only once per Steep instance
- Improve graceful shutdown
- Add possibility to restore cluster members on startup from VM registry
Maintenance:
- Run unit tests in parallel
- Speed up MongoDB unit tests
- Update web UI dependencies
Bug fixes:
- Allow runtime plugins to access current Vert.x context
- Do not fail to delete OpenStack block device if it does not exist