0.54.1

Note about upgrading CDS

Please install version 0.53.0 before installing this new release.

When you are upgrading CDS:

• backup your database before any operation

Notable Changes in version 0.54.1

In the full changelog, you will find a lot of commits about "workflow v2" / "ui" / "hook" / "ascode". All this work is here to prepare the new CDS "As Code". You will alredy find some documentation about new features, but we will make a full presentation of the new engine when it will be ready for production use.

As part of this new engine, we had to refactor the way that CDS commmunicates with the VCS Servers (bitbucket, github, etc...). This refactoring makes it easier to manage authorization based on Personal Access Tokens, instead of using oauth2 as before.
In the latest version 0.53.0, this was already refactored: you could manage VCS Server with the old and new way. With the version 0.54.1, you can only manage your VCS server one way.

If you migrate from a version 0.53, you have two solutions:

• solution A: migrate on the new VCS system before installing the 0.54.1 version
• solution B: install the version 0.54.1 and use the new VCS system. The VCS communication will be unavailable until you set the vcs server in each cds project.

Solution A

The cdsctl binary contains all commands to manage VCS Server on your CDS projects.

cdsctl experimental project YOUR_CDS_PROJECT_KEY, replace YOUR_CDS_PROJECT_KEY with the KEY of your CDS Project.

Github

Generate a new token on https://github.com/settings/tokens with the following scopes:

• repo:status
• public_repo

Create a yml file, example of vcs-github.yml file:

version: v1.0
name: github
type: github
description: "my github"
auth:
    username: your-username
    token: ghp_your-token-here
options:
    urlApi: "" # optional, default is https://api.github.com
    disableStatus: false    # Set to true if you don't want CDS to push statuses on the VCS server - optional
    disableStatusDetails: false # Set to true if you don't want CDS to push CDS URL in statuses on the VCS server - optional
    disablePolling: false   # Does polling is supported by VCS Server - optional
    disableWebHooks: false  # Does webhooks are supported by VCS Server - optional

Then import the configuration:

cdsctl project vcs import YOUR_CDS_PROJECT_KEY vcs-github.yml

Bitbucket Datacenter

Example with vcs-bitbucket.yml file:

version: v1.0
name: bitbucket
type: bitbucketserver
description: "My Bitbucket Datacenter"
url: "http://localhost:7990/bitbucket"
auth:
    user: username-on-bitbucket
    token: the-long-token-here
options:
    disableStatus: false    # Set to true if you don't want CDS to push statuses on the VCS server - optional
    disableStatusDetails: false # Set to true if you don't want CDS to push CDS URL in statuses on the VCS server - optional
    disablePolling: false   # Does polling is supported by VCS Server - optional
    disableWebHooks: false  # Does webhooks are supported by VCS Server - optional

Then import:

cdsctl project vcs import YOUR_CDS_PROJECT_KEY vcs-bitbucket.yml

Solution B

It's the same as solution A, but your will find the useful commands with cdsctl project vcs (instead of cdsctl experimental project vcs)

The version 0.54.1 allows you to use the webui to configure the VCS project.

What's Changed

Features

• api, repositories, hooks: compute semver for workflow run git context by @fsamin in #6769
• api, ui: remove vulnerabilities from CDS by @sguiheux in #6602
• api: cancel abandonned run results by @fsamin in #6854
• api: createdHour in event by @yesnault in #6710
• api: valid email addresses for GPG keys by @fsamin in #6687
• api: worker model docker image whitelist by @fsamin in #6841
• contrib: docker-push plugin by @fsamin in #6813
• contrib: fix preview in vscode / project management by @rclsilver in #6878
• contrib: helm push plugin by @fsamin in #6826
• contrib: manage artifactory promote/release cargo crate by @sguiheux in #6611
• contrib: plugin rt create rb without target by @richardlt in #6730
• doc,vcs: new vcs documentation, cdsctl project vcs cmd by @yesnault in #6773
• engine: admin can delete public integration by @yesnault in #6630
• engine: websocket - rand.New only once by @yesnault in #6729
• hatchery: killAwolWorkerInterval as conf option by @yesnault in #6662
• hatchery: new metrics by @yesnault in #6725
• hatchery/openstack: list worker IP by @fsamin in #6702
• hatchery/openstack: log openstack server info by @fsamin in #6718
• hatchery/vsphere: monitoring vsphere client by @yesnault in #6652
• hooks: trigger on pull-request by @yesnault in #6819
• hooks: trigger on pull-request by @yesnault in #6825
• integration:artifactory: increase upload timeout by @richardlt in #6660
• sdk: allow to stop goroutine by @richardlt in #6754
• ui: add ability to shift click log expand to expand all by @maximecaruchet in #6629
• ui: upgrade to angular 14 by @sguiheux in #6688
• ui: upgrade to angular 15 by @sguiheux in #6691
• ui: upgrade to angular 16 by @sguiheux in #6694
• sdk/interpolate) Add new math interpolations by @bcecchinato in #6716

Bug Fixes

• api, cdn: set timeout for swift client by @richardlt in #6614
• api: avoid double book by the same hatchery by @yesnault in #6720
• api: CGO_ENABLED, strip plugin binaries by @fsamin in #6893
• api: debug constraint fk_worker_workflow_node_run_job by @fsamin in #6678
• api: do not try to send event if integration model does not support that by @yesnault in #6731
• api: don't restart stopped jobs by @fsamin in #6692
• api: favorites in projects list by @rclsilver in #6847
• api: fill favorite project attribute by @rclsilver in #6873
• api: FindOldestWorkflowRunsWithResultToSync by @fsamin in #6638
• api: keep the empty value for git clone tag parameter by @fsamin in #6649
• api: project key / wf name into events by @yesnault in #6723
• api: run result sync goroutine empties promotion data by @fsamin in #6632
• api: skip booked jobs in queue by @fsamin in #6711
• api: try to fix lock between workers and jobs by @fsamin in #6689
• cdsctl: makefile by @fsamin in #6894
• cdsctl: set CGO_ENABLED=0 for nokeychain variant by @fsamin in #6891
• cdsctl: workflow init by @yesnault in #6728
• cdsctl: workflow init detect vcs server by @yesnault in #6810
• contrib/helm-push: type docker and chart path default value by @fsamin in #6861
• deps: upgrade gorepo by @sguiheux in #6800
• docs: LICENSES by @yesnault in #6623
• docs: README by @yesnault in #6768
• engine: context cancel in postTakeWorkflowJob / initWorkflowRun by @yesnault in #6637
• engine: context cancel in postWorkflowJobResultHandler by @yesnault in #6633
• engine: do not force register on update binary by @yesnault in #6634
• engine: dropped cdn test error by @alrs in #6749
• engine: fix some logs by @yesnault in #6677
• engine: logs for compute purge by @yesnault in #6648
• engine: pk uuid by @yesnault in #6801
• engine: resync vcs status v1 by @yesnault in #6843
• hatcheries: do not restart job on dead workers by @yesnault in #6733
• hatchery: check creation timestamp by @yesnault in #6657
• hatchery: do not add same job into workersStarter by @yesnault in #6722
• hatchery: service name in metrics by @yesnault in #6724
• hatchery: worker starter cache clean by @yesnault in #6727
• hatchery/kubernetes: delete secret by @yesnault in #6653
• hatchery/kubernetes: more logs if no spawn by @yesnault in #6659
• hatchery/kubernetes: no need to delete ...

0.53.0

Note about upgrading CDS

Please install version 0.52.0 before installing this new release.

When you are upgrading CDS:

• backup your database before any operation
• follow instructions from the following documentation: https://ovh.github.io/cds/hosting/upgrade/migrate_0.53/

Notable Changes in version 0.53.0

The version 0.52.0 introduced the notion of Organization in CDS for all authentication drivers. In 0.53.0, organizations are now mandatory.

What's Changed

Bug Fixes

• api,vcs: multiple vcs notifications (#6390) (6a166b7)
• api: allow description in application import and cli (#6344) (d8168ff)
• api: check groups on user deletion (#6401) (0da638e)
• api: compute workflow v3 preview (#6346) (0bc7a67)
• api: do not lock workflow run on release and promote (#6332) (c369ff1)
• api: during promotion, get build module in parrallel (#6466) (137d7dd)
• api: export workflow hook (#6495) (a3074a8)
• api: fail job if worker model is disabled (#6552) (6480259)
• api: inject git_branch_exists variable for retention only if vcs info exists (#6394) (ad8631d)
• api: manage variable deletion and env keys as code (#6563) (b1430bc)
• api: nil pointer workflow push (#6321) (5a8f1fa)
• api: on retryOnlyFailedJob, retrieve build variable (#6452) (55071aa)
• api: panic on run result for docker images (#6599) (8f95af1)
• api: return an error if consumer service cannot be loaded (#6533) (e7b773f)
• api: skip missing integration error while loading workflow (#6526) (17ec69d)
• api: template bulk runner count for parallel apply (#6510) (c6e3eae)
• api: workflow push from repository (#6330) (43a1ba1)
• api: workflow push opts initialization (#6324) (1759beb)
• api: workflow search sort runs (#6556) (ca43679)
• api: workflow template check params typos (#6541) (2f668b7)
• cdn, worker: artifact upload error management and logs (#6419) (2f862d1)
• cdn: delete rom db item that is no more on nfs (#6567) (d0949da)
• cdsctl: clean ouput for template bulk track no-interactive (#6391) (b98dae7)
• contrib: artifactory promote always update artifacts (#6415) (b246692)
• contrib: remove deprecated actions (#6612) (2c8cc0f)
• dependencies: upgrade github.com/apache/thrift (#6514) (f1d244c)
• do not ask service question if there is no service name (#6485) (35cb794)
• do not look into sub directories (#6443) (0d8319e)
• engine: dropped api error (#6445) (a0ed12c)
• hatchery:k8s: watch events routine auto restart (#6539) (cfc9eb4)
• hatchery:openstack: set security groups (#6575) (070d3f7)
• hatchery:swarm: unregister worker cpu/memory metrics (#6470) (76e6363)
• hatchery: priority for models registration (#6414) (8a4dc7c)
• resync job outside tx (#6537) (c1e0661)
• sdk: avoid panic during import pipeline (#6343) (32272d8)
• sdk: avoid panic when parsing empty workflow (#6382) (8de3050)
• sdk: goroutine display stack (#6398) (987162f)
• sdk: hatchery step log delay millisecond (#6454) (7417899)
• sdk: hatchery step log delay numeric (#6456) (38c942a)
• sdk: import project vcs interface (#6309) (007aeae)
• sdk: msg if auth error on git repo (#6435) (6deadd1)
• ui,api: redirect to homepage (#6511) (60ac6ba)
• ui: add /concepts in workflow docs URL (#6464) (b2da040)
• ui: set correct doc link for mutex workflow (#6427) (8a3e6b5)
• worker cmd list all run results (#6460) (21ea1e2)

Features

• always pull image (#6455) (d544832)
• api,cli,ui: disable project key (#6429) (5399721)
• api,cli: admin can update username (#6508) (4b0a3e8)
• api,ui: allow to disable project creation (#6403) (49f878a)
• api,ui: allow to disable vcs management on project (#6408) (0a427bf)
• api: add lock to resync workflow run results (#6364) ([c04cf69](https://github.com/ovh/cds/...

0.52.0

Note about upgrading CDS

You have to install version 0.51.0 or newer if you want to upgrade your CDS Instance from a version < 0.51.0 to this newer version.

When you are upgrading CDS:

• backup your database before any operation
• run the database migration, documentation on https://ovh.github.io/cds/hosting/database/
• after running the API, check if there is no error on migration, with cdsctl admin migration list

Notable Changes in version 0.52

Hatchery marathon

As specified here https://ovh.github.io/cds/hosting/upgrade/migrate_0.51/, the hatchery marathon has been deleted.

Organization

This version generalizes the notion of "Organization" in CDS. You can now set organization on your config file (auth section). This is an optional feature but will be mandatory in the next release.

Organization rules

• In a group, you cannot have users from different organizations
• A project created by a user in an organization "foo", cannot have write permissions for users in organization "bar"
• A workflow created in a project in an organization "foo", cannot have an execute permission for users in organization "bar"

What's Changed

Features

• repositories: add cache size in mon/status #6288
• api: load pipelines with usages #6304
• api: workflow search add ascode filter #6299
• api,cli: add CRUD v2 organization and region #6296
• api,cli: add organization #6289
• api: automatic build info #6234
• api,hooks: add v2 hook signature and github repository analysis #6255
• api: add RBAC import handler #6265
• api: clean workflow run secrets #6263
• api: add worker model and worker model template entities #6268
• api: add repository on project #6206
• hooks,api,cli: add analyze endpoint #6236
• hooks, api: manage entities hook #6219
• elasticsearch: tests and improvements #6286
• cli: add delete workflow run command #6273
• cli: add verbosity to cdsctl admin database commands #6246
• db: Add configurable Redis DbIndex #6271
• sdk: allow to pass modifiers on template apply #6249
• sdk: add helper #6266
• hatchery: add CDN configuration to Worker spawn args #6254
• hatchery: remove marathon hatchery #6305
• worker: read CDN info from config #6258
• deps: bump terser from 4.6.9 to 4.8.1 in /contrib/grpcplugins/action/examples/nodejs #6238
• docker: move to alpine root image #6225
• ui: remove semantic from workflow/notification/workflowAdm #6232
• ui: remove suiPopup #6233
• ui: remove semantic from menu #6241
• ui: remove some semantic modal #6242
• ui: remove consumer SUIModal + last suiPopup #6243
• ui: remove semantic from application and env admin views #6228
• ui: list project entities (app/pip/env/workflow) #6217
• ui: refact keys, variables and run sidebar #6221
• ui: remove sui-progress component + add ngZorro layout #6215
• ui: remove some semanticUI components #6245
• ui: remove some ng2 semantic components #6257
• ui: change icon in app-table #6261
• ui: remove some semantic style #6269
• ui: remove semantic from project admin view and integration view #6224
• ui: remove ngsemantic ascode modal and ng-semantic run workflow modal #6212

Bug Fixes

• doc: explicit exit code in worker exit doc #6227
• docs: remove outdated comparison matrix #6230
• docs: log level doc #6291
• repositories: remove Lock Access in unlockfunc #6295
• ui: load jsonschema && remove some semantic style #6285
• ui: set redirect uri when navigate on protected project #6231
• ui: redirect mfa on project pages #6235
• ui: redirect mfa on home page #6237
• ui: avoid button infinite spinner #6276
• ui: listen change on authenticationState #6220
• api: manage space in action name #6279
• api: improve the error msg returned to UI #6275
• api, worker: auto build info #6264
• api: delete workflow run with artifact manager #6260
• api: run result file info panic #6267
• api: workflow run secret clean interval #6272
• api: disableStatusDetails flag #6256
• api,worker: handle multiple release maturity #6280
• api,ui: load as code pipeline + ascode application #6214
• vcs: missing data when getting hook by its ID. #6292 #6293
• vcs: compute default url for github #6250
• vcs: check auth on vcsType and not url #6226
• sdk: json annotation #6270
• sdk: promote action export entities #6259
• engine: json encryption key rollover #6247
• engine: rollover encrypted map #6248
• contrib: handle multiple releases repositories #6208
• contrib: add os-ansible-inventory #6218

New Contributors

• @marie-j made their first contribution in #6249
• @akosdudas made their first contribution in #6271

Full Changelog: 0.51.0...0.52.0

0.51.0

Note about upgrading CDS

You have to install version 0.50.0 or newer if you want to upgrade your CDS Instance from a version < 0.50.0 to this newer version.

When you are upgrading CDS:

• backup your database before any operation
• run the database migration, documentation on https://ovh.github.io/cds/hosting/database/
• after running the API, check if there is no error on migration, with cdsctl admin migration list

Notable Changes in version 0.51

More info on https://ovh.github.io/cds/hosting/upgrade/migrate_0.51/

Note that the presentation framework is being migrated (from ng-semantic to ng-zero).

Features

• add rbac middleware system (#6103) (fce41f8), closes #6103
• api,cdsctl: remove group and project permission import (#6068) (ab62f66), closes #6068
• api,cli: remove old /mon/errors admin route (#6064) (2b56b04), closes #6064
• api,ui: rm jabb notifications (#6036) (3016fac), closes #6036
• api,vcs: use new vcs project (#6160) (c5cd38f), closes #6160
• api: add checks for workflow groups (#6077) (f260e1e), closes #6077
• api: add pipeline name to event job summary (#6106) (2876a3c), closes #6106
• api: check consumer service definition on signin (#6119) (3f02930), closes #6119
• api: compare hatchery and job region for register (#6156) (152dda5), closes #6156
• api: do not consider service as admin or maintainer (#6121) (3a70613), closes #6121
• api: don't send event to public event integrations (#6066) (008ec54), closes #6066
• api: feed a specific kafka topic with jobs (#6070) (c7080c4), closes #6070
• api: handle RSA key rollover (#6154) (a28b9d2), closes #6154
• api: hatchery config property IgnoreJobWithNoRegion on consumer (#6162) (b2b2dbc), closes #6162
• api: log new session with mfa (#6151) (607c015), closes #6151
• api: maintenance allow worker and service calls (#6163) (386edb1), closes #6163
• api: same checks for project and workflow permissions (#6092) (38e5730), closes #6092
• api: silently remove duplicate hooks (#6116) (5a33878), closes #6116
• cdn,sdk,cdsctl: remove old artifacts engine (#6117) (23e0866), closes #6117
• cdsctl,sdk: manage vcs on projects (#6132) (d3c2b12), closes #6132
• delete ratio service option (#6099) (b4967bd), closes #6099
• docs: add example of ascode artifact_manager integration (#6139) (67d92cb), closes #6139
• go 1.18 (eb47ae1)
• hatchery:k8s: set memory and CPU limits for workers and services (#6187) (2e8f30c), closes #6187
• hatchery:k8s: worker config as secret (#6105) (f0e0d92), closes #6105
• hatchery:openstack: default api is v3 (#6144) (6afd8ae), closes #6144
• sdk: improve luascript performance (#6138) (d9edaeb), closes #6138
• sdk: return error messages on ascode import (#6165) (0c58c4b), closes #6165
• sdk: ternary, urlencode, dirname, basename interpolate helper (#6057) (df7bcf9), closes #6057
• ui,cli: set service (name, type, region) on new consumer (#6118) (a73cbcb), closes #6118
• ui: upgrade from angular 11 to angular 13 (#6082) (88119af), closes #6082
• worker: multiline job variable to one line environment variable (#6071) (59f84be), closes #6071
• worker: show running worker hooks in job spawn info (#6174) (eb4c592), closes #6174

Bug Fixes

• api: check name pattern on postProjectIntegrationHandler (#6169) (9aa776a), closes #6169
• api: do not export encrypted placeholder (#6148) (a50c704), closes #6148
• api: don't change workflow permission for as-code (#6090) (b9151b4), closes #6090
• api: getWorkerModelSecretHandler permission (#6141) (e80eb41), closes #6141
• api: hooks uservice calls getWorkflowRunHandler (#6134) (f9a5b4d), closes #6134
• api: include repo type for run result unicity (#6184) (dc7444b), closes #6184
• api: inherit parent status for join and fork (#6186) (70e3ef9), closes #6186
• api: let hooks call getWorkflowJobHookDetailsHandler (#6143) (7d031e4), closes #6143
• api: load missing fields from app deployment strategies (#6140) (5f4b6c5), closes #6140
• api: new session with mfa log info (#6170) (edb9ff8), closes #6170
• api: raise error if git.branch and git.tag are set (#6104) (bf5d1b7), closes #6104
• api: same run result with different type (#6158) (6e9168d), closes #6158
• api: use final status for pipeline parent parameter (#6164) (7b1dd61), closes #6164
• api: workflow node commits history (#6083) (c679d54), closes #6083
• contrib/artifactory: push build info retry (#6159) (8e3de55), closes #6159
• engine/ui: disable directory listing (#6191) (515b469), closes #6191
• hatchery: log level for hatchery book model (#6190) (a302813), closes #6190
• hatchery:k8s: add delay before pending workers cleanup (#6107) (0030f53), closes #6107
• sdk: ignore invalid notif type (#6153) (156c3cf), closes #6153
• sdk: set worker name max length to 63 characters (#6097) (845efc5), closes #6097
• ui, api: remove import workflow template from URL (#6192) (dfa3c70), closes #6192
• ui: add title on permission tab (#6155) (97a158c), closes #6155
• ui: allow to edit notifications without pipelines (#6111) (3c0a76c), closes #6111
• ui: do not paginate job variables (#6198) ([e73fe81](https://github.co...

0.50.0

Note about upgrading CDS

You have to install version 0.49.0 or newer if you want to upgrade your CDS Instance from a version < 0.49.0 to this newer version.

When you are upgrading CDS:

• backup your database before any operation
• run the database migration, documentation on https://ovh.github.io/cds/hosting/database/
• after running the API, check if there is no error on migration, with cdsctl admin migration list

Notable Changes in version 0.50

More info on https://ovh.github.io/cds/hosting/upgrade/migrate_0.50/

Features

• api: allow variable interpolation in notification recipients (#5946) (9f81302)
• api: delete workflow with dependencies (#5957) (a7a1201)
• api: disable project secrets auto injection for given regions (#6048) (1ac7500)
• api, sql: worker hooks (#5980) (4e320d2)
• api,cli: delete secret from commandline (#5933) (833ebb3)
• api,plugin-kafka: allow set the sarama version (#6029) (3803658)
• artifactory: allow to re run actions (#5913) (5b9b536)
• cdsctl: cdsctl admin database for cdn service (#5930) (5e6911e)
• cli: max number of retries for HTTP calls from env variable (#5934) (f0337e4)
• cli,cdn: add command to migrate artifact into CDN (#5884) (4b6bb08)
• engine: add region requirement on integration (#5932) (0c4b620)
• harchery:swarm: improve stability when sharing host (#6000) (3641057)
• hatchery: allow to disable or change max job start attempts count (#5914) (8dc52d6)
• hatchery,worker: worker start command (#5986) (d9eec0f)
• hatchery:swarm: login on private registry (#5908) (c7f3334)
• sdk: reduce memory allocations (#5909) (c80557c)
• ui: use local fonts (#5966) (c44b723)
• allow lazy binary download (#5915) (d5d094e)
• postgresql 14.0 recommanded (#5990) (37d5efc)
• worker run-result add static-file (#5941) (48aa23d)
• ui,api: workflow v3 preview (#5927) (cf2d21b)

Bug Fixes

• api: always add log fields from auth middleware (#6046) (dc93280)
• api: purge check vcsclient (#6019) (982f2a1)
• api: select requirements from plugin binary depending os/arch (#6017) (fb437e9)
• api: take care of integration model requirements for region check (#6021) (4421b59)
• api: validate username with regex (#6026) (ccfee12)
• api,cdsctl: download artifacts and coverage from artifacts manager (#6041) (139ea40)
• contrib: use markdown as default release note format (#5993) (fd7b0a4)
• doc: see also link to root command (#5994) (fec0611)
• hooks: remove deprecated branch deletion from hooks (#5955) (08ef0d8)
• interpolate: wrong µµµ replacement (#6047) (8d8a5b4)
• docker-compose upgrade (#6043) (8d175a0)
• ui: lint + websocketV3 (#5989) (96f5e87)
• worker: base dir + env variable (#6024) (2da9361)
• worker: executing plugin is failing for windows worker (#6042) (74696a1)
• worker: remove all download artifact code (#6001) (af0a7e3)
• worker: warn for invalid junit file (#6011) (20cd0e1)
• worker: worker set version message typo (#5981) (2df6cc0)
• remove default group (#5956) (a0b430b)
• api: craft workflow run now locks repository (#5943) (a752427)
• api: protect api from bad interpolation sequences from git info (#5944) (ed59ae3)
• api: recursively stop parent run using workflow hook (#5906) (9fee75d)
• api: remove deacklock on database (#5945) (ef9d729)
• cdn: do not compute size from hidden directory (#5883) (b4d99f6)
• cdn: do not stack purge goroutine (#5862) (2f6a6fc)
• cdn: insert item unit on deduplication (#5882) (c7e1148)
• cdn: reduce transaction duration (#5878) (ae86426)
• cdn: update query to use index (#5873) (cf7caf9)
• doc: add missing builtin actions (#5917) (63d48e9)
• engine: add missing index + improve query (#5876) (8e4c7de)
• engine: do not load all branches all the time ([#5900](https://github.com/ovh/cds/issu...

0.49.0

0.49.0 (2021-06-22)

Note about upgrading CDS

You have to install version 0.48.1 or newer if you want to upgrade your CDS Instance from a version < 0.48.1 to this newer version.

When you are upgrading CDS:

• backup your database before any operation
• run the database migration, documentation on https://ovh.github.io/cds/hosting/database/
• after running the API, check if there is no error on migration, with cdsctl admin migration list
• follow the migration guide for CDN service https://ovh.github.io/cds/hosting/upgrade/migrate_0.48/

Notable Changes in version 0.49

CDN service

The release 0.49.0 introduced a new parameter to disable and remove a CDN Unit.

More info about it at https://ovh.github.io/cds/hosting/upgrade/migrate_0.49/

Features

• feat(api, cli, ui): auth consumer token expiration and last authentication (#5822) (2581044), closes #5822
• feat: /mon/status returns details only for maintainer (#5795) (a4e32e5), closes #5795
• feat: add artifactory integration (#5802) (d993fac), closes #5802
• feat(cdn): add route to be able to resync a backend with database (#5829) (6b120b8), closes #5829
• feat(cli): read variable from stdin (#5812) (be677c9), closes #5812
• feat(engine): add builtin action PushBuildInfo (#5824) (7f439a6), closes #5824
• feat(engine): add release action for artifactory (#5849) (095de77), closes #5849
• feat(hatchery): inject env variables into workers from hatchery config (#5806) (70909f2), closes #5806
• feat(sdk): restart goroutines (#5821) (f8d2b30), closes #5821
• feat(service): add ip address in router logs (#5789) (137da27), closes #5789
• feat(ui): check proxy empty config (#5790) (14e10a3), closes #5790
• feat(ui): display run result in artifacts list (#5815) (762f22c), closes #5815
• feat(ui): flag to enable service proxy, filter proxy routes (#5800) (5222d9c), closes #5800
• feat(ui): handle fragment auth callback (#5791) (a232316), closes #5791
• feat(worker): add workflow command add run result (#5805) (521a739), closes #5805
• refactor: rename integration plugin method (#5787) (10eebf1), closes #5787
• refactor: rename release action (#5825) (266f0bc), closes #5825

Bug Fixes

• fix: do not get item from an unsync backend (#5836) (dde3834), closes #5836
• fix: increase time for read an item (#5832) (861df2d), closes #5832
• fix: use wrong unit (#5830) (b2559e4), closes #5830
• fix(api): authentication consumer migration (#5850) (09c1cf0), closes #5850
• fix(api): do not check empty size for artifact (#5801) (368caaf), closes #5801
• fix(api): for retry on timeout with service (#5784) (4d4a2d8), closes #5784
• fix(api): http cookies properties (#5792) (0a1ebf7), closes #5792
• fix(api): missing app secret for git checkout with https (#5833) (df9af61), closes #5833
• fix(api): rename EventIntegration (#5786) (00d5b6f), closes #5786
• fix(api): return more info for error on worker set-version (#5813) (8303faa), closes #5813
• fix(api): return public status with maintenance flag (#5799) (c72c813), closes #5799
• fix(api): set the stacktrace as a field (#5803) (06b65d2), closes #5803
• fix(api): take groups from previous wf (#5797) (c49ef96), closes #5797
• fix(api): update http client between services (#5779) (c029a08), closes #5779
• fix(cdn): add deduplication by item type (#5804) (3348ec6), closes #5804
• fix(cdn): clean old worker cache items (#5856) (a0920e1), closes #5856
• fix(cdn): deduplicate run-result on getItems (#5841) (8d4f75e), closes #5841
• fix(cdn): finfo.Name is always empty (#5828) (c39725e), closes #5828
• fix(cdn): get item unit from buffer while deleting it (#5831) (b8c656e), closes #5831
• fix(cdn): random position (#5842) (a28bde4), closes #5842
• fix(cdn): remove lock + increase delay of buffler clean (#5837) (1f88624), closes #5837
• fix(cdn): select item inside at least n storage_unit (#5847) (b746891), closes #5847
• fix(cdsctl): run with resync (#5819) (43cffbd), closes #5819
• fix(engine): check md5 on artifact download + fix error printing (#5846) (e825a5c), closes #5846
• fix(engine): env ascode with many secrets (#5839) (65c6e95), closes #5839
• fix(engine): upgrade go-git (#5818) (066372c), closes #5818
• fix(hooks): sync outgoing tasks (#5854) (266b47c), closes #5854
• fix(ui): add outgoing ui in ascode workflow (#5782) (e2e9e35), closes #5782
• fix(ui): artifact download links (#5826) (2c2861b), closes #5826
• fix(ui): signin callback redirect only on other pages (#5814) (824c027), closes #5814
• fix(ui): spawninfo polling wait for request end (#5855) (8849693), closes #5855
• fix(ui): update dependencies (#5827) (6d959bf), closes #5827
• fix(worker): improve error message + fix retry on cdnDownloadItem (#5838) (70a24b0), closes #5838
• fix(worker): use no timeout client on cdn upload (#5794) (b7adeeb), closes #5794
• docs: add pipeline parameter in yaml (#5783) (bfe36c6), closes #5783
• docs: disable and remove cdn unit (#5853) (0c1946b), closes #5853
• fix(api,cdn): restart some goroutines (#5852) (12a4efb), closes #5852
• fix(worker,engine): return 1 when command not found (#5851) (2c22655), closes #5851
• Updated plugin-a...

0.48.1

0.48.1 (2021-03-23)

Note about upgrading CDS

You have to install version 0.46.0 or newer if you want to upgrade your CDS Instance from a version < 0.46 to this newer version.

When you are upgrading CDS:

• backup your database before any operation
• run the database migration, documentation on https://ovh.github.io/cds/hosting/database/
• after running the API, check if there is no error on migration, with cdsctl admin migration list
• follow the migration guide for CDN service https://ovh.github.io/cds/hosting/upgrade/migrate_0.48/

Notable Changes in version 0.48

CDN service

The release 0.48.0 introduced a new CDS service called CDN. This service is dedicated to receive and store CDS’s job logs.
We created this service to be able to move out job's logs from CDS database to an object storage provider.

In this release, logs are stored both in CDN storage units and CDS database to facilitate migration. Old log data and database table will be removed in a future release.

More info about CDN service at https://ovh.github.io/cds/docs/components/cdn/.

We also completely rework the UI log page to improve logs readability. Thanks to CDN paginated and streaming apis, rendering logs is now faster and efficient.

Workflow Run retention policy (feature preview)

Workflow run retention policy is now expressed with LUA, this allows to define custom rules for workflows. The default rule will keep a run for one year. More information at https://ovh.github.io/cds/docs/concepts/workflow/retention/.

Features

• api: add has_git_branch variable to purge (#5765) (a7cc2bc)
• api: export workflow purge info (#5769) (bf30f99)
• api: improve error for callback repo manager (#5732) (9a35539)
• api: strict variable usage for lua, check result is bool (#5757) (a239e4a)
• api: template skip empty files (#5763) (3475549)
• cdn: remove storage unit (#5729) (9ff91bd)
• cdn: requires at least one storage that is not cds (#5747) (855efde)
• cdsctl: search by commit instead of run number (#5751) (a6d8fe4)
• cli, cdn: database keys rollover (#5771) (eafe13d)
• hatchery: allow to set specific api url for workers (#5773) (25dda4d)
• sdk: add json lib to lua script (#5774) (aaeea69)
• cdn: disable backend synchronization (#5734) (b50d664)

Bug Fixes

• api, ui: run retention policy with feature flipping (#5755) (6a77e84)
• cdn: clean Incoming item without item unit (#5722) (3dc3293)
• cdn: do not select all database for item unit purge (#5739) (97bc1ef)
• cdn: missing default config panic (#5744) (e388c07)
• cdn,api: check workflow access with id, use workflow name from run (#5753) (28fcaed)
• doc: search links (#5725) (2a0f93d)
• engine: download - check config (#5727) (0f356b4), closes #5716
• hatchery: double check flag need registration (#5752) (7adfafa), closes #5750
• ui: use node pipeline parameters from run's workflow data (#5772) (4f55b1a)
• worker: flush graylog hook logs when run or register ends (#5768) (ab7b0b3)
• worker: reduce httpclient timeout and remove some context (#5735) (c61ab8b)
• change error to have more details (#5724) (25a13f2)

0.48.0

0.48.0 (2021-02-25)

Note about upgrading CDS

You have to install version 0.46.0 or newer if you want to upgrade your CDS Instance from a version < 0.46 to this newer version.

When you are upgrading CDS:

• backup your database before any operation
• run the database migration, documentation on https://ovh.github.io/cds/hosting/database/
• after running the API, check if there is no error on migration, with cdsctl admin migration list
• follow the migration guide for CDN service https://ovh.github.io/cds/hosting/upgrade/migrate_0.48/

Notable Changes in version 0.48.0

Workflow Run retention policy

Workflow run retention policy is now expressed with LUA, this allows to define custom rules for workflows. The default rule will keep a run for one year.
⚠ Be careful, when installing this release the following rule will be applied on all your workflow. Workflow runs that are older than one year will be automatically deleted. You can skip this release and install version 0.48.1 if you don't want to use the retention policy feature. ⚠

If you want to keep Workflow Runs that are older than one year you can edit the default rule in database after running the CDS SQL migrations and before starting CDS API using the following query.

# Example: set default rule to keep Workflow Runs for 2 years
UPDATE workflow SET retention_policy = 'return run_days_before < 730';

CDN service

The release 0.48.0 introduced a new CDS service called CDN. This service is dedicated to receive and store CDS’s job logs.
We created this service to be able to move out job's logs from CDS database to an object storage provider.

In this release, logs are stored both in CDN storage units and CDS database to facilitate migration. Old log data and database table will be removed in a future release.

More info about CDN service at https://ovh.github.io/cds/docs/components/cdn/.

We also completely rework the UI log page to improve logs readability. Thanks to CDN paginated and streaming apis, rendering logs is now faster and efficient.

Features

• api: add authentication and metadata log fields (#5672) (0088472)
• api,cli: new feature to list and check encrypted secrets (#5648) (44eae68)
• cdn: add pprof debug routes (#5531) (3094e35)
• cdsctl: allow manual run a pipeline not launched (#5552) (9069da2)
• cdsctl: pipeline import --force to detach from repo (#5575) (05a32fa)
• engine: support scram-sha-256 authentication (#5690) (365a17a)
• harchery:swarm: improve compatibility with docker windows (#5636) (d1af983)
• hatchery/kubernetes: various improvement (#5679) (ca4786a)
• ui: add button to copy run number (#5640) (843cc5d)
• ui: migrate to angular 11 (#5576) (d2a558a)
• ui,api: allow maintainer to get services status (#5638) (3fc30e4)
• detach fromRepository with force import (#5578) (e56ff40)
• vcs: display gh response (#5561) (5b2fe68)
• api: basic support for openid-connect authentication provider (#5393) (7288e75)
• api: validate yml plugin before import it (#5378) (6beace0), closes #5329
• api,ui: add workflow retention policy (#5474) (1830951)
• cdsctl: aliases cmd (#5376) (7cf81cc)
• cdsctl: export template instances in yaml (#5382) (bcf8355)
• engine: conf insecureSkipVerifyTLS on smtp (#5480) (c8dbc04)
• ui: add filters on monitoring status page (#5456) (e6f0eb1)
• status for all instances (#5437) (679eb5e)
• cdsctl: cdsctl queue stopall (#5401) (5dec37d)
• cdsctl: cdsctl template instances (#5396) (9557901)
• hatchery/k8s: force pull if latest (#5405) (2e409b2)
• ui: serve current doc on /docs (#5364) (df0c712)
• ui: title on delete buttons (#5379) (4072a8c)
• worker,api: allow to override cds.version value (#5365) (d933943)

Bug Fixes

• action/plugin-kubernetes: improve error (#5389) (04dd681)
• api: manage change not found in gerrit (#5514) (9f00a81)
• api: craft workflow runs (#5355) (a111301)
• api: do not clean requirements when job is disabled (#5483) (e1a5c36)
• api: don't load service on repositoriesManagerOAuthCallbackHandler (#5461) (2d8dd4a)
• api: event username should be consumer name for services (#5390) (8295a9d)
• api: import workflow with perms on node (#5451) (283a7fe)
• api: improve display err to user (#5680) (b643bee)
• api: missing workflow context fields for audits (#5562) (e044518)
• api: return more details on template apply error (#5548) (9de27a5)
• api: safely stop engine (#5403) (7ed8d51)
• api: sendmail utf-8 (#5472) (b7205f5), closes #5351
• api: support utf8 in mail subject (#5464) ([c75fecc](https://github.com/ovh...

0.47.0

0.47.0 (2020-07-08)

Note about upgrading CDS

You have to install version 0.46.0 if you want to upgrade your CDS Instance from a version < 0.46 to a newer version.

When you are upgrading CDS:

• backup your database before any operation
• run the database migration, documentation on https://ovh.github.io/cds/hosting/database/
• after running the API, check if there is no error on migration, with cdsctl admin migration list

Notable Changes in version 0.47.x

The release 0.47.x contains a lots of bug fixes & performance improvements.

Websocket is replacing SSE

The CDS UI and hatcheries will now use a Websocket to listen for API events. We also removed the browser shared workers that were used in UI, each CDS tab will use its own Websocket connection.

Remove old secrets and user infos

This release include a migration script to remove some old database tables from CDS. Those tables contains mainly secrets and informations related to user accounts, all that data were migrated in previous release.

Features

• api: config to set timeout and tls config (#5270) (a0a0966)
• api: improve error msg on workflow import (#5288) (d177c8f)
• api: improve router logs fields (#5299) (25787f1)
• api: improve tracing (#5286) (922c076)
• api: remove default region (#5231) (614b834)
• api: return workflow runs URLs (#5220) (e5876a9)
• api,ui: edit application as code in CDS ui (#5246) (69fff58)
• api,ui: enable websocket for all (#5249) (069cee4)
• api,ui: improve error and display request id in ui (#5225) (02c2eab)
• hatcheries: ignoreJobWithNoRegion conf (#5233) (0dc2ef8)
• hatchery: enable websocket (#5199) (00d584c)
• hatchery: openstack max CPUs count and flavor weigth (#5190) (80140d7)
• ui: add ascode environment edition in UI (#5269) (bf1afd3)
• venom plugin: add stop-on-failure parameter to venom plugin (#5274) (114967f)

Bug Fixes

• api: append log when retry job (#5294) (ee08e26)
• api: avoid panic on err msg deleted (#5245) (0d46f8a)
• api: call services retries (#5232) (4678ade)
• api: change json tag (#5226) (f841412)
• api: default env vars should always be merged for docker model (#5293) (1c64809)
• api: do not load all components of project when running workflow (#5285) (b404792)
• api: do not loose logs when api restart (#5298) (10e3e4b)
• api: don't try to load cds.scheduler user (#5239) (37334d6)
• api: jsonschema wrong tag (#5279) (fd95ff9)
• api: load application no-ascode (#5255) (c855935)
• api: lock noderun before updating it (#5300) (92aa63f)
• api: missing group for template when loaded from audit (#5273) (511cb03)
• api: permission on worker that are no more linked to an hatchery (#5303) (dae0ac7)
• api: remove client when close message was send (#5227) (dc2af41)
• api: resync repo managers to compute template default params (#5260) (1e8bc3a)
• api: resync workflow run status when stopping a job (#5275) (c51b3b3)
• api: set hatchery_name default value for worker (#5306) (0651356)
• api: stop ticker + time.sleep to reduce cpu load (#5304) (b64bba5)
• api: update permission for workflow when exists in upsert (#5256) (a0d8409)
• api: worker should not be removed when hatchery is lost (#5296) (84f4ed5)
• api, ui: synchronous ended child workflow + navigation parent/child (#5261) (4c98c9c)
• api, vcs, repositories: repository web hook eventFilter (#5266) (8e83900)
• api,repositories: clean not pushed branch when checkout (#5251) (78d9ac2)
• api,sdk: allow to retry request when conflict due to database lock (#5244) (f549c2f)
• api,ui: return empty app stats and timeline if no elastic service (#5248) (1b83551)
• cdn: add public addresses (#5221) (feb094d)
• cdn: processLogs rollback (#5276) (27fd6a6)
• cdsctl: avoid potential panic (#5253) (a83e1a2)
• cdsctl: worker model list - image field (#5267) (11d37a6)
• engine: keep the main trace.span in context (#5287) (5871741)
• hatchery: avoid panic on queuePolling (#5302) (74e167a)
• hatchery: count CPUs using flavor list instead of data from server (#5254) (90ac6a7)
• hatchery/k8s: enable service requirement (#5292) ([4747e92](https://github.com/ovh/cds/c...

0.46.0

0.46.0 (2020-05-28)

Note about upgrading CDS

You have to install version 0.45.2 if you want to upgrade your CDS Instance from a version < 0.45 to a newer version.

When you are upgrading CDS:

• backup your database before any operation
• run the database migration, documentation on https://ovh.github.io/cds/hosting/database/
• after running the API, check if there is no error on migration, with cdsctl admin migration list
• if there is an error, especially on RefactorApplicationCrypto, you can:
  • cdsctl admin migration list --fields=name,id|grep RefactorApplicationCrypto # to get the ID of migration to reset, then run
  • cdsctl admin migration reset <ID_Number>
  • then restart the API

Notable Changes in version 0.46.x

The release 0.46.x contains a lots of bug fixes & performance improvements.

Rolling key and secret encryption

Like in previous releases, CDS store encrypted secret in database but now the key that is used to encrypt this secrets can be updated. This key is named the rolling key.

See section [api.database.encryptionRollingKeys] and [api.database.signatureRollingKeys] in file configuration.

Workflow ascode from template

You can now use a workflow template with a workflow ascode. When using a template your repository will only contains only one yaml file with a reference to the template that you want to use and the values for all the required template parameters.
You can create this file manually or directly with the CDS command line.

Example of .cds/workflow.yml file with some parameters

name: demo
from: shared.infra/example-with-repository@1
parameters:
  deployWhen: manual
  repo: github/your-org/demo
  ssh_key_name: proj-ssh-key

You will find some template documentation on https://ovh.github.io/cds/docs/concepts/template/.

Version v2.0 in your workflow yml file

You can send a notification on all the pipelines, without having to write the name of each pipeline.
The shortest syntax for workflow with only one pipeline was removed in the v2.
If you have a doubt about the syntax, you can export your workflow from CDS (ui or cdsctl) in v2 to see the good syntax.

Features

• api: add hatchery and worker details in job run (#5101) (11d98eb)
• api: defaultRegion (#5212) (132c980)
• api: get workflows by repository (#5141) (d06e020)
• api: load runs with searchWorkflows (#5201) (8cd9356)
• api: load workflow without its base64 icon (#5140) (8729e39)
• api: search workflow handler (#5175) (ab1bf78)
• workflow ascode from template edit with merge request (#5154) (35db63f)
• api: lock update ascode with template, workflow pull with template (#5134) (0834d56)
• api,cdsctl,ui: workflow template with as code workflow (#5053) (9c88e9e)
• api,ui: websocket server + client for admin (#5128) (7bcdada)
• cdn: use logrus to send worker and service log (#5153) (1a6173f)
• cdn: add metrics (#5181) (2497dfa)
• cdsctl: action usage command (#5025) (320bf76)
• cdsctl: manage workflow label (#5028) (449ef4d), closes #5023
• contrib: select helm version (#5058) (#5066) (e1e4fc3)
• docs: add doc on notification in yaml files (#5110) (e671a3d)
• engine, api: metrics through REST API (#5089) (e7198f8)
• hatchery: region prerequisite (#5151) (05f8a8e)
• hatchery/openstack: check flavor (#5148) (1e25af6)
• hooks: add logs and balance in/out in status (#5131) (7c9d5fc)
• sdk: add version 2 of workflow export entities (#5021) (eb3c114)
• sdk: add WithQueryParameter request modifier (#5139) (06dc738)
• sql: add view on run nodes (#5108) (add4d15)
• ui,api: disable ascode edit for template generated components (#5138) (15a5574)
• compute default values for template parameter (#5092) (0eedf85)
• ui: update loader (#5029) (1de9f1f)

Bug Fixes

• api: action param interpolate (#5107) (661af83)
• api: add a global timeout when checking network requirement (#5070) (53fb4a2)
• api: add new application to ascode workflow (#5152) (8c856e3)
• api: add node run ID on workflowNodeRunEvent (#5167) (896dd46)
• api: Allow to get every service through getServiceHandler (#5173) (21b845b)
• api: allow to stop a workflow run marked as to delete (#5207) (71bb5bf)
• api: application crypto migration with null vcs_strategy (#5135) (c6a8f2c)
• api: avoid error if project cannot be loaded (#5096) (dc5da00)
• api: check and grant user before creating pullrequest or comment (#5080) (ebc811a)
• api: check if workflow exists found before permission (#5084) (b289f6b)
• api: clean duplicate hooks (#5094) (b83d05d)
• api: clean duplicate wnode migration (#5057) (f483fe5)
• api: crypto layer now supports complex types (#5116) (2f7acdc)
• api: default version is 1 for pipeline (#5036) (4077bb8)
• api: don't warn if no previous run was found (#5203) (2c5b0eb)
• api: fix duplicate notif for empty pipeline (d8e7c1b)
• api: fix hooks with empty UUID (#5098) (5386f91)
• api: forbid update with duplicate hooks (#5091) (88936dc)
• api: get integration with clear password from services (#5149) (423fb27)
• api: getProjectsHandler_FilterByRepo (#5136) (382c6a2)
• api: getWorkflowJobQueueHandler filter by group for all hatcheries (#5079) (8a6f308)
• api: header date in mail (#5172) (667e606), closes #5171
• api: ignore error if wkf holder not found when loading pipeline (#5184) ([7491dbd](7491d...