Release management

[brevilo]

Introduction

Release management is currently either lacking or sub-optimal, hindering the release and maintenance of BOINC components which affects downstream projects as well as software distributors. This issue is meant to spur discussion and work on a sustainable solution.

State of things:

• Client: error-prone undocumented release process based on SVN practices (see below)
• Manager: based on client release?
• Android: based on client release? Has separate set of related issues (Android release management #1846)
• API: no release management whatsoever
• Library: no release management whatsoever
• Server: no release management whatsoever
• Web: no release management whatsoever

SCM workflow used for client releases

Right now client releases are prepared and maintained in a way that dates back to the SVN-ages. AFAIK, this is how things are done today (please correct me if I'm wrong):

• master is used for all development
• "feature branches" are seldom (mostly for PRs only)
• Development versions/releases happen in master by tagging commits with odd minor numbers, eg: client_release/7.7/7.7.59
• When the current development version appears to have stabilized, a release branch is created off master, eg: client_release/7.8
• Further fixes/improvements are cherry-picked from master into the respective release branch
• Every release gets tagged, eg: client_release/7.8/7.8.0

Disadvantages

1. The BOINC repo is often regarded as a set of seemingly independent files, which doesn't make any sense - this is what SVN did
2. However, when a release gets build the whole repo is effectively used (side-effects!)
3. Therefore the branch used for a build needs to be in a more or less known and stable state (across all components)
4. Development in master precludes this:
   • You can't build because master is broken
   • You mustn't build because master is silently broken but you don't know it
   • If you could build, you don't really know what makes up your build
   • You branch off a broken master because you think it doesn't affect your component, yet it does
5. "Feature branches" and PRs would solve this part of the problem (no direct dev commits to master)
6. Cherry-picking single commits into the release branch is error-prone
   • It's very time consuming (e.g. searching potential dependencies)
   • You are bound to miss relevant commits
   • Non-atomic commits preclude cherry-picking (another issue to be fixed once and for all!)
   • You lose coherence
   • Ergo: don't be stuck in SVN-thinking, use branches and merging
7. Only client releases are covered by this

Conclusion

In order to produce reliable releases we need to optimize the SCM workflow. Git makes this task easy as long as we define a suitable process. My prerequisites would be the following:

• Atomic commits only (don't combine unrelated changes)
• Development in feature branches only (off master), covered by an issue
• Integration of (tested) features in an integration branch (could be master), via PR
• Integration branch could be used for nightly builds (no more explicit development releases?)
• Release branch gets branched of integration branch regularly ("feature freeze")
• Fixes are developed in branches off the respective release branch and merged back into it and the integration branch
• No more cherry-picking for integration purposes, only merges
• Release builds get tagged as usual
• Release branches could be separated for the various components above (where appropriate/useful, the process stays the same)

Proposed solution

One implementation of the above could be GitFlow which is an established best-practice workflow that's even directly supported by various tools and front-ends to streamline the process. Note: this is not to be confused with the "GitHub Flow"!

(to be refined)

[Overtonesinger]

Thanx, @brevilo . Brilliant idea! GitFlow looks really good. I love it!

[AenBleidd]

I suppose this can be closed

CC: @brevilo, @TheAspens

[brevilo]

Yeah, that's probably right. There's a somewhat related follow at BOINC/boinc-policy#17, though.