Reduce noise for in-range dependency updates #22206
Replies: 9 comments
-
you can use branch automerge feature, so renovate will push a branch with update(s), if ci succeeds and branch is exactly one commit ahead, it will merge and push directly to target branch, so no pr required. also with default |
Beta Was this translation helpful? Give feedback.
-
the branch auto merge feature does not work when branch protection is enabled |
Beta Was this translation helpful? Give feedback.
-
I've helped @gr2m and the Here's their current Hi @gr2m, nice to see you here. 👋 😄 |
Beta Was this translation helpful? Give feedback.
-
@gr2m I observed Gatekeeper's approach and found it caused a lot of unnecessary noise in its own way, especially due to false positives. One additional problem with implementing it in the Renovate context is that it would require state, and mo' state mo' problems. You wrote:
Unfortunately, that's where the simple ends. Because deleted branches disappear (unlike closed PRs, of which there's a record), it means you can't query the repository/project to know what's been done before, so you need to store it somewhere else. Then sometimes state gets out of sync and you end up spamming - their - repo. You also wrote:
This part can already be done. Renovate by default won't raise PRs for in-range updates unless they're security updates. And if you enable lockFileMaintenance then you can get a weekly lock file refresh. I give a thumbs-up to this approach. What you won't get from this approach is the continuous testing of new releases and Greenkeeper-style issues created as soon as tests fail. It's my belief:
To attempt to add some metrics to my gut feel on this, I checked the greenkeeper repo itself. I count 22 times the bot thought the Greenkeeper repo itself was breaking. 10 of them remain open and un-actioned to this day. Of the other 12, they were mostly ignored and closed, but sometimes explicitly noted as false positives: Examples: I could not observe a single one of those 22 instances where the bot's issue caused the team members to make any change to the project dependencies. In conclusion, I still don't like the idea of this feature. Here's my rationale:
|
Beta Was this translation helpful? Give feedback.
-
I cannot argue with that :)
I don't agree here, it's not Greenkeepers fault. If your CI tests are unreliable, you have a whole set of problems, false positives by Greenkeeper's background checks was just a symptom. The difference would also be that the background check was default behavior for Greenkeeper, but would be opt-in for Renovate.
I did it several times and the value was very high in these cases. It's not about the instances of when a warning about a broken test from an in-range CI test resulted in pinning a dependency, but about about the amount of dependents and the impact on them. to summarize, I would appreciate the feature for my own libraries, which usually have very reliable testing and 100% coverage, in order lower the barrier to accept contributions. But if the current architecture does not keep the necessary state then I wouldn't add it just to cover this particular use case. How about a different approach: renovate could trigger repository dispatch events in case there is an update for a dependency, and leave the handling of that information up to the user? I could build my own action/app that implements the background check and share it with other renovate users, and enable an ecosystem of renovate-powered actions/apps that would use it as a means to act on npm registry events |
Beta Was this translation helpful? Give feedback.
-
Do I understand this right? Although you didn't pin or downgrade, it was helpful to be aware that people downstream using a fresh install of your package might be broken for some period? And if so, does that mean that the concern/threat is over once a working release has been published, or the non-working release has been pulled? If "noise" wasn't a concern then couldn't you satisfy that today using the update-lockfile strategy? i.e. Renovate would bump the version in the lock file each time a new release is out, which triggers a build, and typically would be green and available for merge (including automatic). Alternatively - a feature not supported today but pretty easy to add - Renovate could create branches to bump lock files and trigger tests, and only raise a PR if they fail. So normal scenario is that you might have 5-10 branches and no PRs but occasionally one branch will be updated and then raised as a PR if testing fails. You could still get a wholesale lock file maintenance PR ~weekly or whenever you want. Repository dispatch events would have the same state challenge btw - Renovate would need a way to prevent duplicate notifications for the same version. |
Beta Was this translation helpful? Give feedback.
-
I did pin until the problem was resolved. The ideal flow would be:
Yeah that makes sense. If GitHub's API would return the information if a branch existed for a given name, e.g. by returning |
Beta Was this translation helpful? Give feedback.
-
@rarkins I'm not sure if you actually want to implement this feature or not? The general feeling I'm getting from your posts is that doing this would lead to "more state more problems", and that you didn't like the idea of it, but you might have changed your mind or found an easier way to solve the problem?
|
Beta Was this translation helpful? Give feedback.
-
If it's a valid dependency update problem, we'd like to solve it. Sometimes difficult problems get an elegant solution eventually, once the code base progresses or maybe we just think about it more. I'll leave it open so that it's not forgotten and others can add ideas or sentiment. |
Beta Was this translation helpful? Give feedback.
-
follow up to https://mobile.twitter.com/rarkins/status/1330928203343548424
What would you like Renovate to be able to do?
I'm migrating some of the Open Source projects I maintain to Renovate. A major concern is to limit the noise from dependency update pull requests, to reduce our maintenance burden, and more importantly, to not put off potential contributors who start watching the repository.
Did you already have any implementation ideas?
Greenkeeper had a great feature to address that problem: silent check for in-range updates. It would create a branch for the in-range update and await the CI checks. If all checks passed, the branch would simply be deleted. If the build fail, an issue would be created to inform the maintainers about a potential breaking change.
That means pull requests would only be raised for out-of range updates (as well as security updates). A weekly *.lock file update would make sure that there are no problems coming from the combination of multiple in-range updates.
Beta Was this translation helpful? Give feedback.
All reactions