Implement GitHub App authentication

[Kobzol]

This means that the rustbot account will no longer be sending the triagebot comments, which can have... consequences 😆 Opening as a draft for now.

[ehuss]

Is there a general overview of what will be required? Some questions:

• Does someone need to create the App, or is one already created? What's its name?
• Is the App installed at the org-level? Does it need to be installed in every repo?
• What permissions does the App need?
• How slow is generating the access token? (Just wondering how it affects our 10s response window.)
• What concerns do you have if it posts under the new App account?
• For posterity, we should probably explain why we are switching to an App. I assume it is for improved security? Is the short-lived token the main thing? Are we changing permission scopes?
• Does the rate limit stay the same? I seem to recall that App limits are complex (but should be the same or better than a PAT?).
• What else might go wrong?

Can we add a little more detail to "How triagebot works" in the README? Something I have neglected to do is fill that out more¹, and I don't want to get further behind. Just briefly explain that the production server runs as a GitHub App, and how the app is installed (org-wide?). And reiterate it doesn't have to run as an App and can run with a PAT.

Footnotes

1. For context, the thing that is missing that I've meant to include is the deploy documentation. From building the image to ECS deployment. I always forget how ECS deployment, shutdown, draining, restart, and management works. ↩

[Kobzol]

I'll answer all these, just wanted to wait for the discussion in https://rust-lang.zulipchat.com/#narrow/channel/242791-t-infra/topic/Triagebot.20GitHub.20App.20migration/with/573317366 to finish, because right now it's not 100% clear that we can even technically make the switch (though probably we should be able to).

[Kobzol]

Looks like we should be able to use the app, though it will require some other infra work, so it will take some time. So the promised answers:

Does someone need to create the App, or is one already created? What's its name?

Yes, it needs to be managed by infra admins. That being said, we found out that we already have an existing rust-triage app, so if the name is considered to be fine, we can just use that.

Is the App installed at the org-level? Does it need to be installed in every repo?

Both are possible. Since we already configure rustbot for repositories explicitly in team, we can make it granular and enable it only for specific repositories. When a GH app is enabled globally per org, it also works with private repositories, which is a bit of an unexpected edge case (rustbot can't write to private repositories today, though maybe it receives webhooks for them?).

What permissions does the App need?

We can add it very granular permissions, unlike the current write permission of the rustbot app. So we'd give it the ability to modify issues and PRs.. and that's more or less it, I think.

How slow is generating the access token? (Just wondering how it affects our 10s response window.)

Locally it takes ~300-400ms, just a network request. The token lasts for an hour and it is cached by octocrab, so I wouldn't be worried about that.

What concerns do you have if it posts under the new App account?

None, really. Though comments will no longer be posted by the rustbot account directly, it will look like this instead, the account will have the bot marker next to it.

For posterity, we should probably explain why we are switching to an App. I assume it is for improved security? Is the short-lived token the main thing? Are we changing permission scopes?

Security is the main reason, yeah. We can assign fine-grained permissions to the app, rather than just using a catch-all write access (for example triagebot is now able to push to repositories that don't have branch protections scoped to specific users or teams). The token will also be short lived, which is in general better, and the account will clearly be marked as a bot. Also, PATs were deprecated by GitHub some time ago, and it seems harder and harder to use them, e.g. we now can't even see them in the GitHub UI on the enterprise level, IIRC. I'll let @marcoieni chime in, in case you want to add something.

Does the rate limit stay the same? I seem to recall that App limits are complex (but should be the same or better than a PAT?).

The base rate limit should be the same as for PATs, so 5k reqs/hour (https://docs.github.com/en/rest/using-the-rest-api/rate-limits-for-the-rest-api?apiVersion=2022-11-28#primary-rate-limit-for-github-app-installations). In fact, it can be even higher than for PATs (up to 12.5k or even 15k/hour).

What else might go wrong?

It's infrastructure, so pretty much anything, really 😆 We should ensure that PATs still continue work for local experiments and debugging, but that should already be the case in this PR. That being said, the exact same approach is used in bors and team, and we haven't really ran into any issues related to GitHub Apps so far.

Can we add a little more detail to "How triagebot works" in the README? Something I have neglected to do is fill that out more1, and I don't want to get further behind. Just briefly explain that the production server runs as a GitHub App, and how the app is installed (org-wide?). And reiterate it doesn't have to run as an App and can run with a PAT.

Sure, I'll do that once we're sure that it is possible to use a GH app for triagebot.

[ehuss]

rustbot can't write to private repositories today, though maybe it receives webhooks for them?

Yes, I believe that is true. I would guess that it should be able to write to private if it is added to the bot list, but none of them are.