T358307 - Update Release Variables Script

[RickiJay-WMDE]

Updates the commit hashes in variables.env, specifically from Gerrit, GitHub, and Bitbucket. The first, it parses the relevant webpage using BeautifulSoup; the latter two, it pulls the relevant data from their respective APIs.

While I did need to tweak some of the urls provided for consistency (and accuracy, in the case of WDQS GUI - the link was broken), I did not commit variables.env updated hashes, so that devs can test the script.

Notes:

• Mediawiki extensions include the compatible version of Mediawiki in the URL. I have the script fetch this from the MEDIAWIKI_VERSION variable and automatically update the commented URLs to the correct version.
• As mentioned, for both GitHub and Bitbucket, the script pulls data from their respective APIs. The base URLs for each link are changed to https://api.github.com/repos/ and https://bitbucket.org/!api/2.0/repositories/ respectively. I didn't make this change in variables.env, because I assumed we wanted that file to link to human-readable pages rather than API output.

[lorenjohnson]

Looks great, with one caveat. Yes, effectively currently our only dev dependency is Docker. Probably should stick with that unless there is compelling reason not to. See here for how we currently run your other Python script, and apply that here?:

wikibase-release-pipeline/lint.sh

Lines 43 to 45 in f8a4ea7

	$TEST_COMPOSE run --rm --build -v "$(pwd):/tmp" test-runner -c "
	python3 scripts/add_newline.py /tmp '$MY_FILES' $PYTHON_FLAGS
	"

Perhaps add a "update_hashes" task to the build.sh script to run the Python script in the test (or build) container?

[rti]

Haha. Great work. Nice, thanks a lot.

I think the automatic Gerrit commit updating can be easily done like this as the REL branches ensure compatibility. No doubt there.

For the others, I would like to have an explicit "review the changes" step in our release process, as the maintainers of these repositories give no guarantees at all (AFAIK). So I frighten having this step automatic could lead to accidental breaking changes. Or can we just assume: "What the test suite does not catch is not breaking"?

[lorenjohnson]

Great! ✨