Automate verification of build results

[dexX7]

It would be awesome, if we had a way to automate the build result verification, for example via Travis CI.

The process would roughly look like this:

1. Clone gitian.sigs
2. Clone gitian-builder
3. Clone omnicore
4. Find build results
5. Extract meta data
6. Checkout related commit
7. Verify results

There are a few issues though:

• extracting and handling the data is a bit tricky, at least for me
• all GPG keys must be available, otherwise the verification fails
• we could add the keys manually (we should publish them somewhere anyway)
• but submissions from new users would fail

I hacked together a Python script: https://gist.github.com/dexX7/4fed3b3e1177097110c5

It can be used as follows (all steps already possible right now):

# one time setup
sudo pip install pyyaml
gpg --recv-keys 0xA6308A6C5B1E67CC7CF7002FBCC02B71BE91B32B 0xF43718054C3E7C5CFB33E8257675E31CF5719832

# project specific setup
PATH_GITIAN=/tmp/verifytest/gitian-builder
PATH_RESULTS=/tmp/verifytest/gitian.sigs
PATH_PROJECT=/tmp/verifytest/omnicore

git clone https://github.com/devrandom/gitian-builder.git $PATH_GITIAN
git clone https://github.com/OmniLayer/gitian.sigs.git $PATH_RESULTS
git clone https://github.com/OmniLayer/omnicore.git $PATH_PROJECT

# get script
wget https://gist.githubusercontent.com/dexX7/4fed3b3e1177097110c5/raw/a0b99320c6aab2f0e467aba67c289c569985ba1c/gveryify_all.py

# ... and go!
python gveryify_all.py $PATH_GITIAN/bin/gverify $PATH_RESULTS $PATH_PROJECT

Why all this? Because it takes of some load, and because I'm obviously not reliable. :/

It's probably also useful, if we start to publish nightly builds, because then we might fire up two build bots, and if their results match, the binaries are published, etc..

I'm going to integrate the script, once all results of this repo match, but it would be great, if you have some input regarding the points I raised above.

[zathras-crypto]

This is really, really cool :)

So when would this run? After we publish our results?

It's probably also useful, if we start to publish nightly builds, because then we might fire up two build bots, and if their results match, the binaries are published, etc..

I'm a little unsure here - what would be the purpose of nightlies? Facilitating more testing? I'm worried end users could end up using them...

[dexX7]

So when would this run? After we publish our results?

Currently the approach would be: whenever there is a new PR or a merge, Travis CI runs the script and verifies all results currently in the repo (either all that there are, or all there are + the ones submitted via the PR). I'm going to take a look at the integration via Travis within the next days, but I assume it's pretty similar to what I posted in the code block above.

I'm a little unsure here - what would be the purpose of nightlies? Facilitating more testing? I'm worried end users could end up using them...

That's an interesting quesiton. How would you currently describe the purpose of "developer previews"? In the end I'm mostly looking for more automation, whether this includes the build process, verification or publishing of results; I haven't really thought about actual use-cases to be honest. :)