Consider using the OpenSFF Scorecard

[danshearer]

This issue originally by @scp93ch .

From the Scorecard GH project:

What is Scorecard?

We created Scorecard to help open source maintainers improve their security best practices and to help open source consumers judge whether their dependencies are safe. Scorecard is an automated tool that assesses a number of important heuristics ("checks") associated with software security and assigns each check a score of 0-10. You can use these scores to understand specific areas to improve in order to strengthen the security posture of your project. You can also assess the risks that dependencies introduce, and make informed decisions about accepting these risks, evaluating alternative solutions, or working with the maintainers to make improvements.

[danshearer]

OpenSFF might be useful or a gimmick, but it seems we can get them to do all the work to try it out at no risk to us. It is one of the Linux Foundation's efforts to improve supplychain quality.

"Risk to us" would include things like the potential problem of introducing a dependency to check if our dependencies are good, and, potentially adding a GitHub Action which is something we want to be pretty cautious about as previously discussed.

But here's how we can seemingly get OpenSFF to do regular runs of system-modeller, from https://github.com/ossf/scorecard :

The list of projects that are checked is available in the cron/internal/data/projects.csv file in this repository. If you would like us to track more, please feel free to send a Pull Request with others. Currently, this list is derived from projects hosted on GitHub ONLY. We do plan to expand them in near future to account for projects hosted on other source control systems.

I looked at that file:

$ wc  -l /tmp/projects.csv
1276587 /tmp/projects.csv 
$ grep -i wordpress\/wordpress /tmp/projects.csv
github.com/WordPress/WordPress,criticality_score:0.722220

So they seemingly scan 1.3 million projects themselves, and require only a GitHub repo and some optional metadata (I chose WordPress because it is not renowned for tight security.) The results are in a useful dataset for online and offline use, and they also provide an online viewer. Here is the Viewer output for WordPress.

Therefore I will submit a pull request for system-modeller to be added to this giant file of projects, and we can inspect the results. I have no idea what the metadata values are yet, but since they can be omitted we'll start with that.

If it useful and sufficient, then great, we are using OpenSFF. If its promising but for some reason we need to run it ourselves, then we can look at the costs and benefits.

[danshearer]

Done: ossf/scorecard#4072 .

[danshearer]

Actually not done, I need to change the format and resubmit.