Improve the risk score section

[ermyas]

This issue raises areas of improvement for the risk scoring section, that we can discuss and address progressively:

• What is our approach to the risk scoring section? Do we try and score all sources of risk we identify or just the most significant ones? My 2 cents is that we do the latter, while still leaving flexibility for others to extend the core risk scoring model for their own purposes.
• The magnitude of risk we assign needs to be consistent and comparable across different sources of risk. Currently, we have a few instances where things that are stronger indicators of risk have a lower risk score than those that are weaker signals of risk.
• How we aggregate the risk scores will likely need to change along the lines of the feedback we got during our recent talk.
• The set of questions we have for some risk indicators might require important nuance and expansion (e.g. audit score)

[chen-robert]

@ermyas Definitely agree with the need for more nuance. For example, the "formal verification risk score" seems quite vague and I'd argue it should be removed or clarified heavily. Does having a single specification count as "formally verified"? How much of the protocol needs to be formally modeled before the binary flips from no -> yes?

For the code auditing risk score, I feel like it should take into account some qualitative factors such as (some of which have been discussed in #91):

1. I'd argue at least some general reputation of the audit firm should be taken into account. Maybe this could be a binary such as a "generally reputable audit firm" with low requirements to avoid controversy, but having no requirements at all feels abusable.
2. Consider a tighter requirement for I1013 -- I'd argue all vulnerabilities should at least be acknowledged/addressed

On the vulnerability response score, there's some duplication. For example, O504 overlaps with I1102.

[ermyas]

Thanks for your feedback @chen-robert.

I agree with the points you've raised.

As mentioned above, I think it is better to have an approach that focuses on the most salient risk indicators for the scoring section while aiming to be comprehensive in our coverage of risk in other sections. I think that "formal verification" is an example of this. We can remove it from the scoring section and expand on the discussions we have about it in the implementation risk section.

For I1013, I would suggest we address it by adding a new score that captures the point you raised. That is, continue to have a score about whether major and critical issues have been addressed, and add another about whether all other issues have been mitigated or acknowledged.

Thanks for pointing out the duplication in the vulnerability score section. I hadn't caught that.

We'd very much welcome a PR addressing these if you're able to contribute. If not, I will address these and more over the next couple of days.

@drinkcoffee any thoughts on the points above?

[drinkcoffee]

There is an open PR that updates the scoring for network issues. I suggest others comment / make improvements to that PR. I am busy on another project at the moment and won't get to updating it for some days.