Scoring for code audit: auditor quality

[drinkcoffee]

In the PR for code audit scoring, there was a discussion around code audit quality. This should be addressed as a follow-up PR.

@ermyas said:
I agree that determining the quality of an audit firm can be somewhat subjective. However, I fear that equating the quality of audits from established players with those from new companies with little to no track record might conceal an important risk signal from our scoring. While we don't have to do it in this PR, I would suggest we think about a way to at least distinguish established from novice players based on their public track record.

@drinkcoffee said:
Could we have a "quality" factor:
Audit firm has done fewer than 10 audits: Score 1.
Audit firm has done between 10 and fewer than 100 audits: Score 2.
Audit firm has done 100 or more audits: Score 3.

Or something like that?

@ermyas said:

This looks ok to me for now. A few thoughts below:

I think that the credibility of an audit largely depends on the calibre of the auditor. Hence, we should perhaps weigh this factor a bit more. Having only a two-point difference between entities that have done thousands of audits vs those that are novices might not capture the importance of this metric well.

It might be useful to capture whether the auditor has experience auditing cross-chain protocols, which might suggest better familiarity with crosschain architecture, common security pitfalls etc.

It might be useful to capture whether the scope of the audit includes execution environments, runtimes or language that the auditor does not have experience in (e.g. fewer auditors might have experience in non-EVM and non-Solidity environments)

Of course, I don't think we should try and capture every consideration in these risk scores, only the most salient ones, and where to draw the line in this regard can be tricky.

I am not opposed to starting with the current proposal and iterating at a later point.

On the point assignment specifically: If there are multiple auditors, do we average out the quality scores for each auditor or just consider the score of the most experienced?