#271 LUNA16 evaluation

[vessemer]

This allows to compute FROC and CPM metrics with bootstrapping for the classification and detection tasks.

Description

Free-Response Receiver Operating Characteristic (FROC) and competition performance metric (CPM). It computes an average of the seven sensitivities measured at several false positives per scan (FPPS) thresholds, more concretely, at each FPPS ∈ {0.125, 0.25, 0.5, 1, 2, 4, 8} true positive rate was computed. Mean of which forms the CPM as discussed in #271.

Reference to official issue

This reference to the #271.

How Has This Been Tested?

CPM over 10-Fold cross validation:

0.125	0.25	0.5	1	2	4	8	Score (CPM)
0.595	0.670	0.731	0.793	0.835	0.868	0.887	0.76

CLA

• I have signed the CLA; if other committers are in the commit history, they have signed the CLA as well

[WGierke]

Nice! You mentioned that you used 10-fold cross validation. Can you document somewhere how you trained and evaluated the model using your newly introduced methods?

[vessemer]

I've used my old pipeline since it fits my currently available one TITAN X GPU, though the pipeline code in my repo looks not that good :)

[vessemer]

Three months ago I've worked on algorithms unification and training parts, but then master branch diverges from mine and I abandoned that idea for a while. The code I've written lies here. I hope to return to this soon.

[vessemer]

Btw, for 10 fold cross-validation was used split from the data page suggested by the LUNA16 authors.

[lamby]

This is cool!