update processing of scancode output to work with the latest version

[elrayle]

Description

Starting with nexB/scancode-toolkit/v31.0.0, the format of the output of scancode has changed. Any part of the service that processes scancode output will need to be updated to work with the new format.

Reference:

• Calculating Definition Scores

Questions:

• Do past output formats need to also be supported? Use Case: After scancode output has been processed, does it remain in ClearlyDefined data and get reprocessed at a later date?

Related Work

• Update ScanCode Toolkit version crawler#502
• explore scancode.io as a replacement for including scancode-toolkit as a library crawler#527

[qtomlinson]

@elrayle The past output formats still need to be supported. The scan results from older versions of scancode in the previously harvested raw files are still used by the summarizers to compute definition. The ScanCodeSummarizer will require changes to process the newer versioned scancode result.

[qtomlinson]

See #999 (comment) for details on improvements in the latest version of ScanCode.