Update ScanCode Toolkit version

[qtomlinson]

In the scancode result, warning is reported:

"OUTDATED": "WARNING: Outdated ScanCode Toolkit version! You are using an outdated version of ScanCode Toolkit: 30.1.0 released on: 2021-09-24. A new version is available with important improvements including bug and security fixes, updated license, copyright and package detection, and improved scanning accuracy. Please download and install the latest version of ScanCode. Visit https://github.com/nexB/scancode-toolkit/releases for details."

[elrayle]

ClearlyDefined/crawler is 2 major versions behind on nexB/scancode-toolkit (referred to generically as scancode). There have been output format changes in at least one of the updates. ClearlyDefined/service processes raw tool output and needs to be updated to work with the latest scancode output format.

Related Work

• explore scancode.io as a replacement for including scancode-toolkit as a library #527
• update processing of scancode output to work with the latest version service#1006

[qtomlinson]

Changes needed:
Crawler:

• Command-line options in config/cdConfig.js
• Dockerfile and DevDockerfile

Service:

• providers/summary/scancode.js needs to be modified to extract the ScanCode output to the summary.

Test cases:

• Cases in point 1. These licenses have been verified.
• Another interesting case to cover is when the license from the package summary differs from the license from the root files. For example, composer/packagist/mmucklo/krumo/0.7.0 (see definitionServiceTest).

Other considerations:

• Memory requirements for the newer version compared to the version in use. Is any adjustment to the configuration of our infrastructure necessary?

[lumaxis]

Thank you @qtomlinson! I started looking into this and will report back with any findings 🙂

[qtomlinson]

@lumaxis For cases where the license of the package differs from the license of the root files (e.g., composer/packagist/mmucklo/krumo/0.7.0), it is interesting to discover the new behavior, which may be different or better than what we currently have.