Handle Duplicate Keys in package.json and pnpm-lock Files

[zahidblackduck]

Ticket

IDETECT-4594

Summary

This Merge Request resolves the issue where Detect fails to handle package.json and pnpm-lock files containing duplicate keys, causing a JsonSyntaxException. The proposed solution sanitizes the JSON files by removing duplicate keys, retaining the last occurrence of each key. This ensures Detect can gracefully handle duplicate keys without scan failures.

Details of the Fix

• A new utility class JsonSanitizer has been introduced.

  • The sanitize(String json) method uses Gson's JsonParser.parseString(json).getAsJsonObject() to remove duplicate keys and returns a sanitized JSON string.
  • This approach leverages the behavior of Gson's JsonObject, which uses a LinkedTreeMap to silently overwrite duplicate keys, keeping the last key-value pair.

• The sanitize() method is called in the following locations:

  • PackageJsonReader class: Before passing JSON content to the gson.fromJson() method during deserialization.
  • CombinedPackageJsonExtractor class: To handle JSON data processing that also involves the gson.fromJson() method.

• By sanitizing the JSON before parsing, Detect ensures that duplicate keys are removed while retaining valid JSON syntax. Invalid JSON files will still result in a JsonSyntaxException.