Use enhanced PurlDB "package set" lookup in d2d pipeline

[pombredanne]

No description provided.

[JonoYang]

@pombredanne

We would like some clarification about how we should be using package sets during the d2d process. Would this be a new step in the d2d pipeline where, after the purdb matching step, we request the enhanced package data for the packages that we matched to? Or would this be a new endpoint where we would be calling to get package data when we are in the matching process?

Tushar suggests that we should return the enhanced package data whenever we query for a package by sha1 from the packages endpoint to skip the extra API call.

[pombredanne]

We would like some clarification about how we should be using package sets during the d2d process. Would this be a new step in the d2d pipeline where, after the purdb matching step, we request the enhanced package data for the packages that we matched to? Or would this be a new endpoint where we would be calling to get package data when we are in the matching process?

Tushar suggests that we should return the enhanced package data whenever we query for a package by sha1 from the packages endpoint to skip the extra API call.

I would likely see this as a single step, e.g., the standard end point to get data for a purl or to get a match and it would have an extra argument/option flag to return enhanced data. This would mean that in these cases that the "enhanced" query (likely an extra call) would done when required server-side.

An alternative could be to do it at a later stage at once for a bunch of packages, but the weird thing would be that we could have stored a package first with some data and later with enhanced data which seems a little odd.

[TG1999]

Done in aboutcode-org/purldb#157