SortingHat: Keep track of which identities I already reviewed

[GeorgLink]

When reviewing an identity to reduce the number of Unknowns, I would like to keep track in SortingHat, which identities I reviewed.

This creates different Unknown stages:

• Unknown Unknown -- I don't yet know if I can find the affiliation because I haven't looked yet.
• Known Unknown -- I checked and was not able to find the affiliation, so it's known to me that this is an unknown.

And then for the affiliated identities, I will have:

• Unknown Affiliations -- The system automatically set an affiliation but it was not confirmed by a human.
• Known Affiliations -- A human verified the affiliation

Further, by keeping track of the date of last review, we can cycle back identities that haven't been checked in several months and may have changed since.

[canasdiaz]

This is very very important to be efficient when improving the identities data set.

Let me use an example: Juan and Emily are working with the same data set. They are improving the identities data set. They have several views on the dashboard to focus on specific sets of users. When they go to SortingHat to enhance the data about the developers with the "Unknown" organization, they don't know whether this was already reviewed by anyone else, so they review all the data.

This situation between Juan and Emily is not efficient. We have a similar issue with our customers. They maintain a list of unknown folks, and from time to time they revisit it and repeat what was already done months ago just because they don't remember a contributor was already reviewed.

[evamillan]

We can add an indicator to show if and when a contributor's profile has been reviewed. A simple approach could be something like this:

Users can mark a profile as reviewed using any criteria, it wouldn't necessarily be linked to affiliations so it could work for more use cases. If there have been any changes to the profile since it was marked as reviewed, eg. an automatic affiliation, we can include a warning. The review date can be refreshed if the profile is checked again.

We can also add filters to the dashboard table to only show identities that have or haven't been reviewed and order them by review date.

@GeorgLink what do you think?

[canasdiaz]

@evamillan It is unclear to me how a profile is set as reviewed with your proposal. Can you please explain that?

[evamillan]

@canasdiaz The idea is to add a button on the user's profile to set it as reviewed (the "Mark as reviewed" button in the example image of my previous comment, but it can look different if it's not clear enough). When you click it, the date would be saved on the database and the profile would show a text or tag with "Last reviewed " (the second line of the example).

[GeorgLink]

I like this idea a lot.

I like the idea of showing the three statuses:

1. "not reviewed"
2. "last reviewed"
3. "last reviewed" + "unreviewed changes"

Would I be able to filter in SortingHat based on these two statuses?

Use Case: One process that we have is to find "Unknowns" in the Affiliations dashboard and go through the contributors from the most active to the least active. Would this new status propagate to the affiliations dashboard? If not, what is the envisioned process?

[sduenas]

We have a first approach to this implementation: chaoss/grimoirelab-sortinghat#923

[dicortazar]

Perhaps the main issue with having a reviewed button is that this may be forever. As an extreme case, you could have one person reviewed 6 years ago. Is this information still valid? For how long does the reviewed flag works?

Can we add perhaps the date when this flag was updated? Or at least have somehow information to give you a perception of how old that review is?

[dicortazar]

@evamillan would it be possible to have a small paragraph updating/adding the new behaviour to the BAP documentation at https://bap.bitergia.com/?

[canasdiaz]

The images are not correctly rendered

[evamillan]

I created this MR to fix the documentation images: https://gitlab.com/bitergium/bm/bap/documentation/-/merge_requests/97

[dicortazar]

I think this task is done. Thanks! Moving this.