Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

oai.samvera mapper validation results #498

Closed
christinklez opened this issue Aug 23, 2023 · 3 comments · Fixed by #514
Closed

oai.samvera mapper validation results #498

christinklez opened this issue Aug 23, 2023 · 3 comments · Fixed by #514
Assignees
Labels
data validation Rikolti mapper data validation review needed
Milestone

Comments

@christinklez
Copy link
Collaborator

christinklez commented Aug 23, 2023

See validation reports: #27086 (08-17); #27156 (08-17); #5105 (08-17); #154 (08-17); #159 (08-17)

  • type: Missing data. map type (See 27086 and 27156)
  • language: Missing data. map language (See 27156 and 154)
  • source: Missing data. map source (See 5105, 154, and 159)
  • alternative title: Missing data. map alternative title (See 5105)
  • is_shown_at: Missing data. Map is_shown_at (all collections)
  • is_shown_by: Missing data. Map is_shown_by (all collections)

Also a quick note that collections 27086 and 27156 are coming from UCSC--it looks like they are no longer exposing some fields that we have previously harvested from. We will be reaching out to them to ask about that.

Otherwise, we've verified that all of the mapping requests indicated above are showing up in the vernacular. Thanks!

@christinklez christinklez added this to the #4 CIC work milestone Aug 23, 2023
@lthurston lthurston linked a pull request Sep 18, 2023 that will close this issue
@lthurston
Copy link
Contributor

lthurston commented Sep 18, 2023

@christinklez with regards to type, the values appear to come from a controlled vocabulary. For instance, images are http://id.loc.gov/vocabulary/resourceTypes/img. I'm not seeing these values being mapped to other values in the legacy mapper, but it's happening somewhere. In any case, I can do this value mapping in the samvera mapper, and if we need to move it up a level later, so that other OAI mappers can use it, we can easily do that.

Update: this only applies to SOME of the collections. Others have explicit type values set.

@lthurston
Copy link
Contributor

Further comments re: type. The value is getting mangled by enrichments in some cases, resulting in the value not getting assigned to the final mapped data. One example is 27156, in which the following enrichment converts "image" into [None]:

https://registry.cdlib.org/required-values-from-collection-registry?field=type&mode=overwrite

That's not the only code that touches the type field in the course of mapping. There's an enrichment called enrich-type and some code in the conversion to solr_doc related to type.

I'm able to follow the course of these changes through the mapping and enrichment process, but it's not intuitive to me what or where the actual issue with the mapping of this collection, or what side effects there will be in other mappers if I change code. I think this requires the eyes of someone who knows this project a little better, but I'm happy to make any changes that come out of that review.

The rest of the mapping issues have been addressed.

lthurston added a commit that referenced this issue Sep 19, 2023
@christinklez
Copy link
Collaborator Author

Thank you for these notes, @lthurston !

@christinklez christinklez added the data validation Rikolti mapper data validation review needed label Oct 3, 2023
lthurston added a commit that referenced this issue Oct 5, 2023
lthurston added a commit that referenced this issue Nov 10, 2023
lthurston added a commit that referenced this issue Nov 13, 2023
lthurston added a commit that referenced this issue Nov 14, 2023
lthurston added a commit that referenced this issue Dec 2, 2023
lthurston added a commit that referenced this issue Dec 2, 2023
amywieliczka pushed a commit that referenced this issue Dec 20, 2023
amywieliczka pushed a commit that referenced this issue Dec 20, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
data validation Rikolti mapper data validation review needed
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants