Pulling Geology data from APIs

[Nicole-Ridgwell-NMMNHS]

We've discussed in issues ways that we can utilize APIs to help manage our geology data. For example: see #2244. I was notified of this new api script for Geolex (https://github.com/ben-norton/HarvestGeolex) which seems useful, and so I thought it might be helpful to have a general discussion on the best way to take advantage of these resources. Can we use resources like Macrostrat and Geolex to manage at least our lithostratigraphic data? What would that look like and how would we deal with the obscure but valid units that haven't made it into these resources?

[Jegelewicz]

We need to figure this out - for all kinds of things - and stop making our own code tables.

how would we deal with the obscure but valid units that haven't made it into these resources

How do we get them into those resources? If we are making active use of their data in a very public way, it seems like it would be a great argument to them to add our obscure resources. But we also have to be community minded with them - will our obscure thing make their resources better or will it contribute to messes and instability? If our resources are too obscure - then we need a different place to put them, separate from the curated resources that Macrostrat, Geolex, WoRMS, PBDB, and others can provide.

[dustymc]

manage

Yes, I think, but it's complicated - moving all code tables to some external source would be much easier than moving a code table or two. Still, I could figure out some sort of hybrid "if fail then try insert and check again" approach or something (and that might still be scalable to anything with a webservice, but probably as a general model rather than reusing code).

(Or what @Jegelewicz said...)

how would we deal with the obscure but valid units that haven't made it into these resources?

MAYBE that would be somehow possible in a complicated hybrid model, but the general answer is that you can't - if the external resource won't work with you on things like that, then they're probably not capable of acting as an authority.

A big concern on these kinds of things would be stability. In this case the github thing seems to just be compiling CSV from https://ngmdb.usgs.gov/connect/apiv1/geolex/units/3/, so we could go straight to the source.

will our obscure thing make their resources better or will it contribute to messes and instability?

If they're willing to accept the latter they're probably not something we want acting as an authority....

different place to put them

For at least some things (taxonomy and Attributes), that's built in to Arctos. Whatever's not appropriate for https://arctos.database.museum/info/ctDocumentation.cfm?table=ctchronostrat_stage_age (maybe according to some API or whatever) can be entered into https://arctos.database.museum/info/ctDocumentation.cfm?table=ctchronostrat_informal. Those aren't quite equal and I'd not see that sort of thing as a workaround to having a good relationship with some external authority, but it's probably OK for dealing with the fringe cases.