Add new vocabs API

[edmondchuc]

Closes #131.

Summary:

• New API for client fetching of concept schemes and concepts
• Prefix generation is set by default on startup
• Add identifier endpoints to provide resolution for IRIs to CURIEs and vice versa
• Add parametrised tests

New API for client fetching of concept schemes and concepts

Adds a new set of APIs as described in the linked issue above to enable the ability to fetch a vocabulary of any size in an incremental manner.

Prefix generation is set by default on startup

This PR creates prefixes by default on start up and can be disabled by setting the DISABLE_PREFIX_GENERATION environment variable to false. This default behaviour is required as the new API assumes the endpoints are stateless with no side effects.

Add identifier endpoints to provide resolution for IRIs to CURIEs and vice versa

New identifier endpoints to provide resolution of IRIs and CURIEs.

• /identifier/curie/{iri:path}
  • With an IRI as input, return its identifier in the form of a CURIE.
• /identifier/iri/{curie}
  • With a CURIE as input, return its fully qualified IRI.

These are required to set up parameterised tests by converting IRI inputs into their CURIE identifiers to perform requests to the new vocabulary endpoints. For example, converting parameterised IRIs to perform test requests to /v/vocab/{concept_scheme_curie}/top-concepts.

Add parametrised tests

Basic structure of parameterised tests for each new vocabulary endpoint added with different scenarios tested. New scenarios can be added in the form of another parameterised test.

[jamiefeiss]

Looks good so far, works as advertised locally against the IDN triplestore.

One thing to note though, testing the /identifier/curie/<iri> & /identifier/iri/<curie> endpoints cause internal server errors when providing invalid iris or curies. These cases should be catered for and have appropriate HTTP error responses in the API.

[edmondchuc]

Thanks for pointing that out @jamiefeiss about the error handling for invalid values on the /identifier/curie/<iri> and /identifier/iri/<curie> endpoints. I've added error handling and tests for them in e13e596.

[edmondchuc]

Won't merge this until prez-ui has an implementation to consume this new API as described in RDFLib/prez-ui#84.

[jamiefeiss]

Both the /top-concepts & /narrowers endpoints don't return sorted concepts. This should be updated to match the recent changes to paginated items being sorted.

[edmondchuc]

@jamiefeiss I just checked both /top-concepts and /narrowers and they appear to be giving me sorted paginated concepts by label.

Maybe we can catch up tomorrow and you can show me the issue?

[recalcitrantsupplant]

See comments here #131

[edmondchuc]

Chat in Prez meeting:

Support for client-side "expand all" functionality.

Count

/count

Query parameters:

• curie - Prez identifier
• inbound e.g. skos:broader
• outbound e.g. skos:narrower, skos:inScheme

Returns the entire vocabulary in the old way.

/v/vocabs/{curie}/all

Had discussion with a potential future design where it's a single endpoint and allows for retrieving vocabs at a depth given by query parameter. May follow up in a separate issue.

[edmondchuc]

The sorted pagination issue that @jamiefeiss raised is likely due to triplestore implementation of ORDER BY on strings that have language tags. Solution is probably casting the ?label variable to a string and binding that as a new label with ORDER BY applied to it.

[edmondchuc]

Two new commits:

• 0cb5873 - Improve error handling for curies and IRI resolution by reusing endpoint functions that already manage HTTP error handling
• 3e63b1a Order top-concepts and narrower concepts pagination endpoints by casting ?label variable to string using the str() SPARQL function

[jamiefeiss]

Looks good! Thanks Edmond