object endpoint to find internal links for resources (#145)

* Object endpoint returns system links. Add endpoint reference data to support system link generation. Rename inbound/outbound children/parents to focus to/from child/parent terminology to align with SHACL. Rename general_class to base_class. Resolves #101 * Update documentation. Remove commented code. * Update documentation. Remove commented code. Fix object tests. * Object endpoint returns system links. Add endpoint reference data to support system link generation. Rename inbound/outbound children/parents to focus to/from child/parent terminology to align with SHACL. Rename general_class to base_class. Resolves #101 * Update documentation. Remove commented code. * Update documentation. Remove commented code. Fix object tests. * Roll out link generation across catprez / vocprez / spaceprez * Test bugfix: best guess the prefixes are not actualy generated in a deterministic way on startup and we shouldn't rely on them being deterministic in tests. Both ns1 & ns2 come back in responses for the same link (for a given URI) if the test is run multiple times. Workaround implemented. The test is not intended to test link generation or prefix generation. * Bugfix for profiles test * Add prez:endpointComponentURI for parents in endpoints such that the URIs of parents are included in the response, and get annotated, such that labels are available for breadcrumbs. Update spaceprez profile to include dcterms:title for the OAS profile. * Add TODOs. Add setting to make ordering optional via ORDER_LISTS_BY_LABEL environment variable, which is on by default. * Update tests to include endpointComponentURIs * Update profiles endpoints to include prez links Add vann prefixes; correct prefix format * Remove inadvertently added OGC content * Change use of prez:endpointComponentURI -> dcterms:identifier "abc"^^prez:identifier for providing parent information that can be used in breadcrumbs. Update tests accordingly. Bugfix: Add endpoint name so vocprez endpoint functions correctly. * Fix bug with identifiers not being included for concepts * Update test responses, comment one test. One bugfix to get_classes * Update prez/cache.py cheers Co-authored-by: Edmond Chuc <37032744+edmondchuc@users.noreply.github.com> --------- Co-authored-by: Edmond Chuc <37032744+edmondchuc@users.noreply.github.com>
RDFLib · Aug 31, 2023 · df00e01 · df00e01
1 parent a56cb9a
commit df00e01
Show file tree

Hide file tree

Showing 69 changed files with 10,657 additions and 1,871 deletions.
diff --git a/README-Dev.md b/README-Dev.md
@@ -77,7 +77,21 @@ using the properties listed below.
 | provenance  | dcterms:provenance  | dcterms:source                                           | altr-ext:hasExplanationPredicate                  |
 | other       | (None)              | schema:color                                             | altr-ext:otherAnnotationProps                     |
 
-## High Level Sequence
+## High Level Sequence `/object` endpoint
+
+Prez provides a `/object` endpoint as an endpoint that supplies any information known about a given URI. If an annotated
+mediatype is requested, prez will additionally provide all system links for endpoints which can render the object. The
+high level sequence for this endpoint is as follows:
+
+1. Get the URI for the object from the query string
+2. Get the class(es) of the object from the triplestore
+3. Use prez's reference data for endpoints to determine which endpoints can render this object, and, a template for
+these endpoints, specifying any variables that need to be substituted (such as parent URIs).
+4. Get the object information from the triplestore, using an open profile, and in parallel any system information needed
+to construct the system links.
+5. Return the response
+
+## High Level Sequence listing and individual object endpoints
 
 Prez follows the following logic to determine what information to return, based on a profile, and in what mediatype to return it.
 
@@ -287,8 +301,8 @@ SELECT ?profile ?title ?class (count(?mid) as ?distance) ?req_profile ?def_profi
 WHERE {
   VALUES ?class {<https://prez.dev/DatasetList>}
   ?class rdfs:subClassOf* ?mid .
-  ?mid rdfs:subClassOf* ?general_class .
-  VALUES ?general_class { dcat:Dataset geo:FeatureCollection prez:FeatureCollectionList prez:FeatureList geo:Feature
+  ?mid rdfs:subClassOf* ?base_class .
+  VALUES ?base_class { dcat:Dataset geo:FeatureCollection prez:FeatureCollectionList prez:FeatureList geo:Feature
     skos:ConceptScheme skos:Concept skos:Collection prez:DatasetList prez:VocPrezCollectionList prez:SchemesList
     prez:CatalogList dcat:Catalog dcat:Resource }
   ?profile altr-ext:constrainsClass ?class ;

diff --git a/prez/app.py b/prez/app.py
@@ -18,16 +18,21 @@
 from prez.renderers.renderer import return_rdf
 from prez.routers.catprez import router as catprez_router
 from prez.routers.cql import router as cql_router
+from prez.routers.identifier import router as identifier_router
 from prez.routers.management import router as management_router
 from prez.routers.object import router as object_router
 from prez.routers.profiles import router as profiles_router
 from prez.routers.search import router as search_router
 from prez.routers.spaceprez import router as spaceprez_router
 from prez.routers.sparql import router as sparql_router
 from prez.routers.vocprez import router as vocprez_router
-from prez.routers.identifier import router as identifier_router
-from prez.services.app_service import healthcheck_sparql_endpoints, count_objects
-from prez.services.app_service import populate_api_info, add_prefixes_to_prefix_graph
+from prez.services.app_service import (
+    healthcheck_sparql_endpoints,
+    count_objects,
+    create_endpoints_graph,
+    populate_api_info,
+    add_prefixes_to_prefix_graph,
+)
 from prez.services.exception_catchers import (
     catch_400,
     catch_404,
@@ -58,9 +63,12 @@
 app.include_router(sparql_router)
 app.include_router(search_router)
 app.include_router(profiles_router)
-app.include_router(catprez_router)
-app.include_router(vocprez_router)
-app.include_router(spaceprez_router)
+if "CatPrez" in settings.prez_flavours:
+    app.include_router(catprez_router)
+if "VocPrez" in settings.prez_flavours:
+    app.include_router(vocprez_router)
+if "SpacePrez" in settings.prez_flavours:
+    app.include_router(spaceprez_router)
 app.include_router(identifier_router)
 
 
@@ -106,12 +114,13 @@ async def app_startup():
     setup_logger(settings)
     log = logging.getLogger("prez")
     log.info("Starting up")
+    await add_prefixes_to_prefix_graph()
     await healthcheck_sparql_endpoints()
     await get_all_search_methods()
     await create_profiles_graph()
+    await create_endpoints_graph()
     await count_objects()
     await populate_api_info()
-    await add_prefixes_to_prefix_graph()
 
 
 @app.on_event("shutdown")

diff --git a/prez/cache.py b/prez/cache.py
@@ -5,6 +5,9 @@
 profiles_graph_cache = ConjunctiveGraph()
 profiles_graph_cache.bind("prez", "https://prez.dev/")
 
+endpoints_graph_cache = ConjunctiveGraph()
+endpoints_graph_cache.bind("prez", "https://prez.dev/")
+
 prez_system_graph = Graph()
 prez_system_graph.bind("prez", "https://prez.dev/")
 

diff --git a/prez/config.py b/prez/config.py
@@ -21,21 +21,9 @@ class Settings(BaseSettings):
     system_uri: Documentation property. An IRI for the Prez system as a whole. This value appears in the landing page RDF delivered by Prez ('/')
     top_level_classes:
     collection_classes:
-    general_classes:
+    base_classes:
     log_level:
     log_output:
-    cql_props: dict = {
-        "title": {
-            "title": "Title",
-            "description": "The title of a geo:Feature",
-            "type": "string",
-        },
-        "desc": {
-            "title": "Description",
-            "description": "The description of a geo:Feature",
-            "type": "string",
-        },
-    }
     prez_title:
     prez_desc:
     prez_version:
@@ -52,22 +40,11 @@ class Settings(BaseSettings):
     system_uri: Optional[str]
     top_level_classes: Optional[dict]
     collection_classes: Optional[dict]
-    general_classes: Optional[dict]
+    order_lists_by_label: bool = True
+    base_classes: Optional[dict]
     prez_flavours: Optional[list] = ["SpacePrez", "VocPrez", "CatPrez", "ProfilesPrez"]
     log_level = "INFO"
     log_output = "stdout"
-    cql_props: dict = {
-        "title": {
-            "title": "Title",
-            "description": "The title of a geo:Feature",
-            "type": "string",
-        },
-        "desc": {
-            "title": "Description",
-            "description": "The description of a geo:Feature",
-            "type": "string",
-        },
-    }
     prez_title: Optional[str] = "Prez"
     prez_desc: Optional[str] = (
         "A web framework API for delivering Linked Data. It provides read-only access to "
@@ -104,58 +81,58 @@ def set_system_uri(cls, values):
             )
         return values
 
-    @root_validator()
-    def populate_top_level_classes(cls, values):
-        values["top_level_classes"] = {
-            "Profiles": [
-                PROF.Profile,
-                PREZ.SpacePrezProfile,
-                PREZ.VocPrezProfile,
-                PREZ.CatPrezProfile,
-            ],
-            "SpacePrez": [DCAT.Dataset],
-            "VocPrez": [SKOS.ConceptScheme, SKOS.Collection],
-            "CatPrez": [DCAT.Catalog],
-        }
-        return values
-
-    @root_validator()
-    def populate_collection_classes(cls, values):
-        additional_classes = {
-            "Profiles": [],
-            "SpacePrez": [GEO.FeatureCollection],
-            "VocPrez": [],
-            "CatPrez": [DCAT.Resource],
-        }
-        values["collection_classes"] = {}
-        for prez in list(additional_classes.keys()) + ["Profiles"]:
-            values["collection_classes"][prez] = (
-                values["top_level_classes"].get(prez) + additional_classes[prez]
-            )
-        return values
-
-    @root_validator()
-    def populate_general_classes(cls, values):
-        additional_classes = {
-            "SpacePrez": [GEO.Feature],
-            "VocPrez": [SKOS.Concept],
-            "CatPrez": [DCAT.Dataset],
-            "Profiles": [PROF.Profile],
-        }
-        values["general_classes"] = {}
-        for prez in list(additional_classes.keys()) + ["Profiles"]:
-            values["general_classes"][prez] = (
-                values["collection_classes"].get(prez) + additional_classes[prez]
-            )
-        return values
-
-    @root_validator()
-    def populate_sparql_creds(cls, values):
-        username = values.get("sparql_username")
-        password = values.get("sparql_password")
-        if username is not None and password is not None:
-            values["sparql_auth"] = (username, password)
-        return values
+    # @root_validator()
+    # def populate_top_level_classes(cls, values):
+    #     values["top_level_classes"] = {
+    #         "Profiles": [
+    #             PROF.Profile,
+    #             PREZ.SpacePrezProfile,
+    #             PREZ.VocPrezProfile,
+    #             PREZ.CatPrezProfile,
+    #         ],
+    #         "SpacePrez": [DCAT.Dataset],
+    #         "VocPrez": [SKOS.ConceptScheme, SKOS.Collection],
+    #         "CatPrez": [DCAT.Catalog],
+    #     }
+    #     return values
+    #
+    # @root_validator()
+    # def populate_collection_classes(cls, values):
+    #     additional_classes = {
+    #         "Profiles": [],
+    #         "SpacePrez": [GEO.FeatureCollection],
+    #         "VocPrez": [],
+    #         "CatPrez": [DCAT.Resource],
+    #     }
+    #     values["collection_classes"] = {}
+    #     for prez in list(additional_classes.keys()) + ["Profiles"]:
+    #         values["collection_classes"][prez] = (
+    #             values["top_level_classes"].get(prez) + additional_classes[prez]
+    #         )
+    #     return values
+    #
+    # @root_validator()
+    # def populate_base_classes(cls, values):
+    #     additional_classes = {
+    #         "SpacePrez": [GEO.Feature],
+    #         "VocPrez": [SKOS.Concept],
+    #         "CatPrez": [DCAT.Dataset],
+    #         "Profiles": [PROF.Profile],
+    #     }
+    #     values["base_classes"] = {}
+    #     for prez in list(additional_classes.keys()) + ["Profiles"]:
+    #         values["base_classes"][prez] = (
+    #             values["collection_classes"].get(prez) + additional_classes[prez]
+    #         )
+    #     return values
+    #
+    # @root_validator()
+    # def populate_sparql_creds(cls, values):
+    #     username = values.get("sparql_username")
+    #     password = values.get("sparql_password")
+    #     if username is not None and password is not None:
+    #         values["sparql_auth"] = (username, password)
+    #     return values
 
 
 settings = Settings()
diff --git a/prez/models/__init__.py b/prez/models/__init__.py
@@ -1,7 +1 @@
-from prez.models.catprez_item import CatalogItem
-from prez.models.catprez_listings import CatalogMembers
-from prez.models.vocprez_item import VocabItem
-from prez.models.vocprez_listings import VocabMembers
-from prez.models.spaceprez_item import SpatialItem
-from prez.models.spaceprez_listings import SpatialMembers
 from prez.models.search_method import SearchMethod
diff --git a/prez/models/catprez_item.py b/prez/models/catprez_item.py
diff --git a/prez/models/catprez_listings.py b/prez/models/catprez_listings.py
diff --git a/prez/models/listing.py b/prez/models/listing.py
@@ -0,0 +1,44 @@
+from typing import Optional, FrozenSet
+
+from pydantic import BaseModel, root_validator
+from rdflib import URIRef, Literal, XSD
+
+from prez.cache import endpoints_graph_cache
+from prez.reference_data.prez_ns import ONT
+
+
+class ListingModel(BaseModel):
+    uri: Optional[
+        URIRef
+    ] = None  # this is the URI of the focus object (if listing by membership)
+    classes: Optional[FrozenSet[URIRef]] = None
+    endpoint_uri: Optional[URIRef] = None
+    selected_class: Optional[FrozenSet[URIRef]] = None
+    profile: Optional[URIRef] = None
+    top_level_listing: Optional[bool] = False
+
+    def __hash__(self):
+        return hash(self.uri)
+
+    @root_validator
+    def populate(cls, values):
+        endpoint_uri_str = values.get("endpoint_uri")
+        if endpoint_uri_str:
+            endpoint_uri = URIRef(endpoint_uri_str)
+            values["classes"] = frozenset(
+                [
+                    klass
+                    for klass in endpoints_graph_cache.objects(
+                        endpoint_uri, ONT.deliversClasses, None
+                    )
+                ]
+            )
+            values["base_class"] = endpoints_graph_cache.value(
+                endpoint_uri, ONT.baseClass
+            )
+            tll_text = endpoints_graph_cache.value(endpoint_uri, ONT.isTopLevelEndpoint)
+            if tll_text == Literal("true", datatype=XSD.boolean):
+                values["top_level_listing"] = True
+            else:
+                values["top_level_listing"] = False
+        return values