📝 [#593] Add documentation for logging and log event types

docker/observability/README.md

@@ -6,7 +6,7 @@ a reference for a potential observability stack.
 **Disclaimer**
 
 The chosen vendors/technologies here merely serve as an example - it's simply a stack we're somewhat
-comfortable with. Open Forms itself is vendor agnostic and the principles demonstrated apply to
+comfortable with. Objects API itself is vendor agnostic and the principles demonstrated apply to
 competing vendors too.
 
 ## Bringing up the services
@@ -36,7 +36,7 @@ In the Grafana menu, navigate to "Explore" to create ad-hoc queries.
 **Web service logs**
 
 ```logql
-{job="docker", app="openforms"} | json | __error__ = ""
+{job="docker", app="objects-api"} | json | __error__ = ""
 ```
 
 This ignores logs that cannot be parsed as JSON (such as container/server startup logs).
@@ -46,5 +46,5 @@ This ignores logs that cannot be parsed as JSON (such as container/server startu
 You can filter application logs based on a request ID:
 
 ```logql
-{job="docker", app="openforms"} | json | __error__ = "" | request_id=`1e9e1b9d-4d34-4657-99e4-88673d824724`
+{job="docker", app="objects-api"} | json | __error__ = "" | request_id=`1e9e1b9d-4d34-4657-99e4-88673d824724`
 ```

docs/conf.py

@@ -8,6 +8,9 @@
 import os
 import sys
 
+# Import as private variable to avoid errors on build
+from importlib.metadata import version as _version
+
 import django
 from django.utils.translation import activate
 
@@ -47,6 +50,7 @@
 extensions = [
     "sphinx.ext.todo",
     "sphinx.ext.extlinks",
+    "sphinx.ext.intersphinx",
     "sphinx_tabs.tabs",
     "recommonmark",
     "sphinx.ext.autodoc",
@@ -116,3 +120,11 @@
         "#%s",
     ),
 }
+
+django_structlog_version = _version("django-structlog")
+intersphinx_mapping = {
+    "django-structlog": (
+        f"https://django-structlog.readthedocs.io/en/{django_structlog_version}",
+        None,
+    ),
+}

docs/index.rst

@@ -61,5 +61,6 @@ a national and/or local :ref:`Objecttypes API`.
    api/index
    admin/index
    installation/index
+   manual/index
    examples/index
    changelog

docs/manual/index.rst

@@ -0,0 +1,11 @@
+.. _manual_index:
+
+Manual
+======
+
+.. toctree::
+   :maxdepth: 1
+   :caption: Further reading
+
+   logging
+

docs/manual/logging.rst

@@ -0,0 +1,69 @@
+.. _manual_logging:
+
+Logging
+=======
+
+Format
+------
+
+Objects API emits structured logs (using `structlog <https://www.structlog.org/en/stable/>`_).
+A log line can be formatted like this:
+
+.. code-block:: json
+
+    {
+        "object_uuid":"1734afac-4628-446d-b344-f39b42f48bd9",
+        "objecttype_uuid":"b427ef84-189d-43aa-9efd-7bb2c459e281",
+        "objecttype_version":1,
+        "token_identifier":"application-test",
+        "token_application":"Application (test)",
+        "event":"object_created",
+        "user_id":null,
+        "request_id":"2f9e9a5b-d549-4faa-a411-594aa8a52eee",
+        "timestamp":"2025-05-19T14:09:20.339166Z",
+        "logger":"objects.api.serializers",
+        "level":"info"
+    }
+
+Each log line will contain an ``event`` type, a ``timestamp`` and a ``level``.
+Dependent on your configured ``LOG_LEVEL`` (see :ref:`installation_env_config` for more information),
+only log lines with of that level or higher will be emitted.
+
+Objects API log events
+----------------------
+
+Below is the list of logging ``event`` types that Objects API can emit.
+
+API
+~~~
+
+* ``objecttypes_api_request_failure``: a request to the Objecttypes API has failed. Additional context: ``exc_info``.
+* ``search_failed_for_datastore``: attempted to perform ``jsonpath`` search for a backend that does not support this operation. Additional context: ``exc_info``.
+* ``object_created``: created an ``Object`` via the API. Additional context: ``object_uuid``, ``objecttype_uuid``, ``objecttype_version``, ``token_identifier``, ``token_application``.
+* ``object_updated``: updated an ``Object`` via the API. Additional context: ``object_uuid``, ``objecttype_uuid``, ``objecttype_version``, ``token_identifier``, ``token_application``.
+
+Setup configuration
+~~~~~~~~~~~~~~~~~~~
+
+* ``no_permissions_defined``: while running the token configuration step, it was detected that neither permissions nor ``is_superuser`` was set for the token. Additional context: ``token_identifier``.
+* ``no_tokens_defined``: while running the token configuration step, it was detected that the config file did not define any tokens.
+* ``configuring_token``: attempting to configure a token. Additional context: ``token_identifier``.
+* ``save_token_to_database``: attempting to save a token to the database. Additional context: ``token_identifier``.
+* ``token_configuration_failure``: configuring a token failed. Additional context: ``token_identifier``, ``exc_info``.
+* ``token_configuration_success``: configuring a token succeeded. Additional context: ``token_identifier``.
+
+Data migrations
+~~~~~~~~~~~~~~~
+
+* ``token_identifier_generated``: while migrating, an ``identifier`` was generated for a token. Additional context: ``token_identifier``, ``token_pk``.
+* ``missing_service_for_objecttype``: while migrating, a ``Service`` object is missing for an ``ObjectType``. Additional context: ``object``, ``objecttype``.
+* ``invalid_objecttype``: while migrating, the ``ObjectType`` is not valid, because it was not possible to parse a UUID from it. Additional context: ``object``, ``objecttype``.
+
+Third party library events
+--------------------------
+
+For more information about log events emitted by third party libraries, refer to the documentation
+for that particular library
+
+* :ref:`Django (via django-structlog) <request_events>`
+* :ref:`Celery (via django-structlog) <request_events>`

src/objects/setup_configuration/steps/token_auth.py

@@ -42,7 +42,7 @@ def _full_clean(self, instance: Any) -> None:
             ) from exception
 
     def _configure_permissions(self, token: TokenAuth, permissions: list) -> None:
-        if len(permissions) == 0:
+        if len(permissions) == 0 and not token.is_superuser:
             logger.warning("no_permissions_defined", token_identifier=token.identifier)
 
         for permission in permissions:
@@ -85,7 +85,7 @@ def execute(self, model: TokenAuthGroupConfigurationModel) -> None:
             logger.warning("no_tokens_defined")
 
         for item in model.items:
-            logger.info("configure_token", token_identifier=item.identifier)
+            logger.info("configuring_token", token_identifier=item.identifier)
 
             token_kwargs = {
                 "identifier": item.identifier,
@@ -115,7 +115,9 @@ def execute(self, model: TokenAuthGroupConfigurationModel) -> None:
 
             except IntegrityError as exception:
                 logger.exception(
-                    "token_configuration_failure", token_identifier=item.identifier
+                    "token_configuration_failure",
+                    token_identifier=item.identifier,
+                    exc_info=exception,
                 )
                 raise ConfigurationRunFailed(
                     "Failed configuring token %s" % item.identifier