Expose reosurce descriptors from manifests

We aim this to be similar to in-toto's `ResourceDescriptor`. To support cases where in-toto cannot be directly used, we make this a dataclass that can be mapped to in-toto when needed, and used as its own otherwise.

Not all fields from in-toto are specified at this moment. All fields here must be present, unlike in-toto, where all are optional.

See https://github.com/in-toto/attestation/blob/main/spec/v1/resource_descriptor.md for the in-toto specification.

This is the first separable PR for the signing support (see full draft on sigstore#253)

Signed-off-by: Mihai Maruseac <mihaimaruseac@google.com>

.github/workflows/lint.yml

@@ -66,7 +66,7 @@ jobs:
           pip install -r model_signing/install/requirements_test_Linux.txt
           pip install -r model_signing/install/requirements_dev_Linux.txt
           # TODO: https://github.com/sigstore/model-transparency/issues/231 - Support all repo
-          pytype --keep-going model_signing/{hashing,manifest,serialization,signature}
+          pytype --keep-going model_signing/{hashing,manifest,serialization,signature,signing}
 
   pylint-lint:
     runs-on: ubuntu-latest
@@ -90,4 +90,4 @@ jobs:
           pylint \
             --max-line-length 80 \
             --disable C0114,C0115,C0116,R0801,R0903,R0904,R0913,R0914,R1721,R1737,W0107,W0212,W0223,W0231,W0511,W0621 \
-            model_signing/{hashing,manifest,serialization,signature}
+            model_signing/{hashing,manifest,serialization,signature,signing}

model_signing/signing/__init__.py

@@ -0,0 +1,13 @@
+# Copyright 2024 The Sigstore Authors
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.

model_signing/signing/signing.py

@@ -0,0 +1,157 @@
+# Copyright 2024 The Sigstore Authors
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Machinery for signing and verification of ML models.
+
+The serialization API produces a manifest representation of the models, and we
+use that to implement integrity checking of models in different computational
+patterns. This means that all manifests need to be kept only in memory.
+
+Hence, for signing, we need a separate class hierarchy to represent the payload.
+This is why we introduce `SigningPayload` abstract class here. Every instance of
+this class is built via a `from_manifest` method (to allow for classes that need
+initialization to be performed only once in their constructor but then convert
+multiple manifests into payloads).
+
+Since we need to support multiple signing methods (e.g., Sigstore, own PKI,
+BCID), we provide a `Signer` abstract class with a single `sign` method that
+takes a signing payload and converts it to a signature in the supported format.
+Since signers may only accept payloads in specific formats, the `sign` method
+can raise a `TypeError` if the provided `SigningPayload` instance is not
+supported (due to typing rules, we cannot just use a subclass type for the
+argument).
+
+Every possible signature will be implemented as a subclass of `Signature` class.
+The API for signatures only allows writing them to disk and parsing them from a
+given path.
+
+Finally, every signature needs to be verified. We pair every `Signer` subclass
+with a `Verifier` which takes a signature, verify the authenticity of the
+payload and then expand that to a `manifest.Manifest` subclass.
+"""
+
+import abc
+import pathlib
+from typing import Self
+
+from model_signing.manifest import manifest
+
+
+class SigningPayload(metaclass=abc.ABCMeta):
+    """Generic payload that we can sign."""
+
+    @classmethod
+    @abc.abstractmethod
+    def from_manifest(cls, manifest: manifest.Manifest) -> Self:
+        """Converts a manifest to the signing payload used for signing.
+
+        Args:
+            manifest: the manifest to convert to signing payload.
+
+        Returns:
+            An instance of `SigningPayload` or subclass of it.
+        """
+        pass
+
+
+class Signature(metaclass=abc.ABCMeta):
+    """Generic signature support."""
+
+    @abc.abstractmethod
+    def write_signature(self, path: pathlib.Path) -> None:
+        """Writes the signature to disk, to the given path.
+
+        Args:
+            path: the path to write the signature to.
+        """
+        pass
+
+    @classmethod
+    @abc.abstractmethod
+    def read_signature(cls, path: pathlib.Path) -> Self:
+        """Reads the signature from disk.
+
+        Does not perform any signature verification, except what is needed to
+        parse the signature file.
+
+        Args:
+            path: the path to read the signature from.
+
+        Returns:
+            An instance of the class which can be passed to a `Verifier` for
+            signature and integrity verification.
+
+        Raises:
+            ValueError: If the provided path is not deserializable to the format
+              expected by the `Signature` (sub)class.
+        """
+        pass
+
+
+class Signer(metaclass=abc.ABCMeta):
+    """Generic signer for `SigningPayload` objects.
+
+    Every signer is allowed to only support some signing payload formats. Every
+    signer produces a signature in its own format. No signer is required to
+    support all subclasses of `SigningPayload` or `Signature`.
+
+    Each signer may implement its own mechanism for managing the key material.
+    """
+
+    @abc.abstractmethod
+    def sign(self, payload: SigningPayload) -> Signature:
+        """Signs the provided signing payload.
+
+        Args:
+            payload: the `SigningPayload` instance that should be signed.
+
+        Returns:
+            A valid signature.
+
+        Raises:
+            TypeError: If the `payload` type is not one of the subclasses of
+              `SigningPayload` that are supported.
+        """
+        pass
+
+
+class Verifier(metaclass=abc.ABCMeta):
+    """Generic signature verifier.
+
+    Every subclass of `Verifier` is paired with a subclass of `Signer`. This is
+    to ensure that they support the same signing payload and signature formats
+    as well as have similar key materialsEvery subclass of `Verifier` is paired
+    with a subclass of `Signer`. This is to ensure that they support the same
+    signing payload and signature formats as well as have similar key materials.
+
+    If the signature is valid, the payload is expanded to a `Manifest` instance
+    which can then be used to check the model integrity.
+    """
+
+    @abc.abstractmethod
+    def verify(self, signature: Signature) -> manifest.Manifest:
+        """Verifies the signature.
+
+        Args:
+            signature: the signature to verify.
+
+        Returns:
+            A `manifest.Manifest` instance that represents the model.
+
+        Raises:
+            ValueError: If the signature cannot be verified.
+            TypeError: If the signature is not one of the `Signature` subclasses
+              accepted by the verifier.
+        """
+        pass

model_signing/test_support.py

@@ -43,6 +43,16 @@
 ]
 
 
+# All directory models to use in testing, where only non empty directory models
+# are supported. See also `all_test_models` comments.
+all_non_empty_directory_test_models = [
+    "sample_model_folder",
+    "deep_model_folder",
+    "model_folder_with_empty_file",
+    "symlink_model_folder",
+]
+
+
 def get_first_directory(path: pathlib.Path) -> pathlib.Path:
     """Returns the first directory that is a children of path.