Add tag namespace concept #715
Merged
Commits on Mar 6, 2024
-
From #390, the desire is to have different users (or CI jobs on the same runner) have a certain amount of isolation from each other. In particular, `spk ls` is expected to not contain any "local" packages that don't belong to the current user/CI job. This commit experiments with this idea. Only the local repo is expected to have an optional tag namespace set, so tar and rpc repos have not been changed. Setting the tag namespace is possible via the spfs config file, but it is expected to typically be set via the environment variable $SPFS_STORAGE_TAGNAMESPACE instead. It may be reasonable to add a [spk] config option to make the tag namespace default to the current user's username. A user sharing a host with other users are unlikely to want to see any local packages made by other users. There is an open question on if spfs runtimes should be subject to the tag namespace. This code currently does make them subject to the tag namespace. Therefore, if a user has a tag namespace specified, `spfs runtime ls` would only show the runtimes created in that same tag namespace, and those runtimes would be hidden from other users/other tag namespaces. This is in the same spirit of users being isolated from each other, however an argument could be made that it is important from an admin perspective to be able to list _all_ the runtimes on a host. `spfs clean` is a special case that always ignores any tag namespace. It should be able to find _all_ tags in the repo and avoid coming to any false conclusions about what objects are garbage on the basis of only seeing tags inside a tag namespace. Signed-off-by: J Robert Ray <jrray@jrray.org>
-
Storage::new will start needing a mutable reference to the repo in order to make it ignore the current tag namespace. Signed-off-by: J Robert Ray <jrray@jrray.org>
-
Keep runtime tags out of the tag namespace
Signed-off-by: J Robert Ray <jrray@jrray.org>
-
Add a naming convention for tag namespace directories
Use a character that is an illegal character for users, '#', so we can prevent users from creating tag names that could be misinterpreted as namespace names. To prevent different namespaces from getting mixed up, include the depth of the namespace component in the directory name. For example the tag "hi" in namespace "foo/bar" would produce this on disk: /spfs-repos/local/tags/foo#ns.0 /spfs-repos/local/tags/foo#ns.0/bar#ns.1 /spfs-repos/local/tags/foo#ns.0/bar#ns.1/hi.tag This prevents tag "bar/hi" in namespace "foo" from colliding with tag "hi" in namespace "foo/bar" since these would have different filenames: - tags/foo#ns.0/bar/hi.tag - tags/foo#ns.0/bar#ns.1/hi.tag Signed-off-by: J Robert Ray <jrray@jrray.org> -
Add an EntryType::Namespace arm
Give ls-tags some awareness of the existence of namespace entries so it can hide these from its output. Note that iter_tag_streams doesn't make use of `EntryType` and will walk into any namespace directories it find, which for the time being is good because it still allows `spfs clean` to find all tags in all namespaces. Signed-off-by: J Robert Ray <jrray@jrray.org>
-
Add test for rejecting namespace-like tag names
Verify that TagSpec will reject a name in the new namespace naming convention (it already does). Signed-off-by: J Robert Ray <jrray@jrray.org>
-
Simple test of tag namespace behavior
The test code requires a way to modify the tag namespace of a TempRepo, but since it can be changed through an Arc, add a `with_tag_namespace` method to make a duplicate handle to an existing repo but with a different namespace set. This required putting the TempDir inside an Arc. Signed-off-by: J Robert Ray <jrray@jrray.org>
-
Add tag namespace collision test
Confirm that tags with names that could possibly collide with tag namespace names are really distinct and don't collide. Signed-off-by: J Robert Ray <jrray@jrray.org>
-
This was deemed unnecessary for disambiguation purposes and would only be useful for supporting nesting namespaces, which we do not expect to support at this time. Signed-off-by: J Robert Ray <jrray@jrray.org>
-
Implement tag namespaces for rpc
This requires adding "*_in_namespace" flavors of the tag methods, because the rpc server needed a way to process requests for tags in a namespace without introducing race conditions by setting the tag namespace on its repo handle. The server is "stateless" in the sense that setting the tag namespace on the client side only changes state on the client side, and then any requests sent to the server include the name of the prevailing tag namespace as part of the request. Signed-off-by: J Robert Ray <jrray@jrray.org>
-
Use to_address to get repository address
Leverage the fact that Config contains the tag address. Signed-off-by: J Robert Ray <jrray@jrray.org>
-
Create newtype for tag namespace
As part of transitioning to use RelativePath instead of Path, create wrapper types to partially hide the underlying type from the public API. Signed-off-by: J Robert Ray <jrray@jrray.org>
-
Use derive_builder in RemoteConfig::from_address
Make it easier to add more query options. Signed-off-by: J Robert Ray <jrray@jrray.org>
-
Implement tag namespace RemoteConfig property
Signed-off-by: J Robert Ray <jrray@jrray.org>
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.