-
Notifications
You must be signed in to change notification settings - Fork 5
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add tag namespace concept #715
Commits on Mar 6, 2024
-
From #390, the desire is to have different users (or CI jobs on the same runner) have a certain amount of isolation from each other. In particular, `spk ls` is expected to not contain any "local" packages that don't belong to the current user/CI job. This commit experiments with this idea. Only the local repo is expected to have an optional tag namespace set, so tar and rpc repos have not been changed. Setting the tag namespace is possible via the spfs config file, but it is expected to typically be set via the environment variable $SPFS_STORAGE_TAGNAMESPACE instead. It may be reasonable to add a [spk] config option to make the tag namespace default to the current user's username. A user sharing a host with other users are unlikely to want to see any local packages made by other users. There is an open question on if spfs runtimes should be subject to the tag namespace. This code currently does make them subject to the tag namespace. Therefore, if a user has a tag namespace specified, `spfs runtime ls` would only show the runtimes created in that same tag namespace, and those runtimes would be hidden from other users/other tag namespaces. This is in the same spirit of users being isolated from each other, however an argument could be made that it is important from an admin perspective to be able to list _all_ the runtimes on a host. `spfs clean` is a special case that always ignores any tag namespace. It should be able to find _all_ tags in the repo and avoid coming to any false conclusions about what objects are garbage on the basis of only seeing tags inside a tag namespace. Signed-off-by: J Robert Ray <jrray@jrray.org>
Configuration menu - View commit details
-
Copy full SHA for 644fe6f - Browse repository at this point
Copy the full SHA 644fe6fView commit details -
Storage::new will start needing a mutable reference to the repo in order to make it ignore the current tag namespace. Signed-off-by: J Robert Ray <jrray@jrray.org>
Configuration menu - View commit details
-
Copy full SHA for 2d23026 - Browse repository at this point
Copy the full SHA 2d23026View commit details -
Keep runtime tags out of the tag namespace
Signed-off-by: J Robert Ray <jrray@jrray.org>
Configuration menu - View commit details
-
Copy full SHA for 8b37677 - Browse repository at this point
Copy the full SHA 8b37677View commit details -
Add a naming convention for tag namespace directories
Use a character that is an illegal character for users, '#', so we can prevent users from creating tag names that could be misinterpreted as namespace names. To prevent different namespaces from getting mixed up, include the depth of the namespace component in the directory name. For example the tag "hi" in namespace "foo/bar" would produce this on disk: /spfs-repos/local/tags/foo#ns.0 /spfs-repos/local/tags/foo#ns.0/bar#ns.1 /spfs-repos/local/tags/foo#ns.0/bar#ns.1/hi.tag This prevents tag "bar/hi" in namespace "foo" from colliding with tag "hi" in namespace "foo/bar" since these would have different filenames: - tags/foo#ns.0/bar/hi.tag - tags/foo#ns.0/bar#ns.1/hi.tag Signed-off-by: J Robert Ray <jrray@jrray.org>
Configuration menu - View commit details
-
Copy full SHA for dbed80e - Browse repository at this point
Copy the full SHA dbed80eView commit details -
Add an EntryType::Namespace arm
Give ls-tags some awareness of the existence of namespace entries so it can hide these from its output. Note that iter_tag_streams doesn't make use of `EntryType` and will walk into any namespace directories it find, which for the time being is good because it still allows `spfs clean` to find all tags in all namespaces. Signed-off-by: J Robert Ray <jrray@jrray.org>
Configuration menu - View commit details
-
Copy full SHA for 2fe5b6e - Browse repository at this point
Copy the full SHA 2fe5b6eView commit details -
Add test for rejecting namespace-like tag names
Verify that TagSpec will reject a name in the new namespace naming convention (it already does). Signed-off-by: J Robert Ray <jrray@jrray.org>
Configuration menu - View commit details
-
Copy full SHA for 300267c - Browse repository at this point
Copy the full SHA 300267cView commit details -
Simple test of tag namespace behavior
The test code requires a way to modify the tag namespace of a TempRepo, but since it can be changed through an Arc, add a `with_tag_namespace` method to make a duplicate handle to an existing repo but with a different namespace set. This required putting the TempDir inside an Arc. Signed-off-by: J Robert Ray <jrray@jrray.org>
Configuration menu - View commit details
-
Copy full SHA for f503d22 - Browse repository at this point
Copy the full SHA f503d22View commit details -
Add tag namespace collision test
Confirm that tags with names that could possibly collide with tag namespace names are really distinct and don't collide. Signed-off-by: J Robert Ray <jrray@jrray.org>
Configuration menu - View commit details
-
Copy full SHA for 9f1a1ed - Browse repository at this point
Copy the full SHA 9f1a1edView commit details -
This was deemed unnecessary for disambiguation purposes and would only be useful for supporting nesting namespaces, which we do not expect to support at this time. Signed-off-by: J Robert Ray <jrray@jrray.org>
Configuration menu - View commit details
-
Copy full SHA for 89239a4 - Browse repository at this point
Copy the full SHA 89239a4View commit details -
Configuration menu - View commit details
-
Copy full SHA for cf145f8 - Browse repository at this point
Copy the full SHA cf145f8View commit details -
Implement tag namespaces for rpc
This requires adding "*_in_namespace" flavors of the tag methods, because the rpc server needed a way to process requests for tags in a namespace without introducing race conditions by setting the tag namespace on its repo handle. The server is "stateless" in the sense that setting the tag namespace on the client side only changes state on the client side, and then any requests sent to the server include the name of the prevailing tag namespace as part of the request. Signed-off-by: J Robert Ray <jrray@jrray.org>
Configuration menu - View commit details
-
Copy full SHA for 635742e - Browse repository at this point
Copy the full SHA 635742eView commit details -
Configuration menu - View commit details
-
Copy full SHA for 0dee728 - Browse repository at this point
Copy the full SHA 0dee728View commit details -
Use to_address to get repository address
Leverage the fact that Config contains the tag address. Signed-off-by: J Robert Ray <jrray@jrray.org>
Configuration menu - View commit details
-
Copy full SHA for d213812 - Browse repository at this point
Copy the full SHA d213812View commit details -
Create newtype for tag namespace
As part of transitioning to use RelativePath instead of Path, create wrapper types to partially hide the underlying type from the public API. Signed-off-by: J Robert Ray <jrray@jrray.org>
Configuration menu - View commit details
-
Copy full SHA for 7da612a - Browse repository at this point
Copy the full SHA 7da612aView commit details -
Use derive_builder in RemoteConfig::from_address
Make it easier to add more query options. Signed-off-by: J Robert Ray <jrray@jrray.org>
Configuration menu - View commit details
-
Copy full SHA for 9be0a36 - Browse repository at this point
Copy the full SHA 9be0a36View commit details -
Implement tag namespace RemoteConfig property
Signed-off-by: J Robert Ray <jrray@jrray.org>
Configuration menu - View commit details
-
Copy full SHA for b9a33a1 - Browse repository at this point
Copy the full SHA b9a33a1View commit details