Clean up Python client auth (#836)

* Clean up Python client auth

* Use expiration and note in examples

* Turn API inside out to simplify.

* Refactor auth tests to new API

* WIP

* Update convenience wrapper to new login API.

* Fix prompt formatting

* Use new login usage in access control tests

* Handle optionality of API key setting correctly.

* Apply comments from Padraic, lost in rebase at some point

* Update usage to new API

* Fix latent usage error

* Update usage in new test

* Test remember_me.

* Test remember_me. Ensure it clears any existing tokens.

* CLI should reuse existing tokens.

* Add whoami

* Pin zarr<3 because awaiting zarr#2619

* Update CHANGELOG

* Add link to authentication guide.

CHANGELOG.md

@@ -23,6 +23,8 @@ Write the date in place of the "Unreleased" in the case a new version is release
 ### Changed
 
 - Change access policy API to be async for filters and allowed_scopes
+- Pinned zarr to `<3` because Zarr 3 is still working on adding support for
+  certain features that we rely on from Zarr 2.
 
 ## 2024-12-09
 

docs/source/how-to/authentication.md

@@ -0,0 +1,158 @@
+# Python Client Authentication
+
+This covers authentication from the user (client) perspective. To learn how to
+_deploy_ authenticated Tiled servers, see {doc}`../explanations/security`.
+
+## Interactive Login
+
+Some Tiled servers are configured to let users connect anonymously without
+authenticating.
+
+```py
+>>> from tiled.client import from_uri
+>>> client = from_uri("https://...")
+>>> <Container ...>
+```
+
+Logging in may enable you to see more datasets that may not be public.
+Log in works in one of two ways, depending on the server.
+
+1. Username and password ("OAuth2 password grant")
+
+   ```py
+   >>> client.login()
+   Username: ...
+   Password:
+   ```
+
+2. Via a web browser ("OAuth2 device code grant")
+
+   ```py
+   >>> client.login()
+   You have 15 minutes visit this URL
+
+   https://...
+
+   and enter the code: XXXX-XXXX
+   ```
+
+In the future, Tiled will log you into this server automatically, without
+re-prompting for credentials, until your session expires.
+
+   ```py
+   >>> from tiled.client import from_uri
+   >>> client = from_uri("https://...")
+   # Automatically logged in!
+
+   # This is a quick way to verify whether you are already logged in
+   >>> client.context
+   <Context authenticated as '...'>
+   ```
+
+To opt out of this, set `remember_me=False`:
+
+```py
+>>> from tiled.client import from_uri
+>>> client = from_uri("https://...", remember_me=False)
+```
+
+```{note}
+Tiled stores OAuth2 tokens (it _never_ stores your password) in files
+with properly restricted permissions under `$XDG_CACHE_DIR/tiled/tokens`,
+typically `~/.config/tiled/tokens` on Linux and MacOS.
+
+To customize the location of this storage, set the environment variable
+`TILED_CACHE_DIR`.
+```
+
+Some Tiled servers are configured to always require login, disallowing any
+anonymous access. For those, the client will prompt immediately, such as:
+
+   >>> from tiled.client import from_uri
+   >>> client = from_uri("https://...")
+   Username:
+   ```
+
+## Noninteractive Authentication (API keys)
+
+There are environments where logging in interactively is not possible,
+such as running a batch script. For these applications, we recommend
+using an API key. These can be created from the CLI:
+
+```sh
+$ tiled login
+$ tiled api_key create --expires-in 7d --note "for this week's experiment"
+```
+
+or from an interactive Python session:
+
+```py
+>>> client = from_uri("https://...")
+>>> client.login()
+>>> client.create_api_key(expires_in="7d", note="for this week's experiment")
+{"secret": ...}
+ ```
+
+The expiration and note are optional, but recommended. Expiration can be given
+in units of years `y`, days `d`, hours `h`, minutes `m`, or seconds `s`.
+
+```
+
+The best way to provide an API key is to set the environment variable
+`TILED_API_KEY`. A script like this:
+
+```py
+from tiled.client import from_uri
+
+client = from_uri("https://....")
+```
+
+will detect that `TILED_API_KEY` is set and use that API key for
+authentication with Tiled. This is equivalent to:
+
+```py
+import os
+from tiled.client import from_uri
+
+client = from_uri("https://....", api_key=os.environ["TILED_API_KEY"])
+```
+
+Avoid typing the API key in to the code:
+
+```py
+from_uri("https://...", api_key="secret!")  # DON'T
+```
+
+as it is easy to accidentally share or leak.
+
+## Custom Applications
+
+Custom applications, such as a graphical interfaces that wrap Tiled, may not be
+able to use Tiled commandline-based prompts. They should avoid using the
+convenience functions `tiled.client.construtors.from_uri` and
+`tiled.client.construtors.from_profile`.
+
+They may implement their own interfaces for collecting credentials (for
+password grants) or launching a browser and waiting for the user to authorize a
+session (for device code grants). The functions
+`tiled.client.context.password_grant` and
+`tiled.client.context.device_code_grant` may be useful building blocks. The
+tokens obtained from this process may then be passed directly in to the Tiled
+client like so.
+
+
+```py
+from tiled.client import Context
+
+URI = "https://..."
+context, node_path_parts = Context.from_any_uri(URI)
+tokens, remember_me = launch_custom_interface()
+context.configure_auth(tokens, remember_me=remember_me)
+client = from_context(context, node_path_parts=node_path_parts)
+```
+
+The client will transparently handle OAuth2 refresh flow. If the session is
+revoked or expires, and an attempt at refreshing the tokens is thus rejected
+by the server, the exception `tiled.client.auth.CannotRefreshAuthentication`
+will be raised. The application should be prepared to catch that exception
+and reinitiate authentication.

docs/source/index.md

@@ -18,6 +18,7 @@ tutorials/plotly-integration
 ```{toctree}
 :caption: How To Guides
 
+how-to/authentication
 how-to/profiles
 how-to/client-logger
 how-to/docker

pyproject.toml

@@ -92,7 +92,7 @@ all = [
     "uvicorn[standard]",
     "watchfiles",
     "xarray",
-    "zarr",
+    "zarr <3",
     "zstandard",
 ]
 # These are needed by the client and server to transmit/receive arrays.
@@ -219,7 +219,7 @@ minimal-server = [
     "starlette",
     "typer",
     "uvicorn[standard]",
-    "zarr",
+    "zarr <3",
 ]
 # This is the "kichen sink" fully-featured server dependency set.
 server = [
@@ -270,7 +270,7 @@ server = [
     "typer",
     "uvicorn[standard]",
     "xarray",
-    "zarr",
+    "zarr <3",
     "zstandard",
 ]
 # These are needed by the client and server to transmit/receive sparse arrays.

tiled/_tests/test_access_control.py

@@ -215,8 +215,9 @@ def context(tmpdir_module):
     }
     app = build_app_from_config(config)
     with Context.from_app(app) as context:
+        admin_client = from_context(context)
         with enter_username_password("admin", "admin"):
-            admin_client = from_context(context, username="admin")
+            admin_client.login()
             for k in ["c", "d", "e"]:
                 admin_client[k].write_array(arr, key="A1")
                 admin_client[k].write_array(arr, key="A2")
@@ -229,16 +230,18 @@ def context(tmpdir_module):
 
 
 def test_entry_based_scopes(context, enter_username_password):
+    alice_client = from_context(context)
     with enter_username_password("alice", "secret1"):
-        alice_client = from_context(context, username="alice")
+        alice_client.login()
     with pytest.raises(ClientError, match="Not enough permissions"):
         alice_client["h"]["x"].write(arr_zeros)
     alice_client["h"]["y"].write(arr_zeros)
 
 
 def test_top_level_access_control(context, enter_username_password):
+    alice_client = from_context(context)
     with enter_username_password("alice", "secret1"):
-        alice_client = from_context(context, username="alice")
+        alice_client.login()
     assert "a" in alice_client
     assert "A2" in alice_client["a"]
     assert "A1" not in alice_client["a"]
@@ -252,26 +255,25 @@ def test_top_level_access_control(context, enter_username_password):
         alice_client["b"]
     with pytest.raises(KeyError):
         alice_client["g"]["A4"]
+    alice_client.logout()
 
+    bob_client = from_context(context)
     with enter_username_password("bob", "secret2"):
-        bob_client = from_context(context, username="bob")
+        bob_client.login()
     assert not list(bob_client)
     with pytest.raises(KeyError):
         bob_client["a"]
     with pytest.raises(KeyError):
         bob_client["b"]
     with pytest.raises(KeyError):
         bob_client["g"]["A3"]
-    alice_client.logout()
-
-    # Make sure clearing default identity works without raising an error.
-    bob_client.logout(clear_default=True)
+    bob_client.logout()
 
 
 def test_access_control_with_api_key_auth(context, enter_username_password):
     # Log in, create an API key, log out.
     with enter_username_password("alice", "secret1"):
-        context.authenticate(username="alice")
+        context.authenticate()
     key_info = context.create_api_key()
     context.logout()
 
@@ -288,8 +290,9 @@ def test_access_control_with_api_key_auth(context, enter_username_password):
 
 def test_node_export(enter_username_password, context, buffer):
     "Exporting a node should include only the children we can see."
+    alice_client = from_context(context)
     with enter_username_password("alice", "secret1"):
-        alice_client = from_context(context, username="alice")
+        alice_client.login()
     alice_client.export(buffer, format="application/json")
     alice_client.logout()
     buffer.seek(0)
@@ -306,8 +309,9 @@ def test_node_export(enter_username_password, context, buffer):
 
 
 def test_create_and_update_allowed(enter_username_password, context):
+    alice_client = from_context(context)
     with enter_username_password("alice", "secret1"):
-        alice_client = from_context(context, username="alice")
+        alice_client.login()
 
     # Update
     alice_client["c"]["x"].metadata
@@ -325,17 +329,19 @@ def test_create_and_update_allowed(enter_username_password, context):
 
 
 def test_writing_blocked_by_access_policy(enter_username_password, context):
+    alice_client = from_context(context)
     with enter_username_password("alice", "secret1"):
-        alice_client = from_context(context, username="alice")
+        alice_client.login()
     alice_client["d"]["x"].metadata
     with fail_with_status_code(HTTP_403_FORBIDDEN):
         alice_client["d"]["x"].update_metadata(metadata={"added_key": 3})
     alice_client.logout()
 
 
 def test_create_blocked_by_access_policy(enter_username_password, context):
+    alice_client = from_context(context)
     with enter_username_password("alice", "secret1"):
-        alice_client = from_context(context, username="alice")
+        alice_client.login()
     with fail_with_status_code(HTTP_403_FORBIDDEN):
         alice_client["e"].write_array([1, 2, 3])
     alice_client.logout()
@@ -397,7 +403,8 @@ def test_service_principal_access(tmpdir):
     }
     with Context.from_app(build_app_from_config(config)) as context:
         with enter_username_password("admin", "admin"):
-            admin_client = from_context(context, username="admin")
+            # Prompts for login here because anonymous access is not allowed
+            admin_client = from_context(context)
         sp = admin_client.context.admin.create_service_principal("user")
         key_info = admin_client.context.admin.create_api_key(sp["uuid"])
         admin_client.write_array([1, 2, 3], key="x")