Login/authentication enhancements (#1895)

* Login enhancements

* Apply suggestions from code review

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* example

* handle google colab errors

* fix contrib test

* fix 1 test

* fix tetst

* fix test

---------

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

Author: Wauplin

contrib/conftest.py

@@ -1,10 +1,11 @@
+import os
 import time
 import uuid
 from typing import Generator
 
 import pytest
 
-from huggingface_hub import HfFolder, delete_repo
+from huggingface_hub import delete_repo
 
 
 @pytest.fixture(scope="session")
@@ -20,18 +21,12 @@ def user() -> str:
 
 @pytest.fixture(autouse=True, scope="session")
 def login_as_dummy_user(token: str) -> Generator:
-    """Login with dummy user token on machine
-
-    Once all tests are completed, set back previous token."""
-    # Remove registered token
-    old_token = HfFolder().get_token()
-    HfFolder().save_token(token)
-
-    yield  # Run all tests
-
-    # Set back token once all tests have passed
-    if old_token is not None:
-        HfFolder().save_token(old_token)
+    """Login with dummy user token."""
+    # Cannot use `monkeypatch` fixture since we want it to be "session-scoped"
+    old_token = os.environ["HF_TOKEN"]
+    os.environ["HF_TOKEN"] = token
+    yield
+    os.environ["HF_TOKEN"] = old_token
 
 
 @pytest.fixture

docs/source/en/guides/cli.md

@@ -92,6 +92,8 @@ Your token has been saved to /home/wauplin/.cache/huggingface/token
 Login successful
 ```
 
+For more details about authentication, check out [this section](../quick-start#authentication).
+
 ## huggingface-cli whoami
 
 If you want to know if you are logged in, you can use `huggingface-cli whoami`. This command doesn't have any options and simply prints your username and the organizations you are a part of on the Hub:

docs/source/en/guides/hf_file_system.md

@@ -97,7 +97,7 @@ The same workflow can also be used for [Dask](https://docs.dask.org/en/stable/ho
 
 ## Authentication
 
-In many cases, you must be logged in with a Hugging Face account to interact with the Hub. Refer to the [Login](../quick-start#login) section of the documentation to learn more about authentication methods on the Hub. 
+In many cases, you must be logged in with a Hugging Face account to interact with the Hub. Refer to the [Authentication](../quick-start#authentication) section of the documentation to learn more about authentication methods on the Hub.
 
 It is also possible to login programmatically by passing your `token` as an argument to [`HfFileSystem`]:
 

docs/source/en/guides/inference.md

@@ -97,7 +97,7 @@ code as before, changing only the `model` parameter:
 
 Calls made with the [`InferenceClient`] can be authenticated using a [User Access Token](https://huggingface.co/docs/hub/security-tokens).
 By default, it will use the token saved on your machine if you are logged in (check out
-[how to login](https://huggingface.co/docs/huggingface_hub/quick-start#login)). If you are not logged in, you can pass
+[how to authenticate](https://huggingface.co/docs/huggingface_hub/quick-start#authentication)). If you are not logged in, you can pass
 your token as an instance parameter:
 
 ```python

docs/source/en/guides/repository.md

@@ -27,7 +27,7 @@ explanation page for more details.
 </Tip>
 
 If you want to create and manage a repository on the Hub, your machine must be logged in. If you are not, please refer to
-[this section](../quick-start#login). In the rest of this guide, we will assume that your machine is logged in.
+[this section](../quick-start#authentication). In the rest of this guide, we will assume that your machine is logged in.
 
 ## Repo creation and deletion
 

docs/source/en/guides/upload.md

@@ -11,31 +11,7 @@ Sharing your files and work is an important aspect of the Hub. The `huggingface_
 - with the `commit` context manager.
 - with the [`~Repository.push_to_hub`] function.
 
-Whenever you want to upload files to the Hub, you need to log in to your Hugging Face account:
-
-- Log in to your Hugging Face account with the following command:
-
-  ```bash
-  huggingface-cli login
-  # or using an environment variable
-  huggingface-cli login --token $HUGGINGFACE_TOKEN
-  ```
-
-- Alternatively, you can programmatically login using [`login`] in a notebook or a script:
-
-  ```python
-  >>> from huggingface_hub import login
-  >>> login()
-  ```
-
-  If ran in a Jupyter or Colaboratory notebook, [`login`] will launch a widget from
-  which you can enter your Hugging Face access token. Otherwise, a message will be
-  prompted in the terminal.
-
-  It is also possible to login programmatically without the widget by directly passing
-  the token to [`login`]. If you do so, be careful when sharing your notebook. It is
-  best practice to load the token from a secure vault instead of saving it in plain in
-  your Colaboratory notebook.
+Whenever you want to upload files to the Hub, you need to log in to your Hugging Face account. For more details about authentication, check out [this section](../quick-start#authentication).
 
 ## Upload a file
 

docs/source/en/package_reference/environment_variables.md

@@ -49,7 +49,7 @@ Defaults to `"$HF_HOME/assets"` (e.g. `"~/.cache/huggingface/assets"` by default
 To configure the User Access Token to authenticate to the Hub. If set, this value will
 overwrite the token stored on the machine (in `"$HF_HOME/token"`).
 
-See [login reference](../package_reference/login) for more details.
+For more details about authentication, check out [this section](../quick-start#authentication).
 
 ### HF_HUB_VERBOSITY
 

docs/source/en/package_reference/hf_api.md

@@ -109,15 +109,6 @@ Below are the supported values for [`CommitOperation`]:
 
 [[autodoc]] CommitScheduler
 
-## Token helper
-
-`huggingface_hub` stores the authentication information locally so that it may be re-used in subsequent
-methods.
-
-It does this using the [`HfFolder`] utility, which saves data at the root of the user.
-
-[[autodoc]] HfFolder
-
 ## Search helpers
 
 Some helpers to filter repositories on the Hub are available in the `huggingface_hub` package.

docs/source/en/package_reference/login.md

@@ -4,8 +4,9 @@ rendered properly in your Markdown viewer.
 
 # Login and logout
 
-The `huggingface_hub` library allows users to programmatically login and logout the
-machine to the Hub.
+The `huggingface_hub` library allows users to programmatically login and logout the machine to the Hub.
+
+For more details about authentication, check out [this section](../quick-start#authentication).
 
 ## login
 

docs/source/en/quick-start.md

@@ -51,45 +51,75 @@ full-length hash instead of the shorter 7-character commit hash:
 
 For more details and options, see the API reference for [`hf_hub_download`].
 
-## Login
+<a id="login"></a> <!-- backward compatible anchor -->
 
-In a lot of cases, you must be logged in with a Hugging Face account to interact with
+## Authentication
+
+In a lot of cases, you must be authenticated with a Hugging Face account to interact with
 the Hub: download private repos, upload files, create PRs,...
 [Create an account](https://huggingface.co/join) if you don't already have one, and then sign in
 to get your [User Access Token](https://huggingface.co/docs/hub/security-tokens) from
 your [Settings page](https://huggingface.co/settings/tokens). The User Access Token is
 used to authenticate your identity to the Hub.
 
-Once you have your User Access Token, run the following command in your terminal:
+<Tip>
+
+Tokens can have `read` or `write` permissions. Make sure to have a `write` access token if you want to create or edit a repository. Otherwise, it's best to generate a `read` token to reduce risk in case your token is inadvertently leaked.
+
+</Tip>
+
+### Login command
+
+The easiest way to authenticate is to save the token on your machine. You can do that from the terminal using the [`login`] command:
 
 ```bash
 huggingface-cli login
-# or using an environment variable
-huggingface-cli login --token $HUGGINGFACE_TOKEN
 ```
 
+The command will tell you if you are already logged in and prompt you for your token. The token is then validated and saved in your `HF_HOME` directory (defaults to `~/.cache/huggingface/token`). Any script or library interacting with the Hub will use this token when sending requests.
+
 Alternatively, you can programmatically login using [`login`] in a notebook or a script:
 
 ```py
 >>> from huggingface_hub import login
 >>> login()
 ```
 
-It is also possible to login programmatically without being prompted to enter your token by directly
-passing the token to [`login`] like `login(token="hf_xxx")`. If you do so, be careful when
-sharing your source code. It is a best practice to load the token from a secure vault instead
-of saving it explicitly in your codebase/notebook.
+You can only be logged in to one account at a time. Logging in to a new account will automatically log you out of the previous one. To determine your currently active account, simply run the `huggingface-cli whoami` command.
+
+<Tip warning={true}>
+
+Once logged in, all requests to the Hub - even methods that don't necessarily require authentication - will use your access token by default. If you want to disable the implicit use of your token, you should set `HF_HUB_DISABLE_IMPLICIT_TOKEN=1` as an environment variable (see [reference](../package_reference/environment_variables#hfhubdisableimplicittoken)).
+
+</Tip>
+
+### Environment variable
+
+The environment variable `HF_TOKEN` can also be used to authenticate yourself. This is especially useful in a Space where you can set `HF_TOKEN` as a [Space secret](https://huggingface.co/docs/hub/spaces-overview#managing-secrets).
+
+<Tip>
+
+**NEW:** Google Colaboratory lets you define [private keys](https://twitter.com/GoogleColab/status/1719798406195867814) for your notebooks. Define a `HF_TOKEN` secret to be automatically authenticated!
+
+</Tip>
+
+Authentication via an environment variable or a secret has priority over the token stored on your machine.
+
+### Method parameters
+
+Finally, it is also possible to authenticate by passing your token to any method that accepts `token` as a parameter.
+
+```
+from transformers import whoami
+
+user = whoami(token=...)
+```
 
-You can be logged in only to 1 account at a time. If you login your machine to a new account, you will get logged out
-from the previous. Make sure to always which account you are using with the command `huggingface-cli whoami`.
-If you want to handle several accounts in the same script, you can provide your token when calling each method. This
-is also useful if you don't want to store any token on your machine.
+This is usually discouraged except in an environment where you don't want to store your token permanently or if you need to handle several tokens at once.
 
 <Tip warning={true}>
 
-Once you are logged in, all requests to the Hub -even methods that don't necessarily require authentication- will use your
-access token by default. If you want to disable implicit use of your token, you should set the
-`HF_HUB_DISABLE_IMPLICIT_TOKEN` environment variable.
+Please be careful when passing tokens as a parameter. It is always best practice to load the token from a secure vault instead of hardcoding it in your codebase or notebook. Hardcoded tokens present a major leak risk if you share your code inadvertently.
 
 </Tip>
 

setup.cfg

@@ -61,5 +61,7 @@ max-line-length = 119
 # --durations=0          -> print execution time of each test
 addopts = -Werror::FutureWarning --log-cli-level=INFO -sv --durations=0
 env =
+    HF_TOKEN=
+    HUGGING_FACE_HUB_TOKEN=
     HUGGINGFACE_CO_STAGING=1
     DISABLE_SYMLINKS_IN_WINDOWS_TESTS=1

src/huggingface_hub/__init__.py

@@ -63,6 +63,7 @@
         "InferenceEndpointType",
     ],
     "_login": [
+        "get_token",
         "interpreter_login",
         "login",
         "logout",
@@ -406,6 +407,7 @@ def __dir__():
         InferenceEndpointType,  # noqa: F401
     )
     from ._login import (
+        get_token,  # noqa: F401
         interpreter_login,  # noqa: F401
         login,  # noqa: F401
         logout,  # noqa: F401