Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make async dependencies optional. #2858

Open
wants to merge 15 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 12 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -97,10 +97,12 @@ Install with pip:
$ pip install httpx
```

Or, to include the optional HTTP/2 support, use:
There are also a number of optional dependancies.

For example to include asyncio and HTTP/2 support, use:

```shell
$ pip install httpx[http2]
$ pip install 'httpx[asyncio,http2]'
```

HTTPX requires Python 3.8+.
Expand Down Expand Up @@ -129,15 +131,17 @@ The HTTPX project relies on these excellent libraries:
* `h11` - HTTP/1.1 support.
* `certifi` - SSL certificates.
* `idna` - Internationalized domain name support.
* `sniffio` - Async library autodetection.

As well as these optional installs:

* `h2` - HTTP/2 support. *(Optional, with `httpx[http2]`)*
* `socksio` - SOCKS proxy support. *(Optional, with `httpx[socks]`)*
* `rich` - Rich terminal support. *(Optional, with `httpx[cli]`)*
* `click` - Command line client support. *(Optional, with `httpx[cli]`)*
* `brotli` or `brotlicffi` - Decoding for "brotli" compressed responses. *(Optional, with `httpx[brotli]`)*
* `anyio` - Async support for `asyncio`. *(Optional, with `httpx['asyncio']`)*
* `trio` - Async support for `trio`. *(Optional, with `httpx['trio']`)*
* `h2` - HTTP/2 support. *(Optional, with `httpx['http2']`)*
* `socksio` - SOCKS proxy support. *(Optional, with `httpx['socks']`)*
tomchristie marked this conversation as resolved.
Show resolved Hide resolved
* `click` - Command line client support. *(Optional, with `httpx['cli']`)*
* `rich` - Command line client support. *(Optional, with `httpx['cli']`)*
* `pygments` - Command line client support. *(Optional, with `httpx['cli']`)*
* `brotli` or `brotlicffi` - Decoding for "brotli" compressed responses. *(Optional, with `httpx['brotli']`)*
* `zstandard` - Decoding for "zstd" compressed responses. *(Optional, with `httpx[zstd]`)*

A huge amount of credit is due to `requests` for the API layout that
Expand Down
18 changes: 18 additions & 0 deletions docs/async.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,24 @@ long-lived network connections such as WebSockets.
If you're working with an async web framework then you'll also want to use an
async client for sending outgoing HTTP requests.

## Enabling Async support

To enable async support you'll need to install some additional dependencies:

If you're using Python's [standard `asyncio` support](https://docs.python.org/3/library/asyncio.html) then:

```shell
$ pip install "httpx[asyncio]"
```

Or, if you're working with the [`trio` third party package](https://trio.readthedocs.io/en/stable/):

```shell
$ pip install httpx['trio']
```

We highly recommend `trio` for async support. The `trio` project [pioneered the principles of structured concurrency](https://en.wikipedia.org/wiki/Structured_concurrency), and has a more carefully constrained API against which to work from.

## Making Async requests

To make asynchronous requests, you'll need an `AsyncClient`.
Expand Down
2 changes: 1 addition & 1 deletion docs/http2.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ trying out our HTTP/2 support. You can do so by first making sure to install
the optional HTTP/2 dependencies...

```shell
$ pip install httpx[http2]
$ pip install 'httpx[http2]'
```

And then instantiating a client with HTTP/2 support enabled:
Expand Down
6 changes: 3 additions & 3 deletions httpx/_transports/asgi.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,6 @@

import typing

import sniffio

from .._models import Request, Response
from .._types import AsyncByteStream
from .base import AsyncBaseTransport
Expand All @@ -28,7 +26,9 @@
__all__ = ["ASGITransport"]


def create_event() -> Event:
def create_event() -> "Event":
import sniffio

if sniffio.current_async_library() == "trio":
import trio

Expand Down
8 changes: 8 additions & 0 deletions httpx/_transports/default.py
Original file line number Diff line number Diff line change
Expand Up @@ -277,6 +277,14 @@ def __init__(
retries: int = 0,
socket_options: typing.Iterable[SOCKET_OPTION] | None = None,
) -> None:
try:
import sniffio # noqa: F401
except ImportError: # pragma: nocover
raise RuntimeError(
"Using httpx in async mode, but neither "
"httpx['asyncio'] or asyncio['trio'] is installed."
)
tomchristie marked this conversation as resolved.
Show resolved Hide resolved

ssl_context = create_ssl_context(verify=verify, cert=cert, trust_env=trust_env)
proxy = Proxy(url=proxy) if isinstance(proxy, (str, URL)) else proxy

Expand Down
17 changes: 2 additions & 15 deletions httpx/_utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,6 @@
from pathlib import Path
from urllib.request import getproxies

import sniffio

from ._types import PrimitiveData

if typing.TYPE_CHECKING: # pragma: no cover
Expand Down Expand Up @@ -291,29 +289,18 @@ def peek_filelike_length(stream: typing.Any) -> int | None:


class Timer:
async def _get_time(self) -> float:
library = sniffio.current_async_library()
if library == "trio":
import trio

return trio.current_time()
else:
import asyncio

return asyncio.get_event_loop().time()

def sync_start(self) -> None:
self.started = time.perf_counter()

async def async_start(self) -> None:
self.started = await self._get_time()
self.started = time.perf_counter()

def sync_elapsed(self) -> float:
now = time.perf_counter()
return now - self.started

async def async_elapsed(self) -> float:
now = await self._get_time()
now = time.perf_counter()
return now - self.started


Expand Down
14 changes: 9 additions & 5 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -29,10 +29,8 @@ classifiers = [
]
dependencies = [
"certifi",
"httpcore==1.*",
"anyio",
"httpcore>=1.0.0,<2.0.0",
"idna",
"sniffio",
]
dynamic = ["readme", "version"]

Expand All @@ -47,10 +45,16 @@ cli = [
"rich>=10,<14",
]
http2 = [
"h2>=3,<5",
"httpcore[http2]",
]
socks = [
"socksio==1.*",
"httpcore[socks]",
]
asyncio = [
"httpcore[asyncio]"
]
trio = [
"httpcore[trio]"
]
Comment on lines 47 to 58
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is interesting... currently we're including the httpcore dependencies explicitly.

Eg. pip install httpx[socks] uses the dependency socksio==1.* instead of the dependency httpcore[socks].

The second style makes more sense, because that's what we're actually trying to effect... but I'm a bit hesitant that some Python install tooling might incorrectly fail to install httpcore[socks] as an optional dependency when the httpcore dependancy is already included. Is my hesitancy well founded here, or are we okay to use this new style throughout?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this should always work but you're right that it's tool dependent and the Python specification around this does not provide guarantees.

zstd = [
"zstandard>=0.18.0",
Expand Down
3 changes: 1 addition & 2 deletions requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
# On the other hand, we're not pinning package dependencies, because our tests
# needs to pass with the latest version of the packages.
# Reference: https://github.com/encode/httpx/pull/1721#discussion_r661241588
-e .[brotli,cli,http2,socks,zstd]
-e .[asyncio,trio,brotli,cli,http2,socks,zstd]

# Optional charset auto-detection
# Used in our test cases
Expand All @@ -24,6 +24,5 @@ mypy==1.11.2
pytest==8.3.2
ruff==0.6.3
trio==0.26.2
trio-typing==0.10.0
trustme==1.1.0
uvicorn==0.30.6
Loading