Skip to content

Feature Proposal: Add optional uvloop support for improved async performance #3451

@jhamman

Description

@jhamman

Summary

Add optional uvloop integration to zarr-python to improve I/O performance for async operations.

Motivation

zarr-python performs many I/O operations, especially when working with remote storage backends (S3, GCS, etc.). uvloop is a fast, drop-in replacement for Python's default asyncio event loop that can provide significant performance improvements for I/O-intensive workloads.

Proposed Solution

  • Automatically use uvloop when available and supported (non-Windows platforms)
  • Make it configurable via async.use_uvloop config setting
  • Add uvloop as an optional dependency: pip install 'zarr[optional]'
  • Gracefully fall back to standard asyncio when uvloop is unavailable
  • No breaking changes - fully backward compatible

Expected Benefits

  • Improved performance for remote storage operations
  • Better concurrent I/O handling
  • Faster network-bound workloads

Configuration

# Enable (default when available)
zarr.config.set({"async.use_uvloop": True})

# Disable
zarr.config.set({"async.use_uvloop": False})
# Via environment variable
export ZARR_ASYNC__USE_UVLOOP=false

Implementation Notes

  • uvloop only works on Unix-like systems (Linux, macOS)
  • Automatic fallback on Windows
  • Zero configuration required - works out of the box when installed
  • uvloop will be an optional dependency

This enhancement would provide measurable performance improvements for users working with remote zarr stores while maintaining full compatibility with existing code.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions