-
-
Notifications
You must be signed in to change notification settings - Fork 361
Open
Description
Summary
Add optional uvloop integration to zarr-python to improve I/O performance for async operations.
Motivation
zarr-python performs many I/O operations, especially when working with remote storage backends (S3, GCS, etc.). uvloop is a fast, drop-in replacement for Python's default asyncio event loop that can provide significant performance improvements for I/O-intensive workloads.
Proposed Solution
- Automatically use uvloop when available and supported (non-Windows platforms)
- Make it configurable via
async.use_uvloop
config setting - Add uvloop as an optional dependency:
pip install 'zarr[optional]'
- Gracefully fall back to standard asyncio when uvloop is unavailable
- No breaking changes - fully backward compatible
Expected Benefits
- Improved performance for remote storage operations
- Better concurrent I/O handling
- Faster network-bound workloads
Configuration
# Enable (default when available)
zarr.config.set({"async.use_uvloop": True})
# Disable
zarr.config.set({"async.use_uvloop": False})
# Via environment variable
export ZARR_ASYNC__USE_UVLOOP=false
Implementation Notes
- uvloop only works on Unix-like systems (Linux, macOS)
- Automatic fallback on Windows
- Zero configuration required - works out of the box when installed
- uvloop will be an optional dependency
This enhancement would provide measurable performance improvements for users working with remote zarr stores while maintaining full compatibility with existing code.
Metadata
Metadata
Assignees
Labels
No labels