Can aiobotocore benefit from awscrt ?

[mdespriee]

Hey there,

The awscrt package is supposed to increase performance in the use of s3 https://aws.amazon.com/fr/blogs/storage/improving-amazon-s3-throughput-for-the-aws-cli-and-boto3-with-the-aws-common-runtime/

From my understanding, botocore automatically imports it if available in the environment, yet I see no clear improvement at all in having it.

Can aiobotocore use it and benefit from the said improvements ?

(Note: I'm not using aiobotocore directly, but s3fs)

[thehesiod]

you know what, I gave a very cursory look and I think it could indeed replace our use of aiohttp given it seems to support async streams. this would probably be a better endeavor than #1085. definitely worth a closer look

[jakkdl]

Given my experience of #1085 (with how tighly coupled the current design of aiobotocore is to aiohttp) I'm a bit scared of how fundamental the changes might be and effects on the codebase, API, tests, etc. But maybe that's worth it if it does turn out to be a better way to go about it

[Zac-HD]

aws-crt-python is syntatically synchronous, with a custom event-loop. It'd be a fair bit of work, but not especially difficult, to connect the callbacks to anyio events and channels and thereby get an idiomatic python library compatible with asyncio and Trio.

[jakkdl]

also see awslabs/aws-crt-python#558 for requesting anyio support be added upstream