Questions on internal path of an SQE

[DimitrisCharisis]

Hello to all.
I'm currently trying to understand the internal workings of io_uring, specifically the path that an SQE follows until it is served. I've been following the missing-manuals, but I'm having trouble reproducing all of the findings. For instance, running with IOSQE_ASYNC gives the following output, which seems odd, as it appears that all the requests pass through both the poll arm and the async path (run in 6.1.0-12-amd64):

 perf stat -a -e io_uring:io_uring_poll_arm,io_uring:io_uring_queue_async_work -- ./udp-read --async
^C./udp-read: Interrupt

 Performance counter stats for 'system wide':

             4,096      io_uring:io_uring_poll_arm
             4,096      io_uring:io_uring_queue_async_work

       3.697246867 seconds time elapsed

Based on my current understanding (which may be wrong) the execution modes of a request are:

• In the process context of the process that submitted the request, without blocking (inline execution). This is what it tries at the beginning. For example, I suspect that a buffered read request with data being in page cache will not block at all and will follow this path.
• If the result is not avalaible, but it can be polled for readiness, then it will follow the io_uring_poll_arm path. I'm not entirely sure how it eventually gets fulfilled from here on. In __io_arm_poll_handler(), which I suspect is where it is executed, I see an io_poll_can_finish_inline() call, which puzzles me regarding what exactly "inline" means.
• Finally, if neither of the above two conditions holds true or if the request was submitted with the IOSQE_ASYNC flag set, then the request will be placed in a queue. Depending on its nature (whether it can be finished in bounded time or not), it will be assigned to a different type of kernel thread.

However, in trying to understand more about the the async path, I wrote a relatively simple program that opens a file and reads 512 bytes with the IOSQE_ASYNC flag set. However, it doesn't hit the io_uring_queue_async_work trace point as I expected. I also tried flushing the caches before running it. Here's a sample of the code and the results from perf record:

[...]
        buf = malloc(BUF_SIZE);

        fd = open("test.txt", O_RDONLY);

        io_uring_queue_init(32, &ring, 0);

        sqe = io_uring_get_sqe(&ring);
        io_uring_sqe_set_flags(sqe, IOSQE_ASYNC);

        io_uring_prep_read(sqe, fd, buf, BUF_SIZE, 0);

        io_uring_submit(&ring);

[...]

And the results are the following, no io_uring_queue_async_work:

# perf script -i perf.data
            read  3401 [000] 37613.375869:           io_uring:io_uring_create: ring 0xffff946d0b903000, fd 5 sq size 32, cq size 64, flags 0x0
            read  3401 [000] 37613.375912:       io_uring:io_uring_submit_sqe: ring 0xffff946d0b903000, req 0xffff946d067b9800, user_data 0x0, opcode READ, flags 0x0, non block 1, sq_thread 0
            read  3401 [000] 37613.375914:         io_uring:io_uring_file_get: ring 0xffff946d0b903000, req 0xffff946d067b9800, user_data 0x0, fd 3
            read  3401 [000] 37613.376202:         io_uring:io_uring_complete: ring 0xffff946d0b903000, req 0xffff946d067b9800, user_data 0x0, result 512, cflags 0x0 extra1 0 extra2 0
            read  3401 [000] 37613.376205:    io_uring:io_uring_task_work_run: tctx 0xffff946d0e739c00, count 1, loops 1

Additionally, I've included an (incomplete) visualization of what I described above. I'm aware that it may not be correct, so I'd appreciate any corrections to help me better understand the entire process. Thank you in advance!

[axboe]

       io_uring_sqe_set_flags(sqe, IOSQE_ASYNC);
       io_uring_prep_read(sqe, fd, buf, BUF_SIZE, 0);

I think you'll have better luck if you do:

       io_uring_prep_read(sqe, fd, buf, BUF_SIZE, 0);
       io_uring_sqe_set_flags(sqe, IOSQE_ASYNC);

as any prep handler will clear the sqe, hence overwriting whatever you've put in it upfront (like the ASYNC flag).

[DimitrisCharisis]

Yes this works as expected, thank you @axboe.

If anyone has any tip on the figure I share in my previous post about the SQE path it would really helpful to share it!

[axboe]

For your diagram, the most important thing it's missing is the fast poll implementation that io_uring has. For that '???' box, the question before that should be "Can this file type be polled for data/space readiness?" and if the answer is yes and you go to that ??? box, the answer is "Arm a poll handler to get notified when we can retry the operation".

The io-wq / io worker side generally isn't very interesting, as most things should never hit that side. They are just a fallback for slower operations, generally, if we can't do sane nonblocking operations on them or they are not pollable (like regular files, for example).

Speaking of regular files, reading or writing to them with buffered IO has some special support as well. On the read side, it works kind of like polling, in that we kick off a read-ahead window that includes the requested range, and then we retry that similarly to reading from a socket when we get a callback that the readahead has finished. For writing, it's mostly just dirtying data. If we need to balance dirty pages, then it'll get punted to io-wq.

[DimitrisCharisis]

That was a very informative response. Thanks!

Speaking of regular files, reading or writing to them with buffered IO has some special support as well. On the read side, it works kind of like polling, in that we kick off a read-ahead window that includes the requested range, and then we retry that similarly to reading from a socket when we get a callback that the readahead has finished.

This action is taking place in process context of the process that made the submission right? The process submits let's say an IORING_OP_READ, in a file which is opened in buffered mode, then the kernel runs in this process context and fires a readahead (if the data are not present in the page-cache) and return control to the process. And when the readahead has finished it runs a callback to fulfill the CQE. This callback runs in interrupt context?

[axboe]

The readahead callback can indeed be in interrupt context, but finishing the original read operation and posting the CQE runs in the context of the original issuing task.