[QST]Inquiry About the Computation Size in a Single cute::gemm Call in CUTLASS

[ziyuhuang123]

What is your question?
Could you please explain how large a single cute::gemm computation is in CUTLASS? Since multiple threads compute together, and it doesn’t explicitly state the number of iterations like CUDA cores do, I find it a bit confusing.

[Junkai-Wu]

A single cute::gemm computes a single MMA instruction. Therefore, how large a single cute::gemm computation is depends on the MMA instruction you use. For example, if you use Ampere MMA with input type is fp16 which computes 16x8x16 (mxnxk) in a single instruction, the corresponding cute::gemm computes matrix multiplication size of 16x8x16 in each call. And since this MMA is a warp level mma, all threads in a warp will compute this instruction together at each time.