In the image, a man dressed in a white shirt and gray pants stands in front of a screen or monitor. He appears to be delivering a speech or presentation, as he is making gestures with his hands. The screen behind him displays numbers, suggesting that he might be explaining or discussing some data or analysis. The man's posture and the presence of the screen indicate that he is the main focus of the image, and he is engaged in an activity that requires his attention and communication skills.
Text transcribed from the image:
Asynchrony: Overlapping GEMM and Softmax
Why overlapping?
Ex: headdim 128, block size 128 x 192
FP16 WGMMA: 2 x 2 x 128 x 192 x 128 = 12.6 MFLOPS, 4096 FLOPS/cycle -> 3072 cycles
MUFU.EX2: 128 x 192 = 24.6k OPS, 16 OPS/cycle -> 1536 cycles
A