In the image, a man wearing a grey shirt is giving a presentation in front of a projector screen. He has his arms stretched out, and his hands are raised, presumably gesturing to emphasize a point. Two people, both with their backs turned, are sitting behind the man, likely attending the presentation. The room features a bookshelf on the wall, with several objects placed on it, including a laptop, a lamp, and a cup. A whiteboard is visible in the background, suggesting that the presentation may touch on topics related to the information displayed on it. Overall, the scene conveys a sense of collaboration and learning as the man shares his insights with the people in the room.
Text transcribed from the image:
Asynchrony: Overlapping GEMM and Softmax
Why overlapping?
Ex: headdim 128, block size 128 x 192
FP16 WGMMA: 2 x 2 x 128 x 192 x 128 = 12.6 MFLOPS, 4096 FLOPS/cycle -> 3072 cycles
MUFU.EX2: 128 x 192 = 24.6k OPS, 16 OPS/cycle -> 1536 cycles
Omniscience
AI PAPERS AND B
OMNISCIENCE.T