AVX512 (3 of 3): Deep Dive into AVX512 Mechanisms
Added 2020-10-20 00:26:22 +0000 UTCFinal vid of the AVX 512 adventures! In this one, we look at the interesting, low level mechanisms that AVX 512 offers. There is a lot more to this instruction set than just doubling the register size. It's really great to see some genuinely new, experimental and colorful and powerful things being added to x86/64.
Thank you for the support, and have a really great day :)
Comments
great! My immediate instinct was to focus my attention on: VFMADD231PS zmm1 {k1}{z}, zmm2, zmm3/m512/m32bcst{er} because it does most of the work: multiplying 2 vectors zmm2 and zmm3/{m32bcst} and adding the result cumulatively to zmm1. Excellent instruction but comes with a latency of 5.8c (at least) and a throughput of 0.75. it can also be used to broadcast 1 scalar from memory into third vector and multiply this vector with zmm2. This is a core operation matrix multiplication when you "parallelize" it. However, the memory access latency will be high (even if m32 is in L1 cach) for every VFMADD231PS execution, compared to using the zmm3 register natively. Therefore it might be better to read A and B by vectors and store in zmm2, zmm3 temporarily and use VPERMPS zmm, zmm, zmm to broad cast from z to z regs - faster with only one read from memory (one entire 64 byte cache line)... So the implementation decision, exactly how to maximize throughput of VFMADD231PS, will come down to how much can be "pipelined", m32 l1 latency/relative to zmm, and then just minimizing the total cycles per loop iteration by TESTING various ideas: unrolled loop, reordering of instructions to keep pipeline as full as possible. Clear to me is that whole 16x16 C matrix needs to be kept in 16 zmm registers the entire mm calculation since it is read/write accessed so much (will also be less confusion in l1/l2 cache competing with A and B matrices reads) and then write the final C result to RAM in the very end. That's the plan.
DriftWood
2020-10-21 18:11:52 +0000 UTCIt’s a fantastic instructions set! Super complicated though, so I’m not sure if we’ll see its true potential for a while yet! Really love matrix kernels!! :) I remember doing a bunch a while back. They were fast, but didn’t tile as well as I’d hoped. I was benching against Intel’s MKL. My kernels were much faster, but the MKL won out for larger matrices. Anywho, I used AVX (didn’t have AVX512 at the time), the elements from the one of the matrices were broadcast, to avoid transposing anything but still reuse the data for as many FMADD’s as possible. Used blocking in L1 and regs. That’s about all I can remember. Maybe I shouldda blocked in L2 as well? Maybe Intel is doing a bit of cheeky Strassen…? Not sure if that will help, but that’s a great topic for a video! Thank you for the suggestion and thanks for watching mate :)
What's a Creel?
2020-10-21 03:02:54 +0000 UTCPlaying around with AVX512, I've fond it to be completely underrated and have gotten an undeservedly bad reputation (LinusT. in particular). The thing is that AVX512, in my limited experience, is extremely powerful and versatile IF you know what you are doing. that that if is a big IF. i can understand that people without a thorough background in linear algebra and who did not invest plenty of time into studying AVX512 assembly, would have a hard time to extract performance out of AVX512 - it's comlex. One thing you could do to help, would be to extend this AVX512 series with a more advanced AVX512 programming and solve a bit more complex problems with focus on efficiency minimizing clock cycles and mem/cache latency. Organizing data the right way in RAM matters hugely so you can feed the 2 AVX512 ports (port 0 and 5) and keep them busy with minimum memory/cache latency. There is so much parallelism in the CPU but it is very hard to use all effciciently. One problem I started to work on is an optimal matrix multiplication of two 16x6 matrix panels, i.e. A*B =C panel multiplications is the inner most loop in most efficient matrix multiplications used.
DriftWood
2020-10-20 23:12:08 +0000 UTC
