bertmaher/simplegemm

Nathan Lambert of the Allen Institute writes abouttheir (very strong) Olmo 2 32B release, and the just released Gemma 3 model from Google. One of the many interesting points:

Jerry Zhang recently posted a coupleof updates on the evolution of the quantization APIs in PyTorch, and the unification around TorchAO.

Performance of the Python 3.14 tail-call interpreter – Made of Bugs

https://www.thonking.ai/p/why-pytorch-is-an-amazing-place-to

The GPU Mode discord has emerged as the preeminent hub for current and aspiring GPU kernel hackers, and several of the folks there have kicked off a project to help make it easier for folks to write and benchmark them. https://gpu-mode.github.io/discord-cluster-manager/docs/intro/ goes over the idea, but it’s a series of leaderboards and runners for different kernel types so you can easily find (and beat!) the state of the art:

https://github.com/deepseek-ai/FlashMLA/

https://arxiv.org/abs/2502.03373

Who needs a tokenizer anyway!

In general, branching in GPU code is considered bad. When you write a kernel, it’s very easy to write the same kind of logic as you would on a CPU. However, GPU kernels execute on blocks of threads scheduled on streaming multiprocessors (SMs), and they are optimized for vectorized (or parallel) computation. This optimization relies on the idea that large groups of threads can execute the same instructions on different data in a lockstep fashion. Practically, these are scheduled as “warps” of 32 threads at a time (on Nvidia, the equivalent in AMD is 64 threads).