Generative modelling in latent space – Sander Dieleman

Kapil Sharma from the PyTorch team has a great series of posts diving into the Triton compiler process: 1, 2, 3. As covered there, Triton lowers to a series of intermediate representations, and each level has a set of transformational passes that implement optimizations. TTIR is the generic Triton IR and leverages a number of standard MLIR passes like common subexpression elimination, as well as some Triton specific passes like managing broadcast ops. That’s then lowered to TTGIR, a GPU-specific IR¹

rui314/mold: Mold: A Modern Linker 🦠

https://www.michaelpj.com/blog/2024/09/03/lsp-good-bad-ugly.html

GPU MODE 4/19 Q&A – Google Docs

torch.compile offers some knobs for controlling the trade-off of execution performance with longer compile times. This is particularly useful for inference, where the same model will be running for a long time.

There are a couple of different options to profile a Triton kernel.

JEPAis an example of a predictive coding architecture. Predictive coding is the idea that the brain works by predicting the next sensory input and learns from the error between the prediction and the actual input. Hierarchies of this kind of prediction allow higher level elements to predict the outputs of lower-level elements, building up deeper and more complex semantics.

As Thomas posted on the Python Discuss board there is a discord for discussing Free-Threaded/NoGil Python: https://discord.gg/rqgHCDqdRr