Serious scientists use FP64 – 64 bit floating point numbers – for high precision simulations, but in the world of machine learning we got by for the longest time with FP32. The perennial quest for increased FLOPS, particularly when memory bound, made even that seem too expensive though.

Helion: A High-Level DSL for Performant and Portable ML Kernels – PyTorch

GPT4o’s image generation was a remarkable event, beyond the brief Ghiblification of all social media.GPT-4o offered significantly more steerability than earlier image generation models,, while offering image quality in the ball park of the best diffusion models. Qwen-Image gives a similar level of fidelity and accuracy and is an open-weights model with a pretty decent technical report: QwenLM/Qwen-Image.

There’s a chart making the rounds that caused Tim Lee over at Understanding AI to rewrite his recent (excellent!) article aboutthe impact of AI on jobs. MIT’s Erik Brynjolfsson and colleagues found¹ that young workers in AI-exposed jobs² have seen their employment drop by 13% since ChatGPT arrived. Meanwhile, their older colleagues in the same fields are doing just fine.

A Primer on LLM Post-Training – PyTorch

You could have invented CuTe hierarchical layout (but maybe not the rest of it?) : ezyang’s blog

Extending AFM-4.5B to 64k Context Length

Pre-training is about making AI correct, post-training is about making AI helpful¹. That helpfulness is (primarily) shaped by reinforcement learning. RL for LLMs really took off with RLHF (RL from Human Feedback), which trained based on the score from a reward model.

I recently read a few posts that helped connect the dots on why Python is a) so successful as the lingua franca of ML b) also seems likely to be successful in the future¹.