A patchwork quilt view of AI Alignment
June 15, 2025
Linear Layouts in Triton
June 11, 2025
Monarch: PyTorch Single Controller
June 10, 2025
I’ve been excited for this to make it to OSS: The PyTorch team at Meta recently soft-launched Monarch on Github.
Toward a Theory of Tokenization in LLMs
June 10, 2025
Analyzing Modern GPU Cores
June 5, 2025
Darwin Gödel Machines
June 1, 2025
Keeping a GPU busy is a lot about tiling
May 30, 2025
File this under the “gross oversimplifications” category. The basic approach to keeping GPUs busy is dividing the work into tiles, smaller sub-problems that make up the larger result. For a GEMM you might break the matrix into 128×128 or 128×64 tiles and let each CUDA thread block (CTA) own one tile. The GPU has many streaming multiprocessors (an A100 has 108) and every SM picks up one CTA at a time. If you want to know how many SMs your own card has you can call:
Metrics for Engineering Teams
May 28, 2025
Don’t blindly tie every piece of work to top-level metrics. Even if technically feasible, the cost is too high and the risk of spurious logic chains significant.
Scaling RL Compute
May 27, 2025
Fused Linear Cross-Entropy
May 24, 2025
Fused Linear Cross-Entropy is a popular optimization that combines the final linear projection and cross-entropy loss into a single operation. This fusion is very valuable for training large language models efficiently, as it can reduce memory usage significant, particularly for larger vocabularies.