PyTorch and Python Free Threading
April 3, 2025
Understanding R1-Zero-Like training
April 1, 2025
Bank Conflicts in Shared Memory
March 29, 2025
When data is in the global memory on a GPU it’s usually in row-major or column-major order. Loading from global memory is quite slow though, so for performance we want to move the data to shared memory for the threads in a warp to work on.
Useful Reasoning Behaviors
March 28, 2025
Scarcity & Abundance
March 27, 2025
Muon optimizer
March 23, 2025
Last year Keller Jordan at OpenAI beat some of the existing NanoGPT speedrun records thanks to some optimizer improvements. Towards the end of the year the work was formalized as the Muonoptimizer, and it’s making waves in a bunch of areas now
What if your worry problem is really a planning problem?
March 21, 2025
| [What if your worry problem is really a planning problem? | Psyche Ideas](https://psyche.co/ideas/what-if-your-worry-problem-is-really-a-planning-problem) |
Libraries not Frameworks & Training vs Agentic loops
March 20, 2025
A couple of conversations I was in last week around agentic system design reminding me of Brandon Smith’s excellent write libraries, not frameworks post
A HBS paper on the PyTorch Foundation
March 19, 2025
Vibe Coding & Code Review
March 19, 2025
Vibe coding is trendy right now, and it’s part of a bigger shift toward AI-generated code. One of the questions I have about this, at scale, is how it will interact with code review.