TIL: weights-only model loading will be the default in PyTorch 2.6
January 8, 2025
I had missed this, but weights-only is going to be the default for torch.load in Pytorch 2.6:
Deepseek V3
January 6, 2025
TIL: torchdbg
January 2, 2025
HuatuoGPT-o1, domain specific reasoning
January 1, 2025
Thunderkittens /GPUs go brr
December 31, 2024
Just got round to reading the intro post to the (now improved) thunderkittens kernel DSL.
Generalization progress in RLHF
December 29, 2024
Ways to use torch.export
December 23, 2024
TLParse
December 22, 2024
https://github.com/ezyang/tlparse – or pip install tlparse
TIL: TunableOp in PyTorch
December 22, 2024
I wasn’t aware of this particular autotuning lever! There is a breakdown of TunableOpon the AMD blog from back in July:
nonzero_static in PyTorch
December 18, 2024
Natalia Gimelshein added cuda supportfor nonzero_staticto PyTorch the other day — its a feature Jax has had for a while that lets you avoid a bit of data dependent annoyingness.