TLParse
December 22, 2024
https://github.com/ezyang/tlparse – or pip install tlparse
TIL: TunableOp in PyTorch
December 22, 2024
I wasn’t aware of this particular autotuning lever! There is a breakdown of TunableOpon the AMD blog from back in July:
nonzero_static in PyTorch
December 18, 2024
Natalia Gimelshein added cuda supportfor nonzero_staticto PyTorch the other day — its a feature Jax has had for a while that lets you avoid a bit of data dependent annoyingness.
Diff Risk Score Model
December 17, 2024
Just heard this was accepted to a conference, but had completely missed the preprint being released. I briefly supported a some of the folks involved in this work at Meta, and am very glad to see it being shared publicly:
Chinese Tech Term Glossary
December 15, 2024
Very interesting list, via Jeff Ding at ChinAI.
ChatGPT Canvas can make API requests now, but it’s complicated
December 15, 2024
Things that can go wrong with priority levels
December 13, 2024
I read this post years back as we did a big migration to Bazel for Android and iOS at Lyft, but was reminded of it from a recent HN comment by jmmv, the author:
Way to use torch.compile
December 11, 2024
Engineering Culture at Meta
December 11, 2024
A question I’ve been asked recently is how Meta compares to other places I’ve worked, or what makes it different.
A very lazy mental model for parallelisms
December 6, 2024
There is a great deal of technical complexity in distributed data parallel, model parallel, tensor parallel, fully-sharded-data-parallel, ZeRO 1/2/3, context parallel etc. To fit ideas into a picture I tend to segment in-practice parallelism into three buckets.