Alphafold3 PyTorch
September 24, 2024
The always prolific Lucidrains has been implementing AlphaFold3 in PyTorch:
Thinking about high quality human data
July 14, 2024
What are NPUs
June 28, 2024
Daniel Schmachtenberger’s Questions
June 24, 2024
HSTU
June 24, 2024
There have been a few, good, attempts at Transformer based recommendation systems, but this one from Jiaqi Zhai and the folks inside the modern recommendation systems team at Meta is particularly strong.
On what neural net architectures actually make a difference
May 24, 2024
TIL: How to measure memory usage from your PyTorchmodel without running it
April 22, 2024
Fantastic tip from Alban, particularly useful when you have a giant model and limited VRAM.
Timeline on the Google Reader shutdown
April 21, 2024
Interesting to read this breakdown of what was happening with Reader: I recall the discussions internally at Google and knew some folks floating around it (including at least one person who volunteered to take on maintenance) but this was largely new to me. Uncomfortably enough was on stage giving a talk for Google the day the shut down happened. I asked for questions, got a lot of hands, then qualified I was looking for questions about the talk and not Reader, and most of the hands went down.
DiLoCo
February 22, 2024
Inductor notes
January 16, 2024
Inductor is PyTorch’s compiler backend designed to optimize and generate high-performance code for arbitrary models. It works over a few phases: