Fantastic tip from Alban, particularly useful when you have a giant model and limited VRAM.
The short answer is this 30 lines TorchDispatchMode that tracks all Tensor memory use
Accidental Factors
Needless text
Needless text
Fantastic tip from Alban, particularly useful when you have a giant model and limited VRAM.
The short answer is this 30 lines TorchDispatchMode that tracks all Tensor memory use