DeepSeek R1 & GRPO

January 25, 2025

DeepSeek dropped another quality release with their R1 series. The models are an exploration of how to improve improving reasoning capabilities in LLMs. They’ve released a crop of models, which, despite some quirks in its licensing, are extremely helpful. The paper continues the trend of being clear and open.

TIL: You don’t need PIL for decoding images with TorchVision

January 21, 2025

pytorch.org/vision/main/generated/torchvision.io.decode_image

Vita 1.5 – how to train an end-to-end speech model

January 20, 2025

GitHub – VITA-MLLM/VITA: ✨✨VITA-1.5: Towards GPT-4o Level Real-Time Vision and Speech Interaction

Functionalization in PyTorch

January 16, 2025

Functionalization in PyTorch: Everything You Wanted To Know – compiler – PyTorch Developer Mailing List

Idle Speculation on GPU Capacity Management

January 12, 2025

Training large models today is tightly coupled to specific hardware. This makes moving workloads across systems or abstracting the hardware almost impossible without losing efficiency, and hence why you don’t tend to see a lot of uptake of the kind of cloud-like abstractions we see elsewhere.