GRPO (Group Relative Policy Optimization) is an RL technique originally proposed in the DeepSeekMathpaper. Instead of using a full-blown value network like PPO does, GRPO samples a group of completions for a given prompt and then computes a relative (normalized) reward for each output. The rewards are “verifiable” because they come from checking the final answer against ground truth and confirming. E.g. does the response follow the expected format (i.e. a <think>…</think> block for reasoning and an <answer>…</answer> block for the solution) and is the answer accurate against a predetermined fact. Not every problem fits this model, but there are a bunch that do, including math reasoning with the GSM8Kdataset of grade-school math word problems. These look like this:

This weekend I was reading the Tulu v3 paper (link), which offers a deep dive into building robust post-training setups. This is an very good resource for anyone aiming to build a really robust fine-tuning workflows. It covers critical elements like data set selection, synthetic data generation (with example prompts!), strategies for SFT and preference tuning, and various things they struggled with.

DeepSeek dropped another quality release with their R1 series. The models are an exploration of how to improve improving reasoning capabilities in LLMs. They’ve released a crop of models, which, despite some quirks in its licensing, are extremely helpful. The paper continues the trend of being clear and open.

pytorch.org/vision/main/generated/torchvision.io.decode_image

GitHub – VITA-MLLM/VITA: ✨✨VITA-1.5: Towards GPT-4o Level Real-Time Vision and Speech Interaction

Functionalization in PyTorch: Everything You Wanted To Know – compiler – PyTorch Developer Mailing List

Training large models today is tightly coupled to specific hardware. This makes moving workloads across systems or abstracting the hardware almost impossible without losing efficiency, and hence why you don’t tend to see a lot of uptake of the kind of cloud-like abstractions we see elsewhere.

I had missed this, but weights-only is going to be the default for torch.load in Pytorch 2.6:

https://github.com/deepseek-ai/DeepSeek-V3/tree/main

https://github.com/ezyang/torchdbg