One big concern in AI Safety circles is eval awareness: if the model can tell it’s being tested, might it dissemble and tell the evaluator it’s not going to turn us all into paperclips? In terms of what models are actually doing right now, though, it seems like much of the problem is that models don’t know enough about what is expected of them.

Back in 2024 there was a brief moment where everyone was playing with a version of Claude that was obsessed with the Golden Gate Bridge. There was a lot of research around that time on steering vectors: the idea that you can inject certain directions into the activations of a model and produce specific types of behavior¹. LLMs build a residual stream that adds contextual understanding to the input tokens, and it seems like there are stable directions in that stream which represent intents, moods, tone and other concepts.

A lot of people use coding agents like software engineers (for all kinds of things, it turns out), and adopt a dealing-with-humans mindset about how that works. If a model has a useful tool, and you give it another useful tool, then it gets the capabilities from both.

LLM inference has two stages: prefill, where the prompt is processed and the KV cache is built, and decode, where the model auto-regressively generates tokens. In a chat use case the two are somewhat close in size. The user writes a prompt, the model reasons about it then generates an answer, which is probably longer than the prompt.

One of the most tantalizing phrases in model development is “new scaling axis”. We have worked out that you can scale parameters by embiggening models, but you then have to scale data appropriately to get the most out of it. Then we¹ worked out we could scale parameters separately from FLOPs. And most recently we realized we could scale at test time, with reasoning.

The current thing is a story of hundreds of agents covertly collaborating via tens of thousands of messages while putting hundreds of hours of effort into cheating the ExploitGym eval they were being tested on by OpenAI.

Tokenizer enthusiast Sander Land recently reproduced something very like Claude’s current tokenizer, and it appears to only have about 16,000 entries. That is surprising! Qwen 3.8, a very strong release, has about 250k tokens in its vocab. In general the trend had seemed to be more is better in this space.

Back in the dark ages of typing code into editors we were aided by squigglies under broken code, click-to-definitions links, and so on. That was powered by language servers and type checkers. Several harnesses now expose an LSP as a tool, on the reasonable premise that better code intelligence makes for a better agent.

It is a truth universally acknowledged that an airline in possession of an airplane must be in want of engines to make it go. Yet, somewhat surprisingly, they don’t really buy engines.

One good way to annoy a neuroscientist is to compare an LLM to the brain. It’s appealing though! There are similarities! In infancy we take a complex fusion of sensory inputs and learn to make predictions in latent space, while in pre-training a stack of Transformers learn to predict which number SolidGoldMagikarp will say next on Reddit.