Llm

Sparse Autoencoders (SAEs) and Cross-Layer Transcoders (CLTs) are two approaches to interpretability of transformer models. Read up on what they're good for and how they differ.

Practical tips on how to design single and multi-agent systems. A synthesis of emerging principles and patterns as of June 2025.

This blog post explores Induction Circuits, an example of a learned mini algorithm of transformer models that runs at inference time.

A brief history of LLM Scaling Laws from compute-optimal training and inference to scaling test-time compute and whether Scaling Laws are coming to an end.

A comparison of Scaling Laws for LLM Pretraining, from Kaplan, to Chinchilla, the Chinchilla Trap, covering compute-optimal training and inference.

An overview of the motivations and techniques used for generating synthetic data for LLM post-training, as seen in the Llama 3.1, AFM, Qwen2 and Hunyuan-Large papers.

An overview of recent research on scaling test-time compute in large language models (LLMs) including CoT, STaR, ReST, RISE, ORMs, PRMs and more.