Emerging Principles of Agent Design

I don't like buying into a hype cycle very much, but I do have a few people whose work I deeply respect; Ilya Sutskever and Andrej Karpathy are two of them. So while everyone is trying to make you believe that agents are the hottest thing since sliced bread, you should give it more credence when it's coming from Ilya and Andrej.

Ilya Sutskever for example hypothesized at NeurIPS 2024 that agents, along with synthetic data and inference time compute, are what's coming next. And Andrej Karpathy just this week said we're at the start of the decade of agents at YC Startup School 2025.

Andrej Karpathy at YC Startup School 2025

It's not just people proclaiming the hype, there's a lot of action happening. So far we've been seeing

coding tools like Cursor and Windsurf adding agentic workflows to their products
the big labs shipping research and coding agents
countless startups building agentic products
the number of agent frameworks exploding

But how exactly is everyone building agents? When I went to the AI Engineering Summit in New York earlier this year I was hoping to get answers. However I came back with more questions; it seems as if everyone was still trying to figure it out themselves!

The solution space is vast: Should you adopt a framework or build your own? How to manage state across potentially very long running, failure prone, potentially human-agent-collaborative workflows? etc. etc.

One blog post alone cannot give justice to the complexity of this. However, in recent months it seems like builders have been converging on at least some principles of agent design.

In this article I'll try to distill some of these articles into a few principles of agent design that are emerging as of June 2025.

Brief overfiew of the literature

Before diving into the details, here is a brief overview of what some people have written about agent design.

In late 2024 Anthropic published a blog article Building effective agents[1] on the distinction of workflows and agents with some fundamental design patterns. It was a popular article that laid out common design patterns and influenced a lot of the discussion around when to use workflows vs agents.

And a little after that OpenAI published a blog called A practical guide to building agents[6] on the same topic. OpenAI discussed single and multi-agent systems and different patterns thereof (manager or decentralized multi-agent patterns).

More on the topic of multi-agents, Cognition[3] recently published a blog post on why not to build multi-agents, where multi-agent refers to a design pattern where a main agent spawns multiple subagents that do work in parallel, before returning their results to the main agent who then synthesizes the final answer. Exactly one day later, Anthropic[2] published a blog post on how they did build their multi-agent research system.

If this sounds like a contradiction to you, it's actually not. There are certain types of tasks that are better suited for multi-agents than others (see below).

Going beyond agents who wait for a human's command to start working, Langchain[5] started the year by coining the term ambient agents, agents that wait for signals from the environment (like receiving emails) to start working.

Bryan Houlton[4] goes a step further and proposes agents that decide how long to "sleep" before their next action, and are also able to define their own workflows. He calls this type of agent an entity.

We'll skip the discussion of whether to use agents at all, or to use workflows instead. This has been covered at length by others.[1][6] Instead we'll go straight into how to design agent systems.

Use multi-agents for breadth-first problems

Anthropic[2] reportedly built their research agent using a multi-agent approach. On their internal evals, a multi-agent system (i.e. one Claude 4 Opus agent spawning several Claude 4 Sonnet subagents to do work in parallel) outperformed a single Claude 4 Opus agent 90.2% of the time.

Critically, they note that this multi-agent approach works best for breadth-first problems: problems that benefit from exploring multiple independent paths first, before synthesizing intermediate results into a final answer.

Research is an example of a breadth-first problem, as it often involves an explorative search phase. Each subagent is able to explore a different aspect of the question in isolation from other subagents. The goal is to conduct a thorough search so it's a feature, not a bug, that there is minimal overlap between the work each subagent does. As a result, it's not necessary to share much context between subagents. Each subagent can have it's own context, it's own tool calls, and all of this can be done in parallel.

Apart from yielding better results than a single agent, they found that spawning multiple subagents in parallel, and allow each subagent to make multiple tool calls in parallel also reduces task execution time by up to 90%.

Search subagents work in parallel and report back to the lead agent. Anthropic 2025

OpenAI[6] briefly mention when to use multi-agents, which is when there is either complex branching logic in play, or when the agent has too many tools to choose from. Remember, with current LLMs performance typically degrades when there are more than a few dozen tools to choose from.

Use single agents for depth-first problems

Cognition[3] on the other hand argues against the use of multi-agents. However, this is in the context of what they are building - a coding assistant.

Coding is an example of a depth-first task, in the sense that it is less suitable for parallelization. This is in line with Anthropic[2] who note that coding tasks involve fewer parallelizable steps. Though it's also a departure from their earlier article[1] where they argued that an orchestrator-worker workflow, which uses parallelism, is a good fit for coding tasks.

Cognition discuss varying degrees to which context can be shared between the main agent and the subagents. They conclude that no amount of context sharing worked as well for them as a single/sequential agent - not even if all context, including conversation history and tool calls, was shared.

The reason is that subagents, if they are not communicating with each other, are not able to coordinate their work. So when they share their individually crafted code back to the main agent, there is a chance that the results are still in conflict with each other.

This is why Cognition argues for using a single, sequential agent, which allows for all context to be taken into account at all times, and which has a higher chance of yielding coherent results.

In reality I think coding is a hybrid problem, where there can be both breadth-first and depth-first steps (there are some gnarly coding problems that would benefit from an initial research phase), though the article doesn't go into those details.

Manage context to avoid context overflow

According to Anthropic[2] agent interactions use 4x more tokens than normal chat-based interactions, and multi-agent systems use 15x more tokens than chats. So on tasks where either approach could be used, it's important to assert that the benefit of the multi-agent approach justifies the cost.

Cognition[3] use the term context engineering, which is basically the same as prompt engineering, except it's done automatically in a dynamic system.

The way they think about it, in a single-agent system as mentioned above, is that after each subagent has finished their work, there is a spezialised context compression LLM that summarises the key moments and decisions that were taking by that subagent.

A context compression LLM manages the context after each step Cognition 2025

Not doing so is fine for small tasks, but for larger tasks there is a risk of context overflow if you don't manage it carefully (and it would also be a waste of compute).

Anthropic[2] mention persisting subagent outputs to external storage as a way to manage context-sharing between subagents. The benefit of not reporting everything back to the main agent (except for lightweight references to those subagent artefacts) is that it can prevent information loss and reduces token overhead.

For really long running tasks, there is still risk of context overflow. This can be mitigated by something called decaying resolution memory[4] (DRM).

The idea of DRM is that it summarises the short term memory of an agent (i.e. the exact conversation history) and stores the summary in long term memory. This reduces the amount of token needed to store the context by 20x.

Sounds similar to the context compression LLM above? The key difference is that DRM uses sliding windows over ground truth memories to create summaries of different granularities. To be more specific: actual messages for anything recent, daily summaries for anything older than a day, weekly summaries for anything older than 2 weeks, and monthly summaries for anything other than 10 weeks. In other words, DRM compresses context more, the older the context gets.

Creating context summaries of different granularities Houlton 2025

In the limit this can still lead to context overflow, but that might not be a concern for most use cases. One downside is that you lose some amount of detail for older messages, but that's the case for any approach that summarises context. One way around this would be to combine this approach with a system (e.g. RAG) than can retrieve the exact messages when needed.

It's unclear how exactly Anthropic[2] do it, but they also mention a pattern by which agents summarise completed work and store it in external memory. When new subagents spawn up, they start with a fresh context window, but are able to retrieve context from external memory.

Prompt engineering is (still) important

Anthropic's article[2] is full of good advice on how to design multi-agent prompts, here are my favorite bits:

teach the orchestrator agent to delegate: a longer, more detailed prompt works better than a short one. It's important to make sure the orchestrator doesn't hand out overlapping tasks to subagents, or leaves out important work streams.
add explicit instructions on tool use to the prompt: i.e. explicitly mention that you only want 1 agent and up to 3 tool calls for simple queries, but 2-4 subagents and 10-15 tool calls each for more complex queries, or even more for really complex queries. You want to avoid the agent spawning too many subagents for queries that are really straightforward.
add explicit heuristics on tool usage to the prompt: Anthropic mention that they add heuristics like "examine all available tools first", or "prefer specialized tools over generic ones".

Write tool descriptions that make sense to the agent

Writing good prompts is one thing, but another important aspect is how you describe tools. Generally the advice is to put yourself into the shoes of the agent and ask yourself whether you would be able to decide which tool is the correct one to use given the task, context and tool descriptions.

One tips how to describe tools is to describe them from the perspective of the agent itself.[4] For example "manage events on your calendar" is a better description than "manage calendar events".

Anthropic[2] also mention the need for each tool to have a distinct purpose and a good description. They also mention that Claude can be used to generate better tool descriptions when given a prompt and failure mode. It's actually such a good prompt engineer that future agents using the improved tool descriptions completed tasks 40% faster, because they were able to avoid the most common mistakes.

Other things than can help tool usage[1] are to (1) include example usage, edge cases, input format requirements in the tool description and (2) change parameter names to make them more descriptive.

Making proactive agents

Both Cognition[3] and Anthropic[2] discuss reactive agents, which wait for a user to query them with a coding or research task.

Langchain[5] published a blog post on a concept they call ambient agents. By this they simply refer to agents that aren't solely triggered by humans, but listen to an event stream and take action based on that.

Similarly, Bryan Houlton[4] proposes a concept he calls entity. Some of the characteristics of an entity are that they

get triggered by events from the environment like ambient agents
run in a loop, but can go into sleep mode for a duration decided by the agent itself
when they wake, can decide their own workflow/tool use rather than following a predefined workflow

How can this manifest in practice? A tool I'm working with is Temporal which allows you to build single or multi-agents that

can wait for signals from the environment (like receiving emails) to start working
allow for human-in-the-loop collaboration
allow for sharing context between agents
allow for workflows that run potentially for days, weeks, or months
are suited for workflows that have high chance of failure at any point in time, like LLM-calls, with retries etc built in

Conclusion

It's still early days with agents, and while there are some examples where I use agents every day like for research or coding, there are still many challenges to be solved.

According to Andrej Karpathy, 2025 is not the year of the agent, but the start of the decade of agents. If that's true, then whoever you think might have a headstart in this space is probably not that far ahead.

It will be fun to see how many of the points in this article will hold true in a couple of years.

Happy building everyone!

References

[1] Anthropic. Building effective agents, 2024.[blog]

[2] Anthropic. How we built our multi-agent research system, 2025.[blog]

[3] Cognition. Don’t Build Multi-Agents, 2025.[blog]

[4] B. Houlton. Building proactive AI agents, 2025.[blog]

[5] LangChain. Introducing ambient agents, 2025.[blog]

[6] OpenAI. A practical guide to building agents, 2025.[blog]