- Anthropic's Claude models excel at agent tasks due to extensive training on open-ended problems, enabling them to take many steps, use tools, and explore environments before providing a final answer.
- Developers can leverage tools like the Claude Code SDK and "Skills" to efficiently build and enhance agents, with coding identified as a fundamental skill that spills over into many other domains.
- Multi-agent systems, where multiple Claude instances work in parallel or delegate tasks, improve performance, optimize context management, and can yield superior final answers compared to single agents.
Building more effective AI agents
- Claude's proficiency as an agent stems from Reinforcement Learning (RL) training on long-running, multi-step tasks across diverse domains, teaching it to learn objectives with limited guidance.
- Coding is a crucial foundational skill for an agent, as it allows Claude to perform varied tasks like web search, scheduling, and generating files (e.g., SVGs, spreadsheets) by writing and executing scripts.
- The Claude Code SDK offers a polished, general-purpose agent loop, encouraging developers to use it as a core scaffold and then add custom business logic and tools rather than rebuilding the loop from scratch.
- Claude "Skills" are an advanced extension of
.mdfiles, enabling agents to incorporate and reuse any file type—such as templates, helper scripts, or images—as resources, akin to "injecting" expertise. - Agents have largely superseded traditional "workflows" for tasks requiring high quality, as they can effectively respond to feedback and self-correct; more complex applications can now chain "workflows of agents" where each step is a closed-loop agent.
- Multi-agent systems facilitate parallel computation, allowing a main agent to delegate tasks to sub-agents (e.g., for simultaneous searches) or preserve context by offloading large subtasks, resulting in faster and more efficient outcomes.
- When designing tools or Model-Computer Protocols (MCPs) for an agent, map them to a user interface (UI) experience rather than a direct API structure, presenting comprehensive information in a single interaction to minimize agent effort.
- Best practices for agent development include starting with the simplest possible solution, adding complexity only when necessary, and always considering the agent's perspective to ensure it receives sufficient context and information.
RL — (Reinforcement Learning) A machine learning paradigm where an agent learns to make decisions by performing actions in an environment and receiving rewards or penalties.
Agent tasks — Open-ended problems that require an AI model to take multiple steps, use tools, explore, and iterate to achieve a goal.
Claude Code SDK — A software development kit that provides a pre-built, general-purpose agent loop, simplifying the creation of AI agents using Claude.
Claude Skills — An extension feature allowing Claude agents to be loaded with reusable resources in various file formats (e.g., code, templates, images) beyond just textual instructions.
Workflows — Traditional, often sequential, predefined steps for an AI model to follow, typically involving single-shot prompts, often contrasted with self-correcting agents.
Workflows of agents — A system where sequential steps are each handled by a closed-loop, self-correcting AI agent, enabling iterative improvement within each stage.
Observability — The ability to understand the internal state and operation of a complex system, such as an AI agent or multi-agent system, based on its outputs and logs.
Multi-agent — A system design where multiple AI agents work collaboratively or delegate tasks to each other, often in parallel, to solve a problem or achieve a goal.
Sub-agent — An AI agent that operates under the direction of a main or parent agent, typically handling specific sub-problems or parallelizable tasks.
MCP — (Model-Computer Protocol) A term used to describe tools or interfaces that enable an AI model to interact with external systems, often designed to mimic a user interface experience for the model.
I think there's also a lot of interesting things to explore of multi-agent as a form of test time compute. Basically, many clods work on a problem. Can be, you know, get you better final answer than just one. Hey, I'm Alex. I lead clod relations here at Anthropic. Today, we're going to be talking about building more effective agents, and I'm joined by my colleague. I'm Eric. I work on multi-agent research here at Anthropic. Eric, take us off here. Can you just explain why clod is so good at agent tasks? Yeah, sure. So during our training, we let clod practice being an agent. We give it open-ended problems for it to work on where it can take many steps and use tools, explore where it is, and what it's working on before giving a final answer. And by getting lots of practice and being an agent, clod becomes really good at this. Okay, so it's these long running tasks and a variety of domains, basically. And through the process of RL and other training mechanisms, clod is learning objective of how to do these things with basically limited guidance or feedback. Exactly. We do lots of RL on coding tasks, on search tasks, lots of things for clod to practice being an agent in different environments. There's kind of this conception, I think, of clod models that they're really, really showing code. But that doesn't always maybe transfer into other domains, or like coding is its own separate thing. What are your views on that generally? So coding has been the first task that we've really focused on. But once you have an amazing coding agent, a coding agent can do any other kind of work. If you need to do search, you can do web search via APIs. You can plan a weekend by creating a schedule. So we really see coding as a very fundamental skill for an agent that's going to have a lot of spillover effect to be able to make clod-graded all sorts of things. And sort of like train on the hardest thing first, and then everything else will become easy. One interesting thing I've seen here recently with a feature that we released in in clod.i on the web was the ability for clod to create actual files through writing code. So it was like writing a Python script. And then the Python script got ran, and all of a sudden you have like an Excel sheet that popped out of that. Is that kind of the future direction that we're headed? Is like clods writing scripts and taking actions on computers to create files or do things that are traditionally not code-related? I think that's one of the really effective ways clod will be able to do these things. Actually just a few days ago, clod was helping me make some diagrams for a presentation. And it was able to create files just by writing out the SVGs. But then I wanted it to make a much more detailed diagram that would need a lot of repetition. And so clod was actually able to do this by writing some code to generate the SVG, which ran much, much faster than clod itself needing to write, you know, it was a very, very repetitive image file with lots and lots of sort of detailed patterns in it. So yeah, I think that for a lot of cases writing code to produce some artifact will be much better than just trying to create that artifact directly. So it's one way to do it for harder cases. Okay, right. Yeah, code allows for kind of this speed up. That's not even possible with like a human like clicking and dragging and using their mouse on a computer like repeated actions. Exactly. Clod gets a for loop. Yeah, if you're a developer and you're building an agent with clod, one thing that we've started to see become really popular is this clod code SDK. Can you walk me through what that is and how you're seeing developers starting to use that? Yeah, so we're really excited about developers using the clod code SDK. This is something where previously if you wanted to build a coding agent or sort of any agent, you had to really go from nothing but hitting an API endpoint, build the loops yourself, build all the tools, build executing these tools, interacting with files, interacting them CP. We basically have already built all that into clod code. And even though its name is clod code, really, clod code is just a general purpose agent. It is most often used for code. Yeah, we are encouraging a lot of developers to use this SDK as the core of their agent loop. And that way, they don't have to spend a lot of time reinventing the wheel that we've already put a lot of time into polishing and perfecting that core agent loop. And instead, they can use that and then just add their tools for their own custom business logic or affordances into that via MCP. Right, so it offers that sort of customizability to where you can remove the coding specific bits and put in whatever sort of prompt or tools that you need, just like slots nicely into the scaffold. Yeah, I think also the people have been using clod code for all sorts of things. I think the my strangest use of clod code is I once had it plan a date for me where I did a bunch of web searches found interesting activities in restaurants in the area. And so not code related at all, but it has all the tools. How is the date? It was pretty good. Yeah, it's great. Yeah, I did a good job. Yeah, flowlly gardens and then a Chinese restaurant nearby. Wow, okay. Clod did a good job. I'm impressed. Yeah. One other thing in the clod code that has been another popular feature I've seen a lot of software engineers use lately is clod md files. So these are files that you define within a project and gives clod relevant information about what your programming style is or like what the layout of the directories are, things like that. We've now launched a similar concept that maybe takes a step further called skills. Can you explain what skills are and how we're starting to see developers use them and what they mean for agents? Yeah, so clod skills are a very exciting extension of clod md files where instead of just giving it notes files, you can give it any sort of file. That can be PowerPoint template files. It can be code and like helper scripts that you want it to use. It can be images or assets. And I think this extension of not just instructions but resources for the agent to use is a really, really powerful tool where you might say not just these otherwise instructions for making PowerPoint presentations, but here's the headshots of all of our company leadership that you might need to reuse in many presentations. I'm just giving it all to clod in a reusable way. So it has everything it needs right there. One analogy I've heard used internally that I really, really liked is it's kind of like in the matrix when Neo is learning kung fu for the first time and they like inject him with the kung fu information and all of a sudden he's like a kung fu master. That feels like very similar to when I give clod a skill of some type. Here's how you create spreadsheets and it's like all of a sudden clods like a banker now and they can create a financial model for me. That and where they load in all the racks of equipment and tools and stuff for them to grab. Yes. You can start with these things not just instructions. Yeah, I love that. Switching gears a little bit. So the last time we chatted on camera here, a few months back and we were talking about agents and at the time we were in this transition from maybe workflows which are like very defined ways of how you chain together prompts to what was just like a single agent system where you're running a model in the loop. Since then what's been the evolution in the space? Yeah. So we've really seen agents take over from workflows where clod has gotten so good at responding to feedback and correcting its own work that now agent loops really dramatically outperform workflows for most things where you care most about absolute quality. Workflows are still great where you need very low latency and you want clod to just give a best answer single shot agents are really really high performance now. I think one of the things that I've seen develop since then is what I call workflows of agents whereas previously an application might have had a workflow that had clod in single shot right SQL command in order to load data and then that would go to another step in the workflow where it would then write a chart to display that data and if the SQL command failed, it doesn't know that it's not returning any data and then the second step of the workflow is completely filled. But now I've seen people where each one of those steps in the workflow is actually a closed loop where instead of just writing a single attempt at a SQL query it then runs, clod sees the output and then it can keep iterating it repeat until it knows that it got the right value and then it transitions to the next step in the workflow. Okay, interesting. So yes, this evolution I guess of like chaining together prompts to now chaining together agents in these loops themselves. We'll see where that goes from there. One other big topic of discussion I feel like that is taking a lot more chatter as of late is this question around observability and verification. Can you explain what that challenges and how people are starting to think about it? Yeah, so observability is very hard for agents, especially as the systems get more complex. And I think that's one of the reasons where I still really believe that even though the models are much more capable today than they were a year ago and they can work better in an agent or even more complex setups, I think that simplicity is still a really important thing and that even though you can build a big workflow of agents, you should still start sort of buy from the simplest possible thing and then work up to a more complex solution. And that's first trying single-shotting things or trying single-shot prompt to Claude Code SDK which is now just sort of such a simple easy thing to use. And then I think only as needed adding layers and layers of complexity because that's going to make the observability harder. Another term here maybe in parallel to work flows of agencies is multi-agent. Is that the same thing or is that something different? Yeah, so multi-agent is my main area of research now. I'd say it's pretty different from a workflow agent. Work flows of agents are sort of one agent goes finishes and then it transitions or its output gets sent to the next agent to work on. Multi-agent is where fundamentally you have multiple agents or multiple clouds working at the same time where maybe one parent agent delegates tasks to five subagents that can each then work in parallel. This is how our deep research, search product works is. The main orchestrator agent will decide and create several subagents that can do lots of searches in parallel. And that's way better for the user because you know all this happens in parallel and you get Ian Sirback much sooner. We also see things like in Claude Code. The model will use a sub agent. So if something if some subtask is going to take tens of thousands of tokens, like maybe finding a certain implementation of a class, but the answer really boils down to something very small, it can do that work in a sub agent to protect the main context from all of those tokens that aren't necessary for the main work. So yeah, basically you can offload this piece of work and just get back the final answer. So are we exposing then this sub agent in this case is like a tool that Claude can call upon. Exactly. Pass in, it will pass in the prompt as like a parameter or something. Exactly. So to Claude subagents look like a tool where it can pass prompts to the subagents that will then go and do work. And part of my research is training Claude to be a better manager and know how to give clear instructions to its subagents and make sure that they get the right things that needs out of them. How is this different than it or is this maybe like a specialized part of tool calling overall or is it different in some ways? I would say that this uses the framework of tool calling for that communication protocol. And it just happens to be a tool that itself is backed by another Claude. Does Claude have like an intuitive understanding of what a sub agent is or do we have to like teach it like you're actually talking to another version of yourself Claude. Like don't get freaked out sort of thing. I would say that Claude makes a lot of the same mistakes that first time managers make of where it will give incomplete or sort of unclear instructions to a sub agent and you know kind of expect the subagent to have the right context when actually it doesn't. And I think something we've seen during training on subagents is that Claude starts to get much more verbose and much more detailed and give it subagents the overall context of what's going on. So that they can do better work that adds up to the whole. So I'd say that you know it definitely Claude. The Claude has a lot to learn and it's learning to get better at this. Okay, cool. What are what are some of the use cases here? So there's search is one in like preserving context. Is there other things that people are using multi agent for right now? Yeah, I think coding is there's a lot of sub agent use in coding. Anything that can be parallelized or map reduced. If you have something where you need to produce a lot of output or there's maybe 10 parts of some output you're creating. If you can split that up among 10 sub agents, that can be really really effective for saving context and getting faster results. I think there's also a lot of interesting things to explore of multi agent as a form of test time compute. Basically letting claw many clouds work on a problem can be you know get you a better final answer than just one. Just like with people, you know a bunch of people putting their heads together can get better results. In that case, are we specializing these agents in any way? Do we gear them towards like one type of persona or another or is it just kind of let them take whatever form? I think you can do either. You know sometimes it's helpful to give a bunch of people the same exact task and see what the different answers they come up with are. Sometimes it's good to have many people or many agents work from different approaches to the same problem or split it up. One thing I've seen a lot is customers that have a lot of tools, maybe a hundred or 200 tools that they want an agent to use. They found that it's really good to split up those tools among sub agents. So the main agent all it has to know is hey, I want to use this bucket of tools and then there's a sub agent that goes and does the actual work there so that each sub agent just has maybe 20 tools that it needs to understand and nobody use. Have we tried like scaling agents like all the way up? What happens if you have like a thousand versions of Claude all working on one problem? Does it just turn into chaos? I've not tried that yet. Okay, I'll get back to you. Good research idea right there. What are some other like failure modes that we're seeing right now with agents or multi agents? Yeah, I think just like any sort of complex system, I think it's easy to over build something and lose a lot of efficiency and just create sort of a lot of like dead weight. And so I've seen over built multi agent systems spend too much time just talking back and forth with each other and not actually making progress on the main task. Human agents or human organizations suffer from this too. As companies get bigger, you have more communication overhead and less and less work is actually the people on the ground making progress on things. And so I think that's another interesting thing to study is like how can we make organizations of clouds very effective while keeping the overheads more? If I'm a developer and I want to get started with agents, whether I'm building on the Claude Code SDK or just trying to roll my own, do you have any tips or best practices that you give them? Yeah, I think the best practices really remain start simple and make sure you only add complexities you need. I think another really important thing is think from the point of view of your agents. If you're giving Claude tools or prompts or sort of any affordances, put yourself in Claude shoes and read what it actually gets what it sees as the model and make sure there's actually enough information there for you to solve the problem. It's very easy to sort of forget that we're seeing everything and the model only sees what we show it. Yeah, I feel like it's always important to go back into the raw transcript of your tool calls and your logs and everything and just view that. Exactly. And I think another thing is that as people are building more things like MCPs and trying to connect Claude to more things, I think a very natural first instinct that people have that's very wrong is that an MCP or tools should be one to one with your API. And I think actually tools for the model or MCPs should be one to one with your UI, not your API. Because ultimately the model is a user of these things. It doesn't work like a traditional program. So if your API might have three separate endpoints for, say, loading a Slack conversation and turning a user ID into a username and turning a channel ID into a channel name, if those are the tools you give the model to understand Slack, for it to understand anything, it's going to have to make three tool calls. For as a user, you know, we just see everything all nicely rendered. And so you want to create a tool or an MCP for the model that it presents everything all at once with as little interaction as possible. Just like for a user, it'll be terrible. Every time you add Slack, you had to like click on a user ID to see what the name was, etc. I like that. It's kind of working back from the end-state almost instead of just trying to map the technical specs one to one. Exactly. And sort of surround whatever context you need. What do you think the future of agents has in store for us? Any predictions on these next six to 12 months? I think agents are going to become a lot more pervasive sort of starting in areas that are verifiable like software engineering, you know, coding agents have already changed how I work and how tons of people at Anthropic work. And I think there's still a huge amount to be began there. I think one of the really exciting things is if agents can start getting better at verifying their own work with things like computer use of they can write a web app but can they go actually open it up and test it and then find their own bug instead of you needing to do that. I think that's one of the most exciting things. It's a closing that loop of testing so that I don't have to be Claude's QA engineer. Right. It's kind of combining all these things from the software engineering abilities to the computer use abilities once we put all these pieces together. Yeah. And I think the computer use is also going to really open up a lot of other avenues and domains where agents have been sort of locked out of so far. What would be an example of that? I think that if you want to have Claude sort of do work for you in a Google Doc, right now it's you know, Claude can write for you but you're copy and pasting back and forth. Right. But if you have computer use and you say, Hey, Claude, can you clean up this Google Doc? It can just do it right there for you scrolling around, clicking, editing the text. And that's just such a nicer experience than needing to copy and paste back and forth. Yeah. So wherever you are, Claude can be there with you if it had with computer use. Well, I'm very excited to have Claude, right? My Google Docs and respond to all my comments for me exactly. That would be a very nice future. Eric, this has been great. Thank you so much for the conversation. Absolutely. Thank you.
TL;DR
- Anthropic's Claude models excel at agent tasks due to extensive training on open-ended problems, enabling them to take many steps, use tools, and explore environments before providing a final answer.
- Developers can leverage tools like the Claude Code SDK and "Skills" to efficiently build and enhance agents, with coding identified as a fundamental skill that spills over into many other domains.
- Multi-agent systems, where multiple Claude instances work in parallel or delegate tasks, improve performance, optimize context management, and can yield superior final answers compared to single agents.
Takeaways
- Claude's proficiency as an agent stems from Reinforcement Learning (RL) training on long-running, multi-step tasks across diverse domains, teaching it to learn objectives with limited guidance.
- Coding is a crucial foundational skill for an agent, as it allows Claude to perform varied tasks like web search, scheduling, and generating files (e.g., SVGs, spreadsheets) by writing and executing scripts.
- The Claude Code SDK offers a polished, general-purpose agent loop, encouraging developers to use it as a core scaffold and then add custom business logic and tools rather than rebuilding the loop from scratch.
- Claude "Skills" are an advanced extension of
.mdfiles, enabling agents to incorporate and reuse any file type—such as templates, helper scripts, or images—as resources, akin to "injecting" expertise. - Agents have largely superseded traditional "workflows" for tasks requiring high quality, as they can effectively respond to feedback and self-correct; more complex applications can now chain "workflows of agents" where each step is a closed-loop agent.
- Multi-agent systems facilitate parallel computation, allowing a main agent to delegate tasks to sub-agents (e.g., for simultaneous searches) or preserve context by offloading large subtasks, resulting in faster and more efficient outcomes.
- When designing tools or Model-Computer Protocols (MCPs) for an agent, map them to a user interface (UI) experience rather than a direct API structure, presenting comprehensive information in a single interaction to minimize agent effort.
- Best practices for agent development include starting with the simplest possible solution, adding complexity only when necessary, and always considering the agent's perspective to ensure it receives sufficient context and information.
Vocabulary
RL — (Reinforcement Learning) A machine learning paradigm where an agent learns to make decisions by performing actions in an environment and receiving rewards or penalties.
Agent tasks — Open-ended problems that require an AI model to take multiple steps, use tools, explore, and iterate to achieve a goal.
Claude Code SDK — A software development kit that provides a pre-built, general-purpose agent loop, simplifying the creation of AI agents using Claude.
Claude Skills — An extension feature allowing Claude agents to be loaded with reusable resources in various file formats (e.g., code, templates, images) beyond just textual instructions.
Workflows — Traditional, often sequential, predefined steps for an AI model to follow, typically involving single-shot prompts, often contrasted with self-correcting agents.
Workflows of agents — A system where sequential steps are each handled by a closed-loop, self-correcting AI agent, enabling iterative improvement within each stage.
Observability — The ability to understand the internal state and operation of a complex system, such as an AI agent or multi-agent system, based on its outputs and logs.
Multi-agent — A system design where multiple AI agents work collaboratively or delegate tasks to each other, often in parallel, to solve a problem or achieve a goal.
Sub-agent — An AI agent that operates under the direction of a main or parent agent, typically handling specific sub-problems or parallelizable tasks.
MCP — (Model-Computer Protocol) A term used to describe tools or interfaces that enable an AI model to interact with external systems, often designed to mimic a user interface experience for the model.
Transcript
I think there's also a lot of interesting things to explore of multi-agent as a form of test time compute. Basically, many clods work on a problem. Can be, you know, get you better final answer than just one. Hey, I'm Alex. I lead clod relations here at Anthropic. Today, we're going to be talking about building more effective agents, and I'm joined by my colleague. I'm Eric. I work on multi-agent research here at Anthropic. Eric, take us off here. Can you just explain why clod is so good at agent tasks? Yeah, sure. So during our training, we let clod practice being an agent. We give it open-ended problems for it to work on where it can take many steps and use tools, explore where it is, and what it's working on before giving a final answer. And by getting lots of practice and being an agent, clod becomes really good at this. Okay, so it's these long running tasks and a variety of domains, basically. And through the process of RL and other training mechanisms, clod is learning objective of how to do these things with basically limited guidance or feedback. Exactly. We do lots of RL on coding tasks, on search tasks, lots of things for clod to practice being an agent in different environments. There's kind of this conception, I think, of clod models that they're really, really showing code. But that doesn't always maybe transfer into other domains, or like coding is its own separate thing. What are your views on that generally? So coding has been the first task that we've really focused on. But once you have an amazing coding agent, a coding agent can do any other kind of work. If you need to do search, you can do web search via APIs. You can plan a weekend by creating a schedule. So we really see coding as a very fundamental skill for an agent that's going to have a lot of spillover effect to be able to make clod-graded all sorts of things. And sort of like train on the hardest thing first, and then everything else will become easy. One interesting thing I've seen here recently with a feature that we released in in clod.i on the web was the ability for clod to create actual files through writing code. So it was like writing a Python script. And then the Python script got ran, and all of a sudden you have like an Excel sheet that popped out of that. Is that kind of the future direction that we're headed? Is like clods writing scripts and taking actions on computers to create files or do things that are traditionally not code-related? I think that's one of the really effective ways clod will be able to do these things. Actually just a few days ago, clod was helping me make some diagrams for a presentation. And it was able to create files just by writing out the SVGs. But then I wanted it to make a much more detailed diagram that would need a lot of repetition. And so clod was actually able to do this by writing some code to generate the SVG, which ran much, much faster than clod itself needing to write, you know, it was a very, very repetitive image file with lots and lots of sort of detailed patterns in it. So yeah, I think that for a lot of cases writing code to produce some artifact will be much better than just trying to create that artifact directly. So it's one way to do it for harder cases. Okay, right. Yeah, code allows for kind of this speed up. That's not even possible with like a human like clicking and dragging and using their mouse on a computer like repeated actions. Exactly. Clod gets a for loop. Yeah, if you're a developer and you're building an agent with clod, one thing that we've started to see become really popular is this clod code SDK. Can you walk me through what that is and how you're seeing developers starting to use that? Yeah, so we're really excited about developers using the clod code SDK. This is something where previously if you wanted to build a coding agent or sort of any agent, you had to really go from nothing but hitting an API endpoint, build the loops yourself, build all the tools, build executing these tools, interacting with files, interacting them CP. We basically have already built all that into clod code. And even though its name is clod code, really, clod code is just a general purpose agent. It is most often used for code. Yeah, we are encouraging a lot of developers to use this SDK as the core of their agent loop. And that way, they don't have to spend a lot of time reinventing the wheel that we've already put a lot of time into polishing and perfecting that core agent loop. And instead, they can use that and then just add their tools for their own custom business logic or affordances into that via MCP. Right, so it offers that sort of customizability to where you can remove the coding specific bits and put in whatever sort of prompt or tools that you need, just like slots nicely into the scaffold. Yeah, I think also the people have been using clod code for all sorts of things. I think the my strangest use of clod code is I once had it plan a date for me where I did a bunch of web searches found interesting activities in restaurants in the area. And so not code related at all, but it has all the tools. How is the date? It was pretty good. Yeah, it's great. Yeah, I did a good job. Yeah, flowlly gardens and then a Chinese restaurant nearby. Wow, okay. Clod did a good job. I'm impressed. Yeah. One other thing in the clod code that has been another popular feature I've seen a lot of software engineers use lately is clod md files. So these are files that you define within a project and gives clod relevant information about what your programming style is or like what the layout of the directories are, things like that. We've now launched a similar concept that maybe takes a step further called skills. Can you explain what skills are and how we're starting to see developers use them and what they mean for agents? Yeah, so clod skills are a very exciting extension of clod md files where instead of just giving it notes files, you can give it any sort of file. That can be PowerPoint template files. It can be code and like helper scripts that you want it to use. It can be images or assets. And I think this extension of not just instructions but resources for the agent to use is a really, really powerful tool where you might say not just these otherwise instructions for making PowerPoint presentations, but here's the headshots of all of our company leadership that you might need to reuse in many presentations. I'm just giving it all to clod in a reusable way. So it has everything it needs right there. One analogy I've heard used internally that I really, really liked is it's kind of like in the matrix when Neo is learning kung fu for the first time and they like inject him with the kung fu information and all of a sudden he's like a kung fu master. That feels like very similar to when I give clod a skill of some type. Here's how you create spreadsheets and it's like all of a sudden clods like a banker now and they can create a financial model for me. That and where they load in all the racks of equipment and tools and stuff for them to grab. Yes. You can start with these things not just instructions. Yeah, I love that. Switching gears a little bit. So the last time we chatted on camera here, a few months back and we were talking about agents and at the time we were in this transition from maybe workflows which are like very defined ways of how you chain together prompts to what was just like a single agent system where you're running a model in the loop. Since then what's been the evolution in the space? Yeah. So we've really seen agents take over from workflows where clod has gotten so good at responding to feedback and correcting its own work that now agent loops really dramatically outperform workflows for most things where you care most about absolute quality. Workflows are still great where you need very low latency and you want clod to just give a best answer single shot agents are really really high performance now. I think one of the things that I've seen develop since then is what I call workflows of agents whereas previously an application might have had a workflow that had clod in single shot right SQL command in order to load data and then that would go to another step in the workflow where it would then write a chart to display that data and if the SQL command failed, it doesn't know that it's not returning any data and then the second step of the workflow is completely filled. But now I've seen people where each one of those steps in the workflow is actually a closed loop where instead of just writing a single attempt at a SQL query it then runs, clod sees the output and then it can keep iterating it repeat until it knows that it got the right value and then it transitions to the next step in the workflow. Okay, interesting. So yes, this evolution I guess of like chaining together prompts to now chaining together agents in these loops themselves. We'll see where that goes from there. One other big topic of discussion I feel like that is taking a lot more chatter as of late is this question around observability and verification. Can you explain what that challenges and how people are starting to think about it? Yeah, so observability is very hard for agents, especially as the systems get more complex. And I think that's one of the reasons where I still really believe that even though the models are much more capable today than they were a year ago and they can work better in an agent or even more complex setups, I think that simplicity is still a really important thing and that even though you can build a big workflow of agents, you should still start sort of buy from the simplest possible thing and then work up to a more complex solution. And that's first trying single-shotting things or trying single-shot prompt to Claude Code SDK which is now just sort of such a simple easy thing to use. And then I think only as needed adding layers and layers of complexity because that's going to make the observability harder. Another term here maybe in parallel to work flows of agencies is multi-agent. Is that the same thing or is that something different? Yeah, so multi-agent is my main area of research now. I'd say it's pretty different from a workflow agent. Work flows of agents are sort of one agent goes finishes and then it transitions or its output gets sent to the next agent to work on. Multi-agent is where fundamentally you have multiple agents or multiple clouds working at the same time where maybe one parent agent delegates tasks to five subagents that can each then work in parallel. This is how our deep research, search product works is. The main orchestrator agent will decide and create several subagents that can do lots of searches in parallel. And that's way better for the user because you know all this happens in parallel and you get Ian Sirback much sooner. We also see things like in Claude Code. The model will use a sub agent. So if something if some subtask is going to take tens of thousands of tokens, like maybe finding a certain implementation of a class, but the answer really boils down to something very small, it can do that work in a sub agent to protect the main context from all of those tokens that aren't necessary for the main work. So yeah, basically you can offload this piece of work and just get back the final answer. So are we exposing then this sub agent in this case is like a tool that Claude can call upon. Exactly. Pass in, it will pass in the prompt as like a parameter or something. Exactly. So to Claude subagents look like a tool where it can pass prompts to the subagents that will then go and do work. And part of my research is training Claude to be a better manager and know how to give clear instructions to its subagents and make sure that they get the right things that needs out of them. How is this different than it or is this maybe like a specialized part of tool calling overall or is it different in some ways? I would say that this uses the framework of tool calling for that communication protocol. And it just happens to be a tool that itself is backed by another Claude. Does Claude have like an intuitive understanding of what a sub agent is or do we have to like teach it like you're actually talking to another version of yourself Claude. Like don't get freaked out sort of thing. I would say that Claude makes a lot of the same mistakes that first time managers make of where it will give incomplete or sort of unclear instructions to a sub agent and you know kind of expect the subagent to have the right context when actually it doesn't. And I think something we've seen during training on subagents is that Claude starts to get much more verbose and much more detailed and give it subagents the overall context of what's going on. So that they can do better work that adds up to the whole. So I'd say that you know it definitely Claude. The Claude has a lot to learn and it's learning to get better at this. Okay, cool. What are what are some of the use cases here? So there's search is one in like preserving context. Is there other things that people are using multi agent for right now? Yeah, I think coding is there's a lot of sub agent use in coding. Anything that can be parallelized or map reduced. If you have something where you need to produce a lot of output or there's maybe 10 parts of some output you're creating. If you can split that up among 10 sub agents, that can be really really effective for saving context and getting faster results. I think there's also a lot of interesting things to explore of multi agent as a form of test time compute. Basically letting claw many clouds work on a problem can be you know get you a better final answer than just one. Just like with people, you know a bunch of people putting their heads together can get better results. In that case, are we specializing these agents in any way? Do we gear them towards like one type of persona or another or is it just kind of let them take whatever form? I think you can do either. You know sometimes it's helpful to give a bunch of people the same exact task and see what the different answers they come up with are. Sometimes it's good to have many people or many agents work from different approaches to the same problem or split it up. One thing I've seen a lot is customers that have a lot of tools, maybe a hundred or 200 tools that they want an agent to use. They found that it's really good to split up those tools among sub agents. So the main agent all it has to know is hey, I want to use this bucket of tools and then there's a sub agent that goes and does the actual work there so that each sub agent just has maybe 20 tools that it needs to understand and nobody use. Have we tried like scaling agents like all the way up? What happens if you have like a thousand versions of Claude all working on one problem? Does it just turn into chaos? I've not tried that yet. Okay, I'll get back to you. Good research idea right there. What are some other like failure modes that we're seeing right now with agents or multi agents? Yeah, I think just like any sort of complex system, I think it's easy to over build something and lose a lot of efficiency and just create sort of a lot of like dead weight. And so I've seen over built multi agent systems spend too much time just talking back and forth with each other and not actually making progress on the main task. Human agents or human organizations suffer from this too. As companies get bigger, you have more communication overhead and less and less work is actually the people on the ground making progress on things. And so I think that's another interesting thing to study is like how can we make organizations of clouds very effective while keeping the overheads more? If I'm a developer and I want to get started with agents, whether I'm building on the Claude Code SDK or just trying to roll my own, do you have any tips or best practices that you give them? Yeah, I think the best practices really remain start simple and make sure you only add complexities you need. I think another really important thing is think from the point of view of your agents. If you're giving Claude tools or prompts or sort of any affordances, put yourself in Claude shoes and read what it actually gets what it sees as the model and make sure there's actually enough information there for you to solve the problem. It's very easy to sort of forget that we're seeing everything and the model only sees what we show it. Yeah, I feel like it's always important to go back into the raw transcript of your tool calls and your logs and everything and just view that. Exactly. And I think another thing is that as people are building more things like MCPs and trying to connect Claude to more things, I think a very natural first instinct that people have that's very wrong is that an MCP or tools should be one to one with your API. And I think actually tools for the model or MCPs should be one to one with your UI, not your API. Because ultimately the model is a user of these things. It doesn't work like a traditional program. So if your API might have three separate endpoints for, say, loading a Slack conversation and turning a user ID into a username and turning a channel ID into a channel name, if those are the tools you give the model to understand Slack, for it to understand anything, it's going to have to make three tool calls. For as a user, you know, we just see everything all nicely rendered. And so you want to create a tool or an MCP for the model that it presents everything all at once with as little interaction as possible. Just like for a user, it'll be terrible. Every time you add Slack, you had to like click on a user ID to see what the name was, etc. I like that. It's kind of working back from the end-state almost instead of just trying to map the technical specs one to one. Exactly. And sort of surround whatever context you need. What do you think the future of agents has in store for us? Any predictions on these next six to 12 months? I think agents are going to become a lot more pervasive sort of starting in areas that are verifiable like software engineering, you know, coding agents have already changed how I work and how tons of people at Anthropic work. And I think there's still a huge amount to be began there. I think one of the really exciting things is if agents can start getting better at verifying their own work with things like computer use of they can write a web app but can they go actually open it up and test it and then find their own bug instead of you needing to do that. I think that's one of the most exciting things. It's a closing that loop of testing so that I don't have to be Claude's QA engineer. Right. It's kind of combining all these things from the software engineering abilities to the computer use abilities once we put all these pieces together. Yeah. And I think the computer use is also going to really open up a lot of other avenues and domains where agents have been sort of locked out of so far. What would be an example of that? I think that if you want to have Claude sort of do work for you in a Google Doc, right now it's you know, Claude can write for you but you're copy and pasting back and forth. Right. But if you have computer use and you say, Hey, Claude, can you clean up this Google Doc? It can just do it right there for you scrolling around, clicking, editing the text. And that's just such a nicer experience than needing to copy and paste back and forth. Yeah. So wherever you are, Claude can be there with you if it had with computer use. Well, I'm very excited to have Claude, right? My Google Docs and respond to all my comments for me exactly. That would be a very nice future. Eric, this has been great. Thank you so much for the conversation. Absolutely. Thank you.