Tips for building AI agents

• Agents are LLM systems that autonomously decide their actions and iteration count until a resolution, fundamentally differing from workflows which are fixed, predefined sequences of LLM calls.
• Effective agent development requires "model empathy," focusing on clear prompts and detailed tool descriptions that consider the model's perspective and context.
• While agents show strong potential for tasks with clear feedback loops like coding and iterative search, their widespread consumer adoption for complex, high-risk tasks is currently overhyped due to high verification costs and the need for extensive personal context.

• An agent is an LLM system that autonomously decides its actions and iteration count until a resolution, unlike a workflow which follows a predefined, fixed sequence of LLM calls.
• Agent prompts are open-ended, providing tools and instructions to iterate until completion, whereas workflow prompts are specific and chained in a linear fashion.
• Treat tool descriptions and parameters as meticulously as you would code documentation; clear, detailed descriptions are essential for the model to effectively use tools.
• Develop agents with "model empathy," designing prompts and environments from the model's perspective, understanding its context limitations.
• Coding agents are a strong use case due to the inherent verifiability provided by unit tests, enabling the model to iterate and converge on correct solutions through feedback.
• For any agent, embed robust feedback loops and measurement mechanisms to track performance and enable continuous improvement, especially in scenarios lacking perfect tests.
• Prioritize starting with simple solutions and progressively adding complexity, ensuring each step is measurable to avoid over-engineering.
• Focus on building systems where your product's value increases as underlying LLM capabilities improve, rather than becoming redundant.

• LLM — Large Language Model; the core AI component capable of understanding and generating human-like text.
• Agent — An LLM system that autonomously decides its actions and iteration count, looping until it achieves a resolution without predefined steps.
• Workflow — A sequence of fixed, predefined steps involving multiple LLM calls, typically pre-orchestrated by code for a specific outcome.
• Prompt engineering — The practice of carefully designing inputs (prompts) to guide an LLM to perform specific tasks or generate desired outputs.
• Tool use — The capability of an LLM agent to interact with external functions, APIs, or data sources to perform tasks beyond its core language generation.
• Function calling — A specific type of tool use where an LLM is instructed to generate a call to an external function with appropriate parameters.
• Context window — The limited amount of text or tokens an LLM can consider at any given time when processing a prompt and generating a response.
• Verifiability — The property of an agent's output or actions that allows for objective checking or testing of its correctness.
• Multi-agent environment — A system where multiple independent agents interact with each other and their shared environment to achieve collective or individual goals.

I feel like agents for consumers are like fairly pro-hyped. Okay, here we go, hot day. Trying to have an agent like fully book a vacation for you. Almost just as hard as just going and booking it yourself. One, two, one. Today we're going behind the scenes on one of our recent blog posts, Building Effective Agents. I'm Alex, I lead Claude Relations here at Anthropic. I'm Eric, I'm in the research team at Anthropic. I'm Barry, I'm on the Applied AI team. I'm gonna kick us off here for viewers just jumping in. What's the quick version of what an agent actually is? I mean, there's a million definitions of it. And why should a developer, somebody that's actually building with AI care about these things? Eric, maybe we can start with you. Sure. Yeah, so I think something we explored in the blog post is that, first of all, a lot of people have been saying everything is an agent. Referring to almost anything more than just a single LLM call. One of the things we tried to do in the blog post is really kind of separate this out of like, hey, there's workflows, which is where you have a few LLM calls chain together. And really, what we think an agent is is where you're letting the LLM decide sort of how many times to run. You're having it continuing to loop until it's found a resolution. And that could be talking to a customer for customer support. That could be iterating on code changes. But something where you don't know how many steps it's gonna take to complete, that's really sort of what we consider an agent. Interesting. So in the definition of an agent, we are letting the LLM kind of pick its own fate and decide what it wants to do, what actions to take. Instead of us pre-defining a path for it. Exactly. It's more autonomous. Whereas a workflow, you can kind of think of it as like a, you know, a workflow or sort of like it's on rails through a fixed number of steps. I see. So this distinction, I assume this was the result of many, many conversations with customers and working with different teams and even trying things ourselves. Barry, can you speak more to maybe what that looks like as we got to create this divide between a workflow and an agent and what sort of patterns surprised you the most as you were going through this? Sure. Also, I think all of those kind of evolved as like model got better and like teams got more sophisticated. We both work with a large number of like customers were very sophisticated and we kind of went from like having a single LLM to having a lot of LLM's and like eventually having like arms orchestrating themselves. So, you know, like one of the reasons why we decided to create this distinction is because we started to see these two distinct patterns where you have workflows that's pre-orchestrated by code. And then you also have, you know, agent which is like a simpler but, you know, complex in other sense, like different shape that we're starting to see. Really, I think like as the models and all of the tools start to get better, you know, agents are becoming more and more prevalent and more and more capable. And that's when we decided like, hey, this is probably the time for us to like get give a formal definition. So in practice, if you're developing implementing one of these things, what would that actually look like in your your code as you're starting to build this like the differences between like maybe we actually go down to like the prompt level here. What does an agent prompt to look like or flow and what does like a workflow look like? Yeah. So I think a workflow prompt looks like you have one prompt, you take the output of it, you feed it into prompt B, take the output of that, feed it into prompt C, and then you're done. Kind of there's this straight line fix number of steps, you know exactly what's going to happen. And maybe you have some extra code that sort of checks the intermediate results of these and makes sure they're okay. But you kind of know exactly what's going to happen in one of these paths. And each of those prompts is a sort of a very specific prompt just sort of taking one input and transforming it into another output. For instance, maybe one of these prompts is taking in the user question and categorizing it into one of five categories so that then the next prompt can be more specific for that. Okay. In contrast, an agent prompt will be sort of much more open-ended and usually give the model tools or multiple things to check and say, hey, here's the question, and you can do web searches or you can edit these code files or run code and keep doing this until you have the answer. I see. So there's a few different use cases there. That makes sense as we start to arrive at these different conclusions. I'm curious, as we've now kind of covered at a high level how we're thinking about these workflows and agents and talking about the blog post, I want to dive even further behind the scenes. Were there any funny stories, Barry, of wild things that you saw from customers that were interesting or are just kind of far out there in terms of how people are starting to actually use these things in production? Yeah, this is actually from my own experience like, buting agents. I joined about a month before the Son of V2 refresh and one of my onboarding tasks was to run OS World which was a computer-yose benchmark. For a whole week, me and this other engineer were just staring at this sort of agent trajectory, they were like counterintuitive to us. We weren't sure why the model was making decision. You know, it was given the instructions that we gave it. So we decided we're going to act like Claude and put ourselves in that environment. So we would do this really silly thing. We close our eyes for a whole minute and then we blink at a screen for a second. We close our eyes again and just think, well, I have to write Python code to operate in this environment. What would I do? I've suddenly made a lot more sense. I feel like a lot of agent design comes down to this. There's a lot of context and a lot of knowledge that the model, maybe there's not half, and we have to be empathetic to the model. We have to make a lot of that clear in the prompt, in the two description, in the environment. I see. A tip here for developers is almost like to act as if you are looking through the lens of the model itself in terms of what would be the most applicable instructions here. I was the model like seeing the world, which is very different than how we operate as a human, I guess, for the additional context. Eric, I'm curious if you have any other stories that you've seen. Yeah, I think actually in a very similar vein, I think a lot of people really forget to do this. I think maybe the funniest things I see is that people will put a lot of effort into creating these really beautiful detailed prompts and then the tools that they make to give the model are these incredibly bare bones. No documentation, the parameters were named A and B, and it's kind of like an engineer wouldn't be able to work with this as a work with this as if this was a function they had to use because there's no documentation, like how can you expect how can you expect quality to use this as well. It's like that lack of putting yourself in the model shoes. I think a lot of people when they start trying to use tool use and function calling, they kind of forget that they have to prompt as well and they think about the model just as this as a more classical programming system, but it is still a model and you need to be prompt engineering in the descriptions of your tools themselves. Yeah, that's like, yeah, I've noticed that. It's like people forget that it's all part of the same prompt. It's all getting fed into the same prompt in the context window and writing a good tool description influences other parts of the prompt as well. So that is like one aspect to consider. Agents is this kind of all the hype term right now. A lot of people are talking about it and there's been plenty of articles written and videos made on the subject. What made you guys think that now is the right time to write something ourselves and talk a little bit more about the details of agents? Sure, yeah. I think one of the most important things for us is to be able to explain things well. I think that's like a big part of our motivation, which is like we walk into customer meetings and everything is referred to as a different term even though they share the same shape. So we thought like you'll be really useful if you can just like have a set of definition and a set of like diagrams and code to explain these things to our customers. And we are getting to the point where the model is capable of doing a lot of the like the agentic workflows that we're seeing. And that seems like the right time for us to have some definitions or just to make these conversations easier. I think for me I saw that there was a lot of excitement around agents but also a lot of people really didn't know what it meant in practice. And so they were trying to bring agents to sort of any problem they had even when much simpler systems would work. And so I saw that as one of the reasons that we should write this is like guide people about how to do agents but also like where agents are appropriate. And that you shouldn't go after a fly with a bazooka. I see. I see. So that was a perfect part of my next question here. There's a lot of talk about the potential of agents and every developer out there and every startup in business is trying to think about how they can build their own version of an agent for their company or product. But you guys are starting to see what actually works in production. So we're going to play a little game here. I want to know one thing that's over hyped about agents right now and also one thing that's under hyped just in terms of implementations or actual uses in production or like potentials here as well. So Eric let's start with you first. I feel like under hyped is like things that save people time even if it's a very small amount of time. I think a lot of times if you just look at that on the surface it's like, oh this is something that takes me a minute. And even if you can fully automate it, it's only a minute. What help is that? But really that changes the dynamics of now you can do that thing a hundred times more than you previously would. So I think I'm most excited about things that if they were easier could be really scaled up. Yeah, I don't know if this is like necessarily related to hype. But I think it's really difficult to calibrate right now like where agents are really needed. I think there's this intersection that's a sweet spot for using agent and it's like a set of tasks that's valuable and complex. But also like maybe the cost of error or cost of monitoring error is relatively low. That set of tasks is like not like not like super clear and obvious unless like you know we actually like look into the like the existing processes. I think like coding and search are like two pretty canonical examples where like agents are very useful. Like take search as example right like you know it's it's it's a really valuable task. It's very hard to do like deep iterative search. But you can always trade off some like precision for recall and then just get a little bit more documents or a little bit more information that nice need to infiltrate down. So like we've seen a lot of success there was agent search. What does a coding agent look like right now? Coding agents I think are super exciting because they're verifiable at least partially. You know code has this great property that you can write tests for it and then you edit the code and either the tests pass or they don't pass. Now that assumes that you have good unit tests which I think you know every engineer in the world can say like we don't yeah. But at least it's better than than a lot of things. You know there's no equivalent way to do that for many other fields. So this at least gives this gives a coding agent some way that it can get more signal every time it goes through loop. So you know every time it's running the tests again it's seeing what the error the output is. That makes me think that you know the model can kind of converge on the right answer by getting this feedback. And if you don't have some mechanism to get feedback as you're iterating you're not injecting any more signal you're just going to have noise. And so there's no reason without something like this that an agent will converge to the right answer. I see. So what's the biggest blockers then in terms of improving agenting performance on like coding at the moment? Yeah. So I think for coding you know we've seen over the last year like on sweet bench like results have gone really from like very very low to like I think you know over 50% now which is really incredible. So the models are getting really good at writing code to solve these issues. I feel like I have a slightly controversial take here that I think the next limiting factor is going to come back to that verification. It's great for these cases where we do have perfect unit tests and that's starting to work. But for the real world cases we usually don't have perfect unit tests for them. And so that's I'm thinking now like finding ways that we can verify and we can add tests for the things that you really care about so that the model itself can test this and no whether it's right or wrong before it goes back to the human. I see. So making sure that we can embed some sort of feedback loop into the processes. Exactly. The right or wrong. Okay. What's the future of agents look like in 2025? Very rinse out with you. Yeah. I think that's a really difficult question. This is like probably not like a practical thing. But one thing I've been really interested in just like how like a multi-agent environment will look like. I think I've already shown Eric this like a building environment where like a bunch of Claude can like spin up other clouds and play like werewolf together. And it's like a completely what is werewolf. Werewolf is a social deduction game where all of the players are trying to figure out what each other's role is. It's very similar to mafia. It's entirely text space which is great for for Claude to play. I see. So we have clouds, multiple different clouds playing different roles within this game all communicating with each other. Yeah, exactly. And then you see a lot of like interesting interaction in there that you just haven't like seen before. And that's something I'm really excited about as like you know very similar to how we went from like single LM to multi LM. I think by the end of the year we can potentially see us going from like agent to multi agents. And there are some like I think interesting research questions to like figure out in that domain. In terms of how the agents interact with each other what is like this kind of emergent behavior look like in that one as you coordinate between agents doing different things. Exactly. And just like whether this is actually like going to be useful or better than like a single agent was access to a lot more resources. Do we see any multi agent approaches right now that are actually working in production? I feel like in production we haven't even seen a lot of like successful single agent. Okay, interesting. But like you know this is kind of like a potential extension of like successful agents with the like as like improved capabilities of the next couple of generations of models. Yeah, so this is not advice that everyone should go explore about the agent environment. It's just I think like you know to understand the models behavior like this provides us was a better way to understand model behaviors. I see. Okay, Eric. What's the future of agents? Yeah, I feel like in 2025 we're going to see a lot of business adoption of agents starting to automate a lot of repetitive tasks and really like scale up a lot of things that people wanted to do more before but were too expensive you can now have 10x or 100x how much you do of these things. I'm imagining things like you know every single pull request in triggers a coding agent to comment update all of your documentation. Things like that will be cost prohibitive to do before but once you think of agents is sort of almost free you can start doing these you know adding these bells and whistles everywhere. I think maybe something that's not going to happen yet going back to like what's over hyped. Yeah, I feel like agents for consumers are like fairly prepared. Okay, here we go hot time. Because I think that like like we talked about like a verifiability. I think that for a lot of consumer tasks it's almost as much work to sort of fully specify your preferences and what the task is as to just do it yourself and it's very expensive to verify. So like trying to have a agent like fully book a vacation for you describing exactly what you want your vacation to be and your preferences is like almost just as hard as going and booking it yourself. Interesting. And you it's like very high risk you don't want the agent to go like actually go book a plane flight. Interesting. Without you first accepting it. Is there a matter maybe context that we're missing here too from like the models being able to infer this information about somebody without having to explicitly go ask and learn the preference over time. Yeah, so I think that these things will get there but first you need to build up this context so that the model already knows your preferences and I think that takes time. I say and we'll need some stepping stones to get to bigger tasks like planning a whole vacation. I say okay very interesting last question any advice that you give to a developer that's exploring this right now in terms of starting to build this or just thinking about it from a general future proofing perspective that you can give. I feel like my best advice is like make sure that you have a way to measure your results because I've seen a lot of people will go and should have build in a vacuum without any way to get feedback about whether their building is working or not and you can end up building a lot sort of without realizing that it's either it's not working or maybe something much simpler would have actually done just as good a job. Yeah, I think very similarly like starting as simple as possible and having that measurable result as you are like building more complexity into it. One thing I've been really impressed by is like I work with some really resourceful startups and they just like they can do everything within one LLM call and the orchestration around like the code which will persist even as the model gets better is like kind of their niche and I always like get very happy when I see one of those because I think they can reap the benefit of like future capability improvements. And yeah, I think like realistically you know we don't know what use case would be great for agents and like the landscape is going to shift but it's probably a good time to like start building up some of that like muscle to like think in the agent and just to understand the capability of it better. Yeah, I think I want to double click on something you said of like being excited for the models to get better. I think that if you look at your startup or your product and think oh man if the models get smarter all of our modes going to disappear that means you're building the wrong thing. Instead you should be building something so that as the models get smarter your product gets better and better. Right. That's great advice. Eric Barry, thank you guys. This is Building Effective Agents. Thank you. Thanks.