- Product managers can leverage AI tools like Claude to significantly accelerate product iteration and operate more independently, reducing reliance on specialized teams for data and testing.
- AI empowers PMs to perform complex data analysis and visualization without needing to write SQL, by connecting directly to product data tables like those in BigQuery.
- Beyond automation, AI extends a product manager's capabilities, enabling rapid generation of test cases for AI system evaluations and shifting focus towards strategic decision-making.
How Anthropic uses Claude in Product Management
- Utilize AI tools such as Claude to accelerate product ideation and iterate on concepts much faster.
- Set up a
BigQuery MCP(or similar integration) to connect product data tables directly to AI code environments for streamlined analysis. - Employ AI to generate and execute SQL queries on product data, eliminating the need for product managers to write complex queries manually.
- Generate
synthetic product datausing AI to test product ideas and explore usage patterns (e.g., light vs. dark mode, plan type usage) before live implementation. - Leverage AI to rapidly expand the options base for
e-bells, generating numerous test cases for evaluating AI systems from just a few examples. - Delegate data gathering, analysis, and operational tasks to AI to allow product managers to focus more on product strategy, customer conversations, and core decision-making.
- Recognize that AI can extend a product manager's individual capabilities, enabling tasks that might otherwise be unachievable independently.
Claude — An AI assistant or model that can perform tasks such as code generation, data analysis, and content creation.
Product Manager — A professional responsible for the strategy, roadmap, and feature definition of a product.
SQL — (Structured Query Language) A standard programming language used for managing and querying relational databases.
BigQuery MCP — A specific setup or integration within Google Cloud's BigQuery service that connects product data tables to an AI code environment (like Claude Code) for direct analysis.
Synthetic product data — Data generated artificially to simulate real product usage or behavior, often used for testing and development.
E-bells — A method or framework used for evaluating the performance, safety, or effectiveness of AI systems and products, typically involving the generation of various test cases.
Rolling average — A statistical calculation that averages data points over a specified period, continuously updating as new data becomes available, to smooth out short-term fluctuations.
Plan type — A classification or category used to segment product users or subscriptions based on the features, tiers, or pricing plans they have opted for.
What excites me about being a product manager now is that I can move and iterate much faster. I can use Claude to test my product ideas before I even get anyone else in the loop. I can operate much more independently, which is extremely empowering. Getting data as a product manager is like a pain point. Usually, you have to ping a data science person to help you. Or often, product managers can write some kind of basic SQL themselves and will have to query a database that they don't know that much about. I started using Claude Code, pretty quickly figured out that we could use it for data analysis workflows. Our data science team set up a BigQuery MCP, which essentially connected all of our BigQuery product data tables to Claude Code. need to know how to write SQL. You just need to be a human interpreter of that data and that result. So, here I am in Claude Code and I want to show how you can use Claude to access product data. Before I started this demo, I used Claude to generate some synthetic product data that shows light mode versus dark mode usage on a product. I've said I want to explore the fraction of dark mode usage over the past 3 months. I'm going to tell it the data table to look for. So, it just popped open this graph. This is like pretty impressive. It's done a whole bunch of stuff I didn't ask for, like add a 7-day rolling average, add the overall average. The graph is nicer than what I would have put together if I had done this myself. Say I want to iterate on this a little bit and take a look. Maybe we want to look at light mode and dark mode usage by plan type. Can you actually plot it by plan type? It's asking me for permission to make edits. Sure. Now we have a plot of the data by plan type. If I had to do this on my own, it would take me possibly hours, if not more. Another way that my team uses Claude is to help with generating e-bells. E-bells are a way that we evaluate AI systems and AI products. Claude is really great at this actually. So, you can kind of just give Claude the situation and the and the product experience that you're trying to build, maybe a couple example test cases, and ask it to expand the options base of test cases. So, really quickly you go from one or two examples to test to like 50 examples to test. So, my hope is that with Claude product managers are able to spend more of their time on product strategy, customer conversations, and decision-making, and less time on coordination and operations. I would say it's like more than automation for me. Like, it's things that I wouldn't have otherwise necessarily even been capable of doing independently. So, it's actually like extending what I can accomplish on my own.
TL;DR
- Product managers can leverage AI tools like Claude to significantly accelerate product iteration and operate more independently, reducing reliance on specialized teams for data and testing.
- AI empowers PMs to perform complex data analysis and visualization without needing to write SQL, by connecting directly to product data tables like those in BigQuery.
- Beyond automation, AI extends a product manager's capabilities, enabling rapid generation of test cases for AI system evaluations and shifting focus towards strategic decision-making.
Takeaways
- Utilize AI tools such as Claude to accelerate product ideation and iterate on concepts much faster.
- Set up a
BigQuery MCP(or similar integration) to connect product data tables directly to AI code environments for streamlined analysis. - Employ AI to generate and execute SQL queries on product data, eliminating the need for product managers to write complex queries manually.
- Generate
synthetic product datausing AI to test product ideas and explore usage patterns (e.g., light vs. dark mode, plan type usage) before live implementation. - Leverage AI to rapidly expand the options base for
e-bells, generating numerous test cases for evaluating AI systems from just a few examples. - Delegate data gathering, analysis, and operational tasks to AI to allow product managers to focus more on product strategy, customer conversations, and core decision-making.
- Recognize that AI can extend a product manager's individual capabilities, enabling tasks that might otherwise be unachievable independently.
Vocabulary
Claude — An AI assistant or model that can perform tasks such as code generation, data analysis, and content creation.
Product Manager — A professional responsible for the strategy, roadmap, and feature definition of a product.
SQL — (Structured Query Language) A standard programming language used for managing and querying relational databases.
BigQuery MCP — A specific setup or integration within Google Cloud's BigQuery service that connects product data tables to an AI code environment (like Claude Code) for direct analysis.
Synthetic product data — Data generated artificially to simulate real product usage or behavior, often used for testing and development.
E-bells — A method or framework used for evaluating the performance, safety, or effectiveness of AI systems and products, typically involving the generation of various test cases.
Rolling average — A statistical calculation that averages data points over a specified period, continuously updating as new data becomes available, to smooth out short-term fluctuations.
Plan type — A classification or category used to segment product users or subscriptions based on the features, tiers, or pricing plans they have opted for.
Transcript
What excites me about being a product manager now is that I can move and iterate much faster. I can use Claude to test my product ideas before I even get anyone else in the loop. I can operate much more independently, which is extremely empowering. Getting data as a product manager is like a pain point. Usually, you have to ping a data science person to help you. Or often, product managers can write some kind of basic SQL themselves and will have to query a database that they don't know that much about. I started using Claude Code, pretty quickly figured out that we could use it for data analysis workflows. Our data science team set up a BigQuery MCP, which essentially connected all of our BigQuery product data tables to Claude Code. need to know how to write SQL. You just need to be a human interpreter of that data and that result. So, here I am in Claude Code and I want to show how you can use Claude to access product data. Before I started this demo, I used Claude to generate some synthetic product data that shows light mode versus dark mode usage on a product. I've said I want to explore the fraction of dark mode usage over the past 3 months. I'm going to tell it the data table to look for. So, it just popped open this graph. This is like pretty impressive. It's done a whole bunch of stuff I didn't ask for, like add a 7-day rolling average, add the overall average. The graph is nicer than what I would have put together if I had done this myself. Say I want to iterate on this a little bit and take a look. Maybe we want to look at light mode and dark mode usage by plan type. Can you actually plot it by plan type? It's asking me for permission to make edits. Sure. Now we have a plot of the data by plan type. If I had to do this on my own, it would take me possibly hours, if not more. Another way that my team uses Claude is to help with generating e-bells. E-bells are a way that we evaluate AI systems and AI products. Claude is really great at this actually. So, you can kind of just give Claude the situation and the and the product experience that you're trying to build, maybe a couple example test cases, and ask it to expand the options base of test cases. So, really quickly you go from one or two examples to test to like 50 examples to test. So, my hope is that with Claude product managers are able to spend more of their time on product strategy, customer conversations, and decision-making, and less time on coordination and operations. I would say it's like more than automation for me. Like, it's things that I wouldn't have otherwise necessarily even been capable of doing independently. So, it's actually like extending what I can accomplish on my own.