Capabilities & limitations

Understanding Generative AI: Capabilities and Limitations

Let's now examine what generative AI can and cannot do, focusing on Large Language Models (LLMs) such as Claude. Think of this as getting to know a new colleague; understanding their strengths and limitations helps you collaborate more effectively.

The Strengths of Modern LLMs

To start, we'll focus on what these systems do remarkably well. You might be amazed at how versatile modern language models can be. They are skilled with language in ways that seemed impossible just a few years ago: crafting emails that capture your voice, condensing lengthy reports into clear summaries, translating between languages, and explaining complex topics across countless fields—from microbiology to marketing strategy.

What's particularly notable is how these models can shift between different tasks without needing additional training. The very same system that helps you write poetry or brainstorm ideas for your birthday party can turn around and help you understand quantum computing concepts or analyze quarterly business trends, all through simple conversation.

These models can also maintain the thread of a conversation, remembering what you discussed earlier and building upon it. If you mention your project deadline in passing, for example, and refer back to it later within the conversation, they typically understand what you're talking about, much like a human conversation partner would.

Many modern LLMs can now also reach beyond their own knowledge by connecting to external tools and information sources, allowing them to search the web, process files, or even use other applications to enhance their capabilities. This dramatically expands what they can help with.

Knowledge Cutoffs and Hallucinations

However, just like any technology, LLMs as they exist today also have certain limitations. First, AI models are bounded by their training data. LLMs have a "knowledge cutoff date" based on when they were trained—the point after which they have no innate knowledge of the world. For example, a model with a cutoff date of November 2024 means that it wasn't trained on any data after that time. Imagine someone who went into a retreat without internet access at a specific date; they wouldn't know about events that happened after they left. Models need tools like web search to learn more about recent developments.

Additionally, the training process doesn't verify every fact in the training data. This means models can sometimes learn and reproduce inaccuracies that were present in their training data. They can also make mistakes when trying to piece together information they've learned. This leads to what is often called a "hallucination": the AI confidently stating something that sounds plausible but is actually incorrect. Unlike search engines that simply retrieve existing documents, LLMs generate responses based on statistical patterns. Imagine a friend who tells a story with absolute confidence, only to have the details completely wrong; AI can sometimes be like that.

Technical Constraints: Context Windows and Non-Determinism

Another important constraint is the "context window." As a reminder, that's the amount of information an AI can process at one time. Every LLM has a maximum limit to how much information it can consider during a single interaction. If this limit is exceeded, the AI won't be able to remember information that falls outside the window, usually on a "first-in, first-out" basis. Depending on the size of the model, this can limit its ability to process large documents or remember the entire conversation.

Furthermore, unlike traditional software that produces identical outputs given the same inputs, LLMs are somewhat unpredictable by default, also known as being "non-deterministic." Ask the same question twice, and you might get slightly different responses each time. This variability stems from the nature of how these models generate text; they are making probabilistic decisions about what text should come next based on patterns in their training data and certain settings that developers can tweak.

This creative variability can be great for brainstorming and generating diverse ideas, but it requires awareness when consistency or accuracy are critical. Some LLM interfaces offer settings to control this randomness when needed; this setting is often referred to as "temperature."

Reasoning and Data Access

Additionally, while these models are improving rapidly, they've historically shown limitations with complex reasoning tasks, particularly with mathematical or logical problems requiring multiple steps. The good news is that newer "reasoning" or "extended thinking" models, specifically designed to think step-by-step, are showing strong progress in these areas.

Finally, while models like Claude can now access external tools, they may still lack access to specific data sources or specialized tools needed for certain tasks. It's like having a brilliant colleague who can't access your company's internal database; their ability to help will be limited no matter how smart they are. If a model doesn't have access to a piece of data or a tool that is needed to answer a question, it cannot provide a complete answer.

The Path Toward AI Fluency

The field of generative AI is rapidly evolving. Researchers are working to address current limitations through techniques like Retrieval-Augmented Generation (RAG), which connects models to external knowledge and data sources, as well as expanding their ability to use tools and improving their reasoning capabilities. That said, some limitations will likely remain for the foreseeable future.

Understanding what AI can or cannot do is essential for AI fluency. It helps you determine when and how to best incorporate these systems effectively into your work and daily life. The most effective applications will leverage the complementary strengths of humans and AI. We bring critical thinking, judgment, creativity, and ethical oversight that AI may struggle to replicate, while AI offers speed, scale, pattern recognition, and the ability to process vast amounts of information.

These complementary strengths will evolve as the technology evolves. That's why continued learning and experimentation are so valuable. They help you stay abreast of these changes and discover new possibilities. Through exercises in this course, you will have a chance to explore these concepts first-hand through conversations with Claude. This direct experience will help you develop an intuitive feel for what generative AI can do, what it can't do, and how best to work with it.