Skip to main content

A closer look at Discernment

📖 Lesson content

What you'll learn

By the end of this lesson, you'll be able to:

  • Understand how to evaluate AI outputs and processes thoughtfully
  • Develop critical thinking skills for your AI interactions
  • Learn to identify and address quality concerns in your AI interactions

A Closer Look at Discernment

(5 minutes)

This video explores Discernment, the AI Fluency competency focused on thoughtfully evaluating AI outputs, processes, and behaviors. We explain that Discernment is the flip side of Description. While Description helps you communicate your intentions clearly, Discernment helps you evaluate whether what you receive meets your needs. The video introduces three types of Discernment:

  • Product Discernment: Evaluating the quality of AI outputs
  • Process Discernment: Assessing how the AI approached the task
  • Performance Discernment: Evaluating how the AI behaved during the interaction itself

Together, these skills help ensure that your AI collaboration remains guided by thoughtful human judgment.

Key takeaways

  • Discernment is your ability to thoughtfully evaluate what AI produces, how it produces it, and how it behaves
  • Product Discernment focuses on evaluating the quality of actual outputs (accuracy, appropriateness, coherence, relevance)
  • Process Discernment involves assessing how the AI arrived at its output, looking for logical errors, attention gaps, or inappropriate reasoning
  • Performance Discernment evaluates how the AI behaves within the collaboration process itself, considering whether its communication style is effective for your needs
  • Discernment works hand-in-hand with Description in a continuous feedback loop
  • Even the most advanced AI systems benefit from human judgment and oversight

Discernment — evaluating AI output

3 components

Product discernment — output quality

Process discernment — how AI arrives

Performance discernment — AI's behavior

Thoughtful human judgment

Feedback into Description — improves next round

Exercises

Expert Discernment: Evaluating AI Responses in Your Domain

Activity Goal

Practice Product, Process, and Performance Discernment by evaluating AI-generated content in a domain where you have expertise, recognizing how your knowledge enhances your ability to critically assess AI outputs.

Instructions

Step 1: Return to Your Area of Expertise

Recall the topic you discussed with Claude in the earlier exercise (Lesson 2, Exercise 2: "Explore something you love"). This was a topic where you had strong knowledge and passion.

Step 2: Ask for Multiple Explanations

Start a new conversation with Claude and ask it to generate three different explanations or analyses about a specific aspect of your expert topic. For example:

  • If your topic was photography, you might ask for three different explanations of depth of field
  • If your topic was cooking, you might ask for three different analyses of fermentation techniques
  • If your topic was history, you might ask for three different perspectives on a specific historical event

Step 3: Apply Your Expert Discernment

Drawing on your expertise, carefully evaluate each explanation provided by Claude:

Product Discernment:

  • Which explanation contains the most accurate information?
  • Are there any factual errors or misconceptions?
  • Is the level of detail appropriate for someone learning about this topic?

Process Discernment:

  • Does Claude follow logical reasoning in each explanation?
  • Are there gaps in its analysis or thinking process?
  • Does Claude make appropriate connections between concepts?

Performance Discernment:

  • Was Claude attentive to your specific question and responsive to feedback and direction?
  • Is terminology used appropriately for the topic?
  • How does the tone and style affect the clarity of the explanation?

Step 4: Provide Feedback and Refinement

Based on your evaluation:

  1. Identify the strongest explanation and specifically tell Claude specifically why it's effective
  2. Identify the weakest explanation and provide specific feedback on what makes it problematic
  3. Work with Claude to create an improved version that addresses the issues you identified

Step 5: Reflection

Discuss with Claude (in the same chat):

  • What specific knowledge did you have that allowed you to identify strengths or weaknesses?
  • How might someone without your expertise struggle to discern the quality of these explanations?
  • What does this experience teach you about the relationship between domain knowledge and effective Discernment?

For a more playful Discernment workout, you might want to try some of the “Game Night” suggestions in the final lesson ("Additional Activities").

Reflection

Before moving on, take a moment to consider:

  • Which type of Discernment (Product, Process, or Performance) do you find most challenging to apply, and why?
  • How does Discernment complement Description? How do they work together?
  • What signals or patterns might indicate that an AI output requires closer scrutiny?

What’s next

In the next lesson, you'll have the opportunity to apply both Description and Discernment skills to your overarching course project. You'll put into practice what you've learned about effectively communicating with AI and critically evaluating its outputs to produce results that leverage the best of both human and AI capabilities.

Feedback

As you progress through the course, we'd love to hear from you about how you are using concepts from the course in your life, work, or classes and any feedback you may have. Share your feedback here.

🎬 Video transcript

Source video: Y0KidGr9Z2Y

📜 Click to expand transcript (cleaned + AI-translated)

AI Fluency Competency: Discernment

AI fluency means working with AI effectively, efficiently, ethically, and safely. Discernment is specifically about evaluating AI outputs, processes, and behaviors—essentially acting as your quality control system for AI collaboration.

Discernment is your ability to critically evaluate what the AI produces, how it produces it, and how it behaves. It is the flip side of description. If description is about clearly communicating what you want, discernment is about deciding whether what you get back actually meets your needs.

Developing your discernment helps you:

  • Identify when AI outputs are valuable versus problematic.
  • Recognize strengths and limitations.
  • Determine when outputs are ready to use or need more work.

Doing this well requires both domain expertise (knowing enough to judge quality) and an understanding of how AI systems work, including their typical shortcomings. Remember, even the most advanced AI systems can make reasoning errors, produce factual mistakes, or act in unexpected ways. Your capacity for discernment acts as an essential safeguard.

Product Discernment: Evaluating Output Quality

The most straightforward form of discernment is evaluating the quality of what the AI actually produces. We call this Product Discernment: the ability to judge the accuracy and value of AI-created output.

When reviewing AI-generated content, ask yourself:

  • Is this factually accurate?
  • Is it appropriate for my audience and purpose?
  • Is it coherent and well-structured?
  • Does it meet my requirements?
  • Does it add value or solve the problem I intended?

Process Discernment: Evaluating the AI's Logic

When interacting with AI, you need to assess not just what the AI produces, but how it got there. Process Discernment is the ability to judge the quality and effectiveness of the AI's reasoning process.

Some things to look out for include:

  • Logical errors or lapses in the AI's attention.
  • Taking inappropriate steps or getting stuck on one small detail.
  • Being unable to consider alternatives or getting trapped in circular reasoning.

For example, imagine you are working with an AI to expand on one of five outline options it offered. After several rounds of back-and-forth ideation, you might notice elements of rejected ideas being reinserted by the AI. Recognizing this allows you to ensure you and the AI are thinking in sync, guiding the system toward your vision for success. This is especially important for complex tasks where the correct answer isn't immediately obvious and trust in the process is paramount.

Performance Discernment: Evaluating Interaction Quality

It is also valuable to evaluate and guide how the AI behaves during your interaction. We call this Performance Discernment.

The difference between process and performance is subtle:

  • Process is the work the AI is doing.
  • Performance is how well it interacts with you while it does that work.

When evaluating an AI system's performance, ask:

  • Is there a better way for the AI to communicate for greater ease and productivity?
  • Is it providing information in a helpful way?
  • Does it respond well to feedback and direction?
  • Is the interaction efficient or unnecessarily complex? (e.g., Is the AI asking too many questions when you need concise answers, or is it too brief when you need comprehensive information?)

Performance discernment helps you shape a more productive working engagement with AI systems.

Closing the Loop: Feedback and Refinement

Discernment does not end with evaluation; you must provide feedback to improve future deliveries. When discernment flags a problem, effective feedback includes:

  1. Specifying exactly what the problem is.
  2. Clearly explaining why it is a problem.
  3. Providing concrete suggestions for improvement.
  4. Revising your instructions or examples.

In many cases, better description is the solution to a discernment problem. However, sometimes you may need to rethink your delegation decisions entirely, as you might be using the wrong tool or approaching the problem in the wrong way.

Conclusion

Product, process, and performance discernment combine to form the discernment competency. Discernment works hand-in-hand with description: while description focuses on communicating your needs, discernment evaluates how well those needs were met. Together, they form a continuous loop of instruction and evaluation that drives quality. By developing these skills, you ensure that your AI collaboration remains guided by human judgment—a critical element of truly effective AI fluency.

🔁 Related lessons

📚 Source & attribution

Was this lesson helpful?

Feedback / ReportSpotted an issue or have an improvement idea?