Getting better results

Building Trust in AI through the Delegation Diligence Loop

In our last lesson, we dealt with data privacy and security—what you absolutely need to protect and how to do it. Now, let's address the question that has likely stopped you from using AI for data analysis in the first place: How can I trust the results?

Today's lesson is about the Delegation Diligence Loop. Specifically, it is about building confidence in AI's analytical capabilities for your specific work by systematically testing it against data you already understand. By doing this, you can better understand how AI will support your specific circumstances.

The Process of Delegation

The process starts with delegation. Here is how this works:

1. Identify a specific analytical task you do regularly that you want to delegate to AI.
2. Find past data where you have already completed the analysis.
3. Work with AI to reproduce what you did, evaluating what works and what doesn't.
4. Refine your approach and test again.

If AI can match your known results, you know how to use it and trust it for similar future tasks. If not, you have learned that this specific task is something you should not delegate.

Case Study: Practical Application in Program Management

To see what this looks like in practice, let's look at Rio, the program director at Valley Veterans Services. Every quarter, he analyzes program attendance alongside employment outcomes, calculating participation rates, tracking monthly changes, and determining whether attendance correlates with job placement success. This analysis consistently takes him hours.

Setting Up the Test Case

Considering delegation, Rio knows he wants to continue using the results of this analysis to improve his program. He wants to interpret the results himself, but he could do without the data cleaning and formula mayhem he usually finds himself in.

To test whether AI is appropriate, he evaluates it using last quarter's data. He knows exactly what this data showed after he analyzed it without AI, and he has the raw, messy data from before he started. This is his test case.

Executing the Diligence Loop

Rio uploads the data and starts to work with AI, using description and discernment to perform his analysis. Each time the AI responds, Rio checks the results against what he knows to be true and jots down potential gaps in AI's reasoning.

• Refining Description: Sometimes additional description helps AI get the outcome he is looking for. In these cases, Rio knows he must include that kind of information for future tasks.
• Identifying Capability Gaps: Other times, Rio might find legitimate capability gaps. This diligence changes what he chooses to delegate in the future.

Iterative Testing and Refinement

Rio's first attempt might look like this: "I'm sharing attendance data and employment outcome data from our job training program last quarter. Please analyze the participation patterns across the three months and graph the correlations between attendance levels and employment success."

AI responds with a summary, but rather than assuming this is fact, Rio checks it against his records. He notices that while AI correctly identified the correlation, it missed a critical insight regarding the combined housing assistance and job placement program.

Rio refines his description, asking AI to try again but pay special attention to the program type. This time, AI catches its mistake. Rio notes that for future quarters, he will need to specifically request the AI to consider the program type.

Next, he tests something harder: "Can you also look at this based on when participants enrolled?" Rio observes that despite not having the enrollment data explicitly formatted, AI could help extract it. He makes a note to cross-reference these results later.

Validated Confidence vs. Guesswork

By going through this process, Rio has systematically validated what AI can and cannot do for his quarterly reporting. He has learned that with the right description, AI can accurately reproduce manual analysis. However, he has also identified clear limitations:

• AI needs explicit enrollment dates to do cohort analysis; otherwise, it tries to infer them inaccurately.
• He now has a tested approach and clear notes on what context he needs to add himself.

When Rio uses this validated approach with new data, his diligence continues. He checks whether numbers make sense, takes accountability for the final report, and remains transparent about AI's role. He is now working from validated confidence, not guesswork.

Framework for Data Delegation

If you want to apply this yourself, follow this framework:

1. Identify a specific analytical task that you want to delegate. Be precise about what you need.
2. Find past data where you already completed that analysis. You need the right answers to evaluate whether AI can arrive at them.
3. Work with AI to reproduce your past analysis and systematically evaluate the results.
   • What did AI produce?
   • How did it approach the task?
   • How did it communicate findings?
4. Identify gaps, refine your delegation, and test again.

If you can validate that AI produces correct results, you have built an approach you can use on new data. If you cannot get there after several refinements, you have learned that this is not a task you should delegate.

Support for Non-Data Experts

What if you are not comfortable with data to begin with? AI can still be a useful tool to brainstorm and implement solutions. Because AI models are uniquely good at coding, they can help with:

• Writing Excel formulas.
• Reformatting messy data.
• Explaining complex data concepts.

In these cases, bring your question to AI and ask for help understanding what a solution could look like, just as you would work with a data analyst on your team. Ask for clarifications and explanations so you can follow the process and understand the final output.

Final Principles of AI Analysis

Validation builds confidence, but it does not eliminate responsibility. You are still accountable for checking that results make sense and being transparent about AI's role in your analysis process.

This testing works for any analytical task:

• Donor analysis.
• Budget forecasting.
• Survey synthesis.
• Outcome tracking.

Test first, validate what works, and then apply with confidence—or learn what you shouldn't delegate at all.