Stop AI Hallucinations

The new consulting edge isn't AI.

It's knowing when your AI is wrong.

Every consultant has been there: You ask AI to analyze documents and generate insights. During review, you spot a questionable stat that doesn't exist in the source!

AI hallucinations are a problem.

The solution? Implementing "prompt evals".

→ Prompt evals: directions that force AI to verify its own work before responding.

A formula for effective evals:

1. Assign a verification role

   → "Act as a critical fact-checker whose reputation depends on accuracy"

2. Specify what to verify

   → "Check all revenue projections against the quarterly reports in the appendix"

3. Define success criteria

   → "Include specific page references for every statistic"

4. Establish clear terminology

   → "Rate confidence as High/Medium/Low next to each insight"

Here is how your prompt will change:

OLD: "Analyze these reports and identify opportunities."

NEW: "You are a senior analyst known for accuracy. List growth opportunities from the reports. For each insight, match financials to appendix B, match market claims to bibliography sources, add page ref + High/Med/Low confidence, otherwise write REQUIRES VERIFICATION.”

Mastering this takes practice, but the results are worth it.

What AI leaders know that most don't:

"If there is one thing we can teach people, it's that writing evals is probably the most important thing." Mike Krieger, Anthropic CPO

By the time most learn basic prompting, leaders will have turned verification into their competitive advantage.

Steps to level-up your eval skills:

→ Log hallucinations in a "failure library"

→ Create industry-specific eval templates

→ Test evals with known error examples

→ Compare verification with competitors

Next time you're presented with AI-generated analysis, the most valuable question isn't about the findings themselves, but:

'What evals did you run to verify this?'

This simple inquiry will elevate your teams approach to AI & signal that in your organization, accuracy isn't optional.