Send us a text

Validation Architecture, Not Validation Effort | Human Centered AI Ep 007

Deloitte Australia delivered a $440,000 AI-assisted report. The client discovered fake citations, non-existent authors, and books that were never written.

This isn't about criticizing Deloitte - they're tackling what we're all facing. How do you validate AI output without destroying the speed advantage?

The Speed Paradox
AI generates a 100-page report in 3 hours. Human validation takes 2 weeks.
You can't slow back to human speed (defeats the purpose). You can't trust blindly (Deloitte proved that costs $440,000).

So what's the answer?

In This Episode:
→ What actually broke at Deloitte (and why it's a process problem, not a technology problem)
→ Why LLMs are eloquence engines, not truth engines
→ The validation architecture we use for AI-assisted reports
→ How to build checkpoints that preserve speed advantage
→ Why transparency about AI use becomes competitive advantage
→ Managing AI agents vs. managing humans (completely different principles)
→ Four implementation guidelines you can use immediately

Key Insights:
The validation bottleneck is real. If you're reading every word, you're back to human speed with added risk.
Transparency must come first. The AI conversation happens before the project, not after someone finds hallucinations.
Speed without checkpoints is just risk. Build validation milestones throughout creation, not just at the end.

Our Approach:
- Declare sources first (set boundaries or you'll get books that don't exist)
- Cross-validate patterns, not sentences
- Build checkpoints throughout (like data packets - check key milestones, not every byte)
- Human expertise where it matters (evaluate output quality, not proofread words)

Three Questions for Your Practice:
- What's your validation framework that doesn't require reading every word?
- Have you told clients HOW you use AI before they discover it themselves?
- Are you validating during creation or only after?

How you validate matters more than how much you validate.

Deloitte paid $440,000 for this lesson publicly. Learn it here for free.

RESOURCES:
📊 AI Future Signals 2025 Report (with full methodology): https://www.designthinkingjapan.com/#futuresignals

Send us a text

"This is the next generation AI-powered convenience store that will become the standard."

When Lawson and KDDI made this bold claim about their Real Tech store in Tokyo, we had to see it ourselves. As AI implementation practitioners, we learn as much from ambitious attempts as we do from polished successes.

The press releases promised, 14 AI cameras for personalized recommendations, intelligent avatars, robot food prep, adaptive shopping experiences.

Unfortunately, what we found was No camera disclosure, "AI avatar" was a human on video call, branded Roomba, staff manually counting inventory beside computer vision equipment, non-interactive screens, zero personalization.

We spoke English to the "smart avatar." It replied, "A little." That's when we realized we were talking to a person, not AI.

This isn't about criticizing Lawson or KDDI, they tackled an impossible challenge. Japanese convenience stores are already efficiency masterpieces.

The promise gap between AI marketing and AI reality is widening across industries.

Three Critical Insights

1. Marketing promises can sabotage good work. Customers felt misled by "AI-powered" experiences that were actually human-powered—even though human solutions might be better.

2. Integration trumps innovation. 14 cameras don't automatically create personalized experiences. The hard work is connecting cameras to inventory systems, recommendation engines, and displays in ways that actually help customers.

3. Expectation management matters. When you promise "the future," customers expect something genuinely different from everywhere else.

The technology exists: facial recognition for greetings, real-time inventory tracking, gaze detection, automated checkout. The challenge isn't capability, rather it's system integration and user experience design.

Here's Four Implementation Guidelines

1. Start with specific friction. Not "AI-powered store," but "What customer problems can technology solve? Long lines? Product discovery? Language barriers?"

2. Test quietly, announce loudly. Build it, validate it works, then tell people. Order matters.

3. Be honest about automation. Customers handle knowing humans help remotely. They can't handle feeling deceived.

4. Under-promise, over-deliver. Surprise beats disappointment every time.

Lawson and KDDI deserve credit for pushing boundaries publicly. Most companies play it safe.

But their experience reminds us that customer trust comes from honest, valuable experiences and not impressive press releases.

The future of retail will involve AI. But it'll be shaped by companies solving real customer problems, not showcasing impressive technology.

Send us a text

Security expert Jonathan Baier joins Brittany Arthur to explore how business leaders can approach AI security strategically. Learn practical frameworks for implementing AI while protecting what matters most to your organization.

What You'll Learn

Reframe Security as Strategy

• How security teams can accelerate AI initiatives rather than slow them down
• The three types of AI security every executive must understand: of AI, from AI, and with AI
• Moving from "we need AI" to identifying specific value-creating opportunities

Bridge the Gap Between Business and Security

• Practical frameworks for non-technical leaders to have meaningful security conversations
• Why 70% of AI success depends on people, not algorithms
• How to ask the right questions when you don't have deep technical knowledge

Master the Innovation-Protection Balance

• When to take calculated risks vs. when to proceed more cautiously
• Real examples of companies navigating AI security decisions
• Starting with business problems rather than AI solutions

Key Takeaways

✅ Security becomes a competitive advantage when technology access is democratized

✅ Your differentiators are data, process, and people—not the technology itself

✅ Every business question becomes a security question at scale

✅ Curiosity and mindful experimentation beat both paralysis and reckless confidence

✅ Small companies need shared security understanding, not dedicated security officers

3 Power Quotes for Sound Bites

1. On AI's Limitations

"AI gives us what we ask for and not necessarily what we need."

2. On Taking the Right Approach

"You actually shouldn't start with AI. You should start with what is your problem that you're trying to solve?"

3. On Security Team Partnerships

"I think many security teams want to be helpful, but they've sort of gotten stereotyped as they're going to get in the way. And so they're not used to someone coming and saying, hey, let's work together and figure out how to do this. But I think they're very much excited to do that."

Connect with Jon: https://www.linkedin.com/in/jonathanbaier/