Designing for Predictability: How Mental Models of AI Error Boundaries Shape Human-AI Team Performance in High-Stakes Decision Making cover image

Designing for Predictability: How Mental Models of AI Error Boundaries Shape Human-AI Team Performance in High-Stakes Decision Making

Designing for Predictability: How Mental Models of AI Error Boundaries Shape Human-AI Team Performance in High-Stakes Decision Making

Listen for free

View show details

Season's Savings | 0.99 €/mo for 3 months

6.99 €/month thereafter - terms apply. Cancel monthly.
Abstract: As artificial intelligence systems are increasingly deployed to advise human decision-makers in high-stakes domains—including healthcare, criminal justice, and financial services—organizations face a critical but often overlooked challenge: the accuracy of an AI system alone does not determine the performance of the human-AI team. This article examines the role of human mental models of AI capabilities, specifically mental models of AI error boundaries, in shaping collaborative decision-making outcomes. Drawing on foundational experimental research demonstrating that properties such as the parsimony and stochasticity of an AI's error boundary significantly influence whether humans can learn when to trust or override AI recommendations, this article translates those findings into actionable organizational strategies. Evidence-based interventions are presented across interface design, model selection, training protocols, and governance structures, illustrated with examples from healthcare, surgery, criminal justice, and AI research and development. Forward-looking pillars for building long-term human-AI collaboration capability—including psychological contract recalibration, continuous learning systems, and distributed oversight—are proposed for practitioners seeking to maximize the return on AI-augmented decision systems.
adbl_web_anon_alc_button_suppression_t1
No reviews yet