By Winnie Tam, Centre for University and School Partnership, The Chinese University of Hong Kong
Decades of research and technological progress have facilitated more sophisticated natural language-based interactions and adaptive learning through artificial intelligence. A recent meta-analysis by Dai and colleagues examined the impact of AI-powered virtual agents in computer-based simulations on learning performance.
The meta-analysis included experimental studies comparing learning outcomes between groups using computer-based simulations featuring AI-powered versus non-AI virtual agents. Performance-based measures were analysed from 22 studies published mainly between 2018-2021 (90% of studies), yielding 49 effect sizes. Random effects modeling revealed a moderate overall effect (Hedges’ g = 0.43).
Moderation analysis uncovered several insights:
• Module-based AI technology achieved the highest impact (g = 0.50), followed by natural language processing/machine learning (g = 0.42) and rule-based designs (g = 0.23).
• Human-like with text agents produced the greatest gains (g = 0.78) relative to fictional characters (e.g. cartoon; g = 0.55) or human avatars (g = 0.35).
• Interaction modality (text, voice, multi-modal) and agent role (e.g., guidance, feedback) had no significant differential impact.
• While intervention duration significantly moderated outcomes, the findings were indeterminate as to whether longer and shorter intervention lengths would produce better results.
Though limited in number of studies, this work provides initial empirical support for AI-powered virtual characters to augment learning via computer simulations.
Best Evidence in Brief Index 