Researchers Evaluate Alert Systems to Flag AI Hallucinations in Virtual Reality Avatars

Computer science researchers at North Carolina State University and partner institutions have evaluated three interface techniques designed to alert virtual reality users when an artificial intelligence avatar speaks inaccurate information. The proof-of-concept study, reported by TechXplore (https://techxplore.com/news/2026-09-ways-users-ai-hallucinations-embodied.html), investigates how systems can flag generative AI hallucinations in 3D environments without disrupting user presence.

Embodied conversational agents use graphical models capable of speech, physical gestures, and facial expressions in interactive settings like educational software or customer service applications. While standard chat tools can highlight text or attach hyperlinks to convey uncertainty, immersive virtual reality introduces a distinct interface challenge because spoken responses are dynamic and transient.

Qiao Jin, an assistant professor of computer science at NC State and the paper's corresponding author, noted that communicating uncertainty and provenance is straightforward in text interfaces, but less clear in immersive environments where humanlike avatars converse directly with users.

To measure how different alerts perform, the researchers tested four conditions with 24 college student participants using a VR system featuring an avatar rendered as a woman wearing a suit. The conditions included embodied cues, where the avatar made physical gestures such as crossing its arms during uncertain claims; icon cues, where a floating notification appeared beside the agent; text cues, where subtitles changed color to indicate potential inaccuracies; and a control condition with no cues.

All three alert mechanisms outperformed the control condition in helping users identify hallucinated statements, though each introduced specific trade-offs. According to Xiaoran Yang, an NC State Ph.D. student and lead author, text cues delivered the clearest interpretability for catching false statements but caused the most distraction from the broader virtual environment. In contrast, embodied physical gestures preserved the highest levels of immersion and avatar trust, while floating icon indicators provided a middle ground between clarity and distraction.

The authors noted that practical deployment of physical cues will require addressing cultural variation in body language, as gestures such as breaking eye contact can indicate uncertainty in some cultures while signaling deference or respect in others.

The paper, titled "Signals of AI Hallucination: Designing Hallucination-Aware Cues for Embodied Conversational Agents in VR," is published on the arXiv preprint server and scheduled for presentation on October 8 at the IEEE International Symposium on Mixed and Augmented Reality in Bari, Italy. Co-authors include researchers from NC State, Carnegie Mellon University, and Xi'an Jiaotong-Liverpool University.