Making Faces: Evaluating Facial Control Methods in VR for Live Conversations

Facial expressions are a crucial part of affective communication in our daily lives, and the same applies to virtual humans. However, achieving a full range of believable and naturalistic emotional expression via VR devices still remains a challenge, particularly when constrained by the capabilities of midrange consumer VR devices without face tracking as opposed to the more expensive versions with facial tracking capabilities. In this study, we evaluated and compared three methods to control avatar facial expression in VR: continuous face tracking (FT); thumbstick label (TL), which is a VR controller's thumbstick-based selection method with visually labeled expression; and thumbstick label with confirmation button (TB), which is a variation of TL with an additional step of pressing a button. Thirty participants took part in the experiment. Participants found FT to be significantly more usable than TL and TB. While TL was comparable to FT on co-presence and several items of expressiveness, with significant gaps emerging mainly for naturalness and smoothness in expression change. These findings suggest that a well-designed control method with low latency and low motor and cognitive load can serve as an alternative method of avatar control in VR devices without facial tracking technology.