Real Time Detection and Analysis of Facial Features to Measure Student Engagement with Learning Objects

This paper describes a software application that records student engagement in an on-screen task. The application records in real time the on-screen activity and simultaneously estimates the emotional state and head pose of the learner. The head pose is used to detect when the screen is being viewed and the emotional state provides feedback on the form of engagement. The application works without recording images of the learner. On completing the task, the percentage of time spent viewing the screen and statistics on emotional state (neutral, happy, sad) are produced. A graph depicting the learner’s engagement and emotional state synchronised with the screen captured video is also produced. It is envisaged that the tool will find application in learning activity and learning object design.