Hybrid Models of Performance Using Mental Workload and Usability Features via Supervised Machine Learning

Mental Workload (MWL) represents a key concept in human performance. It is a complex construct that can be viewed from multiple perspectives and affected by various factors that are quantified by different collection of methods. In this direction, several approaches exist that aggregate these factors towards building a unique workload index that best acts as a proxy to human performance. Such an index can be used to detect cases of mental overload and underload in human interaction with a system. Unfortunately, limited work has been done to automatically classify such conditions using data mining techniques. The aim of this paper is to explore and evaluate several data mining techniques for classifying mental overload and underload by combining factors from three subjective measurement instruments: System Usability Scale (SUS), Nasa Task Load Index (NASATLX) and Workload Profile (WP). The analysis focused around nine supervised machine learning classification algorithms aimed at inducing model of performance from data. These models underwent through rigorous phases of evaluation such as: classifier accuracy (CA), receiver operating characteristics (ROC) and predictive power using cost/benefit analysis. The findings suggest that Bayesian and tree-based models are the most suitable for classifying mental overload/underload even with unbalanced data.