Case Study on Performance and Acceptance of Computer-Aided Assessment

Computer aided assessment (CAA) comprises a set of assessment techniques that are conducted on a computer. These forms of assessment may be purely online or network based or single, stand-alone device based [1]. The device may be a desktop computer or include the use of mobile devices. This paper investigates if CAA leads to improved performance and satisfaction [2]. The student cohort that participated in this pilot study came from a
degree in computer science. A preliminary investigation was performed by executing two
continuous assessments with one cohort of four students in an advanced databases class. One of the assessments was purely paper-based (PBA), the other assessment was fully computer-aided. Both assessments were conducted in an open-book manner. Additionally, both contained an element of replication tasks and applied knowledge task [3]. It
was anticipated that computing students would excel in computer-aided assessment, and that this form of assessment would significantly improve both satisfaction and task performance on the side of the students. However, detailed analysis revealed that there was no statistically significant increase using CAA versus traditional PBA. Furthermore, contrary
to initial hopes, students did not appear to have gained a higher level of satisfaction conducting tasks on a computer. One of the reasons given was that most exams at the host institution are PBA; consequently, assessments should prepare for the related exams, and should also be paper-based. As the research reported here is a pilot study on a small
cohort, follow up studies on larger and more diverse cohorts will further inform the body of knowledge in this area.