A process for appraising commerical usability evaluation methods.

Recent international quality standards and European Community legislation have identified new software quality factors. These new factors include suitability, installability and adaptability. Other quality factors need to be reviewed in the light of these developments. This has impacted on established commercial usability evaluation methods to the extent that it is appropriate to ask if these evaluation methods comply with the new standards and legislation. In order to answer this question the commercial evaluation methods need to be appraised (meta-evaluation) using a suitable method appraisal process. This paper describes such an appraisal process which specifically addresses the many considerations raised by the standards and the law. The appraisal method consists of two parts which provide an overview of the commercial method and a methodical analysis of how it complies. By combining this analysis with a weighting and rating technique the appraised method can achieve a score which can be compared with other commercial methods. The process is an essential tool for strategic managers who are responsible for usability evaluation during systems acquisition. It is also of benefit to supplier organisations who, in their efforts to develop the highest quality systems, need to demonstrate compliance with international standards and development process maturity models.