Epistemologies Of Assessment Instruments

Understanding our epistemological perspective when conducting engineering education research is important for situating the knowledge claims we are making. Depending on that perspective, we may situate the knowledge claims as definitive, representing an absolute Truth, or as contingent, representing a contextualized truth. Traditionally, quantitative research has been identified as positivist, while qualitative research is diverse in its epistemological assumptions, ranging from positivist to interpretivist to Critical and the “posts.” Thus, results from quantitative studies are often treated as generalizable, absolute, and decontextualized, while quantitative studies are treated as particular, contingent, and contextualized. Assessment instruments, being quantitative, are associated with positivist forms of knowledge. We argue that it is more appropriate to treat quantitative assessments as interpretivist. Development of assessments is based on particularized knowledge that is created through a dialogue between the developers and the pilot participants. Interpretation of assessment results is dependent on the particular contexts in which they are used. In this paper we describe the interpretivist roots of assessment using the example of our current project on developing an instrument for engineering quantitative literacy. In the first phase of this project we have used qualitative content analysis to identify the ways in which quantitative literacy is assessed in first-year engineering courses in the United States. This analysis is contextualized by the particulars of these courses, and the results are contingent on the interpretations we make as researchers. We discuss how this interpretivist perspective carries through the entire project as we create and implement a measure of quantitative literacy for engineering students.