An evaluation of a computational technique for measuring the embeddedness of sustainability in the curriculum aligned to AASHE-STARS and the United Nations Sustainable Development Goals

Introduction: SDG 4.7 mandates university contributions to the United Nations (UN) Sustainable Development Goals (SDGs) through their education provisions. Hence,
universities increasingly assess their curricular alignment to the SDGs. A common
approach to the assessment is to identify keywords associated with specific SDGs and
to analyze for their presence in the curriculum. An inherent challenge is associating
the identified keywords as used in the diverse set of curricular contexts to relevant
sustainability indicators; hence, the urgent need for more systematic assessment as
SDG implementation passes its mid-cycle.
Method: In this study, a more nuanced technique was evaluated with notable
capabilities for: (i) computing the importance of keywords based on the term
frequency-inverse document frequency (TF-IDF) method; (ii) extending this
computation to the importance of courses to each SDG and; (iii) correlating
such importance to a statistical categorization based on the Association for the
Advancement of Sustainability in Higher Education (AASHE) criteria. Application
of the technique to analyze 5,773 modules in a university’s curriculum portfolio
facilitated categorization of the modules/courses to be “sustainability-focused” or
“sustainability-inclusive.” With the strategic objective of systematically assessing the
sustainability content of taught curricula, it is critical to evaluate the precision and
accuracy of the computed results, in order to attribute text with the appropriate
SDGs and level of sustainability embeddedness. This paper evaluates this technique,
comparing its results against a manual and labor-intensive interpretation of expert
informed assessment of sustainability embeddedness on a random sample of 306
modules/courses.
Results and discussion: Except for SDGs 1 and 17, the technique exhibited a
reasonable degree of accuracy in predicting module/course alignment to SDGs and
in categorizing them using AASHE criteria. Whilst limited to curricular contexts from
a single university, this study indicates that the technique can support curricular
transformation by stimulating enhancement and reframing of module/course
contexts through the lens of the SDGs.