Meta2: A meta‐analysis and psychometric evaluation of the Metacognitive Awareness Inventory (MAI) in the context of health professions education

Abstract

To keep pace with medicine's unpredictable changes, medical trainees must learn to accurately monitor and evaluate themselves via metacognition (i.e., thinking about thinking). The Metacognitive Awareness Inventory (MAI) can assess and guide the metacognitive development of trainees. This study summarizes existing psychometric evidence and meta-analyzes the internal consistency of the MAI across studies involving health professions education. In 2023, a literature search was performed across six databases for records that met predefined inclusion criteria. After two rounds of screening, MAI validity and reliability data were extracted from the eligible studies for analysis. Validity evidence for the dichotomous, five-point Likert, and sliding analog versions of the MAI were rated and analyzed according to "test content," "response processes," "internal structure," "relations to other variables," and "consequences of testing." The internal consistency range (min-max) of the MAI was estimated using random-effects, restricted maximum likelihood (REML) meta-analysis. Among the 1966 records screened, 23 studies met the inclusion criteria. The five-point Likert version of the MAI was the most supported, with validity evidence in the "test content," "internal structure," and "relations to other variables" categories. None of the MAI versions presented evidence related to "response processes" or "consequences of testing." The lowest aggregated internal consistency was estimated to be 0.805 and the highest as 0.844, suggesting very good reliability. The five-point Likert scale version of the MAI is most supported for use in health professions education, though validation efforts should continue to address all five sources of validity.