A multilevel factor analysis of students evaluations of teaching

This study examined the factorial validity of scores on the newly developed Students' Evaluation of Teaching Effectiveness Rating Scale (SETERS) through a series of confirmatory and multilevel structures. Conventional confirmatory factor analyses using the total covariance and pooled within-covariance matrices from two midwestern universities indicated that a reduced 25-item SISTERS fit the data better than the original 34-item SETERS. Furthermore, multilevel factor analysis was conducted on the combined samples. This analysis suggested that one or three factors at the between and within levels were a plausible representation of SETERS scores. Pearson's correlations between individual scores on the SETERS and the Students' Evaluation of Educational Quality questionnaire provided additional validity evidence for the two measures. The need for additional empirical research on the SETERS before widespread use is discussed.