Sensitivity of Goodness of Fit Indexes to Lack of Measurement Invariance

Abstract

Two Monte Carlo studies were conducted to examine the sensitivity of goodness of fit indexes to lack of measurement invariance at 3 commonly tested levels: factor loadings, intercepts, and residual variances. Standardized root mean square residual (SRMR) appears to be more sensitive to lack of invariance in factor loadings than in intercepts or residual variances. Comparative fit index (CFI) and root mean square error of approximation (RMSEA) appear to be equally sensitive to all 3 types of lack of invariance. The most intriguing finding is that changes in fit statistics are affected by the interaction between the pattern of invariance and the proportion of invariant items: when the pattern of lack of invariance is uniform, the relation is nonmonotonic, whereas when the pattern of lack of invariance is mixed, the relation is monotonic. Unequal sample sizes affect changes across all 3 levels of invariance: Changes are bigger when sample sizes are equal rather than when they are unequal. Cutoff points for testing invariance at different levels are recommended.

Cite this paper

Related papers

1. Evaluating Structural Equation Models with Unobservable Variables and Measurement Error1981
2. Coefficient Alpha and the Internal Structure of Tests1951
3. A new criterion for assessing discriminant validity in variance-based structural equation modeling2014
4. A Guideline of Selecting and Reporting Intraclass Correlation Coefficients for Reliability Research2016
5. Significance tests and goodness of fit in the analysis of covariance structures.1980