Title: Differences of Type I error rates for ANOVA and Multilevel-Linear-Models using SAS and SPSS for repeated measures designs,
Abstract: To derive recommendations on how to analyze longitudinal data, we examined
Type I error rates of Multilevel Linear Models (MLM) and repeated measures
Analysis of Variance (rANOVA) using SAS and SPSS.We performed a simulation with
the following specifications: To explore the effects of high numbers of
measurement occasions and small sample sizes on Type I error, measurement
occasions of m = 9 and 12 were investigated as well as sample sizes of n = 15,
20, 25 and 30. Effects of non-sphericity in the population on Type I error were
also inspected: 5,000 random samples were drawn from two populations containing
neither a within-subject nor a between-group effect. They were analyzed
including the most common options to correct rANOVA and MLM-results: The
Huynh-Feldt-correction for rANOVA (rANOVA-HF) and the Kenward-Roger-correction
for MLM (MLM-KR), which could help to correct progressive bias of MLM with an
unstructured covariance matrix (MLM-UN). Moreover, uncorrected rANOVA and MLM
assuming a compound symmetry covariance structure (MLM-CS) were also taken into
account. The results showed a progressive bias for MLM-UN for small samples
which was stronger in SPSS than in SAS. Moreover, an appropriate bias
correction for Type I error via rANOVA-HF and an insufficient correction by
MLM-UN-KR for n < 30 were found. These findings suggest MLM-CS or rANOVA if
sphericity holds and a correction of a violation via rANOVA-HF. If an analysis
requires MLM, SPSS yields more accurate Type I error rates for MLM-CS and SAS
yields more accurate Type I error rates for MLM-UN.