 |
International Journal of Internet Science |
A peer reviewed open access journal for empirical findings,
methodology, and theory of social and behavioral science concerning the
Internet and its implications for individuals, social groups,
organizations, and society.
Method Effects on Assessing Equivalence of Online and Offline Administration
of a Cognitive Measure: The Exchange Test
Michael Schreiner1, Siegbert Reiss2, & Karl Schweizer2
1University of Education, Heidelberg, Germany,
2Goethe University Frankfurt, Frankfurt a. M., Germany
Abstract: Internet administration of established offline measures has become more and more common over the past years. Research has shown that online administration of a measure does not necessarily change its psychometric properties but one cannot simply assume that online versions of any measure are equivalent to offline counterparts. In a counterbalanced test-retest-design the web-enabled version of the Exchange Test was administered offline and online. Data were analyzed by means of repeated measurement ANOVA. Results indicated that both administration forms were parallel at the first measurement. Hence, online and offline administration seemed equivalent. However, data from the second measurement show they were not: Analyses revealed a main effect of repeated measurement and a systematic interaction of “repeated measurement” and “order of administration” (online-offline vs. offline-online). Comparable articles often report analyses based on one measurement only or aggregate data from repeated measurements. The present study shows by counterbalanced test-retest-design that there is the possibility of method effects that can only be detected in a cross-mode test-retest design and with appropriate analyses. Potential reasons for the significant differences between offline and online administration at the second measurement are discussed and theoretical explanation approaches are presented that may guide further research into the observed lack of equivalence.
Keywords: Equivalence, Internet, online, working memory, counterbalanced, method-effects
Download full paper