Handle with care: the impact of using Java applets in Web-based studies on dropout and sample composition

In Web-based studies, Web browsers are used to display online questionnaires. If an online questionnaire relies on non-standard technologies (e.g., Java applets), it is often necessary to install a particular browser plug-in. This can lead to technically induced dropout because some participants lack the technological know-how or the willingness to install the plug-in. In two thematically identical online studies conducted across two time points in two different participant pools (N = 1,527 and 805), we analyzed whether using a Java applet produces dropout and distortion of demographics in the final sample. Dropout was significantly higher on the Java applet questionnaire page than on the preceding and subsequent questionnaire pages. Age-specific effects were found only in one sample (i.e., dropouts were older), whereas sex-specific effects were found in both samples (i.e., women dropped out more frequently than men on the Java applet page). These results additionally support the recommendation that using additional technologies (e.g., Java applets) can be dangerous in producing a sample that is biased toward both younger and male respondents.