18
Experimental Setting and Procedures
All research sessions took place in a university computer lab familiar to all study
participants because their class met in it. The lab was equipped with iMac computers loaded
with Netscape Web browsers. The lab was limited in size so six sessions were necessary to
accommodate the participants. For each of the six sessions study participants were randomly
assigned to the four treatment conditions or the control condition.
4
Measuring the Dependent Variable
After reviewing the treatments study participants took a two tests. The recognition test
contained 20 questions about the euro Web story and two questions (near the beginning) about
the control topic so that poor performance on the test by the control groups would not be
attributable to test panic. The second test required participants to answer two essay questions –
this was the comprehension measure of learning. Participants were asked to write as much as
they could and were not given a time limit.
Reliability of measures
Grading of the comprehension measure involved what was essentially a content analysis.
I graded all the essays and a second coder (also a doctoral student) graded 10 percent of the
sample. Reliability between the coders was .82 (Pearson's R).
For the recognition measure I computed a correlation coefficient to compare results on
the 10 even-numbered items and the 10 odd-numbered items. The result was a significant split-
halves correlation of .36 (Pearson, p<.01, N=59). I excluded the scores of control group
4
This was accomplished by shuffling the pre-treatment questionnaires which each contained a number
corresponding to one of the computers in the lab. Prior to the arrival of study participants, each computer was
loaded with one of the four treatments or with the control Web site.
Convention