An article about handwriting appeared in the October 11, 2006 issue of the Washington Post. The article mentioned that among students who took the essay portion of the SAT exam in 2005-06, those who wrote in cursive style scored significantly higher on the essay, on average, than students who used printed block letters.

Research question: Is writing in cursive a way to increase exam scores? 

Goals: In this lab, you will

• Explore the distinction between an observational study and an experiment
• Explore the benefits of random assignment and the impact on the scope of conclusions that can be drawn from a study

In comparing essay scores for cursive writers and printed-block-letter writers, identify and classify the explanatory variable and the response variable.

The article also mentioned a different study in which the same one essay was given to all graders. But some graders were shown a cursive version of the essay and the other graders were shown a version with printed block letters. The average score assigned to the essay with the cursive style was significantly higher than the average score assigned to the essay with the printed block letters.

Definition: An observational study is one in which the researchers passively observe and record information about the observational units. In an experimental study, the researchers actively impose the explanatory variable (often called treatment) on the observational units (can also be called the experimental units in an experiment).

Because the researchers actively imposed which type of essay (the explanatory variable) each grader saw, rather than allowing the students to choose their own style of writing, the second study is considered an experiment, with the graders as the experimental units.

In designing an experiment, we want all conditions and variables between the treatment groups to be as similar as possible. Some conditions (e.g., time of day) can be controlled by the researchers, but some characteristics (e.g., handedness of the grader) cannot. But what if left-handed graders tend to give higher grades?? Assigning all the right-handed graders to evaluate block letter essays and all of the left-handed graders evaluate cursive writing essays, would mean handedness of the grader is confounded with the style of writing. This would prevent us from drawing a cause-and-effect conclusion between the type of writing and the score, because we wouldn't know whether the higher score for the cursive group was due to the style of writing or the left-handedness of the graders.

Definition: In a well-designed experiment, experimental units are randomly assigned to the treatment groups. Each unit is equally likely to be assigned to any of the treatments.

To properly carry out such random assignment for say 24 graders, we could label each grader, say 01-24. Then use a random number generator select 12 of the names. These 12 individuals could be assigned the block letter essay, and the rest the cursive essay.

The goal of the random assignment is to create groups that are as similar to each other as possible. To explore whether this actually does “work,” we want to see compare the groups that are created from random assignment. We will do this with the Randomizing Subjects applet.

Your graph of the differences in sample proportions created from the random assignment process should look something like this:

The random assignment process isn't guaranteed to create groups with the same proportion of left and right handers, but on average it does. Meaning, there is no reason to suspect one treatment group is more likely to have right-handers than the other. This means, we have made it hard for handedness to be a confounding variable in this study.

What about age? Maybe older graders tend to give higher scores than younger graders?

The graph below uses the same applet to compute the mean age of each treatment group after each random assignment and then create a graph of the difference in average age (group 1 - group 2).

Graders are volunteer high school or college instructors. Suppose college instructors tend to score differently than high school instructors. If we select the Reveal school? radio button and then select the school variable from the pull-down list, the applet shows this school information for each subject and also the differences in the proportions of high school instructors in the two treatment groups. 

Suppose there were other variables that might impact scoring accuracy that we might not even keep track of, like how much sleep the grader got the night before. If we select the “Reveal sleep?” button in the applet and select “Sleep” from the pull-down list the applet will display the distribution of the difference in average amount of sleep (group 1 - group 2) from the 2000 random assignments.

The primary goal of random assignment is to create groups that equalize any potential confounding variables between the groups, creating explanatory variable groups that overall differ only by the explanatory variable imposed.

Note that this “balancing out” applies equally well to variables that can be observed (such as educational background and age) and variables that may not have been recorded (such as amount of sleep) or that cannot be observed in advance (such as when they will grade the essays).

Although we could have forced variables like age and handedness to be equally distributed between the two groups, the virtue of random assignment is that it also tends to balance out variables that we might not have thought of before the start of the study and variables that we might not be able to see or control (e.g., eyesight).

Thus, when we observe a “statistically significant” difference in the response variable between the two groups at the end of the study, we feel more comfortable attributing this difference to the explanatory variable (e.g., style of writing on the essay) because that should have been the only difference between the groups.

Now what population you are willing to generalize this cause-and-effect relationsnhip to is still another important question...