Question

Suppose we want to do our own test of whether the way students take notes affects their memory for material (like the Mueller and Oppenheimer study), but we have a three condition experiment, where students are randomly assigned to: 1) a "b.y.o." room where students bring in their backpacks and can use whatever device (laptop/ipad, paper notebook, livescribe pen) and whatever style of note-taking / note-taking sofware they think works best for them; 2) a "laptop" room where students can only takes notes on a word processing program on a blank laptop with no internet connection and 3) a "paper & pen" room where students can only take notes in blank paper notebooks with a pen. After students watch a TED talk and take notes for 30 minutes, they will be given a quiz, and we want to know whether students with different kinds of note-taking opportunities differ in their average quiz scores.

We collect the data and find the following statistics from three groups of 50 students each (total N=150):

Quiz scores:  

byo group: Mean = 21.0,

laptop group: M = 28.0,

paper&pen group: M = 33.5

also, SSb = 38.0                     SSe = 672.5

Now...

1) First explain why we need to know not only the mean quiz score for each group, but also the between groups variance and within groups variance of this data?

2) If we are going to test a significant effect of note-taking device on quiz score a) what statistical test are we going to have to do? b) what are the degrees of freedom for this test (hint: there should be two numbers reported here) and so c) what are our MSB and your MSEd) what is the F-ratio?

3) Does this support (with a reasonably low probability of making a Type 1 error) the claim that there is a significant effect of note-taking device on quiz scores? Explain your reasoning.

4) At this point, what (statistically speaking) more might you want to know?

5) If the standard error of the mean is  sqrt(4.6/50=0.092) = .303 then show me how you can use that to conduct the three post-hoc Tukey HSD tests.

6) What did you just do those tests for? And, what did you learn from them?

7) After going through all of this would you still (statistically speaking) have anything else you might like to know? If so, what, and how would it help you better understand the relationship between note-taking and retention?

Answer 1

a) We need to know the between groups variance & within groups variance as while comparing between the three groups, we need to account the within groups variance & between groups variance. As not all the values in the group will not be the same, within group variance will account for that variation. Between group variation is used to account for variability across the groups.

b) i) We will run a one-way ANOVA test to check the difference between the three groups.

df (Between) = K - 1 = 3 - 1 = 2

df(Error) = N - K = 150 - 3 = 147

	SS	df	MS	F
Between	38	2	19	4.15316
Error	672.5	147	4.57483
Total	710.5	149

MS = SS/df

F = MSB/MSE

c) The p-value associated with the F-statistic is 0.018 at df = 2 and 147. The significance level is 0.05. Hence, as p-value<0.05, we can reject the nul hypothesis and conclude that there is a difference in means between the groups.

d) We might want to know that which pair of groups are significantly different from each other as it might not be true that all pairs have different means.

e) Standard error of the mean is calculated by:

The tukey HSD statistic is: q*Standard Error

Hence, the standard error is used to calculate the tukey HSD statistic.

Here, q-critical value = 3.36

Hence, tukey HSD = 1.02

Now, if the difference between the groups is more than 1.02, it is significant.

The groups have means:

byo group: Mean = 21.0,

laptop group: M = 28.0,

paper&pen group: M = 33.5

We can tell that, difference between byo and laptop group: 7 > 1.02 (significant)

Difference between laptop & paper/pen group = 5.5>1.02 (significant)

Difference between byo and paper/pen group = 12.5 > 1.02 (significant)

f) We did the test to check the significant difference between the pairs of the groups. We learnt that all the pairs of the groups have significant difference between them. Hence, we can say that all the groups have significant difference in means from each other.

g) After all this, we can conclude that there is a significant difference between the three groups & there is an association between note taking and retention. We could also tell that paper& pen group has the most retention followed by laptop group and then followed by byo group. Hence, we have understood the relationship between the variables statistically by doing the ANOVA and post-hoc Tukey test. This is enough & we do not need to do any other tests to check the desired relationship.

If possible, we could check if there are any confounding or lurking variables which could affect our result in this experiment.