Question

What is Statistical Significance? Date 01/29/2019

What is statistical significance? If we randomly assign subjects to two groups we would expect there to be some difference in the groups just by chance. If a difference is statistically significant then it is large enough that we would not expect it to happen just by chance. When we compare the results of an experiment, if there is a statistically significant difference then we can conclude that there is a cause-effect relationship between the explanatory variable and the response variable. In this activity we will explore what size difference that is due to chance. This will help us determine statistically significant differences.

The Experiment

Have you ever used music at work to jack up productivity or change your mood? Interestingly some rhythms such as​ ​baroque​, induce enzymes in the brain and add amazing well being and focus. Other tunes leave you punchy ... and unable to focus. Classical​ ​music, such as Haydn and Mozart, often improves concentration and memory when played in the background.

I would like to determine if listening to classical music while studying improves exam scores. Twenty students volunteer to participate in my experiment. Outline the experiment. Use the example shown below as a guide.

What is the explanatory variable? ________________________________________________

What is the response variable? ___________________________________________________

Now it is time to randomly assign the students to the two treatments: to study while listening to classical music or to study in a quiet environment. In order to determine the magnitude of differences that happen by chance, we will assume that listening to classical music while studying does not improve exam scores.

1. We must randomly assign the 20 students to a treatment group. Use the random number table (last page) to make this assignment. Generate 20 one-digit random numbers and indicate the line you used.

Line ​134 20 one-digit numbers ​2, 7, 8, 1, 6, 7, 8, 4, 1, 6, 1, 8, 3, 2, 9, 2, 1, 3, 3, 7

If the number is odd (1, 3, 5, 7, 9), assign the student to study with music. If the number is even (0, 2, 4, 6, 8), assign the student to study with no music. As soon as you have 10 in one treatment group, put remaining students in other group so that there are 10 students in each treatment group.

Student

Random Number

Group (Circle)

Student

Random Number

Group (Circle)

1

​2

Study with music

Study with no music

11

​1

S​tudy with music Study with no music

2

​7

Study with music

Study with no music

12

​8

Study with music

Study with no music

3

​8

Study with music

Study with no music

13

​3

Study with music

Study with no music

4

​1

Study with music

Study with no music

14

​2

Study with music

Study with no music

5

​6

Study with music

Study with no music

15

​9

Study with music

Study with no music

6

​7

Study with music

Study with no music

16

​2

Study with music

Study with no music

7

​8

Study with music

Study with no music

17

​1

Study with music

Study with no music

8

​4

Study with music

Study with no music

18

​3

Study with music

Study with no music

9

​1

Study with music

Study with no music

19

​3

Study with music

Study with no music

10

​6

Study with music

Study with no music

20

​7

Study with music

Study with no music

2. The students take the exam and the grades they earned are recorded below:

3. Record the average score for the subjects in each of the two groups. Also calculate the

difference between these group averages (‘study with music’ average minus ‘study without music’ average). Be sure to indicate if your difference is negative or positive.

Study with music: ​70.7​ Study without music: ​62.9 Difference: ​7.8

4. Below I have recorded the differences determined by students in previous classes. Add

your difference to the results given below.

Match the comparison with the correct interpretation:

___​A​____ What does a positive difference mean? A. The students who studied without music did better than the students who studied with music.

____​B​___ What does a negative difference mean? B. The students who studied with music did better than the students who studied without music.

5. Examine the differences reported above.

Student

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

Score

97

61

83

57

79

63

75

67

83

82

77

87

62

81

78

77

52

77

95

73

Study with music – study with no music

- 4.2

2.2

- 2.6

3.4

0.2

3.6

- 3.8

- 4.6

5.2

  

What is the biggest difference that you observe?

What is the smallest difference you observe?

What is the typical difference that you observe? Explain how you determined this value.

6. Suppose I found that there was a difference of 3 points in the average score of the two groups in my experiment. Do you feel this difference is likely to happen just by chance? Explain your reasoning.

7. Suppose I found that there was a difference of 10 points in the average score of the two groups in my experiment. Do you feel this difference is likely to happen just by chance? Explain your reasoning.

8. Which of the differences discussed above (3 points and 10 points) is a significance difference? Explain.

Answer 1

Explanatory variable: Classical music

Response variable: Exam score

4.

A. A positive difference means that the students who studied with music did better than the students who studied without music.

B. A negative difference means that the students who studied without music did better than the students who studied with music.

(since, we are doing ‘study with music’ average minus ‘study without music’ average).

5.

From the given 9 differences:

Biggest difference =5.2

Smallest difference =0.2

Typical difference =Average difference =(- 4.2+2.2 - 2.6+3.4+0.2+3.6 - 3.8 - 4.6+5.2)/9= - 0.067

6.

Suppose you found that there was a difference of 3 points in the average score of the two groups in your experiment, I feel this difference is likely to happen just by chance.

Reasoning: The difference of 3 points for a maximum score of 100 is negligible (It's only 3%).

7.

Suppose you found that there was a difference of 10 points in the average score of the two groups in your experiment. I don't feel this difference is likely to happen just by chance.

Reasoning: The difference of 10 points for a maximum score of 100 is considerable. (It is 10%).

8.

The differences of 10 points is a significant difference because 10 points out of a maximum of 100 points is 10% which cannot occur just by chance and thus, it is significant.