In: Statistics and Probability
In the course of the thesis work, a student develops a new approach for the solution of a problem (here referred to as method B). The current state-of-the-art approach, method A, is well published in the literature and has been applied to a large standard problem set where its average performance was discovered to be (and published in the main paper by the developers as) 7 with a standard deviation of 3 across the different problems in the problem set. In addition to the publication, the developers of method A also provide their code for anyone to be able to experiment with and the student decides to pick a random set of 15 problems from the standard problem set and apply both methods to these problems, resulting in the following performance numbers for method A: {8, 3, 10, 8, 11, 4, 6, 4, 12, 4, 5, 10, 6, 2, 10}, and the following performance numbers for the student’s method B: {9, 5, 9, 10, 15, 4, 7, 4, 12, 7, 8, 10, 6, 4, 12}. Looking at this data, the student discovers that it seems that method B outperforms method A and sets out to prove this using significance testing with a two-tailed 5% significance threshold. Given that both published performance results as well as the student’s experimental results are available, a number of tests can be performed.
Evaluate the results in terms of the hypothesis that method B has a higher performance than method A. List all the steps (and formulas) involved in the test and what the result implies for the significance of the hypothesis.
a.
Given that,
mean(x)=6.866
standard deviation , s.d1=3.204
number(n1)=15
y(mean)=8.133
standard deviation, s.d2 =3.313
number(n2)=15
null, Ho: u1 = u2
alternate, H1: u1 != u2
level of significance, α = 0.05
from standard normal table, two tailed t α/2 =2.145
since our test is two-tailed
reject Ho, if to < -2.145 OR if to > 2.145
we use test statistic (t) = (x-y)/sqrt(s.d1^2/n1)+(s.d2^2/n2)
to =6.866-8.133/sqrt((10.26562/15)+(10.97597/15))
to =-1.0647
| to | =1.0647
critical value
the value of |t α| with min (n1-1, n2-1) i.e 14 d.f is 2.145
we got |to| = 1.0647 & | t α | = 2.145
make decision
hence value of |to | < | t α | and here we do not reject
Ho
p-value: two tailed ( double the one tail ) - Ha : ( p != -1.0647 )
= 0.305
hence value of p0.05 < 0.305,here we do not reject Ho
ANSWERS
---------------
null, Ho: u1 = u2
alternate, H1: u1 != u2
test statistic: -1.0647
critical value: -2.145 , 2.145
decision: do not reject Ho
p-value: 0.305
we do not have enough evidence to support the claim that difference
of means between
method A and B.
b.
Given that,
mean(x)=8.133
standard deviation , s.d1=3.313
number(n1)=15
y(mean)=6.866
standard deviation, s.d2 =3.204
number(n2)=15
null, Ho: u1 = u2
alternate, H1: u1 > u2
level of significance, α = 0.05
from standard normal table,right tailed t α/2 =1.761
since our test is right-tailed
reject Ho, if to > 1.761
we use test statistic (t) = (x-y)/sqrt(s.d1^2/n1)+(s.d2^2/n2)
to =8.133-6.866/sqrt((10.97597/15)+(10.26562/15))
to =1.0647
| to | =1.0647
critical value
the value of |t α| with min (n1-1, n2-1) i.e 14 d.f is 1.761
we got |to| = 1.0647 & | t α | = 1.761
make decision
hence value of |to | < | t α | and here we do not reject
Ho
p-value:right tail - Ha : ( p > 1.0647 ) = 0.15251
hence value of p0.05 < 0.15251,here we do not reject Ho
ANSWERS
---------------
null, Ho: u1 = u2
alternate, H1: u1 > u2
test statistic: 1.0647
critical value: 1.761
decision: do not reject Ho
p-value: 0.15251
we do not have enough evidence to support the claim that method B
has a higher performance than method A