Question

Major car manufacturers make use of computer simulations to experiment with possible plant layouts. The simulations are designed to reflect stochastic effects such as different demand patterns, chance variations in processing times and breakdowns. Hence the results of running a simulation to calculate shifts’ production rates need to be analysed using appropriate statistical methods. Often these computer simulations are quite complex with quite long runtimes, and hence there is an interest in using as few runs of the simulation as possible, whilst still being able to detect meaningful differences in performance between different layouts.

The performance measure on which layouts are being compared is hourly production rate. Suppose that two possible plant layouts are being considered and that each been simulated for 12 shifts. The observed production rates are tabulated below.

Plant Layout Production rates (cars per hour)

Layout 1 128 112 111 114 121 119 131 121 122 114 116 129

Layout 2 126 112 107 114 118 114 131 122 118 113 112 131

a) Suppose that the results from the two layouts are independent samples. What conditions must be satisfied if we are to apply a parametric (i.e. z or t) test to compare the production rates of the two layouts?

b) Describe a way in which the observation rates from the two layouts could be paired. What conditions must be satisfied if we are to apply a parametric (i.e. z or t) test to compare the production rates of the two layouts if the samples are paired?

c) Supposing that the conditions you outlined in part (b) are satisfied, conduct the appropriate parametric test to compare the production rates of the two layouts. Clearly justify your choice of test, state your null and alternative hypotheses and explain your conclusion. Use a 5% significance level.

d) What is the approximate power of the test you have applied in part (c) if the true difference in production rates is that Layout1 is on average 2.0 cars per hour faster than Layout2?

Answer 1

a.

Conditions for independent samples t-test/z-test:

1. Independence: The two populations from which the two samples are taken for comparison must be independent of one another.
2. Normality: The variable of interest must have a normal distribution, at least approximately. If the population standard deviation is known, then the two samples z-test can be used; if the population standard deviation is unknown and has to be estimated by using the sample standard deviation, then the two samples t-test has to be used.
3. Homogeneity of variance: The two populations of interest must be homogeneous in variance, that is, must not have significantly different variances.

b:

Conditions for paired samples t-test/z-test:

1. Paired data: The two populations from which the two samples are taken for comparison must be paired, that is, dependent on one another in some way.
2. Normality: The difference must have a normal distribution, at least approximately. If the population standard deviation is known, then the two samples z-test can be used; if the population standard deviation is unknown and has to be estimated by using the sample standard deviation, then the two samples t-test has to be used.
3. Simple random sample: The paired data must be a simple random sample from the corresponding population.

It must be kept in mind that, even if the populations are not normally distributed, if the sample size is at least 30, it is safe to assume an approximate normal distribution and proceed with the above tests.

c:

It is assumed that the conditions of a paired samples t-test/z-test are satisfied. Note that the population variance of the differences is unknown. As a result, it has to be estimated by the sample standard deviation of differences. Hence, it is suitable to use the paired samples t-test.

A scenario in which the paired samples t-test can be used is, if the layouts are somehow related, such as, if the layouts are based on the same machinery, or same operators, etc.

Suppose μ_d denotes the true mean difference between the mean of Layout 1 and Layout 2, that is, μ_d = μ _{Layout 1} – μ _{Layout 2}. The null and alternative hypotheses are as follows:

H₀: μ_d = 0, vs. H₁: μ_d ≠ 0.

The formula for the test statistic is t = (x̅_d – μ_d)/s_e, where x̅_d is the sample mean difference, and s_e is the sample standard error for differences. Since there are n pairs of observations, the degrees of freedom is (n – 1).

The calculations have been done using Excel. Enter the data for the two layouts in two columns. Go to Data > Data Analysis > t-Test: Paired Two Sample for Means > OK; In Variable 1 Range, enter $A$1:$A$13, in Variable 2 Range, enter $B$1:$B$13; Tick on Labels; Enter Alpha as 0.05; Enter Hypothesized Mean Difference as 0. Click OK.

The output is as follows:

Since the alternative hypothesis is two-tailed, the p-value is: P(T<=t) two-tail ≈ 0.0295.

Since p-value < α (level of significance = 0.05), the null hypothesis is rejected.

Conclusion:

The hourly production rates of the two layouts are significantly different from one another.

d:

Power = P_H1 (| t | > t*), where t* is the test statistic value when H₁ is true.

Here, since Layout 1 is on average 2.0 cars per hour faster than Layout 2, and μ_d = μ _{Layout 1} – μ _{Layout 2}, under H₁, the value of μ_d = 2. Using Excel formulae =AVERAGE(C2:C13) and =STDEV.S(C2:C13), the mean and standard deviation of the sample differences are respectively, x̅_d = 1.67, s_e = 2.3094. Then, the value of t* is:

t

= (x̅_d – μ_d)/s_e

= (1.67 – 2)/2.3094

≈ –0.1429.

Using the Excel formula =T.DIST.2T(0.1429,11) [the negative sign is ignored here; degrees of freedom is n – 1 = 12 – 1 = 11], Power = P_H1 (| t | > t*) ≈ 0.8890.

Hence, the approximate power is 0.8890.