Question

Give a real life example of a type 1 error as well as a type 2 error and in the context of your example explain how you could reduce the chances of each of them.

Answer 1

Type I and Type II errors are two well-known concepts in quality engineering, which are related to hypothesis testing. Often engineers are confused by these two concepts simply because they have many different names. We list a few of them here.

Type I errors are also called:

1. Producer’s risk
2. False alarm
3. error

Type II errors are also called:

1. Consumer’s risk
2. Misdetection
3. error

Type I and Type II errors can be defined in terms of hypothesis testing.

• A Type I error () is the probability of rejecting a true null hypothesis.
• A Type II error () is the probability of failing to reject a false null hypothesis.

Or simply:

• A Type I error () is the probability of telling you things are wrong, given that things are correct.
• A Type II error () is the probability of telling you things are correct, given that things are wrong.

The above statements are summarized in Table 1.

Assume an engineer is interested in controlling the diameter of a shaft. Under the normal (in control) manufacturing process, the diameter is normally distributed with mean of 10mm and standard deviation of 1mm. She records the difference between the measured value and the nominal value for each shaft. If the absolute value of the difference, D = M - 10 (M is the measurement), is beyond a critical value, she will check to see if the manufacturing process is out of control.

What is the probability that she will check the machine but the manufacturing process is, in fact, in control? Or, in other words, what is the probability that she will check the machine even though the process is in the normal state and the check is actually unnecessary?

Assume that there is no measurement error. Under normal manufacturing conditions, D is normally distributed with mean of 0 and standard deviation of 1. If the critical value is 1.649, the probability that the difference is beyond this value (that she will check the machine), given that the process is in control, is:

So, the probability that she is going to check the machine but find the process is, in fact, in control is 0.1. This probability is the Type I error, which may also be called false alarm rate, α error, producer’s risk, etc.

The engineer realizes that the probability of 10% is too high because checking the manufacturing process is not an easy task and is costly. She wants to reduce this number to 1% by adjusting the critical value. The new critical value is calculated as:

Using the inverse normal distribution, the new critical value is 2.576. From the above equation, it can be seen that the larger the critical value, the smaller the Type I error.

By adjusting the critical line to a higher value, the Type I error is reduced. However, the engineer is now facing a new issue after the adjustment. Some customers complain that the diameters of their shafts are too big. Instead of having a mean value of 10, they have a mean value of 12, which means that the engineer didn’t detect the mean shift and she needs to adjust the manufacturing process to its normal state.

What is the probability of failing to detect the mean shift under the current critical value, given that the process is indeed out of control? In this case, the mean of the diameter has shifted. The answer for this question is found by examining the Type II error.

The engineer asks a statistician for help. The statistician uses the following equation to calculate the Type II error:

Here, is the mean of the difference between the measured and nominal shaft diameters and is the standard deviation. The mean value of the diameter shifting to 12 is the same as the mean of the difference changing to 2. From the above equation, we can see that the larger the critical value, the larger the Type II error.

The result tells us that there is a 71.76% probability that the engineer cannot detect the shift if the mean of the diameter has shifted to 12. This is the reason why oversized shafts have been sent to the customers, causing them to complain.

It seems that the engineer must find a balance point to reduce both Type I and Type II errors. If she increases the critical value to reduce the Type I error, the Type II error will increase. If she reduces the critical value to reduce the Type II error, the Type I error will increase. The engineer asks the statistician for additional help.

The statistician notices that the engineer makes her decision on whether the process needs to be checked after each measurement. This means the sample size for decision making is 1. The statistician suggests grouping a certain number of measurements together and making the decision based on the mean value of each group. By increasing the sample size of each group, both Type I and Type II errors will be reduced. However, a large sample size will delay the detection of a mean shift. The smallest sample size that can meet both Type I and Type II error requirements should be determined. The engineer provides her requirements to the statistician.

The engineer wants:

1. The Type I error to be 0.01.
2. The Type II error to be less than 0.1 if the mean value of the diameter shifts from 10 to 12 (i.e., if the difference shifts from 0 to 2).

Tables and curves for determining sample size are given in many books. These curves are called Operating Characteristic (OC) Curves. From the OC curves of Appendix A in reference [1], the statistician finds that the smallest sample size that meets the engineer’s requirement is 4.

This sample size also can be calculated numerically by hand. Assume the sample size is n in each group. The mean value and the standard deviation of the mean value of the deviation (difference between measurement and nominal value) of each group is 0 and under the normal manufacturing process. Based on the Type I error requirement, the critical value for the group mean can be calculated by the following equation:

Under the abnormal manufacturing condition (assume the mean of the deviation shifts to 2 and the standard deviation is unchanged), the Type II error is:

The smallest integer that satisfies the above two equations is the value of n.

Let’s set n = 3 first. The critical value is 1.4872 when the sample size is 3. Using this critical value, we get the Type II error of 0.1872, which is greater than the required 0.1. So we increase the sample size to 4. The critical value becomes 1.2879. The corresponding Type II error is 0.0772, which is less than the required 0.1. Therefore, the final sample size is 4.

By using the mean value of every 4 measurements, the engineer can control the Type II error at 0.0772 and keep the Type I error at 0.01.

Sometimes, engineers are interested only in one-sided changes of their products or processes. For example, consider the case where the engineer in the previous example cares only whether the diameter is becoming larger. The hypothesis test becomes:

Assume the sample size is 1 and the Type I error is set to 0.05. The critical value will be 1.649. For detecting a shift of , the corresponding Type II error is . Readers can calculate these values in Excel or in Weibull++. The relation between the Type I and Type II errors is illustrated in Figure 1: