Question

1. the probability of type 2 error will be affected by the choice of test statistics. but why???

Please draw the picture to explain and step by step

follow the comment as well

Answer 1

A significance level α corresponds to a certain value of the test statistic, say tα, represented by the orange line in the picture of a sampling distribution below (the picture illustrates a hypothesis test with alternate hypothesis "µ > 0")

Since the shaded area indicated by the arrow is the p-value corresponding to tα, that p-value (shaded area) is α.
To have p-value less than α , a t-value for this test must be to the right of tα.
So the probability of rejecting the null hypothesis when it is true is the probability that t > tα, which we saw above is α.
In other words, the probability of Type I error is α.

Rephrasing using the definition of Type I error:

The significance level α is the probability of making the wrong decision when the null hypothesis is true.

Pros and Cons of Setting a Significance Level:

• Setting a significance level (before doing inference) has the advantage that the analyst is not tempted to chose a cut-off on the basis of what he or she hopes is true.
• It has the disadvantage that it neglects that some p-values might best be considered borderline. This is one reason2 why it is important to report p-values when reporting results of hypothesis tests. It is also good practice to include confidence intervals corresponding to the hypothesis test. (For example, if a hypothesis test for the difference of two means is performed, also give a confidence interval for the difference of those means. If the significance level for the hypothesis test is .05, then use confidence level 95% for the confidence interval.)

Type II Error

Not rejecting the null hypothesis when in fact the alternate hypothesis is true is called a Type II error. (The second example below provides a situation where the concept of Type II error is important.)

Note: "The alternate hypothesis" in the definition of Type II error may refer to the alternate hypothesis in a hypothesis test, or it may refer to a "specific" alternate hypothesis.

Example: In a t-test for a sample mean µ, with null hypothesis ""µ = 0" and alternate hypothesis "µ > 0", we may talk about the Type II error relative to the general alternate hypothesis "µ > 0", or may talk about the Type II error relative to the specific alternate hypothesis "µ > 1". Note that the specific alternate hypothesis is a special case of the general alternate hypothesis.

In practice, people often work with Type II error relative to a specific alternate hypothesis. In this situation, the probability of Type II error relative to the specific alternate hypothesis is often called β. In other words, β is the probability of making the wrong decision when the specific alternate hypothesis is true.


Considering both types of error together:

The following table summarizes Type I and Type II errors:

	Truth (for population studied)
Null Hypothesis True	Null Hypothesis False
Decision (based on sample)	Reject Null Hypothesis	Type I Error	Correct Decision
Fail to reject Null Hypothesis	Correct Decision	Type II Error

The null hypothesis is "defendant is not guilty;" the alternate is "defendant is guilty." A Type I error would correspond to convicting an innocent person; a Type II error would correspond to setting a guilty person free. The analogous table would be:

	Truth
Not Guilty	Guilty
Verdict	Guilty	Type I Error -- Innocent person goes to jail (and maybe guilty person goes free)	Correct Decision
Not Guilty	Correct Decision	Type II Error -- Guilty person goes free

The following diagram illustrates the Type I error and the Type II error against the specific alternate hypothesis "µ =1" in a hypothesis test for a population mean µ, with null hypothesis ""µ = 0,"  alternate hypothesis "µ > 0", and significance level α= 0.05.

• The blue (leftmost) curve is the sampling distribution assuming the null hypothesis ""µ = 0."
• The green (rightmost) curve is the sampling distribution assuming the specific alternate hypothesis "µ =1".
• The vertical red line shows the cut-off for rejection of the null hypothesis: the null hypothesis is rejected for values of the test statistic to the right of the red line (and not rejected for values to the left of the red line)
• The area of the diagonally hatched region to the right of the red line and under the blue curve is the probability of type I error (α)
• The area of the horizontally hatched region to the left of the red line and under the green curve is the probability of Type II error (β)

Deciding what significance level to use:

This should be done before analyzing the data -- preferably before gathering the data.
The choice of significance level should be based on the consequences of Type I and Type II errors.

• If the consequences of a type I error are serious or expensive, then a very small significance level is appropriate.

Example 1: Two drugs are being compared for effectiveness in treating the same condition. Drug 1 is very affordable, but Drug 2 is extremely expensive. The null hypothesis is "both drugs are equally effective," and the alternate is "Drug 2 is more effective than Drug 1." In this situation, a Type I error would be deciding that Drug 2 is more effective, when in fact it is no better than Drug 1, but would cost the patient much more money. That would be undesirable from the patient's perspective, so a small significance level is warranted.

• If the consequences of a Type I error are not very serious (and especially if a Type II error has serious consequences), then a larger significance level is appropriate.