Question

9. let us consider the following validation data set confusion matrix is the result of a logistic regression model which includes if the patient will have a heart attack as a dependent variable which is connected to the range of independent variables. In this model y=1 indicates heart attack and y=0 indicates not having a heart attack. Cutoff value is considered 50 per cent.

1. Calculate sensitivity, specificity and overall error of the model.
2. Considering this confusion matrix do you think shall we make a change in the cut-off value? Justify your answer.

		Predicted class
		1	0
Actual class	1	2700	1000
	0	70	3068

Answer 1

		Predicted Class
		1	0
Actual Class	1	2700	1000
0	70	3068

a.

Sensitivity:

True Positive Rate = (True Positive)/Positive = (predicted as 1 given it was 1)/(all predicted as 1) = 2700/(2700+70)

= 2700/2770 = 0.9749

or Sensitivity = 0.9749

Specificity:

True Negative Rate = (True Negative)/Negative = {predicted as 0 given it was 0)/(all predicted 0) = 3068/(3068+1000) = 3068/4068 = 0.7542

or Specificity = 0.7542

Total Error: (all false predictions)/total = (predicted as 1 but was 0 and predicted as 0 but was 1)/total

= (1000+70)/(2700+70+1000+3068) = 1070/6838 = 0.1565

or Total error = 0.1565

b. The total accuracy for Logistic Regression model is = 1 - 0.1565 = 0.8444

Important:

Here, we are predicting if a patient has heart attack, now try to understand two scenario:

1. Sensitivity : Predicting that patient has heart attack given that he actually had it

2. Specificity: Predicting that patient didnt have heart attack given that he didnt actually had it

So, here in confusion matrix, if we are supposed to increase the Specificity we will have to reduce Sensitivity.

In medical diagnosis, predicting patient has heart attack given that he didnt have it actually is less riskier than predicting patient does not have heart attack when he actually had it

So, we can compromise Specificity given that we have good Sensitivity.

Further, if we increase the cut-off we will be able to increase Specificity but that can lead to reduced Sensitivity which we do not want.

Hence, we will not change cut-off from 0.5 to any other probability.

Please rate my answer and comment for doubt