Question

A student stated: “I fail to see why the response function needs to be constrained between 0 and 1 when the response variable is binary and has a Bernoulli distribution. The fit to 0, 1 data will take care of this problem for any response function.” Comment.

Answer 1

The definition of Bernoulli Distribution:-

Suppose you perform an experiment with two possible outcomes: either success or failure. Success happens with probability p, while failure happens with probability (1-p). A random variable that takes value 1 in case of success and 0 in case of failure is called a Bernoulli random variable (alternatively, it is said to have a Bernoulli distribution).

Now that the response variable or the support space is Binary means that it takes only two values with their respective probabilities. But why 0 and 1?

Because it comes from the concept of the Indicator function.

Definition of Indicator Function:-

The indicator function of an event is a random variable that takes value 1 when the event happens and value 0 when the event does not happen. Indicator functions are often used in probability theory to simplify notation and to prove theorems.

Any Indicator function is always a Bernoulli Random Variable.