Question

Logistic Regression and the Cross Entropy Cost. What are they similar in? Give me please some explanation or examples, that shows their similarity

Answer 1

Logistic regression:

Logistic regression is just a generalized linear model, so there is a linear regression lurking in here whose errors are supposed to be normally distributed, and from which you can get confidence intervals in the usual way.

It can also be defined as Logistic regression is a classification algorithm used to assign observations to a discrete set of classes. Unlike linear regression which outputs continuous number values, logistic regression transforms its output using the logistic sigmoid function to return a probability value which can then be mapped to two or more discrete classes.

Cross-Entropy :

Cross-entropy cost measures the performance of a classification model whose output is a probability value between 0 and 1. Cross-entropy loss increases as the predicted probability diverge from the actual label

For logistic regression, it’s a bit more complicated. The cost function we use is called Cross-Entropy, also known as Log Loss. Cross-entropy loss is split into two separate cost functions when dealing with a binary classification

The cross-entropy between two probability distributions p and q over the same underlying set of events measures the average number of bits needed to identify an event drawn from the set if a coding scheme used for the set is optimized for an estimated probability distribution q, rather than the true distribution p.

Cross-entropy loss can be divided into two separate cost functions: one for y=1 and one for y=0

for example:

def costFunction(features, labels, weights): # define a cost function

    observations = len(labels)

predictions = predict(features, weights)

class1Cost = -labels*np.log(predictions)    # It will take error when table is 1

class2Cost = (1-labels)*np.log(1-predictions) # It will take error when label is 0

cost = class1Cost + class2Cost # It will find the sum of two costs

cost = cost.sum() / observations # it will give average cost

return cost

The above code will return the cost of the feature.