Maximum likelihood

Link: Logistic regression

The goal of maximum likelihood §

First of all, it’s a general approach that can be applied to both linear and non-linear. The goal is to find out the best line that fit the data y

Why do we need maximum likelihood in logistic regression? §

Because the least square method in linear regression does not work, as the transformed log involves +infinity and -infinity

The intuition of maximum likelihood §

The idea is to find the constants so that the $p(x)$ is the closest to the observed $p(x_i)$

Understand the intuition of maximum likelihood with math

It’s actually calculating a sum of series of independent events, each event is the observed outcome of each feature. The goal is to find the max p(x) so that it maximize the sum probablity of all events

Assume feature 1 is when $x_1,y=1$, feature 2 is $x_2,y=0$, …

$$p(x)=p(y=1|x_1)\cdot (y=0|x_2)\cdot (y=1|x_3)\cdot ..\cdot (y=0or1|x_i)$$

Maximum likelihood is to get the max of p(x). So, for logistic regression, we substitute the logistic function to $p(x_i)$, and let the computer to find the best constant for us.

See more in details in Likelihood function.