Probability and Likelihood

Probability and Likelihood – looking through the other end of a telescope.

 

Probability:

Figure 1 – Probability density showing the probability that an observation will fall between two given values of \(x\). The red ordinate shows the (relative) probability of a single observation.

 


  • The probability density function, pdf, tells how probable a value of \(x\) is, given the model parameters, \(\theta\). For example, for a Normal density: \(\theta= (\mu, \sigma)^T\)\[f(x | \mu, \sigma) = \frac{1}{\sqrt{2 \pi}} e^{-\frac12 \left(\frac{x-\mu}{\sigma} \right)^2 }\]
  • Probabilities integrate to one.

 

Likelihood:

Figure 2 – Likelihood function showing the likelihood of the parameter values, \(\theta= (\mu, \sigma)^T\), given the observations,.

\[f(\mu, \sigma | x) = \frac{1}{\sqrt{2 \pi}} e^{-\frac12 \left(\frac{x-\mu}{\sigma} \right)^2 }\](Notice that the likelihood and probability formulations are identical; the only difference is in what is known: probability (i.e. the model parameters) or observed results, \(x\).)

  • The likelihood function tells how likely the model parameters, \(\theta= (\mu, \sigma)^T\) are, given the observed value of \(x\).
  • Likelihood can be defined for both censored and uncensored data.(Uncensored example shown here.)
  • Likelihoods do not necessarily integrate to one.