Bayesian Updating

 Home Up We use Bayesian Updating every day without knowing it.

Engineers see references to Bayesian Statistics everywhere.  Here is a ten-minute overview of the fundamental idea.  The concept is easy - we do it every day .  But there's a catch:  Sometimes the arithmetic can be nasty.

 Situation # 1:Given: The median height of an average American Male is 5'10".  (I don't know if this is accurate; that's not the point.)  You are on a business trip and are scheduled to spend the night at a nice hotel downtown. Wanted:  Estimate the probability that the first male guest you see in the hotel lobby is over 5'10". Solution:  50%  (Well, that's certainly self-evident.)

 Situation # 2:On your way to the hotel you discover that the National Basketball Player's Association is having a convention in town and the official hotel is the one where you are to stay, and furthermore, they have reserved all the rooms but yours. Wanted:  Now, estimate the probability that the first male guest you see in the hotel lobby is over 5'10". Solution:  More than 50%   Maybe even much more, and that's obvious too.

So what?  You just applied Bayesian updating to improve (update anyway) your prior probability estimate to produce a posterior probability estimate.   Bayes's Theorem supplies the arithmetic to quantify this qualitative idea.

How does Bayesian Updating Work?

The idea is simple even if the resulting arithmetic sometimes can be scary.  It's based on joint probability - the probability of two things happening together.

Consider two events, A and B.   They can be anything.  A could be the event, Man over 5'10" for example, and B could be Plays for the NBA  The whole idea is to consider the joint probability of both events, A and B, happening together (a man over 5'10" who plays in the NBA), and then perform some arithmetic on that relationship to provide a updated (posterior) estimate of a prior probability statement.

 IF: Prob(A|B) is the conditional probability of A, given B, and Prob(A and B) is their joint probability Some definitions THEN: Prob(A|B) x Prob(B) = Prob(A and B) = Prob(B|A) x Prob(A) By the definition of conditional probability So that: Prob(A|B) = Prob(A and B) / Prob(B) By algebraic manipulation.
 Numerical Example:Let Prob(A) = 0.5 Let Prob(B) = 0.000001 Let Prob(B|A) = 0.00000198 Probability of seeing a man over 5'10" Probability of playing for the NBA Probability of playing for the NBA, given that you're over 5'10"

Wanted: an updated (a posteriori) probability estimate that the first guest
seen will be over 5'10", i.e: Prob(A|B)

 Prob(A|B) = Prob(A and B) / Prob(B),   =Prob(A|B) = Prob(B|A) x Prob(A) / Prob(B) = Prob(A|B) = 0.00000198 x 0.5 / 0.000001 = Bayesian updating begins with the conditional probability, Prob(B|A) as given, when what is desired is the other conditional orobability, Prob(A|B) Prob(A|B) = 0.00000099 / 0.000001 = 0.99 Updated probability of seeing a man over   5'10" given that he plays for the NBA

A Venn Diagram shows that once the universe has been narrowed to NBA players (crosshatched area), the fraction of that universe that is taller than 5'10" is very large.

(In fairness, a warning is in order:  This example is very simple, and real problems seldom provide the required conditional probabilities - they must be inferred from the marginals - and real problems are seldom binary - black or white - but consist of many possible outcomes, with only one of primary interest.)

So What?

Suppose Event A were your analytical predictions of some physical phenomenon, and Event B the ex post facto physical measurements (complete with their uncertainty).  Bayesian updating could be used to improve your analytics in light of the new experimental information.  Note that this is NOT equivalent to "dialing in a correction" between what was predicted and what was measured.

 [ HOME ] [ Feedback ] Mail to Charles.Annis@StatisticalEngineering.com Copyright � 1998-2008 Charles Annis, P.E. Last modified: June 08, 2014