Day 37: Bayes’ Theorem

We’re talking Bayes for days.

Can you guess who came up with Bayes rule? Early 1700’s smart guy Thomas Bayes (among other things) said we can use prior knowledge of an event to predict the outcome of some related event. Remember our coin flipping experiment? I don’t blame you if you forgot already, that was quite a few posts back! We said that the probability of getting a heads or a tails was independent of the outcome of the previous flip. What if we could predict the outcome of the next flip more accurately based on the previous one? That is the idea behind Bayes’ rule.

Before we get into the math let’s look at a general example. Let’s say the life of a cell phone battery (since we just talked cell batteries!) decreases in some way as age of the cell phone increases. Well, we could better predict how long the battery of the cell phone would last if we knew the age of the phone before making our prediction.

For those of you who are more math inclined Bayes’ rule is as follows:

For those of you confused, P(A|B) is read as, “the probability of event A happening if event B occurs.” When we write it in reverse, P(B|A) we just say that is the probability of event B given event A (usually just read as B given A). Going back to our cell phone example, we are wondering the probability that our battery life has decreased given a certain age of phone. We already said that as cell phone age increases, battery life decreases, so by using the age of the phone we can better predict the battery life.

Let’s look at another simple (but not cell phone related) example. Let’s say that you are developing a test for cancer that is 99% sensitive and specific (meaning that the test will produce 99% true positives for people with cancer and 99% true negatives for people without cancer). Furthermore, let’s say that 0.2% of the sample population are people who actually have cancer. Now we want to know, what is the likelihood that a person selected randomly will test positive. Well let’s do the math!

This means that even if a person tests positive, it is more likely a false positive. That is because the number of people with no cancer is so much higher than the number of people with cancer. For example if our sample population was 1000 people, we would expect to see ~2 people with cancer and 998 without. From the 998 people without cancer, 0.01998 = 9.8 or roughly 10 false positives. From the ~2 people with cancer, 0.992 = 1.98 or roughly 2 true positives. So out of 12 positive results, only 2 are true positives.

One important thing to remember, these are not independent events, going back to our first example, our cell phone battery age and the battery life are linked; one affects the other. We cannot use this when the two events are independent of each other. For example we cannot better predict a fair coin flip if we know what the previous flip result was. Or to use our phone example, we cannot use cell phone color to predict battery life, they are two independent variables and one does not impact the other. A subtle, but important thing to remember when you try to use Bayes’ rule for things.

Well this seems like a good point to call it for the post. Next we can get into the Poisson distribution (it’s french, so not pronounced like poison). Then bring it all home and get into how we can use them in combination. In any case, we’ve now introduced Bayes and hopefully helped demystify it a bit.

Until next time, don’t stop learning!

*My dear readers, please remember that I make no claim to the accuracy of this information; some of it might be wrong. I’m learning, which is why I’m writing these posts and if you’re reading this then I am assuming you are trying to learn too. My plea to you is this, if you see something that is not correct, or if you want to expand on something, do it. Let’s learn together!!