Day 16: Type 1 errors

We did it, we cracked the coin conundrum! We managed the money mystery! We checked the change charade! We … well you get the idea. Last post we (finally) determined if our coin was bias or not. Don’t worry, I won’t spoil it for you if you haven’t read it yet. I actually enjoyed working through a completely made up problem, so if you haven’t read it, you really should. Today we’re going to talk dogs, you’ll see what I mean, so let’s dive in.*

For those of you who are reading daily (thank you), you know that not only did we determine if our value was significant, we also introduced the idea that significance was variable. That is to say that we had a certain amount of variability what we considered significant. Furthermore, without giving away the spoilers to our final solution, we saw that we had the ability to change whether our result was significant or not by how much confidence we wanted to have with our result.

This introduced the idea of confidence. We can never be 100% sure that our observed data is not the result of some fluke. I could flip a coin 10,000 times and get heads every time, but I still would not be able to say with 100% certainty that the coin is bias. However, I could say that it was overwhelmingly likely because there is only roughly 1 in 1.995*10^3010 chance that you would get that result so it is safe to assume that we have a bias coin because the alternative is for practical purposes all but impossible.

Okay, so we can’t be too strict with our acceptable level of confidence, but we cannot be too relaxed either. This leads to two different types of error, today let’s touch on type one error or the dog alarm.

Let’s use an example. You train your dog to bark when someone breaks into the house. They are a good doggo and you do your best to train them. One day the dog goes nuts, but it turns out that it is just the mailman. A few days go by and this happens again, this time when you have guests coming over!

This is the essence of a type 1 error and it is sometimes referred to as the “dog alarm,” which is why I used the dog example. You see type 1 errors when you decrease your acceptable level of certainty and the worst part about errors like this, you don’t know it is an error. Formally speaking, a type 1 error occurs when you wrongfully reject the null hypothesis. A less formal way of thinking about type 1 errors would be false positives.

What’s the opposite of a false positive? Well that would be a false negative also known as a type 2 error, which we will cover tomorrow! I did say today would be short. Don’t worry we will compare the two next time and hopefully it will make more sense.

Until next time, don’t stop learning!

*As usual, I make no claim to the accuracy of this information, some of it might be wrong. I’m learning, which is why I’m writing these posts and if you’re reading this then I am assuming you are trying to learn too. My plea to yu is this, if you see something that is not correct, or if you want to expand on something, do it. Let’s learn together!!