The hypothesis in statistics

As promised today we’re talking statistics for experiments! It’s more interesting than it sounds… okay it’s exactly as exciting as it sounds. Depending on who you are that’s a lot or not at all. No matter where on the spectrum you fall, knowing how it works is useful. So we’re starting at the beginning and discussing what a hypothesis is and how we test it.

Semi-regular introduction to my blog. I’m a third year PhD candidate in neuroengineering and this is my daily blog about the journey.We’re in year two of this project so there’s a whole lot of posts, some talking about experiments, a lot about scientific writing, and a few on hobbies or why it’s important to have a hobby/way to relax. This was also a sort of expanding notebook for my classes, but since I haven’t done that in awhile I figure we can give that a shot again and try to teach the things I’m learning.

In statistics a hypothesis is something we can test. For example, we can determine if a coin is fair by flipping it several times (the more the better) and doing a little bit of math to determine if our result was statistically significant or not. We won’t be covering the math behind that just yet, but we can talk about some of the terms that we will be using as we dive into this subject.

Statistical significance is up first and that just means we can reject the null hypothesis. The null hypothesis is the “status quo,” using our coin example, the null hypothesis is that the coin is fair. The alternative hypothesis going to be the “more interesting” outcome, in our example the alternative hypothesis is the coin is not fair. Typically if we’re bothering to do the statistics we want the more interesting outcome (the alternative hypothesis) because it tells us something new. In research it’s a way to show we’ve found something more interesting than an unfair coin, but that’s a simple example so we’ll continue using it.

In a perfect world if we flipped a coin you would get 50% heads and 50% tails, but if you flipped a coin and found 48% heads and 52% tails you couldn’t say right away that the coin was not fair. There is a certain amount of variability that is “acceptable” because you can only approach the ideal 50/50 outcome as your flips approach infinity (if the coin is fair). Since you can’t flip a coin an infinite amount of times, we need to decide the range of values that we call “a fair coin.”

There is a “region” of values that we could get and would still mean we could not reject the null hypothesis (the coin is fair). Quick sidenote, we never “accept” the null hypothesis, but that’s just the convention used, not because accepting it is somehow wrong. If the value we got fell outside of that region of acceptable values we would be in something called the rejection (or critical) region. This is where the “interesting” stuff lives.

The rejection region are the values we would get that give us the statistical power to reject the null hypothesis. We can never say with 100% certainty that the coin is not fair. If we flipped the coin 10,000,000,000,000 times and got only heads there is a non-zero chance (granted it’s vanishingly small at that number of flips) you would get that outcome with a fair coin so instead we instead select a “confidence interval.” Which is just how confident we are that our conclusion is correct, if you got all heads for example and it turns out the coin was fair (maybe because you only flipped it 4 times and not 10 million times) and said the coin wasn’t fair even though it was, then you made an error. There are two types of errors you can have and we’ll cover that next time.

The confidence interval is something you decide before you run the test and it depends on what you’re testing. If we’re testing a coin for fun we may say that we want 95% confidence in our outcome, that means there is a 5% chance you made an error and that is usually good for whatever we do so you’ll see that selected most of the time. However, in some cases you set the confidence interval to 99% , 99.9% or even 99.9999%. Basically you can make the test as strict as you want, but there are downsides to this (you can make a certain type of error).

Tomorrow we’ll talk about the types of errors you can make and then we can dive into how exactly we would test a hypothesis (probably using the coin example). Side note, most of this I covered way back in my stochastic processes notes! That would be year one starting at day seven or eight… I believe. That was back when I counted the days, but the titles for my posts got shorter as the days progressed into years and now I just title my posts with the topic and keep a running count behind the scenes so I know when to switch to year three. In other words, if you’re interested in this stuff or need to learn it right now and don’t feel like waiting for a revisit you can look at those posts.