Day 20: Independent Events

By: xkcd

Because we introduced the central limit theorem last post, it’s time to introduce another important concept. The idea of independent events, while this may seem intuitive, it is one of the assumptions we make in parametric statistics, another concept we will define, but for now let’s jump into independence.*

While we haven’t actually defined it explicitly, when we use parametric statistics (I promise we will define this soon, maybe next post), we make two very important assumptions. The first is that our data has a normal distribution. Yesterday we talked about the central limit theorem and how we can find the normal distribution just about everywhere, which in and of itself is amazing that nature would take that approach.

Well today we are going to talk about the second assumption, the assumption each point of our data is independent from every other point. Independence is an interesting concept because while it may seem simple, there are pitfalls to assuming independence.

Since the genie is already out of the bottle so to speak and we’ve started cracking into some of the math behind these topics, we can just go ahead and define statistical independence using some math and then dive into the implications behind it. When our data has independence, we say that they satisfy the following

P(A B) = P(A)P(B)

What this means is that if we have two events, event A and event B, the probability that event A will happen given event B is equal to the two individual probabilities. We saw this type of probability in our coin flipping experiment. That is where we saw that the probability of a heads P(H) = 1/2. We also saw that the probability of getting two heads in two flips was P(HH) = 1/2*1/2. This is because event A (we get a heads on our first flip) has no impact on event B (we get a heads on our second flip).

This was a valid assumption because we did not change anything about the way we flipped the coin when we got a heads compared to when we got a tails. This is not to say that events that are not independent are created by a bias, just that we can make the events dependent by changing how we flip a coin after we get a heads or a tails.

If that was confusing, then let’s look at a good example of dependentance that may be less confusing is a ball picking experiment. Let’s look at one such example:

Say we have 5 balls in a box, 3 white and 2 black. You draw 2 balls WITHOUT replacing the first draw.

What are the odds that we will draw a black ball on the first selection?

P(B) = 2/5 or we have a 40% chance of drawing a black ball the first pick.

Here is where the dependence comes into play, say we do draw a black ball the first pick. Let’s look at what are the odds that we will draw a black ball on the second selection.

P(B) = 1/4 meaning we have a 25% chance of drawing a black ball on our second pick GIVEN we drew a black ball the first time.

If these were independent events the probability of drawing a black ball twice would just be P(B B) = P(B) P(B) = (2/5)(2/5) = 0.16 or 16% and is not what we see here. The odds of drawing a black ball the second time given that we drew a black ball the first time is actually P(B B) = (2/5)(1/4) = 0.10 and NOT equal to 0.16, thus we have a situation where we do not have independence.

Now, you may be inclined to think that the only time event A and event B could be linked is when we draw from a pool and do not replace the removed value (using our example, removing a ball, then removing a second ball). However, this is why this topic is such an important concept to cover because this isn’t the case. We see dependent events all the time in other context, sometimes in situations where it isn’t always as apparent as drawing balls from a box.

One good example of this is working with EEG data. When we collect the data we cannot assume independence because the data collected is not independent. Why is this? Well for one thing we have to deal with a phenomenon called volume conduction. When we measure EEG signals, we are measuring the changes of electrical potential in the brain, whole groups of neurons, hundreds of thousands at a time. This means several things:

• We cannot resolve single neuron activity using this method
• Because of the way these networks operate, the behavior of one neuron influences the neighboring neurons
• Because the body conducts electricity, the recordings at one sensor will “leak” to another sensor.

Let’s do a quick thought experiment to highlight what volume conduction looks like. If we set several microphones around the outside of a room, and have a group of people talk inside the room, you will see multiple microphones with the same voice at different amplitudes based on how far away the microphone was from the person speaking. This is the essence of volume conduction.

To our point, this means that the data collected is not independent for several reasons and none of them are due to removing something from the system. There are other examples of this concept, but I think that is probably one of the best we can discuss now.

Okay, so we’ve talked independence, one might even say we’ve had our independence day (cue alien invasion). We’ve also looked at some examples of independent events and data that was dependent! It seems like we’ve covered quite a bit for such a “simple” looking topic, right? So what’s the topic for tomorrow? Well that … depends! Okay, bad joke, but really I think we should talk about parametric statistics and define it explicitly.

Until next time, don’t stop learning!

*As usual, I make no claim to the accuracy of this information, some of it might be wrong. I’m learning, which is why I’m writing these posts and if you’re reading this then I am assuming you are trying to learn too. My plea to you is this, if you see something that is not correct, or if you want to expand on something, do it. Let’s learn together!!