Two populations in statistics

As a mad scientist, or maybe just a grumpy scientist, you want to test a new mind control technique! To do this you decide that you want to test this works by having people select one of two objects set in front of them. *Insert evil laugh* Using your mind control technique you want your unwitting participants pick the object on the left. You don’t get 100% success, but suspect it’s working, how do we know for sure?

Mad scientist you say? No, not me… I’m just a tired scientist. For those of you new here, I’m a third year PhD candidate in neuroengineering definitely not attempting mind control, or am I?! I have a BS and MS in mechanical engineering and I’m researching how neural communication changes after spinal cord injury. In an effort to distract encourage myself to work on the big assignment I have due this weekend I’m covering a broad overview of what I’ve been working on, so let’s get into it.

Being a mad scientist must be difficult, mostly because getting IRB approval (the board whose job it is to make sure you’re doing ethical research) to do experiments is difficult as it is, getting funding to do some evil experiments must be impossible. Thankfully our mad scientist has funding and wants to test mind control. The question is how do we know its working and the answer is, as with most things, statistics.

Unlike yesterday’s example, we don’t know anything about the population we’re sampling from. Even though you have a 50/50 chance of selecting one object over the other, people could prefer one over the other, it may be tied to left or right handedness, there could be a lot of factors. So what do we do to mitigate these preferences? Ideally we would test the sample group for both experiments!

The first experiment would measure preference of the left or right object and we would then do our thing and calculate the mean and standard deviation of our group. Ideally this would be a larger group, but it doesn’t always need to be, 30 or more people would be great. This is especially the case if this mind control device doesn’t appear to be working 100% of the time.

We would then perform a second experiment with the same group and use our mind control device! There are some caveats here. A better experiment would be to split the group into two and one group would do the control test first the other would do the mind control test first then flip for the second experiment. This is called counterbalancing and attempts to try to control for preference changing for the second experiment (IE maybe the participant for experiment 1 selects the left object, but decides to take the right object for experiment 2 because they took the left in the first).

There are other ways to design the experiment of course, we could control for handedness by selecting only right or left handed participants for example. We could have multiple objects, not just two, we may have multiple buttons or other things instead of objects, etc. This is why experimental design is so important, because it dictates what we can and cannot conclude based on what we find.

In the end we can test our two populations using something called a t-test and we’ll go over what that is tomorrow, but for today I will stop here because I feel like this is all a lot. Tomorrow we’ll make up some data… err test our mind control device and find out if our mad scientist has been successful or not.