Finding answers

Okay, today is basically the final day I have to get the answers I want. While I’ll have time to work on this during the week (with a little luck that is) I don’t want to rely on that. Instead today I’m going to make the push to find out if my “super secret technique” (SST) actually works. I have the data, I have most of the code written, and I have some new equipment to help me out (after putting that mechanical engineering degree to good use and reinforcing my desk a bit). So the rest is me and how much I can do.

Since yesterday I’ve managed to segment the data to the conditions I want. For those who don’t read my blog often or if you’re just confused by the arrangement of the words in that first sentence I’ll explain. When we collect data it’s a stream of numbers, 64 channels sampled (measured) every 1 ms so 1000 samples x 64 channels (that’s one channel for each EEG sensor we use) over the course of about two to three hours. That’s a lot of data. Thankfully the software we use lets us add markers for the important bits or we couldn’t do anything with all that data since it loses context the second we collect it.

Each marker or flag as I like to call them has a note that we can add. For example we can have an “eyes closed condition” and we may mark it in or data with a flag called “eyes closed” now alone that would be useless, but the flag also contains the exact instance when the flag was added, so if we added it 10 seconds after we hit the start recording button, we would have a flag called “eyes closed” at latency 10,000 (assuming 1000 hz or 1000 samples per second sample rate, we can speed that up or slow it down if need).

Those flags are important because we go in and grab the bit that we want. Most of the time we use two flags a start and end flag and even if they use the same name it’s obvious which is which because the second is obviously the stop. Sometimes we can add “double flags” which gives us 3, but that’s an easy fix too and I’ll often mark that there was a double flag and which one to pay attention to when we’re recording data or I’ll make a note in my experiment notebook (because I have one).

Since we also have EMG, which is its own stream of data (as in collected in different software with no way to hit the start button at the exact same time between the two), we use a second trigger (or flag) that synchronizes the systems. We do this mostly because we can’t start them in sync, but even if we could there’s no guarantee that they will stay in sync, so we add regular flags to the data to make sure we have points were we can realign. The other problem is sampling rate, or the rate at which the systems take measurements. The EEG and EMG sample at different rates, so we need to either upsample (add samples to fill in the “gaps”) or downsample (remove samples to match the data being sampled at a lower rate).

There are a few other technical details that make this a pain, but I’ve managed to come up with some pretty good code to do that for me and it appears, from what I’ve seen anyway, to be reliable enough to do the job blindly. So say that we got through all this, well then we’re done and can now move onto the next step.

Oh all this assumes we’ve already preprocessed our data and made sure that the obvious artifacts and what not are filtered out of the data. But we’re well past that, so let’s talk about where we are now.

Segmenting the data!

We had a whole lot of conditions and 10 datasets, so we need to get the bits of data from that stream that we want (as I explained above, that’s not a trivial task). We’ll we’re officially past that step as of yesterday and part of today. Now that I have the data roughly segmented, I need to do one more “fine” segment and just look at the parts of the data where the person did the task, this is where the other streams of data come in. EMG in this case since people were doing different tasks that involved movement, we can use the EMG to find the points where the person started and stopped the task.

Once I manage that, I can do the (comparatively) easy part and compare the differences between conditions, get answers, etc. But since I’m still working on it I can’t say one way or the other what we have yet. And since I’m still working on it, I really need to just end the post here and get back to work now that the data I had saving is done, saving takes forever since we’re talking about ~20 GB of data. But the computer tells me it’s finished so back to work I go.

Vague answers coming… soon I hope!