The value of planning your code

It’s no secret I need to write code, a lot of code. A mind numbingly large amount of code. Researching in a brain-machine interface lab is more about programming than it is about scifi brain hacking or anything the latest techno-thriller movies would have you believe. Anyone can do it, but it takes a certain amount of practice and a whole heck of a lot of time. Over time your skills develop and things get to be easier, but it’s not a linear progression (is anything really?) so things can be frustrating. These days planning saves me tons of time and effort, so today I want to pass that bit of knowledge on to you.

In its most basic form, a computer program is just a to-do list for your computer. It’s a set of instructions that your computer follows and the result is whatever you want to get out of this to-do list. Imagine if you wanted to bake a cake, you would follow a recipe, but if there was a typo in the recipe or you skip a step… well that could be disastrous or you may not even notice. The same thing can happen when you code, you can accidentally add an extra zero, miss a decimal, or multiply when you really want to divide. You can skip over a step by mistake or you can tell the computer to keep doing the step until a certain condition is met and that condition never actually gets met (the dreaded infinite loop).

Really I’m a glorified instruction manual writer. The thing that seperates a good writer from a bad writer is time spent on the writing and more importantly how clever you can be with the instructions. That’s where having a plan comes into play. I’ve written a whole series on MATLAB for beginners (here), but I’ve also passed tips I’ve learned along the way (like how to make your code easy to change). So when I say that planning your program is the best advice I can give a person, I mean it.

Almost a full year ago I wrote the longest code I’ve ever written for a single project. It was over 9,000 lines of code. It did its job, it did it well, and truthfully I still have that code and refer to it from time to time. It was a monumental effort and it took several days of tedious edits and trials. The data went in and it returned a nicely organized structure (think folders storing data, they are labeled and are very organized). I’m proud of that code and the amount of time I took to get it working correctly. But the sad truth is it’s awful code. Don’t get me wrong, I’m not mad. I just improved and that is the whole point of doing something, it was the best I had at the time and while it’s cool, it just wasn’t very good.

Sometimes I forget how far I’ve come. Yesterday (here) I sat down and shared something somewhat mundane, but I was also being — perhaps overly — honest. I had tried to do something I didn’t think I would be able to do and I said yesterday that I was going to do it all that day. Since we’re being honest, I thought for sure I would fail that one and I wanted to give up several times yesterday in favor of putting it off. The thing was, I needed this code to do what my >9,000 lines of code did, PLUS more. I didn’t have the energy or the desire to do all that again. I am still traumatized from the effort I put into writing that old code.

But that was over a year ago, I’ve gotten better at doing this stuff and it takes days like yesterday for me to be reminded of this growth.

In yesterday’s post I sat down and gave a rough overview of what I needed this code to do. Granted I didn’t get too into details (because I really cannot), but yesterday’s post really let me come up with a plan of attack (just like the post from the day before). With a clear plan and a rough outline I sat down and got to work. I don’t want to spoil it, but I did it. I did all of it, it works, it’s awesome, and it wasn’t as hard as I thought it would be.

The section that does the heavy lifting ended up being 80 lines of code exactly. The entire code start to finish is 350 lines exactly which plugging in the numbers is ~24.7x shorter than that infamous code I wrote a year ago. If I removed all the comments and strategic spacing (because separated code is easier for me to follow), which the comments alone make up >60% of that code and we’re left with something even shorter. If I wanted to condense it as small as I could make it, I didn’t check, but I would bet you’re looking at less than 50 lines of (very ugly, very hard to follow) code. If I tossed two of those steps into functions, it would be less than a handful of lines (10 on the high end), but that would be cheating since the other lines of code would just be hidden in a function.

All of that thanks to practice and planning. Since practice takes a lot longer than planning, my advice to anyone just starting out is to plan your code. Figure out what you want it to do, start and finish, then fill in the steps in between. My main-PI recommends a flowchart, which is how I used to work when I was writing code, but I’m more of a visual person and I like to imagine my data as something physical I can cut into or section different ways, which is probably why I hate working in higher dimensions, but if you haven’t and are interested in how I think about my data, this post covers it.

Thinking of my data as a physical object has probably been the biggest help for me and what I do. I’m not sure if it’s a good habit or one that everyone should imitate, but it’s the best thing I’ve found for me and I’ve been writing code pretty much daily for the past three years or so now. The point here is just to find what works best for you, but more importantly, plan your code before you start writing it. If I hadn’t I could’ve easily ended up with another 9,000 line mess of code to do something I figured out how to accomplish using a small fraction of that.

I’m incredibly excited to be finished with this project. Between COVID and my own lack of experience, it’s taken far longer to do this than we anticipated. The data are super hard to work with too, so maybe I can cut myself a little slack, but really all this extra time has given me a chance to really grow and figure out how to do the best job I can. If I had finished this project at the beginning of the year like we had originally planned, I wouldn’t have been able to do what I did yesterday and it will be worth it once it’s done and I can show off all the work I’ve put into this (and it was a lot!).

I’m rarely proud of the work I do. I’ve shared I think exactly one example of something I’ve done recently that I was truly proud of and that was my COVID model video. This will be something that I can be proud of, but also something I can share, that others could potentially use for themselves, and will probably benefit the rest of the lab should they do a similar analysis (which I’m guessing will become a regular occurance now that I’ve succeeded).

I’m also excited to share this work here with everyone. I’ll probably just give up my anonymity completely at that point, but it could be worth it. While I’m still not a huge fan of this experiment, I am grateful that I got to learn so much from it. Maybe I’ll make a stripped down version of what I’m doing to show here, but in any case, you’ll get to see the full thing once we publish, which will hopefully be soon! So something to look forward to.

Maybe I’ll dive in-depth on how I like to plan with an example tomorrow… who knows, but we’ll see tomorrow.