There’s something deeply ingrained in many developers, including myself, that creates a tendency to over-engineer. Maybe it’s how we’re taught or maybe it’s a natural desire to “future proof” our code. Regardless, this tendency is so strong that even being aware of it is not enough to prevent the behavior.

Earlier this year, I was working on a system for helping users filter their emails. While I love Gmail filters, the experience of creating them and debugging them is a bit lacking. I wanted something much simpler for Maleega. I started off with just two very broad options:

• Mark emails from every contact as important
• Mark emails from important contacts as important

Storing and applying this setting isn’t rocket science, but that didn’t stop me from trying to make it so.

In my head were all these grand plans for everything I would build in this new system. I had brainstormed a few dozen and had only selected this as the first “rule” in my rule engine. Because of this, I couldn’t just store the setting! I needed to have a data model that could store the setting for any possible rule I could think of. This would make it much easier to add new rules in the future.

This flexibility promises many benefits, but it doesn’t come free. I wrote hundreds of lines of complex code to handle all my theoretical cases. I ended up spending 2 weeks building this framework. It would have taken a day to build this one feature with its one use case. That’s alright though! I enjoyed every minute of it and I would soon reap the benefits of all this extra work!

Fast forward 7 months.

While I had plenty of brainstormed ideas for rules when I first built my rules engine, I realized that most of them were of questionable value. Instead I spent most of my time working on more important things based on user feedback and my own usage of the product. It took 7 months before I had come up with another rule worth building.

The problem with most email filtering is that the rules are usually generated after an email comes in from a certain person or about a certain subject. However, a lot of important email comes from people we have just met or are contacting us for the first time about buying our product, offering us a job, or just networking in general. There’s a novelty in this email that makes it important, which is why the next filter I implemented was to mark all novelty emails as important.

That alone isn’t so bad and would have been covered easily by my rules engine on the backend. The problem is that this is a very distinct experience from the previous rule on the frontend. The instructions/tutorial/onboarding is going to be different. The warnings for certain actions based on these settings will be different. The interactions available based on these settings will be different.

The extra layer of abstraction I had built to make my life easier was actually going to make things harder when dealing with all these specific cases. While I had enjoyed writing this code, it was abstraction for the sake of abstraction. This was something I had vowed never to do again years ago and here I was repeating the mistake.

What I had built felt like sophisticated engineering, but the unneeded complexity makes things harder to test, harder to track production bugs, and harder to build new things in. This is especially true when doing user research because the point of doing user research is that I’m unsure of what may be needed 6 months down the line.

I honestly can’t fully explain this fallacy. Maybe it’s because 90% of software isn’t rocket science and good products don’t need complex code, but complex code is much more interesting to write. I could have some deep programming need that’s not necessarily being met just by building a great product. Or maybe the habits I had developed in my younger years are still causing me to make certain decisions reflexively.

Regardless, I hope I’ve finally kicked the habit. I deleted my rules engine and rebuilt my old rule and my new rule in less than two days. There’s still some abstraction, but it’s a lot simpler and it provides an actual benefit to the codebase. The abstraction I have now has a purpose beyond being just abstract.