YAGNI

"Solve only the problems you do have,
not the problems you might have"

‌
"I find that when someone's taking time to do something right in the present, they're a perfectionist with no ability to prioritize, whereas when someone took time to do something right in the past, they're a master artisan of great foresight."
Munroe, R. (2011) "The General Problem", xkcd.com/974

In the broader engineering discipline, it is often very easy to fall into the trap of over-engineering a solution. Here are two primary driving reasons I've identified, and hopefully this helps you avoid this trap.

The Desire For a Perfectly Optimised, Ultimately Efficient Solution

This leads to wasting a lot of time when all of that optimisation and efficiency is either completely unneeded for a rarely-run piece of software, or is completely eclipsed by some other, much larger, cost.

When all that's needed is something quick that will only get occasionally used, it doesn't really matter how long it takes to run_{unless it's dealing with some kind of big data where a single run of the software can take hours, ain't nobody got time for that}. If there is some task that needs to be run once a week and it take fifteen minutes to complete, then if you were to spend about four hours trying to shave off five minutes you wouldn't break even until next year. One year to gain the benefit of a 33.333% improvement that only took about half a days work to do. Maybe it would have been better to just deal with those five minutes.

This is a very useful chart for quickly estimating the usefulness of spending time on workflow improvements.

‌
"Don't forget the time you spend finding the chart to look up what you save. And the time spent reading this reminder about the time spent. And the time trying to figure out if either of those actually make sense. Remember, every second counts toward your life total, including these right now."
Munroe, R. (2013) "Is It Worth the Time?", xkcd.com/1205

Given similar timeframes, equally unoptimised subsystems can be optimised by comparable ratios not comparable amounts

​When there's a much larger aspect of the system affecting the time, then any savings in one area might be completely unnoticeable. For example: if you are working on a web application, optimising your algorithm that takes a hundred milliseconds is not going to be worth it if the page takes five seconds to load. When optimising anything it is important to focus on what is actually taking the longest not just what is most interesting to optimise. The only way to actually do optimisation properly is with profiling; find what is taking up the most time / memory / bandwidth / etc. and figure out how to reduce it.

Generally speaking: Given similar timeframes, equally unoptimised subsystems can be optimised by comparable ratios not comparable amounts. This means that you can spend a week getting a 20% improvement in your 0.1s algorithm or spend a week getting a 20% improvement in your 5s page load time. Which sounds like a better use of your time?

The Desire For a Perfectly Generalised Solution

Another very common reason for over-engineering is the desire to craft the most perfectly generalised solution ever seen. A single tool that solve any problem, the last system you'll ever need.

As anyone with even the most cursory experience of engineering will know this is not possible. The old saying "A jack of all trades, master of none." applies here as well; The only way to make a tool or system that can handle a wide variety of situations is to compromise on its effectiveness and efficiency in any given situation. If XYZ needs to be done then a tool should be made that does XYZ as best as possible in the given timeframe. That is all that can be done, there's no point in worrying about what problems may be encountered later on, they don't exist now and may never exist. Who knows what could have been done with the time saved by simply doing what needs to be done in the moment_{see first image} rather than wasting time trying to solve hypothetical scenarios that may never materialise.

This is not to say that engineers should be completely myopic and ignore future problems or not allow for future improvement but just to make sure that we aren't losing sight of what's important: the actual problem in front of us.

And if you don't want to listen to me, then listen to Donald Knuth:

"Premature optimization is the root of all evil"