One of the truths I’ve constantly re-arrived at over the past several years is that optimization is boring. It’s a bit of a shame, since I tailored most of my senior engineering courses toward optimization. But although I understand many of the numerical techniques and general engineering principles, optimization itself remains super boring.


Mostly because it’s the part of engineering you can commoditize. When a problem fits neatly into the category of optimization, you can confidently declare: “No problem, throw two smart guys at it for a week.” Or worse, “we’ll just lock a bunch of machine learning kids in the basement until they make it better.”

It’s a bit harsh for me to claim that doing optimization is a commodity. Of course, engineers familiar with the space will be able to do a much more accurate and faster job of it. But at the end of the day, chances are you can just buy your way out of the problem one way or another. (See: HipHop compiler). Other examples include “make this faster”, “make this smaller”, “make this use less energy”. Optimization results tend to take the form of diminishing returns, so you can usually get a lot out of the first bit of work done on it.

Problems that can’t be solved with optimization tend to excite me: my brain thinks they’re way cooler. This is when you sort of get down to the raw conceptual design stuff, or when you start re-thinking the whole approach. Examples include: “make this fly”, “make it never fail”, and “I want to be able to do my e-mail on it”.

If you wanted to twist language or be particularly pedantic, you could claim that all design is just optimizing a really fancy function, but I don’t make friends with those people.

What’s interesting is that a lot of engineers actually prefer optimization problems, while only a few seem to like to deal with more poorly-defined problem spaces.

This is a shame, because I think optimization vs holistic design sort of aligns with different types of innovation. I tend to write-off or view optimization work as sort of free, or guaranteed. Like, we’ll see predictable growth in numbers of processor cores by optimizing the heck out of what we have. (Albeit at some point we’ll have to re-think everything, but for now, let’s say it’s “just” optimization.)

Many of Alan Kay’s themes touch on this as well.

Some problems, I think society misunderstands as being optimization problems which most definitely are not. Solving cancer, for example, is actually a very wide open problem. We’re not in a position to say “we can probably just crack this open by locking a bunch of standard-issue biologists in a basement for a couple years.”

Takeaways: society misunderstands that there are at least two fundamentally different types of engineering; similarly misunderstand there are different types of problems; poor predictability of outcomes.

Not yet discussed: operations research/taylorism/MBAs/McKinsey.


Now read this

Programming without Text

I am disappointed with the current state of programming today, largely because it used to be one of my favourite pasttimes. It’s not the logic, or the projects, or the prevailing attitudes, or even the languages employed, per se, which... Continue →