Known and unknown unknowns

Jeff Atwood has an article revealing some things that I’ve learned about software and business in general: the dangerous belief in certainty.

Software is unique in that solved problems are easy to build on. Once a problem is considered “solved”, the solution can be exploited with little or no cost. (When’s the last time you worried about the TCP/IP stack when building a web app?)

Therefore, if a software developer is asked to create something new, she is in all likelihood solving a problem that has not been solved before. After all, if there’s already a solution, why write new code?

Because it’s a new problem, we don’t have much real certainty about the difficulty of the problem, and any predictions we make are speculative. Jeff says: […] control is ultimately illusory on software development projects. If you want to move your project forward, the only reliable way to do that is to cultivate a deep sense of software craftsmanship and professionalism around it.

This is an empirical and experimental view, and it is the right one, IMHO. Predictions about where a software project will go, and how long it will take, are educated guesses. Sometimes we recognize what we don’t know; usually, we don’t.

This needn’t be a showstopper; in fact, it’s the thing that thrills me about software. The danger is in placing too much weight on your predictions.

I’ve found this very hard to communicate to clients. They want two contradictory things: on the one hand, they want their feedback to be respected and built into the project. On the other hand, they want certainty about schedules and costs.

Both are perfectly reasonable desires. But they are at cross-purposes — each piece of feedback generates a new feedback cycle. The list of to-do’s grows exponentially with time.

I think continuous feedback is a great way to make really robust, personable, organic software. It gives businesses real proprietary advantages: the software reflects the uniqueness of the organization.

But it comes at the expense of certainty. Recognizing that we don’t know what we don’t know — and that that’s OK — is a leap that’s hard for many to take.