Software development is notoriously difficult to estimate. For this reason, I know few people who take the Waterfall process seriously. There are many factors that create a level of unpredictability. One is that there are usually pieces that need to be built that a developer has never built before. The lack of domain experience will result in something being missed and all the planning in the world won’t catch everything. Is it worthwhile to restrict ourselves to only building things we’ve built before? Probably not.

Another factor is that aspects of a software product are going to change as more information is discovered about how users interact with the product. These changes are often unpredictable as well. Would you enjoy working with developers who refuse to let you change requirements because it would throw off their estimates? Probably not.

The end goal with software is to have something that people find useful and valuable. Discovering what this is takes time and leads us down unfamiliar paths. This makes dealing with unknowns a fact of life when building software.

That doesn’t mean we can’t manage our time effectively.

I’ve used a number of tactics to do so in the past. The most effective is time boxing. The idea is to pick a set amount of time, e.g. 2 hours, 1 day, or whatever feels reasonable, to do some discovery. At the end of the timebox, you sync back with your team and discuss the results. The discovery can be pure research, but often times it involves some prototyping. It’s hard to know what’s really involved in building a piece of software unless some time is spent actually building it.

Obviously an unknown task can take an unpredictable amount of time, so how does creating a fixed amount of time to do discovery help? Why not just let someone take as long as they need to finish?

The problem with this is that discovering new things is like trying to slay a hydra. Learn one thing and you find 3 more things you need to learn about. It’s easy to get lost in the ocean of potential knowledge. A team can act like a compass and help provide focus in one direction. Everything else is just noise.

The opposite can also happen. A person can easily get stuck trying to understand something new or getting a prototype working. Having a set time to go back to the team and discuss the issues can help that person find ways to move forward.

And there is no reason why you can’t have multiple time boxes for a single feature. Time boxing isn’t about forcing things to get done in a certain amount of time. They’re about providing a structure for check ins to make sure the right progress is made.

More importantly, time boxing provides an opportunity to re-evaluate whether a task is worthwhile or not. Maybe there’s a feature seems important at first. After multiple time boxes with little progress made though, that feature may seem like less of a priority. The opportunity cost of working on a feature with an unknown amount of work left is greater than other features. Time boxing provides an opportunity to re-prioritize things because it starts to reveal how expensive something may actually be to develop.

There are other ways to manage unknowns in software. I just happen to have found time boxing to be the most effective for the projects I’ve worked on. The important thing to note is that good project management isn’t about pretending that unknowns don’t exist in software development. Good project management is about accounting for them.