The Problem With Software Estimates
I’ve written about the technical challenges of coming up with estimates in software development before. What I’ve neglected to talk about is that one of the biggest problems with getting accurate estimates, particularly for large projects, isn’t technical. The biggest problem is getting to understand your user.
The idea that you can get accurate estimates for a large project relies on the waterfall model working. You come with some requirements that you know to be excellent. Then you come up with estimates on the implementation.
The problem is that requirements are almost always a hypothesis of value for users. You can try to run experiments with paper prototypes or wireframes, but that requires your demographic to actually be familiar enough with the development process to provide good feedback for such low fidelity prototypes.
The most effective experiment is to launch. Get users to use a working product and then collect their feedback. Sometimes their feedback isn’t in what they say, but what they do. I still remember hearing a talk from a Facebook engineer about the initial launch of the news feed, which is now the core of every social network (can you believe social networks used to not have feeds?) Everyone’s Facebook news feed was filled with posts about how news feed was terrible. But those people were complaining with the news feed. And those people were also constantly refreshing the news feed.
NOTE: I’m ignoring the ethical debates of this particular product and focusing solely on the strategy of collecting feedback.
One day of real user usage can give you more information than months of internal debate.
That data allows you to refine your hypothesis/requirements resulting in better requirements. You can now iterate and improve on the product to hopefully deliver more value to your users.
The problem is that before you get that data, you don’t know what those iterations will look like. All the technical estimation skill in the world won’t help if you don’t know how to change your requirements. You may be able to provide accurate estimate on a particular iteration, but not what accounting for feedback after that iteration is released.
The problem with software estimates is that you need requirements to provide them, but you need to release working software to know what your requirements should really be.
I keep this blog around for posterity, but have since moved on. An explanation can be found here
I still write though and if you'd like to read my more recent work, feel free to subscribe to my substack.
