Accounting For Inaccurate Software Estimates
Estimates are inevitably going to become inaccurate. There are simply too many factors to take into account to create an accurate estimate in a reasonable manner. As much as we want to try and make our estimates more accurate, a better use of our efforts would be to look at how we handle things when an estimate is missed.
The best way to handle a missed estimate is to find out sooner rather than later. If you have a two week task, it makes no sense to find out that the task won’t be finished after working on it for two weeks. Daily standups are a popular trend for a reason and one of those is to help bring up difficulties that could cause an estimate to be missed. It is difficult to see two weeks into the future, but nearly every developer has some sense of what they want to accomplish in a day. If they don’t make it, they tend to know why.
One of these reasons is when the scope of the task has increased unnecessarily. Sometimes this happens because of office politics. Someone had an idea to make a feature “cooler” or they just have a pet project they want done. This is a hard situation and the solution for handling it differs by company, but the end result should never be to acquiesce and allow the scope increase.
Scope can also increase unnecessarily because developers always find edge cases while writing code. Our natural tendency is to simply account for that edge case in our code without discussion. However, some edge cases can take a long time to account for. Even if they don’t, a large number of edge cases that eat up 15-20 minutes each can eat up days.
Not all edge cases matter. Every edge case has a probability of occurrence and a consequence. Many edge cases have a low probability and low consequence. For example, let’s say there is a 1 in a 1000 chance your system shows someone the wrong recommendation in an e-commerce site. Given that a single person sees hundreds to thousands of recommendations a day (and how irrelevant most of them end up being), a single bad recommendation is not going to matter that much.
An edge case may not matter even with a high probability. Let’s say you’re making a single player video game. There is an exploit players can use to get some benefit. Does fixing the exploit matter? The game is single player so they can’t use it to get a competitive advantage against other players. If they also get more enjoyment out of the game that way, then one could rightfully call the bug a “feature”. These edge cases rarely need to be accounted for. A developer can bring it up during a standup before working on it and a quick discussion can be had. Five minutes of talking can save hours of development.
Of course sometimes edge cases do matter quite a bit. An email notification system that is constantly sending out duplicate emails has both high probability and high consequence. A bug like this would be extremely annoying for users, especially with the importance placed on inboxes.
And even low probability edge cases can have consequences high enough that even a single occurrence can be devastating. No billing system should double charge a customer. It’s a display of gross incompetence, makes customers angry, and probably will get you into legal trouble if it isn’t fixed quickly.
Since edge cases like this are necessary scope increases, something needs to happen with the project plan. Either we need to accept that the project will take longer or we need to reprioritize something else. Daily standups should be short, but waiting for scheduled planning sessions isn’t always feasible either.
I’m a big fan of impromptu planning sessions, maybe with a subset of the team, to help reassess the needs of the project. Features can be changed to ease development or other features can get cut in order to make room for more important work. So long as these sessions aren’t happening every day (or, ugh, multiple times a day), the time spent should help accelerate development rather than hinder it. This impromptu re-planning session can also be useful for other factors such as someone being out sick or a task being more technically challenging than originally thought.
So far we’ve only discussed cases where estimates will become inaccurate because of unforeseen circumstances. There are plenty of times where developers know an estimate has a high margin of error. This can happen if a feature is inherently challenging.
For example, let’s say you are working on a social feature that requires data or actions taken for contacts separated by multiple degrees. How many people on LinkedIn have more than 1000 contacts? If you need data from the contacts of those contacts, and those contacts also have 1000 contacts each, then you’re looking at data for 1 million people (assuming no duplicate contacts). Add another degree of separation and things just got worse. This is not an insurmountable problem, but will be difficult if you need the data quickly or if it has to be repeated for multiple users in quick succession.
Some features are just known to have a lot of edge cases. Let’s say you wanted to build your own user permission system. Everyone who has built a permission system has gone in thinking “oh this isn’t that hard” and ended up suffering. If you’re a user of G-suite, then you know even Google with all it’s engineering resources hasn’t quite figured it out. Odds are your smaller team will have trouble as well.
There are also features that your team may not have a lot of experience with. For example, you want a recommendation engine but don’t have a single developer who has worked on one before. One option here is to spend time doing research and going through several prototyping cycles. The other is to try and hire someone with the experience. Both options will take an unknown amount of time.
Expectations need to be set properly for all these cases. Estimates are probably not the right tool here. If you need to work on a feature that falls into one of these buckets, then you need to accept that you won’t really know when it’ll be done at the point when you start. A better tool in this situation is to start noting discoveries and accomplishments made on a daily basis. You may not be able to control the completion date, but you can control whether progress is being made or not. And if progress is not being made at a rate necessary to justify the project, then you can choose to cut your losses and have your team work on something else.
Many managers prefer their teams work on learning how to estimate more accurately. However, teams can also work on how to account for estimates that are missed. You can’t always control the factors that affect an estimate, but you can always control how you react. That makes learning to account for inaccurate estimates a more valuable skill.