Human Behavior in Software Development
I find behavioral economics fascinating. Many economists assume rational behavior among all people and it results in economic models that seem good in theory, but end up being completely inaccurate. Humans aren’t 100% rational and the real decisions we make are difficult to account for.
We have a similar problem when it comes to the software development process. A lot of it is designed with the assumption that software developers, who are still human, can act in a completely organized and logical fashion with no errors. The waterfall model is the most extreme example of how this problem manifests. You plan out what you want to build, then you design the system, then you build the system, then you test, and finally you release. It’s perfectly organized and makes a lot of sense in theory.
It took decades of struggling projects to prove that this doesn’t work. It doesn’t follow how humans actually think or work naturally. The original concept of agile software development was meant to fix this. The first line is even “Individuals and interactions over processes and tools” Focus on the people in your team and build a process that fits them. You can still have a process. You can still talk about how people can work more effectively. You can still improve human behavior slowly. However, the people come first. The process needs to adapt to them, not the other way around.
Which leads us to the next example. It has become common practice to estimate software tasks with story points. Many consider it the “proper” way to do “agile”. However, I once had a team that really struggled with story points. The abstract nature of story points made it difficult to understand. Even after months of trying, we still couldn’t get on the same page of what 3 or 5 story points meant. So I switched them to time estimates. Humans literally deal with time every day of our lives. It makes time easier to think about and so the act of estimating comes more naturally. Granted, we completed about 60 hours of estimated work with 200 working hours in a sprint, but that ratio doesn’t matter. What mattered is that we now had some semblance of what we could accomplish. We used an estimated hour as a semi-abstract unit of measure of what we thought we could accomplish in an hour. Our team’s velocity was based on how many estimated hours we could accomplish rather than trying to account for all the things people struggle to think about when estimating such as meetings, code reviews, or spontaneous discussions. I made the process fit the people on my team.
It doesn’t matter how well thought out a given practice is or how widely used it is. Its utility is essentially zero if it doesn’t work for the people you have. The process must always be molded to fit those people. Those who stubbornly stick to agile practices because everyone else uses those practices have not learned from the underlying failings of waterfall. A process created in isolation without accounting for human behavior is destined to fail. They’re still essentially using the same underlying assumptions of waterfall, but are calling it “agile”.
It is an unfortunate reality that it is also natural for humans to want easy answers and managers are still human as well. They want a set of practices that other people have used to make their own jobs easier. But just because those practices worked with other teams with other people, does not mean it will work for yours or mine. If it did, we wouldn’t need managers. We’d just program those practices into some algorithm and let that manage our teams. A large part of having managers is because teams need to have a process customized for them.
Human behavior is unaccounted for in areas outside of management as well. Take automated testing. There is a very large difference between the number of developers who say it is a good idea versus the number of developers who actually build those tests. Many hold on to a fallacy that if they make building tests easier, then developers will start writing tests. I used to believe this as well. It’s a compelling thought because it’s relatively easy to do. Look at where building tests have friction and then write some code to remove it.
However, the real problem is not that building tests has a lot of friction. I’ve seen tests take minutes to write and people will still not write them. The problem is that while everyone says they think tests are a good idea, they don’t feel the value of those tests after writing them. Writing functioning code for an application gives most developers a sense of accomplishment. Writing tests does not. This isn’t an easy problem to solve, but it is the real problem that needs solving.
Speaking of real problems, let’s talk about code reviews. I personally like code reviews for many reasons, but they are rarely conducted in a way that’s productive. For instance, it’s common practice to have someone write code and then have someone else review it when the code is complete. It sounds like a sensible and organized practice. Unfortunately, this leaves you open to the problem where the entirety of the code should be rewritten. I’ve seen this happen more times than I’d like. No one wants to rewrite all the code they spent days working on and few people are emotionless enough to force them to do so. That means this situation usually results in the code being merged in anyway with a note that it needs to be rewritten at some point in the future (i.e. never).
While my preferred way of preventing this problem is a design review, that can still miss some big things. Code reviews really should happen periodically while the code is being written. Big red flags can be seen early and resolved early. The reviewer will have more context in the code base because they are digesting it in small pieces over time. This makes it easier to do the code review in the first place because there’s fewer mental gymnastics needed to understand the code.
I’ve glossed over a number of these topics rather quickly, and there are many more examples of human behavior being unaccounted for in software development. When I think about all the blog posts I’ve written over the past few years, the common umbrella for most of those posts is trying to account for how humans naturally think or act. Going forward, I will be writing more posts around that particular focus and I will certainly write posts digging deeper into everything I mentioned in this one.