Software Developers Should Have Sysadmin Experience
Being a software developer and being a system administrator are very different things. Many folks lump the two professions together, but the skillsets do not overlap much. Software developers write code. System administrators maintain the computer systems that the code runs on.
Both are critical. Software provides value to users such as yourself, but software can’t exist without an environment to run on.
Many companies like to keep developers and sysadmins on separate teams. This makes sense in theory. You have two different skillsets for two different professions. Why not have two different teams?
The biggest issue with this is that context is really important when building software. Software developers need to understand the environment where their code will be running or they may not build it properly. An analogy: imagine you were tasked with building a house without knowing where it was. You’d probably design a decent enough house.
But then someone takes that design and builds it… on an incline.
If a software developer has never done any sysadmin work, then they will build code that works in theory. The developer tends to build software on their single computer. Most software on the internet runs on multiple computers. The bigger sites like Google or Facebook have thousands and thousands of computers. But like our theoretical house that worked on flat land, code that works in theory can completely fall apart when it becomes live in front of users. This can come in the form of bugs or the software crashing.
For example, think of a website where you upload images such as Facebook or Twitter. Facebook and Twitter have way too many people using them to have those services run on a single server/computer. So they have multiple web servers set up to deliver their website to you.
Why does this matter? Remember that developers tend to write code on a single computer. They could write code that stores the image on the same hard drive as the web server, which for them is the same. Yet problems arise when that code goes live in front of users like you.
If there are 3 web servers and the image is stored on the hard drive for one, then 2 out of 3 people will be unable to see it. If you had 300 friends, then only 100 people would be able to see the image you uploaded. What a terrible service!
The proper way to fix this is to use a distributed file system that is accessible by all web servers.
A developer with little experience will not realize this though. They write code on a single computer. The image will show up 100% of the time when they test their code on their one computer. It is only after the code is being used by users on live web servers does the problem occur.
There are dozens if not hundreds of other examples. Someone needs to explain to the developer how these things work, but being told something is not nearly as effective as experiencing it for yourself. Experiences create a deeper understanding.
That understanding will help catch errors much earlier in the process. A developer with no sysadmin experience will go through a flow where:
- They write code
- They send it over the wall for code review and/or testing
- The code is sent back to them to be fixed.
Things can also get worse because often times sysadmins won’t look at a developer’s code. That means that users could see bugs first! These kind of issues are also hard to investigate because it will work perfectly on a developer’s computer. They won’t be able to recreate the issue easily.
Admittedly, I hate doing sysadmin work. I know there are people who enjoy it, but to me it is just a constant source of frustration. It’s a separate skillset from writing code, but it stands in my way to get people to use the software that I built.
But I do it anyway. I do it because the context helps me write better code. I do it because code that works on my computer is useless. The code that matters is the code that works on web servers that are live in front of everyone else. Just writing code is only doing half of the job a software developer needs to do.