The invisibility of software and the coding warroom

Software is one of the few things built by humans that is incredibly hard for us to fully see. This is true during the construction process and after it is 'complete' (one problem is that good software is never really complete- new features are constantly added, it is made easier to use, and it gets fixed when bugs are found). It is very difficult to get a sense of how all the pieces fit together.

When asked by a layman to describe how modern software (i.e. large software) is built I like to use an analogy. Building a large piece of software is like building a skyscraper where the construction workers can only work at night and each worker has only a small flashlight to guide them. One can only see about ten feet in front of them at any one time.

We write code on small screens and can only see a tiny fraction of it at any one time. We use certain models to help get different perspectives of our systems but we cannot pick up and examine the pieces and see how they fit together like we can with almost every other thing that is engineered.

It is even harder to visualize how software changes over time. With time lapse photography one can see how a skyscraper evolves but the same is not possible with software. How does one see how some exceptional work has evolved from the early beginnings to a mature and extensible solution? Most of the code I am proud of was grown over many iterations. There is usually a good story about how the code changed over time but these stories rarely get told to anyone and almost never get written down anywhere. They stay in our heads for a little while but they inevitably escape into the ether.

In order for developers to grow and learn we must open up for examination how we do our work. One would think that knowledge workers, because of their ability to gather and manipulate data, would excel at capturing and presenting this information to their peers. However, if you look at some of our tools (our version control systems in particular) we are pretty bad at it. For example, some of the most valuable thought work that we do happens at the whiteboard. Inevitably, it gets erased shortly after we create it. We rely on our memories to store much of this valuable information and the stories about how the work has evolved. I think a more reliable storage mechanism should be used.

I have often wondered what it would be like for software developers to project their work in 'warroom' like environments. Imagine groups of developers working together where everyone could see what is happening by members of the team. This would certainly help (at least partially) solve the 'invisibility' problem by expanding our view of the system beyond the 10 foot limit.