Atomic Spin :: Elements of Software

Every human society has a similar expression about the nature of complexity. In English we say, “The devil is in the details”. In Sweden they say “Djävulen finns i alla detaljer”. Same idea, exactly. Customers of software development firms often underestimate, with a vengeance, the truth of this well-worn aphorism. Maybe that’s a necessary thing, as their vision needs to be unfettered by details in order to bring to life something new, innovative, and valuable. Software developers, on the other hand, are required to deal with the devilish details. Computers demand precise, detailed instructions. Software craftspeople must turn a customer’s vision into a working system, devilish details included.

Smashing the vase

I’ve thought of a metaphor to help describe how details absolutely explode out of a seemingly simple application. Think of a simple ceramic vase, decorated with pleasing patterns and colors in the glaze. The simple form makes it clear that this vase does its job very well: water is retained, flowers are displayed, a room is brightened. Most customers have a similarly clear and simple description of their idea for a new application: users do this, the app does that, time is saved, errors are reduced, customer is happy, we all get rich. The software craftsperson starts from this description by taking the beautiful, simple, whole vase and smashing it on the floor. The smashing is called decomposition. It’s how we break big un-manageable things into little, manageable pieces. Successful software development requires not only skills in decomposition, but also skills in composition. Taking the pieces of the vase and putting them back together into a functional, beautiful, whole requires dealing with a bunch of devilish details.

The second law of thermodynamics says that entropy increases over time unless energy is put into a system. In terms of our vase, the second law means that smashing the vase onto the floor will result in increased entropy (disorganization). (Limitation of the metaphor: Smashing a real vase requires almost no effort. De-composing a software system accurately and completely requires experience and skill.) Putting the vase together again will require much more effort than smashing it. Composition into a functioning whole is even more difficult than decomposition. Decomposition is the result of questioning, separating, pondering, and identifying possibilities. Composition is the result of deciding, designing, building, creating, and verifying.

The relevance to software of the second law of thermodynamics is the tricky and time-consuming business of handling all the details involved in building an application from the decomposed pieces. If you only see the complete, whole, simple vision, the details are not readily apparent. But when you have to build the vision, the details become excruciatingly relevant. Getting lost in those details, being overwhelmed by them and losing your way is a common cause of failure in software projects.

Happily, the agile software development community has created specific practices to address this problem. Making only small changes, testing as you go, keeping the system working continuously, integrating frequently, delivering regularly – these practices are all aimed at dealing with the inherent complexity of all the devilish details of interesting software projects.

The tip of the iceberg

Software systems are like icebergs – you often only see their tips. Their bulk and complexity remain hidden beneath the water.

A web application is a good example of this. The user sees a simple web page with a few input forms. Everyone knows how easy it is to use and create a simple web page. Consider, though, what’s behind that web page. Business logic, data storage, input validation, business process integration – the fan out of details can be astounding. And this list doesn’t even include mundane details like browser dependencies, testing, and user interface design.

Sometimes a customer is too close to the problem to fully appreciate its size. Knowing too much can be worse than knowing too little when it comes to creating an effective application or estimating the effort required to build it. Familiarity breeds blindness to the complexity inherent in a system. The agile practice of “whole team”, where the customer works directly with the developers, promotes opportunities for developers and customers together to translate the customer’s expertise into a working software system. Building a shared mental model and translating that into working software takes time, and hence money.

Part of a series. See the syllabus and outline at the Software Customer 101 page.

Previous: Welcome to Software Customer 101 Next: “Understanding Risk”