Finding Zen Solutions to Problems

Can we derive a systematic procedure for finding high quality solutions to design problems?

I have so far given specific solutions to specific problems for illustration purposes but this website is not meant to be a recipe book; my goal is to give you the means of finding your own solutions to the problems you encounter in the course of your programmer’s life.

This is a rough outline of the process I use while programming.

  1. Divide and conquer
  2. Identify data (state) and behaviour
  3. Identify and classify constraints on data and behaviour
  4. Find matching programming constructs
  5. Determine zen level of each
  6. Take a decision

Divide and Conquer

The first step is to decomposing a complex problem into a series of smaller ones until reaching problems small enough that I can keep the entirety of any one of them in my mind.

Indentify Data and Behaviour

I then need to decide what data I need to store. Whether it is stored as local variables, into the callstack, as the position of the program counter, as attributes of a singleton object, as a file on disk or an entity in a database is irrelevant at this point. What matters is that I need information available somewhere. In parallel to this reflexion I need to make clear what behaviour must be encoded, in terms of changes to that data. Note that it is more important to express behaviour as a before-and-after comparison, than as how you can achieve that. Don’t say “loop over those numbers, and every time you reach one that is bigger than the current biggest one, replace that current biggest one by that value”. Say instead “before starting I have a bunch of numbers, and when I am done I have the largest of those numbers available somewhere”.

Identify and Classify Constraints

The third step is to find what constraints you have. Currently the model of your data and your behaviour is completely flat and unstructured. Try to find relationships between data elements: this is always smaller than that, this always has the same type as that, this key is always present in that map. Try to find intrinsic restrictions on individual elements: this value is never null, this string is always one of these five possibilities, this number is never negative. Try to find constraints on behaviour: this piece of code will never modify those objects, or inversely these objects may only be modified from this piece of code. This piece of code will never acquire a lock on more than one object (or, will never acquire any lock at all), and so on.

Once you have identified your constraints, classify them. The examples mentioned in the previous paragraph could be classified as nullability, type restriction to a finite set, type restriction to a practically infinite set (even though 32bit, 64bit or any xbit integers are definitely finite in number you aren’t going to write all of them in your source code, so they can be considered as infinite while you make your design decisions), access restriction to certain functions on certain objects (accessible by default, then non-accessible by default), restriction on simultaneous access to certain functions. The “will never acquire any lock at all” example is no different from “will never modify those objects”, both are about restricting access to certain functions.

Find Matching Programming Constructs

The fourth step is to mentally find features in your programming language (or build chain, or development environment, or testing libraries) that enforce similar properties. To perform this step effectively there are no other ways than experience. Know all features of your programming environment, use them and use them again until they become an extension of your brain. It is important to realise that it doesn’t matter what those features were designed for. What does matter is what they actually do. For instance you can implement singletons with a single-element enumeration in Java. That’s not what enumerations were designed for, enumerations are meant to represent types covering a finite collection of values, and certainly should not be stateful. And yet it works!

Once you found those features, construct a design to encode your data or behaviour constraint in such a way that violating the constraint triggers that environment feature. With practice you will be able to do that mentally, and cover possibly dozen of designs in a matter of minutes.

Determine Zen Levels

Fifth step: For each constraint you found in step three, and for each design you found in step four, evaluate the corresponding “zen level”. Try breaking the constraint. Do you get a compilation error? A compilation warning? Does the error jump into your face when you look at the erroneous code section? Does a unit test fail? Do you get an error at startup, or when you use the associated feature?

Some constraints can be handled separately, but some will typically overlap (especially if the piece of logic that you isolated in step 1 is small), in which case the zen level is really a matrix. Each row of the matrix is a design, and one design may be highly resilient with respect to constraint one, but very fragile with respect to constraint two.

Take a Decision

Sixth step: based on the data from the previous step, choose a design. Your decision should take other dimensions into account, such as how verbose, complex, readable a candidate design is. Here you can’t just use a mechanical approach such as summing all levels or rejecting outright a design that has level seven in any column, or favouring a design because it is level one in one column (but scores badly in others).

A good approach is to estimate the cost of a design as a product of the expected number of bugs times the cost of an unfixed bug, plus the maintenance cost (the effort required to do a change in the code times the frequency of changes).

The expected number of bugs over time can itself be estimated as the product of the risk an individual developer making an individual change introduces a bug times the frequency of updates (or copy-pastes followed by an update of copied code). A pattern that will be used hundreds of times over the code base has better be Level One everywhere, while a piece of code that will be written once and never touched again.

Finally, we have to accept that some bugs will slip through and make it to production. Instead of attempting to reach zero bugs, we should estimate which types of bugs are acceptable and which are not. I capture this with a simplistic BusinessValue factor: BusinessValue=0 means we don’t care if a bug happens, we won’t attempt fixing it anyway. Higher values means we will have to watch for bugs and fix them.

We get something like this:

Cost = UpdateFrequency * (BugRisk * BugFixEffort * BusinessValue + MaintenanceEffort)

A design with high Zen level has a low BugRisk because it’s harder to make a mistake without noticing the consequences. High Zen level code tends to have a low BugFixEffort because the higher Zen levels require error and warning messages to be explicit (making analysis quick), and high levels can only be achieved with low redundancy, making mistakes highly localised, and so quick to fix.

The MaintenanceEffort is mostly independent of the Zen level, although Level Four typically has a higher cost as you need to maintain your unit tests. You can have extremely complex Level One designs, which gives them a high maintenance cost, which in turn may negate the advantage provided by your Level Oneness, no matter how beautiful it is.