The modern world is made of lots of moving parts. Nothing is orchestrated, there is no grand plan, but they come together to make the world we live in.

It is this ‘complexity’ that makes predicting anything impossible. It also makes predicting solutions to the worlds problems, be it economic, crime, world hunger or even how our business will run in 10 years time impossible. We turn to our leaders and experts for guidance, to steer us in the right direction, but it turns out that for long term goals, they have as much influence as the rest of us. Where the leaders and the experts go excel at is leading in the short an medium term. The reason for this is that their expertise can only go so far before entropy kicks in and it just makes it impossible to be right. So how should you solve problems, in incremental steps.

Surprisingly, this method of solving problems has been around for billions of years. In Nature, its called Evolution.

In evolution, each incremental step starts off with a mutation (variation). If the mutation gives the organism an advantage, the organism lives and procreates spreading that mutation. If the mutation causes a disadvantage, it is more likely that the organism dies, and the mutation is no longer spread.

Karl Sims a computer graphics artist tried simulating evolution in a virtual world. From his videos, you can see that his virtual creatures started having traits that living creatures have. This just shows how well nature has used evolution to solve problems.

Types Of Complex Solutions

There are 3 types of solutions to complex problems.

Mount Fuji

photo credit:Reaching Out via photopin (license)

There is one, and only one perfect solution to the problem. As you improve, you get to the ultimate solution, any more effort and you are just going down hill.

Very few problems fall in this category, but when we solve a problem, we seem to think we have found the best way.

Rugged Landscape

photo credit:Sunny Mont Blanc via photopin (license)

Here, when you solve a problem, you have one solution of many possible solutions. Some better than others. It is very hard to find the perfect solution, but you need to keep trying to see if there is a more efficient, or better solution.

Dancing Mountain

Small Changes

So what constitutes a small change? Let me give you a real life example that I had. As part of the daily health check that I do for the company I work for. The health check steps are documented in a wiki page with checkboxes so you know which steps are executed as you go along.

Once of these steps was check if a file was transferred successfully to its destination. To do this, we would FTP to the remote host and see if the file was present every morning. This process didn’t take long. About 5 min each day. This was one task out of many as part of our health check that would take about an hour to accomplish if there was no issues. Over a day at times if there was. To save time with this check I wrote a python script that would show a web page (Using SimpleHTTPServer) of the remote directory to be viewed. The script took about 15 min to write. Now we have a visual indicator that is embedded in the wiki page (through iframes) to see if the file is present. This change alone saved 5 min. Yes, the script could be made better, a better solution would be to send an alert email or SMS etc if the file isn’t present or other countless improvements, but that is not the point.

This small change which saved 5 min was a first step. Other small improvements followed. Provided there are no issues, the health check is now down to 5-10 min. With issues, its is now rare that they would take a day or more. Most of the time its an hour or two to resolve the issues. We still have a long way to go, but we are on the journey.

The point is, that you start with something and build upon it and keep trying to improve it. Some things work, some don’t, but you won’t know until you try.

Galls Law

John Gall, an American paediatrician wrote a book “General systemantics : an essay on how systems work, and especially how they fail.”

In this book (Which I admit I haven’t read) there is one statement his is known for

A complex system that works is invariably found to have evolved from a simple system that worked. A complex system designed from scratch never works and cannot be patched up to make it work. You have to start over with a working simple system

Here John Gall advocates building complex systems through evolution by starting with a simple system and building upon it. Pretty much how an Agile project should work.

 W. Edwards Deming

Now we come to the tail end of this post. Back in World War 2, while the men were fighting, the women would be the ones working in the factories. Building every day products and the machinery of war. With limited resources, the factories had to work smarter and harder to get reasonable productivity. Deming, through all this worked out a system to achieve this.

At the end of the war, all the men came back and the women were out and everything went back to the way it was. At least in America and the allied nations.

In Japan, they were just trying to re-build from after the War. Deming was brought over  the techniques that he developed during the war. The Japanese embraced this. None more than Toyota. Which is where we get the Toyota Production system and the start of Lean.

One of these techniques that Deming brought over was the PDCA cycle. Also known as the Deming Cycle.

• Plan your change and determine its success criteria
• Do your change
• Check that it meets the success criteria
• Act accordingly
  • If the success criteria is met, Yay complete.
  • If the success criteria is not met, the re-evaluate and plan the next step and repeat the cycle.

Conclusion

So as you can see, through independent means, many people have found that the evolutionary way to trial and error seems to be the best means to achieve the best solution to a problem. Be it nature, manufacturing or computer science.