All posts by admin

Cargo Cults

Back in World War II, in what is now known as Indonesia, Japanese and American forces set up base camps. These base camps were set up to provide supplies and ammunition to the forces in the region. Planes were called in using radios. The operators would wear headphones and talk into microphones and call in the planes.

To the primitive indigenous people, this was like magic. They would see the actions of the soldiers and see the outcomes of cargo, the precious cargo drop from the heavens via air drops. Be brought in by planes. Spectacular battles would be fought on the ground and in the air. Then all of a sudden it would all stop. The cargo that they treasured so much that would come from the heavens no longer came. The people that brought it, gone. What were they to do.

So, the indigenous people would try to mimic what the soldiers did. They made radios out of wood, Headphones out of coconuts, planes out of sticks and straw. No matter what they did, no matter how much they tried, the cargo would not come.

The same can be said about those that try agile and DevOps without understanding. For whatever reason, for example, a manager or C level executive reads about how Agile is getting results much quicker and with much better quality than traditional methods. They may even read a couple of blog posts. They read that Agile/Scrum has stand ups, Sprints of 2 weeks. So they implement that.

You might even get a planning session, only a hour or so, just long enough to assign tasks to the team members for the sprint.

Here, we see Cargo Cult activities in action. These practitioners of agile go through the motions. There may be some improvement (That just shows how dysfunctional Waterfall is), but most of the time, they will fail and not understand why. Therefore the thinking is that Agile is a failure and does not work.

In these cases, the thinking is that the process has all the magic.

Another variation is that CI or CD is implemented. Again, the system may have some benefits, but most of the time it fails. I this case, the thinking is that the magic is in the technology.

Again, cargo cult. Going through the motions. Following the process without understanding, and expecting the magic to happen.

Primarily, these approaches are both wrong and miss the point. The magic itself is in the people. When the people accept agile, work towards making it work. Try to learn, try to improve. That is where you get the speed, and quality. The process and technology are supporting roles to the people.The process is there to help the people communicate. The technology is just a tool to help embed quality and speed up development.

When the people stop “doing agile” and start “being agile”, the cargo cult phase morphs into the being phase. It just clicks.

At least  that is how I see it. I’m still waiting for things to click. If it has clicked for you, I’d like to know your story.

Toyota Tour

Late last week, I had the opportunity to visit the Toyota factory at Altona in Melbourne. If you have the opportunity to visit the plant before it closes in late 2017, I highly recommend you do. It’s around about a 6 month wait for groups (At least it was when I signed up in March this year, but you may get in if you are a small group. You never know, there may be a late cancellation. The tour is free, and you get to see the Toyota Production System first hand. i.e., Genchi Genbutsu (Go and See). 

The tour guide took us first to the pressing plant where the sheet metal for the car bodies were being pressed. Those dies are huge and heavy. I remember reading in the Toyota Way where in other car companies previously, it would take hours to change over dies to different molds. On the tour, we were told that it would take minutes. A few for the lighter dies, and up to 12 for the heavier. Everything was controlled by one crane operator who controlled everything from the ground. So he know where to place everything. There was still a cabin for the crane where the operator previously sat, and they would have someone below guiding, but by having the operator on the ground, it made things more efficient, and that was the whole point. Efficiency as simply as possible. We saw simple solutions to get efficient. For example, when bonnets are pressed and stacked. To prevent them from banging into each other and causing dents in the metal, tennis balls on string were used to separate the bonnets. A simple solution, but it did the job. There was also complex solutions. They had autonomous rovers that would carry parts to and from assembly areas. These rovers would follow a magnetic strip along the ground. They would stop if anyone got within 50-100cm from the rover. And they would run every where at a slow walking pace.

I could see first hand the limiting of parts of each station. There would be a bin for each station for the required parts that the operator would take from. A second bin would be present for when the first one was used up. And possibly a third. When the bin was empty, an electronic Kanban system was triggered to fetch another load if required. Speaking of Kanban’s, these were gathered every 38 minutes. An announcement was made at one facility for them to be gathered by Supervisors.

There was only one area where there was inventory. That was for parts that came directly from Japan. We were told that the only reason they had inventory was in case of Typhoons and other delays. Otherwise everything comes as required.

For every car that comes off the line in one end, the materials to make another car come in from the other. The whole system is a highly coreographed dance. 

Everything is made in the order it was ordered. For example, you have one red Camry, another white Aurion. A blue Camry hybrid etc. Each car is made in that order. They do not do batches. For example, 50 Camry’s white, 50 Camry’s blue etc. We didn’t get to see inside the paint facilities, but they showed us a video from Megafactories of the painting. Each car is painted individually. You may have one white car, and then next to it, only 50 to 100cm away, a blue car. The painting is so accurate, that there is no splatter. To reduce splatter, the car is charged negative and the paint is charged positive so that they are attracted to one another. Exhaust fans drive excess paint down through the floor – what little waste there is. Everything is paced based on “takt” Time, which is the frequency it takes to produce vehicles. The takt time is based on orders. For example, a slow day may produce 80 vehicles per shift, a fast paced one would be 200 vehicles per shift. Everything is based on the number of orders. Painting is also where the most time taken to manufacture a vehicle takes place. It takes 12-15 hours for the paint to dry and 24 hours total for a car to be manufactured.

If it did happen where a major part was for the wrong vehicle, for example the seats, then the right seats would be taken from the limited stock on site. By the time the vehicle for that stock was taken is about to be put together, a replacement would have been on site. It takes around about 16 minutes for local suppliers to get replacement parts where required.

As I mentioned previously, the Toyota Factory is closing late next year. It’s going to be a hard time for the workers there, but from what I saw, the management does not take this lightly. There is a training center where employees get skilled up to help them find a job later. Management has a “Food for Thought” drive, where senior management will have a meal together with every employee who wants one.  This in my opinion shows respect for their employees. I have never seen something like this happen in my entire working life. 

While I was on the tour, I did ask one question. I’m curious about Kaizen. I’m the sort of person who comes up with a lot of ideas, and usually comes up with some sort of prototype to test the theory. Anyway, my question was “How often is the Kaizen workshops done? Is it monthly or as required?” I was told that it is as required. When an employee comes up with an idea, it is evaluated straight away. No idea is too stupid. 

I like this sort of thinking in companies. It uses their people as a resource for innovation. I saw something similar when I visited the Telstra innovation hub late last year. Telstra has a system where employees could submit ideas online, they would be evaluated by their peers through a Monopoly money type System where each employee has $100 and can invest that money however they see fit in other people’s ideas to fund them. 

If you want to learn more about the Toyota Production System and Lean manufacturing, or Lean in general, I highly recommend the following books.

The Toyota Way – Jeffrey Liker
Lean Thinking – Womack and Jones

Thoughts on Focusing On Cost

A company that predominantly focuses on cost is a company that is going out of business.

Why?

There is a lowest level you can go with cost. If you cut costs too low, you limit your ability to make more revenue. You go down a deadly spiral till you are out of business.

The opposite is true, a company that focuses on revenue has no limits.

Why?

There is no known ceiling to how much revenue you can make.

In the theory of constraints, the focus is on increasing throughput, in lean it is flow.

Both methods look at getting the product out to the customer with quality sooner. This can result in more sales, better market response time and thus more revenue.

Flow or throughput is made faster by making the system more efficient. By increasing efficiency, you reduce waste, be it materials or effort. This results in reduce cost.

This means that reduced cost is a byproduct of the theory of constraints and lean. The first goal is getting a quality product out to the customer as efficiently as possible.

Speed of delivery is also a byproduct of lean and theory of constraints thinking.

By increasing efficiency, you reduce the manufacturing time, and therefore increase the tie of delivery.

Lean Terms

Agile, or at least Scrum is based on the manufacturing practices of Japanese car companies from the 1980’s. At the time, Japanese car companies were building cars with better quality, faster and cheaper than their American counterparts. The most successful of these Japanese companies was Toyota with their Toyota Production System. From this, evolved the Lean process.

Myself and my team are about to go visit the Toyota plant in Altona Victoria later this month. So I thought it would be a good idea to get myself reaquainted with Lean terms that I read in “The Toyota Way” by Jeffrey Liker.

I’m going to try to relate these terms to the CAMS (Culture, Automation, Measurement/Monitoring, Sharing) principles developed by John Willis and Damon Edwards that was developed during their DevOps Cafe podcast and give my explanation of what I think the terms mean in terms of software development.

 

Poka Yoke : Mistake Proofing (culture, automation) – In my thoughts, this is trying to prevent Defects in the software development process. One possible method of mistake proofing is through Test Driven Development where you build your tests before you start your development. It can also mean automation. There are a number of tasks that we that are manual, repetitive, and or simply done so infrequently that it’s hard to remember all the steps. By removing the component that can cause mistakes in tasks of this nature (The human) you can prevent mistakes from happening.

Hansei : Self Reflection (culture) – This I think is the retrospective in Scrum, but in a more personal level. It is simply to think over what you have done whether it’s in the past hour or the past sprint and try to think of a better way to improve.

Andon : Sign or signal to highlight action required (measurement, monitoring) – This is simply a signal to tell when a developer is in trouble or when the system is in trouble. In software development, it can take the form of an alert, an email, even a red light.

Jidoka : automation (automation – obviously) – To put simply, all types of automation. Automation of the build process, automation of the development process. Automation of tasks you do every day. For example a daily health check email. Anything that you can automate that will improve the system.

Just in Time (culture) : This is simply to have resources, requirements come just as they are required. Contrary to popular thinking about Agile that you do not need planning, to get Just in Time to work, you need to plan. If for example, to make a change to the firewall to get some ports opened up, and there is a 2 day lead time, you need to make that request 2 days before you need the ports opened up. Not when you need the ports, nor significantly in advance as requirements may change.

Henjunka : production smoothing or leveling (culture) – This is leveling the workload. How many of you have been on a project where at the beginning there is a lot of slack time. Nothing to do. Then at the end there is a mad rush to get the project completed in time. By doing the work in small chunks (sprints) you should have a constant flow of work.

Kaizen : Continuous improvement (culture) – This is where you try to continuously improve the system. Try something, try it at a scale that will not cause significant impact, see if it improves the situation. If it does, decide whether to adopt the changed process or not. If there is no improvement, determine whether or not to modify the trial procedure or drop the change completely. Don’t be afraid of failure, as each failure should be treated as a learning experience. Kaizen is one of the methods where PDCA (Plan Do check Act) learning cycle is used to try improvements.

Genchi Genbutsu : go and see for yourself (culture) – I see it a lot, I’ve done it myself a lot of times. I would come up with a theory on how something works. Or why it’s not working. The best way to determine how something works is to actually run it. Try it.Be it an API, a new technology etc. With the advent of Open Source and trial software, it’s easy to try products out yourself and see if they work for you.

Nemawashi : laying the groundwork or foundation for consensus (culture) – This I think is when the Team decides. It can be in the form of team meetings where you discuss the use of a new process, plan the work to be done. Just get everyone together and discuss.

kanban : signboard (culture) – This is where the Kanban board got its name from. With the Kanban board, you have a card which represents a piece of work. As it flows through the board, you see it’s progress. This gives a good visual queue to anyone who may be passing by as to the progress of any single piece of work.

gemba : the place where the actual work is done (culture) – This is a little tricky in Software development. In manufacturing, it’s simply the factory floor. In software development, it’s done at people’s desks. On the computer and in their heads.  For managers though, if you want to know what is happening with your people, go out and talk to them. Don’t just rely on reports, the wall, burn down charts etc.

zero quality control (culture, automation, measurement, monitoring) – This I think is part of the “Shift Left” movement where quality is being pushed towards the developer. The idea being that you build in quality as part of the process. Not as a process you do at the end of the development cycle as is done with traditional testing. Zero Quality Control is an asymptotic goal. A goal that is to be strived for but may never be reached. By continuously improving and trying to remove the cause of defects (not the actual defects themselves) you work towards the goal of never having any defects.

kaikaku : radical improvement (culture) – This is where significant change happens. It can mean moving from Waterfall to Agile, Agile to DevOps. Changing company direction. Anything that has a significant, radical change to they way work is done. This is opposed to Kaizen which is incremental change.

 

This is only a quick overview and my own explanations of what I think these terms mean to me. I may at some point in the future go through these in more depth.  I really need to re-read the Toyota Way. This time around, take notes as it sticks in my head better.

Water Levels

One of the things that I think is missing in Agile training is that agile highlights problems and that through learning and solving those problems, you get faster and better.

The flow of work is a river. If the river flows slowly, the deeper it is. 


Traditional methodologies such as Waterfall slow things down. This hides problems. They remain hidden under the surface and are never addressed.

The same thing can be said with Agile, if you do not work towards lowering that water level. For example, focusing only on getting the work done instead of focusing on improvement and learning as well. 

So, how do you lower the water level? Well, you push the limits. You try to go faster. Try something different. Experiment.

For example, If you go a little faster, what happens? If you continue to do things the same way, only faster, you start running into problems. You make compromises, either in quality or scope. For example, code gets messy. You leave out documentation or testing. In these cases, you are not getting better. In fact you are getting worse.

So what do you have to do to keep the quality up? Well, remove waste, get better at getting better, find different ways of doing things. Find out what is valuable and work towards that. 

This of course is easier said than done, but that is why Agile is considered hard and not easy.


As you lower the water level, you start to see problems. This is a crucial time. Most companies balk at this stage and say that Agile doesn’t work. Those that are truly successful in agile fix these problems and move on.


As you get faster, better, you start to see more problems. Solve these problems and continue to get faster and better.

Keep going.

There will always be problems. Those that continue to solve the problems and get better, become highly successful. Those that don’t remain mediocre. They may be successful to a certain extent, but not as successful as those that embrace continuous learning and improvement.

Apologies for the crude diagrams. My artistic ability isn’t that great. I recently got the Apple Pencil for my iPad Pro that I’m using to write this post. 

Hot Potato

There is a Childs game called “Hot Potato” where a small object such as a bean bag is tossed around (as if it was a Hot Potato) while music plays. When the music stops, the person with the Hot Potato is out.

There is a communications method I would like to call the Hot Potato method where when you try to disseminate communication, you do a blast. Such as an email to your group, a poster on the notice board or some other passive means. Why would I like it to be called the Hot Potato method, its simply because it seems that the goal is to get rid of the communication as easily and quickly as possible.

My thoughts are that this method of communication is fine, if the information is inconsequential.

If the information is important, then I suggest other means be employed.

Why? well, if you disseminate information that is not currently relevant to all individuals at the time, then you run the possibility that it will be ignored or forgotten when the time is crucial.

For example, you have a procedure for a particular circumstance that comes around once a month. You send an email out to your team a month beforehand. If they are inundated with a number of emails per day. They are going to read it and mentally ignore it, or ignore it in the first place. Its 30 days away. They are busy, its not going to be at the top of their thoughts. Then a month later, the procedure needs to be implemented. You are going to have people who have no idea what you are talking about. You can complain, and say that you sent an email, but it was sent at a time that it was irrelevant for those people.

A quote by George Bernard Shaw comes to mind in this circumstance. “The single biggest problem in communication is the illusion that it has taken place”.

So what can you do? Well, my thinking is that its not going to be easy and there may be different methods to try.

For example, timing might be everything. A day before the procedure needs to be performed, notify then. Not a month before hand. Physically go to the people affected and let them know about the procedure. A conversation may stick more in their mind than an email. Teach those affected. Write a tutorial, doing sticks more in the mind than telling.

The whole point is that if you really, really want to communicate something to someone, you have to be pro-active, you cannot be passive. You have to get their attention. Tell them and/or teach them. Get confirmation that they have received the information. Reinforce the information.  If after all that, you still fail to get the message across to all relevant people, re-evaluate and try something different.

People retain information through different means. Some through written word, others audibly, others through pictures and diagrams. Some, by doing and some a combination of all the above. It becomes your job as the person who needs to communicate the information to make sure it is properly communicated as only you know if its importance.

 

The 7 Principles of Bloat Software Development

No doubt you have heard of Lean Software Development Agile and this DevOps stuff, well, it doesn’t work. Bloat Software Development is the way of the future.

In this post, I will go through the 7 key principles of Bloat Software Development and you can happily delay your projects indefinitely.

1. Incorporate Waste – This isn’t as bad as it sounds. You are not systematically trying to delay by slacking off. No, you delay by adding manual processes (If you haven’t got a process for adding a process, you need one), increase lead times, do incorrect work or add unrequired features. Also, don’t forget to document everything up front. The more pages the better. There is nothing like a 700 page requirements document that says nothing about the end product. Automation is good too. Have a task that will save you a small amount of time, automate it, but spend 18 months for its automation. 6 months sitting on it(we should automate that), 6 months development, then trying to perfect it for another 6 months before releasing it to your development team (if ever).

2. Check Quality at the End – You have to make sure that everything works right. The best time to check quality is after development. Hand off the finished product to the testers. Make sure that the testers have no idea what they are testing. (See principle 3, obscure knowledge) and hand back a bunch of defects back to the Developers. After a few rounds of this cycle, you will eventually go to User Acceptance Testing, only to find out you have built something completely different to the customers requirements. You will then either go back to the drawing board, or the customer would have spent so much money that they will need to accept the final product, with a few major changes.

3. Obscure Knowledge – The whole idea here is not to lie or mislead, the whole idea is to describe the requirements as vague as possible.

One of the keys to obscuring knowledge is cyclic logic.

For example:

The purpose of this requirements document is to fulfill the requirements of the feature. The feature will fulfill the requirements as specified in this requirements document.

A true master of knowledge obsuration can make a document flow, without actually saying anything.

4. Get Commitment Early – Oh yes, we have have that for you in 12 months. When the 11th month comes along, then you let the customer know about the delays. Sorry, we have only just finished writing the requirements, it will take another 6 months. 5 months later. Sorry, development has been delayed, you have changed the requirements based on the original requirements document, it will take another 6 months…… You get the drift.

5. Deliver Eventually – That is unless the project is canned before it is completed.

6. People are Resources – You can chop and change people at any time. They are mere cogs in the machine. Add more people when deadlines are looming. Work your people 24 hours a day. So what if they are exhausted, sick, tired. They bounce back. If they don’t – replace them.

7. Optimise the areas of least return- If you have a task that takes 10 days out of a 12 month project, and you automate it (while having a single resource for 6 months working on the automation) so the time is halved to only 7 days, it is well worth it.

/S

As much of a joke that this post is, sometimes some of this can be seen as the norm.

What Makes People Happy

There is a great talk by Dan Ariely on TED where he talks about what motivates people to work.

The simplistic view is that motivation = money. This isn’t always the case, but it seems to be what a lot of people think. The term “You’re getting paid good money to…”

This isn’t always the case, Dan goes through several experiments with psychology students to check out the findings.

What he found is that the more meaning you have for your work, the more you will love it, and be more productive.

He also looked into other methods of motivation. How we feel if our work is not acknowledged or ignored completely. He found that ignoring work gave the worker the same amount of de-motivation as if you destroyed the work in front of them. Simply acknowledging the work, i.e. Saying “Good Job” can significantly increase a persons motivation.

At the end of the talk, Dan goes through that motivation is money, meaning, creation, challenge, ownership, identity, pride etc.

This ties in with the book “Lean Thinking, bu Womack and Jones”  Pg 65 of the first edition talks about what makes a person happy.

  • A Clear objective.
  • A need for concentration so intense that no attention is left over.
  • A lack of interruption and distractions
  • Clear and immediate feedback on progress towards an objective
  • A sense of challenge.
  • The perception that ones skills is adequate, but just adequate to cope with the task at hand.

This makes sense. Have you ever been in the “zone” while developing. Time seems to fly by, you get a lot done, but don’t know where the time went.

This is what I like about Agile. You have a plan, clear objectives that you have worked out with the product owner. Using Specifications by Example/BDD techniques, you understand what you need to accomplish. You work in a small dedicated team of professionals, any interruption is on task. you also get feedback quickly. With the short iterations of scrum, you get feedback fairly regularly and you can implement that feedback. 

Then you have the opposite, which despite best intentions, I see in traditional Command and Control organizations.

The following has been taken from a talk by John Willis and Damon Edwards from the DOES 2016 conference with my own explanations.

The recipe for burnout

  • Work Overload – Have you ever been on a death march project. Too many tasks on your plate you feel overwhemed or even just ridiculous deadlines. 
  • Lack Of Control – Having multiple bosses telling you to what to do, therefore context switching all the time. Inability to do tasks the way you want to. For example, not allowed to improve but must follow a process.
  • Insufficient Rewards – As we mentioned above from Dan’s talks, even just a simple acknowledgement can go far. But having them too few and far between can do wonders to demotivate.
  • Breakdown of Community – Basically having a non supporting work environment. You would like to try different things (but still get you work done) but no, you can’t because your boss doesn’t like it. You want to try TDD, but told “No” it doesn’t work. Sigh.
  • A sense of Fairness – Your view is consistently not taken on board, or worse still dismissed. Someone else comes up with the same thing, and then it’s implemented.
  • Value Conflcts – A mismatch between the organizations values and individuals. For example, the organization is Waterfall, but you want to try Agile.

 To put it simply, you don’t need significantly high renumeration to keep your people happy. Just give them enough challenging work and get them involved. Acknowledge what work they do do, whether used or not, so long as it’s generally relevant or with good intentions. Help them guide their own destiny. Give them the ability to explore, with full backing, but keep them on point. Give them a goal to strive towards whatever way they wish to try. There will be constraints, but removing as many constraints as possible and giving them some freedom. I like the concept from “The Toyota Way” where Toyota builds everyone up to be a leader. Not just the chosen few. 
Finally, inspire your people. Inspire your colleagues and avoid situations of burnout. It is not a badge of honor.  

Scrum for Cars!

I have just watched an interesting TEDx talk. Joe Rainier talks about wikispeed. An organisation he started where the goal was to build a car that can go 100 mpg. Be road worthy and safe.

They did it!

They used scrum and software development techniques to iterate over a 7 day period. They had their first fully working prototype done in 3 months, and an auto show version done in 6 months.

The video is only 10 minutes long and I think its well worth a watch.

Black Swans

I have just started “reading” Black Swan by Nassim Nicholas Taleb, I just the term “reading” loosely as I’m not quite reading the book, I bought the Audiobook version.

What is a Black Swan

According to the book, a black swan is an event that satisfiles the following criteria.

  • It is unexpected to the observer.
  • It has a major event after it first occurs
  • It is rationalized by hindsight as if it could be expected. The event that Taleb uses for the name sake could be considered a black swans.

From Roman times till the 16th century, Europe only had white swans. By all empirical evidence at the time, all swans were white. By the 16 the century, it was considered that to have a black swan was impossible. It went into common language at the time, something was “as impossible as a black swan”. Then Dutch explorer Willem de Vlamingh became the first European to see a black swan in Western Australia and then brought a pair back to Europe. This changed the perception and subsequently, the saying changed to show later “perceived” impossibility might later be shown to be disproven.

I haven’t finished the book yet, but it does talk about trying to avoid black swans that are bad and exploiting those that are good.

My thoughts for this are, for example, Netflix with their chaos monkey that randomly kills servers is one possible way to try to avoid negative black swans. By causing unpredictable events, learning from them and then trying to prevent them, Netflix’s systems become more resilient to actual black swan events such as unexpected disruptions.

On the other hand, exploiting the black swan can be seen as experimentation. Trying something different, something new and if it works, then exploitng it, and innovating.

The book “Adapt: Why Success Always Starts with Failure : Tim Harford” also goes down this line, where constant experimentation, trying new things increases your chances of finding something new and exciting. A positive black swan.

This can be seen through the Unicorn companies such as Netflix, Facebook, Etsy, Google, Amazon etc. These companies continuously try something new. They deploy new functionality multiple times per day. Get feedback and then determine if they need to keep the new functionality, drop it or modify it and try again.

On Audiobooks

I am finding it a little difficult to actively read an audiobook. By actively read, I mean take notes while reading. I find it easier to do with a written book as opposed to a spoken book. I find the audiobook reading a little more passive. Hopefully enough will sink in that I will be able to produce more posts.