Is Going Agile Too Expensive?

I have heard the argument that agile is a con. A way to get the certification industry money and a case to get high paid consultants to fill their coffers.

I don’t think this is true for the simple reason that if you look and you don’t have to look hard, you can find resources to learn agile techniques for free. The way I see it, most agile methodologies such as scrum and Kanban are like open source. Free to use, but if you want more support you can pay for it.

For example, with scrum, you do not need to get a CSM or PSM to practice scrum. You can download the scrum guide for free. Learn from blog posts, videos, podcasts and do your own practice. If you want more support, then you can buy books. Still need more, do a course, still need more then bring in a consultant, Even if you do a training course or bring in a consultant, you still have to do the work. There are no short cuts.  The concepts of Agile simple simple, but they not easy.

As for the perceived requirement that you need a certification. Well, that started by a requirement by businesses. Not the people who developed the methodologies in the first place. A lot of businesses generally do not want to invest in their people. They are more likely to buy or rent expertise already than grow internally. They will try any shortcut whether or not it works in the long term. What better way to know if someone knows Agile or any other technology/methodology than to have them certified in such technology/methodology. People see the requirement, get certified so they can be hired. Right or wrong this becomes a vicious cycle that feeds itself and creates the requirement for certification. 

If you or your company is serious about being agile. Not for the kudos or bragging rights, but just to be better, I suggest learning the history, reasons and purposes behind the methodology.

The following resources both from a google search and my own resources should be of help.

Free Resources

Scrum

Conference Videos

Software Houses

There are software companies that want to share their methods, help people learn.

  • Mountain Goat Software
  • Atlassian (Yay and Aussie Company)
  • Most tech companies have a blog where they share their development methodology which is usually some form of agile.

Web Sites

Pod Casts

Paid Resources

Books

This set of books is specifically not Agile related, they deal with Lean and TPS and learning from mistakes. Agile/scrum have their roots in Lean. Agile is also a learning methodology, and to learn, you have to make mistakes.

Online Courses

This is only a handful of resources, there is probably a billion other resources out there that are free or cheap that will help you learn any type of agile methodology that takes your fancy.

I’m always interested in learning new sources. If you have a favorite free or cheap learning resources, please add them in the comments.

Disclaimer

I myself have gone through the Certified Scrum Master course. I had done this after spending about a year learning about Agile and DevOps using some of the above resources.  I did the course because I wanted the alignment of what I had learned myself, and unashamedly the piece of paper.

My Views On Automation

One of the things I like doing at my job is automating repetitive tasks. For example, when a particular error occurs as part of the morning health check, an email needs to be sent out to the business owners of the source application (I work in middle ware) to notify them of the situation. (I know I’m doing it wrong – Special Cause/Common Cause, Burn’t toast etc – but that’s the situation I’m in) the process was to manually, so extract the relevant data, compose and email and send it out. All in all, about 5 minutes worth of work. Not long in the grand schema of things, but very boring. So I automated it. Now an email gets sent out automatically every morning. We don’t have to worry about it anymore.

Now, many times when you want to automate a task, there is the argument of “is it worth doing or not” comes up. In the above example, we are saving a task that only took 5 minutes a day. How long should we spend automating this task. 1 hour, 4 hours, 1 day, 1 week? (For the record, it only took me about 20 min for first version, then about another 20 min over several days to get the formatting right) for something so trivial.  Then again, lets look at another scenario. Say you have a task that will take you 5 min to do. But to after automating it, the automated process itself takes an hour. Not the writing of the process, but the actual execution. Should it have been automated?

In both these cases, there are arguments against automation because the time involved to either build or run the automation take too long. In my view, you are looking at the wrong metric.

My view is that automation should be used primarily for consistency. Spending a day on writing an automated script that saves 5 min will mean that that process will be run consistently forever and a day. The secondary benefits are that you don’t have to worry about the process any more – at least until it is no longer required. You have the knowledge on how it was automated which can be applied to automating future tasks. Finally, if you are lucky, the task may run quicker.

The same goes for a task that takes 5 minutes manually, but 1 hour automatically. You have the primary benefit of having the task consistently done.  Human error has been removed.  If the task needs to be done in a quicker time, then spend the time optimizing the automation script.

Now,  there are always exceptions to this rule, there always is. You have to use good judgement as to when to and when not to apply the rule.

Now, how do you determine what priority to automate. I’m not taking here as a manager (I’m not one) but as a person doing the manual work. Well, I work on one simple question. “What annoys me the most at this point in time?”.  I then spend time outside of my normal tasks trying to make that process better. Then I repeat. For the more common tasks, they get fixed pretty quickly. If they are not fixed properly, it annoys me and I go through the process again. For tasks that have long periods between them, I start working on them, I may not finish, but I document what I have found for the next round the task occurs and then work on it a bit more then.
The benefit of doing the “What annoys me at this point in time?” is that it is something that directly affects me. It becomes personal and thus I’m more likely to fix it. That is not to say that management cannot assign improvements, but generally, they are not the ones doing the daily work, and therefore do not see the pain points.

Striving To Be Happy

Have you ever sat in one of those emotional retrospectives where everyone draws a happy or sad face depending on how they felt the sprint went, or a little graph to show how you felt during the sprint?

They can and do feel like BS, especially to programmers like myself who are introverted, would prefer to sit in a dark room all day coding rather than share their feelings, but there is some merit to measuring happiness during a sprint.

Now we have to do some clarification about what happiness is work wise. Its not playing around all day, browsing facebook or shooting each other with Nerf guns, its more when are you happy being productive. You know the feeling, when you are in the zone. Time seems to pass by without you noticing and at the end of the day, when its home time (you have to be reminded its home time as you are so focused) that you have a feeling that you have accomplished something. These days are as rare as hens teeth, but damn they feel good.

So what is the magical formula that brings us those days. According to Lean Thinking, it is when the following criteria are met.

  • A clear objective – In other words, you know exactly what you need to accomplish.
  • A need for concentration so intense, that no attention is left over to think about other things.
  • A lack of interruptions and distractions (This rules out the modern workplace 🙂
  • Clear and immediate feedback on the progress towards the objective. In other words, you have the feeling you are getting somewhere.
  • A sense of challenge. Not too much of a challenge. Your skills need to be just adequate enough to complete the task. Too hard and you get frustrated. Too easy and you get bored.

Jeff Sutherland, one of the co-creators of Scrum. In his book Scrum, The Art Of Doing Twice The Work In Half The Time summarizes it to 3 points.

  • Autonomy
  • Mastery and
  • Purpose

This definition I like more. Short and Direct. So what are these?

Autonomy comes from having a certain level of control over how you do a task or more likely how you accomplish a goal. The more vague the goal, the more the person has control on how you accomplish the end result. The more detailed the task, the less choices the person has on how to accomplish. The more they feel a cog rather than a thinking person.

There was recently a scientific study that discussed the mortality rate of a person based on the level of control they had over their job.  The findings were that if you have low control, in a high demand job, you have a 15.4% increase in the chance of death compared to a low control, low demand job.

For a high control, high demand type job, there was a 34% decrease in the odds of death compared to high control, low demand job. So having more control and a little bit of stress is actually good for you.

Mastery is where we want to learn more. Get better. The struggle for perfection. It is what gives us the sense of challenge.
When we are in the zone, we are learning. We are trying to figure out that little puzzle, put all the pieces together. Once they are all together, we go ‘ahh’. We have mastered that problem, we have accomplished  something and that little snippet of knowledge is tucked away for next time. Not to mention the dopamine shot we get to activate the pleasure centers.

Finally there is purpose. This is what drives us to keep going. Without a purpose all our work feels like it is being done for nothing. That purpose needs to be perceived, for without the perception of purpose, it doesn’t matter how important the job is, it will have no meaning to the person carrying out the task.

So why is happiness important? Quite simply, a happy worker is a productive worker. A Happy worker is a more accomplished worker and a happy worker is a more healthy worker.

I’d like to say, for your next retrospective that you go through and do an emotional check. Rather than draw happy faces, discuss the above points and see what can be done to increase the likely hood of reaching them, or even if they are relevant.

Please let me know how you go in the comments.

Technology Can Bring Benefits…

I have started listening to Beyond The Goal: Theory Of Constraints by Eliyahu Goldratt.

In the first chapter, Goldratt has come up with an explanation as to why, if tried, his Theory of Constraints has failed. He doesn’t mention this, he keeps saying that everyone who has tried TOC has succeeded, but I think his argument can be applied to Agile/DevOps or any other radical system/process/framework in any industry that fails to succeed. This is my interpretation of what he says.

The first thing he asks is the following….

Technology can bring benefits, if and only if it diminishes a limitation.

Note, The statement does not say that Technology that does diminish a limitation will bring benefit. This is why I have underlined the can. Its is not a will.

Now before you agree or disagree with the statement, Eliyahu does not say what the limitation is. It could be known or unknown.  If it is not known, is the statement still true?

So what is a limitation – well, its something that prevents us or limits us from doing something. When we as humans hit a limitation, what do we do? Give up and do nothing? Not likely, we work around it. We develop policies, rules, measurements, processes etc to accommodate the limitation.

Now, lets say we introduce a technology that eliminates this limitation. Yay, its gone.  Now, what happens if we do not change the policies, rules, measurements, processes etc that were developed to accommodate the limitation? Well, not a hell of a lot. Despite the fact that the limitation has been removed, we act as if it is still there. The limitation is virtually still there.

As an example, Eliyahu brings up the MRP (Material Resource Planning) software industry.  Before MRP software was developed, it was typical for a factory of 300 workers to have about 20 staff work on their material requirements planning. Calculations were tedious and error prone, they were done manually. It would take about a month to work out the requirements. So orders for materials were done monthly. Lets say you are a customer and make an order for 100 widgets. The factory will receive your order, it would then go into the system. The calculations would be made and if you were lucky, the materials for your order would reach the factory in a month. Then you had to wait for the factory to make your widgets and then finally you would receive them. If you missed the montly cut-off, you might get the orders fullfilled in 2 months.

Then the first MRP software was developed. One of the first companies to use MRP software was Black and Decker in 1964. At the time, MRP was in response to the Toyota Production System which was barely known in the west at the time. Black and Decker did so well with MRP, they became one of the most profitable companies in the world. You have to remember too that at the time, software was done by punch cards, not terminal screens like it is today. We’re talking a very basic system compared to what is available today.

It took time, but by 1975, about 700 companies world wide were using MRP software. These companies were getting the same results as Black and Decker. Reduced inventory, more profitable results etc. At this point, the rest of the world started to notice. More and more companies wanted MRP software. So, by 1981, about 8000 companies were using MRP software. Then the murmurs of not getting the same results started to happen. Most of these companies didn’t get the magical results. By 1984, this turned into a scream. 90% of companies that used MRP software were not getting the benefits. In some cases, companies were worst off.

No one at the time knew why. The going theory at the time was that in order to get the benefits, your inventory needed to be 97% or higher in accuracy. No ones inventory is that accurate. Then you were not trained correctly on how to use the software. So, vendors started selling training. An industry of certified implementers was developed. Still, many businesses were not seeing the benefits. It became so bad, that the cost of the implementation wiped out any savings. If there were any.

Since then, we have moved on to MRP2 (Which is where I started my career in the mid to late 90’s) and now ERP systems.

So, why did the early pioneers succeed where the later adopters didn’t.

Eliyahu has a theory, which he admits, he didn’t think of at the time all this was happening.

What is the most important part of the data? Its the orders. The new recipients of MRP software forgot about this. What they did was run their MRP system overnight – once a month. So the customer orders were still fulfilled a month or 2 later.  It was if the limitation of the calculation time was still there. The procedures and processes were not changed. So no net benefit. That is not to say that there wasn’t benefit, they no longer had 20 people taking a month to do the calculations, it could now be done overnight. But this benefit does not lead to the magical profits the early adopters were getting. So why did the early adopters get the benefits? These were the more forward thinking companies. They ran their systems more frequently. Fortnightly, weekly or every few days. They were able to get their orders in sooner. Get their products made sooner and get the end result into the customers hands sooner.

Eliyahu has come up with 4 questions to ask when looking into new technology.

  • What is the power of the technology?
    This is an easy question to answer. Just speak to the Vendors. They will talk all day about the power.
  • What technology does this technology diminish for us?
    This gets harder. Trying to identify what limitation is diminished. Remember that some are seen and obvious, some, not so obvious. You need to identify precisely what is affected.
  • What rules, policies, procedures, measurements etc helped us accommodate the limitation?
    It gets harder and harder. Here you have to identify what you do to accommodate the limitation.If you do not identify the rules, procedures, processes etc you run the risk of leaving them in place. Then you will find yourself still in the same position.
    In MRP’s case above, it was processing orders monthly.
  • Finally, What rules, policies, procedures measurements etc should we be doing now?
    Now that the limitation has been removed, how far can we go before we hit the next limitation?

So, how does this affect Agile. Well, I see Agile as another technology. It may not be a new piece of software, but it is a new way of doing things. If you are not careful and identify what needs to be changed from a traditional Waterfall implementation. If you just focus on the superficial such as stand-ups and Kanban boards, and not focus on the planning, or retrospectives (At least in Scrum’s case) where you try to continuously improve, then you are still working under the limitation or Waterfall. Yes, there are some benefits. Just like the late adopters of MRP software saved some money by not having 20 people doing the calculations (It really sucked for those 20 people being laid off) but the massive benefits were not realized.

 

Lunchtime Meetup Tomorrow

I almost forgot, Elabor8 have another lunch time meetup tomorrow at 12.
http://www.meetup.com/en-AU/Elabor8-Lunch-and-Learn-Melbourne/events/234930794/

I like these meetups because they are at a great time of day. Lunchtime. Other meetups at 6PM or later are a little hard to get to when your wife works at nights and you have a 4yo to take care of while she is working.

One Piece Flow – Part 2

I’m a little late this week, due to the Melbourne Cup holiday.

The following video I found on youtube is what inspired me for the card game from the previous post.

This video clearly shows that the smaller the batch, the quicker you process the work.

So, how does this relate to Agile, well, the sooner you can complete work – get it into production, or ready for production, the quicker you should finish your project. Think of each process as dev/test/preprod etc with the end goal ready for prod.

Don’t get fooled though, completing tasks themselves does not mean you are doing the above. For example, completing the development of a feature, completing another feature, completing another feature, then migrating to test. Testing feature 1, testing feature 2 etc. Where you have one card per task. ie, one card for developing feature one, then moving it to done once developed. In this case, you are still doing batches.

The above is for the life cycle of the feature. Not the task. If a feature is not being actively worked on, it should either be in the backlog, blocked or ready for migration to production or in production. Anything less and you are not doing agile.

For more information, look up continuous flow with regards to manufacturing, the Toyota Production System or Lean.

Another good book that explains this is “The Phoenix Project” which I’m currently re-reading (This time as an audio book).

One Piece Flow vs Batch Experiment

This is something you can do with your kids, or with a couple of other people in the office.

You will need a few things for this experiment.
– A deck of cards sorted in suits. I.e. All hearts, then all diamonds, spades then clubs.
– A stopwatch
– A pen and paper to record the results

What we are going to do is create a little assembly line.

The layout of the assembly line is as follows (pleased excuse my lack of drawing capability).

image1

The deck of cards is the work that needs to be done, and the people are the process that works on them.

The first run is a batch size of 52 cards.
The cards are put in the first persons inbox.
Then the person starts work on them. Before work starts, please start the stop watch.
The first person takes the first card flips it over then places it in the outbox. The flipping of the card simulates work being done.
They take the second card, do the same thing for all 52 cards.
It is important that this be done at a steady pace. No rushing, say about 1-2 seconds per card.
Once all 52 are flipped, the batch is moved to the inbox for the next person.
The next person does the same thing. Again at a steady pace.
At the end of 52 cards, The batch is moved to the inbox for the third person. Their job is to turn the card upside down, also at a steady pace.
Please record the following.
The time it takes for the first card to reach the completed queue.
The time it takes for the first suit to reach the completed queue.
The time it takes to process all 52 cards.

The second batch size is 13 cards. Please split the cards based on suits. You should now have 4 batches.
Again run the same process again starting the stop watch when the first card is picked up.
It is important again that a steady pace remain. No rushing. The same pace as done at the first round.
At the end of 13 cards , the first person can move the batch from their outbox to the next persons inbox.
They can then move the next batch to their inbox and start processing while the second person processes the first batch and so forth.
Keep going until all 52 cards are processed.

Please record the following again.
The time it takes for the first card to reach the completed queue.
The time it takes for the first suit to reach the completed queue.
The time it takes to process all 52 cards.

The third and final run is for a batch size of 1.
Take the first card of the deck, flip it over then place it in the outbox, then place it in the inbox of the next person.
That person processes that card while person 1 pick up the next card off the pile.
repeat again until all 52 cards are processed.
Please record the following yet again.
The time it takes for the first card to reach the completed queue.
The time it takes for the first suit to reach the completed queue.
The time it takes to process all 52 cards.

The whole experiment should take about 15 min, once done, we can talk about the results which hopefully if this works, you should find interesting. Please let me know how you go in the comments below.

Agile Antipatterns

What is an Anti-pattern. Firstly, it’s a pattern that you think will improve things, but in actual fact, it does the opposite. It makes things worse. Sometimes this is visible, sometimes it isn’t. The following is a list of Anti-patterns that I have observed.

Backlog

In scrum, the purpose of a backlog is to give an idea of the work that is to be done for the project/product to make it a reality. At a high level it is the product owners vision in coarse grain. When the team gets the backlog, they break down the requirements to determine what needs to be involved.

The undefined backlog takes many forms. One form is that the product owner is too busy to define at a high level what they want. They only define the immediate sprint or two sprints worth of work. This means that the team has no idea of the long term vision. They only have immediate goals. This does not give the team ownership of the tasks.

The product backlog becomes a list of tasks where the product owner has done the breakdown themselves. Again takes ownership away from the team. It also means that there may be no coherence between tasks giving the developer little context to go by.

The product backlog does not result in a working component. The best method of measuring an agile project is in the working software. Something needs to be running at the end of each interval (sprint) and be production ready. That way everyone along the SDLC is working on getting something done earlier. Developers develop a working instance, no matter how primitive, testers can test that primitive instance, with caveats, documentation is done etc and at the end, there is always something of ship-able quality.

The product backlog is not prioritised, or is prioritised in multiple streams even if they require the same skill set. I.e., the items in the backlog are “assigned” to individuals, and not the team. This again takes ownership of the development from the team, and the individual especially if the way the task is to be executed is dictated rather than worked out.

The final, which I have already mentioned is that tasks are assigned to individuals before the team has a chance to break the tasks down and understand what is involved.

Planning

In scrum, there are 2 planning sessions. Sprint planning 1 and Sprint Planning 2. Sprint planning 1 is to go through the backlog with the product owner and determine what stories or task the team will accept for that sprint. This gives context to the team so they are across what the product owners vision is and can get first thoughts of how long a task will take. It gets everyone in line with the product owners vision.

The second sprint planning session is where the team deep dives into the stories and does the nitty gritty design. Granular tasks are determined and this is where you get the cards for the sprint. It also serves the purpose of making sure that everyone on the team is across the details of what is happening. It allows team members to contribute to tasks even if they are not involved directly with the task. This also gives the team a sense of ownership on the execution of the task.

Where the Anti-patterns is, is when management or the team think that they know what they are doing and decide they do not need a planning session  or they need minimal planning. This could be to save time, reduce meetings etc. It is actually pretty rare that this is the case and what ends up happening instead is that team members have no idea what is happening during the sprint. Time gets wasted explaining the situation and somethings the situation needs to be explained more than once as one team member may ask the same questions as another.

Daily Stand Ups

Daily Stand Ups are there to show the daily progress of the work. You talk to 3 questions. What did I do, What am I going to do and what is blocking me.

Where the Anti-patterns occur is when the meeting becomes a status meeting. News is given and the conversation veers away from the progress of the work. This can lead to another Anti-patterns where Stand Ups take too long. I’ve been in Stand Ups that have gone consistently over an hour because of this. The purpose of the stand up is to make people uncomfortable by standing so the meeting is short. No more than 15 min. If it goes more than 15 minutes, people tune out.

Another anti pattern is giving a status update to the product owner or manager. This is a meeting for the team. It is to let the team know what you are doing and an opportunity to offer help if required.

Another Anti-pattern is talking about tasks that are not recorded either on the board or in the task management software. All work should be logged.

No Showcase

The showcase is where the working incremental, warts and all is shown to the stake holders. It can also serve as a mini handover session. If there is no showcase, then there is the “opportunity” to skip the working incremental requirement.

Retrospective

The purpose of the retrospective is to look back at what has been done in the last interval and see what can be improved. It is not wishy washy improvement either. It needs to be measurable.

A common Anti-pattern when teams first start out is that they struggle with the retrospective and find little to no value in it. So they think about dropping it or extending the period between retros in order to save time and do the work. When you do this, you miss the opportunity to analyse what you have done and figure out ways to improve.

Another anti-pattern is to change how often you have the retrospective. If you are not doing fixed length sprints (eg, maybe doing Kanban) then there is the tendency to hold off the retrospective until the end of a development phase, this may be months later. A lot can happen within those months, and without looking back over a short interval, you miss the opportunity to improve.

Retrospective when you first start them out, can be a little difficult. Its hard to know how to improve and what to improve without an Agile coach or someone who understands the process. What then happens is that the retrospective gets a little superficial. If you are looking back and not seeing any problems, then that is a problem. One thing that Agile does is try to highlight problems. Through fixing these problems do you get better. If things are getting a little superficial and you are glossing over the problems, then you are not doing retrospectives right.

Finally, not having any tasks set up to do that are measurable. A good guideline is SMART goals.

  • Specific
  • Measurable
  • Agreed Upon
  • Realistic and
  • Time Based

is a good standard for adding tasks to the backlog from the results of the retrospective. Again, this is how you improve.

Command And Control

When starting out with Agile/Scrum for the first time, it is not unusual for the product owner or scrum master to be a manager. Be it the team leader or higher. When this happens in these cases is that old habits of Command and Control kick in. The manager assigns tasks to individuals in the development team. Tasks are broken down based on hours of work. Not story points. In this case, the manager is taking ownership of the tasks away from the team. By assigning tasks, and worse breaking them down themselves, the manager has kept ownership, and the development team is just a cog in the engine.

This can lead to failure as the team is no longer empowered. They have no say in the solution and thus mindlessly do the work as required.  Yes, it is the scrum masters role to protect the team from this sort of thing, but if the scrum master has no power, and all the power is in the product owner, the role becomes moot.

Big Bang improvement

Another anti-pattern is big bang improvement. A good example is that you need automated deployment. This has been determined part way through a project. Big Bang improvement is spending the next 3+ months (in my case 12+ months) developing an automated deployment system without iterative or incremental use by the rest of the development team.

Part of Agile is getting value as soon as possible, even if its not full value. Some value early even if its not directly related to the production system is better than nothing short term and everything much much later.

You also run the risk that your efforts in getting the big bang solution in place do not live up to scratch and add only a little value in real life.

Another anti-pattern is delaying improvements until there is slack in the schedule. Slack rarely happens, and more than likely will never happen if you do not include the improvements early. The reason for this is that your technical debt racks up, you spend more time fixing the issues rather than improving.

There is an old joke, “How do you eat an elephant? One bite at a time!” but if you never take that first bite, you will never eat the elephant.

Education

If you do not know what is required for Scrum or Agile, and you only go by what you see superficially, you are going to fail. Learn what scrum is. Learn its history. Learn how it has been implemented in other industries. For example, it all started with Toyota. Read up on the Toyota production system. A good book is “The Toyota Way” by Jeffrey Liker.

If you jump in without understanding, you run the risk of doing Cargo Cult Agile.

Quality and definition of done

An agile anti-pattern that tends to be done is missing the definition of done, or skipping quality checks during the development process.

For quality checks, this could take the form of not doing unit tests, code reviews etc until after development is complete. This could happen days or weeks later and is usually skipped in the interest of time. Why is this a problem? Well, usually when problems are found out after development, the developer has to get back into the mindset that they were in while developing the feature. This takes time. Then they have to do the implementation again and recheck etc.

If the checks are done while development is still in progress, then you save that time that it took the developer to get back into the rhythm. This time saved is usually significantly shorter than the testing task in the first place.  There will be a productivity hit initially as you are not use to doing the checks while working, but you will find in the long run that adding the checks actually saves time.

The other anti-pattern is not defining a definition of done or acceptance criteria. Again, left off due to time constraints. But what happens here is that work comes back to the developer because it is incomplete. Why is it incomplete? Well, in this case, the developer didn’t know that they had to cater for that situation. Having a definition of done, or acceptance criteria gives the developer a perspective on what is required and gives solid checks that they can measure their code by.

Lack Of Long Term Thinking

What is lack of long term thinking? This is where the immediate problem is looked at and the overall solution is glossed over. This could take the form of doing the work immediately. You have to repeat the same task anther 4 times for other environments. Instead of thinking of ways you can quickly automate the process or even semi-automate the process to save a little time in the long run, while sacrificing the short term implementation, you just go ahead and brute force the implementation. This means that when you do the task in the other 4 environments, its going to take the same amount of time.

An example. Say you have to do a deployment from Dev to Test. It takes about a day to do the deployment. It is tedious work, but not that complicated.

If you spent 2 days automating the task, the deployment could be done in an hour. But you don’t have time, You need to deploy now. So you spend the day deploying from Dev to Test. Then when you have to deploy to Pre-prod, it takes you a day.

So far you have taken 2 days. If you had automated the task, it would have taken 2 days and 2 hours.

Now, you have to deploy to prod. If you do the brute force method, you would have taken another day. So 3 days all up. If automated, 2 days and 3 hours. A saving of 5 hours.

The more you have to repeat the task in the long term. Lets say, there is a bug fix and you have to redeploy to all environments again, the more savings you make time wise.

i think of this problem a bit like the poverty trap. For example, you cannot afford to go to a supermarket. No car or public transport. So you have to buy groceries from the local convenience store at significantly higher prices. Which prevents you from saving money so that you can afford to go to the supermarket to save money.

You need to transcend the work. Look at ways to improve the process in the long term. Not the short term. Do this continuously. This is called Kaizen, and is a major part of the agile philosophy.

In the anti-pattern, you are stuck in the poverty trap based on time rather than money.

Another anti-pattern is that of outsourcing. You need a project done. So you bring in outsiders to do the project. One the project is complete, they hand over to the BAU (Business As Usual) team, which then has the problem of having to understand what has been built. In the mean time, the people that have all the knowledge have long since left.

Another anti-pattern is buying pretty toys (tools) without knowing what your requirements are just because it looks pretty and simple. But no one ends up using it because it doesn’t meet the use case.

Lack of communication

Have you ever been in the situation where you feel that you have been left off something that everyone else knows about? A conversation may have happened at the “water cooler”, or the modern equivalent, the coffee machine, that may have affected you, but you were not present when the conversation happened.

Or have you been in the situation where you are told “You should know that”, but this is actually the first time you have heard this.

Or, an email was sent months ago. You brushed it off because it was irrelevant at the time and subsequently forgot about it, but when you ask about the topic, you are told that they sent an email – why don’t you know about it!

This anti-pattern is a big one. Communication can kill an agile project, especially when you are berated for not knowing something, Morale of the person goes down and so does their productivity.

You need to re-think your communication goals. It is not efficient to send out an email if only half your team takes in the knowledge, or do a hand over for something that won’t be used for 6 months. The efficiency isn’t in the dissemination of the knowledge, but in making sure that the recipient retains the knowledge.  this can take the form of a one-on-one session, writing details in a wiki page for future reference (Do not use email – it is a horrible interface for knowledge). Write a tutorial.

If something is discussed at the water cooler, get everyone else together and re-discuss the topic. Even if you don’t think its relevant to someone, it might actually be.

Make sure that everything needed at some future point is documented in enough detail that it can be picked up at a later date. For example 6 months down the track.  Make sure its in an easily searchable format – such as a wiki.

Finally, do pair programming or mob programming. This is an excellent method of getting everyone on the same page. you may think that it isn’t efficient as you have 2 or more people doing one task on one computer, but there is nothing better for handover than doing the work together.

Not making it a safe environment

This final lot of anti-patterns has to do with not making the environment safe. I don’t mean taking care of frayed power leads (although you should take care of those straight away), but I mean giving an environment where people can speak their mind without being shut down. Where dissent is tolerated or even encouraged – if its relevant to the job.

If you do tell those that have different opinions to shut up effectively, they may just do that. Then you lose your most valuable feedback – the check to see if you are doing things correctly or not. Or if you are going down the wrong route.

These are the anti-patterns (at least in my opinion) that I have encountered.

If you agree or disagree or have some of your own anti-patterns that you have encountered, let me know in the comments.

 

Testing Grid

The following table is a grid of testing terms that I put together to explain at least in my mind where these terms reside.

Since I work in the integration space, some are more inclined towards integration. If you see anything out of place, or missing, please let me know and I will include.

 

UNIT TESTING SYSTEM INTEGRATION TESTING
(SIT)
USER ACCEPTANCE TESTING
(UAT)
WHO Who is the stakeholder/interested party? Developer Current
Develpoer Future
Developer Current
Tester Current
Developer Current
Tester Current
End User/Consumer/End Point
(Also known as the 3 Amigos)
WHAT What is tested? Components/Services/Units Plumbing/Connectivity Does what the stakeholder expects
What level of detail? As much as possible to narrow down errors/defects Works end to end without issues Works end to end, data is as expected
What Code Coverage? 100% (Or as close to as possible – within reason)
– Error Paths
-Edge Cases
50% Enough to prove working 20-30% Enough to see that the requirements have been met.
WHEN When executed? – Development
– Regression
– Bug Fixing
– Working out what the code does
When more than 2 components are complete
(Currently done after development)
– Once End-To-End complete (Even partial)
(Currently done after SIT)
WHY Why is it done? – Verification Of Unit
– Boundary Conditions
– Regression
– Verify End-To-End
– Regression
-Verify works as customer expects
WHERE Where is it tested? – Development Environment – Build Environment
– Test Environment
– Stage Environment
– Preprod Environment
HOW How is it tested? -NUnit (Junit/xmlUnit/WMUnit etc)
– Stubs
– Mocks
– Virtual Services (CA DevTest, IBM Rational Tester etc)
– Commercial Frameworks
Test Driven Development(TDD)
-NUnit (JUnit, xmlUnit, WMUnit etc)
– Virtual Services
– Mocks
– Commercial Frameworks
– Actial End Points
– NUnit (Junit, xmlUnit, WMUnit etc)
– Commercial Frameworks
– Actual End Points
– Behaviour Driven Development (BDD), Acceptance Test Driven Development (ATDD)
How long should a test take to run? Seconds Seconds -> minutes minutes -> hours

 

I have added this grid to a page in this blog. I think it makes a good reference point. At least for myself.

Baking

Have you ever tried baking a cake? Let’s say you want to bake a cake, you need flour, eggs, sugar, milk etc. let’s say you look into the pantry, no sugar, but you have honey. Honey is sugar. I’ll use that instead. You need self raising flour, but you only have plain flour. So you use that instead. No milk, let’s use water. It’s a a liquid. Oh and I don’t have eggs, oh well. Let’s leave them out. You have no measuring cups or scales so you use your best judgement.

So you mix your ingredients together and put it in the oven for 30 min. Out comes this flat hard mess that tastes sort of strange.  You throw your hands in the air and say “Baking doesn’t work”.

The same can be said about when agile goes wrong. I have read that 73 percent of companies that are doing scrum are doing it wrong. The most likely reason I think is that they have left something out of the process or substituted something else in its stead expecting the same result. For example, saying “We don’t need to plan, we know what we are doing” or the product owner who also doubles as the manager assigns tasks to the team instead of the team working out the best way to do the work. Or worse yet, they make it up as they go along without understanding the underlying principles.
The project goes sideways and they blame Agile. Ummm, what do you expect. You cut corners and rather than learn a tried and true process that has been around for more than 20 years, you decided to go it alone without understanding.

Get up to speed. Learn why something works, don’t just ‘think how it works’ or go by your thoughts as gospel or feel your way through. Read a book or two on agile, listen to podcasts, watch videos, read the scrum guide. Learn.

Agile is suppose to make you more flexible, but it doesn’t mean ‘do what you like’. If you do, your project will end up like the cake.  Just plain wrong.

Thoughts, Tutorials and Learnings specifically with regards to Agile, DevOps and SOA/EAI