Ok, here is another chapter in my upcoming (hopefully) book on Agile.
This post is the raw text, it still needs refinement, but I just wanted to record my unfiltered thoughts first. At some point this will be reposted as an edited version, or I’ll just release the book if it is ever done.
Agile Methodologies
Most likely if you are thinking about going agile, you are thinking Scrum. Yes, Scrum is the most popular method of Agile, but there have been many that have been developed over the years. Some have stood the test of time and are still in use today. Others have dropped off to the wayside. Some are new comers and have managed to gain a following. In this section, I want to go through some of the agile Methodologies that are out there, but I will concentrate specifically on Scrum for two reasons. Firstly, it is the most popular implementation of Agile, and secondly, it is the agile methodology that I am most familiar with.
The Toyota Production System (Lean)
The Toyota Production System was the first Agile approach although it was developed long before the term Agile. It even pre-dates software development. In fact it started in the Car manufacturing industry. The approach was simple. Elimination of waste and continuous improvement. I won’t go through details of how it was developed, but eventually it was taken up by other car manufacturers in the 1980’s and 1990’s and changed its name to Lean as you could not have car manufacturers saying they worked the same way as Toyota. It wasn’t until Tom and Mary Poppendieck published the book “Lean Software Development: An Agile Toolkit” in 2003 that the Lean software movement took off, even though a majority of Agile Methodologies developed earlier used Lean Principles as a basis.
The lean Software development approach is summarized in the following principles.
- Eliminate Waste.
- Amplify Learning.
- Decide as late as possible
- Deliver as fast as possible
- Empower the team
- Build integrity in
- See The Whole
Lean Software Development Wikipedia Entry
The Lean Mindset – Tom and Mary Poppendieck
DSDM (Dynamic System Development Method)
DSDM was first developed in 1994 and was thought to provide discipline to RAD (Rapid Application Development) software development.
DSDM is an iterative and incremental approach to software development that has elements of the agile development approach and encouraged s end user/customer involvement of the process. DSDM is also where the MoSCoW prioritization method was developed.
MoSCoW uses the four following priorities.
M – Must Have Requirements
S – Should Have Requirements
C – Could Have Requirements
W – Won’t Have Requirements
DSDM is owned and managed by the Agile Business Consortium, a not-for-profit vendor independent organization.
Dynamic System Development Method
Extreme Programming (XP)
Extreme Programming was created by Kent Beck during his work at Chrysler Comprehensive Compensation payroll project which he started working on in 1996. He refined his approach and released the book Extreme Programming Explained in 1999. The premise of extreme Programming was to take the best practices of Software Development at the time and take them to extreme levels. At the time Extreme Sports was very popular. Principles of Test First development and Pair Programming were formalized through the extreme Programming
The lifecycle of Extreme Programming consists of 5 phases.
- Exploration
- Planning
- Iterations to Release
- Productionising
- Maintenance and
- Death
The 4 basic activities for Extreme Programming are
- coding
- Testing
- Listening
- Designing
The 5 values of Extreme Programming are
- communication
- Simplicity
- Feedback
- Courage
- respect
Extreme Programming Wikipedia Entry
Kent Beck – Extreme Programming Explained
Agile RUP (Rational Unified Process) or AUP (Agile Unified Process)
Agile RUP is a simplified version of the Rational Unified Process. The Rational Unified Process is an iterative software process originally developed by Rational Software in the 1990’s. Rational was later purchased by IBM in 2003.
AUP has seven disciplines.
- Model
- Implementation
- Test
- Deployment
- Configuration Management
- Project Management
- Environment
Philosophies
- Your staff know what they are doing
- Simplicity
- Agility
- Focus on high value activities
- Tool independence
- You’ll want to tailor the AUP to meet your own needs
Wikipedia Entry For Agile Unified Process
Kanban
Kanban in software development is a method for managing knowledge work. It is a visual method that specifically uses “pull” to control the flow. This is in contradiction of more traditional methods of work management where work is “pushed” into process.
Kanban was developed by David Anderson n his 2010 book “ Kanban” where he used the Theory of Constraints methodology at a project at Microsoft. The name Kanban comes from the Toyota Production System where it means “signal” and is used to signal the need for something.
A simple example of a Kanban system in manufacturing would be where an assembler might have 2 tubs of screws. The first tub he takes from, the second is standby. When he runs out of screws in the first tub, a card attached to the tub is taken to signal that the assembler has run out of screws. The assembler continues to use the second tub. The card, known as a Kanban card is then taken to inventory where the screws are kept. This signals the inventory to send out another tub of screws to the assembler, which now becomes his second tub.
That is not the end of the card though. That card is then sent to the purchasing department where it triggers he purchasing of another tub of screws. As a side note, Toyota does do orders at least daily for the work to be done the next day, if not more often. Very rarely do they order in bulk. That would only be for times when there is long shipping times for parts directly from Japan, at least here in Australia. Most parts are sourced locally, delivered either that day or the next day for the cars to be manufactured that day.
Back to software development, David Anderson modeled work in a similar manner. Cards represent the work that needs to be done. There are lanes that indicate the process the “work” needs to go through. The idea is that the worker pulls the cards as it progresses from one process to the next. Work in process is limited only to a maximum of 3 items per worker. This helps alleviate some inconsistencies. If a worker has more than 3 items in progress in a process, then the person giving the work needs to step in, and help that worker get back on track.
For example, a developer develops 4 features quickly in succession, but the tester is stuck on the first feature still. The developer stops further development and then helps the tester with testing. The developer may help test 2 features. Now when the testers work in progress drops down to 2, the developer goes back to development.
The advantage of the system is that it gets things done quicker. It also allows you to see where the problems are in the system and know where to add additional resources. Kanban is also very visual. You can see at a glance the work in progress, what is remaining to be done and what has been done. Especially when a physical board is used. Some people also like virtual boards. The advantage of a virtual board is that it can help teams with remote members, automatically generate reports and not be knocked over, but you loose the visibility. You have to “open” the software to view the board and thus it can fall to the “out of sight, out of mind” problem.
The Kanban system has been so successful that is is used on many agile Methodologies as a means to manage work to be done. Many teams that use agile of varying types can be seen to have physical boards around their office as a means to manage work, be they use Kanban itself or Scrum, XP or even Lean Software development.
Wikipedia – Kanban Development>)
Crystal
Crystal is a family of Agile methodologies such as Crystal Clear, Crystal Yellow, Crystal Orange and Crystal Red. The darker the colour, the more heavy the methodology. Each approach is used depending on the project size, team size, critically, project priorities etc.
Several of the key properties of crystal include teamwork
, communication and simplicity. There is an element of frequent reflection to adjust and improve the process. Like other Agile processes, Crystal promotes the frequent delivery of working software, high user involvement, adaptability and the removal of waste in the form of bureaucracy or distractions.
Crystal was started by Alistair Cockburn when he released the book “Crystal Clear: A Human-Powered Methodology for Small Teams” published 2004.
Wikipedia Entry for Crystal Clear)
DevOps
The DevOps movement was started by Patrick Debois in 2009 when he saw the live presentation streamed over the internet by Paul Hammond and John Alspaw at the 2009 Velocity Conference.
The talk, “10 Deploys Per Day, Dev and Ops Co-operation at Flickr”.
Through the twitter stream, Patrick sited how much he would have liked to be at the conference and through a joke was told to start his own conference.
Patrick did just that and in 2009 in Ghent Belgium, the first ever DevOps Days conference was held. Due to Twitters 140 character limit, the “Days” portion of the name was dropped and the hash tag #DevOps was born and has stuck since.
DevOps was created by people in operations who were frustrated at the way work was being done. They would see Agile being used by Developers, but would stop once the features were in production. There had to be a better way. And so people in operations started their own methodology which included culture, tools and philosophy. Technology such as Infrastructure as Code, Continuous Deployment and culture borrowed from other Agile processes such as Theory of Constraints, Lean. They embraced the teachings of W. Edwards Deming, Elyahu Goldratt and Taichi Ohno. Learned lessons from the Manufacturing industry from 30 years previous. They embrace Agile methodologies such as Scrum, but expand on the process to continue into operations.
DevOps promotes continuous delivery, automation of the mundane, the difficult and frequent deployments. Some companies deploy multiple times per day. Amazon, one of the most prolific users of DevOps deploys on average once every 11.6 seconds. DevOps encourages experimentation to try to improve processes and continuously strive to get better.
Some say that DevOps may be the next evolution of Agile. They may be right.
Damon Edwards – The History Of DevOps
John Alspaw and Paul Hammond – 10 Deploys Per Day
Amazon Deploys Every 11.6 Seconds
Scrum
We finally come to Scrum. Scrum is a methodology and framework developed by Jeff Sutherland and Ken Schwaber and presented to the OOPSLA conference in 1995. Scrum is an open source framework. You do not have to pay any licence fees to use Scrum, but you do need to follow the required portions of Scrum, otherwise as mentioned in the Scrum Guide, although you can use portions of Scrum, if you are not following the requirements as per the guide, you are not doing Scrum.
Scrum is a framework. It has been created to form a base process. Once you have the basics in place you can develop on top. Practices such as User Stories and Kanban Boards are examples of this.
Scrum consists of 3 roles.
The Scrum Master who is in charge of making sure that the team follows the Scrum principles. Without someone driving the principles, teams tend to regress to more familiar development processes such as Waterfall, or start doing the “Easy” Parts of Scrum without doing the hard parts, thus not getting the full benefit.
The second role is the “Product Owner”. The Product Owner is the advocate for the business. They provide the requirements. What the business wants. They are also in charge of what work is to be done which is in the form of an ordered backlog. The backlog is ordered based on what gives the most value to the business. The product owner does not dictate how the work is to be done.
The third and final role is the Development Team. This is a self managed , cross functional dedicated team who performs the work. The team is in charge of how to implement the requirements. There are no leaders within the team. The whole concept is very egalitarian.
In Scrum there are 4 ceremonies.
The Sprint Planning Sessions where planning of the sprint, a short period of development usually about 2 weeks.
The Daily Standup where the Development Team members synchronize on what they are doing.
The Sprint Review where the development team shows the stake holders what they have done during the sprint and finally the
Retrospective where the team reflects on the sprint. Determine what practices went well, what didn’t and try to find out how to improve on the next sprint.
This is another form of the feedback cycle.
Finally, there are 3 artifacts in Scrum.
The Product backlog.
The Sprint Backlog and the
Product Increment. A portion of the product that should be in a state that is releasable to production after each sprint.
We will go through the details of Scrum later in more detail.
Wikipedia Entry – Scrum Software Development)
Scrum Hybrids
Finally there are Scrum Hybrids. These are where 2 or more methodologies are combined with Scrum. Some notable hybrids are Scrumban, a combination of Scrum and Kanban and Water-Scrum-Fall where the waterfall methodology is combined with Scrum. It should be noted that Water-Scrum-Fall is not generally an accepted methodology in the Agile community, but is seen as an example of what can happen when the easy bits of scrum are combined with the Waterfall Methodology. Generally the benefits of this approach are minimal compared to properly implemented Scrum.
Not all hybrids are bad – Scrumban is considered a success, but you do need to know what you are doing to get the full benefits.
Notable Mentions
Other notable mentions are
The Theory of Constraints. Not considered an Agile Methodology, but can be adapted to Software development and in the case of DevOps has been incorporated. Released when Elyahu Goldratt released his Management teaching novel “The Goal”.
Lean Six Sigma. Another manufacturing methodology that can be adapted to Software Development that combines the elimination of Waste part of Lean with the Continuous Improvement parts of Six Sigma.
Six Sigma uses the DMAIC process for problem resolution where it stands for
D – Define
M – Measure
A – Analyze
I – Improve
C – Control