Smarter ideas worth writing about.

Using Kanban in a Waterfall Shop

Tags: Agile Enablement

What does a strict waterfall development organization do when faced with a deadline that cannot be met using their existing process?

Maybe becoming more agile using Kanban would help.

The Situation

While the names are fictional the project described here is all too real. 

The “Helping Hand” agency within the state of “Jefferson” manages the delivery of certain services to eligible citizens. The process begins when a citizen enters their local Helping Hand office and provides a case worker with information relevant to their situation. At the end of the day the case worker sends a data file to the Helping Hand processing center, which processes the data overnight to determine if the citizen is eligible for assistance. Another process is then run to enroll everyone who is eligible. The case worker then receives a report the next day and follows up with the citizen to let them know the result. 

Helping Hand has for many years outsourced the support and maintenance of their processing system to “IQ”, a large well-known consulting firm. IQ’s contract with Helping Hand requires IQ to provide an agreed upon number of IT personnel to support Helping Hand’s system in exchange for a fixed payment. The system was originally created in the early ‘90’s, with many of the IQ consultants having worked on the system for ten or fifteen years. As a result, they have become intimately familiar with the inner workings of the system. 

The Helping Hand agency has a small team that oversees the work being done by IQ. Their job is to ensure that the state of Jefferson receives the best possible service from IQ. To support this mission, they implemented a very strict stage-gated process for creating and deploying changes to the system. The process includes the following phases: 

  1. The business analyst team creates a business system design document (BSD) that details the functional and non-functional requirements of the changes. 
  2. The development team reviews the BSD and the software code, then designs the technical solution. This is captured in a technical system design document (TSD). 
  3. Once the TSD is approved the development team can start coding the changes. 
  4. The test team creates test plans based on the BSD and TSD, and then performs tests and records results. 
  5. The operations/deployment team creates a plan for placing the changes into production. 

The process mandates that the output of any phase must be reviewed and approved by the Helping Hand team before the next phase in the process can be started (see figure 1). Any attempt to work on a phase without the previous phase having been approved is frowned upon by the Helping Hand team. Due to the volume of changes requested the Helping Hands team can take from several days to a few weeks to give their approval. 

The IQ delivery team has been organized to support the Helping Hand-mandated process by creating the following teams:

  1. Business Analysts (BA)
  2. Software Developers (3 separate teams)
  3. Project Managers (PMO) 
  4. Testers (QA) 
  5. Operations/Deployment 

Each team is in its own physical area with its own manager. While the managers meet weekly to coordinate their efforts, for the most part the teams work independently on their portion of the process as the work flows through. The Problem In an attempt to provide better service, Helping Hand decided to purchase a new web-based eligibility determination system to intake information from potential clients. Originally developed by MF Corp. for another state, the new system determines in real-time if the citizen is eligible for assistance. If eligible, the existing system would still be used to enroll them into the Helping Hand assistance program during the nightly run. The enrollment process itself would not change, but extensive changes would need to be made to it to interface with the new eligibility determination system. 

MF Corp. assured the state that only minor modifications would be required to meet their needs, and that the new eligibility determination system could be ready to go in six months. Each of the IQ teams reviewed the changes that would be required to integrate the new eligibility determination system into the existing enrollment system, and saw no technical challenges. Due to state contracting regulations, the two teams were not allowed to communicate without including someone from the state to ensure that state interests were protected, which limited their ability to collaborate. Each team would be working separately, with regular meetings scheduled with the two teams and the state to review any issues. 

The good news was that the IQ teams had a very good idea of what changes needed to be made because of their intimate knowledge of their system. The IQ team would be able to make the required changes to the enrollment system in parallel with the changes the MF Corp. team would need to make to their eligibility determination system. 

The bad news was that following the existing process of “create – review – approve” for gathering requirements, writing the code, and testing there was no chance that IQ could meet the six-month deadline. Something had to change to have any hope of meeting the deadline. 

The Solution

One of the development team leads had experience using agile techniques at other organizations, and suggested that the work might be completed on time using a more agile process. Since the IQ team knew what needed to be done for significant portions of the work, why not begin making the known changes right away? Could the process be changed so that work could be started immediately on what was known, and fill in the details for what was not known in parallel? 

Working with the PMO and the team leads, an alternative process was proposed using a Kanban board to visualize and manage the flow of work and improve collaboration between IQ teams. The developers would start work right away on what was known, allowing the team to quickly get feedback on any issues interfacing with the MF Corp. system. The BA team would begin working on documenting the requirements for what was not clearly known and start getting approvals, then go back and document the known changes later. The QA team could do the same, writing test plans for what was known up front, and then completing the remaining documents as the information became available. By eliminating the wait time for Helping Hand approval for every activity, individual changes could be completed much quicker (see figure 2). 

Kanban, one of several ‘agile’ software development methods, is based on lean manufacturing principles. One of the key lean principles is the elimination waste. Lean methodology identifies eight sources of waste, one of which is waiting time between actual work activities. It became clear to the IQ team that waiting on the Helping Hand team to approve work between phases was a waste that, if reduced or eliminated, may allow the project to be completed on time. The new process would still give the Helping Hand team the documentation they required and they would still have to give their approval, except that large portions of the coding and testing would already be done by the time of their review. If the IQ team was wrong about a requirement, they could quickly go back and make the change. 

A Kanban board was created on one of the conference room whiteboards with columns for backlog, in progress, done and approval tasks (see figure 3). Individual tasks were created for each document that needed to be created, as well as for the coding and testing activities. Different colors were used to represent which team was responsible for the work. The Kanban board made it easy to see progress and to also see any bottlenecks that needed attention. Borrowing from the Scrum practice of a daily scrum, representatives from the IQ teams met each morning to update tasks on the board and to coordinate their efforts. 

The Result

Removing the wasted waiting time allowed the IQ team to complete all of the modifications to the existing system on schedule. The Kanban board successfully kept the team on track and focused, and quickly made any blockers to progress visible. The daily scrum greatly improved the collaboration between the IQ teams. 

The MF Corp. team, however, was not so fortunate. Making changes to a system created for one state for another state turned out to be a bigger job than they thought, and a year past the deadline they had still not completed their changes. The project was eventually cancelled, at a significant cost to MF Corp. Maybe if MF Corp. had been a little more agile this story would have a happier ending.

Interested in learning more about Kanban? Join our Agile Leadership Series event in Cincinnati on February 23 to learn key Kanban concepts and principles through an intereactive team game.


About The Author

Scrum Master/ Agile Coach

Michael started using agile in 1998 when he stepped out of day-to-day software development and needed a process to help his team deliver projects successfully. He is a Scrum Master/Agile Coach and has taught in the graduate programs at OSU and DeVry University Keller Graduate School of Management. Michael has published several articles and books on business and technology topics. When not at work, Michael enjoys golf, home brewing, and traveling with his wife.