Does adopting Agile approaches really improve your and your team’s software delivery performance? And if yes, which methodology is better? Scrum, Kanban, or eXtreme Programming?
It’s not easy to measure the positive impact of adopting Agile practices. No software project is the same. And going back in time to do it the other way is unfortunately impossible.
But there’s a great way of visualizing the impact of different Agile methodologies:
The Penny Game
The Penny Game is a fun activity for your team and helps to grasp the counter-intuitive effects of reducing batch sizes in production systems.
The goal of the Penny Game is to manufacture and ship 20 products represented by 20 coins, all showing either Heads or Tails.
Each coin (product) has to pass 4 workstations before it can be handed over to the client for inspection. Each workstation is represented by one team member (worker), and each team member has to flip each coin twice – one by one, using only their left hand.
Round 1 of 4
In the first round, the batch size will be 20 (equal to the amount of all coins). That means worker 1 has to flip all 20 coins in the batch before they are allowed to pass the batch on to the next workstation of worker 2. Then worker 2 has to flip all coins before they are allowed to pass the batch on to worker 3. And so on, and so on.
The rest of the team observes the workers and measures the time that passes until the client gets the first coin and the last coin (all coins).
| Round 1 | Round 1 | Round 2 | Round 3 | Round 4 |
|---|---|---|---|---|
| Batch size | 20 | |||
| First delivery | 5:17 | |||
| Last delivery | 5:17 |
In the above case, it takes the team 5 minutes and 17 seconds to deliver the 20 coins (products).
We have only one batch of 20 coins, so the first coin arrives together with the last.
Defect rate
I usually play the client, and this is where I typically spice things up a little:
As soon as I receive the batch of coins, I toss a coin and announce that I actually want Heads instead of Tails. I do this to simulate the ever-changing requirements in software projects.
If all coins show Heads and Tails randomly in the beginning, and if all the workers flip the coin exactly twice, I will, on average, receive 10 Heads and 10 Tails.
I actually receive 11 coins showing Tails instead of 20 Heads. That represents a defect rate of 55%!
| Metrics | Round 1 | Round 2 | Round 3 | Round 4 |
|---|---|---|---|---|
| Batch size | 20 | |||
| First delivery | 5:17 | |||
| Last delivery | 5:17 | |||
| Defect rate | 55% |
It looks like the team will have to spend some time on unplanned rework.
Waterfall
I like to compare this first round with only one batch to a software project that uses a traditional Waterfall approach.
In a Waterfall project, the project time is separated into distinct phases:
The batch of coins represents a batch of all features. All features will be planned together, implemented, tested, and delivered to the client as one final product (one batch) at the very end of the project.
Often, people from different departments are responsible for the various project phases. Between the departments, they use formal and time-consuming hand-overs to ensure that it’s not them who jeopardize the project.
Whether the features are of good quality is determined at the end of the project. There’s a manual testing phase, and the end-users can only provide feedback after they receive the product.
In a Waterfall project, there’s essentially one big feedback loop:
But the feedback arrives too late. Many features of the delivered product are faulty or, in their current state, unusable.
Waterfall projects often miss deadlines and go over budget because of unplanned additional work discovered only at the end of the project or after the product was delivered.
Round 2 of 4
Back to the Penny Game.
In round 2, the batch size is cut in half. That means each worker has to flip the first 10 coins and is then allowed to pass the batch on to the next workstation before they continue flipping the remaining batch of 10 coins.
The rest of the team again observes the workers and times the process.
As soon as the client receives the first coin (as part of the first 10-coin-batch), I toss a coin and announce:
“The client wants Tails; please ensure that all undelivered coins show Tails!”
The workers may now flip a coin a third time to accomplish that.
The first batch contained 5 unwanted Tails, and the team managed to deliver only Heads with the second batch.
Here are the results of round 2:
| Metrics | Round 1 | Round 2 | Round 3 | Round 4 |
|---|---|---|---|---|
| Batch Size | 20 | 10 | ||
| First delivery | 5:17 | 2:39 | ||
| Last delivery | 5:17 | 3:12 | ||
| Defect rate | 55% | 25% |
We first notice that the client receives the first product (coin) after 2 minutes and 39 seconds. That’s about twice as fast when compared to round 1.
And with 3 minutes and 12 seconds, it took 2 minutes less to deliver all products.
The defect rate went down from 55% to 25%.
Scrum
The second round can be compared to a project that uses Scrum as a development approach.
In Scrum, a cross-functional development team works on the product in 4-week iterations called Sprints.
Per Sprint, the team focuses only on a few features. The team plans, implements, and tests the features. And at the end of each Sprint, they deliver the set of features, and they receive feedback.
Earlier feedback and close collaboration (cross-functional teams vs. departments) help to reduce project time and increase quality significantly.
Round 3 of 4
The batch size is again cut in half in round 3 of the Penny Game. Now each worker has to flip 5 coins before they pass 5 coins to the next workstations and continue flipping the next set of 5 coins.
While the rest of the team tracks the process with their stopwatches, the client decides whether they want Heads or Tails. As soon as I receive the first batch of 5 coins, I toss a coin and announce, “Tails!”.
The first batch contains 2 coins showing Heads.
For the remaining 3 batches, the team is allowed to flip a coin three times to ensure the client only gets coins showing Tails.
Here are the results of round 3:
| Metrics | Round 1 | Round 2 | Round 3 | Round 4 |
|---|---|---|---|---|
| Batch Size | 20 | 10 | 5 | |
| First delivery | 5:17 | 2:39 | 1:15 | |
| Last delivery | 5:17 | 3:12 | 2:56 | |
| Defect rate | 55% | 25% | 10% |
We notice, with 1 minute and 15 seconds, the duration until delivery of the first coin is again slashed in half (a fourth when compared to round 1).
The overall delivery time is now less than 3 minutes, and the defect rate is only 10%.
Kanban
We can compare the third round to a project that uses Kanban as a development approach.
A Kanban team works in weekly iterations, and limits work in progress to avoid context switching.
Every week, they reflect on improving the development process, and they ask for feedback from their client or end-users.
A Kanban board helps the team visualize and optimize the work in progress.
No developer is allowed to work on more than one feature simultaneously (One-Piece-Flow). Before a developer can start a new feature, they have to ensure that no other unfinished feature is still work-in-progress that needs support (e.g., a pending code review).
An important Kanban principle is:
Stop starting; start finishing!
The result of even shorter feedback loops and even closer collaboration among the team members is a constant flow of fully-tested and integrated features throughout the workweek.
With Kanban, overall delivery time and defect rate are further reduced.
Can we top that?
So far, we’ve seen that reducing the batch size leads to shorter feedback cycles. And shorter feedback cycles lead to a reduction in delivery time and increased quality.
It seems reasonable to assume that if we manage to reduce the batch size further, the delivery time will go down, and the quality will increase even more.
But suppose the batch size in software engineering is already reduced to one feature at a time (Kanban).
What practices could possibly result in even shorter feedback cycles and closer collaboration?
Round 4 of 4
It’s time for the final round of the Penny Game.
This time the batch size will be 1 coin. That means each worker flips a coin twice and then passes it immediately to the next workstation.
As soon as the client receives the first coin (it shows Tails), I announce that the client wants only “Heads!”.
Here are the results of round 4:
| Metrics | Round 1 | Round 2 | Round 3 | Round 4 |
|---|---|---|---|---|
| Batch Size | 20 | 10 | 5 | 1 |
| First delivery | 5:17 | 2:39 | 1:15 | 0:36 |
| Last delivery | 5:17 | 3:12 | 2:56 | 2:42 |
| Defect rate | 55% | 25% | 10% | 5% |
After only 36 seconds, the client receives their first coin (product). It’s faulty, but the early feedback allows the team to deliver 19 coins of good quality. The defect rate shrinks to 5 percent.
The overall delivery time is 2 minutes and 42 seconds, again a bit lower compared to round 3 and about half of round 1.
eXtreme Programming (XP)
I like to compare round 4 to a project that uses eXtreme Programming as a development approach.
eXtreme Programming (XP) takes everything that works in software engineering to the “extreme.”
XP is based on agile values and principles, which are not much different from the ones of other agile approaches. But XP comes – unlike Scrum or Kanban – with a concrete set of software engineering practices that will supercharge software delivery performance.
Pair Programming and Test-first Programming are two of the more famous XP practices.
Pair Programming takes collaboration among team members to the next level. It means that all developers work in pairs most of the time. 2 Developers, 1 Computer, 1 Problem. The continuous improvement process happens all the time. A pair of developers constantly exchange knowledge and provide feedback and guidance.
Test-first Programming is the discipline of developing code in short cycles of 5-15 minutes. Each cycle begins with writing a test (that will fail), adjusting the code (so that the test passes), and refactoring the code (to ensure the code stays maintainable).
With Test-first Programming, the batch size is practically reduced to less than one feature at a time. Each test deals with one of the many aspects of a single feature. Each test drives the implementation of one feature closer to completion.
Summary
The Penny Game shows that reducing the batch size will lead to shorter feedback cycles. And shorter feedback cycles will profoundly impact delivery time and quality.
Agile methodologies try to accomplish the same for software projects. All Agile methods have in common that they reduce the batch size to get earlier feedback from the client. And to reduce the batch size, they foster closer collaboration among all the people involved in getting a feature in front of the client.
Scrum introduces the concepts of 4-weekly Sprints and cross-functional teams.
Kanban uses weekly iterations and work-in-progress limits forcing the team to work even closer together.
And eXtreme Programming (XP) takes closer collaboration and shorter feedback loops to the extreme with Pair Programming and Test-first Programming.
See you around,
-David