Project management is all about maximising productivity and efficiency while minimising resources and costs. This becomes especially important in the Agile world, where you need to deliver constant improvements to the customer. There are many approaches to achieving Agile Development, including Scrum, Kanban and extreme programming.
This article is aimed primarily at Scrum Masters, but there are valuable lessons for anyone leading or running software development projects. So, read on to learn how to achieve the seemingly impossible: doing less while delivering more.
The Scrum Masters’ myth of full task lists
The world is full of managers who subscribe to the myth that unscheduled time is wasted time. If your staff haven’t been assigned back-to-back tasks then they will be unproductive. To see why this is a myth, let’s look at the problem of doctors’ appointments.
Everyone knows that sinking feeling of arriving on time for an appointment only to be told: “please take a seat”. You know that you are now in for an unpredictable wait until the doctor is ready to see you. But why does this happen? Why, when we schedule the exact date and time in advance, do we end up waiting a long time? Let’s look at this example from a flow perspective.
In this model, patients are tasks that the doctor (the resource) needs to process. Usually, patient visits are planned one by one, not leaving any empty gaps in the doctor’s schedule. This is an attempt to optimise resource efficiency by making sure 100% of the doctor’s time is utilised.
| Tuesday | Wednesday | |
| 9:00 | John | |
| 9:30 | Tom | |
| 10:00 | Paul | |
| 10:30 | Anna | |
| 11:00 | Michael | Mona |
| 11:30 | Andrea | Phil |
| 12:00 | Stephen | Mike |
| 12:30 | Elton | |
| 13:00 | Alex | |
| 13:30 | Janet | |
| 14:00 | ||
| 14:30 |
In an ideal scenario, every patient comes on time and their consultation lasts no more than 30 minutes. Let’s ask ourselves: how often does this ideal scenario become a reality? The answer: almost never! The reason is that it is very difficult to predict how long the visit will last and what treatment or tests the doctor will need to perform for each patient.
The analogy with Agile software development
Why talk about doctors waiting lists in an article about software development? What is the relevance to Scrum Masters or Agile development? Well, as you can see, there is actually a clear analogy here. If we add urgent cases (expedited items) to the example, it looks even more similar.
The two mentioned factors are key: the unpredictability of work and expedited items. These factors are the main reasons why high resource utilisation does not equate to work optimisation in software development. Indeed, instead of benefits, we get big delays, long queues, low employee satisfaction, and high pressure.
What can we do about it?
Fortunately, there are better alternatives for achieving efficient delivery. We need to start by optimising flow efficiency and then moving on to resource utilisation. The quadrant chart below compares resource and flow efficiency.
Our aim is to reach box 4 at the top right, where both the flow and resource utilisation are at their most efficient. However, all too often we end up in box 1.
What does this mean in practice? It means that we first aim to make sure that we deliver early and often rather than keep everyone busy with 100% utilisation.
Achieving flow optimisation
Flow optimisation isn’t unique to software development. Indeed, it is a widely studied field of research. I always encourage fellow Scrum Masters to look to other fields for inspiration. One of the most important applications is in highway management.
A 100% utilised highway is simply one big traffic jam. No one can move anywhere, which is frustrating, expensive and increases pollution.
Highways engineers are constantly looking at ways to improve the situation. The old-fashioned answer was just to build more and wider highways. Sadly, this just seems to increase the overall volume of traffic and means any traffic jam is even more disruptive!
Instead, the better approach is to limit the number of vehicles occupying each stretch of highway. This can be done in a couple of ways:
- by controlling entry at the slip road, but this just moves the problem to the side roads
- by pacing vehicles on the highway using lane markings
To see why pacing works, think about what happens when you are travelling at full speed and suddenly come across a traffic jam. Every car now slows down rapidly, triggering a traffic wave that eventually leaves the highway at a standstill. When it starts to flow again, all the vehicles are closely packed and the traffic jam will stretch a long way.
Rather than allowing all the cars to bunch up and create a full highway, it’s more efficient to adjust the flow rate. This is done by pacing vehicles and keeping them at a correct distance. That means they have time to react to any problem, thus improving the overall efficiency of the highway.
Now, cars have time to react more safely to the heavier traffic and no one comes to a full halt. The result is a better-utilised highway. This may seem counter-intuitive, since it seems to create wasted capacity, but it is proven to lead to an overall increase in throughput over time.
“Just in time” and lean production
Over several decades, Toyota developed its Toyota Production System. This approach to manufacturing is sometimes called just-in-time. It is based on the principle of delivering parts to the factory just in time for them to be used. This has a number of benefits.
Firstly, it cuts the need to store parts, allowing factories to be smaller. Secondly, it allows flexibility in your supply chain. Thirdly, it means you can create a more predictable manufacturing process. This has since evolved into the Lean Manufacturing or Lean Production approach.
One of the basic principles of the Lean approach is to deliver just in time. This means that we need to constantly limit the work in progress and review our queue. This needs you to ask yourself the following questions:
- What is needed?
- When is it needed?
- What amount do I need?
This approach leads to a continuous reduction of lead time. In practice, that means that we can deliver more items in total with a shorter time needed for each of them. Or as Womack and Jones put it in their seminal book, Lean Thinking, it is:
“A way to do more and more with less and less.”
Little’s Law
Lean isn’t just a management idea. It is underpinned by flow theory. In particular, Little’s Law tells us that the average number of items in progress is the rate of items that we can process in a given time multiplied by the average time that it takes to process an item. Or more concretely, the average number of customers in the store L, is the effective arrival rate λ, times the average time that a customer spends in the store.
Applying this to software delivery
The key question now is how can we apply Lean Production to software development? Probably, the best-known approach is the use of Kanban. Any Scrum Master, Project Manager or Developer will be familiar with Kanban boards. These simply present the work in a number of columns. Typically, these might be:
- Backlog, where all remaining tasks for the release sit
- Sprint Backlog, which lists all the tasks for the current sprint
- Work in progress, which are the tasks being worked on currently
- With QA, which are completed tasks waiting to be tested
- Done, which includes all completed and signed-off tasks for the sprint
Typically, each organisation will have a slightly different Kanban board, but the principles remain the same for all of them.
Limiting work in progress
Kanban is a pull-based system—each column on the board pulls in work from the column to the left. It is essential to ensure there is always work available, but you also need to prevent work from piling up. Often, you will be under pressure to complete more work. However, as we saw, adding more tasks to complete within the same time actually leads to less work getting completed. The main tool Kanban uses to control this is to limit work in progress (WIP). Limits can be applied for each column, or across the whole board.
To see why this works, we can look again at Little’s Law. Within a development team, the rate of processing items is largely static. Developers can only do a finite amount of work each. Therefore, the only way you can control the overall throughout is by limiting the number of items being processed. Interestingly, just increasing the number of developers can exacerbate the problem, since it directly leads to more work in progress. Thus the actual throughput may not increase unless you impose a new limit on WIP.
The need for prioritisation
Prioritisation is key to efficient sprint planning. You want to ensure you complete all high priority tasks first. But you also want to avoid completing all the work before the end of the sprint—that is a waste of resources. One popular approach is the MoSCoW analysis. Tasks are divided into:
- Must have, which have to be completed
- Should have, which ideally need to be completed
- Would be nice to have, which are optional
However, this can only be done properly if you work with the product managers to set the priorities.
Applying Lean in practice
Hopefully, you are now convinced that Lean can offer real benefits to the software delivery process. But you may be wondering how to actually achieve this. The prerequisite for achieving this is to optimise the process systematically (full product development). Local optimisation at the level of a single team isn’t enough.
The problem with local optimisation
Local optimisation happens when just one team decides they should try to change their practices. Or when each team decides to do their own thing to improve things. This results in every party only caring about their little island, which is just part of the process. As a result, the overall performance may even reduce with local optimisation.
This sort of local optimisation triggers its own set of problems. In particular, most of the waiting time now comes in between steps (islands). We can use our medical analogy again here.
In a hospital, patients are seen by one doctor but then get sent to another department for tests. Even if every department runs as efficiently as possible, waits will happen. This is because no department can predict how often or when a new patient will be sent to them. The result is increased waits and even resources standing idle. Better is to turn to your Scrum Master for help across the teams.
The important role of Scrum Master
To the uninitiated, the Scrum Master role may seem unnecessary. After all, this is just someone that seems to swan about between teams, arranging meetings and calls. It is all too easy to dismiss them as just another unwanted manager.
But Scrum Masters play a vital role in the whole Scrum methodology. Firstly, they act as a source of knowledge and experience for teams. This helps teams to avoid mistakes. Secondly, the Scrum Master has a global view of the process. This avoids the local optimisation trap. Thirdly, by being detached from the actual work, the Scrum Master can focus on improving efficiency across the whole organisation.
In effect, the Scrum Master is there to eliminate bottlenecks and improve the flow of completed tasks. However, it is important to ensure that scrum masters and project managers don’t just focus on their own teams. There are various techniques to help them see the bigger picture. One of these is “flight levels”. Here, there is a Kanban board at every level of the organisation, each with a different level of abstraction. Now, everyone can see how their work fits into the bigger picture and can focus on global optimisation.
The benefits of getting it right
If you successfully apply these lessons, you will see some real improvements in productivity and efficiency. These improvements will be seen across all the development teams.
- Better focus on the work that matters
- Fewer distractions from the current task
- Less complexity and less need to balance competing demands
- Fewer incidents because developers are less likely to make mistakes
All this just by applying common sense and lessons learned from other sectors.
Summary
As we have seen, maximising workload leads to a reduction in work done. High utilisation of resources causes problems whenever unforeseen or high priority work comes up. Instead, we should look to optimise flow efficiency.
This results in the ability to respond to customer needs more quickly with higher quality and more focused teams. In my role as an Engineering Manager at Luxoft, I see this constantly.
