Managing Lean Software Development with Cumulative Flow Diagrams
mber of hours remaining to complete the Sprint backlog. More
recently burn up charts have become popular. These plot the number of completed
Stories, Tasks or Features on the project with a projected completion target. It is possible
to extrapolate the plot with a trend line to estimate the completion date of a project.
The concept of a burn up chart had been used in Feature Driven Development
since its inception in the late 1990s. Its called a Feature Complete Graph as shown in
Figure 1.
200
400
600
800
1000
1/3/
2003
1/17
/2003
1/31/
200
3
2/14/
200
3
2/28/
2003
3/14/
200
3
3/28/
2003
4/11/
200
3
4/25/
2003
5/9/
2003
5/23
/20036/6/20036/20/20037/4/20037/18/2003
Date
F
eatu
r
es
C
o
m
p
l
e
te
Figure 1. FDD Feature Complete Graph
However, burn up charts arent really sufficient for managing a project. They have no
concept of work-in-progress (WIP) and simulating the anticipated end date is problematic.
With Agile Management [Anderson 2003] I introduced Cumulative Flow Diagrams
(CFDs) as a better visualization tool. This paper explains why and shows how to use
CFDs to better understand the health of a project and to predict its outcomes.
Managing Lean Software Development with Cumulative Flow Diagrams
Borcon 2004 - Revision 1.0
The S-Curve
The completion of Features tends to follow an S-Curve model, as shown in Figure 2. The
S-curve effect makes it difficult to predict the end-date of a project from a single plot of
Features complete.
0
100
200
300
400
500
600
700
800
900
1
3
5
7
9
11
13
15
17
19
21
23
25
27
29
W eek
F
eat
u
r
e
s
Co
m
p
l
e
t
e
S-Curve
Max Rate
Optimal
Average
Figure 2. S-Curve for Features Complete
Bottom of the S-Curve
It is very difficult to sprint right out of the gate. It is probably impossible to
achieve optimum throughput in software development at the start of a project. There are
several reasons why this should be: feature inventory blocked due to ambiguous
requirements; training of personnel; lack of understanding of the project domain; poor
team working; changing designs; availability of tools; availability of environments;
facilities; lack of technical infrastructure; minimal domain specific tools; and staffing
levels. Some can be eliminated through better planning, others appear to be endemic.
They represent the nature of the beast software development.
Team Formation
When a new team is formed for a project, there will be an impact on
velocity. This is endemic to the business. The team must work through its
forming and storming [Tuckman 1965, Weaver 1997] phases before settling
into its new normal habits. As the team gets to know each others strengths and
Managing Lean Software Development with Cumulative Flow Diagrams
Borcon 2004 - Revision 1.0
weaknesses, there will be some friction. This friction will reduce the production
rate of the team.
Team Dysfunction
Occasionally, a team is dysfunctional. This will manifest itself as a low
production rate and a flat curve. Dysfunction may result in poorly executed design
or peer reviews, or petty arguments which block development. Monitoring the
cumulative flow diagram arms the manager with a lot of information which can
indicate a dysfunctional team, but only the manager can actually investigate and
discover dysfunctional behavior. It is important to understand that a dysfunctional
dispute between two individuals hurts the performance of the whole team.
Knowledge Sharing
Knowledge sharing is important to performance. A team which works well
together will share knowledge. A team which is dysfunctional and poor at sharing
will under perform. It is important to create an environment which encourages
individuals not to hoard information such as measuring and rewarding individuals
rather than teams. However, a full discussion of this is out-of-scope of this paper.
Encouraging openness and knowledge sharing and creating a means for
communicating the knowledge, such as a news server, or intranet knowledge base,
will improve performance of the team.
Tools & Environments
If a development team is ready to start a project but they dont have all the
tools and environments needed to make progress then the production rate will
suffer badly. Capacity constrained development resources must be exploited to
the full. They must not be idle because tools or environments are not available.
Hence, it is important that the project manager ensure tools and environments are
in place early. A flat line at the start of a project may be a strong indication that
the team doesnt have the tools or environments that it needs.
Ambiguity in Requirements
Design changes will hurt at any time in a project. When someone
discovers a requirement which necessitates the modification of a widely used
interface, the regression effect across the system, causes velocity to drop
dramatically. This problem is endemic. There will always be a little unforeseen
uncertainty related to analysis and design.
Top of the S-Curve
There are four reasons why the production rate tends to drop off in projects
regardless of software development method employed: increased integration effort; bug
fixes; inventory blocked due to unresolved issues; and refactoring.
Managing Lean Software Development with Cumulative Flow Diagrams
Borcon 2004 - Revision 1.0
Integration
Increased integration effort is almost unavoidable. As a project becomes
large, code from a new iteration must be integrated with the large existing code
base. The number of problems found during build-integration and integration
testing is likely to increase. The only way to avoid this is to never work on a large
project or system, though the effects can be mitigated through a good loosely
coupled architecture. Hence, large projects, even those built in smaller iterations,
should anticipate this effect and allow for it in the planning of the project buffer.
The effect of increased problems due to build-integration can be measured
by measuring the time from the start of the build until the build is declared
successful. This is the local lead time through Integration Test. The manager can
then assess what the overall downtime for the development team might be. In
extreme cases, all development stops until the build is declared successful. The
effect on the production rate can be calculated. The trend in the build-integration
time can be used to predict the future impact of build-integration on velocity.
Hence, the contribution of build-integration to the cumulative flow diagram can
be predicted.
It should also be possible to use a tool, such as Borland Together, to
calculate a metric for the degree of coupling in the architecture. The trend in this
metric could be used to predict the increase in integration time.
Bugs
As a project matures and more code is available for system and product
test, the number of bugs reported may rise. As the bug database rises, there is a
tendency to increase the mix of bugs to new client-valued functions being coded.
The result is that the overall speed of the team is maintained but an increasing
percentage of the effort is devoted toward fixing bugs. Bug fixes are related to
feature inventory which is already shown as code complete and hence no
additional value is earned on the cumulative flow diagram by working on them.
To minimize the effect of bugs on the code complete slope and to maintain
velocity, it is vital to maintain high quality standards. Fewer bugs have a direct
positive effect on the production rate of the system. The s-curve clearly shows
how and why high quality makes a project go faster.
Unresolved Issues
As a project nears completion, inventory which has been blocked due to
unresolved issues raised during analysis or design, re-enter the critical path.
Eventually all client-valued functionality must be coded, or the client must agree
to remove them from the scope. If they remain in scope, the issues must be
resolved. Issues which remain open near the end of a project, reduce the number
of client-valued functions which can be progressed to completion. The result is
that the speed of the team slows. Some developers may become idle, or more and
more effort is concentrated on bug fixes, because other inventory is blocked.
To avoid a tail off in velocity it is vital to resolve issues before the client-
valued functions re-enter the critical path. Hence, the project manager should be
focused on fast and effective issue resolution. This maintains the overall
Managing Lean Software Development with Cumulative Flow Diagrams
Borcon 2004 - Revision 1.0
throughput of the system. A flattening in the curve is a good visual indicator that
the project manager needs to work harder to resolve open issues.
Refactoring
Refactoring will also impact velocity, if it involves features already shown
as complete. When working code already completed is being re-worked then it is
not being shown as new productivity. For every client-valued function being re-
worked, the system is losing the possibility of a unit of new production. For this
reason, I prefer to plot refactoring as new dark matter features which are added
to the total inventory on the cumulative flow diagram.
Where refactoring is unavoidable due to a change in market conditions
which force new architectural requirements upon the system or through sloppy
coding which has li