 |
A critical concept that motivates full-lifecycle
testing is the cost of change.
Figure 1 depicts the traditional cost of
change curve for the single release of a project
following a serial (waterfall) process. It
shows the relative cost of addressing a changed
requirement, either because it was missed or
misunderstood, throughout the lifecycle.
As you can see the cost of fixing errors
increases exponentially the later they are
detected in the development lifecycle because
the artifacts within a serial process build on
each other.
|
|
For example, if you make an requirements error and find
it during the requirements phase it is inexpensive to
fix. You merely change a portion of your requirements
model. A change of this scope is on the order of $1 (you
do a little bit of retyping/remodeling). If you do not
find it until the design stage, it is more expensive to
fix. Not only do you have to change your analysis, you
also have to reevaluate and potentially modify the
sections of your design based on the faulty analysis.
This change is on the order of $10 (you do a little more
retyping/remodeling). If you do not find the problem
until programming, you need to update your analysis,
design, and potentially scrap portions of your code, all
because of a missed or misunderstood user requirement.
This error is on the order of $100, because of all the
wasted development time based on the faulty requirement.
Furthermore, if you find the error during the
traditional testing stage, it is on the order of $1,000
to fix (you need to update your documentation and
scrap/rewrite large portions of code). Finally, if the
error gets past you into production, you are looking at
a repair cost on the order of $10,000+ to fix (you need
to send out update disks, fix the database, restore old
data, and rewrite/reprint manuals).
Figure 1. Traditional cost
of change curve.
It is clear from Figure 1 that you want to test often and test
early. By reducing the feedback loop, the time between
creating something and validating it, you will clearly
reduce the cost of change. In fact,
Kent Beck
argues that in
eXtreme Programming (XP) the cost of
change curve is flat, more along the lines of what is
presented in Figure 2. Heresy you say! Not at all,
Beck’s curve reflects the exact same fundamental rules
that
Figure 1 does. Once again, heresy you say!
Not at all, the difference is that the feedback loop is
dramatically reduced in XP. One way that you do so is
to take a
test-driven design (TDD) approach (Beck
2003; Astels
2003). With a TDD approach the feedback
loop is effectively reduced to minutes – instead of the
days, weeks, or even months which is typical in serial
processes – and as a result there isn’t on opportunity
for the cost of change to get out of hand.
Figure 2.
Kent Beck’s cost of change curve.
Although many people have questioned Beck’s claim,
which was based on his own anecdotal evidence and
initially presented as a metaphor to help people to
rethink the things that they have believed for years.
Frankly all he’s done IMHO is found a way to do what software
engineering has recommended for a long time now, to test
as early as possible – testing first is about as early
are you’re going to get. To be fair, there’s more
to this than simply TDD. With XP you reduce the
feedback loop through
pair programming as well as by working closely with
their customers (project stakeholders). One
advantage of working closely with stakeholders is that
they are available to explain their requirements to you,
increasing the chance that you don’t misunderstand them,
and you can show them your work to get feedback from
them which enables you to quickly determine if you’ve
built the right thing. The cost of
change is also reduced by an explicit focus on writing
high-quality code and by keeping it good through
refactoring, a technique where you improve the design of
your code without adding functionality to it. By
traveling light, in other words by retaining the minimum
amount of project artifacts required to support the
project, there is less to update when a change does
occur.
|
|
|
Teams that pair together stay together. |
|
Fundamentally,
as
Figure 3
shows,
the
reason
why the
agile
cost of
change
curve
has
flattened
is
because
we
follow
techniques
which
reduce
the
feedback
cycle.
Agile
techniques,
shown in
green,
have
short
feedback
cycles
and
therefore
are at
the flat
end of
the
curve.
Traditional
techniques,
shown in
red,
have
longer
feedback
cycles
and
therefore
are at
the
higher-cost
end of
the
curve.
Figure 3.
Comparing
the
feedback
cycle of
various
development
techniques.
Figure 4
presents a cost of change curve that I think you can
safely expect for agile software development projects.
As you can see the curve doesn’t completely flatten but
in fact rises gently over time. There are several
reasons for this:
- You travel heavier over time. Minimally
your business code and your test code bases will grow
over time, increasing the chance that any change that
does occur will touch more things later in the
project.
- Non-code artifacts aren’t as flexible as code.
Not only will your code base increase over time, so
will your non-code base. There will be documents such
as user manuals, operations manuals, and system
overview documents that you will need to update.
There are models, perhaps a requirements or
architectural model, that you’ll also need to update
over time. Taking an
Agile Modeling Driven
Development (AMDD) approach will help to reduce the
cost of this but will not fully eliminate it.
- Deployment issues may increase your costs.
When it becomes expensive to release software,
perhaps you choose to distribute CDs instead of
releasing software electronically to shared servers,
your cost of change increases because you begin to
follow more conservative, and more expensive,
procedures (for example you may be more inclined to
hold reviews).
- You may not be perfectly agile. Many agile
software development teams find themselves in very
non-agile environments and as a result are forced to
follow procedures, such as additional paperwork or
technical reviews, that increase their overall costs.
These procedures not only increase the feedback loop
they are very often not conducive to supporting
change.
- We travel lighter. Agilists create
high-value documentation, or in other words far less
documentation than traditionalists do, and we don't
tolerate unnecessary bureaucracy. As a result
we can act on changes quicker because we have less
work to do -- in other words, we maximize the amount
of work not done.
Figure 4. A realistic cost
of change curve for agile software development.
An important thing to understand
about all three cost curves is that they represent the
costs of change for a single, production release of
software. Over time, as your system is released over
and over you should expect your cost of change to rise
over time. This is due to the fact that as the system
grows you simply have more to code, models, documents,
and so on to work with, increasing that chance that your
team will need to work with artifacts that they haven’t
touched for awhile. Although unfamiliarity will make it
harder to work with and change an artifact, if you
actively keep your artifacts of high quality they will
be easier to change.
Another important concept
concerning all three curves is that their scope is the
development of major release of a system. Following the
traditional approach some systems are released once and
then bug fixes are applied over time via patches. Other
times an incremental approach is taken where major
releases are developed and deployed every year or two.
With an agile approach an incremental approach is
typically taken although the release timeframe is often
shorter – for example releases once a quarter or once
every six months aren’t uncommon, the important thing is
that your release schedule reflects the needs of your
users. Once the release of your system is in production
the cost of change curve can change. Fixing errors in
production is often expensive because the cost of change
can become dominated by different issues. First, the
costs to recover from a problem can be substantial if
the error, which could very well be the result of a
misunderstood requirement, corrupts a large amount of
data. Or in the case of commercial software, or at
least “customer facing” software that is used by the
customers of your organization, the public humiliation
of faulty software could be substantial (customers no
longer trust you for example). Second, the cost to
redeploy your system, as noted above, can be very large
in some situations. Third, your strategy for dealing
with errors affects the costs. If you decide to simply
treat the change as a new requirement for a future
release of the system, then the cost of change curve
remains the same because you’re now within the scope of
a new release. However, some production defects need to
be addressed right away, forcing you to do an interim
patch, which can clearly be expensive. When you include
the cost of interim patches into the curves my
expectation is that Figure 1
will flatten out at the high level that it has reached
and that both Figure 2
and Figure 3
with potentially have jumps in them depending on your
situation.
What is the implication for modeling? Although it
may not be possible to reduce the feedback loop for
non-code artifacts so dramatically, it seems clear that
it is worth your while to find techniques that allow you
to validate your development artifacts as early as
possible.
Note: Portions of this article has been excerpted from
The Object Primer 3/e.
 |
|
The Object Primer 3rd Edition: Agile Model Driven
Development with UML 2 is an
important reference book for agile modelers,
describing how to develop 35
types of agile
models including all 13
UML 2 diagrams.
Furthermore, this book describes the techniques
of the
Full Lifecycle Object Oriented Testing
(FLOOT) methodology to give you the fundamental
testing skills which you require to succeed at
agile software development. The book also
shows how to move from your agile models to
source code (Java examples are provided) as well
as how to succeed at implementation techniques
such as
refactoring and
test-driven development
(TDD). The Object Primer also includes a
chapter overviewing the critical database
development techniques (database refactoring,
object/relational mapping,
legacy analysis, and
database access coding) from my award-winning
Agile Database Techniques
book. |
|
|
|
Agile Modeling: Effective Practices for Extreme
Programming and the Unified Process is the seminal
book describing how agile software developers approach
modeling and
documentation. It describes principles and
practices which you can tailor into your existing
software process, such as
XP, the
Rational Unified Process (RUP), or the
Agile Unified Process (AUP), to streamline your
modeling and documentation efforts. Modeling and
documentation are important aspects of any software
project, including agile projects, and this book
describes in detail how to
elicit requirements,
architect, and then
design your system in an agile manner. |
 |
|
The Elements of UML 2.0 Style describes a collection
of standards, conventions, and
guidelines
for creating effective
UML diagrams. They are based on sound, proven
software engineering principles that lead to diagrams
that are easier to understand and work with. These
conventions exist as a collection of simple, concise
guidelines that if applied consistently, represent an
important first step in increasing your productivity as
a modeler. This book is oriented towards
intermediate to advanced UML modelers, although there
are numerous examples throughout the book it would not
be a good way to learn the UML (instead, consider
The Object Primer). The book is a brief 188
pages long and is conveniently pocket-sized so it's easy
to carry around. |
I actively work with clients around the world to
improve their information technology (IT) practices as
both a mentor/coach and trainer. A full description of
what I do, and how to contact me, can be
found here.
|
