When I initially started work on Agile
Modeling (AM) I wanted to focus solely on principles and
practices for effective modeling but quickly discovered
that this scope was not sufficient, that I also needed
to consider the issue of how to be effective at the
creation and maintenance of documentation too. Some agile models will "evolve" into official
system documentation, although the vast majority will
not, and therefore it is relevant to discuss how to be
agile doing so. This article explores agile and
lean philosophies towards documentation within
In this article I address the following
documents, and source code
Why do people
What is the
relationship between documentation and project
When does a model become
What are the issues associated with
What does it mean to travel light?
When is a document agile?
What type of documents do you need?
Are agilists actually creating documentation and is
it any good?
When should you create documentation?
should you update documentation?
Can templates help?
How can you reduce documentation CRUFT?
When is documentation the best option?
practices for increasing the agility
The fundamental issue is
Agilists write documentation if that's the best way to
achieve the relevant goals, but there often proves
to be better ways to achieve those goals than
writing static documentation.
Document stable things, not speculative things.
Take an evolutionary approach to documentation
development, seeking and then acting on feedback on
a regular basis.
Prefer executable work products such as customer tests and
developer tests over static work products such as
plain old documentation (POD).
You should understand the total cost of ownership (TCO)
for a document, and someone must explicitly choose
to make that investment.
Well-written documentation supports
organizational memory effectively, but is a poor
way to communicate during a project.
Documentation should be concise: overviews/roadmaps are
generally preferred over detailed documentation.
With high quality source code and a test suite to back it
up you need a lot less system documentation.
Travel as light as you possibly can.
Documentation should be
barely good enough.
Comprehensive documentation does
not ensure project success, in fact, it
increases your chance of failure.
Models are not necessarily documents, and
documents are not necessarily models.
Documentation is as much a part of the system as
the source code.
primary goal is to develop
secondary goal is to enable your next
The benefit of having documentation must be
greater than the cost of creating and maintaining it.
Developers rarely trust the documentation, particularly
detailed documentation because it's usually out of
sync with the code.
Each system has its own unique documentation
needs, one size does not fit all.
Ask whether you NEED the documentation, not
whether you want it.
The investment in system documentation is a
business decision, not a technical one.
Create documentation only when you need it at the
point in the life cycle.
Update documentation only when it hurts.
Let's start with understanding the relationships
between models, documents, source code, and
documentation, something depicted in Figure
1. From AM's point of view a document is any
artifact external to source code whose purpose is to
convey information in a persistent manner. This is different from the concept of a model,
which is an abstraction that describes one or more
aspects of a problem or a potential solution addressing
a problem. Some
models will become documents, or be included as a part
of them, although many more will simply be discarded
once they have fulfilled their purpose. Some models will be used to drive the development of source code, although
some models may simply be used to drive the development
of other models.
Source code is a sequence of instructions,
including the comments describing those instructions,
for a computer system. Although source code is clearly an abstraction,
albeit a detailed one, within the scope of AM it will
not be considered a model because I want to distinguish
between the two concepts. Furthermore, for the sake of discussion the term
documentation includes both documents and comments in
Figure 1. The relationship between
models, documents, source code, and documentation.
documentation is an intrinsic part of any system, the
creation and maintenance of which is a "necessary
evil" to some and an enjoyable task for others, an
aspect of software development that can be made agile
when you choose to do so. There are several valid reasons to create
- Your project stakeholders require it.
The creation of documentation is fundamentally a
business decision, you are investing the resources of
your project stakeholders in the development of the
documentation therefore they should have the final say
on whether their money is to be spent that way, not a
technical one. If your project stakeholders request a document from you,
perhaps at your suggestion, and understand the
trade-offs involved (more on this later), then you must
create the document.
It is important to note that
(XP) is very explicit about documentation being a
decision. You should create
documentation only when your project stakeholders ask
you to? Preposterous
you say! Well,
my experience is that this isn't preposterous. Your
project stakeholders include a wide variety of people,
including all of the clients of your system, and
therefore they should have a reasonably good idea what
they want. Maintenance
developers, or someone representing them if they are not
in place yet, will request system overview
and their management will likely request user
staff will request operations documentation.
Yes, you will need to work closely with them to
determine what they actually need, someone is going to
have to decide to pay for the development and subsequent
maintenance of the documentation, and you may even need
to explain the implications of what is being requested,
but this is doable.
- To define a contract model.
Contract models define how your system and an
external one interacts with one another, some
interactions are bi-directional whereas others are uni-directional,
making the interaction(s) explicitly to everyone
involved. Contract models are often required when an external group
controls an information resource that your system
requires, such as a database, legacy application or
information service. The AM practice Formalize
Contract Models states that a contract model is
something that both parties should mutually agree to,
document, and change over time if required.
It is important to understand that the
development of a contract model should still be verified
by your project stakeholders-it is their money that
you are spending, and if they choose to go at risk and
not have the contract model in place then that is their
- To support communication with an external
group. It isn't always possible to co-locate a development team
and it isn't always possible to have project
stakeholders (or at least the ones you need at the time)
available at all times.
When you need to work with an external group of
people you need to find ways to communicate with them,
and shared documentation is often part of the solution
in combination with occasional face-to-face discussions,
teleconferencing, email, and collaborative tools. It is a mistake to use documentation as your primary means of
communication because it's far too easy to
misunderstand something that has been written, but it is
a good supporting mechanism.
A good way to think of documentation in this
situation is that it is your option of last resort.
Note that this in effect is an extension of the
practice Model to Communicate into the realm of
- To think something through. Many
people will write documentation to either to verify
for themselves some group work they had just been
involved with or simply to increase their own
understanding. The act of writing, of putting your ideas down on
paper, can help you to solidify them and discover
problems with your thinking. What appears clear and straightforward in your
mind can often prove to be very complicated once you
attempt to describe it in detail, and you can often
benefit from writing it down first. It is for this reason that I suggest that people
write comments before they write their code, a practice that I have been
following for years.
Your stakeholders must understand the total cost of ownership (TCO) for a document, and your stakeholders must explicitly decide to invest in that document.
This very likely differs from what you are
used to doing, which makes sense because I'm talking
about how to become more agile in your approach to
is this different, and why is it more agile? My experience is that developers in non-agile
environments are often forced to create documentation
for less-than-ideal reasons, often based on political
reasons and sometimes due to sheer ignorance, and
therefore may not have been allowed to be as effective
as they possibly can. Questionable reasons for creating
documentation, and how to combat them, include:
- The requester wants to be seen to be in
control. People will request documents, such as specifications and
documents that they can sign off on and say
"yes, go ahead and build us one of these."
Whenever I find myself in this situation I ask
the individual(s) requesting the documents if they
also want to be seen as responsible for the
project's failure because the development team was
too busy focusing on needless documentation and not
on building software. I'll then suggest that
instead of requesting documentation they should
instead request access to the software itself, even
if it is just an internal release of the software,
so they can provide constructive feedback about it.
They can still be seen to be an active
participant in the project and can do so in a
- The requester mistakenly thinks that
documentation has something to do with project
could be further from the truth.
- The requester wants to justify their
existence. This typically occurs when someone who is "dead wood" is
desperate to be seen doing something.
This is an insidious problem because the
requester often has what appears on the surface to
be a good reason for requesting the documentation,
it's something they've been doing for years, and
management often believes it's necessary.
To address this problem I will
requester what they intend to with the document, why
they need it, why creating that documentation for
them is more important than other work that my team
needs to do, and so on to try to determine the
actual value of what it is that they're doing.
These are valid questions to ask, albeit
uncomfortable ones for someone that doesn't add
much value to the development effort.
- The requester doesn't know any better.
Many people have been working in
organizations that have been following non-agile
processes for years, processes that were likely
documentation centric, processes that produced a lot
of documents for review throughout the process and
finally software at the end of it.
This is what they're used to so they are
simply asking you for something they've gotten in
the past. The
idea that you can produce software early in the
project, that it's your primary goal, is new and
often radical to many people.
- Your process says to create the document.
Although this isn't a problem with agile
software processes it definitely can be one with
prescriptive software processes.
The most common reasons for this problem
include people wanting to justify their existence
(see above), people not understanding the software
development process or at least the implications of
what they are requesting, and situations where the
primary goal is to bill for hours as opposed to
develop software effectively. Once again, the best strategy to address this problem is to
explore whether the creation of the document
actually provides value to your efforts.
- Someone wants reassurance that everything
is okay. Your
project stakeholders are investing significant
resources in your project team, they're taking a
risk on you, and they want to know that their
investment is being well spent.
To get this reassurance they'll ask you for
documentation, status reports or overview documents
perhaps, not understanding that you need to take
time away from your true goal of developing software
and not realizing that a better request would be to
see the software itself (as indicated earlier, they
don't know any better).
You need to recognize when your project
stakeholders are looking for reassurance, something
common at the beginning of a project if they don't
trust your team yet, and find an alternative way to
provide that assurance.
- You're specifying work for another group.
Although I have identified this situation as
likely isn't appropriate it's still a common
one for justifying the creation of significant
amounts of documentation.
Documentation is one way to communicate but
it isn't the best way. Try to find alternative
approaches, such as occasional meetings with the
other group or use of collaborative tools, to reduce
your reliance on documentation.
development contracts are routinely subject to
re-competition. This problem is endemic in
firms that work for government agencies, although
businesses will often threaten their contractors
with putting a project up for bid again if they
don't perform. So what? If your primary
goal is to develop software then focus on doing so
and you're much more likely to perform adequately
enough to keep the contract. The direct client
in this situation is often operating under the
misguided belief that if you don't perform they can
take the documentation that you produce and provide
it to the next contractor who will start from there.
This borders on delusional in my opinion. If
you're doing such a bad job that you lose the
contract chances are very good that you've also done
a bad job of the documentation and therefore the
next contractor will need to rework it. Even
if you've done a perfect job of the documentation,
yet still lose the contract, the next contractor
will very likely have people with different skills
and enough time will have passed that they will need
to revisit the requirements anyway. No matter
how you look at it, the next contractor is very
unlikely to take advantage of the documentation you
4. What is the Relationship Between Documentation
and Project Success?
There is no solid relationship between project
success and writing comprehensive documentation, and in
fact it is more likely that the more documentation that
you write the greater the chance of project failure.
Have you ever seen a project team write a comprehensive
requirements document, get it signed off by your
stakeholders, only to have the developers build
something else? Or the team actually builds to spec,
only to have your stakeholders say that's not really
what they wanted? Or the team delivers late or over
budget, even though everything looked good most of the
way through the project? Have you ever seen a team
create a comprehensive architecture model, have it
reviewed and accepted by really smart people, only to
see the architecture fail in practice anyway? Or the
developers simply ignore the model and then go off and
build it the way that they want to? Have you seen all
this happen under the guise of management oversight,
without management realizing what was happening until
long after it occurred?Figure 2 is
taken from my article "Examining
the Big Requirements Up Front (BRUF) Approach" which
explores the effectiveness of writing a detailed
requirements specification early in the project
life cycle. As you can see, it seems that the
traditional approach to requirements appears to put your
project at risk, at least where return on investment (ROI)
The effectiveness of a serial
approach to requirements.
Why do people mistakenly think that documentation is
a critical success factor in software development?
My theory is that in the 1970s and 1980s many
organizations moved their IT departments from a "code
and fix" hacking mentality to a documentation-heavy
serial waterfall process. When they did so their
success rate actually improved. We even see this
today with CMM/CMMI -- when you move from the code and
fix CMM(I) level 1 to levels 2 or 3, you do in fact see
improvements in productivity even though you've added a
lot more documentation development to your process.
They've "learned" that documentation improves the
software development effort, and were satisfied with the
answer. That's truly unfortunate.
The proponents of CMM(I), or other strategies which
inevitably end up to be documentation heavy, never seem
to ask the question of whether there are better ways to
work. Wouldn't focusing your efforts on writing
high quality software (e.g.
test-driven development which in effect produces
executable specifications) provide greater payback
than writing documentation? Wouldn't finding ways
to keep your code of high quality (e.g.
refactoring) provide greater payback than writing
documentation? Isn't the true value of
documentation the increase understanding of the problem
domain, or the solution domain in the case of
architecture/design, and not the documentation itself?
If so, perhaps focusing just on the modeling effort
agile model driven development) and not on writing
documentation be a better strategy?
On the surface the life cycle of an agile
model is fairly straightforward-
machine diagram for models. Models will start out as an idea-"we need
to understand how we're going to build this", "how
do the users want this to work", or "we need to show
what we're going to deliver"-that you either
choose to model or abandon (for the sake of our
discussion, any activity you choose to perform instead
of modeling counts as abandonment). The model starts out being temporary, one that
you intend to discard once it has fulfilled its
immediate purpose, the typical fate of the vast majority
of models created by agile developers. These models are typically hand-drawn sketches
which discarded once the idea has
been communicated to your intended audience. You'll
naturally update temporary models as needed, often
creating them, working on them, and then discarding them
over a short period of time: many agile models are
created be two or three people in a couple of minutes to
facilitate their discussion whereas other agile models
are created in modeling sessions of a several people
over a period of several hours. The interesting transition depicted in
3 is when you decide to keep a temporary model, to
make it permanent, to make it one of your project
team's official documents. A temporary model becomes a "keeper" when it
meets all of the following criteria:
There is a clear and value reason to make it
There is audience for which the model provides
Your project stakeholders are willing to invest in
having it made into documentation.
3. Life Cycle of an agile model.
Points #1 and #2 are driven by the principle
With a Purpose: you should have a valid reason
for creating the model in the first place but your
documents should also have a definite purpose as well. You should know who you are creating the
documentation for-perhaps a subgroup of your
development team, your users, or the team(s) that will
maintain and operate your system once you've released
it-and what they need from that documentation. The fundamental idea is that the creation and
maintenance of a document is a burden on your
development team, and if you want to increase
someone's burden you should be able to justify why. It's a simple as that. Documentation is a burden, and when people
recognize this simple fact they put themselves in a
significantly better position to consider it
effective documentation can provide significant benefits
to your project team and project stakeholders, and as
maximize stakeholder ROI
tells you the benefits should outweigh the increased
burden on your team. It's important to note that sometimes the
benefits are received by someone else other than the
people experiencing the costs, for example your
development team is impacted by the costs of creating
system documentation that your maintenance developers
benefit from. The
implication is that when you are considering the costs
and benefits of documentation you need to look at the
entire picture, the trade-offs of which are described in
the What Are the
Issues Associated with
Documentation? section. Point #3 is also driven by the principle
maximize stakeholder ROI and the concept that because
it is the resources of your project stakeholders that
are being invested that they should be the one to direct
how those resources are invested, for better or worse.
you can see in Figure
you can rethink your decision
about making a model permanent, often because you
realize that the benefit provided by the model is far
less than the burden of maintaining it. When this happens the model is either discarded
outright or more commonly the owners of that model
simply stop keeping it up to date and it starts to
"gather dust". Sometimes these models are revived months or
years later by the development team, the maintenance
team, or the "redevelopment team" if the system is
being rewritten. These
stagnant models are often reviewed, recognized as being
significantly out of date, and either then discarded or
used as a template from which to create a new version of
the model. Ideally
this new version is leaner than the original model,
because if the original model didn't provide positive
value to your effort then an updated version of it
following the same approach likely won't provide value
for doing so are discussed in the Section Strategies
for Increasing The Agility of Documentation.
Agile developers recognize that effective
documentation is a balancing act, the goal being to have
just enough documentation at just the right time for
just the right audience. To accomplish this you must address the following
Software development versus documentation
development. This is the fundamental issue that you need to grapple with
-any time spent creating documentation is time spent
not developing new functionality for your users.
At one end of the spectrum are projects where no
documentation is written at all whereas at the other end
no software is written at all, neither extreme is likely
to be appropriate for your situation.
Remember that your
primary goal is to produce
working software-you want to support your users business
activities, you want to generate revenue for your
organization, you want to receive feedback on your work,
you want to prove to your users that you can produce –
but that you need to counteract this with your secondary
goal which is to enable the next effort.
Therefore you're still going to need to write some
documentation (user manuals, operations manuals,
system overviews, ...).
- Executable specifications offer far more value
than static documentation.
specifications, for example a customer-test suite
which specifies the majority of your requirements,
or a developer test-suite which specifies the
detailed design, offer significant value to
developers because they not only specify your system
they also help to validate it. Because these
artifacts add value there is a significantly greater
chance that developers will keep them up-to-date.
Yes, you will still need some static documentation
because tests certainly don't cover the entire range
of your documentation needs, but where you can
develop executable specifications you should.
- Software developers have the knowledge,
technical writers have the skill.
Like it or not few technical people have good
writing skills, if only for the simple reason that they
haven't taken the time to gain them.
The problem is that the best person suited to
write documentation is the one that knows the topic
being written about, in this case the developers of the
system. Many teams will simply hand off a system, or portion thereof,
to a technical writer and ask them to "figure it
out". This has the advantage of minimizing the effort on the part
of the developer but increases the effort required by
the technical writer and increases the chance that
they'll get it wrong.
A better approach is for the developer to write
the initial version of documentation and then
off to a technical writer for clean-up.
This has the advantage that it is likely more
effective, the developer does a "knowledge dump" and
the technical writer refactors the material to present
it effectively, but has the disadvantage that the
developer may not know how to get started or even what
to write about. A
third approach, the best one in my opinion, is to have
the technical writer and developer work together
to write the documentation, learning from each other as
they do so.
- What is required during development is often
different than what is required after development.
You have different needs during development than
you do post-development-for the sake of discussion
post-development activities include the period where you
a transitioning a release into production as well as
when the release is in production, the Transition and
Production phases of the
Enterprise Unified Process.
During development you're exploring both the
problem and solution spaces, trying to understand what
you need to build and how things work together.
Post-development you want to understand what was
built, why it was built that way, and how to operate it.
Furthermore, during development you are much more
willing to tolerate rough drafts, sketches, and greater
inconsistency-it's your own work after all –
whereas post-development you typically want more formal
during development you likely want less documentation,
you prefer to travel light, than you do during
- Do you document as you work or when you are
extreme is to write all of your documentation in
parallel with developing software. The advantage is that you capture relevant information as you
progress but the disadvantage is that as your software
evolves, as you refactor it, you will also need to
rework your documentation. Not only does this slow your development efforts down it also
results in wasted effort-documentation that you
wrote yesterday will need to be rewritten or discarded
today, in effect with this approach you are no longer
traveling light. When
your requirements have not yet stabilized, when you are
taking an iterative approach to development, excessive
documentation can become very expensive to maintain
because you are constantly updating it to reflect
other extreme is to wait until you are finished and then
write the documentation, the primary advantage being
that you are writing about a known and stable thing (the
release of the software that you just built).
There are clearly several disadvantages to this
have likely forgotten some of the reasons behind the
decisions that you made, you may not have the right
people anymore to write the documentation, you may not
have funding to do the work, or the will to write the
documentation may no longer exist.
An effective middle ground is to capture information
throughout the project as it
- Can you wait until the information has stabilized?
If you write documentation containing information
which has not yet
stabilized then you are at risk of
having to rework the documentation once the
information has changed. In other words, you
do not want to invest much time documenting
speculative ideas such as the requirements or design
early in a project. Instead, wait until later
in the life cycle when the information has stabilized
and when you know what information is actually
useful to you. The implication is that your
documentation effort may be a few iterations behind
your software development effort.
- Code documentation versus external
documentation. Do you place all of your documentation in your code, do you
write "self-documenting" code for that matter, or do
you place all of your documentation in external
again, you need to find an effective middle ground.
When your audience is developers the best place
to put the majority of the documentation is the source
code. Yes, you will likely need a system overview
document for this group as well but the reality is that
these people aren't going to trust, let alone read,
documentation outside of the code-and if they're
really smart they won't trust the documentation in the
code either. However,
the audience for documentation is much wider than just
will likely need to write documentation for people that
won't have access to the source code, or at least
don't have the ability to understand it, such as
users, senior management, and operations staff.
These audiences will require external
documentation written to meet their exact needs.
The AM practice
Single Source Information suggests that you
strive to capture information once, in the best
place possible. This is often in the source
In short, strive to
capture and communicate information using the best
- Project-level versus enterprise-level
Not all of the documentation that you will write
will be specifically for your project team, or for the
team(s) taking over your system, some of it may need to
be made available at an enterprise level. You
should to take
advantage of existing artifacts, including but not
limited to system documentation and models within your
may include existing definitions of business rules,
existing interfaces to legacy systems and their
documentation (effectively existing contract models), a
corporate meta-data repository describing the data
resources throughout your company, or an enterprise
business model. Where
did these resources come from?
The source information came from other project
teams such as yours and is likely administered by a
corporate team of specialists.
Yes, this is clearly a situation that is likely
to promote needless bureaucracy it is still possible to
be agile-the centralized administration teams need
to find a way to work with yours effectively.
During development they should provide the
resources such as existing models and meta data that you
need and act as consultants when you need help
understanding and working with those resources.
Post development they should help you to capture
relevant information to feed back into the shared
enterprise resources, for example part of the process of
transitioning your system into production may be to
ensure that updates to your corporate business rules and
meta-data repositories are made.
Centralized administration teams need to be
customer focused to be truly agile, they must provide
real business value for their efforts, and actively
strive to understand how and why the resources that they
manage are used by their customers.
- Quantity versus quality.
The basic trade-off is the "security" of
having the document against your trust in its accuracy.
What would you rather have, 2000-page system document
that is likely to have a significant number of errors in
it but significant details or a 20-page, high-level
overview? The large document would very likely
have most of the information that you need to maintain
and enhance your system, but would you trust the
information contained in it? The shorter document
very likely wouldn't contain the detailed information
you need but it would provide a map from where you could
dive into the source code, or other documents, for
details. You'd be more likely to trust this document
because it's shorter, worst case you could easily update
or simply rewrite it if you found it to be grossly
inaccurate, and because it deals with high-level
concepts such as your system architecture which will
change more slowly than the detailed minutiae contained
in the larger document. It is important to understand
that I am not saying that a larger document is
automatically of lower quality than a shorter one, but I
am saying that it is likely to be perceived as such
until proven otherwise.
Document stable things,
not speculative things.
7. What Does it Mean to Travel
One of the greatest misunderstandings
people have about the concept of
traveling light is that
it means you don't create any documentation. The reality is that nothing could be further from
the truth. What
traveling light does mean, at least in the context of
AM, is that you create just enough models and just
enough documentation to get by.
On extremely rare occasions that may mean you
create no models whatsoever, perhaps on a very small
project, or even no documentation, perhaps on a very
small project. But
for the vast majority of projects you need to create
some models and some documents.
How can you ensure that you're traveling
good rule of thumb is that you shouldn't create a
model or document until you actually need it –
creating either thing too early puts you at risk of
wasting your time working on something you don't
actually need yet. I once worked on a commercial software system,
one that organizations could either buy and install out
of the box or more typically would purchase and then
have the company that I worked for modify it to meet
their exact needs. One day our sales person got a new lead on a company that
wanted our system, but this company was in a new domain
that we didn't have experience in yet. We figured we were going to get this sale, we
eventually did in fact get it, so before we even met
with the client myself and a couple of others started
modeling the expected changes that we thought we would
need to make. After
several days of modeling sessions we came to the
conclusion that the changes were going to be easy, our
system was amazing after all, and we would have no
problem supporting this new client. Then we talked to them. Of the seventy or so new requirements we had
identified we had gotten about half of them right, not
bad considering we didn't know what we were talking
about, but we had missed several hundred others. Naturally our models based on our faulty
requirements were of little value and we had to discard
them and start fresh, something that was made a little
harder because we needed to shift our mindset away from
our original approach. Even worse, we went into the first meeting with a
cocky attitude because we thought we knew it all, an
attitude that would have lost us the customer if it
wasn't for the fancy maneuvering of our sales person. The bottom line is that we didn't need the
model at the time we created it, instead we should have
waited to create the model after talking to our client
and after securing the sale.
As another example, the
RUP suggests that you create a Software Architecture
Description (SAD) document to define your architecture. Yes, defining your system's architecture is
very good thing to do, as I discuss in Agile
Architecture, but do you need to create a document
early in the project? Sometimes yes, sometimes no. A couple of years ago I worked on a
mission-critical system where we developed our
architecture model on a
plain old whiteboard (POW), we were
following the practice
Display Models Publicly,
as a collection of free-form diagrams. Because the process told us to create a SAD we
did so, even though most of the developers were working
from the whiteboard sketches. Because the RUP
product provides a pretty good Microsoft Word
template for a SAD we used it as the basis from which to
work. A few of the developers printed out diagrams from
the SAD document, we had transcribed our sketches into
Microsoft Visio, to tack to their cubicle walls because
they couldn't easily see the diagrams from where they
time the whiteboard sketches changed sufficiently we
would have to update the electronic versions of the
diagrams plus corresponding sections in the SAD
advantage of having the SAD document was that it
provided a description of what we were building, a
document that we distributed to management and made
available to anyone within the organization that was
disadvantage, from what I could tell, was that nobody
was using the SAD document effectively. I'm not sure the managers ever looked at it,
and if they did it was a cursory inspection at best
because we never received any feedback from them. The developers that needed copies of the diagrams
could just as easily taken a digital snapshot of the
sketches and either created a shared architecture web
page or just printed those out.This would have also been a SAD document, albeit in a
different form, according to the principle
More Important Than Representation because it still
would have described our architecture (which is the
fundamental purpose of a SAD document).By the same reasoning our whiteboard sketches were also a SAD
document, although one that wasn't permanent and one
that fixed in a permanent location. In my opinion we had made three fundamental
mistakes and thus were traveling heavier than we needed
to: we chose the wrong format for the documentation, we
documented too early, and we created too much
added overhead of maintaining the SAD as a Word document
slowed us down because we effectively had that
information in several locations-the whiteboard
which is what we were working from, the Word document
which was almost superfluous, and in our code itself. Furthermore, we could have been developing
software instead of documentation, and worse yet the
update effort took up a large portion of the time of one
of our senior developers so he couldn't help to mentor
and coach the junior developers. Granted, it was useful to have this document at
the end of the project as one of our deliverables to the
maintenance team but we certainly didn't need it
during development and it clearly did the project team
more harm than it did good.
Traveling light requires you to think about what
you are doing. At the beginning of your project
ask yourself what you think you're going to need based
on the nature of your project. Development of an
air traffic control system will likely require greater
documentation than development of a web site made from
static HTML pages. As the project progresses
you'll find that your initial estimate of your
documentation needs changes with experience, perhaps
you'll need more or less.
to use the analogy of hiking, he's a mountain climber -
packing too light or too heavy can lead to disaster, at
worst it kills you and at best forces you to turn back
and rethink your strategy. Imagine crossing a
desert with insufficient water, you're traveling too
light, or trying to cross the same desert with a
100-pound pack strapped to your back, now you're
traveling too heavy. Now imagine building a
mission-critical e-commerce application without
providing any documentation describing how to operate
it, your project effectively fails because you've
traveled too light. Now imagine building the same
system with thousands of pages of documentation that you
must update and validate every time you change the
system, you fail again because you're traveling so heavy
that you cannot respond quickly enough to changes in the
marketplace. Traveling light means just enough
models and documentation, too little or too much puts
you at risk.
Regardless of what some people will tell
you, documentation can in fact be quite effective. When is a document agile? When it meets the following criteria:
Agile documents maximize stakeholder ROI. The
benefit provided by an agile document is greater than
the investment in its creation and maintenance, and
ideally the investment made in that documentation was
the best option available for those resources.
In other words documentation must at least
provide positive value and ideally provides the best
value possible. For example, if the creation of a user
document provides a 50% payback on investment but
providing a training course to your users provides 100%
payback then you're better off choosing the training
because it's a better investment.
Stakeholders know the TCO of the document.
Stakeholders must understand the total cost of
ownership (TCO) of the document and be
invest in the creation and maintenance of that
Agile documents are "lean and sufficient".
An agile document contains just enough
information to fulfill its purpose, in other words it is
as simple as it can possibly be.
For example, portions of an agile document could
be written in point form instead of prose-you're
still capturing the critical information without
investing time to make it look pretty, which is in
accordance to the principle
Content is More Important
Agile documents will often provide references to
other sources of information, for example a
model describing the interface to an external system
could indicate that the SOAP 1.1 protocol is being used
and provide a reference to the XML DTD and schema
definition which define the XML documents transmitted
between systems. When
writing an agile document remember the principle
Simplicity, that the simplest documentation will be
sufficient, and follow the practice
Content whenever possible.
Agile documents fulfill a purpose.
Agile documents are cohesive, they
single defined purpose. If you do not know why you are
creating the document, or if the purpose for creating
the document is questionable (see earlier), then you
should stop and rethink what you are doing.
Agile documents describe "good things to
know". Agile documents capture critical
information, information that is not readily obvious
such as design rationale, requirements, usage
procedures, or operational procedures.
Agile documents do not capture obvious
example, documentation indicating that the F_NAME column
of the CUSTOMER table captures the first name of a
customer really doesn't provide a lot of value to me.
Documentation indicating that the customer table
does not include data for customers residing in
Canada's Yukon territory because that data is stored
in an ASCII flat file on another system due to
regulatory reasons is likely good information to know.
Agile documents have a specific customer and
facilitate the work efforts of that customer.
System documentation is typically written for
maintenance developers, providing an overview of the
system's architecture and potentially summarizing
critical requirements and design decisions.
User documentation often includes tutorials for
using a system written in language that your users
understand whereas operations documentation describes
how to run your system and is written in language that
operations staff can understand.
Different customers, different types of
documents, and very likely different writing styles.
must work closely with the customer, or potential
customer, for your documentation if you want to create
something that will actually meet their needs. For
example, I would be reticent to write the system
documentation for the maintenance developers without
involving some of them in the effort.
Yes, sometimes you can't get these folks involved (you
may have none on staff at the moment) or you may not be
able to identify who within your maintenance
organization will be the eventual "owners"
of your system. When you don't have the
customers involved you're at risk of creating too much
documentation or unnecessary documentation and
hence becoming less agile.
You will often discover that when you involve the
customers that they often have a very good idea of what
they actually need, and can often provide examples of
what works well for them and what doesn't.
Agile documents are sufficiently accurate,
consistent, and detailed.
Have you ever learned how to use new software by
using a book describing a previous version of that
you succeed? Likely.
Was it a perfect situation?
Did it cover all the new features of the
course not, but it still got you up and running with the
software package. Were
you willing to spend your own money, perhaps on the
order of $30, to purchase the latest version of the book
you needed? Likely
not, because it wasn't worth it to you. Agile documents do not need to be perfect, they just need to
be good enough.
However, having said this you need to
take the advice that agile documents need to be just
good enough with a grain of salt. If you are writing the operations manuals for
a software system for a nuclear power plant then I
highly suggest you get it right! However, few systems are truly that critical, so investing
the effort required to get their documentation
perfect isn't appropriate. You need to make a judgment call when it
comes to writing documentation, the key issue being
to identify how much ambiguity can the customers of
the document accept and still be effective at their
You will need to create documentation on
your project, this is true even of the most
XP projects let alone
RUP projects. But what types of documentation will you potentially
need to create? Table 1
lists some of the most common documents that you
may decide to create as part of your development effort,
documents that you will deliver as part of your overall
system. Table 1
does not include management artifacts such as
project schedules, software deliverables such as source
code and test suites, or interim work products such as
Table 1. Potential documents to be
created by your development team.
A document describing the technical
interface to a system or portion of a system.
Developers, Maintenance Developers,
A summary of critical decisions
pertaining to design and architecture that the
team made throughout development
Question the need for most of this information.
Are people really going to care several years
from now why you did something?
Focus on decisions that are not
obvious, which had other reasonable alternatives,
or ones where you explored what initially looked
like a reasonable alternative that in the end
proved insufficient for your needs.
The goal of this effort is to avoid
needless refactoring at some point in the future
or rehashing a previously made decision.
Design decisions are often
documented throughout other artifacts, such as
system overviews and source code, although you may
choose to have a separate document when
appropriate to the situation.
Senior Management, User Management,
A definition of the vision for the
system and a summary of the current cost
estimates, predicted benefits, risks, staffing
estimates, and scheduled milestones.
including listing of major stakeholder groups,
their representatives (if any), and their
This document is typically used to
gain funding and support for your project as well
as provide status updates to important project
stakeholders who may not be actively involved with
your project on a day-to-day basis.
Keep it short,
ideally a few pages, and consider writing it in
Have the courage to make this
publicly available to everyone that should have
access to it, even when it contains bad news,
according to the principle
Open and Honest
Valuable interim document
This documentation typically
includes an indication of the dependencies that
your system is involved with; the nature of its
interaction with other systems, databases, and
files; references to backup procedures; a list of
contact points for your system and how to reach
them; a summary of the availability/reliability
requirements for your system; an indication of the
expected load profile of your system; and
Developers, Managers, Maintenance
Developers, Operations Staff
A summary of critical information
such as the vision for the system, primary user
contacts, technologies and tools used to build the
system, and the critical operating processes (some
applicable to development, such as how to build
the system and some applicable to production, such
as how to back up data storage).
Also provides references to critical
project artifacts such as the source code (the
project name in the source code control system is
often sufficient), where the permanent models
pertaining to the system (if any) are, and where
other documents (if any) are.
I typically create and maintain this
document during development.
Keep it short and to the point, I
don't ever remember one getting beyond ten
printed pages and most are less than five.
This document serves as a starting
point for anyone new to the team, including
maintenance developers, because it answers
fundamental questions that they are likely to
Whenever you first share this
document with anyone ask them to keep track of
major issues that they were not able to easily
resolve using this document, or any misinformation
they discover, to provide insight into potential
updates for the document.
I often call this a "Good Things
to Know" document.
Developers, Maintenance Developers,Users, User Managers
This document defines what the
system does, summarizing or composed of
requirements artifacts such as business rule
definitions, use cases, user stories, or essential
user interface prototypes (to name a few).
Writing a requirements specification early
in the life cycle should be seen as a
very large risk.
XP projects typically favor
inclusive requirements techniques such as user
stories and CRC cards.
RUP projects tend towards more
formal requirements artifacts and documentation.
Disciplined agile teams capture the
majority of their requirements as
executable specifications. As a result
these teams only need brief overviews of the
scope which would be part of the system
In scaling situations, particularly
when regulatory compliance, large teams, or
geographically distributed teams are an issue
This documentation includes training
materials specific to support staff; all user
documentation to use as reference when solving
problems; a trouble-shooting guide; escalation
procedures for handling difficult problems; and a
list of contact points within the maintenance
You will find that your
department often has a standard escalation
procedure in place, therefore you will not need to
write one, and like the operations department may
have standard templates or examples that you can
Maintenance Developers, Developers
The purpose of this document is to
provide an overview of the system and to help
people understand the system.Common information in this document includes an overview of
the technical architecture, the business
architecture, and the high-level requirements for
the system. Detailed
architecture and design models, or references to
them, may also be included where appropriate.
System documentation helps to reduce
perceived risk on the project by providing
"truck insurance", the assurance that if the
development team leaves, or gets hit by a truck,
that critical information about the project is
Unfortunately this is typically false
insurance-if you lose someone(s) then no
matter how good the documentation is you have a
serious problem on your hands because new people
still need to be identified, assigned to your
system, and need to learn the system.
You're much better off increasing your
project's "truck number", the minimum number
of people that if you lost them would put your
project at risk, by supporting knowledge sharing
practices such as
Model With Others.
Users, User Managers
Your users may require a reference
manual, a usage guide, a support guide, and even
It's important that you distinguish
between these different types of documents because
the way that each one is used varies: one is for
quick lookups, one is for discovering about how to
work with the system, one is for how to obtain
additional help, and one is for training.
.I typically base my usage guide and
training materials on the use cases for the system
-the use cases describe how the actors work
with the system, therefore they should be a very
good foundation on which to base both of these
.User documentation should be
considered part of the user interface for your
system and therefore should undergo usability
Observation: Sometimes Un-Agility Enables
Agility. Agile software development is new to many
organizations and as a result there is significant fear
and uncertainty concerning its viability, and although
it's frustrating at times this is actually a good
thing because it means that people care. Because of this fear you may be required by your
project stakeholders to create extraneous documentation
to help put them at ease, something that is particularly
true of senior management who have likely been burned in
the past by other new techniques and technologies. Often the creation of a single document, perhaps
an executive overview, will give your project
stakeholders enough of a "warm and fuzzy feeling"
about your project to allow you to work in the manner of
your choice (e.g. in an agile way).
part of the
a definite yes.
were just as
and so on.
to lay waste
to some of
have when it
Figure 4. Deliverable documentation creation.
part of the
of the IT
of which are
As you can
is at the
as that of
Figure 5 is
it came to
the bars on
Figure 5. Quality of "deliverable documentation".
when in the
In this case
that you may
to retain at
the end of
the type of
this list is
not meant to
Figure 6 in
through the SDLC.
continuously such as
docs as you
The idea is
that if a
system is to
the end of
time in sync
If you adopt
iteration (although strive to write it that iteration).
of done for
In this respect documents are just
like models, my recommendation is to follow the practice
Update Only When It Hurts. Agile documents, like agile models, are just good
times a document can be out of date and it doesn't
matter very much.For example, when I wrote the first version of this article I
was working with
the early release of the JDK v1.3.1 yet I regularly used
reference manuals for JDK 1.2.x-it's not a perfect
situation but I get by without too much grief because
the manuals I'm using are still good enough.Would the manuals for Java v1.0.x be sufficient? Likely not, because there has been a significant
change since that release and my productivity loss would
be much greater than the cost of new manuals.
In addition to the fundamental
requirement of your project stakeholders authorizing the
investment of resources required to do the work, you
should use the follow heuristics for deciding when to
Update contract models and
re-release them ideally before, and minimally in
parallel to, releasing the item(s) that the model
Documentation that is intended
as part of the system, such as operations and user
manuals, should be updated and released before or
with the system itself.
The customer of the
documentation is being inordinately harmed,
including a significant loss of productivity,
because the documentation is not updated (e.g. it
Yes, it can be frustrating having
models and documents that aren't completely accurate
or that aren't perfectly consistent with one another. This frustration is offset by the productivity
increase inherent in traveling light, in not needing to
continually keep documentation and models up to date and
consistent with one another. Do you want to write software or write
A documentation handoff occurs when one
group or person provides documentation to another group
or person. Agile
developers desperately try to avoid documentation
handoffs because they are not a very good way for people
to communicate. Unfortunately
documentation hand-offs are a reality in some situations
-often your development team is so large it cannot be
co-located, perhaps a subsystem is being created by
another company (implying the need for a contract
model), perhaps important project stakeholders are not
readily available to your team, perhaps regulations
within your industry or organization require the
production of certain documents, or perhaps your team is
made up of specialists instead of
generalizing specialists. The following strategies can help to increase the
effectiveness of documentation handoffs:
- Avoid documentation handoffs.
As you migrate to an
development process you will constantly run into
people who are not as agile, people who see nothing
wrong with documentation handoffs.
Point out that there are better ways to
communicate-face-to-face conversations, video
conferencing, telephone conferencing-that you
should consider before writing documentation, and
whenever possible try to find a better way that
fulfills your needs.
- Support handoffs with other means of
communication. If you can't avoid providing documentation to someone else
you should at least strive to support the handoff
face-to-face communication or
This may enable you to write less
documentation, therefore allowing you to focus on
other activities, and will help you to avoid some of
the common disadvantages of documentation such as
misunderstanding the material.
- Avoid documentation handoffs.
Chances are good that the people you are
interacting with don't like writing and receiving
documentation either-at least it doesn't hurt
See the section
Strategies for Increasing
the Agility of Documentation.
I can't stress this enough.
Yes and no.
require in a
this and try
all of the
prove to be
is a slang
part on the
has of an
when out of
the cruft in
being a bad
100% - C * R
* U * F * T,
- C =
- R =
- U =
- F =
- T =
- Minimize the information contained in the document
- Single source information by capturing it in one place only
- Have more than one person work on the document together (i.e. pair write the document and thereby do inspections in progress)
- Have shared ownership of all documentation so that multiple people will work on it
- Document stable, not speculative concepts
- Take an evolutionary (iterative and incremental) approach to its development so that you gain feedback as to its actual value
- Prefer executable specifications, such as customer tests and developer tests, over static specifications such as traditional requirements and design documents. By doing so the specification actually provides value to the audience
- Capture high-level information in documentation, not details. When you specify details of how to do something and then hand the document to someone for them to follow, you effectively take all of the joy out of the activity for that person because you've done the thinking for them. What would the motivation be for an intellectual worker, such as an IT professional, to follow detailed instructions?
- Work closely with the audience of the document
- Take an evolutionary approach
or to be
If you can't avoid writing a
document, how can you at least write it in an agile
best practices should help:
Agile Documentation describes a collection of
patterns for writing effective system documentation.
The Object Primer 3rd Edition: Agile Model Driven
Development with UML 2 is an
important reference book for agile modelers,
describing how to develop 35
types of agile
models including all 13
UML 2 diagrams.
Furthermore, this book describes the techniques
Life Cycle Object Oriented Testing
(FLOOT) methodology to give you the fundamental
testing skills which you require to succeed at
agile software development. The book also
shows how to move from your agile models to
source code (Java examples are provided) as well
as how to succeed at implementation techniques
(TDD). The Object Primer also includes a
chapter overviewing the critical database
development techniques (database refactoring,
legacy analysis, and
database access coding) from my award-winning
Agile Database Techniques
Agile Modeling: Effective Practices for Extreme
Programming and the Unified Process is the seminal
book describing how agile software developers approach
documentation. It describes principles and
practices which you can tailor into your existing
software process, such as
Rational Unified Process (RUP), or the
Agile Unified Process (AUP), to streamline your
modeling and documentation efforts. Modeling and
documentation are important aspects of any software
project, including agile projects, and this book
describes in detail how to
architect, and then
design your system in an agile manner.
The Elements of UML 2.0 Style describes a collection
of standards, conventions, and
for creating effective
UML diagrams. They are based on sound, proven
software engineering principles that lead to diagrams
that are easier to understand and work with. These
conventions exist as a collection of simple, concise
guidelines that if applied consistently, represent an
important first step in increasing your productivity as
a modeler. This book is oriented towards
intermediate to advanced UML modelers, although there
are numerous examples throughout the book it would not
be a good way to learn the UML (instead, consider
The Object Primer). The book is a brief 188
pages long and is conveniently pocket-sized so it's easy
to carry around.
We actively work with clients around the world to
improve their information technology (IT) practices,
typically in the role of mentor/coach, team lead, or trainer. A full
description of what we do, and how to contact us, can be
found at Scott W.
Ambler + Associates.