Document Late: An Agile Best Practice

A common agile practice is to defer the creation of all deliverable documentation as late as possible, creating them just before you need to actually deliver them. For example, system overviews are best written towards the end of the development of a release because you know what you've actually built. Similarly, the majority of user and support documentation is also best written towards the end of the lifecycle. Figure 1 compares the traditional and agile approaches to documentation, showing the relative total amount of effort invested in creating and maintaining documentation throughout the SDLC. Fundamentally, this strategy is to wait until the information has stabilized before you capture it in documentation.

Note that it is generally risky to capture speculative information, such as requirements or design details, early in the lifecycle because those details are likely to change. These sorts of artifacts are typically not included in your deliverable documentation, except in cases where regulatory issues are a concern, so it doesn't make a lot of sense to put much effort into their creation and maintenance.


Figure 1. Documentation throughout the software development lifecycle.

It's interesting to note the differences in the two strategies:

  1. Agilists model and plan early in the project, but they don't write detailed documents. Agilists will invest a bit of time at the beginning of the project to envision the initial requirements and the initial architecture. They do this because people will ask questions such as what are you going to do, how much will it cost, how long will it take, and how are you going to build it. The implication is that you need to do some high-level requirements and architecture modeling, but that doesn't mean that you need to write mounds of detailed specifications. Get the value out of modeling, which is to think things through, without the cost of needless documentation. Agilists will also create a high-level plan showing projected delivery dates, major dependencies, and so on.
  2. Agilists still create detailed specifications. Agilists capture the details on a just-in-time (JIT) basis, not up front. They also prefer to write executable specifications in the form of tests, not static documents. When you take a Test-Driven Development (TDD) approach this is quite easy to achieve. In the end, Agilists often do more specification than traditionalists but we do it in a different manner.
  3. Agilists still write supporting documents. Supporting documentation, such as user manuals, operations manuals, and support manuals, are an important deliverable of any system development project. Agilists, like traditionalists, will write these documents towards the end of the lifecycle once the system has stabilized. However, because Agilists treat documentation like a requirement we will estimate the effort to write it and ask our stakeholders to prioritize it along with all other requirements. Because of this greater level of stakeholder participation and control Agilists will tend to be more efficient than traditionalists when it comes to that sort of documentation (when you put a visible price tag on things people get really smart really fast over how they invest their money).
  4. Agilists still write system overview documentation. There will still be some need for high-level overview documentation for the people who will need to maintain the system. However, because Agilists write high-quality code than traditionalists and have a regression test-suite for their code (which in effect is the detailed specifications for the system) you will find that you only need a concise overview consisting mainly of critical high-level diagrams. The greater discipline of Agilists, and the resulting higher levels of quality, result in less need for detailed documents (which are invariably out of date anyway, further contributing to the quality challenges on traditional projects).
  5. Agilists have a lower cost curve. Not only is the agile curve in Figure 1 lower, it is also less steep. By documenting early in the lifecycle traditionalists incur increased documentation maintenance costs throughout the lifecycle.

Risk Profile of this Practice

By waiting to document information once it has stabilized you reduce both the cost and several risks associated with documentation:
  1. Financial risk. Cost is reduced because you won't waste time documenting information that changes, which in turn motivates you to update the documentation. If you write documentation containing information which has not yet stabilized then you are at risk of having to rework the documentation once the information has changed. In other words, you do not want to invest much time documenting speculative ideas such as the requirements or design early in a project. Instead, wait until later in the lifecycle when the information has stabilized and when you know what information is actually useful to you. The implication is that your documentation effort may be a few iterations behind your software development effort.
  2. Accuracy risk. Accuracy is improved because you are focusing on what you actually built and did, not on what you speculate you'll be doing. One of the challenges with writing documentation early in the lifecycle is that you don't maintain it well throughout the lifecycle -- it's too easy to assume you'll get to doing the appropriate updates next iteration.
Agile Documentation

However, this doesn't mean that all documentation should be left towards the end because there some risks which are increased:

  1. Delivery risk. You may not complete the required documentation due to lack of resources (financial, time). To counteract this risk make the creation of those documents explicit requirements which should be prioritized and estimated like any other requirement.
  2. Accuracy risk. You may forget some information. To counteract this risk you might still want to take notes for these sorts of documents throughout development so that you don't lose critical information. These notes may be nothing more than point-form information as there is no need to "polish" documents until just before final delivery of them.

Shouldn't My Team Document Continuously?

Another, different strategy, for approaching documentation on an agile project is to document continuously. The basic idea is that you write your deliverable documentation throughout the project, updating it as you update your software. This strategy supports the agile philosophy of always producing a solution (disciplined agile delivery teams recognize that there is more to it than creating great software, what you're really producing is a great solution which includes software, hardware, supporting documentation, changes to the business process, and potentially changes to the organization structure) that is potentially shippable. To do this you really need to document continuously, but in doing so you start travelling heavier and thereby take on the risks associated with doing so. So, the real question is what trade-offs are you willing to make?