Document Late: An Agile Core Practice

A common agile practice is to defer the creation of all deliverable documentation as late as possible, creating them just before you need to actually deliver them. Fundamentally, the document late strategy is to wait until the information has stabilized before you capture it in documentation. For example, system overviews are best written towards the end of the development of a release because you know what you’ve actually built. Similarly, the majority of user and support documentation is also best written towards the end of the lifecycle. Figure 1 compares the traditional and agile approaches to documentation, showing the relative total amount of effort invested in creating and maintaining documentation throughout the lifecycle.

Figure 1. Documentation throughout the software development lifecycle (click to enlarge).

Agile Documentation - Document Late strategy

Deliverable Documentation on Agile Teams

I’ve been using the term “deliverable documentation”, but what does that mean? Deliverable documentation is the documentation that you need to include as part of your overall solution. This will of course vary between teams, but deliverable documentation typically includes user manuals, training materials, operations manuals, support manuals, and system overviews. It typically does not include requirements specifications or design specifications, except of course in regulatory situations where such documentation is required or in contract negotiations where it’s required as part of the contract. Of course, with all such artifacts, I highly suggest following the core practices of agile documentation.

Note that it is generally risky to capture speculative information, such as requirements or design details, early in the lifecycle because those details are likely to change. These sorts of artifacts are typically not included in your deliverable documentation, except in cases where regulatory issues are a concern.

Document Late in Practice

It’s interesting to note the differences in the two strategies:

Agilists model and plan early in an initiative, but they don’t write detailed documents. Agilists will invest a bit of time at the beginning of the initiative to envision the initial requirements and the initial architecture. They do this because people will ask questions such as what are you going to do, how much will it cost, how long will it take, and how are you going to build it. The implication is that you need to do some high-level requirements and architecture modeling, but that doesn’t mean that you need to write mounds of detailed specifications. Get the value out of modeling, which is to think things through, without the cost of needless documentation. Agile teams will also create a high-level plan showing projected delivery dates, major dependencies, and so on.
Agilists still create detailed specifications. Agilists capture the details on a just-in-time (JIT) basis, not up front. They also prefer to write executable specifications in the form of tests, not static documents. When you take a Test-Driven Development (TDD) approach this is quite easy to achieve. In the end, Agilists often do more specification than traditionalists but we do it in a different manner.
Agilists still write supporting documents. Supporting documentation, such as user manuals, operations manuals, and support manuals, are an important deliverable of any system development initiative. Agilists, like traditionalists, will write these documents towards the end of the lifecycle once the system has stabilized. In other words, they tend to document late. Because Agilists treat documentation like a requirement we will estimate the effort to write it and ask our stakeholders to prioritize it along with all other requirements. Because of this greater level of stakeholder participation and control Agilists will tend to be more efficient than traditionalists when it comes to that sort of documentation. When you put a visible price tag on things people get really smart really fast over how they invest their money.
Agilists still write system overview documentation. There will still be some need for high-level overview documentation for the people who will need to maintain the system. However, because Agilists write high-quality code than traditionalists and have a regression test-suite for their code (which in effect is the detailed specifications for the system) you will find that you only need a concise overview consisting mainly of critical high-level diagrams. The greater discipline of Agilists, and the resulting higher levels of quality, result in less need for detailed documents (which are invariably out of date anyway, further contributing to the quality challenges on traditional initiatives).
Agile documentation has a lower cost curve. Not only is the agile curve in Figure 1 lower, it is also less steep. By documenting early in the lifecycle traditionalists incur increased documentation maintenance costs throughout the lifecycle.

Shouldn’t My Team Document Continuously, Not Document Late?

Another complemetary strategy for approaching documentation on an agile team is to document continuously. The basic idea is that you write your deliverable documentation throughout the initiative, updating it as you update your software. This strategy supports the agile philosophy of always producing a solution. There is more to software development than creating potentially shippable software. What you’re really producing is a great solution that includes software, hardware, supporting documentation, changes to the business process, and potentially changes to the organization structure. To do this you really need to document continuously, but then you “travel heavy” and thereby take on the risks associated with doing so.

The real question is what trade-offs are you willing to make?

Document Late in Context

By waiting to document information once it has stabilized you reduce both the cost and several risks associated with documentation:

Financial risk. Cost is reduced because you won’t waste time documenting information that changes, which in turn motivates you to update the documentation. If you write documentation containing information which has not yet stabilized then you risk having to rework the documentation once the information has changed. You do not want to invest much time documenting speculative ideas such as the requirements or design early in an initiative. Instead, document late by waiting until the information has stabilized and when you know what information is actually useful to you. The implication is that your documentation effort may be a sprint or two behind your software development effort.
Accuracy risk. Accuracy is improved with a document late strategy because you are focusing on what you actually built and did, not on what you speculate you’ll be doing. One of the challenges with writing documentation early in the lifecycle is that you don’t maintain it well throughout the lifecycle: it’s too easy to assume you’ll get to doing the appropriate updates next sprint.

Note that some risks increase when you leave documentation to the very end:

Delivery risk. You may not complete the required documentation due to lack of resources (financial, time). To counteract this risk make the creation of those documents explicit requirements which should be prioritized and estimated like any other requirement.
Accuracy risk. You may forget some information. To counteract this risk you might still want to take notes for these sorts of documents throughout development so that you don’t lose critical information. These notes may be nothing more than point-form information as there is no need to “polish” documents until just before final delivery of them.

In practice you can reduce these risks by writing any required documentation one-sprint after a requirement was implemented, effectively a combination of document late and document continuously.