This article presents a coherent strategy for bringing usability practices
into agile project, summarizing Chapter 4 of
Maturing Usability: Quality in Software, Interaction, and Value.
User experience (UX) and
agile practitioners need to learn about and respect each other's philosophies
and techniques, and then actively find ways to work together. This requires
both communities to make minor changes to the way that they work, but if they
choose to make these changes I suspect that their work products will be much
better for it.
With an agile UX approach you will do high-level, very broad modeling at the
beginning of the agile project to address the majority usability and UI issues
facing the team. You don't need a detailed answer, the details can come later
on a JIT basis during development, but you do need a strategy. During
development usability practitioners should be embedded within the development
team, working on UX tasks when needed but also working with their teammates on
non-UX activities. Agile teams are made up of
generalizing specialists with
the ability to work on a wide range of things, instead of narrowly focused
specialists typically found on traditional teams. User testing, including
usability testing, is done throughout the lifecycle, it isn't left to the end of
the project where it is invariably too late to act.
In The Inmates Are Running The Asylum Alan Cooper
many of today's software-based products suffer from usability challenges. He
believes that the key to solving the problem is to perform what he calls
interaction design before the programming effort to design the way that software
behaves before it is built. Cooper believes programmers consciously design the
code inside programs but only inadvertently design the interaction with humans.
Although programmers work hard to make their software easy to use, their frame
of reference is themselves and as a result they make it easy for other software
engineers, not normal human beings. He argues that programmers have too much
influence over the design of the human interface and due to a lack of skills in
this area do a poor job of it. In other words, the programming inmates are
running the software development asylum.
My experience is similar to Cooper's, there is a need for improved
interaction design skills within the programming community, although I don't
agree with his proposed solution. Reading between the lines of both
Are Running the Asylum and a discussion between
Kent Beck and Alan Cooper, it seems to me that most of Cooper's experiences are with traditional
development teams but not agile teams. Although I believe that his observations
about the interaction design skill levels of developers apply to both the
traditional and agile software development (ASD) communities, I fear that his
advice is difficult for traditionalists to implement and all but impossible for
agilists. The goal of this article is to show how to make the advice of
the user experience community palatable, or perhaps "usable" is a better term,
to the agile community.
Table of Contents
Software Development (ASD)
Experience (UX) Activities
Current state of the art
experience modeling on an
User testing on an agile
A call to action
To address the challenges faced by software developers an initial group of 17
methodologists formed the Agile Software
Development Alliance, often referred to simply as the Agile Alliance, in
February of 2001. An interesting thing about this group is that they all came
from different backgrounds, yet were able to come to an agreement on issues that
methodologists typically don't agree upon. This group of people defined a
manifesto which defines 4 values and 12 principles for encouraging better ways
of developing software; this manifesto defines the criteria for ASD processes.
Just as the Software Engineering Institute's
Capability Maturity Model
Integrated (CMMI) defines the requirements for a heavy-weight software development process,
the agile manifesto defines the requirements for an agile software process.
Agile processes which reflect these requirements include:
The vast majority of agile projects are teams of less than ten people, are
co-located, have direct access to stakeholders, have access to inclusive
modeling tools such as whiteboards and corkboards, have their own development
machines, and have access to the development tools that they require, including
testing tools. Having said that, some agile teams are very large (upwards
of several hundred people), some are dispersed geographically, and some do not
always have easy access to stakeholders (Eckstein 2004). Although most agile
teams take an
development (TDD) approach where they write a unit test before writing just
enough production code to fulfill that unit test, they typically do not have
access to UI testing tools. Furthermore, they rarely have access to a usability
lab, so in this respect agile is little different than traditional development.
Figure 1 depicts my rendition of a generic
which is comprised of six phases: Concept, Inception, Construction, Transition,
Production, and Retirement. Although many agile developers may balk at the idea of phases, the
fact is that it's been recognized that processes such as XP and DAD do in fact have phases.
Figure 1. The Agile SDLC.
Let's consider each phase in turn:
- Concept. The initial idea/concept for the product or project is typically identified
as part of your organizations product management process.
- Inception. The first week or so of an agile project is often
referred to as "Iteration 0" or "Cycle 0" (cycle is an alternative term for iteration). The
goal during this period is to initiate the project by garnering initial
support and funding for the project; actively working with stakeholders to
envision the requirements of the system at a high-level; starting to build
modeling an initial architecture for the system; and setting up
- Construction phase. During development iterations agilists
incrementally deliver high-quality working software which meets the changing
needs of stakeholders.
- Transition phase. During the release iterations(s) agile practitioners
transition the system into production.
- Production. The goal of this phase is to keep systems useful and
productive after they have been deployed to the user community. The
fundamental goal is to keep the system running and help users to use it.
- Retirement. Eventually your solution may be removed from production, perhaps because
it supports a line of business your organization has exited or because it has been replaced
by a more effective solution.
On the surface, the agile SDLC of Figure 1 looks very
much like a traditional SDLC, but when you dive deeper you quickly discover that
this isn't the case. Because the agile SDLC is highly collaborative, iterative,
and incremental the roles which people take are much more robust than on
traditional projects. In the traditional world a business analyst created a
requirements model that is handed off to an architect who creates design models
that are handed off to a coder who writes programs which are handed off to a
tester and so on. On an agile project, developers work closely with their
stakeholders to understand their needs, they pair together to implement and test
their solution, and the solution is shown to the stakeholder for quick
feedback. Instead of specialists handing artifacts to one another, and thereby
injecting defects at every step along the way, agile developers are
generalizing specialists with full lifecycle skills. More
importantly, from a user experience point of view, they take a very different approach to modeling and
testing than traditionalists do.
|Usability is a quality attribute of a system which encompasses learnability,
efficiency, memorability, error recovery, and end-user satisfaction (Neilson
1994). User-centered design, also known as UCD although I will use that
abbreviation for usage-centered design, is a highly structured, product
development process where the focus is on understanding the needs and goals of
the user of the product.
Interaction Design (ID) is a methodology described by Alan Cooper where the goal is
to provide end users with functions that are both desirable and useful. In
ID, interaction designers focus on what's desirable, engineers on what they're
capable of building, and business stakeholders on what's viable I use the term
user experience (UX) to encompass all of these concepts – although there
is good reason to distinguish between the various ideas, that isn't relevant for
my current discussion.
An important question to ask is why should agile practitioners consider
Jeff Patton believes that
UX addresses several issues which are
critical to the success of ASD teams:
- UX places emphasis on the usage necessary for roles to meet their
- UX helps meet the goal of identifying the behavior the software should
- UX practices can be applied with varying degrees of formality, thereby
making them compatible with agile methodologies.
Other important terminology which I use in this article includes:
- System. The product, which often includes software, under
- User. Also known as an end user, a user is a person who will
actually work with the system/product being built.
- Developer. An IT professional involved with the creation of the
- Stakeholder. A stakeholder is anyone who has a stake in the
creation or operation of the system. This includes people who are a direct
user, indirect user, manager of users, senior manager, developer, operations
staff member, support (help desk) staff member, developers working on other
systems that integrate or interact with the one under development, or
maintenance professionals potentially affected by the development and/or
deployment of a software project. Some agile methodologies, XP in
particular, uses the term "customer".
- Acceptance testing. A testing technique, the goal of which is to
determine whether a system satisfies its acceptance criteria and to enable
the stakeholder(s) to determine whether to accept the system
- Usability testing. A method by which users of a system are asked
to perform certain tasks in an effort to measure the system's ease-of-use,
task time, and the user's perception of the experience. Usability testing
can be both formal and informal, using dedicated rooms and equipment to
simply using physical mock ups of the system.
- User testing. Testing activities, including both acceptance and
usability testing, where stakeholders are actively involved.
First the good news. I believe that there is a growing recognition within
both the agile and UX communities that they can benefit by working closely with
the other. Within the agile community
Larry Constantine's UCD work is well
respected and the UX community seems intrigued by the promises of ASD. There
is an active
Agile Usability mailing list where ASD and UX practitioners interact
regularly. There are agile usability tutorials at conferences, including both
UPA 2005 and Agile 2005
recently. There seems to be a will to bring the two communities together.
Now for the bad news. There are several challenges which need to be overcome
if we're to work together effectively, and just the fact that we talk about two
different communities indicates that we have a problem. These challenges
- Different goals. Software engineers focus on the technical
design, implementation, and maintenance of software system. UX
practitioners focus on developing systems so end-users can use them
effectively. Unfortunately each group seems to all-but-ignore the issues
addressed by the other.
- Different approaches. UX methodologies are centered on the user
whereas agile methodologies take a broader view and focus on the
stakeholder. With UX methods one tries to get a holistic view of the user
needs and comes up with an overall plan for the user interface before
starting implementation. Agile methods favor little up front design and
instead focus on delivering working software early.
- Organizational challenges. The agile community follows a highly
collaborative and fluid organizational strategy where teams are self
organizing. This doesn't appear to always be the case with the centralized
UX groups within some organizations. While a center for UX is important
to provide the needed practices, tools, and standards, a strong
organizational and management hierarchy can be problematic.
- Process impedance mismatch. The agile community forgoes detailed
modeling up early in the project, something they refer to as "Big Design Up
Front (BDUF)". Many within the UX community prefer more comprehensive
modeling early in the project to design the user interaction "properly"
before construction begins.
- UX practitioners struggle to be heard. Although this is true
within traditional teams as well, my experience is that this isn't as
problematic with agile teams for the simple reason that agile practitioners
favor high levels of collaboration. Jokela and Abrahamsson (2004) believe
that UX practitioners often complain that the results of their work are not
considered in the design decisions. They also point out that no UX
practitioners were invited to participate in the formation of the
which may be why there has been insufficient UX influence to date within
the ASD community.
- Our thought leaders may be a bit too extreme. This was
the discussion between Kent Beck and Alan Cooper, the founders of Extreme
Programming (XP) and Interaction Design (ID) respectively.
Many interesting points came out of the discussion pertaining to the
differences between the agile and UX philosophies, summarized in
Unfortunately, both Beck and Cooper seem to be at the extreme end of the
discussion, and you can see in the interview there were bones of contention
that weren't resolved.
Table 1. Comparing agile and UX philosophies.
Asks "How can what we
have now be improved this iteration?"
You should work
closely with your stakeholders/customers to identify their exact
requirements can be identified on a just-in-time (JIT) basis during
modeling is a risky endeavor at best.
- Interaction design has a role to play from the beginning of a
project, as a way to come up with the metaphors for the project.
Asks "What is the
Defining the behavior
of software-based products and services is very difficult and it has
to be done from the point of view of understanding and visualizing
the behavior of complex systems, not the construction of complex
- All behavioral issues need to be addressed before construction
To help promote effective collaboration between the two communities we need
to clear up a few misconceptions that each community may have with the other.
There are several that UX practitioners may have about the agile community:
- Agilists don't model. The actual fact is that agile
practitioners do in fact model, it's just that they discourage extensive
up-front design work in favor of more agile approaches to modeling.
- Agilists are continually deploying software into production.
Although some teams do this, it's not the norm. What is common is to
deliver working software on a regular basis, perhaps every few weeks, into
an internal environment for system and user testing. Deployment into
production may occur every six-to-twelve months, if not longer, based on
need and the ability of our end users to accept new releases.
- XP is the only game in town. This is a serious misunderstanding
because UX practitioners who believe this miss agile methods such as
Scrum, Agile Modeling, DAD,
and DSDM which are arguably more attuned to their philosophies.
- There is no role for UX practitioners. Many agile methods forgo
the concept of specific roles in favor of more generic roles such as
developer/programming, coach/leader, and customer/stakeholder.
- Agilists aren't specialists. This is partly true because agilists
prefer to be "generalizing specialists".
- User interfaces shouldn't be refactored. Many people fear
refactoring the user interface, often because they believe the UX issues
will be forgotten or that end users will not be able to deal with continual
change. The reality is that UI
refactoring results in the slow but safe evolution of the UI, thereby
improving its design. To ensure that UX issues aren't forgotten,
people with UX skills should be involved with the development and evolution
of the UI. Yes, the UI changes, hopefully for the better, but
the only people affected by the changes on a continual basis are those
actively involved in user testing. And when you stop to think about it,
shouldn't developers act on the findings of usability testing efforts and
thereby improve the UI?
The agile community equally suffers from debilitating misperceptions about
the UX community:
- All you need is a good set of UI guidelines. That's a good
start, but there is a fair bit more to UX than creating consistent UIs.
- Working closely with stakeholders is good enough. That's also a
good start, but Jokela and Abrahamsson (2004) found that even a close and
frequent co-operation between developers and stakeholders does not ensure
good usability at all.
- UX is just about UI design. UI design is clearly a part of
but so is understanding how your users will work with your system and what
their goals for using the system are so that you can build something that is
usable by them. This requires significant modeling and collaboration skills
- UX relies on comprehensive up-front modeling. Although some
people (Cooper 2004) in the
UX community want you to believe that, many
others believe different. In many ways getting the UI right is similar
to getting your
architecture right, it often requires some up front thought although
that doesn't imply that you need to take a serial approach to development.
ASD practitioners are very pragmatic when it comes to modeling. The Agile
Modeling (AM) methodology describes in detail how agilists approach both
modeling and documentation. Figure 2 overviews the
lifecycle of an Agile
Model Driven Development (AMDD) approach to ASD, an approach which
originally grew out of the XP community but which seems to capture the essence
of the modeling approach on agile projects in general. Each box in the diagram
represents a development activity. The initial modeling activity during
includes two main sub-activities,
architecture envisioning which are done iteratively in parallel. The
model storming and
implementation activities potentially occur during any iteration, including
iteration 0 (yes, the rumors are true, agile practitioners will often implement
working software the very first week of a project). The time indicated in each
box represents the length of an average session: for example, during development
you'll often model storm for a few minutes with a stakeholder to explore a
requirement and then code for several hours.
Figure 2. The Agile Model Driven Development (AMDD)
The initial envisionment effort is typically performed during the first week of a
project. For short projects (perhaps several weeks in length) you may do this
work in the first few hours and for long projects (perhaps on the order of
twelve or more months) you may decide to invest up to two weeks in this effort.
There are two aspects to the initial envisioning effort:
- Requirements modeling. You need to identify the
requirements as well as the scope of the current release. The goal is to
get a good gut feel what the project is all about, and to do that you will
likely need to create an initial usage model to explore how users will work
with your system (e.g. use cases or usage scenarios), an initial domain
model which identifies fundamental business entity types and the
relationships between then, and optionally other important artifacts
exploring technical requirements.
- Architectural modeling. The goal of the
modeling effort is to try to identify an architecture that has a good chance
of working. 99% of the time agile practitioners simply gather around a
whiteboard and create free-form diagrams as they discuss various
architectural strategies. When the UI architecture is an important
consideration, agile modelers will create a
diagram (see Figure 3) which explores initial
relationships between important
screens/pages and reports (thereby giving you a birds-eye view of the UI,
enabling you to ask fundamental usability questions).
Figure 3. A UI navigation model for a university
system (hand drawn).
In later iterations your initial models will evolve as you learn more, but during
Iteration 0 the goal is to get something that is just barely good enough so that the
team can get going. You don't need to model a lot of detail, and I cannot
stress this enough: the goal is to build a shared understanding, it isn't to
write detailed documentation.
During development iterations the vast majority of modeling sessions involve a
few people, usually just two or three, who discuss an issue while sketching on
paper or a whiteboard. These model storming sessions are "just in time" (JIT)
efforts: you identify an issue which you need to resolve, you quickly grab a few
team mates who can help you, the group explores the issue, and then everyone
continues on as before.
Sometimes modeling storming isn't enough. Perhaps you need to model complex
requirements which require input from someone outside of your immediate team, or
perhaps you need to model a legacy asset which can take a significant amount of
time. In other words, you may need to
model a bit ahead of actually implementing
a requirement. This is actually a rare occurrence, regardless of
what traditional modelers may hope for, but it does happen every so often.
All of this begs the question "how can UX activities fit in to an agile project?" The
easy answer is that agilists need to adopt usage-oriented requirements
artifacts, such as personas,
The quickly explore user interface requirements, it is common to create
fidelity prototypes out of paper, such as the one in
Figure 4 created from a flip chart sheet and sticky notes. This allows
to you quickly explore the UI without doing a lot of up front work, and even
enables agile usability testing. In fact several agile
methodologies have done so, in particular Disciplined Agile Delivery (DAD). After
this, it's not so easy.
An abstract UI prototype created
UX practitioners can rail about the need for doing lots of design up front,
but that message falls on deaf ears within the agile community. The bottom line
is that from an agile perspective, traditional UX techniques aren't very usable
for them, which is rather ironic when you stop and think about it. To make
techniques usable to agile practitioners, they must reflect the
agile lifecycle. Luckily, this can be accomplished by:
- Doing some UI modeling up front. You need to establish 3 things:
Overall organization for the parts of the UI that fits with the structure of
user tasks; a common scheme for navigation among all the parts; a visual and
interaction scheme that provides a consistent look-and-feel to support user
tasks. Yes, this takes some up-front work, but for the vast majority of
systems this could easily be accomplished during Iteration 0. Remember
that the amount of modeling that you need to do up front is dependent upon
the context -- some teams will need to do more than others.
- Using modeling tools which reflect agile practices. For example,
XP teams prefer to work with index cards, not documents, and
prefer whiteboard sketches. Luckily paper and whiteboards are common tools
with many UX practitioners.
- Modeling a bit ahead when appropriate. If you need to, explore
important aspects of the UI before you implement them. Conducting a user
session often involves scheduling specialized facilities that are in high
demand, and making appointments with the appropriate stakeholders.
Therefore you need to
adopt the practice of
- Doing UI development on a JIT basis the majority of the time.
The UI is important, but then again so are many other aspects of a system
(such as the network, the database design, and so on) and therefore the UI
should be treated differently only when appropriate. When the goal of
design is limited to a single feature or small set of features it is
effective to perform UX activities as part of the development of those
features during the iteration.
- Adopt UX-friendly requirements artifacts. As I pointed out
earlier, some agile methods such as DSDM and DAD already do this. This same philosophy could be applied in
XP, and frankly many XP teams already capture the need to implement a UI
view as a user story.
For the sake of this article, user testing encompasses:
The agile community has embraced the importance
of acceptance testing, having built tools such as Fit (Mugridge and Cunningham
2005) to help automate it. The automated tests will be run often, at least
daily if not several times a day. Manual user testing, on the other hand, is
typically done an iteration behind. At the end of an iteration, many agile
teams will deploy the working system into a QA/testing environment where user
and system testing is performed. The team continues on, developing version N+1
of the system while obtaining defect reports pertaining to version N. Defect
reports are treated just like any other requirement: they are estimated,
prioritized, and put on the requirements stack to be addressed at some point in
Usability testing is considered optional, and I have no doubt that many UX
practitioners will find that frustrating. Agile teams are little different from
traditional teams in this respect, they very likely don't appreciate the need
for usability testing (or other UX practices for that matter). True usability
testing requires repeated testing with numbers of users under controlled
settings. Just like acceptance testing can be done regularly throughout
development, so can usability testing. On agile projects usability testing
should occur during the user testing effort after each iteration, assuming of
course that someone on the team has those skills.
There is a range to usability
testing. At the "agile
end" of the spectrum is
Patton's agile usability
testing. In a workshop at
the Agile 2006 conference Patton
described a technique where
people simulate the system using
abstract prototypes (see
Using this strategy one
developer simulates the system.
They're not allowed to speak,
only help the user navigate
between "screens" (the paper
prototypes). At least one
user, although two seems to work
well, works through a scenario
via the paper prototype.
For example, in a university
system a scenario might be that
they try to enroll in a seminar.
The user(s) should
narrate/discuss what they're
thinking as they use the system.
One or more developers act as
observers, taking notes but not
fixing the paper prototypes as
the users work with the system.
At the formal end of the
spectrum is traditional
usability testing where users
are observed in a
usability lab which is
typically a room where a user
can be observed working with the
system. Usability labs
often have one-way windows which
enable people to watch the user,
often a valuable learning
experience for developers who
mistakenly think that they've
done a great UI design job.
Some labs will even be outfitted
with cameras so that you can
record the exact interactions,
in the hope of replaying the
usage and hopefully acting to
improve the design to support
more effective usage.
There is nothing stopping you
from taking an agile approach to
usability testing at some points
in your project and formal
usability testing at other
points. Nor is there
anything stopping you from doing
something in between.
Usage scenario testing is a
technique that can be used to
test the logic captured within
your design before you invest in
implementing it. This is
an inclusive technique which
stakeholders can be actively
involved with which can and
should be performed in parallel
with your modeling efforts to
ensure that your model
accurately reflects your
Using one ore more usage
scenarios (a series of steps
describing how someone works
with your system) you
walkthrough your class model and
validate that it is able to
support the scenario(s). If it
does not, you update your model
appropriately (or your code, as
the case may be). Figure 5
presents usage scenario testing
from the point of view of
walking through a class model,
although the same logic can also
be followed to validate your
user interface prototype (even
if it is an abstract prototype).
The process of usage scenario
testing (for a class model).
If the agile and UX communities are going to work together effectively, they
need to find a middle ground. I believe that middle ground exists, but that
both communities need to adopt several changes in order to succeed. First,
agile professionals must:
- Learn UX skills. Developers should be trained in, and adopt
into their practices, UX techniques. This will enable developers to work
more collaboratively and effectively with UX practitioners.
- Accept that usability is a critical quality factor. Luckily,
agile practitioners are "quality infected" – they understand the importance
of doing high-quality work and have a proven track record of adopting
techniques such as test-first programming, code refactoring, and database
refactoring. Good usability of an end product can be ensured only by
systematic usability engineering activities during the development
- Adopt UI and usage style guidelines. Developers must understand
that not only should their code follow common guidelines, so should their
Similarly, UX practitioners must make some changes. They need to:
- Go beyond UX. I believe that many of the challenges experienced
between programmers and UX practitioners are due to
overspecialization of roles and hand-offs between people in those roles.
Agilists have mostly abandoning the concept of
building teams of specialists and instead favor teams of
generalizing specialists. The implication is that although UX practitioners bring a critical skillset to a development team, they still need to learn a wider range of
skills to become truly effective. Furthermore, agile practitioners
have tightened the feedback loop on software projects, and thereby reduced
both risk and cost, by increased collaboration which in turn results in
- Become embedded in ASD teams. By embedding UX practitioners on
agile teams, not only will this increase the chance UX issues are
addressed, it will help to promote UX skills within the agile community
because people pick up new skills from one another as they collaborate.
- Give ASD approaches a chance. Kent Beck suggested to Alan Cooper
that a week be invested at the beginning of a project to explore interaction
issues, although Cooper believed that wasn't sufficient. The easiest way to
find out who is right is to actually try it in practice.
- Start looking beyond XP. I've said it before and I'll say it
again, there is more to ASD than XP. Agile methodologies are flexible,
they're not meant to be used "out of the box" but instead to be tailored to
meet the exact situation which the project team finds itself in. To address
UX concerns you will very likely find that you need to tailor some of the
principles and practices of Agile Modeling and/or the techniques of
User-Centered Design into your base software process.
It is very easy to suggest that agile practitioners take the time to learn
UX skills, and to adopt appropriate guidelines, but the reality is that these
skills are competing for attention along with other equally important skills
such as database design and modeling. To make matters worse, few
developer-oriented books cover UI/usability issues, and the few that do such as
The Object Primer rarely seem to devote more than a chapter to it. I fear
that many agile practitioners aren't even aware of the issue.
Similarly, UX practitioners receive mixed signals. Although I'm calling for
them to become
generalizing specialists, the industry still rewards specialization – UX
specialists are paid very well, and most organizations expect them to focus on
doing that specific sort of work. Agile practitioners also suffer from this
challenge: Why take an introductory UI design course when you can take a Java
programming course which leads to certification and more pay?
It's also easy to say that UX professionals should be embedded into agile
projects, but that only works when you have UX professionals available. Few
organizations have such people on staff, and worse yet believes that few
organizations think in terms of interaction design as part of the requirements
planning or project planning process. Therefore, many organizations may not see
the need to hire anyone with these skills.
The lack of UI design ownership means that everyone wants to be involved with
UI design, regardless of their skill level, which can lead to design by
committee. Although a common philosophy throughout the agile community is that
agile practitioners should have the humility to know their limits, and the
respect of others with the appropriate skills to address a specific issue, it
doesn't always work out that way – apparently agile practitioners are still only
It is possible for UX practitioners to make a valuable contribution on agile
teams, and I suspect that there is a much greater chance of them succeeding as
compared to working with traditional teams. Up until this point the UX
community has had little success in their attempts to become an active part of
mainstream development teams, so it is time for a new approach. My suggestion
is that UX practitioners should work closely with the agile inmates to try to
get the prison under reasonable control.
I'd like to acknowledge the input from Paul Oldfield and Tim Tuxworth on the
Agile Modeling list which helped me to evolve this article.