Agile Business Analysts: Rethinking the Role of Business Analysts

Agile business analysts (agile BAs) work with stakeholders to explore and capture their needs so that they can be shared with software developers. Traditionally, organizations have an IT role called analyst, and will often differentiate between various types:

Requirements analysts who are responsible for requirements elicitation
Systems analysts who are responsible for analyzing the requirements to determine the system needs to fulfill those requirements
Business analysts who are responsible for understanding the business and making recommendations for improvement
Business system analysts whose responsibilities are a combination of those of a requirements analyst, business analyst, and a system analyst.

The focus of this discussion is on business system analysts (BSAs) even though many of the issues (or flavors thereof) are pertinent to the other analyst types. BSAs typically have experience in a wide range of techniques, including interviewing, structured meeting approaches such as Joint Application Development (JAD), modeling sessions, and model reviews. Good BSAs have a good understanding of the business domain and are typically “people persons”. This article covers:

1. Why Have BSAs?

Why have many traditional organizations adopted the idea of having BSAs on staff? There are three general reasons, all of which I believe are misguided:

Developers can’t elicit requirements. The common stereotype is that developers don’t have the communication and modeling skills necessary to elicit requirements effectively, and sometimes they don’t even have the inclination to do so. At best this is a self-fulfilling prophecy: if you believe that developers can’t do this sort of work then you won’t give them the training nor the opportunities to gain the skills on the job. My experience is that it is possible to train all but one or two percent of developers in the sort of communication skills and business knowledge required to successfully perform business analysis, although that doesn’t imply that all of them understand that this is important nor does it imply that they want to gain these skills.
Stakeholders can’t model and document their own requirements. In one respect this is true, particularly when stakeholders aren’t given any sort of training or support. With the use of inclusive tools and techniques, and with a bit of training, stakeholders can become active participants in the development efforts. Yes, stakeholders still need someone to guide and facilitate their efforts, but they can do the majority of the work.
You need analysis experts. You definitely need to do analysis, but whether you need someone who just does that is a really big assumption. Agile developers are generalizing specialists, people with one or more specialties, a general understanding of the software process, and a knowledge of the domain. One of their specialties might be in analysis, or then again it might not. It is unreasonable to expect everyone to be an expert at every aspect of software development, but it is reasonable to expect IT professionals to have some analysis skills and for some people to have deep skill in this activity (amongst many of their skills).

Unfortunate observation: You need to do analysis, but that doesn’t imply that you need analysts.

2. The Traditional Activities of an Analyst

A BSA on a traditional software development team will perform one or more of the following activities:

Scope the system. During the initial phase of an initiative, often called “iteration zero” or simply the inception phase, BSAs may be the only “development staff” assigned to the team. At this point they will work with key stakeholders to formulate and communicate the business vision, to envision initial requirements , and to scope the effort. Their fundamental goal is to get the team focused early by translating the initial high-level vision into something realistic. They may also help to identify potential areas of automation and even to aid in reengineering the underlying business process.
Translate business needs. A major responsibility of BSAs is to work with stakeholders to translate their requirements into something that developers can understand as well as to translate the resulting questions that the developers have into something the stakeholders can understand. This is an iterative process. An important part of this is to the distillation of the differing messages of various stakeholders into a single, consistent vision. This task often includes significant negotiation and political maneuvering. In this respect a BSA will act as a knowledge integrator. Furthermore, BSAs often find themselves spending significant time in meetings, thereby saving the rest of the development team from this inefficient use of their time.
Translate technical issues. BSAs will also explain technical/architectural complexities to stakeholders, such as why your HTML-based application can’t have as slick of a user interface as a Visual Basic application. BSAs often explain what the developers are doing and why they need to do it, including explanations of the basis of schedules and estimates.
Model and document. BSAs will often work with stakeholders to identify, model, and then document their requirements and business domain details.
Act as a communication broker. BSAs typically have very good connections within the business community and therefore are in a position to help development teams find the right people to work with.
Political mentor. BSAs often help teams through the political minefields within their organizations, particularly when the BSA has worked within the same organization for several years.
Test and validation. BSAs will work with stakeholders to validate their requirements and analysis models via techniques such as reviews, walkthroughs, and play acting. BSAs will often aid in writing user acceptance test (UAT) cases and will be a liaison between stakeholders and your testing organization during UAT.
Represent stakeholders. When teams don’t have direct access to their stakeholders, clearly not a good situation, BSAs will act as “stakeholder surrogates”. Typically developers will treat a BSA as the “customer” from which requirements, domain information, and business priorities are provided. The BSA in turn will work with the stakeholders to obtain information and to verify decisions.

3. Business Analysts Gone Awry

In theory the idea of having traditional BSAs involved with a team should work quite well, and in practice it often does. The best analysts are organized and great communicators, having the ability to distill the critical information from your stakeholders out of the “information noise” that they provide – often through a wide range of modeling techniques. For many organizations the addition of BSAs clearly improved the quality of the requirements elicitation and analysis modeling. It also opened up communication between the “tech weenies” in IT and the “business morons” that the system was being built for.

Clearly this was a step in the right direction. However, some very common problems have been known to occur:

BSAs often lack the right skills. Many organizations have difficulties hiring the right people and/or nurturing the right skills in people. The end result is that people are often thrust into the role of BSA but do not have the skills to fulfill that role, nor do they have an opportunity to gain those skills — this is clearly a mistake.
BSAs can have undue influence. A strong-willed BSA may inadvertently influence a team, perhaps by playing down requirements that they don’t agree with or even influencing architecture decisions by being biased towards one type of analysis technique (such as focusing just on use cases or just on data models). This is particularly dangerous when BSAs act as stakeholder surrogates AND the developers and stakeholders have little interaction other than via the BSA
BSAs can be out of date. Having previous development experience is critical for a BSA because it grounds them in reality. However, it may also lead them astray. For example someone with a data-intensive background may struggle when working with a development team that is using object technology (and vice versa) because they don’t know which issues they need to focus on. Minimally BSAs should have an understanding of the technologies that their team is working with, but this isn’t as good has having hands-on experience with those technologies
BSAs can act as a communication barrier. Although BSAs can act as a communication bridge between the two groups they also act as a communication barrier. On the Agile Modeling (AM) mailing list Ron Jeffries depicted the communication approach of traditional BSAs to look like “stakeholder => BSA => developer => BSA => stakeholder”. This looks a lot like the game of telephone tag, doesn’t it? When you play this game you quickly discover that the signal degrades between hops, decreasing the chance that the actual message gets through successfully. Although a highly skilled BSA, one with excellent communication skills, will reduce this problem it will never go away completely.
BSAs can reduce stakeholder influence. An interesting implication is that your stakeholders don’t have as much influence over the software as they may think. They’re influencing the traditional BSA, and the models and documentation that the BSA creates, but have no direct influence over what the developers are creating. This can even be said of user interface prototyping efforts, particularly when those efforts are led by a traditional BSA.
BSAs often over analyze. Another inherent problem with traditional BSAs is their tendency to actually do their perceived jobs. What? When you specialize in something you will naturally focus on that task. The implication is that the task of business analysis will be stretched out, instead of Iterating to Another Artifact as AM suggests, such as a design model or even source code, a BSA will likely focus on expanding the artifacts that they specialize in. This isn’t a problem that is specific to BSAs, instead it is a problem of people who are overly specialized (agile teams prefer people who are generalizing specialists, not just specialists). Not only is your development effort likely to devolve into a more serial process, instead of an evolutionary approach, the BSAs are likely to develop more documentation than is required. Your goal should be to create models and documents which are just barely good enough.
BSAs can reduce feedback. When analysts are only responsible for analysis efforts, for creating models and documents that are meant to be handed off to developers (something which occurs far too often on geographically distributed teams), there is less opportunity for feedback. There is the danger that they will create theoretically sound models, and make unrealistic promises based on those models to your stakeholders, that don’t work in practice. The feedback of people working with software is critical, feedback that you can’t get from models and documents. The BSA quickly learns what actually works, and from the resulting changes requested by stakeholders quickly improves their analyst skills because they recognize mistakes they made.
BSAs can reduce opportunities for developers to gain communication skills. With traditional BSAs in place developers aren’t given as many opportunities to improve their own communication skills, skills that are critical to success in today’s environment. Nor are they given the opportunity to work closely with business experts, opportunities that would allow them to learn about the nuances of the business environment and thus increase the chance of them building a system that actually meets their users needs. Nor are they given the chance to discover that the “business morons” are actually pretty smart after all, with interesting knowledge worth learning. Similarly, stakeholders miss the opportunity to learn about how software is developed, arguably a good thing in some organizations, and to discover that the “tech weenies” are actually very interesting people. Houston, we have a problem.

The role of BSA makes sense when barriers to communication (such as time, distance, or lack of communication skills among developers) have been erected between stakeholders and developers. BSAs compensate for these barriers.

4. Towards Agile Business Analysts

Fundamentally the activities performed by traditional BSAs are varied, but a crucial goal was always to improve the communication between developers and stakeholders. In many “traditional” organizations that meant that BSAs formed a bridge between the two groups, a definite improvement in many situations, although at the same time erected barriers. Now is the time to take the next step, to become more agile and have BSAs become the communication mentors/coaches on teams. This isn’t to say that every existing BSA is qualified to become a communication mentor, but chances are that your pool of BSAs is a good place to start looking for likely candidates.

How can BSAs become more effective? AM suggests that development teams follows the practice Active Stakeholder Participation , that they work closely with their stakeholders. The goal of this practice is to reduce the feedback loop and thereby improve communication. However, the way that you work with your stakeholders will in part be determined by your team’s organization. An interesting implication is that the role that a BSA takes on changes dramatically based on the communication options available to your team. There are three categories of team organization that need to be considered with respect to how an “Agile BSA” will act on a team:

4.1 One Room – Developers and Stakeholders are Co-Located

First, let’s assume that you are able to co-locate your development team with your stakeholders, an ideal situation that you should always strive to achieve. In this situation an agile BSA should be just another one of the developers on the team, likely leading requirements and analysis modeling efforts as needed. They would actively work to facilitate communication within the team, mentoring developers in BSA skills and perhaps even mentor stakeholders in development skills. Ron Jeffries says it well in his article “Business Analysis in Extreme Programming”:

This begins with a simple change of focus, from “We’ll go out and find out what the customers want and bring it back”, to “We’ll go help the customers figure out what they want, so they can tell us”. This change of focus leaves control in the hands of the customers, and makes all the difference.

A critical skill that an agile BSA brings to the table is the ability to investigate and understand how the business actually works, as opposed to how developers think it works or want it to work, an ability that they should be able to mentor developers in. When co-located, Agile BSAs would also perform non-BSA activities as well, perhaps pair programming with one of the developers on some of the business code or working with users to develop acceptance tests. This way the agile BSA would grow his or her own skill set over time and would help others to do so as well.

An implication of this approach is that the “traditional BSA” is getting dropped in favor of a developer with very good communication skills. This reflects AM’s philosophy that good developers are generalizing specialists, people who have a general understanding of the complete software development lifecycle and one or more specialties: in this case business system analysis. The danger of this approach is that the BSA can’t be in the role of “conflict arbiter” when developers and stakeholders don’t see eye to eye – the BSA is a developer and therefore is likely to be seen as biased in such situations. Conflicts would need to be resolved by the team leader/coach or one or more senior stakeholders.

4.2 Over the Wall – Single Location But Not Co-Located

A common situation is for teams and stakeholders to be located in the same building, or geographical area, but not together – perhaps the team is one floor and the stakeholders are on different floors, or perhaps they’re in different buildings in the same city. Furthermore, the stakeholders are typically not available on a full time basis to the team. Common scenarios include stakeholders that can co-locate one or two days a week and be available as needed on other days; stakeholders that can be available for physical and teleconference meetings several times a week; and stakeholders whose availability is severely limited and often can only give you an hour or two a week. Clearly none of these scenarios are ideal, but unfortunately they are all too common.

In these situations the agile BSA takes on some of the aspects of a traditional BSA. In particular, in situations where the stakeholders are not able to partially co-locate with the development team the BSA will need to work with them in their own environment. This will likely entail modeling sessions and interviews with the stakeholders. The Agile BSA will typically bring one or two other developers with them to these sessions, that way the developers can not only learn the business knowledge that they need they can also gain BSA skills in the process. This will also help to build bonds between the developers and stakeholders.

The agile BSA will also model a bit ahead of the team, gathering the information which they need a few days or weeks before it’s actually needed: too much modeling ahead may result in the sort of wastage seen with a big requirements up front (BRUF) approach.

4.3 Across the Network – Dispersed/Distributed Development

In this scenario the team is located in a different location than their stakeholders, and in the extreme the team itself is in several locations as are the stakeholders. Large organizations, consortiums of organizations, and organizations that outsource development to other companies (often overseas) often find themselves in this situation. I highly suggest that you avoid distributed development situations such as this because of the increased communication risk inherent in this approach. When a team is dispersed/distributed the opportunities for face-to-face discussion, the most effective way to communicate, are reduced. Instead you are forced to rely on less productive communication strategies – such as video conferencing, telephone conversations, and written documentation – and thereby dramatically increase risk. Rich communication between developers and stakeholders is a fundamental aspect of agile software development. It is possible to take an agile approach in these situations, but it’s very hard.

In this situation the role of an Agile BSA devolves into something closer to that of a traditional BSA, that of trying to bridge the gulf between developers and stakeholders. They would realize that this situation is not in the best interest of the initiative, that they really want to have the stakeholders working with the developers and therefore should try to get these people together as often as possible. Perhaps they’ll facilitate conference calls, video conferences, and online meetings between the two groups. Perhaps they’ll help to arrange periodic co-locations where some stakeholders travel to the developers, or vice versa, to enable some direct interaction between the groups.

5. Agile Business Analysts as Product Owners?

An agile business analyst make take on the agile role of “product owner” on an agile team. There are several aspects about this role which I think are critical to understand:

Product owners bridge the communication between the team and their stakeholders . As Figure 1 shows, in practice there proves to be two critical aspects to this role: first as a stakeholder proxy within the development team and second as a team representative to the overall stakeholder community as a whole.
Product owners are empowered. The product owner is the single person whom is responsible for prioritizing requirements, as Figure 2 depicts (Note: Figure 2 shows just one of several strategies supported by the DAD framework), and for providing the details explaining the requirements to the team.
Product owners facilitate communication. Product owners need good communication skills, including agile modeling skills. They also need to know who the stakeholders are, interact with them regularly, and when needed facilitate the interaction which the team has with them.
Product owners prefer direct means of communication. There is significant evidence that the worst way to communicate information between people is via documentation, and that the most effective way is face-to-face around a shared sketching environment. For some BSAs, particularly those working in traditional environments, this can be a significant change for them as product owners, just like agilists in general, prefer to avoid interim documentation and instead use more effective means for agile communication.
Product owners need many business analysis skills. They need to be skilled in techniques to identify stakeholder needs, negotiate priorities between repeating stakeholder factions, and then collaborate with developers to ensure that the requirements are implemented effectively. This is business analysis 101 stuff.

Bottom line is that in many ways a product owner is a business analyst who is empowered to make decisions in a timely manner who focuses on high value activities (e.g. they find ways to avoid interim specifications).

Figure 1. Scaling the role of Product Owner (click to enlarge).

Product owners

Figure 2. Agile requirements change management process.

Although a traditional business analyst could fill the role of product owner, particularly those from the business side of your organization, there is a risk that they will fall into their old habits of communicating via documentation. If they can overcome these tendencies and instead focus on collaboration, knowledge transfer, and skills transfer then becoming a product owner on an agile team is clearly a viable strategy for existing BSAs.

6. Business Analysis at Scale?

As the situation becomes more complex, the need for someone(s) focused on business analysis grows. In particular, this seems to be true for the following way of working (WoW) tailoring factors (overviewed in Figure 3 below):

Team size. Large agile teams, like large traditional teams, are typically organized as teams of subteams. Each of these subteams requires a product owner, sometimes a single individual can be the product owner for several subteams (although this can be difficult), and these product owners need to coordinate their work stacks between them. The product owner team needs someone in the chief product owner role to coordinate their overall activities, and a business analyst is a very good option for this role. As pointed out earlier, it is possible for BSAs to fill this role. Furthermore, the complexities of large teams will motivate you to have product owners with solid business analysis skills.
Geographical distribution. The more geographically distributed a team is the greater the communication risk. Although it isn’t ideal, the reality is that there will be greater need for documenting requirements and other business information to share it with distributed team members, as I described earlier in the over the wall and across the network scenarios. Someone(s) in a BSA role can potentially help the team overcome these communication risks. Note that geographical distribution and large team size have a tendency to go hand-in-hand.
Regulatory compliance. When an agile team finds itself in a regulatory situation it often discovers that it must increase the formalism in its approach to requirements, potentially requiring someone with more traditional BSA skills. Or perhaps the implication is that you need someone on the team with technical writing ability, also a viable option. The best advice that I can give you with respect to regulatory situations is to invest the time to actually read the regulations. Many teams suffer from overly formal or bureaucratic strategies for complying to regulations because they allowed bureaucrats to interpret the regulations instead of practical people.
Organization distribution. When you have several organizations involved with a development initiative, perhaps you have a consulting organization involved or you’re partnering with other companies/divisions, there is often the need for greater formalism in your approach to requirements.
Domain complexity. As the domain complexity rises, so does the need for business analysis. You will approach the development of an air traffic control system differently than the development of informational website, for example.

Figure 3. WoW tailoring factors (click to learn more).

The bottom line is that when you find yourself in a scaling situations, and yes there is e vidence that agile scales, the need for specialist roles such as BSAs often increases. However, just because there is greater need for someone with more traditional BSA skills you must still remember that agile documentation principles and practices are still applicable — just because one or more scaling factors applies it doesn’t imply that detailed speculations are required.

7. Parting Thoughts

The traditional software development community has significant cultural barriers to overcome when it comes to over modeling and overly detailed specifications, and as a result experienced business analysts often struggle to fit into agile teams at first. We need to do business analysis on agile teams but that doesn’t imply that we need specialized business analysts. There is an opportunity for BSAs to become effective members of agile teams, they clearly have value to add, but they need to be prepared to rethink they way that they approach their jobs. This includes a greater focus on greater collaboration, on knowledge sharing, on skills transfer, and on becoming a generalizing specialist. Agile teams require people with greater flexibility, greater discipline, and the willingness to work in an evolutionary manner. This will take time and effort, but in the end I expect that this investment will be well worth it.