Agile Modeling (AM) is a practices-based software process whose scope is to describe how to model and document in an effective and agile manner.  The practices of AM should be used, ideally in whole, to enhance other, more complete software process such as eXtreme Programming (XP), the Microsoft Solutions Framework (MSF) for Agile, the Rational Unified Process (RUP), the OpenUp, the Agile Unified Process (AUP), and the Enterprise Unified Process (EUP) to name a few.  These processes cover a wider scope than AM, in the first three cases the development process and in the fourth the full software process including both development and production.  Although these processes all include modeling and documentation activities, in one form or the other, there is definitely room for improvement.  In the case of XP and MSF the modeling processes should be better defined, and in the case of both the RUP and the EUP the modeling processes could definitely stand to be made more agile.

In this article I explore in detail how AM can be used in conjunction with the various instantiations of the Unified Process (UP), including but not limited to the RUP and the EUP.  To do so, I discuss

• How modeling works in the Unified Process

• How good is the fit between AM and RUP?

• Case study

• Adopting AM on an UP project

• How does this work?

• References and Suggested Reading

How Modeling Works in the Unified Process

All efforts, including modeling, is organized into disciplines (formerly called workflows) in the UP and is performed in an iterative and incremental manner.  The lifecycles of the AUP and EUP are presented in Figure 1 and Figure 2 respectively.  The AUP is a subset of the RUP and the EUP a superset of the it.  I like to say that the UP is serial in the large and iterative in the small.  The six phases of the EUP clearly occur in a serial manner over time, at the beginning of an UP project your focus is on project initiation activities during the Inception phase, once your initial scope is understood your major focus becomes requirements analysis and architecture evolution during the Elaboration phase, then your focus shifts to building your system during the Construction phase, then you deliver your software during the Transition phase, you operate and support your software in the Production phase, and finally you remove it from production during the Retirement phase.  However, on a day-to-day basis you are working in an iterative manner, perhaps doing some modeling, some implementation, some testing, and some management activities.

Figure 1. The lifecycle of the Agile Unified Process (AUP).

Figure 2. The lifecycle for the Enterprise Unified Process (EUP).

Table 1.  The Modeling Disciplines of the Unified Process.

How Good is the Fit Between AM and UP?

Now that we understand the basics of how modeling in the UP works, we can examine how well AM fits in with it.  Luckily many of AM’s principles and practices are arguably a part of the UP already, although perhaps not as explicitly as I would like.  Table 2 presents an examination of how well each individual AM practice is currently implemented in the UP, if at all, and discusses how to adopt the practice within the scope of the UP.  My experience is that it is relatively straightforward for UP teams to adopt AM practices if they choose to do so.  This is because the UP is very flexible, one of its underlying principles is that you should tailor it to meet your unique needs, making it easy to merge AM practices into the UP.

Table 2. The Fit Between UP and AM.

Something that is important to understand is that for AM to be successful the culture of your organization must be open to the concepts, values, and principles of agile software development.  The problem is that the UP is often adopted by organizations that either implicitly or explicitly do not accept the values of agile software development.  Their focus is often on following processes and using tools, the RUP product clearly defines many processes and describes how to apply Rational’s tools effectively on software projects, and therefore the RUP is clearly attractive to them.  Unfortunately this goes against the agile value of preferring individuals and interactions over processes and tools.  When the culture of an organization is documentation centric they may find the UP appealing because you can instantiate it in such a way as to result in the creation of significant amounts of documentation (you can also instantiate it to result in very little documentation, remember, the UP is flexible).   If an organization is documentation centric then this aspect of its culture goes against agile software development’s value of preferring working software over comprehensive documentation.  This organization may still successfully adopt, tailor, and instantiate the UP but be unable to follow many of AM’s principles and practices effectively because it does not have an agile culture (see the article When Does(n't) AM Make Sense).  My point is that how well AM and UP will fit together in your organization depends largely on your organization’s culture and not so much on the UP itself.   You can easily use the techniques of AM to improve your UP modeling efforts, but to be effective you will find that you need to overcome cultural challenges within your organization. 

Case Study: Wayne Enterprises

In early 2001 I was involved with a development project at Wayne Enterprises, the name has been changed to protect the innocent, a medium-sized financial institution.  They had adopted the RUP and wanted to tailor it with Enterprise Management concepts from the EUP, in particular they wanted to ensure that the Inception phase and Elaboration phase efforts of individual projects were supported by enterprise-level requirements, architecture, and reuse efforts.  They also wanted to ensure that their modeling processes were effective, they weren’t convinced that the UML was sufficient for their needs (it wasn’t) and they wanted to ensure that legacy integration modeling was reflected in their tailoring of the RUP.  I was eager to apply AM on a RUP project and they liked the basic concept of the methodology and were willing to give it a try. 

The culture of Wayne Enterprises was conducive to change, which was a huge benefit.  Management was open minded and willing to try new techniques, hence the RUP.  As you would expect the developers were also eager to try new techniques and work with new technologies, although nowhere near as enamored with the RUP as management was.  Our approach was to pilot the RUP on a small project – there was seven developers, one manager, one senior manager and four users – over a period of seven months.  The project was of mid-level importance, it would be noticeable if we failed but it wouldn’t put the company at risk.  It was their first real J2EE project, and the team was fairly typical: there was two mid-level Java developers; a senior consultant experienced in EJB; another senior developer with a C++, OMT, and UML background; a experienced business analyst that was new to use cases and object-oriented development; a DB2 database administrator (DBA) learning Oracle; and a good QA/tester person with some Java testing background.

At the beginning of the project we tailored the RUP with the practices of AM and a portion of the EUP, the end result of which was the creation of something the RUP calls a development case.  This effort took several days, I had all the raw material that we needed to add, and everyone recognized that there was little value in spending much more time than that – everybody wanted to get to work on the project.  My involvement with the project was to help them with the initial creation of the development case and initial overview training to the development team.  I also periodically reviewed the effort, both from an architectural point of view and from a software process point of view – their first goal was to develop a good system and their second goal was to ensure that they were learning how to do so effectively. 

During the Inception phase the team produced a high-level requirements model, about 20 use cases and a 15-page supplementary specification that the business analyst evolved throughout the project.  A lot of this documentation was initially written by the user representatives, after breaking out of requirements modeling sessions led by the business analyst and attended by one or two developers at a time.  The Elaboration phase was mostly white boarding performed by the entire team and some coding and testing of an end-to-end prototype to prove the architecture.  They kept the models as simple as they possibly could, some of this was motivated by the realization that they weren’t familiar with the technology and therefore it was futile to over model things, and part of it was because they were eager to code EJBs, servlets, and JSPs.  This phase went a little slower than usual due to the team’s inexperience with setting up EJB, developing EJBs, and deploying them.  The Elaboration phase was a good learning experience for the team because it gave them experience in the technology and in following their tailored process.  It also helped to dampen the DBA’s argument that the data architecture was the primary concern, granted, the legacy data issues were challenging, and was an important stepping stone towards building a common understanding of how to work together as a team.   During the Construction phase the use of Together/J and the data modeling tool was very common, although white boarding was still used for high-level thinking and discussions.  From what I could gather the architecture models stayed up on the white boards, the critical ones were a layered deployment diagram and a high-level sketch of the structure of the object schema.  The DBA maintained and evolved the data model, outputting it regularly to their plotter and tacking to the wall.  The organization already made it a practice to Display Models Publicly long before they adopted RUP and AM. The team had five four-week iterations in the Construction phase, implementing between two and five use cases at a time.  The Transition phase was fairly standard, from an AM point of view the most important effort was a documentation clean up at the end.

As the project progressed the team ran into several issues:

1. Personnel. The Java/EJB developers took to the process pretty quickly, much of the object-oriented development practices were familiar to them already and they preferred an iterative and incremental approach to development.  The C++ developer had a “big design up front” (BDUF) mindset that he started to overcome mid-project after seeing that our emergent design approach actually worked in practice (see the Agile Design Modeling article a discussion of these two approaches to design).  The DBA had a similar mindset although he wanted to create a data model early in the project, which was clearly inappropriate because we wanted to let our object design drive our database design so we diverted him to focus on legacy analysis and the formalization of contract models with the relevant groups. The business analyst struggled with learning how to apply use cases effectively in combination with other requirements artifacts.  The tester was initially concerned that we weren’t producing detailed models that could be inspected and peer reviewed, but he recognized the closer collaboration among the team and the combination of significant developer testing using JUnit (www.junit.org) and an EJB testing tool and testing in the large activities such as system testing and user acceptance testing (UAT) was sufficient for our needs.

2. Management. The team had good senior management support throughout the effort as it was clear to them that the team was delivering.  The team’s senior manager focused on technical issues such as scheduling and estimating and the junior manager focused on softer issues such as ensuring the team had the resources they needed, including training, and that they weren’t being delayed by politics. 

3. Tools.  The team could have been more agile when it came to tool usage.  The practice of Use the Simplest Tools wasn’t fully accepted, whiteboards made sense to the team but index cards and essential user interface prototypes using Post It Notes and flip chart paper were too foreign for them.  Initial diagrams were drawn on the whiteboard and in the case of structural diagrams such as UML class diagrams and data diagrams they were then transcribed into the appropriate CASE tool, Together/J and Oracle Developer respectively.  The use case diagram was transcribed into Microsoft Visio, they needed to present it to senior management and my suggestion that we simply provide them with a digital picture of the whiteboard didn’t go over well.  The diagram also appeared in the final documentation so the effort was worthwhile.

4. Cultural.  The hardest thing to overcome was the serial/waterfall mindset of some of the people involved.  The C++ developer, who was learning EJB, and DBA both had a BDUF mindset and the tester wanted to see more formal documentation.  The data-driven mentality of the DBA was typical but eventually overcome when he saw that an object-driven approach actually worked in practice – this was clearly a struggle for him at first and I suspect that he was initially giving the team enough rope to hang themselves with, but slowly saw that there are in fact more effective ways to work than what he was familiar with.  The managers were open to iterative and incremental development, I suspect that not only did they purchase Kruchten’s RUP book (2000) they actually read it, and had communicated the advantages of this approach to the user representatives we were working with.  One of our user representatives had been involved with a “challenged project” in the past and definitely didn’t want to repeat that experience and as a result was incredibly open to following the RUP. 

5. Documentation.  In my opinion the team wrote too much documentation, but at the same time it increased their comfort level with the new approach so that wasn’t too bad.  They stuck to the traditional RUP artifacts.  Their requirements model was focused around the use case diagram and a collection of use cases, business rules, technical requirements, and glossary written in English prose.  The business analyst wrote and maintained this document, keeping it up to date as the requirements evolved.  The Software Architecture Document (SAD) was maintained as point form notes on an HTML page, whiteboard sketches, and arguably in the CASE tool models.  The notes, basically an overview of the team’s decisions, and the whiteboard diagrams were quite agile.  However, the team complained that there was nothing in between the high-level sketches and the detailed diagrams in their CASE tools – both Together/J and the data modeling tool are oriented towards detailed design and development, not high-level architecture.  In the end they discovered that they didn’t need the intermediate-level models, but this was a point of contention for awhile.  The data model appeared to be of value to the team and was considered a permanent artifact, although the diagrams in Together/J weren’t perceived to be as important – Together/J writes most information to Java class files, which were clearly considered critical to success.  The Test Model, the definition of the test cases, was maintained by the tester, and although I never looked at it from what I could tell the tester invested too much effort in its development.  To be fair, the tester was new to J2EE development and wanted to err on the side of caution.

Adopting AM on an UP Project

For an UP project team to adopt AM they will need to overcome the common misconceptions that developers have about the UP as well as several cultural barriers that are common within organizations that instantiate the UP. they need to begin by thinking outside of the current UML-centric nature of the UP.  To do this, you should:

1. Forget the term “use-case driven”.  Yes, it’s a wonderful marketing term but the reality is that use cases aren’t sufficient to drive much of anything.  Use cases are a good technique to document behavioral requirements but that’s only a small part of the functional requirements picture and an even smaller part of the total requirements picture – they aren’t very good at documenting business rules, user interface requirements, constraints, or non-functional requirements – which is why the UP includes something called a supplementary specification to contain all these other things.  Requirements drive things, use cases don’t.  Your modeling efforts will always remain hobbled if you don’t separate the UP’s software engineering wheat from its marketing-rhetoric chaff.

2. Recognize that there are more modeling artifacts than those described by the UML.  AM’s principle Multiple Models tells you that you have many modeling artifacts at your disposal – change cases, user stories, business rules, UML activity diagrams, UML class diagrams, data models, and external interface specifications – to name a few.  An interesting implication of this principle is that you have far more than just the diagrams of the UML at your disposal.  The good news is that the UP recognizes that a wide range of models is needed to explore the complexities of modern software, recent versions do in fact include data modeling and user interface design activities that are currently outside the scope of the UML, the bad news is that it many people erroneously perceive that the UP is simply a process for using the UML. 

3. Recognize that the UP is not inherently documentation centric.  The UP is actually very clear that you should only develop the artifacts that you actually need, however, this is good message is something that often gets missed by many software professionals so it is something that is worth repeating here.  You should question every single model that the UP suggests creating because it purposely describes a wide variety of artifacts, many of which your project simply doesn’t need.  The UP includes three major sets of modeling-oriented artifacts – the business modeling set, the requirements set, and the analysis & design set – each of which in turn is composed of several detailed artifacts.  For example the business modeling set includes a business use-case model, business rules, a business architecture document, and a business supplementary specification.  Do you actually need all of these things?  Likely not.  If you do need them, do you need them as formal documentation?  Likely not.  Communicate AM’s Travel Light and Model With a Purpose principles to your project stakeholders, as well as the practices Update Only When It Hurts and Discard Temporary Models.

4. Build a common process vision between developers and project stakeholders.  Managers often tend towards a prescriptive software process, something that appears well defined and comprehensive such as the UP, one with a perceived focus on control.  Developers, on the other hand, gravitate towards agile techniques such as XP and AM due to their perceived focus on what is important to developers: building software.  Because management holds the purse strings many developers find themselves in a situation where their managers have chosen to adopt the UP and are now being required to follow it.   Luckily the UP is flexible enough that it can be tailored to be reasonably agile, but to do so developers and project stakeholders will need to come to an agreement as to the extent of the tailoring.

5. Actively promote iterative and incremental development.  Agile Modeling’s practices of Model in Small Increments, Iterate to Another Artifact and Create Several Models in Parallel can be tough ones for experienced modelers to adopt – and chances are that your experienced modelers are already chaffing at the UP’s concepts of iterations, let alone an even greater emphasis on iterative and incremental modeling.  Traditional modeling techniques often promoted a single artifact approach, such as use-case modeling or user-interface prototyping sessions, and often even modelers that focused, for example data modelers.  Also, they often promoted a BDUF approach where you modeled everything in detail before you started coding.  These concepts were great in theory, focusing on a single artifact at a time should have allowed the modelers to get it right quickly, but unfortunately practice shows this not to be the case.  A good way to ease into these practices is instead of use-case modeling sessions run requirements modeling sessions where you work on use cases, Class Responsibility Collaborator (CRC) cards, business rules, and user interface prototypes simultaneously. Similarly, hold analysis sessions where you are use case modeling, sequence diagramming, user interface prototyping, and class modeling may make sense, and design sessions where you are class modeling, state modeling, data modeling, component modeling, user interface prototyping, and hopefully even developing business code.  Once you are comfortable with these practices the next step is to then merge your modeling efforts in with your implementation efforts, applying multiple artifacts including all your potential models, source code, and test cases as needed – truly iterative development.  While you are doing this, keep their focus on the requirements that you are implementing in the current iteration, resulting in an incremental delivery of functionality each iteration.

6. Actively promote simplicity.  Simplicity is a fundamental value of AM, one that motivates several critical principles that can dramatically improve the effectiveness of your modeling efforts.  Many experienced modelers want to specify everything they possibly can, for example, not only do they wish to model the overall structure of their software in UML class diagrams they also want to specify the scaffolding code needed to implement that structure.  This is a lot of effort which provides very little actual value.  A better approach is to create a class diagram that is just barely good enough for your purpose, to depict the likely structure of your classes, and then start coding from there.  Agile modelers assume that the programmers, often themselves, can figure out the details at the time and instead will focus on issues that may not be so obvious.  This approach implies less work for the modeler and less modeling noise for the actual programmer to wade through.  When an agile modeler is creating a class diagram they realize that they don’t need to model all of the classes required to build the software, instead they focus on getting the core classes right and assume that the programmers are competent enough to handle the rest[1].  By keeping your models simple you are likely to work faster while at the same time create something that is actually of value to your programmers – models that focus on critical issues that are devoid of fluff.

7. Staff projects with generalizing specialists.  Many organizations have separate positions for modelers, motivating their staff to focus on specialties, a practice that in my experience reduces your ability to be agile.  Although the UP is very clear that individual developers can and should take multiple roles on a project my experience is that this is advice that falls on deaf ears within many organizations.  Instead what happens is that organizations adopting the UP tend to introduce positions along the lines of UP’s modeling roles – for example Requirements Specifier, System Analyst, User-Interface Designer, Database Designer – and therefore slots people into individual roles, going against the advice of both AM and the UP.  If your only job on a software project is to produce models then there is a tendency for you to over model things, first because you naturally want to do a good job and second because if it’s your job to model then that means it’s likely someone else’s job not to model (e.g. their job is to program).  Because there is a hand-off there is now greater motivation to add greater detail into your model, details that likely wouldn’t be needed if the people writing the code also developed the models that helped them to identify what needed to be coded in the first place. 

8. Live AM’s principle of Open and Honest Communication.  I’ve run into several situations where a development team was reluctant to follow the Display Models Publicly practice, one way to promote open and honest communication with people external to your team, often because they were afraid of what another political faction within the organization would do with the information.  However, when they finally worked up the courage to display their models publicly they quickly discovered that the politicos they were so afraid of couldn’t find much to criticize, and when they did this was great input that the developers quickly acted on and benefited from. 

How Does This Work?

It is clearly possible to tailor the Unified Process (UP) with the practices of Agile Modeling (AM). To succeed, your organization’s culture must be receptive to both the UP and to AM, and therein lies the rub – the goal of organizations that adopt the UP is often to instantiate it as a fairly rigid and prescriptive process whereas organizations that adopt AM typically want to work in a more fluid manner (in fact, to be truly effective AM requires such an environment).  Luckily the UP is flexible enough so that it can be instantiated to be reasonably agile.  When this is the case, when you have instantiated a light-weight version of the UP, then the UP and AM fit together well.  Both the UP and AM are based on the idea that the majority of software is best delivered iteratively and incrementally.   Because the UP explicitly includes modeling disciplines it is very easy to identify where AM practices should be used to enhance your own tailoring of the UP.  This works when your project team and your project stakeholders choose to make it work.

Recommended Resources

		The Object Primer 3rd Edition: Agile Model Driven Development with UML 2 is an important reference book for agile modelers, describing how to develop 35 types of agile models including all 13 UML 2 diagrams.  Furthermore, this book describes the techniques of the Full Lifecycle Object Oriented Testing (FLOOT) methodology to give you the fundamental testing skills which you require to succeed at agile software development.  The book also shows how to move from your agile models to source code (Java examples are provided) as well as how to succeed at implementation techniques such as refactoring and test-driven development (TDD).  The Object Primer also includes a chapter overviewing the critical database development techniques (database refactoring, object/relational mapping, legacy analysis, and database access coding) from my award-winning Agile Database Techniques book.
		Agile Modeling: Effective Practices for Extreme Programming and the Unified Process is the seminal book describing how agile software developers approach modeling and documentation.  It describes principles and practices which you can tailor into your existing software process, such as XP, the Rational Unified Process (RUP), or the Agile Unified Process (AUP), to streamline your modeling and documentation efforts.  Modeling and documentation are important aspects of any software project, including agile projects, and this book describes in detail how to elicit requirements, architect, and then design your system in an agile manner.
		The Elements of UML 2.0 Style describes a collection of standards, conventions, and guidelines for creating effective UML diagrams. They are based on sound, proven software engineering principles that lead to diagrams that are easier to understand and work with.  These conventions exist as a collection of simple, concise guidelines that if applied consistently, represent an important first step in increasing your productivity as a modeler.  This book is oriented towards intermediate to advanced UML modelers, although there are numerous examples throughout the book it would not be a good way to learn the UML (instead, consider The Object Primer).  The book is a brief 188 pages long and is conveniently pocket-sized so it's easy to carry around.
		The Enterprise Unified Process: Extending the Rational Unified Process describes the EUP, an extension to the RUP.  Whereas the RUP defines a software development lifecycle, the EUP extends it to cover the entire information technology (IT) lifecycle. The extensions include two new phases, Production and Retirement, and several new disciplines: Operations and Support and the seven enterprise disciplines (Enterprise Business Modeling, Portfolio Management, Enterprise Architecture, Strategic Reuse, People Management, Enterprise Administration, and Software Process Improvement).

Footnotes

[1] If this isn’t the case then you have a serious human resources issue that you need to deal with appropriately.  Excessively complex models aren’t going to help address this situation in practice, and their creation will help management avoid making the hard decisions regarding the quality of their staff.

Let Me Help

I actively work with clients around the world to improve their information technology (IT) practices as both a mentor/coach and trainer.  A full description of what I do, and how to contact me, can be found here. 

Copyright © 2001-2009 Scott W. Ambler This site owned by Ambysoft Inc. Agile Data (AD)  |  Agile Unified Process (AUP)  |  Enterprise Unified Process (EUP)  |  My Writings   |  IT Surveys