Generalists or Specialists?

A critical issue that you need to address when recruiting members to your team is the ratio of generalists to specialists that you wish to include.  To answer this question you need to consider the nature of modern software development.  The lifecycle of the Enterprise Unified Process (EUP)  depicted in Figure 1.  The process workflows listed along the left-hand side hint at the potential complexity of software development – you may need to business model, gather requirements, analyze and design your system, and so on – and this is clearly just the tip of the iceberg.  The phases, Inception through to Production, listed across of the diagram, are indicative of a changing focus throughout your project that requires different skills at various times.  The point to be made is that software development is clearly very complex, an effort that requires significant skill and experience.  It is important to understand that this complexity is inherent in software development in general, not just in the EUP – project teams taking an XP, Agile Unified Process (AUP), DSDM, or Scrum approach also need to deal with these complexities.  Although their lifecycles may not depict it as explicitly as the EUP’s project teams following these methods still perform configuration management activities, still perform management activities, and so on (albeit in different manners).

 

 

Figure 1. The lifecycle for the Enterprise Unified Process (EUP).

 

Many organization’s first reaction is to build a team of specialists.  The basic idea is that specialists are very proficient at a specific task and are therefore more efficient, so to become efficient at software development all you need to do is put together a team of specialists who work on their part of the project and hand-off their work to other specialists who evolve it from there.  This is effectively the “assembly-line” school of thought, which works very well if you are mass producing cars but in my experience does not work very well when you are attempting to hand-craft software.  Furthermore, this approach is geared towards larger teams – if there are X distinct tasks required to develop software then you need at least X specialists following this approach.  How big is X?  Twenty?  Fifty?  One hundred?  Depends on how finely you define the specialties, doesn’t it?  If you are of the single artifact developer mindset then just to handle modeling alone you may need over twenty specialists, there are that many artifacts listed in the modeling artifacts article, and if you are of the single role developer mindset then you have eleven roles on an EUP project just to cover the workflows.  Specialists often have difficulties working with others, either they lack the humility to recognize that people with different specialties than their own have something of value to add or they are so narrowly focused that they don’t realize that what they are doing is causing someone else later on to do significant rework (or perhaps the work that they are doing is simply being ignored).  Another problem with specialists is their skills may not be very good at all, even in their own specialty.  The high rate of technical change within the IT industry provides an environment where developers can work with a new technology for several months, become reasonably familiar with it, and claim to be an expert because few others are available with the same level of experience.  Clearly there is a problem with building a team from people who are only specialists. 

So how about building a team of only of generalists?  Everyone would have a fairly good understanding of how to develop software, but unfortunately wouldn’t have the detailed knowledge required to get the job done.  Your project will need people intimately familiar with the techniques and technology that you are using.  If you’re working with Enterprise JavaBeans (EJB) then you want developers with expertise in Java programming as well as in EJB development.  A team working with Oracle on the backend will also want someone with Oracle database administration experience, and a team developing software for a brokerage will want people that understand the nuances of stock and bond trading. 

My experience is that neither extreme works well, that instead what you want is something in the middle.  One approach is to build a team with some people who are generalists and some who are specialists, the generalists provide the glue within the team and focus on the bigger picture whereas the specialists focus on the detailed complexities of your project.  This works well because the strengths of generalists balance the weaknesses of specialists and vice versa, and it is often quite useful for a generalist to pair with a specialist because of this balance.  A better approach would be to build a team comprised of people who are generalists with one or two specialties -- generalizing specialists.  For example, I would claim that I am a generalist, I have a pretty good handle on how it all fits together, yet I have specialties in business application software modeling, object persistence, and Java programming.  One of my current co-workers is a generalist with specialties in modeling, EJB development, and testing whereas another is a generalist with specialties in telecommunications networking and Java programming. The advantage of building teams from generalists that have one or more specialties is that they quickly find common ground with their co-workers, they’re all generalists after all, and have the necessary background to appreciate the importance of other specialties.  The main disadvantage is that these people are often very senior, it can easily take ten to twenty years to gain sufficient experience to become a generalist, and thus are difficult to obtain.  You’re very lucky if a portion of your team is formed by people such as this. 

It's important to recognize that novices often specialize at first.  People who are new to development are typically overwhelmed by the vast range of knowledge that they need to gain, that’s only natural.  Most people will start out by focusing on one or two aspects of development, perhaps programming in Java or gathering user requirements, and then using that experience as a base branch out from there.  As time goes on they’ll slowly build up their skillset with experience, perhaps gaining a specialty or two in the process as well as a better understanding of how it all fits together.

It's also important to understand that many developers specialize to their detriment. Because of this inherent complexity of software development a common trap is for developers to fall into the Single Artifact Developer anti-pattern where someone defines himself as a person that works on one type of artifact (e.g. code, a use case model, or a data model) or into the Single Role Developer anti-pattern where someone defines herself as a person that performs one kind of task (e.g. modeling, testing, or coding).  In other words the person is specializing in a specific role, a tendency that is often encouraged within large organizations following prescriptive processes (if they’re following any process at all).  The problem is that developers who fall into this trap are too narrowly focused to be productive on an agile software development project, although if they are willing to expand their horizons this issue can be swiftly overcome.