Agile: Diagram-Driven Development

Agile methodologies have been helping software developers produce higher quality software in less time for almost ten years now.

However, there are always ways to improve. After observing teams function in some form of Agile for the last decade, I am left with the impression that there was a baby in the bathwater who may have been unwittingly tossed out.

Agreed - software documentation is not something that customers typically ask for, hence it is of lower value in Agile methodologies. It is not entirely without regard - some Agile processes like FDD and XP, for example, include short Design phases in their process, which one can interpret as generating some form of Documentation.

In practice though, teams trying to become Agile tend to cling to the concepts in the Agile Manifesto, which extoll production over documentation.

The Oversight

I believe there is room for better balance here.  The original intention of "documentation for developers" - i.e. Diagrams - was to save time and money. It was a good idea, but it was hard to get right.  Hence the entire practice was largely thrown out with Agile.

The Proof

People tend to do what is necessary to get their job done.  Normally that includes installing and configuring an IDE, reading an Ant or Maven build script, and digesting source code.

In my experience, unless a project is very simple, as I am trying to digest source code, I tend to jot down class diagrams, activity diagrams, and deployment diagrams.  I do this because with larger systems I cannot remember all of the business rules, the entity relationships, and where all the servers are.

A developer must understand the codebase to be productive. The more clarity, the higher quality the solution tends to be.

This requires a significant investment by each team member.  Requiring each developer to take on this cost multiplies overall cost across the project - developers either bear the cost directly by doing the research, or indirectly via longer execution time for each task.  A development task cannot be performed without some knowledge of the system.

The Ideal Solution

Of course we could say that ideally, there would be diagrams documenting each aspect of the system and they would be magically updated as things change. It is easy to see that this is not Reality.

Challenges include:

• comprehensive production of diagrams - do we have diags for everything?
• maintenance of diagrams - is anyone updating these as things change?
• finding a tool that does what we need and is easy to use

Mitigation

An effective way to relieve this tension is by clarifying and simplifying the documentation needs.  I suggest reducing the set of documents to:

• Activity diagrams - Essential - for complex business rules
• Entity diagrams - Helpful - both class and database
• Deployment diagrams - Helpful - for servers in use on the project

Entity diagrams are the easiest to manage, as they should always be generated.   This requires a tool that will reverse-engineer classes.  Open source solutions tend to fall short of this, however.  Current options include:

• the diver project 
• Object Aid UML Explorer, an Eclipse plugin available at http://www.objectaid.com/update
• UMLet - an Eclipse plugin available at Eclipse Marketplace  (although it seems un-downloadable at the moment?)

Database tools tend to have diagramming capability at some level, but investigation is required to find the right solution for the primary database in use during development.

Capturing business rules, however, is a different story.  Its challenges are unique:

• worthwhile sets of rules tend to be complex, meaning they are difficult to actually diagram
• diagramming tools often require manual layout, which imposes maintenance costs for complex diagrams

The team needs to decide which business rules require diagramming. The guiding principal should be:

• waste no time diagramming simple rules
• waste no time debating complex rules - let the diagram speak 

These diagrams must be required to be maintained by any contributor working on the code.  I would suggest that Test-Driven Development be augmented by Diagram-Driven Development in these cases.

Definition:  Diagram-Driven Development

Test-Driven software development which is directed by business rules captured in diagrams.  Can be applied to any significant body of business rules which will be implemented in code.

1. developers and stakeholders agree on the set of rules, or modifications to existing rules
2. ideally: rules are diagrammed in real-time during user story discussions
3. rules are coded into test cases
4. rules are coded into source code

If we turn the production of Diagrams upside down, as leading the process rather than following it, we have a process which ensures that important diagrams are maintained, just as our test cases are.

And in return, we have useful documentation which can be used to reassure The Customer that we know what we built.