HelpMatch: Redefining the Humanity of Humanity!Join the cast of characters who are seizing their defining moment; pitch in and help create something that will redefine the humanity of humanity! Sounds big, overblown even, but such is the potential of HelpMatch! If you just think of a “MySpace” for help, for connecting people who are touched by an event or a story, a trip, something that makes them want to reach directly to a person in need and help them, then you are onto the potential of the idea. Big huh?

I have learned what I know about good architecture from working with good architects, many of whom have set the bar for excellence in architecture documentation. That said, I am too often caused to lament that the only thing harder than getting engineers to read the architecture documentation is getting architects to write it!. So why bother? Who cares? It is only going to get out-of-date. It takes work, and isn’t that time better spent coming up with better solutions to the challenges we face? Or better still, writing code, which is the ultimate in system documentation–right???

An architectural decision that isn’t written down has a lifespan of, let’s see, what is the memory span of a goldfish? Actually, it turns out that a goldfish memory span is not just a few seconds as previously thought. And we’re even smarter than goldfish. Can’t we just rely on our individual and group memory? Well, we have all been in the situation where a decision we thought we made yesterday, is still under debate today, and we have to persuade and coax, inform and influence, brow-beat and defend all over again.

Writing the decision down doesn’t get us entirely away from this perpetual churn, but it does help. We get sign-off on the decision. Once that is achieved, we can demand that only counter-arguments that have a strong link to architectural requirements can be brought up in contention with the decision.

Still, the half-life of an architectural decision depends very much on the authority vested in the architects, and how this authority is formally and informally reinforced in the organization. And it depends on what the architects do to communicate the decision, and what support and follow-through they apply.

So, if we cowtail to the push-back against architecture documentation from extreme agilists, and our own disinclination to write down architecture decisions and thinking that went into them, then we are sending the message that the “architecture” is not a set of decisions but just a fluid initial starting point that we expect everyone to remold and reshape actively and without constraint–or restraint.

In some situations this may be just fine. If we are working on a novel system, without precedent in our experience and in the collective experience of our industry (so we can’t hire in the experience we need), then it would be foolhardy to create an architecture specification early on and expect to live by it through the life of the system.

Architecture is intended to constrain–and enable, but the price is that we must constrain; that’s what decisions do. Once made, they have eliminated some alternatives we might otherwise have picked. Yes, like stop lights, they enable and constrain. So, is it reasonable to constrain (and enable) ourselves early on in the project? While we are always pushing boundaries beyond what we already know, we are generally also working with a good deal of experience to leverage. To the extent that we can create an architecture that we can validate and build confidence that it will serve us through the first release, and prepare us well for subsequent releases, we should do due diligence when it comes to documenting our architecture.

But what is architecture documentation? In good part, the architecture documentation consists of the very models we use to think through and make the architecture decisions. In short, much of the work is already done, assuming that we use models to visualize and evaluate architectural approaches. The diligence part has to do with:

1. making sure we write down the thinking behind the models

2. keeping the models (and supporting explanations) up-to-date

Architecture documentation explains and justifies the decisions that embody the architecture. So we need to articulate the reasoning, tracing the decisions back to the requirements that drove them, keeping track of alternatives we weighed but ruled out and why (so we don’t have to make the same arguments again and again), and writing down assumptions we made (so if these assumptions are invalidated, we know we have to revisit the decision).

When we do this, we also help educate the engineering community, sharing the experiences that shaped our decisions. By documenting our reasoning like this, we make our knowledge explicit and shareable. Further, it makes us more careful, because we leave a record that can be assessed and debated.

One of the things we have to do as architects is figure out where to push back against the status quo, to lead out of the rut we are in to a better way of doing things; a better way that enhances our community’s quality of life. And we have to figure out which battles just aren’t worth it, because it takes energy and passion to lead these changes, and some changes are more important than others.

For the things that we see rising above this cut-line, we need to do what it takes to be effective. We must not allow ourselves to be lulled by the cries of “it’s all going to change anyway” to escape the effort it takes to write good documentation for architecturally-significant decisions. If we want these decisions to impact the behavior of people implementing them, we need to do our part in communicating them.

The architecture decisions must be recorded, so we have a ready reference that doesn’t depend on us being always in the room, ready to explain and defend each decision. The decisions must be communicated and understood. It is worth it, if the architecture decisions are worth it. So the decisions must be strategic and minimalist, and relevant. As soon as they are not, we must be on hand to adapt the architecture decision set.

Once written down, to be sure not everyone who should read the various architecture documents, will read them. But those that do will have a much better understanding of the architecture, and the rationale for the decisions it encompasses. Good models and well-written explanations get right into the head of the reader in a personal and effective way. The reader can engage, backtrack, hold an internal dialog with the material until it is well understood, or at least clear where the questions are. Each reader will be better positioned to explain to their  peers and reports what the architecture means, in the narrow direct sense and in the broader sense of its intent. This very effectively expands your capacity to champion and explain architecture decisions and catch misunderstandings and misapplication of the architecture.

All this presumes that you and your team can come up with an architecture. It does not presume that you make every architectural decision in advance of all coding. Not at all! But when you have a complex organizational setting (large number of developers, distributed teams, etc.), then you need to do more of the architecture work upfront, and document AS YOU GO, not afterwards! There never is an “afterwards” and even if there was, you’ll have forgotten much of the rationale, if not many of the important decisions. Besides, though we need architecture documentation to help us with system evolution, we need architecture to create a system that addresses our architecturally significant requirements in the first place.

Early on, figure out what the architecturally significant uncertainties and risks are, and figure out what you must do to resolve these risks. Leaving them until “you must deal with them” is risky business. Then work quickly, with a focus on architectural priorities, to get a minimalist set of architecture decisions explored, validated (through models, reviews and assessments, simulations, mock-ups, and early, focused development of critical pieces of the code) and documented!

The bottom line: No architecture documentation –> no architecture; no architecture and we rely on organic people-intensive communication processes that, on average, don’t scale too well. No architecture + big project –> project wipe-out.

Other Perspectives on Architecture Documentation

For Architecture Documentation:

• Documenting your Software Architecture, by Jim Alateras on March 15, 2006

• Why Documentation Matters, by Frank Kelly, posted on June 15, 2006

On-the-fence about Architecture Documentation:

• Agile Architecture Modeling, by Scott Ambler, last updated on April 29, 2006

Creating Architecture Documentation:

• Software Architecture Documentation, by Ruth Malan and Dana Bredemeyer, March 2003

• Software Architecture Action Guide Book (draft), by Ruth Malan, and Dana Bredemeyer, 2006

• Less is More with Minimalist Architecture, by Ruth Malan and Dana Bredemeyer, published in IEEE’s IT Professional, September/October 2002.

• A Template for Documenting Software Architectures, by Mike Ogush, Derek Coleman, and Dorothea Beringer, March 2000

• Architectural Blueprints - The 4+1 View Model of Software Architecture, by Philippe Kruchten 

• IEEE Std 1471-2000 IEEE Recommended Practice for Architectural Description of Software-Intensive Systems

• Software Architecture Documentation in Practice: Documenting Architectural Layers, by Felix Bachmann, Len Bass, J. Carriere, P. Clements, D. Garlan, J. Ivers, R. Nord, and R. Little, 2000

• Architecture Documentation — Choosing the Views, by Paul Clements, Jan 31, 2003.

See also Software Architecture Books and Architecture Documentation (links) in my Trace in the Sand Architecture Journal

[8/25/06: Had to republish this post so the sidebars would show up.]

[9/16/06: Arnon Rotem-Gal-Oz is blogging on the Software Architecture Document – on 9/12/06. See also Deliverables sections of our Software Architecture Action Guide book chapters on Meta, Conceptual and (soon) Logical Architecture.]

I love my Trek bicycle, but with Shimano gears and brakes, and Bontrager frame, wheels, tires, and pedals, the ineffable Trek quality cannot be pinned down to any Trek-branded component. The success factor here: modularity with clear interfaces, and clear design requirements pushed down to the modules so that excellence at the level of the overall system is achieved through the composition of excellent pieces.

Bicycles have been around a long time, and there is a dominant design. That allows innovation to be focused on the components and subsystems, with less frequent revolution in the overall decomposition. Together these 3rd party pieces fitting into a standard decomposition nonetheless yield a system that is unique and differentiated. Modular components driven to excellence in of themselves, plus a clear sense of the differentiating system qualities, yields a distinctive product. Flawless. So well-designed I can say wthout reservation, I love this product.

It is the architect’s responsibility to create the architecture (with collaborative input from component designers): to make decisions about the system identity and integrity, and decisions about components and their scope and responsibilities, and address cross-cutting concerns and system properties. The architecture provides the context to push excellence, innovation and quality to the level of the components.

In the software world, in the pressure of the moment we allow ourselves to accommodate the architecture. Every compromise to the modularity of the system increases the future cost of change. But we live in the present. We have to get out of this singular mindset, and part of the way out is to have an architect with the clear and well-known charter to preserve the structure of the system—to defend modularity and ensure that the modular pieces do indeed work together to deliver the functions and properties required.

Feature set teams or storyline teams (or whatever else you call them in your group) tend to work across the layers in a system. This work partitioning helps drive out features that users can respond to and give feedback on, so they can be improved. Great! But it also makes it important to empower an architect to defend and preserve the architecture, and yes, evolve the architecture when, on balance, that is the best route.  

Objects, then components, now services, are all hyped because they promise to address the build-from-parts need in our software world, at ever greater levels of granularity. But solving our problem isn’t simply a matter of picking a programming paradigm that supports (larger-grained) parts. We have to become good at designing parts—parts that really fit our system; really fit our need.

I acknowledge that I am fundamentally, completely, biased here, but I can see no other course than through getting better at architecture, and better at protecting and preserving our architecture, and better at investing in evolving and regenerating our architecture. And better at investing in what it takes to completely revolutionize our architecture, even though that means we have to do so with a whole new set of people, which is counter-intuitive (see Rechtin’s book Why Eagles Can’t Swim, 2000).

To set the context for subsequent posts, I thought we’d start with the topic of “what is software architecture?” Bass, Clements and Kazman’s definition of software architecture  (in essence, the high level structure of the system, described in terms of components and their externally visible properties and the relationships among them) has been very influential. While this definition has resonated with many people, the continuing discussion indicates some remaining uneasiness with definitions proposed so far.  We prefer to use the Bass et. al. definition and focus on the central concerns that software architecture addresses. (You might also like to take a look at Chapter 1: Software Architecture: central concerns and key decisions.) 

Complexity and Cost of Change

We need architecture to manage complexity and cost of change. Grady Booch puts it like this: “all architecture is design, but not all design is architecture. Architecture represents the significant design decisions that shape a system, where significant is measured by cost of change.” Cost of change goes up as scope of impact increases, so this definition covers decisions relating to system decomposition as well as those addressing cross-cutting concerns. Cost of change is also high if a significant chunk of the system has to be revised or rewritten, so this speaks to challenging pieces of the system.

Big Rocks First

We use the “big rocks” metaphor (see slide 6 and 7): If we start with the big rocks, then add the pebbles, and last the sand, we fit them all into our jar. And yes, we could even add water. The point is not that we can always ask developers to do more! The point is that if we start with the sand, add the pebbles, and then try to add the big rocks, we cannot fit them all in the jar. To fit them all in, we must start with the big rocks. Architecture is about getting the big rocks in place first.

But what are the “big rocks”? The architectural elements—the components and their relationships, yes. And architectural mechanisms addressing cross-cutting concerns or systemic properties, yes. Big rocks bear a high cost of change, yes. Is there more?

Architecture Implements Strategy

The architect of early generations of HP OpenView said “Architecture is the translation of business strategy into technical strategy.” This definition focuses on the strategic nature of architecture. What is significant, in this view, is driven by what is strategic, and what is strategic determines how we will compete. How we will differentiate dictates the big things we must get right, what hard problems we will tackle, where we will innovate, and where we must be ahead of competition. And it allows us to accept good-enough along those dimensions where we are not trying to create competitive advantage. Of course, even “good enough” may be challenging, especially when taken in conjunction with where we aim to differentiate.

The business or product strategy needs to establish what differentiating value we will deliver to our customers, our shareholders, our partners in the value network and our people. The architect needs to assess what capabilities will deliver this value. Architecture is about designing system capabilities that deliver the value propositions and reinforce the identity of the system (application, product, product-line, etc.) in alignment with the business strategy.

Architecturally significant decisions are those that must be made by the person or team who has influence and perspective across the system in order to deliver on the strategic objectives of the system.

Architecture Balances Differentiation, Complexity and Cost

The architect needs to balance the need to differentiate, with the lifecycle cost of the features and quality attributes we pursue. What we need to do, just to be in the game, constrains what we can accomplish in order to distinguish our products or services and business. For example, in many systems some level of security, scalability, and disaster recovery are threshold attributes not our avenues for differentiation. So we must develop mechanisms (authentication, encryption and firewalls; load balancing; failover, etc.) to address these challenges. Just delivering the base level of features and qualities is challenging, given the threshold set by intense competition in most markets.

Others will have a say in what our opportunities are to differentiate (e.g., marketing). And others will have a say in identifying the challenges we face to build and field systems in our domain (e.g., development). The architect needs to play a role in balancing what we would like to do, with what we are able to do given our resources and capabilities.

Architecture Expands Our Capability

Moreover, the architecture needs to play a role in increasing what we are able to do, and increasing the value of what we attempt to do, by allowing focus. Focusing attention, enabling specialization and separation, understanding where outsourcing or licensing can be leveraged, reducing complexity and scope where it is not essential to our value proposition. Knowing what to focus on and knowing what we can ignore—both are key to success.

Architecturally Significant Decisions

So architecture helps us manage complexity and cost of change, and deliver differentiating value in alignment with our business and product strategy. Architecturally significant decisions are those that the architect (or architecture team) needs to make in order to address these concerns (strategy, complexity, and cost of change). They generally include the high-level decomposition of the system and address cross-cutting concerns. What we do is driven by our business strategy, how we do it is driven by cost of change.

Furthering the State of our Understanding

Whatever else you might add, I hope that in the commentary on this post, you will share as concretely as you are comfortable with given the public forum (and your level of identity disclosure),

• what are the major concerns you address through architecture, and
• what kinds of decisions are architecturally significant for your system?

To set context for your observations, it would be very helpful if you would provide some background, such as

• what kind of system you are working on (embedded system, e-commerce system, whatever),
• some indication of complexity (like size of the development team), and
• what your role is.