Archive for Architecture

Att-e-en-tion! To System Design

What we are paying attention to shapes what we perceive and pay attention to. And paying attention, requires attention. The system, thought of, observed and measured, reasoned about, designed as, a system, needs intentional attention. Attention that competes for bandwidth with the order of the day — delivery of working code (with tests, please). We need to discover on purpose (embracing happy accident along the way), design on purpose (embracing iteration, discovery, accident), and… redesign on purpose (throughout the lifecycle). Using the cheapest design medium that will expose the uncertainties and stress the design (fragment, mechanism, etc.) under consideration sufficiently for the (extraordinary!) moment we’re at. Which may be code. But don’t assume so, when thought experiments, models, sketch-prototypes, cardboard mockups, role-plays, etc. will serve the moment. Yes, yes, “code wins arguments,” but you have to know how to set up your arguments.

The system takes a different kind of attention. Thinking at the interaction between capabilities (and value the system delivers) and system properties (impacting users individually, like usability and performance, and in aggregate, like scalability and availability; and impacting operations and the business, …) and the structures and mechanisms that deliver them, takes a vantage point that is able to move fluidly and adeptly at different scales or scopes (zoom levels, if you like) and across views (frames, cross sections, selective screening, …). But it also takes time, which takes will — the organizational will, and the personal will, to do it. When there is code to be written, and pressure to deliver. Increasing. If we build ourselves into a blind-alley of “technical debt” …which I have come to think of as “decision debt” — where we put off making a decision and the non-decision decision (do nothing, or keeping doing the old thing) or a reactive quick-fix decision gets coupled into the system. Some of these will be of little consequence. But there will be those where the “payback time” comes with stiff interest — and penalties! (Those are the ones an architect should have sniffed out as architecturally significant — and if he didn’t, he will next time! Experience factors. Puns allowed. We’re among friends.)

But wait. We can refactor. Continuously refactor. That’d do it. Well, yes. And no. Imperious Red Queen voice: “I need an architect here… Architect?” “It depends.” What did the Red Queen tell Alice? “it takes all the running you can do, to keep in the same place. If you want to get somewhere else, you must run at least twice as fast as that!” Say what?

Someone needs to be thinking strategically about the system. Not alone; working with the team, but having responsibility for the system so that attention is drawn to the system, attending to structural integrity in the small, sure, but in the large, too. Instrumenting the system — making code health and operational health visible. Anticipating the market and problems in the code, keeping tabs on how the system is doing relative to the design envelope for its various system properties. Finding the weak spots in the design. And doing “model storming” “out loud” “in pairs” or small groups. And all that agile stuff. Just in the cheapest medium for the moment, because we need to explore options quick and dirty. Hack them — with sketches/models, with mockups, with just-enough prototypes. Enough early. And enough along the way so that we course correct when we need to. So that we anticipate enough. Put in infrastructure in good time. Avoid getting coupled to assumptions that cement the project into expectations that are honed for too narrow a customer need/group. And suchness.

Comments

Conceptual Architecture: How

How: Some Comments on Creating and Evolving the Conceptual Architecture

During early system conceptualization, we start to envision the form or shape of the system, its boundaries and interactions, its primary elements and their interactions. The sketchy shapes of these elements, expressed mainly in terms of their responsibilities, analogies, and drawing on patterns and experience, take a rough form, allowing us to investigate the system concept. Then, as the system concept (identity, value propositions and capabilities) takes shape, so too does the Conceptual Architecture.

In designing complex systems, separation of concerns is a key strategy for gaining intellectual traction and enabling ourselves to reason about important design decisions while considering cross-cutting concerns that are part and parcel of the decomposition-composition dual of systems design and development. The concerns focused on at this point are related to giving shape to the system paying attention to user-facing capabilities and developer-facing properties of the system. We seek to bring intentionality and experience to design to achieve more the system properties we want, recognizing that ultimately system properties are emergent and we will need to sense and respond and adapt as we learn more from the system in context.

The architectural elements of software systems (that is, elements significant enough to the system to draw out and deal with in architectural design) are constructs of inventive human thought. We mold and shape them to accomplish a purpose, subject to constraints and forces. We leverage analogies to draw experience and knowledge from other domains, including mechanisms of nature (e.g., swarms), man-made (e.g., mechanical mechanisms like hubs), and social constructs (e.g., social and economic mechanisms like brokers).

The Conceptual Architecture Diagram is a lightweight and yet powerful tool for sketch-prototyping the system structure, and for rendering the structure of an existing system, to explore adaptations to it. Thus it allows us to explore the system from its earliest conception through evolution. As we experiment on paper, it is cheap to postulate various alternatives and explore their ramifications.
Early Conceptual Architecture exploration (depicted figuratively)
Later Conceptual Architecture elaboration and documentation

As system capabilities are defined, the associated responsibilities are allocated to “chunks” or components of the system, and these responsibilities are an important thinking tool for architectural designers who need to make software abstractions cognitively malleable and communicable. With something so simple as elements each with their list of responsibilities and relationships, we can experiment with different factorings and hence different abstractions or elements each with a different basis for cohesion of responsibilities. Further, it becomes possible to assess:

  • clarity of responsibilities
  • whether responsibilities are missing
  • components for cohesion of responsibilities
  • the system shape for balanced distribution of responsibilities

Architectural components may play different roles in the system, so their responsibilities are related within role-scopes. That is to say, for example, a component may play a minor role in system health monitoring and an active role with respect to some other system capability (e.g., providing shopping cart functionality). 

In highly dynamic systems, as software intensive systems are, structure and dynamics are designed together — if we focus on structure we impact dynamics, and vice versa. When we focus on dynamics — on behavior and “how it works” and properties that emerge from the structures as they perform their function and interact — our design decisions have to be expressed in terms of the structure for the structure houses, if you like, the implied responsibilities and interfaces.

The conceptualization of a product isn’t limited to the conceptualization of its use, but the product in use. We’re designing what the product is capable of in conjunction with what the product is and how it is built. A car isn’t a faster carriage, and although early cars had aspects of a carriage, the engine was quite different than a horse, and this translated to mechanisms to make the wheels move (from passive to active). In other words, when we conceptualize a new kind of product we iterate across designing capabilities and the mechanisms and structures that deliver those capabilities, and the product changes the ecosystem (a gasoline powered car raises the need for places to replenish fuel) so to make the product successful we also (impact the) redesign (of) key aspects of the ecosystem or larger system of systems into which our system fits, interacts and in important ways shapes and is shaped by.

“In most people’s vocabularies, design means veneer. It’s interior decorating. It’s the fabric of the curtains and the sofa. But to me, nothing could be further from the meaning of design. Design is the fundamental soul of a man-made creation that ends up expressing itself in successive outer layers of the product or service. … The essence of the iMac is to be the finest possible consumer computer in which each element plays together” — Steve Jobs

Conceptual Architecture grants us passage between the from-the-outside and the from-the-inside views of the system, allowing us to design interactively across the concept of the system and the internal concepts that allow that conception to take form. And it allows us to think about the elements and how they play together. More specifically, it is the conception and narration of the organization of the parts or elements of the system and its key, or architecturally significant, mechanisms — as the system is conceived, developed and evolved. More pivotally, it is an agent of consilience, helping us to bridge various divides in system design.

Some related resources and ideas:

Comments

Conceptual Architecture: Why

Why: Motivation for the Conceptual Architecture View

Conceptual Architecture is a key medium for describing the “big picture” and essential design ideas of our system, helping others to more rapidly comprehend a complex system, how it is composed and its critical mechanisms or interworking to achieve some key internal system capability essential to sustaining itself. The Conceptual Architecture Diagram serves as a high level map of our system, providing for navigation around complex systems, location of responsibilities, and identification of dependencies.

But Conceptual Architecture is also a focus of essential design work! As we conceive of the system structure, we’re tackling questions of decomposition and modularity, with the dual of composition and emergent properties. We do so to make the system organizationally and cognitively tractable, bringing order to the system. We might think of this as a kind of “social order” if we see its elements as agents of system responsibilities, working with cohorts to deliver system capabilities and in this interaction effecting emergent properties. But even if we don’t go that far, we are at least bringing a kind of mechanical order, complete with hierarchically composed sub-orders, to the system.

Modular systems enable us to:

  • simplify the design with cohesive units of responsibility
  • partition realms of uncertainty and experiment, separating them from more stable and understood areas of the system;
  • separate areas of the system that change for the same reason (for example, dependencies on underlying technology), from other areas of the system that have different change impulses;
  • create areas of focus for investments in highly tuned performance or extreme robustness and resilience;
  • separate elements for reuse or leverage; and
  • create units of accountability. (Whether we organize development primarily by features or by components, it is helpful to have clear component ownership so that the structural health of the component is an explicit charter.)
  • Early on, we’re dealing with our conceits of formative system elements — on paper. We’re crafting lo-fidelity sketch-prototypes of the system structure, and iterating across the conception of the system and our discovery and exploration of how we might conceive of, arrange and build the system. Because it is cheap to do, relying more on how generative our imagination is leveraging the grist of experience, patterns, capabilities in other systems and frameworks, and so forth, we can explore various alternatives, discovering elements and relationships, illuminating the system conception or possibilities we might offer in terms of system capabilities. We can play with different factorings of responsibilities into capabilities and onto elements. And we can evaluate our postulated alternatives with respect to coverage or completeness as well as balance, harmony, conceptual integrity and consistency, in addition to the achievement of desired system outcomes. All of which allows for a much more active, adaptive, creative exploration of the system concept and structural organization before the direction is channeled and cast in shaping, anchoring expectations and code.

    We don’t claim to be able to so well conceive of the system in its ultimate form that we go into system development with a perfect and complete design. We do claim that we can begin the evolutionary adaptation process with a more plastic and malleable medium than a growing mass of code. And then we can support code evolution with, again, a more malleable medium for exploring architectural-level refactoring and adaptations — changing sketches and more formal models to explore the impact of ideas before we incur the cost of making the changes in the code base. All the while maintaining greater intellectual traction over a complex system, because we have cognitive aids to system understanding and work with system constructs in their own terms, rather than always and only in terms of code which contains details that obfuscate.

    Some related reading:

  • Decisions, Concerns …(re-)Defining Software Architecture
  • Why Do We Need Software Architecture?
  • What architecture is about
  • Comments

    Conceptual Architecture: What

    What is Conceptual Architecture?

    “Conceptual Architecture” is the conceptual view(s) of the architecture of a system. It describes at a broad brushstrokes conceptual level the significant design ideas of the system. In particular, this view includes diagrams and text which identify, explicate, rationalize and contextualize the key structures and mechanisms of the system, and the relationships among them.

    Conceptual Architecture expresses the key architectural elements and their relationships that give “shape” to the system. Each of these architectural elements, or abstractions, play a critical role in the system which is signified by their responsibility assignments. They collaborate and interact, and these relationships are a significant design consideration not only because interaction points (analogous to articulation points in physical systems) and “seams” in the system are non-trivial, but because interactions give rise to synergy that makes the system more than a simple aggregate of executing parts. The parts, their arrangement and concert, cohere within and give expression to the system “gestalt.”

    Conceptual Architecture Diagram

    The Conceptual Architecture Diagram renders the formative architectural abstractions (named boxes) and their interrelationships (lines). For complex systems, there may be a set of such diagrams, exploring the (de)composition of more complex, architecturally interesting architectural elements.

    Component-Responsibility-Collaborator-Rationale (CRC-R) Descriptions

    The Conceptual Architecture also identifies the responsibilities of each of the architectural elements. We advocate doing this on Component-Responsibility-Collaborator-Rationale (CRC-R) Descriptions* for each (conceptual architecture) component, so that the reasoning behind component design choices is captured along with the responsibilities (connecting the dots to desired outcomes, strategic decisions, lessons from experience and grounding assumptions, etc.), and dependencies and other relationships are also recorded. It is a simple and “just enough” format that matches the spirit and intent of Conceptual Architecture, supporting conception, communication and evolution of the key design ideas, including the fundamental organization, of the system.

    Mechanism Sketches

    Beyond the (set of) Conceptual Architecture Diagram(s) and CRC-R descriptions, architectural mechanisms (i.e., mechanisms of architectural significance) are sketched (in diagrams and descriptions) to conceive and convey their essential design nature — the design intent, contextual assumptions, structure, and “how it works” dynamic considerations. The capabilities of the system emerge from the inter- and intra-working of the parts of the system, and mechanisms allow us to focus on specific processes within the system, conceptually (at this point) designing architecturally significant functions of parts of the system.

    Architecturally significant mechanisms are those that have more diffuse or systemic impact or are make-or-break important to system outcomes. Many of the patterns in our field’s literature formulate tried-and-true mechanism designs addressing very specific system capabilities (often internally focused at system sustaining and structural integrity concerns because these are common across systems irrespective of their specific user-facing functionality). A system has functions just like a body has functions. And many of these are internal system sustaining functions that have to do with continuity rather than serving any immediate external demand being made of the system.

    Together, the expressions of the Conceptual Architecture form a conceptual framework within which we conceive of, reason about, communicate and share, extend and reify and evolve the key design ideas of the system.

    References and More Information

  • Conceptual Architecture [Draft for Review], Ruth Malan
  • Conceptual Architecture and example Conceptual Architecture and Why Bother with Concepts, by Doug Newdick
  • Our draft chapter on Conceptual Architecture (the book has been reconceived and is being rewritten)
  • Key to conceptual architecture is the notion of abstraction. I need to rework it, but this piece on Software Abstractions gives an introduction.
  • Logical Architecture is the actionable set of architectural design descriptions or specifications, and includes interface specification.
  • Context: see Visual Architecting Poster and Core Activities Chart.
  • Conway’s Law and architecture of responsibility
  • The influence here also includes the RDD (responsibility driven design) work of Rebecca Wirfs-Brock, as well as work we were doing on Team Fusion under the leadership of Derek Coleman.
  • Comments

    The Art of Change: Fractal and Emergent

    Our “The Art of Change: Fractal and Emergent” Executive Report covers
    1. a model of change, showing how the vectors of change are different at different points in the lifecycle, so that agility means different things, depending on where in the lifecycle the product-market is
    2. a discussion of how the meaning of business and the meaning of design are shifting
    3. Jeff Bezos notion of fractal strategy, leveraging it to illustrate how fractal strategy enables intrinsic agility
    4. positioning IT as a leading player on a strategic stage where relationships and business intelligence are key drivers of innovation and agility
    5. the tandem role of strategy and architecture in an agile business and the implications for architects
    6. a fractal notion of leadership, in a business that relies on fractal strategy and tandem architecture to combine intentional goal-seeking with emergent responsiveness

    Business strategy and its tandem architecture creates coherence of purpose and concert among the many socio-technical systems, the many smaller pools of action and influence, within an organization, so that bigger, more ambitious, impactful things get done. While embracing emergence or extemporaneous dynamic responsiveness, we also note that strategic differentiation takes intentional focus to align inspired, creative, inventive thought and action so that many contributions of mind, will and hands build the systems that create and sustain competitive distinction in the market.

    We borrowed Jeff Bezos’ image of strategy happening fractally at Amazon, and put words to what is done, varyingly, in organizations. The important thing about creating this image of fractal strategy and tandem architecture though, is that it gives us a way to have the conversation about the relationship between strategy and architecture. Why? Because there is inconsistent understanding of the role of strategy, let alone architecture!

    In some organizations, strategy is ignored or derided — they claim there is “no strategy,” and that is treated as a point of cultural pride. A point of cultural pride. Hmm, that sounds like identity, which is a key part of strategy.  So strategy in the organization is fractal, with an independent “cowboy” (shoot first and aim after) culture set as the unifier at the corporate level, and other elements of strategy pushed out to the business elements. But as soon as that company wants to achieve something more coherent across its businesses, it finds itself needing to work strategically and architecturally to create a shared intent and the relationship platform for enabling that coherence. So, whether “dynamic, organic, fractal strategy” enters their parlance, allowing them to explicitly talk about intentional and emergent strategy or not, they have to get more intentional if they want to do those bigger things that require concert to make them more the way they would like them to be (Herbert Simon’s wonderful way of defining and motivating design).  

    The impetus for writing this report, was an increasing rumbling around the future of IT and EA. Well, of course we know IT and EA has a healthy prognosis. Still, many choose to see IT as a cost-center — one that encumbers with a mish-mash of entangled, brittle systems, and expensive tastes in technology frills that can’t be afforded in lean times, at that. So it is worth articulating the counter-position, don’t you think? Anyway, that’s a key message — articulating the role of IT and architects (product, system and enterprise) in sensing, catalyzing and responding to change.  So the report makes points like:       

    “Many of the business capabilities that IT supports and enables have to do with building and maintaining relationships and their information spaces to run the business and create strategic advantage. … 

    Relationships, both formal (with codified transactions) and informal (with dynamic, even ad hoc, interactions), are enabled through high connectivity. In Connections, James Burke, commenting on the Gutenberg printing press, observed “the easier it is to communicate, the faster change happens.” Alternately put, new ideas come about through conversations, and conversations through relationships, and increasingly these are digitally enabled and/or enhanced.”  …

    “When we recognize that this is a world where organizations increasingly compete on and for relationships, perception, and fidelity, and on information leverage, the strategic role of IT jumps into sharp relief. Place this in a context of change, and IT finds itself with a leading role on the strategic stage. Whether it is playing the role of the proverbial bad guy responsible for runaway costs and change encumbrance or a partner in a landscape-defining dance of change depends very much on how well IT is integrated into strategic decision making — at various levels in a fractal approach to strategy setting.”

    and

    “Complexity is a key driver of architecture. That is to say, as complexity increases, so does the need for architecture. It is not that we want complexity to go away, for value comes hand in hand with complexity. Instead, we want to harness complexity and, as it were, to tame it so that it serves rather than obfuscates and subverts the value we are creating.”

    and

    “The role of architects in an agile enterprise, therefore, includes taming the transmogrifying mess created by responsiveness, dynamic learning, and accommodation, even while leading with intentionality to innovatively envisage, build, evolve, and sustain systems and their explicit, enabling and constraining architecture decision sets.”

    – Ruth Malan and Dana Bredemeyer, The Art of Change: Fractal and Emergent, Cutter Consortium Enterprise Architecture Executive Report, Vol. 13, No. 5, 2010.

    We hope that the Report persuades managers and architects that there is an important relationship between architecture and strategy, and that relationship doesn’t have its foundation entirely in the business side, nor entirely in the technical side — but rather in a partnership where strategy and architecture work together collaboratively. That is, they inform and are informed by each other, enhance and are enhanced by, lead and are led by each other. And I hope that the paper unfolds the salient topics in an accessible manner — accessible across the languages of business and technology — to motivate and enable that dynamic tandem relationship.

    The Art of Change: Fractal and Emergent is the first in a two-part series, and focuses on the what and the why. Part Two, The Art of Change: To Lead is To See, To Frame, To Draw focuses on the how. We hope that you find the Fractal and Emergent paper, with its focus on agility through fractal strategy and tandem architecture, inspiring and useful. If so, you can play a role in Part Two, helping us improve it by becoming a reviewer or simply by providing encouragement.

    You can download a complimentary copy of The Art of Change: Fractal and Emergent at http://www.cutter.com/offers/artofchange.html.

    Comments

    Why Getting Past “But…” is Important

    You’ve probably read Getting to Yes and heard of Getting Past No, so why Getting Past “But”? Well, because “but…” is insidious, making it harder to get past than an outright “no.” The person who says “yes, but…” is ostensibly aligning with you. Ostensibly agreeing but for this teensy caveat—this objection that is a showstopper! It can be resistance in a subtle guise, seeming passive yet inherently active—the kind of action that is actively rationalized non-action. Or it can be genuine goodwill—indicating a real desire to orient with you, and active intellectual, creative engagement. The trouble, though, is that “but” can become a barrier. We need the attitude that looks beyond “but.” If we look only to “but,” only to the objections, the reasons why not, we stop there. We need to look to what we want to accomplish, then figure out how to get there from here. We need to look beyond “but” to get past “but.” Yes, this is the stuff of “kindling the collective mind,” engaging others in seeing how we would like things to be, then engaging their creativity in resolving how to get there.

    I’m told “I agree with you, architects should play an active role in requirements, but reality in my organization is that the structure and process doesn’t allow that.” Yes, that reality is hard to change. And it will not, so long as the one person who could begin to make the change, the person who sees that the change is needed, doesn’t start to lead the change! First, to see the need, then to help others see a better future, then to enroll them in creating that better future.

    This by way of an architect’s signature on an email I received this evening:

    “Reasonable people adapt themselves to the world.  Unreasonable people attempt to adapt the world to themselves.  All progress, therefore, depends on unreasonable people.”

    George Bernard Shaw

    Yes, change can be hard, and it can take a long time to even get to the point where people recognize the need for change. It took Madison five years to get the parties to the table to create change. Hopefully it can take us less time to restructure the status quo in software development. Our Getting Past “But” paper is one resource you have to help shift perception and expectation. The Agile Movement also helps underscore the importance of multi-disciplinary teams. If your organization’s approach to scale and complexity is to do just enough requirements and design upfront (for example, to spin off concurrent teams with enough context), you can still leverage the learning that the Agile Movement has embraced—multi-functional teams, iteration and stakeholder participation allow more concurrency to happen earlier, with better outcomes.

    Comments

    Scaling Agile with VAP: Getting Past “But…”

    Our Getting Past “But…” executive report covers two essential areas:

    1. innovation, the circles of innovation model, the innovation process, and what all this means for architects.

    2. scaling agile development projects with VAP (emphasizing just enough design upfront or JEDUF).

    These map roughly to the first and second halves of the report, though we encourage those interested primarily in the material in the second half to also read the first half. What follows is a brief outline of the contribution of these two parts of the Getting Past “But…” paper.

    Innovation and Architects

    Innovation is rampant, and clearly companies big and small are innovating apace. At the same time, many companies are disappointed with the return on their innovation investments. Industry incumbents are adapted to the status quo and defend their inertial tendencies with “but..”: but we aren’t chartered to do that, but that’s too risky, but our customers aren’t asking for that, but that would cannibalize our market. Focusing on immediate releases and incremental improvements, thwarts competitive landscape reshaping innovation which then tends to come from outside, often from start-ups.

    Getting past “but…” takes a shift in attitude. It is true that this shift is fostered by empowerment, with an innovation culture established by top management. Google is the prototypical example there. It is also true that the shift can start with the individual. The creation of masking tape (the 3M story starts around minute 21 of Scott Berkun’s presentation) is just one example where the inventor used his own initiative and limited budget to fly a skunkwords project under the radar. We need to recognize that generally there isn’t a shortage of ideas. And, in aggregate, there isn’t a shortage of willingness to take risks—witness the number of failures, including startups. What we need are more effective ways to get good ideas on the table, sift for those that make good business sense because they create high customer value that can be used to build differentiated identity and strategic advantage in the industry, put them through early, quick and cheap failure/improvement cycles, and get more and more talented peoples’ heads in the game to build the system.

    Our Getting Past “But”: Finding Opportunity and Making IT Happen report speaks to role and process changes that empower design teams to create a new competitive basis through differentiating innovations. You can download a complimentary copy from Cutter Consortium at http://www.cutter.com/offers/findopportunity.html.

    In this paper, we take the position that architects need to be part of, if not lead, the innovation team. The architect’s role is to help the business identify opportunities to create value through capabilities that technology brings to the table. This leverages the unique perspective of the architect into technology and the organization’s technical capabilities, but it also leverages the architect’s unique skills in system thinking and modeling.

    VAP and Scaling Agile

    VAP (the Visual Architecting Process) is all about being agile even when the complexity of the system, and the organizational unit(s) building it, demands just enough upfront designfor example, to launch concurrent agile teams. What VAP emphasizes and enables is parallelizing the requirements and architecture iterations, with intensive stakeholder involvement as pertinent to the quick cycles. VAP can be applied during coding cycles, but for complex systems, early VAP cycles use the cheapest possible artifacts (e.g., models and prototypes) for learning quickly about stakeholder value and architectural challenge. This concurrency, together with the principle of stakeholder involvement (including but not limited to end users), is a major value and contribution of VAP.

    We write about JEDUF and agile architecture in the Getting past But paper. The content of the report is so important to the conversations we’re having, to the challenges of organization after organization as the hype around agile pushes larger and larger projects to experiment with agile development. As we do so, we need to leverage all the lessons of our histories creating complex systems, as well as the lessons and values of agile development, to adapt a process that works for concurrent development of complex systems.

    References

    This synopsis derives from writing in Ruth Malan’s (almost daily) architecture journal.

    Ruth Malan and Dana Bredemeyer,  “Getting Past “But”: Finding Opportunity and Making It Happen.”  Enterprise Architecture Executive Report, Cutter Consortium, August 2008 You can download a complimentary copy from http://www.cutter.com/offers/findopportunity.html.

    Comments

    Conway’s Law

    The Wikipedia community describes Conway’s Law like this; I paraphrase it like this: if the architecture of the system and the architecture of the organization are at odds, the architecture of the organization wins. The organizational divides are going to drive the true seams in the system.

    The architecture of the system gets cemented in the forms of the teams that develop it. The system decomposition drives team ownership. Then the organizational lines of communication become reflected in the interfaces, with cleaner, better preserved interfaces along the lines where organizational distance increases. In small, co-located teams, short-cuts can be taken to optimize within the team. But each short-cut that introduces a dependency is like rebar in concrete–structurally efficient, but rigid. If the environment changes, demanding new lines to be drawn, the cost becomes clear. The architecture is hard to adapt.

    One could say this is part of the innovator’s dilemma. Sustaining innovations, that is, incremental improvements within the cast of the architecture, are what the organization is adapted to be good at. But when a breakthrough innovation demands a new architecture, a new organization (unencumbered by power trees that grew up around the old architecture) tends to be more fleet in bringing the innovation to market.

    Another implication of Conway’s Law is that if we have managers deciding on teams (what they’ll do, who will be on them, and how they will relate), and deciding which services will be built, by which teams, we implicitly have managers deciding on the system architecture. They determine system chunks (services or components) and capabilities by deciding who will build what.

    Conway’s Law also kicks in if we take an initial guess at the system decomposition (a first-cut conceptual architecture), allocate subsystems to teams, and sally forth–the team boundaries will tend to become boundaries within the system. Anything else will be a feat of architectural heroics; hard to accomplish, when architectural heroics have to compete with schedule heroics driven by the steady beat of integration clocks. Yet, architecture is where we address cross-cutting concerns, or at least those that needs-must be addressed with a system perspective so that when it comes time to compose the system it will have the properties stakeholders care about, rather than emergent properties that may or may not suffice.

    Roles are defined by their responsibilities and associated decisions. Architect is a role. Any person may play one or more roles. That is, the architect role may be shared among a group of people (as in many agile project teams), or one person may hold more than one role (as in many small teams, especially in startups).  This may be overt and declared. And it may be the result of decisions that are actually effected. If management decisions determine the architecture of the system, they are in effect its architects. If developers determine the architectural decisions, they are in effect its architects.

    “Duh!” you might well be saying. Yes, a lot of what is absolutely common sense when it is put plainly, is so obtuse in the face of perplexifying reality. Simply witness all the heated arguments and misunderstandings you get around the topic of the role of the architect.

    But what does it mean? Architecture needs to happen across the interfaces, and this also means across the system/organization interfaces. It means that system architects (who we call architects) and business/organization architects (who we call managers) should not work as if one has no impact on the other.

    Other references:

    Why do we need software architecture (how architecture serves organizational and technical purposes)

    Who needs carrots anyway? (about the iron triangle and the interface between management and architects)

    Comments

    Architecture Documentation: Courage to Fly in the Face of Convention

    I have learned what I know about good architecture from working with good architects, many of whom have set the bar for excellence in architecture documentation. That said, I am too often caused to lament that the only thing harder than getting engineers to read the architecture documentation is getting architects to write it!. So why bother? Who cares? It is only going to get out-of-date. It takes work, and isn’t that time better spent coming up with better solutions to the challenges we face? Or better still, writing code, which is the ultimate in system documentation–right???

    An architectural decision that isn’t written down has a lifespan of, let’s see, what is the memory span of a goldfish? Actually, it turns out that a goldfish memory span is not just a few seconds as previously thought. And we’re even smarter than goldfish. Can’t we just rely on our individual and group memory? Well, we have all been in the situation where a decision we thought we made yesterday, is still under debate today, and we have to persuade and coax, inform and influence, brow-beat and defend all over again.

    Writing the decision down doesn’t get us entirely away from this perpetual churn, but it does help. We get sign-off on the decision. Once that is achieved, we can demand that only counter-arguments that have a strong link to architectural requirements can be brought up in contention with the decision.

    Still, the half-life of an architectural decision depends very much on the authority vested in the architects, and how this authority is formally and informally reinforced in the organization. And it depends on what the architects do to communicate the decision, and what support and follow-through they apply.

    So, if we cowtail to the push-back against architecture documentation from extreme agilists, and our own disinclination to write down architecture decisions and thinking that went into them, then we are sending the message that the “architecture” is not a set of decisions but just a fluid initial starting point that we expect everyone to remold and reshape actively and without constraint–or restraint.

    In some situations this may be just fine. If we are working on a novel system, without precedent in our experience and in the collective experience of our industry (so we can’t hire in the experience we need), then it would be foolhardy to create an architecture specification early on and expect to live by it through the life of the system.

    Architecture is intended to constrain–and enable, but the price is that we must constrain; that’s what decisions do. Once made, they have eliminated some alternatives we might otherwise have picked. Yes, like stop lights, they enable and constrain. So, is it reasonable to constrain (and enable) ourselves early on in the project? While we are always pushing boundaries beyond what we already know, we are generally also working with a good deal of experience to leverage. To the extent that we can create an architecture that we can validate and build confidence that it will serve us through the first release, and prepare us well for subsequent releases, we should do due diligence when it comes to documenting our architecture.

    But what is architecture documentation? In good part, the architecture documentation consists of the very models we use to think through and make the architecture decisions. In short, much of the work is already done, assuming that we use models to visualize and evaluate architectural approaches. The diligence part has to do with:

    1. making sure we write down the thinking behind the models

    2. keeping the models (and supporting explanations) up-to-date

    Architecture documentation explains and justifies the decisions that embody the architecture. So we need to articulate the reasoning, tracing the decisions back to the requirements that drove them, keeping track of alternatives we weighed but ruled out and why (so we don’t have to make the same arguments again and again), and writing down assumptions we made (so if these assumptions are invalidated, we know we have to revisit the decision).

    When we do this, we also help educate the engineering community, sharing the experiences that shaped our decisions. By documenting our reasoning like this, we make our knowledge explicit and shareable. Further, it makes us more careful, because we leave a record that can be assessed and debated.

    One of the things we have to do as architects is figure out where to push back against the status quo, to lead out of the rut we are in to a better way of doing things; a better way that enhances our community’s quality of life. And we have to figure out which battles just aren’t worth it, because it takes energy and passion to lead these changes, and some changes are more important than others.

    For the things that we see rising above this cut-line, we need to do what it takes to be effective. We must not allow ourselves to be lulled by the cries of “it’s all going to change anyway” to escape the effort it takes to write good documentation for architecturally-significant decisions. If we want these decisions to impact the behavior of people implementing them, we need to do our part in communicating them.

    The architecture decisions must be recorded, so we have a ready reference that doesn’t depend on us being always in the room, ready to explain and defend each decision. The decisions must be communicated and understood. It is worth it, if the architecture decisions are worth it. So the decisions must be strategic and minimalist, and relevant. As soon as they are not, we must be on hand to adapt the architecture decision set.

    Once written down, to be sure not everyone who should read the various architecture documents, will read them. But those that do will have a much better understanding of the architecture, and the rationale for the decisions it encompasses. Good models and well-written explanations get right into the head of the reader in a personal and effective way. The reader can engage, backtrack, hold an internal dialog with the material until it is well understood, or at least clear where the questions are. Each reader will be better positioned to explain to their  peers and reports what the architecture means, in the narrow direct sense and in the broader sense of its intent. This very effectively expands your capacity to champion and explain architecture decisions and catch misunderstandings and misapplication of the architecture.

    All this presumes that you and your team can come up with an architecture. It does not presume that you make every architectural decision in advance of all coding. Not at all! But when you have a complex organizational setting (large number of developers, distributed teams, etc.), then you need to do more of the architecture work upfront, and document AS YOU GO, not afterwards! There never is an “afterwards” and even if there was, you’ll have forgotten much of the rationale, if not many of the important decisions. Besides, though we need architecture documentation to help us with system evolution, we need architecture to create a system that addresses our architecturally significant requirements in the first place.

    Early on, figure out what the architecturally significant uncertainties and risks are, and figure out what you must do to resolve these risks. Leaving them until “you must deal with them” is risky business. Then work quickly, with a focus on architectural priorities, to get a minimalist set of architecture decisions explored, validated (through models, reviews and assessments, simulations, mock-ups, and early, focused development of critical pieces of the code) and documented!

    The bottom line: No architecture documentation –> no architecture; no architecture and we rely on organic people-intensive communication processes that, on average, don’t scale too well. No architecture + big project –> project wipe-out.

    Other Perspectives on Architecture Documentation

    For Architecture Documentation:

    On-the-fence about Architecture Documentation:

    Creating Architecture Documentation:

    See also Software Architecture Books and Architecture Documentation (links) in my Trace in the Sand Architecture Journal

    [8/25/06: Had to republish this post so the sidebars would show up.]

    [9/16/06: Arnon Rotem-Gal-Oz is blogging on the Software Architecture Document -- on 9/12/06. See also Deliverables sections of our Software Architecture Action Guide book chapters on Meta, Conceptual and (soon) Logical Architecture.]

    Comments (1)

    Modularity and what we can learn from Trek

    I love my Trek bicycle, but with Shimano gears and brakes, and Bontrager frame, wheels, tires, and pedals, the ineffable Trek quality cannot be pinned down to any Trek-branded component. The success factor here: modularity with clear interfaces, and clear design requirements pushed down to the modules so that excellence at the level of the overall system is achieved through the composition of excellent pieces.

    Bicycles have been around a long time, and there is a dominant design. That allows innovation to be focused on the components and subsystems, with less frequent revolution in the overall decomposition. Together these 3rd party pieces fitting into a standard decomposition nonetheless yield a system that is unique and differentiated. Modular components driven to excellence in of themselves, plus a clear sense of the differentiating system qualities, yields a distinctive product. Flawless. So well-designed I can say wthout reservation, I love this product.

    It is the architect’s responsibility to create the architecture (with collaborative input from component designers): to make decisions about the system identity and integrity, and decisions about components and their scope and responsibilities, and address cross-cutting concerns and system properties. The architecture provides the context to push excellence, innovation and quality to the level of the components.

    In the software world, in the pressure of the moment we allow ourselves to accommodate the architecture. Every compromise to the modularity of the system increases the future cost of change. But we live in the present. We have to get out of this singular mindset, and part of the way out is to have an architect with the clear and well-known charter to preserve the structure of the system—to defend modularity and ensure that the modular pieces do indeed work together to deliver the functions and properties required.

    Feature set teams or storyline teams (or whatever else you call them in your group) tend to work across the layers in a system. This work partitioning helps drive out features that users can respond to and give feedback on, so they can be improved. Great! But it also makes it important to empower an architect to defend and preserve the architecture, and yes, evolve the architecture when, on balance, that is the best route.  

    Objects, then components, now services, are all hyped because they promise to address the build-from-parts need in our software world, at ever greater levels of granularity. But solving our problem isn’t simply a matter of picking a programming paradigm that supports (larger-grained) parts. We have to become good at designing parts—parts that really fit our system; really fit our need.

    I acknowledge that I am fundamentally, completely, biased here, but I can see no other course than through getting better at architecture, and better at protecting and preserving our architecture, and better at investing in evolving and regenerating our architecture. And better at investing in what it takes to completely revolutionize our architecture, even though that means we have to do so with a whole new set of people, which is counter-intuitive (see Rechtin’s book Why Eagles Can’t Swim, 2000).

    Comments

    « Previous entries Next Page » Next Page »