Agile Architecture: An Scalable approach

This is a somehow controversial topic in the Agile world. Where Architects land in an Agile approach, may be challenging to figure out. The reality is that any significant sizable solution will require to make important technical decisions. At the same time, the solution as a whole needs to meet certain non-functional requirements (latency, scalability, security, regulations…), and sometimes is hard to track this into the different components and teams.

Architecture usually involves a heavy decision making process. How to integrate this process into an continuous delivery agile approach, is challenging. There must be a subtle trade-off between detailed upfront analysis/design, and Cowboy coding, so the feedback loop is not broken.

In this article I expose a simple approach to architecture, and some tools that allows an Agile approach to this infamous practice. Notice my definition of architecture is mostly “important decision with big impacts that are hard to revert”. Where impacts are mostly on the non-functional/ilities area of the solution.

This approach was used during more than 5 years in my company, and showed good escalation with more than 10 delivery teams, and more almost 100 people involved ( including product management and delivery teams..)

The Architecture Guild

Following regular community of practice concept in Agile, this guild will be formed as a committee where representatives of different teams participate.

The guild will meet when required depending on the triggering usecase. If the solution is in an early stage it maybe required to have a regular meeting, we found once a week is enough .

As any other meeting the agenda must be planned before the actual date, and supporting material (slides, Arch PR, diagrams..) must be prepared for the rest of the committee to review before the meeting.

Stakeholders:

• Product Owner/Program Manager
• Sales
• Support
• Managers

Responsibilities:

• Helping product owners to shape non-functional requirements into stories and well defined acceptance criteria. Identifying involved teams, and suggesting technologies to meet those requirements.
• Tracking existing solution status in terms of compliance with non-functional requirements.
• Raise concern about existing solution gaps, and propose plans to tackle it.
• Influencing roadmap backlog to cover identified gaps.
• Help to answer questionnaires presented by customers or related to obtain a certification.
• Deliver architecture changes to the teams for implementation, by sharing the decision making process.
• Provide reports to management level on current compliance situation, and ballpark estimations for improvements.

Artifacts

• List of non-functional requirements
• C4 Level 1 System Context diagram
• C4 Level 2 Container diagram
• Container to non-functional requirements matrix
• Inter-Container link matrix and non-functional requirement matrix
• Technical decision log.

Deliverables:

• Archictecture release

Kanban board

A simple Kanban board can be used by the Architecture Guild members to track the action items progress. Because there is usually no recurrent meeting, the Kanban board is better than a regular Sprint board where Sprint start/end time window constrains the board.

Arquitecture as Code: The Git repository

In this DevOps Cloud era everything seems to be turning into Code. If you think about it, architecture artifacts maybe turned into “Code” providing some important qualities that we all link to code. All architecture artifacts can be expressed as text, so we can apply regular SCM practices to track changes on architecture:

• SCM versioning approaches can be applied to architecture artifacts.
• The SCM will provide shareable links to artifacts with a well defined version, avoiding the email attachment chaos.
• SCM provides a built-in peer review process to architecture artifacts, allowing to save comments, and facilitate discussion. PR process becomes important part of the architecture as knowledge base.

Repository Layout:

• ADR: This directory holds all the ADR records.
• c4model: This directory holds all the plantuml diagrams. Level 2 diagrams can be split in different files by subsystem, to avoid
• containers: This directory holds container to non-functional requirement matrices, stored as simple markdown documents. You can use sub-directories to break solution into different subsystems.
• glossary: A simple markdown document with some terms specific to the solution.
• tech_requirements: List of tech requirements managed by the solution. Saved as simple markdown doc.

In this repository we can link artifacts from diagrams to containers, and containers to tech requirements. This facilitates the navigation from the different resources. Simple http links can be used.

ADR

Architectural Decision Records is a well established practice nowadays. It allows to synthesize the context of a technical decision. It also creates a nice decision history backlog, providing any reader the context of the decision given the conditions at that time. There are plenty of simple templates available to follow, and the file can be saved in a markdown file. Files should be created following a naming convention such as “<adr_num>_<adr title>.md”

ADR example

This example is taken Github ADR examples

C4 Model

C4 Model provides a simple, yet powerful abstraction to model the solution context. For the architecture guild level 1 and level 2 are enough detail to track the intended scope.

The Level 1 will provide a basic understanding of the problem, in terms of the actors participating with the system, and how the interactions looks like

Level 1 example

The Level 2 provides a quick view on all the containers required to run the solution, and how the containers are linked together. This helps the guild to drive internal discussions, and also to deliver architecture changes to the different teams.

Level 2 example

PlantUml

If you are wondering how Git can track diagram changes as code, the answer is simple, use PlantUml. PlantUml is de facto standard on text based diagrams, with good support on regular IDEs and even Git repositories. Plantuml will automatically render the diagram from the text description. You can use deployment or component diagrams to create the corresponding C4 model level 1 and level 2.

There are a couple of benefits of using text based diagrams worth highlighting:

• Any change in the architecture will be clearly spotted as part of the PR with text line highlighted automatically by Git. If the change is bringing a new link between existing containers, this will be evident in the PR.
• No one needs to have any special tool installed to create diagrams. A simple text editor is enough.
• Sharing a diagram is as simple as sending an http link to the Git file containing the diagram. No more email attachments with chaotic versioning. You can use external Plantuml servers such as Planttext, to share with external parties in case access to git repo is not possible. The diagram is text, can can be easily encoded in a safe URL.
• Focus on architecture reflection rather than make a pretty diagram. When creating text based diagram, I find myself thinking more about how to accurately describe the solution, rather than making it look good.

The Architecture PR

Once all these elements are established, it’s easy to drive the Architecture Guild activities around the Pull Request concept. Any change in architecture will likely cause a change in the existing container and technical requirement relationships. The PR must be started with a well defined ADR, and modify or add any impacted resources ( new containers, existing links among containers, new non-functional requirements..)

The resulting Architecture change is well defined in a PR, ready to be presented to the guild. Anyone can make comments to the PR, before or after the meeting. The Guild may approve the PR as it is, or require changes, and go into the traditional development loop. Once the PR is approved, the PR is merged to master, and a new architecture is released for the teams to implement.

Spikes / Proof of concept

Sometimes the Architecture Guild may need to evaluate a given technology to evaluate if its fitting to the purpose. Since creating full Proof of Concept working environments is heavy effort, the best way to plan this work is through Spike stories. Product Owner should plan the corresponding Spikes into the different teams and agree a proper time box and acceptance criteria. Once the Spike is completed, the Guild is in better position to approve or reject an Architecture PR.

Conclusion

• There is a place for Architecture practice in Agile deliveries.
• ArchitectureAsCode is possible, and delivers a standard and simple framework to document architecture and the decision process behind it.
• This architecture process can be scaled from small to medium sized solutions. Large solutions maybe supported with a tweak on the committee structure.