Modern Agile software development approaches like Scrum recommend a “just in time” vision of application development that tends to make people focus only on the activities that are directly useful for the current sprint. How can you include an activity with a long-term perspective like enterprise software architecture in the iterative process of Scrum?
Author: Markus van Aardt, Adamantium Software Engineering, http://www.adamantiumse.com/
Agile approaches have brought a renewal in software development and delivery practices over the last decade. Yet, iterative and incremental development methods have been around since the 1980’s and are not really that new. What made Agile stick was the community created by The Agile Manifesto’s rehash of an old idea. Agile approaches are different in that they look outside the boundaries of software development into the softer areas of software requirements, product delivery and people management. They provide a management tool to improve the ability of software engineers to deliver better solutions, in less time and at a lower cost than what was possible with the traditional Waterfall process and even Boehm’ Spiral model. But that is not where this story ends.
I find it often quite amusing how technology trends manage to overrun conventional wisdom as a new truth and immediately negating everything that has gone before. This is the case with Agile and Scrum. So when I recently encountered a senior IT executive who believed his project was exempt from architectural impact due to an Agile development model, I was surprised – in a bad way.
To provide context into this matter, it is important to understand software architecture from an enterprise and general software engineering perspective.
Enterprise software architecture defines and communicates the application landscape in terms of the technical structure, technological environment and business context. Enterprise software engineering needs to happen within the structures provided through the boundaries and guides of the specific enterprise software architecture, also called the reference architecture. Poor alignment during design and development will lead to poor operational alignment. Architectural alignment is a non-negotiable requirement for effective enterprise software engineering. This holds true for every development methodology, whether it is the traditional waterfall process, Scrum or extreme programming (XP).
Let’s consider two scenarios. Although engineers rarely encounter new, uncharted environments, we should consider the clean slate where little or no environment exists that could be referenced. This is usually the case for new businesses. The second, more regular, scenario is the existing (or legacy) target environment. When such environment is encountered, it is critical that it is included and referenced during the design and construction phases of product development.
So how do we get to architectural maturity from either the clean slate or a densely populated environment? This figure below will provide you with a roadmap to achieve a mature and living architectural environment.
Defining a reference architecture
Target environments with no existing reference architecture are usually new and are the simplest starting point for software development. These environments typically have no legacy architecture and provide a clean slate to the engineering team. Part of the team’s responsibility now becomes the definition of the target environment architecture. Although Agile approaches lend themselves to producing reference architecture on the fly, these activities should be carefully separated into different focus areas. You should create your architecture before any other development work, although the process must be aware of the pending technology requirements.
The enterprise’s reference architecture is created and maintained first and foremost by the enterprise architecture team and the associated subject matter experts (SME). Based on the architectural strategy, the reference architecture will reflect the current maturity and plot a map towards the pre-determined end state. Strategy is determined by many factors, but mainly driven by technology deliverables, financial conditions and the larger enterprise strategy. Ongoing design and development further expands the reference architecture as and when required, in a limited way. Since the reference architecture guides these activities, major changes are usually negotiated with enterprise architecture, SMEs and technology leadership.
This provides a starting position that will grow over time as the environment evolves and matures further.
Reviewing a reference architecture
More often than not, a reference architecture already exists (whether documented or not) within which the product needs to be delivered. The project will benefit from reviewing the existing reference architecture or detail undocumented architecture as an input to the project. In the case of an undocumented architecture, it is recommended that this cycle precedes the other delivery cycles to ensure a proper and common understanding of the environment to the project team.
This review should be the initial (and most critical item) in the project backlog of the initial sprint. The only difference between this deliverable and conventional sprint results is that the reference architecture is an input to subsequent sprints within the project.
Update the reference architecture to support the project
Not all projects can simply comply with an existing reference architecture. Many software products will challenge the reference architecture (or technology environment) to create a more suitable landscape for effective operation. This is where the project architect or designer start to engage with the enterprise architecture team / lead. Although the Scrum team does not have a dedicated architect or design role, such a capability has to be included in the team structure, either through the group as a whole or with an individual skill. Effective architecture engagement is highly reliant on such a capability.
Most sprints within a Scrum project will have some dependency on the architecture. This may be inside the project from a previous sprint, or from outside the project in the architecture landscape or another in-process project. Once a requirement is formalized in the project, it has to be evaluated against the reference architecture. If the required component exists already, it can simply be reused with little or no alternative development or configuration required. This provides two critical benefits to both the project and the organization.
Benefits achieved with this model are financial and performance related. If a critical enterprise capability exists and is reused, the project would typically require little additional funding while it increases the total return on investment (or ROI) to the organization. Secondly, in line with Agile approaches’ nature, the reuse of existing artifacts reduces delivery times and can simplify environmental complexities.
The requirements and subsequent development needs to be fed back into the architecture model as revisions. The project architect (see the comments above about this role) needs to clearly identify requirements in terms of the existing reference architecture, to evaluate changes to the architecture and to negotiate the required updates to it. The changes requested by the development team are required to be in support of the existing and future architecture. The final decision and responsibility for the architecture remains with the enterprise architecture team.
Post-project review and confirmation of architecture
Modern organizations change constantly. When an organization employs Agile approaches for software development, it is a clear indication that the organization (or its software engineers at least) understands this. It is therefore critical that the architectural feedback loop is closed by reviewing and confirming the latest architecture landscape in a revised reference architecture guide in parallel with the sprint review and retrospective activities. This reduces the risk and cost usually associated with future development and support activities.
Taking this even further, a mature architecture review process will provide the Scrum team with active feedback on changes in the greater architecture. That marks the importance of architecture collaboration to ensure that engagement on these matters happen both vertically and horizontally. Vertical engagement ensures alignment between individual projects and enterprise architecture and governance while horizontal alignment improves inter-project engagement.
Closing remarks
Agile approaches have without a doubt proven their place among software development methodologies. However, a word of caution should be raised: here be dragons. When it comes to software development, there is no silver bullet. Whether a project selects Agile over traditional methodologies, or vice versa, every solution would require careful consideration of the selected methodology. Agile approaches fit well into the modern and dynamic environments due to their core philosophy. They do however not discount experience, risk management and common sense. The developed solution ultimately still remains a citizen in the enterprise and needs to adhere to the laws of the land: the enterprise architecture governance.
About the Author
Markus van Aardt is a software engineer and application architect in the financial industry. He launched his career as software developer at Deutsche Bank prior to starting Adamantium Software Engineering, a software and technology consultancy which he currently runs while contracting as architect in banking and financial services. Find more on http://www.adamantiumse.com/.
This is an excellent article… I think the Agile community does not put enough thought into incorporating architecture (and design) into Agile development.
In my experience, mostly with Scrum, project go much faster and have higher quality when there is a reference architecture, and the scrum team spends some time early in the project to develop, to review and update, the reference architecture.