Neglect the Meatware at Your Own Peril
It’s the people, stupid!
We have serious problems today with delivering modern digital capabilities. Modern systems architectures cannot be delivered and operated by traditional organizations. Conway’s Law is an ever-present and undeniable force. Against this reality, most organizational architectures do not align with the loosely-coupled software architecture design vision. This is the most overlooked aspect of digital transformation efforts and modernization programs. In fact, this is why your organization’s architecture is eating your systems architecture.
Organizational architecture and the organization’s communications architecture are more of a controlling force over systems architecture than any other factor. Progress and adoption of new technologies and ways of working have been a struggle, even though they are essential for competing and winning on the modern battlefield. An industry has emerged, particularly among the large consultancies, to provide a holistic Agile, DevOps/DevSecOps, cloud-native transformation to the DoD. Large sums of annual spending are going towards “enterprise transformation” projects to address organizational change, technology implementation, scaling Agile, or business process transformation. These often fail to achieve lasting outcomes. Across the Department of Defense (DoD), Digital Transformation has been met with passionate support, focus, hesitance, resistance, or even repentance, even though it has become a nebulous catch-all term.
Re-organizing your schedules into two week blocks and calling them \”sprints\” will not enable you to deliver modern software architectures.
Conducting Daily Standups and Backlog Grooming will not enable you to deliver modern software architectures.
Giving team members new titles and certifications will not enable you to deliver modern software architectures.
I say this because Conway’s Law is always present. And I believe organizational architecture eats culture and systems architecture for breakfast, lunch, and dinner.
You must think about and apply architectural design principles to your teams and how they organize, work, and communicate. API-centered architectures allow systems components to decouple within clearly defined bounded contexts. The organization can apply the same architectural design patterns by establishing Team APIs. That shift to organizational architecture is critical for delivering modern software architectures.
The unbreakable bond between the organizational and systems architectures
There is a direct influence between social (people, processes, policies, etc.) and organizational technical structures. What’s more, as microservices architectures and broader adoption of API-driven development have taken hold, the systems architectures have begun exposing significant challenges in the communications and interaction among the teams that make up organizational systems and architectures. Those within the industry, myself included, have seen that organizational architecture and structures are often overlooked while emphasis is always given to new technical architectures. This results in stalled implementations, frustration, delays, friction among components of the organizations, recidivism, holding onto legacy technical architectures and widespread technical debts, and shadow IT.
It is essential to define a few crucial concepts that shape the organizational architecture design patterns:
Conway’s Law:
Melvin E. Conway’s famous paper from 1961, finally published in 1968, stated:
The basic thesis of this article is that organizations which design systems (in the broad sense used here) are constrained to produce designs which are copies of the communication structures of these organizations. We have seen that this fact has important implications for the management of system design. Primarily, we have found a criterion for the structuring of design organizations: a design effort should be organized according to the need for communication.
This criterion creates problems because the need to communicate at any time depends on the system concept in effect at that time. Because the design which occurs first is almost never the best possible, the prevailing system concept may need to change. Therefore, flexibility of organization is important to effective design.
Ways must be found to reward design managers for keeping their organizations lean and flexible. There is need for a philosophy of system design management which is not based on the assumption that adding manpower simply adds to productivity. The development of such a philosophy promises to unearth basic questions about value of resources and techniques of communication which will need to be answered before our system-building technology can proceed with confidence.
Sociotechnical Systems (STS):
STS is an integration of both the social sciences and systems engineering. In STS, the effects that technical and non-technical components of architecture have on one another are recognized as part of the integrated, holistic architectural model. No system or software exists in a vacuum without societal or organizational factors affecting them. Academic research has provided a layered model highlighting the interconnected elements of any given system – from an organization conducting software engineering, airline connection networks, social networking, and urban traffic. Within the model, the layers represent the following:
- Organizational layer: Strategy, management, and internal regulations and processes
- Social layer: The broader culture, regulatory environment, and laws outside of the organization, as well as the people who are end users and customers
- Business process layer: Business activity-supported processes that define how technology is used internally and how the business operates
- Equipment layer: Hardware the business relies upon for development and operations
- Operating system layer: Systems that bring hardware and other software together
- Data management and communications layer: The layer bridges the operating system and the application so information can be used and managed appropriately
- Application layer: Software that customers or end-users see and interact with. It provides the user interface and is often the most visible layer of the STS.
Bounded Context:
This is an approach in Domain-Driven Design (DDD) based on the concept of an internally consistent system with carefully designed boundaries that mediate what can enter the system and what can exit it. A Bounded Context consists of various components (microservices, web applications, mobile apps, databases, message queues, etc.). It also serves as a logical barrier that encapsulates those components. The details can couple and freely pass data to each other internally. But the Bounded Context helps enforce loose coupling externally, defining explicit points where:
- External data can enter (perhaps via a consumer subscribed to a Kafka topic)
- Internal data can exit (maybe via another Kafka topic or via a well-design GET API, carefully crafted to hide any interior system details)
In November 2022, two experts in the field of software development and DevOps were on a podcast with Thoughtworks titled “Reckoning with the Force of Conway’s Law.” ( https://www.thoughtworks.com/insights/podcasts/technology-podcasts/reckoning-with-the-force-conways-law). Martin Fowler and James Lewis summed up the entire thesis of this discourse with the following statement:
“A lot of what we see out in the industries is people ignoring [the impact of Conway\’s Law] and trying to come up with architectures or just not really thinking about the organizational design and trying to pretend it isn\’t there, and as a result, you get this mismatch where people are trying to do something in software design and their organizational design is pushing against it and the result is a lot of friction and problems.” – Martin Fowler
Being Intentional About Your Architecture
With COVID and the rapid expansion of remote-first communications and collaboration, I believe that being intentional about architecting how teams work together and communicate is even more critical. Agile/Scrum ceremonies only encompass about 3 to 5 percent of the work day; what and how are teams and individuals working together to request or provide services and solutions or enable software delivery? Email still dominates, especially within traditional bureaucratic organizations. Email is the equivalent of a point-to-point monolithic architecture, especially when senders request support from unknown entities or group inboxes.
Within software systems architectures, APIs transform the means by which applications and architectural components communicate and request or provide support and services to one another. APIs focus on organizing and structuring data and services for consumption by external parties with minimal dependency across services or components. The organization must embody loosely-coupled teams and well-architected services and support communications to deliver these architectures. I have not seen any program where this has not been wholly forgotten during “Digital Transformation” projects.
Bottom Line
No matter how much an organization spends on certifications, and consultants delivering and implementing transformation roadmaps, if the communications and dynamics within the teams are not architected for flexibility and simplicity, the likelihood of successfully having loosely-coupled software architectures is near Zero.
Team dependencies, relationships, and communications architecture
The COVID pandemic forced organizations to change working patterns and communication methods rapidly. There was a significant change in the influence of geographic and physical locations on how teams work together and communicate. We’re shifting again into hybrid or predominantly in-person work now that the pandemic is more under control. The evolution of our communications systems and patterns will not allow us to forget Conway’s Law.
In every organization, inter-team dynamics mirror the relationships and bounded context dynamics between various components of the software system architecture. These often involve connections where teams:
- Teams can be mutually dependent
- Groups can provide upstream/downstream services consumed by others,
- Teams can conduct functions that rely on the sequential execution of workflows,
- Teams can be free/independent – changes to their services or products do not impact others.
In 2021, Manuel Pais and Matthew Skelton, co-authors of the book Team Topologies, applied the technical approach to integrating modular, service-oriented software architecture to the communication and collaboration channels across people and teams within organizations. They had experience as technical architects and saw how externally driven transformation efforts missed the target by ignoring how teams worked together. In addition, leveraging an API-centered model for how teams work together within an organization provides a structured approach to flexible communications pathways that allow for delivering modern, loosely coupled software solutions. Very seldom are these inter-team dependencies and relationships identified and documented from an architectural perspective. In fact, poorly architected or non-existent team and organization architectures are a significant anti-pattern.
Applying Intentional Architectural Design
Applying intentional architectural design to the teams and communications patterns will radically shift how the mission is supported, increase alignment among lines of effort, and improve team members’ lives. Furthermore, even within bureaucracies, teams can have flexibility and clarity in communicating across teams and achieve significant gains in their ability to deliver software with modern architectural patterns.
Aim small, Miss Small
There is a pathway to success, even in top-down, bureaucratic organizations. The key is to start small. Avoiding massive enterprise transformation projects is vital. Instead of focusing on practices around daily rituals and ceremonies, focus on ways of working at the team level. It is equally important to see the organization as an integrated sociotechnical system actively. Teams and how they communicate have a direct impact on the systems architecture you are aiming to deliver.
Moreover, leaders at all levels must recognize and emphasize that communications and collaboration practices are not to be assumed.
Try to use the Team API model provided at the end of this article as a starting point. In fact, this lightweight template makes it easy for teams to codify and share considerable amounts of information with the organization to publicly establish structure and boundaries for how they work within themselves and across teams. This makes the dynamics between teams evolve away from insular, segmented parts of the organization chart to a more consumer/customer-centered approach where each team aims to make their capabilities available and discoverable to others, even if they are not directly subordinate/superior within the organization or across organizational lines.
I challenge you to bring this approach into your organization. How are you applying DDD to the teams themselves? Has this had an impact on your software and systems architectures? What challenges and experiences have you had, and how have you adapted?
Team APIs – The Sources, Templates, and TeamAPI-as-Code
https://github.com/TeamTopologies/Team-API-template
https://github.com/TeamTopologies/TeamAPI-As-Code
https://teamtopologies.com/key-concepts-content/remote-first-team-interactions-with-team-topologies
Recommend0 recommendationsPublished in Productivity, Technology & Innovation
Responses