Three Questions That Help You Build a Better Software Architecture

In any software development effort, there is always too much to do and not enough time or resources to do it all. The problem is that the number of things we could build is infinitely large, and our available time and resources are, by comparison, almost infinitely small.

This applies especially to architecting. The art in software architecting is deciding what decisions need to be made now and which ones can wait.

Making decisions involves answering critical questions, and there is an order in which certain questions need to be answered. Success in architecting involves rejecting false assumptions/assertions quickly and cheaply, using empiricism. In other words, find those alternatives that are wrong and costly quickly, and choose a better alternative. The Minimum Viable Product (MVP) and its Minimum Viable Architecture (MVA) approach helps answer these questions.

When faced with the challenge of delivering an MVP with what feels like unrealistic deadlines, a team needs to answer a few critical questions in a specific order:

1. Is the business idea worth pursuing?
2. How much performance and scalability are needed?
3. How much maintainability/modularity and supportability are needed?

Figure 1: Answers to 3 Key Questions Determine the MVA Decisions & Tradeoffs

The answers to these questions determine the architectural decisions and tradeoffs that the team needs to make when designing the MVA (see Figure 1).

This list does not include questions about security (including cybersecurity and data protection), because security is non-negotiable and therefore should not be involved in trade-offs. As we discussed in a prior article, the only way to answer these questions is to use empiricism and run experiments. 

The most important question is "Is the business idea worth pursuing?"

Ideas are cheap, solutions are expensive, and solutions that don’t solve a useful problem are just a waste. The most important decision a team can make is deciding not to build something that people don’t need. The MVP tests the value of a proposed solution. For us, an MVP is a mechanism for determining whether a product idea has value. 

The MVP has to test assumptions (especially technology assumptions) that were made when the business idea was presented for approval. Are these assumptions still valid? How will you know? What data will you need to collect to decide? The MVP will need to be instrumented to collect this data.

How does this relate to software architecture? You don’t want to create an architecture for a product that no one needs. And in validating the business ideas, you will test assumptions that drive quality attributes like scalability and performance needs. To do this, the MVP has to be more than a Proof of Concept - it needs to be able to scale well enough and perform well enough to validate the business case, but it does not need to answer all questions about scalability and performance ... yet.

In addition, cost is always a factor, and the MVP has to evaluate whether the business case can be met. If the solution is too expensive to meet the user’s needs, it’s not worth pursuing. The MVA needs to explore just enough of the cost question to make sure that the solution won’t blow the budget. 

The second most important question is "What About Performance and Scalability?"

Early performance problems are a leading indicator of future scaling problems, and business sponsors understand this at least implicitly. If a system shows slow response times with only one or a few users, the business sponsors will question whether the system will ever meet its performance and scaling goals. As a result, early attention to performance safeguards the future of the development effort.

The challenge with making performance-related decisions is that "performance" is subjective and depends on the user’s expectations. They would, of course, like the system to respond instantaneously, but they are willing to accept small delays. As a result, the team has opportunities to make trade-offs in how they handle processing. For example, things that take a long time might be moved to asynchronous background tasks while the user is doing other things, so long as the results are ready for the user when they need them.

Users also make predictions about scalability based on performance. Poor initial performance may cast doubts in the users’ minds about whether the system would be able to scale and yet provide acceptable performance. This may lead to questions about the feasibility of the MVP. 

There are some certainties akin to physical laws in dealing with performance problems:

• Data access from a storage device is always expensive due to physical throughput constraints. 
• Remote data access is very, very expensive, due to latency constraints. 
• The cost of data access can be reduced by moving data to memory, but this only works if data volumes are small and memory is relatively large.

As a result, there are always trade-offs on where data resides and how it is accessed, so there are trade-offs between the two sources of processing costs.

How do you know? What metrics will you need to collect to decide?

Sometimes executing a complex algorithm takes a long time and is the source of performance problems. Optimizations and approximations aside, these computations are often good candidates for moving to background tasks. 

Verifying that the system will perform well requires the team to build the critical parts of the system in order to test hypotheses about how long certain things will take. The team does not need to build the entire system, but they do need to build the parts of the system that will affect performance to the greatest degree.

A related question to performance is "How Much Scalability?"

Scalability is a kind of invisible quality that manifests itself through poor performance under large loads (users, data, transactions, etc.). As a result, measuring scalability requires the team to measure performance as the load increases. Users want good performance as the system scales; for them, adding load should be "invisible".

Investing in scalability is one of the most expensive decisions a team will make because there is often no limit to what they could do to improve scalability. The problem is knowing how much scalability they actually need. 

A team working on delivering an MVP almost always faces a scalability challenge. Should they design for scalability now or defer that decision? As we discussed in an earlier article, scaling a system is a hard problem to solve. Finding the right balance is crucial. Underinvesting in scalability can limit a system's lifespan, while overinvesting can negatively impact the business case for an MVP due to cost.

Teams often struggle to accurately predict scalability needs, partly because business sponsors may find it difficult to project system usage growth; business sponsors usually want near "infinite" scalability, unfortunately, without the budget to match. However, scalability requirements are always present; they're just not always obvious. Every system has a business case, and within that case, there are implicit scalability needs.

Ultimately, achieving affordable scalability requires making careful trade-offs. As an example, scalability can be improved by making some tasks asynchronous, but doing so can increase the complexity of coordinating those background tasks. This complexity is not free and may affect performance.

Achieving good performance while scaling can also mean reworking parts of the solution that you’ve already built; solutions that perform well with a few users may break down as load is increased. On the other hand, you may never need to scale to the loads that cause those failures, so overinvesting too early can simply be wasted effort.

Many scaling issues also stem from a critical bottleneck, usually related to accessing a shared resource. Spotting these early can inform the team about when, and under what conditions, they might need to change their approach. 

The third important question is "How much Maintainability/Modularity and Supportability do you need?"

Teams develop their systems in modular ways to make them easier to develop, understand, and maintain. A more modular system also helps the team to identify and exploit reuse opportunities. Other techniques like parameterization or configurability make the system easier to maintain because certain kinds of changes won’t require code changes.

There is a limit to the usefulness of these strategies, however. Anticipating a change that will never occur is simply wasted effort. Enabling configurability that is never used is also a waste. And until you know whether the MVP is going to be successful, your efforts to make the system more modular, maintainable, and supportable may simply be wasted.

The challenge in making a system more maintainable and supportable is anticipating the kind of changes that are most likely to occur and preparing for the change in the code.

In a prior article, we identified 5 possible outcomes when developing an MVP and its associated MVA:

1. The MVP is successful and the MVA doesn’t need to change;
2. The MVP is successful but the MVA isn’t sustainable;
3. The MVP is partially but not wholly successful, but it can be fixed; 
4. The MVA is also partially but not wholly successful and needs improvement;
5. The MVP isn’t successful and so the MVA doesn’t matter

In that article, we observed that outcomes #2, #3 and #4 are the most likely situations that a team will have to deal with. If the MVP is successful, teams should quickly ascertain if the MVA is sustainable. That leaves most teams dealing with outcomes #3 and #4.

In case #3, the team should invest in just enough maintainability to help them fix the MVP, since they don’t yet know exactly what the MVP needs to do. There is always a risk that they make some changes to the MVP and find out that they are really dealing with outcome #5, a non-viable MVP. In practical terms, this means investing in building just enough modularity to make their next iteration easier, for example, by defining interfaces to major components but building only enough implementation to support the MVP; the refinement can be left for later.

Case #4 is both simpler and yet more complex; the MVP may need more work, but the MVA definitely needs more work. In other words, you don’t yet have a viable MVA.  It may perform poorly or may not scale, for example. Or it may be missing major components that you don’t know if you will need to build and, if you do need to build them, they will cost too much. In this case, the team’s focus should be on proving the MVP, of course, but also exploring possible MVA trade-offs to determine whether there is an MVA that would meet the needs of the MVP and be affordable.

How this relates to modularity, maintainability, and supportability is that the team should invest just enough to be able to continue to evolve the MVA as long as they need to. There are sometimes trade-offs between modularity and performance/scalability, so these trade-offs need to be examined by running experiments.

Use Technical Debt to assess Supportability and Maintainability

As we discussed in a previous article, technical debt is a popular metaphor for communicating the long-term implications of architectural decisions and trade-offs to stakeholders. Technical debt is one way to measure the supportability and maintainability of a system. Incurring technical debt is unavoidable when delivering MVPs because of the need for fast feedback on the business concept and the technical solution that supports it.

One of the most important architectural decisions that teams must make is to decide how they will know that technical debt has risen too far for the system to be supportable and maintainable in the future. The first thing they need to know is how much technical debt they are actually incurring. One way they can do this is by recording decisions that incur technical debt in their Architectural Decision Record (ADR).

The next thing they need to do is to use empiricism to test maintainability and supportability. One way to do this is to consider the cost of rework while the team is producing releases. In every iteration (e.g. Sprint, for those using Scrum), the team will inevitably incur some rework as they incorporate learnings based on feedback from prior iterations. Measuring this rework can help the team understand the cost of undoing/redoing existing code.

Another way to understand maintainability and supportability is to incorporate change cases into the work. A change case is, very simply, some intentional rework that tests how easily architectural decisions can be changed in the code. It might involve switching out major components or subsystems, or changing the messaging model from synchronous to asynchronous in some areas. The key thing is to anticipate changes that might happen in the future by making that change in a small area of the code and then extrapolating the result to assess maintainability.

Conclusion

To paraphrase Helmuth von Moltke the Elder, no MVP survives contact with customers. This ripples throughout the MVA as well. In terms of architectural decisions, teams need to first consider whether the business idea is worth pursuing, as shown in Figure 1. This, in turn, will change decisions they make about the MVA.

In our experience, the next most important set of architectural decisions the team must make involve performance and scalability, because these affect a wide ranging set of technical choices. Only then are decisions about maintainability and supportability worth considering, as performance and scaling decision changes often affect the composition of the system. In turn, these decisions can affect the business viability of the system by exceeding cost constraints. 

One of the challenges with architectural decisions is that they are never really truly decided once and for all; each new piece of information can change the impact of decisions, so the team is always adapting their decisions to new circumstances. Because the MVP is a series of small experiments, it will evolve over time based on feedback. The MVA needs to evolve as well in a similar way and timeframe.