Software Architecture for Agile Development Hayim Makabee http://EffectiveSoftwareDesign.com
My life in logos: Education: Experience:
Part I: Architecture and Agile  Different approaches for Software Architecture in the context of Agile.  The approach of Adaptable Design Up Front (ADUF).  Guidelines: How to put ADUF in practice.  An example of a real system.
Context  Iterative Software Development  Agile, Incremental
Question  How much design should be done up front?  Up front = Before starting the implementation (coding).
Big Design Up Front  BDUF = Do detailed design before starting to code.  Problems: ◦ Requirements are incomplete. ◦ Requirements may change. ◦ “The Scrum product backlog is allowed to grow and change as more is learned about the product and its customers.”
Emergent Design  “With emergent design, a development organization starts delivering functionality and lets the design emerge.”  First iteration: ◦ Implement initial features.  Next iterations: ◦ Implement additional features. ◦ Refactor.
Just Enough Design  “Just enough sits somewhere in the chasm between big design up front's analysis paralysis and emergent design's refactor distractor.” – Simon Brown  Question: How much design is “just enough”?
Planned: Modiin
Emergent: Jerusalem Old City
Eroding Design
Eroding Design  “The biggest risk associated with Piecemeal Growth is that it will gradually erode the overall structure of the system, and inexorably turn it into a Big Ball of Mud.” - Brian Foote and Joseph Yoder – “Big Ball of Mud”
Software Entropy  Entropy is a measure of the number of specific ways in which a system may be arranged (a measure of disorder).  The entropy of an isolated system never decreases.
Lehman Laws  A computer program that is used will be modified.  When a program is modified, its complexity will increase, provided that one does not actively work against this.
Conclusions so far…  In summary: ◦ We must design up front. ◦ BDUF does not support change. ◦ Emergent design works at low-level. ◦ Just Enough DUF is not clear.  We need: ◦ The design must support change.
Adaptability  “A system that can cope readily with a wide range of requirements, will, all other things being equal, have an advantage over one that cannot. Such a system can allow unexpected requirements to be met with little or no reengineering, and allow its more skilled customers to rapidly address novel challenges.” - Brian Foote and Joseph Yoder – “Big Ball of Mud”
New development mindset  Instead of planning for software development, plan for software evolution.  Instead of designing a single system, design a family of systems.
Family of Systems  Software Product Line: “A set of software-intensive systems that share a common, managed set of features satisfying the specific needs of a particular market segment or mission and that are developed from a common set of core assets in a prescribed way.” – Software Engineering Institute
Adaptable Design  Adaptable Software Design: A generic software design for a family of systems which does not need to be changed to accommodate new requirements.  ADUF = Adaptable Design Up Front!
Open/Closed Principle  “Software entities (classes, modules, functions, etc.) should be open for extension, but closed for modification.” - Bertrand Meyer
Strategy Design Pattern
Adaptable Design Up Front and the Open/Closed Principle  ADUF focuses on the early definition of the “closed” aspects of the design while at the same time allows easy evolution by making this design open to extensions.  Closed: Captures essential entities and relationships.  Open: Provides mechanisms for extensibility.
Architecture vs. Interior Design
Common Errors  Too much design up front  Not enough design up front  Too much openness  Not enough openness
Too much design up front  Capturing aspects that are not essential as if they were such.  The design may need to change (thus it is not really “closed”).  Examples: ◦ Specific details are not generalized. ◦ Designing for a specific system instead of designing for a family of systems.
Not enough design up front  Leaving away some of the essential aspects of the system.  These aspects will need to be added to the design when other parts of the system are already implemented.  Example: ◦ Adding relationships to new entities may cause changes to existing interfaces.
Too much openness  The design is over engineered, resulting in complexity not really necessary.  Example: ◦ It is always possible to reduce coupling through additional layers of indirection. ◦ With many layers, it is difficult to locate where the business logic is implemented.
Not enough openness  The design captures all essential aspects, but there are not mechanisms that allow this design to be easily extended.  Example: ◦ If there are Interfaces for the main entities, there should also be Factories to instantiate concrete subclasses.
Extensible Designs Main alternatives for extensible designs:  Frameworks ◦ Plug-ins  Platforms
Frameworks  Framework: “A software framework is an abstraction in which software providing generic functionality can be selectively changed by additional user-written code, thus providing application-specific software.”
Plug-ins  Plug-in: “A plug-in is a software component that adds a specific feature to an existing software application. When an application supports plug-ins, it enables customization.”
Pluggable Components
Versioning  Traditional: ◦ Version = modifications + additions. ◦ After n iterations: n versions.  Component-based: ◦ Version = combination of new implementations of components. ◦ System with m components, after n iterations: nm versions.
Platforms  Platform: “Technology that enables the creation of products and processes that support present or future development.”  Software platforms provide services that are used by several applications.  Platforms and applications may evolve independently of each other.
Question 1  How do you identify the components in your framework or the services in your platform?
Domain Modeling  “A domain model is a conceptual model of all the topics related to a specific problem. It describes the various entities, their attributes, roles, and relationships, plus the constraints that govern the problem domain.”
Question 2  How do you decouple the components in your framework?
Design Patterns  Reduce coupling using Design Patterns.  Most patterns avoid direct links between concrete classes, using interfaces and abstract classes.  Example: Use Observer when several types of objects must react to the same event.
Example: Recommender System
Recommender System Model Model Recommender Trainer Historic Data
Recommender System Framework After 1 year of development:  Historic Data: 13 subclasses.  Model: 9 subclasses.  Trainer: 9 subclasses.  Recommender: 19 subclasses.
What about YAGNI?  YAGNI = You aren't gonna need it.  A principle of extreme programming (XP) that states that a programmer should not add functionality until deemed necessary.  “Always implement things when you actually need them, never when you just foresee that you need them.” - Ron Jeffries
YAGNI vs. ADUF  YAGNI tells us to avoid doing what we are not sure we will need.  ADUF tells us: ◦ Define the things you are sure you will need in any case. ◦ Prepare for the things that you will probably need in the future (adaptability). ◦ NIAGNI: “No, I am gonna need it!”
Conclusions  Software systems must evolve over time.  This evolution should be planned and supported trough Adaptable Software Designs.  Domain Modeling: Identify the entities and relationships that form the “closed” part of the design.  Framework Design: Design a framework that represents the model and add mechanisms for extensibility, making it “open”.
Related Work  Software Engineering Institute: ◦ Software Product Lines  Alistair Cockburn: ◦ Walking Skeleton ◦ Incremental Rearchitecture  Neal Ford: ◦ Emergent Design ◦ Evolutionary Architecture  Simon Brown: ◦ Just Enough Design Up Front
Adaptability & Evolution “It is not the strongest of the species that survives, nor the most intelligent that survives. It is the one that is the most adaptable to change.” Charles Darwin
Part II: Manifesto for Adaptability  Lean Startup  Minimum Viable Product  Manifesto for Adaptable Software Development  SOA Principles  Microservices Practices
Build-Measure-Learn  Build-Measure-Learn feedback loop:
Building a Pyramid Incrementally (Wrong)
Building a Pyramid Incrementally (Right)
Minimum Viable Product (MVP) The MVP is the “version of a new product which allows a team to collect the maximum amount of validated learning about customers with the least effort.” The MVP should have the core features which allow it to be deployed to real customers to get initial feedback, but not more.
Building a MVP
Manifesto for Agile Software Development Individuals and interactions over processes and tools Working software over comprehensive documentation Customer collaboration over contract negotiation Responding to change over following a plan
Manifesto for Adaptable Software Development Experimentation instead of Specification Evolution instead of Implementation Adaptation instead of Modification Extension instead of Growth
Experimentation instead of Specification  Traditional software development: ◦ Detailed specification of system requirements, functional and non- functional.  Modern software development: ◦ Allows the discovery of requirements through experimentation.
Evolution instead of Implementation  Traditional software development: ◦ Focuses on the prioritized implementation of features according to specified requirements.  Modern software development: ◦ Allows the constant evolution of the system’s functionality to meet the dynamically varying customer needs.
Adaptation instead of Modification  Traditional software development: ◦ The code must be modified in order to meet changing requirements.  Modern software development: ◦ Allows the existing code base to be easily adapted to satisfy any change in requirements.
Extension instead of Growth  Traditional software development: ◦ The system must grow in order to accommodate additional requirements.  Modern software development: ◦ Allows the easy extension of the system’s functionality through modular features.
Architectural needs  The Software Architecture must support: ◦ Experimentation ◦ Evolution ◦ Adaptation ◦ Extension  If the Software Architecture is not adaptable, the Agile process will fail.  Consequences: Increasing Technical Debt, endless Refactoring.
SOA Principles 1. Standardized Service Contracts 2. Service Loose Coupling 3. Service Abstraction 4. Service Reusability 5. Service Autonomy 6. Service Statelessness 7. Service Discoverability 8. Service Composability 9. Service Interoperability
Microservices Practices (I) 1. Separated Build: ◦ Each Microservice is built independently and has its own version. 2. Independent Deployment: ◦ Each Microservice may be deployed without affecting the others. 3. Separated Data Stores: ◦ Microservices should not share data stores. 4. Independence of Platform: ◦ Microservices should be deployed in containers.
Microservices Practices (II) 5. Individual Technology Choice: ◦ Each Microservice may be implemented using a different technology. 6. Confinement of Failure: ◦ If a failure occurs in a particular Microservice it should not propagate to the other ones. 7. Independent Scalability: ◦ It should be possible to scale each Microservice independently of the others.
In Summary  Adaptable Software Development: ◦ Experimentation instead of Specification ◦ Evolution instead of Implementation ◦ Adaptation instead of Modification ◦ Extension instead of Growth
Thanks! Q&A http://EffectiveSoftwareDesign.com

Software Architecture for Agile Development

  • 1.
    Software Architecture for AgileDevelopment Hayim Makabee http://EffectiveSoftwareDesign.com
  • 2.
    My life inlogos: Education: Experience:
  • 3.
    Part I: Architectureand Agile  Different approaches for Software Architecture in the context of Agile.  The approach of Adaptable Design Up Front (ADUF).  Guidelines: How to put ADUF in practice.  An example of a real system.
  • 4.
    Context  Iterative SoftwareDevelopment  Agile, Incremental
  • 5.
    Question  How muchdesign should be done up front?  Up front = Before starting the implementation (coding).
  • 6.
    Big Design UpFront  BDUF = Do detailed design before starting to code.  Problems: ◦ Requirements are incomplete. ◦ Requirements may change. ◦ “The Scrum product backlog is allowed to grow and change as more is learned about the product and its customers.”
  • 7.
    Emergent Design  “Withemergent design, a development organization starts delivering functionality and lets the design emerge.”  First iteration: ◦ Implement initial features.  Next iterations: ◦ Implement additional features. ◦ Refactor.
  • 8.
    Just Enough Design “Just enough sits somewhere in the chasm between big design up front's analysis paralysis and emergent design's refactor distractor.” – Simon Brown  Question: How much design is “just enough”?
  • 9.
  • 10.
  • 11.
  • 12.
    Eroding Design  “Thebiggest risk associated with Piecemeal Growth is that it will gradually erode the overall structure of the system, and inexorably turn it into a Big Ball of Mud.” - Brian Foote and Joseph Yoder – “Big Ball of Mud”
  • 13.
    Software Entropy  Entropyis a measure of the number of specific ways in which a system may be arranged (a measure of disorder).  The entropy of an isolated system never decreases.
  • 14.
    Lehman Laws  Acomputer program that is used will be modified.  When a program is modified, its complexity will increase, provided that one does not actively work against this.
  • 15.
    Conclusions so far… In summary: ◦ We must design up front. ◦ BDUF does not support change. ◦ Emergent design works at low-level. ◦ Just Enough DUF is not clear.  We need: ◦ The design must support change.
  • 16.
    Adaptability  “A systemthat can cope readily with a wide range of requirements, will, all other things being equal, have an advantage over one that cannot. Such a system can allow unexpected requirements to be met with little or no reengineering, and allow its more skilled customers to rapidly address novel challenges.” - Brian Foote and Joseph Yoder – “Big Ball of Mud”
  • 17.
    New development mindset Instead of planning for software development, plan for software evolution.  Instead of designing a single system, design a family of systems.
  • 18.
    Family of Systems Software Product Line: “A set of software-intensive systems that share a common, managed set of features satisfying the specific needs of a particular market segment or mission and that are developed from a common set of core assets in a prescribed way.” – Software Engineering Institute
  • 19.
    Adaptable Design  AdaptableSoftware Design: A generic software design for a family of systems which does not need to be changed to accommodate new requirements.  ADUF = Adaptable Design Up Front!
  • 20.
    Open/Closed Principle  “Softwareentities (classes, modules, functions, etc.) should be open for extension, but closed for modification.” - Bertrand Meyer
  • 21.
  • 22.
    Adaptable Design UpFront and the Open/Closed Principle  ADUF focuses on the early definition of the “closed” aspects of the design while at the same time allows easy evolution by making this design open to extensions.  Closed: Captures essential entities and relationships.  Open: Provides mechanisms for extensibility.
  • 23.
  • 24.
    Common Errors  Toomuch design up front  Not enough design up front  Too much openness  Not enough openness
  • 25.
    Too much designup front  Capturing aspects that are not essential as if they were such.  The design may need to change (thus it is not really “closed”).  Examples: ◦ Specific details are not generalized. ◦ Designing for a specific system instead of designing for a family of systems.
  • 26.
    Not enough designup front  Leaving away some of the essential aspects of the system.  These aspects will need to be added to the design when other parts of the system are already implemented.  Example: ◦ Adding relationships to new entities may cause changes to existing interfaces.
  • 27.
    Too much openness The design is over engineered, resulting in complexity not really necessary.  Example: ◦ It is always possible to reduce coupling through additional layers of indirection. ◦ With many layers, it is difficult to locate where the business logic is implemented.
  • 28.
    Not enough openness The design captures all essential aspects, but there are not mechanisms that allow this design to be easily extended.  Example: ◦ If there are Interfaces for the main entities, there should also be Factories to instantiate concrete subclasses.
  • 29.
    Extensible Designs Main alternativesfor extensible designs:  Frameworks ◦ Plug-ins  Platforms
  • 30.
    Frameworks  Framework: “Asoftware framework is an abstraction in which software providing generic functionality can be selectively changed by additional user-written code, thus providing application-specific software.”
  • 31.
    Plug-ins  Plug-in: “Aplug-in is a software component that adds a specific feature to an existing software application. When an application supports plug-ins, it enables customization.”
  • 32.
  • 33.
    Versioning  Traditional: ◦ Version= modifications + additions. ◦ After n iterations: n versions.  Component-based: ◦ Version = combination of new implementations of components. ◦ System with m components, after n iterations: nm versions.
  • 34.
    Platforms  Platform: “Technologythat enables the creation of products and processes that support present or future development.”  Software platforms provide services that are used by several applications.  Platforms and applications may evolve independently of each other.
  • 35.
    Question 1  Howdo you identify the components in your framework or the services in your platform?
  • 36.
    Domain Modeling  “Adomain model is a conceptual model of all the topics related to a specific problem. It describes the various entities, their attributes, roles, and relationships, plus the constraints that govern the problem domain.”
  • 37.
    Question 2  Howdo you decouple the components in your framework?
  • 38.
    Design Patterns  Reducecoupling using Design Patterns.  Most patterns avoid direct links between concrete classes, using interfaces and abstract classes.  Example: Use Observer when several types of objects must react to the same event.
  • 39.
  • 40.
    Recommender System Model ModelRecommender Trainer Historic Data
  • 41.
    Recommender System Framework After 1year of development:  Historic Data: 13 subclasses.  Model: 9 subclasses.  Trainer: 9 subclasses.  Recommender: 19 subclasses.
  • 42.
    What about YAGNI? YAGNI = You aren't gonna need it.  A principle of extreme programming (XP) that states that a programmer should not add functionality until deemed necessary.  “Always implement things when you actually need them, never when you just foresee that you need them.” - Ron Jeffries
  • 43.
    YAGNI vs. ADUF YAGNI tells us to avoid doing what we are not sure we will need.  ADUF tells us: ◦ Define the things you are sure you will need in any case. ◦ Prepare for the things that you will probably need in the future (adaptability). ◦ NIAGNI: “No, I am gonna need it!”
  • 44.
    Conclusions  Software systemsmust evolve over time.  This evolution should be planned and supported trough Adaptable Software Designs.  Domain Modeling: Identify the entities and relationships that form the “closed” part of the design.  Framework Design: Design a framework that represents the model and add mechanisms for extensibility, making it “open”.
  • 45.
    Related Work  SoftwareEngineering Institute: ◦ Software Product Lines  Alistair Cockburn: ◦ Walking Skeleton ◦ Incremental Rearchitecture  Neal Ford: ◦ Emergent Design ◦ Evolutionary Architecture  Simon Brown: ◦ Just Enough Design Up Front
  • 46.
    Adaptability & Evolution “Itis not the strongest of the species that survives, nor the most intelligent that survives. It is the one that is the most adaptable to change.” Charles Darwin
  • 47.
    Part II: Manifestofor Adaptability  Lean Startup  Minimum Viable Product  Manifesto for Adaptable Software Development  SOA Principles  Microservices Practices
  • 48.
  • 49.
  • 50.
  • 51.
    Minimum Viable Product (MVP) TheMVP is the “version of a new product which allows a team to collect the maximum amount of validated learning about customers with the least effort.” The MVP should have the core features which allow it to be deployed to real customers to get initial feedback, but not more.
  • 52.
  • 53.
    Manifesto for Agile SoftwareDevelopment Individuals and interactions over processes and tools Working software over comprehensive documentation Customer collaboration over contract negotiation Responding to change over following a plan
  • 54.
    Manifesto for Adaptable Software Development Experimentationinstead of Specification Evolution instead of Implementation Adaptation instead of Modification Extension instead of Growth
  • 55.
    Experimentation instead of Specification Traditional software development: ◦ Detailed specification of system requirements, functional and non- functional.  Modern software development: ◦ Allows the discovery of requirements through experimentation.
  • 56.
    Evolution instead of Implementation Traditional software development: ◦ Focuses on the prioritized implementation of features according to specified requirements.  Modern software development: ◦ Allows the constant evolution of the system’s functionality to meet the dynamically varying customer needs.
  • 57.
    Adaptation instead of Modification Traditional software development: ◦ The code must be modified in order to meet changing requirements.  Modern software development: ◦ Allows the existing code base to be easily adapted to satisfy any change in requirements.
  • 58.
    Extension instead ofGrowth  Traditional software development: ◦ The system must grow in order to accommodate additional requirements.  Modern software development: ◦ Allows the easy extension of the system’s functionality through modular features.
  • 59.
    Architectural needs  TheSoftware Architecture must support: ◦ Experimentation ◦ Evolution ◦ Adaptation ◦ Extension  If the Software Architecture is not adaptable, the Agile process will fail.  Consequences: Increasing Technical Debt, endless Refactoring.
  • 60.
    SOA Principles 1. StandardizedService Contracts 2. Service Loose Coupling 3. Service Abstraction 4. Service Reusability 5. Service Autonomy 6. Service Statelessness 7. Service Discoverability 8. Service Composability 9. Service Interoperability
  • 61.
    Microservices Practices (I) 1.Separated Build: ◦ Each Microservice is built independently and has its own version. 2. Independent Deployment: ◦ Each Microservice may be deployed without affecting the others. 3. Separated Data Stores: ◦ Microservices should not share data stores. 4. Independence of Platform: ◦ Microservices should be deployed in containers.
  • 62.
    Microservices Practices (II) 5.Individual Technology Choice: ◦ Each Microservice may be implemented using a different technology. 6. Confinement of Failure: ◦ If a failure occurs in a particular Microservice it should not propagate to the other ones. 7. Independent Scalability: ◦ It should be possible to scale each Microservice independently of the others.
  • 63.
    In Summary  AdaptableSoftware Development: ◦ Experimentation instead of Specification ◦ Evolution instead of Implementation ◦ Adaptation instead of Modification ◦ Extension instead of Growth
  • 64.