A Framework for Classifying and Comparing
Architecture-Centric Software Evolution Research

        Pooyan Jamshidi, Mohammad Ghafari,
             Aakash Ahmad, Claus Pahl
     Lero - The Irish Software Engineering Research Centre,
           School of Computing, Dublin City University
Architecture-Centric Software Evolution
 Software Architecture represents the blue-print of a software as
  topological configuration of computational components and connectors that
  enable component-level interconnections. [ISO/IEC/IEEE 42010 Standard]

 Software Evolution adapts a software to changing requirements and
  operating environment by means of addition, removal and modification of
  software artefacts. [ACM/IEEE Software Engineering Curriculum, 2004]

- Evolution @ Design-time


- Evolution @ Run-time




                                                                          2
Traditional Architecture-Centric Software Evolution




                                                      3
A Paradigm Change (I): models@runtime




                                        4
A Paradigm Change (II): executable models
             Adaptation


                System=Executable Model



                  Run-Time Environment


Executing the model makes sense
An engine is responsible for the execution
An interpreted DSL is needed
                                              5
The Conceptual Framework




                           6
Motivations for this Study
Wide spectrum of approaches
Different paradigms and frameworks
Different problem and solution spaces
Different interpretation of concepts
No systematic insight into state-of-the-art
    Such insight is crucial for practitioners and researchers
To that end, we performed an SLR
    Taxonomic Scheme
    Classification and Comparison



                                                                 7
Agenda
•   Research Methodology
•   Research Questions
•   Search Strategy & Quality Assessment
•   Data Items
•   Results
•   Limitations




                                           8
Systematic Literature Review (Abstract)




                                      9
Research Methodology




                       10
Research Methodology




                       11
Project Scope
• 60 papers selected after exclusion process
• Protocol defined by 4 researchers
• Papers reviewed by 2 researchers
• Data analysis with 4 researchers
• ~30 discussion sessions to get an agreement on
  data extraction
• Duration of the study ~ 9 months
• Special thanks to Excel and Dropbox!!!


                                                   12
Research Questions
RQ1: What types of evolution are supported in
 ACSE?
RQ2: Which formalisms are required to enable an
 ACSE approach?
RQ3: How architectural reasoning is adopted in
 ACSE?
RQ4: Which execution environments and mechanisms
 are needed to enable run-time evolution?
RQ5: What tool supports is available for ACSE?
                                               13
Search Strings and Retrieved Papers

                       No     Name       Results
                        1      ACM         663
                        2      IEEE       1149
                        3 Science Direct   194
                        4 Web Of Science   480
                        5  Springer Link   445
                        6   Pro-Quest      231
                        7 Google Scholar   460
                        8      Wiley       516
                               Total      4138




                                                   14
Quality Assessment of Studies
General
    1.   Problem definition of the study (2)
    2.   Environment in which the study was carried out (1)
    3.   Research design of the study refers to the way the study was organized (2)
    4.   Contributions of the study refer to the study results (2)
    5.   Insights derived from the study (2)
    6.   Limitations of the study (2)

Specific
    1.   Main focus of the paper (2)
    2.   The architecture descriptions presented in the studies (2)
    3.   Architectural properties in terms of constraints (2)
    4.   Evaluation of the study (2)
                        G1+G2+G3+G4+G5+G6         (S1+S2+S3+S4)
 Ranking formula=                            +                   ∗3
                                11                      8
 4= Quality papers; 3: Acceptable; 2, 1, 0: Excluded                            15
Number of Included Studies per Year
12


10


8


6


4


2


0



                                   16
Taxonomical
                             Collected Data Items
                              Sub-
                                                                                     Coded Attributes
  Classification         classifications                                     (Domain Knowledge, Standards, Keywords)
(Higher-Order Theme)       (Sub-themes)
                       Need for Evolution      ISO/IEC 14764 typology of change: Corrective, Perfective, Adaptive, Preventive, All applicable
                                               Static: Transformation, Refactoring, Refinement, Restructuring, Pattern change; Dynamic:
                       Means of Evolution
                                               Reconfiguration, Adaptation
    Type of
                       Time of Evolution       Design-Time, Run-Time
   Evolution                                   Change impact: Consistency checking, Impact analysis, Propagation; Change history: Evolution
                       Support Activity
                                               analysis, Versioning
                       Stage of Evolution      SDLC: Analysis/Design, Implementation, Integration/provisioning, Deployment, Evolution
                       Level of Formalism      Lightweight, Formal
                                               Modeling language: ADL, Programming lang., Domain-specific lang., Type systems, Archface,
                                               Model-based; Formal models: Graph theory, Petri-net, Ontology, State machine, Constraint
                       Type of Formalism
                                               automata, CHAM; Process algebra: FSP, CSP, π-calculus; Logic (Constraint language): OCL, CCL,
   Type of                                     FOL, Grammars, Temporal logics, Rules, Description logic, Z, Alloy, Larch
 Specification                                 Process algebra: Darwin, Wright, LEDA, PiLar; Standards: UML, Ex.-UML, SysML, AADL; Others:
                       Description Language
                                               ACME, Aesop, C2, MetaH, Rapide, SADL, UniCon, Weaves, Koala, xADL, ADML, AO-ADL, xAcme
                       UML Specification       Static: Class, Component, Object; Dynamic: Activity, State, Sequence, Transition, Communication
                       Description Aspect      Structural, Behavioral, Semantic
                                               Structural: Pattern, Architectural style, Primitive, Metaphore, Micro-structure, Crosscutting
                       Architectural           concern, Clue; Invariant: Component-level invariant, Cross-component invariant, Pre-/Post-
    Type of            Constraint              condition, Temporal invariant; Relational constraints: Coding rules, Cardinality, Coordination,
   Reasoning                                   Meta-model, Variability
                       Intent of Reasoning     Specify, Preserve, Change, Enforce, Match, Discover, Analyze
                       Type of Analysis        Consistency checking, Model checking, Pattern conformance, Graph refinement, Enforcement
                                               Component model: Fractal, KOALA, SOFA 2.0, CCM, OpenCOM, KobrA, Middleware: SIENA, OSGi,
                       Runtime Environment
   Run-Time                                    .NET, MS-COM, EJB, JavaBeans, SOA middleware, Coordination model: Reo, Linda, MANIFOLD
     Issue             Mechanism of            Reflection mechanisms: Introspection, Constraint injection, Intercession, Reification, Causal
                       Runtime Evolution       connection, Evolution enablers: Safe stopping, Runtime binding, State transfer
                       Need for Tool Support   Architecture life-cycle: Business case, Creating architecture, Documenting, Analyzing, Evolving
      Tool             Analysis                Simulation, Dependence analysis, Model checking, Conformance testing, Interface consistency,
    Support            Usage of Tool Support   Inspection and review-based
                       Level of Automation     Fully automated, Partially automated, Manual
                                               A particular problem/challenge, Overview or Survey, Formalism for constraint specification,
                       Research Motivation
                                               Formalism for architectural analysis, Formalism for arch. evolution, Formalism for code generation
    Research                                   Development paradigm: SPL, OO, SOA, CBS; Traditional: Embedded, Real-time, Process-aware,
    Method             Application Domain      Distributed, Event-based, Concurrent, Mechatronic, Mobile, Robotic, Grid computing; Emerging:
                                               Cloud computing, Smart-*, Autonomic computing, *-critical, Ubiquitous                         17
                       Evaluation Method       Case study, Mathematical proof, Example application, Industrial validation
Results, RQ1: “types of evolution”
1. Need for Evolution
2. Means of Evolution
3. Time of Evolution
4. Support Activity
5. Stage of Evolution




                                         18
Results, RQ1: types of evolution (I)

                        Need for Evolution

   All applicable

     Preventive

       Adaptive

      Perfective

      Corrective

                    0        5      10       15   20



                                                       19
Results, RQ1: types of evolution (II)
                        Means of Evolution

Change operation
   Arch. decision
  Pattern-based
 Reconfiguration
      Adaptation
   Restructuring
     Refinement
     Refactoring
  Transformation

                    0       5          10    15   20
                                                       20
Results, RQ1: types of evolution (III)

                      Time of Evolution



      Run-Time




    Design-Time



                  0     10     20     30   40   50




                                                     21
Results, RQ1: types of evolution (IV)

                       Support Activity

              Versioning


     Change propagation


       Evolution analysis


    Consistency checking

                            0   2         4   6   8




                                                      22
Results, RQ1: types of evolution (V)

                     Stage of Evolution


         Maintenance



      Implementation



   Analysis and Design


                         0    10     20   30   40




                                                    23
Results, RQ2: “formalisms”
1.   Level of Formalism
2.   Type of Formalism
3.   Description Language
4.   UML Specification
5.   Description Aspect




                                       24
Results, RQ2: formalisms (I)

                  Level of Formalism


    Formal




Lightweight



              0       10      20       30   40




                                                 25
Results, RQ2: formalisms (II)
                        Type of Formalism
        Archface
Temporal logics
Process algebra
    C. Automata
                Z
  State machine
             CCL
             OCL
            Alloy
 Prog. language
       π-calculus
Description logic
   Model-based
        Petri-net
           Graph

                    0      5         10     15   20   26
Results, RQ2: formalisms (II)

                Description Language
    SafArchie
      Darwin
     AO-ADL
Extended UML
        UML
      xAcme
       ACME

                0      5      10       15   20




                                                 27
Results, RQ2: formalisms (III)

                 UML Specification
    Transition
Communication
       Object
   Component
        Class
    Sequence
        State
      Activity

                 0     5      10     15   20




                                               28
Results, RQ2: formalisms (IV)

                  Description Aspect

 Semantic



Behavioral



 Structural


              0    10   20   30    40   50   60




                                                  29
Results, RQ3: “architectural reasoning”
1. Architectural Constraint
2. Intent of Reasoning
3. Type of Analysis




                                      30
Results, RQ3: architectural reasoning (I)
                      Architectural Constraint
              Coding rules
    Coordination constraint
        Pre-/Post-condition
               Meta-model
      Crosscutting concern
 Cross-component invariant
 Component-level invariant
                  Primitive
        Architectural style
                    Pattern

                              0   5    10        15   20   25
                                                                31
Results, RQ3: architectural reasoning (II)

                      Intent of Reasoning
       Analyze
       Discover
         Match
       Enforce
       Change
      Preserve
        Specify

                  0       10     20         30   40




                                                      32
Results, RQ3: architectural reasoning (III)

                       Type of Analysis

       Constraint enforcement

       Graph-based refinement

          Pattern conformance

               Model checking

         Consistency checking

                                0   5     10   15




                                                    33
Results, RQ4: “execution environments”
1. Environment of Runtime Evolution
2. Mean of Runtime Evolution




                                      34
Results, RQ4: execution environments (I)


    Environment of Runtime                 Mean of Runtime
           Evolution                          Evolution
         SIENA                          Runtime
                                        binding
SOA middleware
        PKUAS                           Causal
                                      connection
           CCM
          OSGi                         Reflection
        Fractal
                                                    0   2    4
                  0   0.5   1   1.5




                                                             35
Results, RQ5: “tool supports”
1. Need for Tool Support
2. Analysis Usage of Tool Support
3. Level of Automation




                                      36
Results, RQ5: tool supports (I)

                Need for Tool Support

     Evolving


    Analyzing


 Documenting


     Creating

                0      10     20        30   40




                                                  37
Results, RQ5: tool supports (II)

        Analysis Usage of Tool Support

         Inspection

       Conformance

      Model checking

 Dependence analysis

          Simulation

                       0   2   4   6   8   10   12




                                                     38
Results, RQ5: tool supports (III)

                Level of Automation

 Manual



  Partial



    Full


            0     10       20         30   40




                                                39
Results: Research method
1. Application Domain
2. Evaluation Method




                                   40
Results: research method (I)

               Application Domain

              CBD

Event-based system

 Distributed system

               OO

              SOA

                      0   10   20   30   40



                                              41
Results: research method (II)

               Evaluation Method

Example application



Mathematical proof



        Case study


                      0   5   10   15   20   25   30



                                                       42
Main Results: Summary
Recent inclination towards
  Runtime adaptation
  Self-adaptive applications
  Probabilistic modeling
  Quantitative verification
“Reflective environments” and “Models @ run-
 time” as the key drivers to enable adaptations
Lack of evidences for rigorous validation
Recent evidences of machine learning approaches
 in ACSE
                                              43
Study Limitations
Bias of the reviewers
  Search strategies
  Research protocol
Reliability
  Consensus among researchers
  Scheme
  Pilot study
  Open data
Results are valid, though not conclusive!!
                                              44
Thanks, more details? Check out the link below




           http://tiny.cc/hcg8sw             45

A Framework for Classifying and Comparing Architecture-Centric Software Evolution Research

  • 1.
    A Framework forClassifying and Comparing Architecture-Centric Software Evolution Research Pooyan Jamshidi, Mohammad Ghafari, Aakash Ahmad, Claus Pahl Lero - The Irish Software Engineering Research Centre, School of Computing, Dublin City University
  • 2.
    Architecture-Centric Software Evolution Software Architecture represents the blue-print of a software as topological configuration of computational components and connectors that enable component-level interconnections. [ISO/IEC/IEEE 42010 Standard]  Software Evolution adapts a software to changing requirements and operating environment by means of addition, removal and modification of software artefacts. [ACM/IEEE Software Engineering Curriculum, 2004] - Evolution @ Design-time - Evolution @ Run-time 2
  • 3.
  • 4.
    A Paradigm Change(I): models@runtime 4
  • 5.
    A Paradigm Change(II): executable models Adaptation System=Executable Model Run-Time Environment Executing the model makes sense An engine is responsible for the execution An interpreted DSL is needed 5
  • 6.
  • 7.
    Motivations for thisStudy Wide spectrum of approaches Different paradigms and frameworks Different problem and solution spaces Different interpretation of concepts No systematic insight into state-of-the-art  Such insight is crucial for practitioners and researchers To that end, we performed an SLR  Taxonomic Scheme  Classification and Comparison 7
  • 8.
    Agenda • Research Methodology • Research Questions • Search Strategy & Quality Assessment • Data Items • Results • Limitations 8
  • 9.
  • 10.
  • 11.
  • 12.
    Project Scope • 60papers selected after exclusion process • Protocol defined by 4 researchers • Papers reviewed by 2 researchers • Data analysis with 4 researchers • ~30 discussion sessions to get an agreement on data extraction • Duration of the study ~ 9 months • Special thanks to Excel and Dropbox!!! 12
  • 13.
    Research Questions RQ1: Whattypes of evolution are supported in ACSE? RQ2: Which formalisms are required to enable an ACSE approach? RQ3: How architectural reasoning is adopted in ACSE? RQ4: Which execution environments and mechanisms are needed to enable run-time evolution? RQ5: What tool supports is available for ACSE? 13
  • 14.
    Search Strings andRetrieved Papers No Name Results 1 ACM 663 2 IEEE 1149 3 Science Direct 194 4 Web Of Science 480 5 Springer Link 445 6 Pro-Quest 231 7 Google Scholar 460 8 Wiley 516 Total 4138 14
  • 15.
    Quality Assessment ofStudies General 1. Problem definition of the study (2) 2. Environment in which the study was carried out (1) 3. Research design of the study refers to the way the study was organized (2) 4. Contributions of the study refer to the study results (2) 5. Insights derived from the study (2) 6. Limitations of the study (2) Specific 1. Main focus of the paper (2) 2. The architecture descriptions presented in the studies (2) 3. Architectural properties in terms of constraints (2) 4. Evaluation of the study (2) G1+G2+G3+G4+G5+G6 (S1+S2+S3+S4)  Ranking formula= + ∗3 11 8  4= Quality papers; 3: Acceptable; 2, 1, 0: Excluded 15
  • 16.
    Number of IncludedStudies per Year 12 10 8 6 4 2 0 16
  • 17.
    Taxonomical Collected Data Items Sub- Coded Attributes Classification classifications (Domain Knowledge, Standards, Keywords) (Higher-Order Theme) (Sub-themes) Need for Evolution ISO/IEC 14764 typology of change: Corrective, Perfective, Adaptive, Preventive, All applicable Static: Transformation, Refactoring, Refinement, Restructuring, Pattern change; Dynamic: Means of Evolution Reconfiguration, Adaptation Type of Time of Evolution Design-Time, Run-Time Evolution Change impact: Consistency checking, Impact analysis, Propagation; Change history: Evolution Support Activity analysis, Versioning Stage of Evolution SDLC: Analysis/Design, Implementation, Integration/provisioning, Deployment, Evolution Level of Formalism Lightweight, Formal Modeling language: ADL, Programming lang., Domain-specific lang., Type systems, Archface, Model-based; Formal models: Graph theory, Petri-net, Ontology, State machine, Constraint Type of Formalism automata, CHAM; Process algebra: FSP, CSP, π-calculus; Logic (Constraint language): OCL, CCL, Type of FOL, Grammars, Temporal logics, Rules, Description logic, Z, Alloy, Larch Specification Process algebra: Darwin, Wright, LEDA, PiLar; Standards: UML, Ex.-UML, SysML, AADL; Others: Description Language ACME, Aesop, C2, MetaH, Rapide, SADL, UniCon, Weaves, Koala, xADL, ADML, AO-ADL, xAcme UML Specification Static: Class, Component, Object; Dynamic: Activity, State, Sequence, Transition, Communication Description Aspect Structural, Behavioral, Semantic Structural: Pattern, Architectural style, Primitive, Metaphore, Micro-structure, Crosscutting Architectural concern, Clue; Invariant: Component-level invariant, Cross-component invariant, Pre-/Post- Type of Constraint condition, Temporal invariant; Relational constraints: Coding rules, Cardinality, Coordination, Reasoning Meta-model, Variability Intent of Reasoning Specify, Preserve, Change, Enforce, Match, Discover, Analyze Type of Analysis Consistency checking, Model checking, Pattern conformance, Graph refinement, Enforcement Component model: Fractal, KOALA, SOFA 2.0, CCM, OpenCOM, KobrA, Middleware: SIENA, OSGi, Runtime Environment Run-Time .NET, MS-COM, EJB, JavaBeans, SOA middleware, Coordination model: Reo, Linda, MANIFOLD Issue Mechanism of Reflection mechanisms: Introspection, Constraint injection, Intercession, Reification, Causal Runtime Evolution connection, Evolution enablers: Safe stopping, Runtime binding, State transfer Need for Tool Support Architecture life-cycle: Business case, Creating architecture, Documenting, Analyzing, Evolving Tool Analysis Simulation, Dependence analysis, Model checking, Conformance testing, Interface consistency, Support Usage of Tool Support Inspection and review-based Level of Automation Fully automated, Partially automated, Manual A particular problem/challenge, Overview or Survey, Formalism for constraint specification, Research Motivation Formalism for architectural analysis, Formalism for arch. evolution, Formalism for code generation Research Development paradigm: SPL, OO, SOA, CBS; Traditional: Embedded, Real-time, Process-aware, Method Application Domain Distributed, Event-based, Concurrent, Mechatronic, Mobile, Robotic, Grid computing; Emerging: Cloud computing, Smart-*, Autonomic computing, *-critical, Ubiquitous 17 Evaluation Method Case study, Mathematical proof, Example application, Industrial validation
  • 18.
    Results, RQ1: “typesof evolution” 1. Need for Evolution 2. Means of Evolution 3. Time of Evolution 4. Support Activity 5. Stage of Evolution 18
  • 19.
    Results, RQ1: typesof evolution (I) Need for Evolution All applicable Preventive Adaptive Perfective Corrective 0 5 10 15 20 19
  • 20.
    Results, RQ1: typesof evolution (II) Means of Evolution Change operation Arch. decision Pattern-based Reconfiguration Adaptation Restructuring Refinement Refactoring Transformation 0 5 10 15 20 20
  • 21.
    Results, RQ1: typesof evolution (III) Time of Evolution Run-Time Design-Time 0 10 20 30 40 50 21
  • 22.
    Results, RQ1: typesof evolution (IV) Support Activity Versioning Change propagation Evolution analysis Consistency checking 0 2 4 6 8 22
  • 23.
    Results, RQ1: typesof evolution (V) Stage of Evolution Maintenance Implementation Analysis and Design 0 10 20 30 40 23
  • 24.
    Results, RQ2: “formalisms” 1. Level of Formalism 2. Type of Formalism 3. Description Language 4. UML Specification 5. Description Aspect 24
  • 25.
    Results, RQ2: formalisms(I) Level of Formalism Formal Lightweight 0 10 20 30 40 25
  • 26.
    Results, RQ2: formalisms(II) Type of Formalism Archface Temporal logics Process algebra C. Automata Z State machine CCL OCL Alloy Prog. language π-calculus Description logic Model-based Petri-net Graph 0 5 10 15 20 26
  • 27.
    Results, RQ2: formalisms(II) Description Language SafArchie Darwin AO-ADL Extended UML UML xAcme ACME 0 5 10 15 20 27
  • 28.
    Results, RQ2: formalisms(III) UML Specification Transition Communication Object Component Class Sequence State Activity 0 5 10 15 20 28
  • 29.
    Results, RQ2: formalisms(IV) Description Aspect Semantic Behavioral Structural 0 10 20 30 40 50 60 29
  • 30.
    Results, RQ3: “architecturalreasoning” 1. Architectural Constraint 2. Intent of Reasoning 3. Type of Analysis 30
  • 31.
    Results, RQ3: architecturalreasoning (I) Architectural Constraint Coding rules Coordination constraint Pre-/Post-condition Meta-model Crosscutting concern Cross-component invariant Component-level invariant Primitive Architectural style Pattern 0 5 10 15 20 25 31
  • 32.
    Results, RQ3: architecturalreasoning (II) Intent of Reasoning Analyze Discover Match Enforce Change Preserve Specify 0 10 20 30 40 32
  • 33.
    Results, RQ3: architecturalreasoning (III) Type of Analysis Constraint enforcement Graph-based refinement Pattern conformance Model checking Consistency checking 0 5 10 15 33
  • 34.
    Results, RQ4: “executionenvironments” 1. Environment of Runtime Evolution 2. Mean of Runtime Evolution 34
  • 35.
    Results, RQ4: executionenvironments (I) Environment of Runtime Mean of Runtime Evolution Evolution SIENA Runtime binding SOA middleware PKUAS Causal connection CCM OSGi Reflection Fractal 0 2 4 0 0.5 1 1.5 35
  • 36.
    Results, RQ5: “toolsupports” 1. Need for Tool Support 2. Analysis Usage of Tool Support 3. Level of Automation 36
  • 37.
    Results, RQ5: toolsupports (I) Need for Tool Support Evolving Analyzing Documenting Creating 0 10 20 30 40 37
  • 38.
    Results, RQ5: toolsupports (II) Analysis Usage of Tool Support Inspection Conformance Model checking Dependence analysis Simulation 0 2 4 6 8 10 12 38
  • 39.
    Results, RQ5: toolsupports (III) Level of Automation Manual Partial Full 0 10 20 30 40 39
  • 40.
    Results: Research method 1.Application Domain 2. Evaluation Method 40
  • 41.
    Results: research method(I) Application Domain CBD Event-based system Distributed system OO SOA 0 10 20 30 40 41
  • 42.
    Results: research method(II) Evaluation Method Example application Mathematical proof Case study 0 5 10 15 20 25 30 42
  • 43.
    Main Results: Summary Recentinclination towards Runtime adaptation Self-adaptive applications Probabilistic modeling Quantitative verification “Reflective environments” and “Models @ run- time” as the key drivers to enable adaptations Lack of evidences for rigorous validation Recent evidences of machine learning approaches in ACSE 43
  • 44.
    Study Limitations Bias ofthe reviewers Search strategies Research protocol Reliability Consensus among researchers Scheme Pilot study Open data Results are valid, though not conclusive!! 44
  • 45.
    Thanks, more details?Check out the link below http://tiny.cc/hcg8sw 45