Statistical Debugging of Programs written in Dynamic Programming Language - RUBYMaster Thesis Software Engineering Thesis no: MSE-2010-33 December 2010 AdeelAkhterHassan Azhar
AgendaIntroduction
 Research Questions
 Research Design
 Results
 Contribution
 Future WorkIntroductionSoftware Debugging“Debugging is the activity of identification of actions and events that causes for failure of mid level crash down of normal execution of program before ending to final stage [1]”Debugging Process consist of Detecting failure Localizing fault Correcting fault
Introduction Cont…Statistical Debugging
The Statistical debugging technique sketch out the program’s source code to gather data flow, program flows, program state trace information and then perform the statistical analysis on collected information to figure out the bug traces.
Instrumentation is a process of predicates insertion with in the source code.
Analysis of instrumented code will provide the visualization of  buggy areas with in the source code.Introduction Cont…Statistical Debugging
Goal of Statistical Debugging
Identify distinct bugs in source code.
Ranking the predicate that best predicts the bug.Introduction Cont...Problem Description
Adapting  existing statistical debugging techniques adaptable for Dynamic Programming Languages.
Evaluating  statistical debugging algorithm SOBER and Cause Isolation Scheme for programs coded in RUBY language.
Literature lacks in providing the evaluation process of statistical debugging and fault localization for programs coded in RUBY language.
No empirically validated process for utilizing both testing approaches in combinationIntroduction cont...AimThe aim of this study is to investigate existing statistical debugging techniques for programs coded in modern dynamic programming languages like Ruby.
Introduction cont...Objectives
Identify and evaluate the existing statistical debugging algorithms for dynamic programming languages.
To implement the most mature statistical debugging algorithm and adapt them to the Ruby language.
Evaluation and analysis of SOBER and Cause Isolation Scheme algorithms in RUBY language programs.
Comparative validation of SOBER and Cause Isolation Scheme debugging algorithm on programs developed in RUBY language.Research Questions	RQ1:   What are the existing statistical debugging techniques                   	    adaptable for Dynamic programming languages?	RQ2    How many existing techniques for statistical debugging                  are adapted to work in the context of various dynami                programming languages?	RQ3How to adapt statistical debugging algorithm SOBER               and Cause Isolation Scheme in RUBY?	RQ4  How will SOBER and Cause Isolation Scheme perform              when these will be applied on sample programs                               developed in RUBY?
Research DesignLiterature ReviewQualitativestudy of DebuggingAlgorithmsPhase 1Observations from previous studiesDesign statisticaldebugging process for programs developed in RUBY languagePhase 2Implementation of debugging process in RUBY programsResultsanalysisusing SOBER and Cause Isolation SchemeAlgorithmsProcess validationPhase 3QuantitativeAnalysisPhase 4Analysis, results and conclusion
Objectives & Research MethodologyObjective 1LiteraturereviewProposed process of statisticaldebugging for programs in RUBYObjective 2Objective 3Implementation, Analysis & validation of proposed processObservations & conclusionObjective 4Resultsanalysisusing SOBER and Cause Isolation SchemeAlgorithms
System RequirementsSoftware RequirementsAnalysisVerificationDesignTesting during executionCodingStatic TestingTestingProduction / DeploymentDebuggingSoftware debugging, testing and verification
Existing Techniques of Statistical Debugging
 Process designPrograms will be implemented with standard structures e.g. Loops, conditional statements, method calls and object oriented structures.Programs are then instrumented at the critical areas of source code where bugs or failures are most likely to occurSampling process is next to instrumentation activity. It involves the implementation of predicates in the form of specialized checks at the specified instrumented sites.Next to sampling, ranking of predicates will be performed by using Tarantula fault localization technique. Predicate ranking will be performed by looking at the successful completion of iteration with in specified range.The output generated from the sampled program will be analyzed by using SOBER and Cause Isolation Scheme algorithms. Predicates results will be normalized to visualize the high and low critical bugs within the inspected source code.
Experiment ExecutionSubject selectionWe have selected the source codes related to different scenarios including the implementations of control structures, string manipulation, iteration structures, file streaming process, object initialization, exception processing.Object Studied
Identification of bug localities
Impact of bugs on normal execution of source code.

Statistical debuging for programs written in dynamic programming language ruby

  • 1.
    Statistical Debugging ofPrograms written in Dynamic Programming Language - RUBYMaster Thesis Software Engineering Thesis no: MSE-2010-33 December 2010 AdeelAkhterHassan Azhar
  • 2.
  • 3.
  • 4.
  • 5.
  • 6.
  • 7.
    Future WorkIntroductionSoftwareDebugging“Debugging is the activity of identification of actions and events that causes for failure of mid level crash down of normal execution of program before ending to final stage [1]”Debugging Process consist of Detecting failure Localizing fault Correcting fault
  • 8.
  • 9.
    The Statistical debuggingtechnique sketch out the program’s source code to gather data flow, program flows, program state trace information and then perform the statistical analysis on collected information to figure out the bug traces.
  • 10.
    Instrumentation is aprocess of predicates insertion with in the source code.
  • 11.
    Analysis of instrumentedcode will provide the visualization of buggy areas with in the source code.Introduction Cont…Statistical Debugging
  • 12.
  • 13.
    Identify distinct bugsin source code.
  • 14.
    Ranking the predicatethat best predicts the bug.Introduction Cont...Problem Description
  • 15.
    Adapting existingstatistical debugging techniques adaptable for Dynamic Programming Languages.
  • 16.
    Evaluating statisticaldebugging algorithm SOBER and Cause Isolation Scheme for programs coded in RUBY language.
  • 17.
    Literature lacks inproviding the evaluation process of statistical debugging and fault localization for programs coded in RUBY language.
  • 18.
    No empirically validatedprocess for utilizing both testing approaches in combinationIntroduction cont...AimThe aim of this study is to investigate existing statistical debugging techniques for programs coded in modern dynamic programming languages like Ruby.
  • 19.
  • 20.
    Identify and evaluatethe existing statistical debugging algorithms for dynamic programming languages.
  • 21.
    To implement themost mature statistical debugging algorithm and adapt them to the Ruby language.
  • 22.
    Evaluation and analysisof SOBER and Cause Isolation Scheme algorithms in RUBY language programs.
  • 23.
    Comparative validation ofSOBER and Cause Isolation Scheme debugging algorithm on programs developed in RUBY language.Research Questions RQ1: What are the existing statistical debugging techniques adaptable for Dynamic programming languages? RQ2 How many existing techniques for statistical debugging are adapted to work in the context of various dynami programming languages? RQ3How to adapt statistical debugging algorithm SOBER and Cause Isolation Scheme in RUBY? RQ4 How will SOBER and Cause Isolation Scheme perform when these will be applied on sample programs developed in RUBY?
  • 24.
    Research DesignLiterature ReviewQualitativestudyof DebuggingAlgorithmsPhase 1Observations from previous studiesDesign statisticaldebugging process for programs developed in RUBY languagePhase 2Implementation of debugging process in RUBY programsResultsanalysisusing SOBER and Cause Isolation SchemeAlgorithmsProcess validationPhase 3QuantitativeAnalysisPhase 4Analysis, results and conclusion
  • 25.
    Objectives & ResearchMethodologyObjective 1LiteraturereviewProposed process of statisticaldebugging for programs in RUBYObjective 2Objective 3Implementation, Analysis & validation of proposed processObservations & conclusionObjective 4Resultsanalysisusing SOBER and Cause Isolation SchemeAlgorithms
  • 26.
    System RequirementsSoftware RequirementsAnalysisVerificationDesignTestingduring executionCodingStatic TestingTestingProduction / DeploymentDebuggingSoftware debugging, testing and verification
  • 27.
    Existing Techniques ofStatistical Debugging
  • 28.
    Process designProgramswill be implemented with standard structures e.g. Loops, conditional statements, method calls and object oriented structures.Programs are then instrumented at the critical areas of source code where bugs or failures are most likely to occurSampling process is next to instrumentation activity. It involves the implementation of predicates in the form of specialized checks at the specified instrumented sites.Next to sampling, ranking of predicates will be performed by using Tarantula fault localization technique. Predicate ranking will be performed by looking at the successful completion of iteration with in specified range.The output generated from the sampled program will be analyzed by using SOBER and Cause Isolation Scheme algorithms. Predicates results will be normalized to visualize the high and low critical bugs within the inspected source code.
  • 29.
    Experiment ExecutionSubject selectionWehave selected the source codes related to different scenarios including the implementations of control structures, string manipulation, iteration structures, file streaming process, object initialization, exception processing.Object Studied
  • 30.
  • 31.
    Impact of bugson normal execution of source code.
  • 32.
    Code coverage relatedto identified bugs.
  • 33.
    Predicates results observationtemplate Experiment Execution Cont...Test Candidate Programs
  • 34.
    ResultsResult analysis wasperformed based on observation of following attributes for instrumented predicates:
  • 35.
  • 36.
    Results Single- Bug Version Cont…Mean Median
  • 37.
    Results Single- Bug Version Cont…Triangle
  • 38.
    Results Single- Bug Version Cont…Create_file
  • 39.
    Results Single - Bug Version Cont…Generate_Product_Table
  • 40.
    Results Multi- Bug VersionEvaluate_Polynomial
  • 41.
    Results Multi- Bug Version Cont…Evaluate_Polynomial
  • 42.
    Comparative validation ofSOBER & Cause Isolation Scheme
  • 43.
    Comparative validation Cont... Located Bugs w.r.t Examined code SOBBER - Cause Isolation Scheme debugging of Evaluate_Polynomial.rb
  • 44.
    Comparative validation Cont…Located Bugs w.r.t Examined codeSOBER - LIBLIT debugging of JRuby_Calculator .rb
  • 45.
    Comparative validation Cont…LocatedBugs w.r.t Examined codeSOBER - LIBLIT debugging of JRuby_Calculator .rb
  • 46.
    ContributionIdentification of mostmature existing statistical debugging techniques
  • 47.
    Commencement of statistical debugging algorithms for program developed in Ruby programming language.
  • 48.
    Introducing a dynamicpredicate evaluation process for programs coded in RUBY programming Language.Future WorkValidation of proposed process for complex application developed in RUBY & RUBY ON RAILS.
  • 49.
    Development of statisticaldebugging tool that can highlight the bug localities and also perform the statistical analysis of dynamically generated predicates.Questions???