SQA
Software Quality Assurance
S/W Quality
• Quality definition
• Quality requirements
• Quality Measures
▫ Mc Call’s Quality Factors
▫ ISO 9126 Quality Factors
Intro
• Planned and systematic method that
▫ Evaluate the quality of s/w products, standards, process &
procedures
• Ensure that development follow Standards and Procedures
• Done by
▫ Continuous Monitoring
▫ Product Evaluation
▫ Conducting Audits
Standards & Procedures
• Standards
▫ Established criteria to which product will be compared
• Procedures
▫ Established criteria to which development process & control
process are compared
• Hence,
▫ These both establish the method for developing s/w application
▫ Role of SQA to provide existence & adequacy of standards and
procedures
▫ SQA Activities
 Continuous Monitoring, Product Evaluation, Conducting Audits
Type of Standards
• Documentation
▫ Define the proper content for planning and control
▫ Provide consistency throughout development life of project
• Design
▫ Define the proper form and content of the design product
▫ Methods for translating the software requirements specification
into the actual software design
• Code
▫ Specify the programming language
▫ Also specify various constraints that should be put for usage of
the language features (i.e use of data structures, patterns etc.)
• Procedures must be followed in carrying out a development process
• All the development processes must have documented procedures
• Procedures like i.e.
▫ Configuration Management
▫ Non-conformance Reports
▫ Corrective Actions
▫ Testing & formal inspection
SQA Activities
• Product Evaluation & Process Monitoring are two important
activities
▫ Ensures development processes and the control processes written
in the management plan are carried out effectively
▫ Ensure that all the procedures and standards are correctly
followed
• Products are continuously monitored for checking that it is
following the standards and processes
• Product Evaluation
▫ Assures standards are being followed
▫ Ensures that the software application product is developed by
conforming all the applicable standards as illustrated in the
Management Plan
• Product Monitoring
▫ Ensures that the appropriate steps are carried out during the
development process
▫ Comparing the actual steps carried out with those in the
documented procedures
▫ Ensures that the Management Plan specifies the methods that
should be used by the monitoring activity
• Product Audits
▫ That looks the entire product and all the processes in depth
▫ Done by comparing them with the established standards and
procedures
▫ It’s an important activity to review the management plan,
technical processes and assurance processes to provide the actual
status of the software application product
▫ Main idea behind this is control procedures are properly followed
and desired documentation is properly maintained
• SQA Product
▫ Is nothing but an audit report to display findings and
recommendations to obey standards and procedures
S/W Review
• “filter” for software process
• Servers to uncover errors and defects that can then be removed
• “purify” s/w including
▫ Requirements
▫ Design models
▫ Code
▫ Testing data
• Primary objective is to find errors during the process so that do not
become defects after release of the s/w
Defect Amplification & Removal
Percentage
Efficiency for
error detection
Errors passed through
Amplified Errors 1: x
Newly Generated Errors
Defects Detection
Development Step
Errors from
Previous Step
Errors Passed
To next Step
Reviews: A Formality Spectrum
• Formality of review increase when,
▫ Distinct roles are defined for
reviewers
▫ Sufficient amount of planning
and preparation is available
▫ Distinct structure of review is
defined (task and time)
▫ Follow-up by the reviewers for
any correction that are made
REVIEW
Planning &
Preparation
Meeting
Structure
Correction &
Verification
Roles
Individual
Plays
Informal Review
• Simple desk check or casual meeting held for a review
• No advanced planning or preparation
• No agenda or meeting structure
• No follow-up on the errors
Effectiveness of such review is lower then more formal
approaches
• To improve the efficiency,
▫ Develop a set of simple review checklists for each major work
product
Formal Technical Review (FTR)
• S/Q QC activity performed by s/w engineers
• Objectives
▫ To uncover errors in function, logic or implementation of s/w
▫ To verify s/w review meets its requirements
▫ To ensure s/w represented as per followed Standards
▫ To achieve s/w that is developed in uniform manner
▫ To make project more manageable
• It serves as a training ground, enabling Juniors to observe different
approaches of s/w life cycle
• Serve to promote backup and Continuty
• Its class of review that contain,
▫ Walkthroughs and Inspections
• FTR is conducted as meeting and only be successful if it is well
planned, controlled and attended
• The Review Meeting
▫ 3-5 people should be involved in the review
▫ Advanced preparation should occurred (no more then 2 hours)
▫ Duration should be 2 hours for review meeting
• FTR focus on a specific part of the overall software
• Higher likelihood of uncovering the errors
• FTR focus on work product
• Producer (who has developed the work product)  informs
Project Leader about completion of their work & review should be
done
• Project Leader informs Review Leader
▫ Who evaluate product for its readiness, generate product material
& distribute them to the reviewers for advance preparation
▫ Create notes, reviewing product & establish agenda for review
meeting
• Presence of all above three characters required in a review meeting
• Reviewers become Recorders (who records all important issues
raise during the review)
• At the end of the review, FTR must decide whether
▫ Accept the product without further modification
▫ Reject the product due to several errors (once modified another
review must be performed)
▫ Accept the product provisionally (minor errors have been
encountered & must be corrected but no further review required)
• After final decision
▫ Attendees complete the sign off.
• Review Reporting & Report Keeping
▫ Recorder list out all the issues raised during meeting
▫ Summarized at the end of review and review issue list is
prepared
▫ A formal technical review summary report in completed
that answers,
 What was reviewed?
 Who reviewed it?
 What were the findings & conclusions?
(it is of one single page)
▫ Review issue list serves two purpose
 To identify problem areas within the product
 An action item check list that guides the producer as correction are
made
(normally attached to summary report)
Software Reliability
• “the probability of failure free operation of a computer program in a
specified environment for a specific time”
• Measure of reliability
▫ MTBF – (mean – time – between - failure)
MTBF = MTTF + MTTR
(MTTF - mean time to failure & MTTR - mean time to repair)
▫ Useful measure than other quality related software metrics
▫ Alternative measure of reliability is FIT (failure – in – time)
 A measure of how many failures a component will have over one
billion hours of operation
• Software Availability
▫ The probability that a program is operating according to
requirement at a given point in time
𝑀𝑇𝑇𝐹
𝑀𝑇𝑇𝐹 + 𝑀𝑇𝑇𝑅
𝑥 100%
• Software Safety
▫ Focus on identification and assessment of potential hazards that
may affect s/w negatively & cause an entire system to fail
Quality Standards – ISO 9001
• Issued by International Organization for Standardization (ISO)
• Important part of an ISO 9001 is a Proper Documentation
• ISO 9001 applies to S/W engineering
• It address the topics such as • Management
Responsibility
• Quality System
• Contract Review
• Design Control
• Document
• Data Control
• Product Identification
• Traceability
• Process Control
• Inspection
• Testing
• Prevention Action
• Control of Quality
Records
• Internal Quality
• Audits
• Training
• Servicing
s/w organization must establish
policies & procedures to address
each of the requirements & able to
demonstrates that these all are
being followed
Six Sigma
• A generic quantitative approach to improvement that applies to any
process
• Is a disciplined, data driven approach and methodology for
eliminating in any process from manufacturing to transactional and
from product to service
• Improve the process for the development of the products faster and
at reasonable cost
• Is a systematic approach to achieve perfection
• Is based on measurement strategy and obviously focuses on process
improvement
• Six Sigma have two methodologies,
▫ DMAIC
 Define – define the problem & process to improve upon
 Measure – How can you measure this process in systematic way?
 Analyze – identify the way in which it can be improved – find root
cause within the process to improve it
 Improve – present solution to improve and implement them
 Control – Utilize Statistical Process Control to continuously measure
your results and ensure that you’re improving continuously
▫ DMADV
 Define, Measure and Analyze are same as previous method
 Design – avoid root cause of defects and meet the customer
requirements (redesign)
 Verify – compare the process with standard plan and find differences
CMM (Capability Maturity Model )
• determine an organization’s current state of process maturity that
result in five point grading scheme
• Is a process meta model developed by SEI
• defines the process characteristics that should exist if an
organization want to establish a software process that is complete
• 5 process maturity levels are there
▫ Level 1 – Initial
▫ Level 2- Repeatable
▫ Level 3- Defined
▫ Level 4- Managed
▫ Level 5 - Optimizing
• Initial
▫ s/w process is characterized as ad hoc
▫ Few process are defined & success depends on individual effort
• Repeatable
▫ Basic project management processes are defined to track cost,
schedule & functionality
▫ Process discipline is in place to repeat success of similar kind of
projects
• Defined
▫ Both management & engineering activities Is documented,
standardized & integrated in the organization
▫ All project use documented and approved version of organization’s
process to proceed further
▫ Includes all characteristics defined for CMM level 2
• Managed
▫ Detailed measure of s/w process & product quality is collected
▫ Products and processes are controlled using detailed measures
▫ Includes all characteristics defined for CMM level 3
• Optimizing
▫ Continuous process improvement is done by quantitative
feedback from process and from testing innovative ideas &
technologies
▫ Includes all characteristics defined for CMM level 4
SQA Plan
• Main aim is to control the quality of the product and cover all
quality assurance activities
• Provides clear idea to produce Quality Product & is a backbone of
SQA activities
• Various section of SQA plan is as follow
▫ Documentation Section
 Contain docs related to work of plan of the project, models like ERD &
class, technical docs like test plan and user manual
▫ Standards, Practices & Conventions Section
 List of standards and practices that are followed for development
 Like coding standards, document standards etc.
▫ Review and Audit Section
 Provides overview of the approach of all the reviews and audits
conducted
▫ Test Section
 Use all possible testing techniques & procedures
 Record all errors uncovered and corrective action taken
▫ Management Section
 Uses tools & methods to assure quality of the product like,
 Configuration management procedures to control the change
 Define procedures to maintain the records
 Identified methods to handle risk management activities (RMMM
plan)
SQA ENDS HERE

Software Quality Assurance - Software Engineering

  • 1.
  • 2.
    S/W Quality • Qualitydefinition • Quality requirements • Quality Measures ▫ Mc Call’s Quality Factors ▫ ISO 9126 Quality Factors
  • 3.
    Intro • Planned andsystematic method that ▫ Evaluate the quality of s/w products, standards, process & procedures • Ensure that development follow Standards and Procedures • Done by ▫ Continuous Monitoring ▫ Product Evaluation ▫ Conducting Audits
  • 4.
    Standards & Procedures •Standards ▫ Established criteria to which product will be compared • Procedures ▫ Established criteria to which development process & control process are compared • Hence, ▫ These both establish the method for developing s/w application ▫ Role of SQA to provide existence & adequacy of standards and procedures ▫ SQA Activities  Continuous Monitoring, Product Evaluation, Conducting Audits
  • 5.
    Type of Standards •Documentation ▫ Define the proper content for planning and control ▫ Provide consistency throughout development life of project • Design ▫ Define the proper form and content of the design product ▫ Methods for translating the software requirements specification into the actual software design • Code ▫ Specify the programming language ▫ Also specify various constraints that should be put for usage of the language features (i.e use of data structures, patterns etc.)
  • 6.
    • Procedures mustbe followed in carrying out a development process • All the development processes must have documented procedures • Procedures like i.e. ▫ Configuration Management ▫ Non-conformance Reports ▫ Corrective Actions ▫ Testing & formal inspection
  • 7.
    SQA Activities • ProductEvaluation & Process Monitoring are two important activities ▫ Ensures development processes and the control processes written in the management plan are carried out effectively ▫ Ensure that all the procedures and standards are correctly followed • Products are continuously monitored for checking that it is following the standards and processes
  • 8.
    • Product Evaluation ▫Assures standards are being followed ▫ Ensures that the software application product is developed by conforming all the applicable standards as illustrated in the Management Plan • Product Monitoring ▫ Ensures that the appropriate steps are carried out during the development process ▫ Comparing the actual steps carried out with those in the documented procedures ▫ Ensures that the Management Plan specifies the methods that should be used by the monitoring activity
  • 9.
    • Product Audits ▫That looks the entire product and all the processes in depth ▫ Done by comparing them with the established standards and procedures ▫ It’s an important activity to review the management plan, technical processes and assurance processes to provide the actual status of the software application product ▫ Main idea behind this is control procedures are properly followed and desired documentation is properly maintained • SQA Product ▫ Is nothing but an audit report to display findings and recommendations to obey standards and procedures
  • 10.
    S/W Review • “filter”for software process • Servers to uncover errors and defects that can then be removed • “purify” s/w including ▫ Requirements ▫ Design models ▫ Code ▫ Testing data • Primary objective is to find errors during the process so that do not become defects after release of the s/w
  • 11.
    Defect Amplification &Removal Percentage Efficiency for error detection Errors passed through Amplified Errors 1: x Newly Generated Errors Defects Detection Development Step Errors from Previous Step Errors Passed To next Step
  • 12.
    Reviews: A FormalitySpectrum • Formality of review increase when, ▫ Distinct roles are defined for reviewers ▫ Sufficient amount of planning and preparation is available ▫ Distinct structure of review is defined (task and time) ▫ Follow-up by the reviewers for any correction that are made REVIEW Planning & Preparation Meeting Structure Correction & Verification Roles Individual Plays
  • 13.
    Informal Review • Simpledesk check or casual meeting held for a review • No advanced planning or preparation • No agenda or meeting structure • No follow-up on the errors Effectiveness of such review is lower then more formal approaches • To improve the efficiency, ▫ Develop a set of simple review checklists for each major work product
  • 14.
    Formal Technical Review(FTR) • S/Q QC activity performed by s/w engineers • Objectives ▫ To uncover errors in function, logic or implementation of s/w ▫ To verify s/w review meets its requirements ▫ To ensure s/w represented as per followed Standards ▫ To achieve s/w that is developed in uniform manner ▫ To make project more manageable • It serves as a training ground, enabling Juniors to observe different approaches of s/w life cycle • Serve to promote backup and Continuty
  • 15.
    • Its classof review that contain, ▫ Walkthroughs and Inspections • FTR is conducted as meeting and only be successful if it is well planned, controlled and attended • The Review Meeting ▫ 3-5 people should be involved in the review ▫ Advanced preparation should occurred (no more then 2 hours) ▫ Duration should be 2 hours for review meeting • FTR focus on a specific part of the overall software • Higher likelihood of uncovering the errors
  • 16.
    • FTR focuson work product • Producer (who has developed the work product)  informs Project Leader about completion of their work & review should be done • Project Leader informs Review Leader ▫ Who evaluate product for its readiness, generate product material & distribute them to the reviewers for advance preparation ▫ Create notes, reviewing product & establish agenda for review meeting • Presence of all above three characters required in a review meeting • Reviewers become Recorders (who records all important issues raise during the review)
  • 17.
    • At theend of the review, FTR must decide whether ▫ Accept the product without further modification ▫ Reject the product due to several errors (once modified another review must be performed) ▫ Accept the product provisionally (minor errors have been encountered & must be corrected but no further review required) • After final decision ▫ Attendees complete the sign off. • Review Reporting & Report Keeping ▫ Recorder list out all the issues raised during meeting ▫ Summarized at the end of review and review issue list is prepared
  • 18.
    ▫ A formaltechnical review summary report in completed that answers,  What was reviewed?  Who reviewed it?  What were the findings & conclusions? (it is of one single page) ▫ Review issue list serves two purpose  To identify problem areas within the product  An action item check list that guides the producer as correction are made (normally attached to summary report)
  • 19.
    Software Reliability • “theprobability of failure free operation of a computer program in a specified environment for a specific time” • Measure of reliability ▫ MTBF – (mean – time – between - failure) MTBF = MTTF + MTTR (MTTF - mean time to failure & MTTR - mean time to repair) ▫ Useful measure than other quality related software metrics ▫ Alternative measure of reliability is FIT (failure – in – time)  A measure of how many failures a component will have over one billion hours of operation
  • 20.
    • Software Availability ▫The probability that a program is operating according to requirement at a given point in time 𝑀𝑇𝑇𝐹 𝑀𝑇𝑇𝐹 + 𝑀𝑇𝑇𝑅 𝑥 100% • Software Safety ▫ Focus on identification and assessment of potential hazards that may affect s/w negatively & cause an entire system to fail
  • 21.
    Quality Standards –ISO 9001 • Issued by International Organization for Standardization (ISO) • Important part of an ISO 9001 is a Proper Documentation • ISO 9001 applies to S/W engineering • It address the topics such as • Management Responsibility • Quality System • Contract Review • Design Control • Document • Data Control • Product Identification • Traceability • Process Control • Inspection • Testing • Prevention Action • Control of Quality Records • Internal Quality • Audits • Training • Servicing s/w organization must establish policies & procedures to address each of the requirements & able to demonstrates that these all are being followed
  • 22.
    Six Sigma • Ageneric quantitative approach to improvement that applies to any process • Is a disciplined, data driven approach and methodology for eliminating in any process from manufacturing to transactional and from product to service • Improve the process for the development of the products faster and at reasonable cost • Is a systematic approach to achieve perfection • Is based on measurement strategy and obviously focuses on process improvement
  • 23.
    • Six Sigmahave two methodologies, ▫ DMAIC  Define – define the problem & process to improve upon  Measure – How can you measure this process in systematic way?  Analyze – identify the way in which it can be improved – find root cause within the process to improve it  Improve – present solution to improve and implement them  Control – Utilize Statistical Process Control to continuously measure your results and ensure that you’re improving continuously ▫ DMADV  Define, Measure and Analyze are same as previous method  Design – avoid root cause of defects and meet the customer requirements (redesign)  Verify – compare the process with standard plan and find differences
  • 24.
    CMM (Capability MaturityModel ) • determine an organization’s current state of process maturity that result in five point grading scheme • Is a process meta model developed by SEI • defines the process characteristics that should exist if an organization want to establish a software process that is complete • 5 process maturity levels are there ▫ Level 1 – Initial ▫ Level 2- Repeatable ▫ Level 3- Defined ▫ Level 4- Managed ▫ Level 5 - Optimizing
  • 25.
    • Initial ▫ s/wprocess is characterized as ad hoc ▫ Few process are defined & success depends on individual effort • Repeatable ▫ Basic project management processes are defined to track cost, schedule & functionality ▫ Process discipline is in place to repeat success of similar kind of projects • Defined ▫ Both management & engineering activities Is documented, standardized & integrated in the organization ▫ All project use documented and approved version of organization’s process to proceed further ▫ Includes all characteristics defined for CMM level 2
  • 26.
    • Managed ▫ Detailedmeasure of s/w process & product quality is collected ▫ Products and processes are controlled using detailed measures ▫ Includes all characteristics defined for CMM level 3 • Optimizing ▫ Continuous process improvement is done by quantitative feedback from process and from testing innovative ideas & technologies ▫ Includes all characteristics defined for CMM level 4
  • 27.
    SQA Plan • Mainaim is to control the quality of the product and cover all quality assurance activities • Provides clear idea to produce Quality Product & is a backbone of SQA activities • Various section of SQA plan is as follow ▫ Documentation Section  Contain docs related to work of plan of the project, models like ERD & class, technical docs like test plan and user manual ▫ Standards, Practices & Conventions Section  List of standards and practices that are followed for development  Like coding standards, document standards etc.
  • 28.
    ▫ Review andAudit Section  Provides overview of the approach of all the reviews and audits conducted ▫ Test Section  Use all possible testing techniques & procedures  Record all errors uncovered and corrective action taken ▫ Management Section  Uses tools & methods to assure quality of the product like,  Configuration management procedures to control the change  Define procedures to maintain the records  Identified methods to handle risk management activities (RMMM plan)
  • 29.