Test Management
(10 Marks)
Program: Computer Engineering
Course Name: Software Testing
Course Code:22518
Course Outcome: Prepare Test Plan for Application
Unit Outcome:
Prepare test plan for given application.
Identify the resource requirement of given application.
Prepare test case for given application.
Prepare test report of executed test cases for given
application.
Topic/Sub Topic
1. Test Planning
2. Test Management
3. Test Process
4. Test Reporting
What we will learn today
1. Test Planning
2. Test Management
3. Test Process
4. Test Reporting
Key Takeaway
Test Management
Concept Map
Test Cycle
closure
Test Execution
Environment
Setup
Test Case
Development
Test Planning
Requirement
Analysis
Test Plan:
• A document describing the
scope, approach, resources and
schedule of intended test
activities.
• It is a record of the test
planning process.
Test Planning→Preparing test plan
test items
Roles and Responsibilities
The features to be tested
Test design techniques and environment
Entry and exit criteria
Risk requiring and contingency planning
Test Planning→ Scope management
Understanding what supports a release of product
Breaking down release into features
Prioritizing the features for testing
Deciding which features will be tested and which will not be
Gathering details for estimating resources for testing.
The following factors drive the choice and prioritization of features to be tested
1) Features that are new and critical for the release:
➢ New feature will have new program code and thus have higher susceptibility and
exposure to defects.
➢ Since features are new both development and testing team will have to go
through a learning phase.
➢ So these features are in high priority list for testing.
➢ Here product marketing team and selected customers participate in identification
of features to be tested.
Test Planning→ Scope management
The following factors drive the choice and prioritization of features to be tested
2) Features whose failures can be catastrophic:
Any feature which have highest or adverse impact on business has to be high on the
list of features to be tested.
3) Features that are expected to be complex to test:
Early participation by testing team can help identify features that are difficult to test.
This can help in starting the work on these features early.
Test Planning→ Scope management
The following factors drive the choice and prioritization of features to be tested
4) Features which are extensions of earlies features that have defect prone:
➢ In regression testing, certain areas of code tend to be defect prone and such areas
need detailed testing so that old defects will not come again or occur again.
Test Planning→ Scope management
Test Planning→ Deciding test approach
What type testing would you use for testing the
functionality?
What are the scenarios for testing the features?
Which type integration testing will be performed ?
What type localization would be needed ?
What “non – functional” test would need to do?
Specify the metrics to be collected.
Identify significant constraint on testing, such as test-
item availability and deadline.
Test Planning→ Setting up criteria for testing
For each phase of testing there must be entry and
exit criteria.
The entry criteria for test specify threshold criteria for
each phase or type of test.
The exit criteria specify when a test cycle or testing
activity can be complete.
Suspension criteria specify when a test cycle or test
activity can be suspended.
Resumption criteria specify when the suspended test
can be resumed.
Test Planning→ Identifying Responsibilities
Identify the groups responsible for managing,
designing, preparing, executing, checking and
resolving.
Identify the groups responsible for providing the test
item identified in the test item section.
Identify the groups responsible for providing the
environmental needs.
Test Planning→ Staffing and Training needs
Specify staffing needs by skill level.
Identify training needs for providing necessary skill.
Test Planning→ Resource Requirement
Machine configuration needed to run product under
test.
Different configuration or versions of the software
that must be present.
Need of automation tool like load runner, win runner,
QTP etc.
Supporting tools such as compilers, test data
generators, configuration management tools and so
on.
Appropriate number of licenses of all software.
Test Planning→ Test Deliverable
Test plan
Test case design specification
Test cases (manual and automated)
Test logs produced by running the test.
Test summary report
Bug report written in defect repository
Test Planning→ Testing task → Estimation
Estimation
Size
Estimation
Effort
Estimation
Schedule
Estimation
Test Planning→ Testing task →Size Estimation Method
Size
Estimation
Line of
code
Functional
point
Application
size
Page
19
Test Planning→ Testing task →Effort Estimation
Page
20
Derived from size
estimate
Taking the individual
work break down
structure units
Classifying work as
reusable
modification and
new development
Test Planning→ Testing task →Schedule Estimation
Page
21
Translating the
effort required
into specific
time frames.
Identify the
task necessary
to prepare for
and perform
testing.
Identify all the
task inter
dependency.
Identify any
special skills
required.
Test Management:
• Test management is a
process of managing
testing activities, such as
planning, execution,
monitoring, and
controlling activities.
Test Management →Choice of standards
Standards are of two types:
➢ External standards:-are standards that a product should comply with, are
externally visible are usually stipulated by external consortia.
➢ Compliance to external standards is usually mandated by external parties.
Example: ISO
➢ Internal standards:-are standards formulated by testing organization to bring
consistency.
➢ It standardize the processes and methods of working within the organization
Page
23
Test Management →Choice of standards
Internal standards include
1. Naming and storage convention for test artifacts:-Every test artifact like test cases,
test result, test specification and so on must be named appropriate and
meaningfully.
2. Documentation standards:- Documentation standard specify how to record
information about the test within the test script themselves.
3. Test coding standards:- Test coding standards go one level deepen into the test and
it feels how the test should be written.
Page
24
Test Management →Test Infrastructure Management
Testing infrastructure is made up of three essential elements:
Page
25
Defect
Repository
Test Infrastructure Management→ TCDB
Page
26
Test Case –Defect Cross Reference
Purpose :Provides mapping between test case and defect
Attribute: Test Case ID ,Defect ID
Test case run history
Purpose: When test was run?, What was the
result?
Attribute: Test Case ID ,Run date, Time taken,
Status
Test case product cross reference
Purpose:Provides mapping between test and
corresponding feature
Attributes
Test Case ID ,Module ID
Test Case
Purpose : Record all the static information
about the test.
Attributes
Test case ID ,Test case name, Test case Owner
Test Infrastructure Management→ Defect Repository
Page
27
It captures relevant details of defect.
It is tool of communication.
Defect Metrics are derived from defect repository.
Test Infrastructure Management→ Configuration management repository and tools.
Page
28
Keeps track of change control of all the
files/entities that makeup a software product.
Keeps track of version control of all the
files/entities that makeup a software product.
Test people management
Page
29
People management is an integral part of any project
management.
A person relies on his or her own skills to accomplish an
assigned activity.
People management also requires the ability to hire, motivate
and retain the right people .
Success of testing organization depends on careful people
management skills .
Integration with product release
➢ Success of a product depends on the effectiveness of integration of the
development and testing activities.
➢ Both team should work in co-ordination with each other.
➢ Schedule of testing have to be considered or linked to product release.
Page
30
Integration with product release
Following points to be decided for this planning:
1. Synchronization between testing and development as to when different types of
testing can start. For example When unit testing or system testing could start and
so on.
2. Services level agreement between development and testing as to how long it
would take for testing team to complete the testing.
3. Definition of the various priorities and severities of the defects. Development and
testing team should have same vision.
4. Establish communication mechanisms to ensure that the document is kept in
sync with product development and testing.
Page
31
Test Process
• Testing is not a single activity instead it’s a set of number of
processes.
• It includes following activities:
1. Base Lining a Test Plan
2. Test Case Specification
3. Update of Traceability Matrix
Page
32
Base Lining test plan
• Every organization have template of test plan.
• The test plan is reviewed by a designated set of competent authority in
organization.
• After this plan is approved by authority who is responsible for testing.
Page
33
Base Lining test plan
• After this, the test plan is baseline into the configuration management repository.
• From then baseline test plan becomes the basis for running the testing project.
• Any significant changes in the testing project should thereafter be reflected in the
test plan .
• Changed test plan baseline again the configuration management repository.
Page
34
Test Case Specification
Basis for
Baseline
test
plan
Basis
for
Test team
design
specifications
Individual
Test Case
Page
35
Test Case Specification
Test case specification should clearly identify following :
1. The purpose of the test:-This will list which feature the test is intended for
2. Items being tested along with their version release number
3. Environment that needs to be set up for running the test case.eg hardware
environment set up, software environment set up
4. Input data to be used for the test case. This choice is depending on technique
followed in the case. For example manual or automated or type of testing
(equivalence portioning or boundry value analysis)
Page
36
Test Case Specification
Test case specification should clearly identify following :
5. Steps to be followed to execute test
6. The expected result that are correct results
7. A step to compare the actual result produced with the expected result
8. Any relationship between this test and other tests:-by this we can find
dependencies among the test
Page
37
Requirement Traceability Matrix
Traceability matrix is tool to validate that every requirement is tested. It provides
mapping between requirement and test cases.
• The traceability matrix is created during the requirement gathering phase.
• Unique identifier is assigned to each requirement.
• As project proceeds through design phase identifier for design feature is entered in
matrix.
Page
38
Requirement Traceability Matrix
• When project enters coding phase identifier for program file is entered in the
traceability matrix.
• When test case specification is complete the row corresponding to the
requirement which is being tested by the test case is updated with the test case
specification identifier.
Page
39
Requirement Traceability Matrix
• Testability matrix helps in:
– Ensuring 100% test coverage
– Showing requirement/document inconsistencies
– Displaying the overall defect/execution status with focus on business
requirements.
– Template for RTM(Requirement Traceability matrix) :
Page
40
Test Reporting
Page
41
Executing Test
case
Preparing Test
Summary Report
Executing Test Cases
Proper execution of test cases is essential which will minimize the work and reduce
the time to release s/w product.
Test execution task:
1. Follow the test procedures to execute test cases.
2. Do the confirmation testing for the failed test cases.
3. Log the result for test execution.
Page
42
Executing Test Cases
4. Compare actual and expected results. In case of difference defect occurrence is
reported.
5. Update defect database which is used to communicate between developer and
tester team.
6. So the execution of test cases will decide the suspension or resumption of
further test cases.
Page
43
Test Summary Report
• Test summary report is a document which contains summary of test activities and
final test results.
• After the testing cycle it is very important that you communicate the test results
and findings to the project stakeholders .
• so that decisions can be made for the software release. i.e. If further testing is
required and we need to delay the release.
Page
44
Test Summary Report
• Test summary report will be different for different kind of testing.
• In addition to test coverage and unresolved defects test summary reports should
also contain test strategy, test objectives and overall result of test effort.
Page
45
References
https://www.gcreddy.com/2014/07/software-test-plan-templates.html
https://www.kualitee.com/test-management/best-test-management-tools-must-
used-2019/
47
Thank You
Supriya Kadam
Department of Computer Engineering (NBA Accredited)
Vidyalankar Polytechnic
Vidyalankar College Marg, Wadala(E), Mumbai 400 037
E-mail: supriya.angne@vpt.edu.in
48

Test Management final ppt file for vp(1).pdf

  • 1.
  • 2.
    Program: Computer Engineering CourseName: Software Testing Course Code:22518 Course Outcome: Prepare Test Plan for Application Unit Outcome: Prepare test plan for given application. Identify the resource requirement of given application. Prepare test case for given application. Prepare test report of executed test cases for given application.
  • 3.
    Topic/Sub Topic 1. TestPlanning 2. Test Management 3. Test Process 4. Test Reporting
  • 4.
    What we willlearn today 1. Test Planning 2. Test Management 3. Test Process 4. Test Reporting Key Takeaway Test Management
  • 5.
    Concept Map Test Cycle closure TestExecution Environment Setup Test Case Development Test Planning Requirement Analysis
  • 6.
    Test Plan: • Adocument describing the scope, approach, resources and schedule of intended test activities. • It is a record of the test planning process.
  • 7.
    Test Planning→Preparing testplan test items Roles and Responsibilities The features to be tested Test design techniques and environment Entry and exit criteria Risk requiring and contingency planning
  • 8.
    Test Planning→ Scopemanagement Understanding what supports a release of product Breaking down release into features Prioritizing the features for testing Deciding which features will be tested and which will not be Gathering details for estimating resources for testing.
  • 9.
    The following factorsdrive the choice and prioritization of features to be tested 1) Features that are new and critical for the release: ➢ New feature will have new program code and thus have higher susceptibility and exposure to defects. ➢ Since features are new both development and testing team will have to go through a learning phase. ➢ So these features are in high priority list for testing. ➢ Here product marketing team and selected customers participate in identification of features to be tested. Test Planning→ Scope management
  • 10.
    The following factorsdrive the choice and prioritization of features to be tested 2) Features whose failures can be catastrophic: Any feature which have highest or adverse impact on business has to be high on the list of features to be tested. 3) Features that are expected to be complex to test: Early participation by testing team can help identify features that are difficult to test. This can help in starting the work on these features early. Test Planning→ Scope management
  • 11.
    The following factorsdrive the choice and prioritization of features to be tested 4) Features which are extensions of earlies features that have defect prone: ➢ In regression testing, certain areas of code tend to be defect prone and such areas need detailed testing so that old defects will not come again or occur again. Test Planning→ Scope management
  • 12.
    Test Planning→ Decidingtest approach What type testing would you use for testing the functionality? What are the scenarios for testing the features? Which type integration testing will be performed ? What type localization would be needed ? What “non – functional” test would need to do? Specify the metrics to be collected. Identify significant constraint on testing, such as test- item availability and deadline.
  • 13.
    Test Planning→ Settingup criteria for testing For each phase of testing there must be entry and exit criteria. The entry criteria for test specify threshold criteria for each phase or type of test. The exit criteria specify when a test cycle or testing activity can be complete. Suspension criteria specify when a test cycle or test activity can be suspended. Resumption criteria specify when the suspended test can be resumed.
  • 14.
    Test Planning→ IdentifyingResponsibilities Identify the groups responsible for managing, designing, preparing, executing, checking and resolving. Identify the groups responsible for providing the test item identified in the test item section. Identify the groups responsible for providing the environmental needs.
  • 15.
    Test Planning→ Staffingand Training needs Specify staffing needs by skill level. Identify training needs for providing necessary skill.
  • 16.
    Test Planning→ ResourceRequirement Machine configuration needed to run product under test. Different configuration or versions of the software that must be present. Need of automation tool like load runner, win runner, QTP etc. Supporting tools such as compilers, test data generators, configuration management tools and so on. Appropriate number of licenses of all software.
  • 17.
    Test Planning→ TestDeliverable Test plan Test case design specification Test cases (manual and automated) Test logs produced by running the test. Test summary report Bug report written in defect repository
  • 18.
    Test Planning→ Testingtask → Estimation Estimation Size Estimation Effort Estimation Schedule Estimation
  • 19.
    Test Planning→ Testingtask →Size Estimation Method Size Estimation Line of code Functional point Application size Page 19
  • 20.
    Test Planning→ Testingtask →Effort Estimation Page 20 Derived from size estimate Taking the individual work break down structure units Classifying work as reusable modification and new development
  • 21.
    Test Planning→ Testingtask →Schedule Estimation Page 21 Translating the effort required into specific time frames. Identify the task necessary to prepare for and perform testing. Identify all the task inter dependency. Identify any special skills required.
  • 22.
    Test Management: • Testmanagement is a process of managing testing activities, such as planning, execution, monitoring, and controlling activities.
  • 23.
    Test Management →Choiceof standards Standards are of two types: ➢ External standards:-are standards that a product should comply with, are externally visible are usually stipulated by external consortia. ➢ Compliance to external standards is usually mandated by external parties. Example: ISO ➢ Internal standards:-are standards formulated by testing organization to bring consistency. ➢ It standardize the processes and methods of working within the organization Page 23
  • 24.
    Test Management →Choiceof standards Internal standards include 1. Naming and storage convention for test artifacts:-Every test artifact like test cases, test result, test specification and so on must be named appropriate and meaningfully. 2. Documentation standards:- Documentation standard specify how to record information about the test within the test script themselves. 3. Test coding standards:- Test coding standards go one level deepen into the test and it feels how the test should be written. Page 24
  • 25.
    Test Management →TestInfrastructure Management Testing infrastructure is made up of three essential elements: Page 25 Defect Repository
  • 26.
    Test Infrastructure Management→TCDB Page 26 Test Case –Defect Cross Reference Purpose :Provides mapping between test case and defect Attribute: Test Case ID ,Defect ID Test case run history Purpose: When test was run?, What was the result? Attribute: Test Case ID ,Run date, Time taken, Status Test case product cross reference Purpose:Provides mapping between test and corresponding feature Attributes Test Case ID ,Module ID Test Case Purpose : Record all the static information about the test. Attributes Test case ID ,Test case name, Test case Owner
  • 27.
    Test Infrastructure Management→Defect Repository Page 27 It captures relevant details of defect. It is tool of communication. Defect Metrics are derived from defect repository.
  • 28.
    Test Infrastructure Management→Configuration management repository and tools. Page 28 Keeps track of change control of all the files/entities that makeup a software product. Keeps track of version control of all the files/entities that makeup a software product.
  • 29.
    Test people management Page 29 Peoplemanagement is an integral part of any project management. A person relies on his or her own skills to accomplish an assigned activity. People management also requires the ability to hire, motivate and retain the right people . Success of testing organization depends on careful people management skills .
  • 30.
    Integration with productrelease ➢ Success of a product depends on the effectiveness of integration of the development and testing activities. ➢ Both team should work in co-ordination with each other. ➢ Schedule of testing have to be considered or linked to product release. Page 30
  • 31.
    Integration with productrelease Following points to be decided for this planning: 1. Synchronization between testing and development as to when different types of testing can start. For example When unit testing or system testing could start and so on. 2. Services level agreement between development and testing as to how long it would take for testing team to complete the testing. 3. Definition of the various priorities and severities of the defects. Development and testing team should have same vision. 4. Establish communication mechanisms to ensure that the document is kept in sync with product development and testing. Page 31
  • 32.
    Test Process • Testingis not a single activity instead it’s a set of number of processes. • It includes following activities: 1. Base Lining a Test Plan 2. Test Case Specification 3. Update of Traceability Matrix Page 32
  • 33.
    Base Lining testplan • Every organization have template of test plan. • The test plan is reviewed by a designated set of competent authority in organization. • After this plan is approved by authority who is responsible for testing. Page 33
  • 34.
    Base Lining testplan • After this, the test plan is baseline into the configuration management repository. • From then baseline test plan becomes the basis for running the testing project. • Any significant changes in the testing project should thereafter be reflected in the test plan . • Changed test plan baseline again the configuration management repository. Page 34
  • 35.
    Test Case Specification Basisfor Baseline test plan Basis for Test team design specifications Individual Test Case Page 35
  • 36.
    Test Case Specification Testcase specification should clearly identify following : 1. The purpose of the test:-This will list which feature the test is intended for 2. Items being tested along with their version release number 3. Environment that needs to be set up for running the test case.eg hardware environment set up, software environment set up 4. Input data to be used for the test case. This choice is depending on technique followed in the case. For example manual or automated or type of testing (equivalence portioning or boundry value analysis) Page 36
  • 37.
    Test Case Specification Testcase specification should clearly identify following : 5. Steps to be followed to execute test 6. The expected result that are correct results 7. A step to compare the actual result produced with the expected result 8. Any relationship between this test and other tests:-by this we can find dependencies among the test Page 37
  • 38.
    Requirement Traceability Matrix Traceabilitymatrix is tool to validate that every requirement is tested. It provides mapping between requirement and test cases. • The traceability matrix is created during the requirement gathering phase. • Unique identifier is assigned to each requirement. • As project proceeds through design phase identifier for design feature is entered in matrix. Page 38
  • 39.
    Requirement Traceability Matrix •When project enters coding phase identifier for program file is entered in the traceability matrix. • When test case specification is complete the row corresponding to the requirement which is being tested by the test case is updated with the test case specification identifier. Page 39
  • 40.
    Requirement Traceability Matrix •Testability matrix helps in: – Ensuring 100% test coverage – Showing requirement/document inconsistencies – Displaying the overall defect/execution status with focus on business requirements. – Template for RTM(Requirement Traceability matrix) : Page 40
  • 41.
  • 42.
    Executing Test Cases Properexecution of test cases is essential which will minimize the work and reduce the time to release s/w product. Test execution task: 1. Follow the test procedures to execute test cases. 2. Do the confirmation testing for the failed test cases. 3. Log the result for test execution. Page 42
  • 43.
    Executing Test Cases 4.Compare actual and expected results. In case of difference defect occurrence is reported. 5. Update defect database which is used to communicate between developer and tester team. 6. So the execution of test cases will decide the suspension or resumption of further test cases. Page 43
  • 44.
    Test Summary Report •Test summary report is a document which contains summary of test activities and final test results. • After the testing cycle it is very important that you communicate the test results and findings to the project stakeholders . • so that decisions can be made for the software release. i.e. If further testing is required and we need to delay the release. Page 44
  • 45.
    Test Summary Report •Test summary report will be different for different kind of testing. • In addition to test coverage and unresolved defects test summary reports should also contain test strategy, test objectives and overall result of test effort. Page 45
  • 46.
  • 47.
    47 Thank You Supriya Kadam Departmentof Computer Engineering (NBA Accredited) Vidyalankar Polytechnic Vidyalankar College Marg, Wadala(E), Mumbai 400 037 E-mail: supriya.angne@vpt.edu.in
  • 48.