Unit iv-testing-pune-university-sres-coe

UNIT:IV
Software Testing Strategies
- A strategic approach to testing
- Test strategies for conventional software
- Test strategies for object-oriented software
- Validation testing
- System testing
- The art of debugging

Introduction
•
•
•
•
•

A strategy for software testing integrates the design of software test
cases into a well-planned series of steps that result in successful
development of the software
The strategy provides a road map that describes the steps to be taken,
when, and how much effort, time, and resources will be required
The strategy incorporates test planning, test case design, test
execution, and test result collection and evaluation
The strategy provides guidance for the practitioner and a set of
milestones for the manager
Because of time pressures, progress must be measurable and problems
must surface as early as possible

2

A Strategic Approach to Testing

General Characteristics of
Strategic Testing
•
•
•
•
•

To perform effective testing, a software team should conduct effective
formal technical reviews
Testing begins at the component level and work outward toward the
integration of the entire computer-based system
Different testing techniques are appropriate at different points in time
Testing is conducted by the developer of the software and (for large
projects) by an independent test group
Testing and debugging are different activities, but debugging must be
accommodated in any testing strategy

4

Verification and Validation
•
•

Software testing is part of a broader group of activities called verification
and validation that are involved in software quality assurance
Verification (Are the algorithms coded correctly?)
– The set of activities that ensure that software correctly implements a specific
function or algorithm

•

Validation (Does it meet user requirements?)
– The set of activities that ensure that the software that has been built is
traceable to customer requirements

5

Organizing for Software Testing
•
•

Testing should aim at "breaking" the software
Common misconceptions
– The developer of software should do no testing at all
– The software should be given to a secret team of testers who will test it
unmercifully
– The testers get involved with the project only when the testing steps are
about to begin

•

Reality: Independent test group
– Removes the inherent problems associated with letting the builder test the
software that has been built
– Removes the conflict of interest that may otherwise be present
– Works closely with the software developer during analysis and design to
ensure that thorough testing occurs
6

A Strategy for Testing
Conventional Software
Validation Testing
Integration Testing
Unit Testing

Br Nar
o a ro
de w
r s to
co
pe

System Testing

Ab
co strac
nc t
re to
te

Code
Design
Requirements
System Engineering
7

Levels of Testing for
•

Unit testing
– Concentrates on each component/function of the software as implemented
in the source code

•

Integration testing
– Focuses on the design and construction of the software architecture

•

Validation testing
– Requirements are validated against the constructed software

•

System testing
– The software and other system elements are tested as a whole

8

Testing Strategy applied to
•

Unit testing
– Exercises specific paths in a component's control structure to ensure
complete coverage and maximum error detection
– Components are then assembled and integrated

•

Integration testing
– Focuses on inputs and outputs, and how well the components fit together
and work together

•

Validation testing
– Provides final assurance that the software meets all functional, behavioral,
and performance requirements

•

System testing
– Verifies that all system elements (software, hardware, people, databases)
mesh properly and that overall system function and performance is
achieved
9

Testing Strategy applied to
Object-Oriented Software
•
•
•
•

Must broaden testing to include detections of errors in analysis and design
models
Unit testing loses some of its meaning and integration testing changes
significantly
Use the same philosophy but different approach as in conventional
software testing
Test "in the small" and then work out to testing "in the large"
– Testing in the small involves class attributes and operations; the main focus is
on communication and collaboration within the class
– Testing in the large involves a series of regression tests to uncover errors due
to communication and collaboration among classes

•

Finally, the system as a whole is tested to detect errors in fulfilling
requirements
10

When is Testing Complete?
•
•
•
•

There is no definitive answer to this question
Every time a user executes the software, the program is being tested
Sadly, testing usually stops when a project is running out of time,
money, or both
One approach is to divide the test results into various severity levels
– Then consider testing to be complete when certain levels of errors no
longer occur or have been repaired or eliminated

11

Ensuring a Successful Software
Test Strategy
•
•
•
•
•
•
•
•

Specify product requirements in a quantifiable manner long before
testing commences
State testing objectives explicitly in measurable terms
Understand the user of the software (through use cases) and develop a
profile for each user category
Develop a testing plan that emphasizes rapid cycle testing to get quick
feedback to control quality levels and adjust the test strategy
Build robust software that is designed to test itself and can diagnose
certain kinds of errors
Use effective formal technical reviews as a filter prior to testing to
reduce the amount of testing required
Conduct formal technical reviews to assess the test strategy and test
cases themselves
Develop a continuous improvement approach for the testing process
through the gathering of metrics
12

Test Strategies for

Unit Testing
•
•
•

Focuses testing on the function or software module
Concentrates on the internal processing logic and data structures
Is simplified when a module is designed with high cohesion
– Reduces the number of test cases
– Allows errors to be more easily predicted and uncovered

•

Concentrates on critical modules and those with high cyclomatic
complexity when testing resources are limited

14

Targets for Unit Test Cases
•

Module interface
– Ensure that information flows properly into and out of the module

•

Local data structures
– Ensure that data stored temporarily maintains its integrity during all steps
in an algorithm execution

•

Boundary conditions
– Ensure that the module operates properly at boundary values established
to limit or restrict processing

•

Independent paths (basis paths)
– Paths are exercised to ensure that all statements in a module have been
executed at least once

•

Error handling paths
– Ensure that the algorithms respond correctly to specific error conditions

15

Common Computational Errors
in Execution Paths
•
•
•
•
•

Misunderstood or incorrect arithmetic precedence
Mixed mode operations (e.g., int, float, char)
Incorrect initialization of values
Precision inaccuracy and round-off errors
Incorrect symbolic representation of an expression (int vs. float)

16

Other Errors to Uncover
•
•
•
•
•
•
•
•

Comparison of different data types
Incorrect logical operators or precedence
Expectation of equality when precision error makes equality unlikely
(using == with float types)
Incorrect comparison of variables
Improper or nonexistent loop termination
Failure to exit when divergent iteration is encountered
Improperly modified loop variables
Boundary value violations

17

Problems to uncover in
Error Handling
•
•
•
•
•

Error description is unintelligible or ambiguous
Error noted does not correspond to error encountered
Error condition causes operating system intervention prior to error
handling
Exception condition processing is incorrect
Error description does not provide enough information to assist in the
location of the cause of the error

18

Drivers and Stubs for
Unit Testing
•

Driver
– A simple main program that accepts test case data, passes such data to the
component being tested, and prints the returned results

•

Stubs
– Serve to replace modules that are subordinate to (called by) the
component to be tested
– It uses the module’s exact interface, may do minimal data manipulation,
provides verification of entry, and returns control to the module
undergoing testing

•

Drivers and stubs both represent overhead
– Both must be written but don’t constitute part of the installed software
product

19

Unit Test Environment
driver

interface
local data structures
Module

stub

boundary conditions
independent paths
error handling paths

stub

test cases
RESULTS
20

Integration Testing
•

Defined as a systematic technique for constructing the software
architecture
– At the same time integration is occurring, conduct tests to uncover errors
associated with interfaces

•
•

Objective is to take unit tested modules and build a program structure
based on the prescribed design
Two Approaches
– Non-incremental Integration Testing
– Incremental Integration Testing

21

Non-incremental
Integration Testing
•
•
•
•
•
•
•

Commonly called the “Big Bang” approach
All components are combined in advance
The entire program is tested as a whole
Chaos results
Many seemingly-unrelated errors are encountered
Correction is difficult because isolation of causes is complicated
Once a set of errors are corrected, more errors occur, and testing appears
to enter an endless loop

22

Incremental Integration Testing
•

Three kinds
– Top-down integration
– Bottom-up integration
– Sandwich integration

•
•
•
•

The program is constructed and tested in small increments
Errors are easier to isolate and correct
Interfaces are more likely to be tested completely
A systematic test approach is applied

23

Top-down Integration
•
•

Modules are integrated by moving downward through the control
hierarchy, beginning with the main module
Subordinate modules are incorporated in either a depth-first or
breadth-first fashion
– DF: All modules on a major control path are integrated
– BF: All modules directly subordinate at each level are integrated

•

Advantages
– This approach verifies major control or decision points early in the test
process

•

Disadvantages
– Stubs need to be created to substitute for modules that have not been built
or tested yet; this code is later discarded
– Because stubs are used to replace lower level modules, no significant data
flow can occur until much later in the integration/testing process

24

Top Down Integration
A

B

top module is tested with
stubs

F

G

stubs are replaced one at
a time, "depth first"

C

as new modules are integrated,
some subset of tests is re-run
D

E

25

Bottom-up Integration
•
•

Integration and testing starts with the most atomic modules in the
control hierarchy
Advantages
– This approach verifies low-level data processing early in the testing
process
– Need for stubs is eliminated

•

Disadvantages
– Driver modules need to be built to test the lower-level modules; this code
is later discarded or expanded into a full-featured version
– Drivers inherently do not contain the complete algorithms that will
eventually use the services of the lower-level modules; consequently,
testing may be incomplete or more testing may be needed later when the
upper level modules are available

26

Bottom-Up Integration
A

B

G

drivers are replaced one at a
time, "depth first"

C
D

F

E

worker modules are grouped into
builds and integrated

cluster
27

Sandwich Integration
•
•
•

Consists of a combination of both top-down and bottom-up integration
Occurs both at the highest level modules and also at the lowest level
modules
Proceeds using functional groups of modules, with each group
completed before the next
– High and low-level modules are grouped based on the control and data
processing they provide for a specific program feature
– Integration within the group progresses in alternating steps between the
high and low level modules of the group
– When integration for a certain functional group is complete, integration
and testing moves onto the next group

•
•

Reaps the advantages of both types of integration while minimizing
the need for drivers and stubs
Requires a disciplined approach so that integration doesn’t tend
towards the “big bang” scenario
28

Sandwich Testing
A

B

F

Top modules are
tested with stubs
G

C
D

E

Worker modules are grouped into
builds and integrated

cluster
29

Regression Testing
•
•

Each new addition or change to baseline software may cause problems
with functions that previously worked flawlessly.
Regression testing re-executes a small subset of tests that have already
been conducted.
– Ensures that changes have not propagated unintended side effects
– Helps to ensure that changes do not introduce unintended behavior or
additional errors
– May be done manually or through the use of automated capture/playback
tools

•

Regression test suite contains three different classes of test cases.
– A representative sample of tests that will exercise all software functions
– Additional tests that focus on software functions that are likely to be
affected by the change
– Tests that focus on the actual software components that have been
changed
30

Smoke Testing
•

Taken from the world of hardware
– Power is applied and a technician checks for sparks, smoke, or other
dramatic signs of fundamental failure

•

Designed as a pacing mechanism for time-critical projects
– Allows the software team to assess its project on a frequent basis

•

Includes the following activities
– The software is compiled and linked into a build
– A series of breadth tests is designed to expose errors that will keep the
build from properly performing its function
• The goal is to uncover “show stopper” errors that have the highest likelihood
of throwing the software project behind schedule

– The build is integrated with other builds and the entire product is smoke
tested daily
• Daily testing gives managers and practitioners a realistic assessment of the
progress of the integration testing

– After a smoke test is completed, detailed test scripts are executed

31

Benefits of Smoke Testing
•

Integration risk is minimized
– Daily testing uncovers incompatibilities and show-stoppers early in the
testing process, thereby reducing schedule impact

•

The quality of the end-product is improved
– Smoke testing is likely to uncover both functional errors and architectural
and component-level design errors

•

Error diagnosis and correction are simplified
– Smoke testing will probably uncover errors in the newest components that
were integrated

•

Progress is easier to assess
– As integration testing progresses, more software has been integrated and
more has been demonstrated to work
– Managers get a good indication that progress is being made

32

Example
• Example of Smoke testing is explain below
•

Considering example in a project there are five modules like login,
view user, user detail page, new user creation, and task creation etc.
So in this five modules first of all developer perform the smoke
testing by executing all the extreme (major) functionality of modules
like user is able to login or not with valid login credentials and after
login new user can created or not, and user that is created viewed or
not. So it is obvious that this is the smoke testing always done by
developing team before submitting (releasing) the build to the testing
team.
33

Test Strategies for

Test Strategies for
•
•
•

With object-oriented software, you can no longer test a single operation in
isolation (conventional thinking)
Traditional top-down or bottom-up integration testing has little meaning
Class testing for object-oriented software is the equivalent of unit testing
for conventional software
– Focuses on operations encapsulated by the class and the state behavior of the
class

•

Drivers can be used
– To test operations at the lowest level and for testing whole groups of classes
– To replace the user interface so that tests of system functionality can be
conducted prior to implementation of the actual interface

•

Stubs can be used
– In situations in which collaboration between classes is required but one or
more of the collaborating classes has not yet been fully implemented
35

Test Strategies for ObjectOriented Software (continued)
•

Two different object-oriented testing strategies
– Thread-based testing
• Integrates the set of classes required to respond to one input or event for the
system
• Each thread is integrated and tested individually
• Regression testing is applied to ensure that no side effects occur

– Use-based testing
• First tests the independent classes that use very few, if any, server classes
• Then the next layer of classes, called dependent classes, are integrated
• This sequence of testing layer of dependent classes continues until the entire
system is constructed

36

Background
•
•
•
•
•

Validation testing follows integration testing
The distinction between conventional and object-oriented software disappears
Focuses on user-visible actions and user-recognizable output from the system
Demonstrates conformity with requirements
Designed to ensure that
–
–
–
–
–

All functional requirements are satisfied
All behavioral characteristics are achieved
All performance requirements are attained
Documentation is correct
Usability and other requirements are met (e.g., transportability, compatibility, error
recovery, maintainability)

• After each validation test
– The function or performance characteristic conforms to specification and is
accepted
– A deviation from specification is uncovered and a deficiency list is created

• A configuration review or audit ensures that all elements of the software
configuration have been properly developed, cataloged, and have the necessary
detail for entering the support phase of the software life cycle
38

Alpha and Beta Testing
•

Alpha testing
– Conducted at the developer’s site by end users
– Software is used in a natural setting with developers watching intently
– Testing is conducted in a controlled environment

•

Beta testing
– Conducted at end-user sites
– Developer is generally not present
– It serves as a live application of the software in an environment that
cannot be controlled by the developer
– The end-user records all problems that are encountered and reports these
to the developers at regular intervals

•

After beta testing is complete, software engineers make software
modifications and prepare for release of the software product to the
entire customer base
39

Different Types
•

Recovery testing
– Tests for recovery from system faults
– Forces the software to fail in a variety of ways and verifies that recovery
is properly performed
– Tests reinitialization, checkpointing mechanisms, data recovery, and
restart for correctness

•

Security testing
– Verifies that protection mechanisms built into a system will, in fact,
protect it from improper access

•

Stress testing
– Executes a system in a manner that demands resources in abnormal
quantity, frequency, or volume

•

Performance testing
– Tests the run-time performance of software within the context of an
integrated system
– Often coupled with stress testing and usually requires both hardware and
software instrumentation
– Can uncover situations that lead to degradation and possible system
failure
41

Debugging:
A Diagnostic Process

43

Debugging Process
•
•
•
•
•
•
•
•

Debugging occurs as a consequence of successful testing
It is still very much an art rather than a science
Good debugging ability may be an innate human trait
Large variances in debugging ability exist
The debugging process begins with the execution of a test case
Results are assessed and the difference between expected and actual performance
is encountered
This difference is a symptom of an underlying cause that lies hidden
The debugging process attempts to match symptom with cause, thereby leading
to error correction

44

The Debugging Process
test cases

new test
regression cases
suspected
tests
causes
corrections
identified
causes
45

results

Debugging

Debugging Effort
time required
to diagnose the
symptom and
determine the
cause

time required
to correct the error
and conduct
regression tests

46

Symptoms & Causes
symptom and cause may be
geographically separated
symptom may disappear when
another problem is fixed
cause may be due to a
combination of non-errors
cause may be due to a system
or compiler error
symptom
cause

cause may be due to
assumptions that everyone
believes
symptom may be intermittent
47

Why is Debugging so Difficult?
•
•
•
•

The symptom and the cause may be geographically remote
The symptom may disappear (temporarily) when another error is
corrected
The symptom may actually be caused by nonerrors (e.g., round-off
accuracies)
The symptom may be caused by human error that is not easily traced

(continued on next slide)
48

Why is Debugging so Difficult?
(continued)
•
•
•
•

The symptom may be a result of timing problems, rather than processing
problems
It may be difficult to accurately reproduce input conditions, such as
asynchronous real-time information
The symptom may be intermittent such as in embedded systems involving
both hardware and software
The symptom may be due to causes that are distributed across a number of
tasks running on different processes

49

Debugging Strategies
•
•
•
•

Objective of debugging is to find and correct the cause of a software
error
Bugs are found by a combination of systematic evaluation, intuition,
and luck
Debugging methods and tools are not a substitute for careful
evaluation based on a complete design model and clear source code
There are three main debugging strategies
– Brute force
– Backtracking
– Cause elimination

50

Strategy #1: Brute Force
•
•
•
•

Most commonly used and least efficient method
Used when all else fails
Involves the use of memory dumps, run-time traces, and output
statements
Leads many times to wasted effort and time

51

Strategy #2: Backtracking
•
•
•
•

Can be used successfully in small programs
The method starts at the location where a symptom has been
uncovered
The source code is then traced backward (manually) until the location
of the cause is found
In large programs, the number of potential backward paths may
become unmanageably large

52

Strategy #3: Cause Elimination
•

Involves the use of induction or deduction and introduces the concept of
binary partitioning
– Induction (specific to general): Prove that a specific starting value is true; then
prove the general case is true
– Deduction (general to specific): Show that a specific conclusion follows from a
set of general premises

•
•
•
•

Data related to the error occurrence are organized to isolate potential
causes
A cause hypothesis is devised, and the aforementioned data are used to
prove or disprove the hypothesis
Alternatively, a list of all possible causes is developed, and tests are
conducted to eliminate each cause
If initial tests indicate that a particular cause hypothesis shows promise,
data are refined in an attempt to isolate the bug

53

Three Questions to ask Before
Correcting the Error
•

Is the cause of the bug reproduced in another part of the program?
– Similar errors may be occurring in other parts of the program

•

What next bug might be introduced by the fix that I’m about to make?
– The source code (and even the design) should be studied to assess the
coupling of logic and data structures related to the fix

•

What could we have done to prevent this bug in the first place?
– This is the first step toward software quality assurance
– By correcting the process as well as the product, the bug will be removed
from the current program and may be eliminated from all future programs

54

Testing Tactics
- Testing fundamentals
- White-box testing
- Black-box testing
- Object-oriented testing methods

Characteristics of Testable
Software
•

Operable
– The better it works (i.e., better quality), the easier it is to test

•

Observable
– Incorrect output is easily identified; internal errors are automatically
detected

•

Controllable
– The states and variables of the software can be controlled directly by the
tester

•

Decomposable
– The software is built from independent modules that can be tested
independently

(more on next slide)

56

Characteristics of Testable
Software (continued)
•

Simple
– The program should exhibit functional, structural, and code simplicity

•

Stable
– Changes to the software during testing are infrequent and do not
invalidate existing tests

•

Understandable
– The architectural design is well understood; documentation is available
and organized

57

Test Characteristics
•

A good test has a high probability of finding an error
– The tester must understand the software and how it might fail

•

A good test is not redundant
– Testing time is limited; one test should not serve the same purpose as
another test

•

A good test should be “best of breed”
– Tests that have the highest likelihood of uncovering a whole class of
errors should be used

•

A good test should be neither too simple nor too complex
– Each test should be executed separately; combining a series of tests could
cause side effects and mask certain errors

58

Two Unit Testing Techniques
• Black-box testing
– Knowing the specified function that a product has been designed to perform, test
to see if that function is fully operational and error free
– Includes tests that are conducted at the software interface
– Not concerned with internal logical structure of the software

• White-box testing
– Knowing the internal workings of a product, test that all internal operations are
performed according to specifications and all internal components have been
exercised
– Involves tests that concentrate on close examination of procedural detail
– Logical paths through the software are tested
– Test cases exercise specific sets of conditions and loops

59

White-box Testing
•
•

Uses the control structure part of component-level design to derive the
test cases
These test cases
– Guarantee that all independent paths within a module have been exercised
at least once
– Exercise all logical decisions on their true and false sides
– Execute all loops at their boundaries and within their operational bounds
– Exercise internal data structures to ensure their validity

“Bugs lurk in corners and congregate at boundaries”
61

Basis Path Testing
•
•
•
•

White-box testing technique proposed by Tom McCabe
Enables the test case designer to derive a logical complexity measure
of a procedural design
Uses this measure as a guide for defining a basis set of execution paths
Test cases derived to exercise the basis set are guaranteed to execute
every statement in the program at least one time during testing

62

Flow Graph Notation
• A circle in a graph represents a node, which stands for a sequence of one
or more procedural statements
• A node containing a simple conditional expression is referred to as a
predicate node
– Each compound condition in a conditional expression containing one or more
Boolean operators (e.g., and, or) is represented by a separate predicate node
– A predicate node has two edges leading out from it (True and False)

• An edge, or a link, is a an arrow representing flow of control in a specific
direction
– An edge must start and terminate at a node
– An edge does not intersect or cross over another edge

• Areas bounded by a set of edges and nodes are called regions
• When counting regions, include the area outside the graph as a region,
too
63

Flow Graph Example

FLOW CHART

FLOW GRAPH

0

0

1

1

2

2

3
4
8

6

4

R2

5

7

9
11

R3

3

6
7

R4

R1

8

5

9
10

11

10

64

Independent Program Paths
•
•
•

Defined as a path through the program from the start node until the
end node that introduces at least one new set of processing statements
or a new condition (i.e., new nodes)
Must move along at least one edge that has not been traversed before
by a previous path
Basis set for flow graph on previous slide
–
–
–
–

•

Path 1: 0-1-11
Path 2: 0-1-2-3-4-5-10-1-11
Path 3: 0-1-2-3-6-8-9-10-1-11
Path 4: 0-1-2-3-6-7-9-10-1-11

The number of paths in the basis set is determined by the cyclomatic
complexity

65

Cyclomatic Complexity
•
•
•
•

Provides a quantitative measure of the logical complexity of a program
Defines the number of independent paths in the basis set
Provides an upper bound for the number of tests that must be conducted to
ensure all statements have been executed at least once
Can be computed three ways
– The number of regions
– V(G) = E – N + 2, where E is the number of edges and N is the number of
nodes in graph G
– V(G) = P + 1, where P is the number of predicate nodes in the flow graph G

•

Results in the following equations for the example flow graph
– Number of regions = 4
– V(G) = 14 edges – 12 nodes + 2 = 4
– V(G) = 3 predicate nodes + 1 = 4

66

Deriving the Basis Set and Test Cases
1)
2)
3)
4)

Using the design or code as a foundation, draw a corresponding
flow graph
Determine the cyclomatic complexity of the resultant flow graph
Determine a basis set of linearly independent paths
Prepare test cases that will force execution of each path in the basis
set

67

A Second Flow Graph Example

1
2
3
4

5
6
7
8
9
0

A: x++;
if (x > 999)
goto D;
if (x % 11 == 0)
goto B;
else goto A;

1
2
3

B: if (x % y == 0)
goto C;
else goto A;

4
5

C: printf("%dn", x);
goto A;

6
7
8

D: printf("End of listn");
return 0;
}

3

int functionY(void)
{
int x = 0;
int y = 19;

4
5
6
8

13

9

16

11

10

7

17

12
14
15

68

A Sample Function to Diagram and Analyze

1
2
3

int functionZ(int y)
{
int x = 0;

4
5
6
7
8
9
0
1
2
3
4
5
6
7
8
9

while (x <= (y * y))
{
if ((x % 11 == 0) &&
(x % y == 0))
{
printf(“%d”, x);
x++;
} // End if
else if ((x % 7 == 0) ||
(x % y == 1))
{
printf(“%d”, y);
x = x + 2;
} // End else
printf(“n”);
} // End while

0
1
2

printf("End of listn");
return 0;
} // End functionZ

69

A Sample Function to Diagram and Analyze

1
2
3

4
5
6
7
8
9
0
1
2
3
4
5
6
7
8
9

while (x <= (y * y))
{
if ((x % 11 == 0) &&
(x % y == 0))
{
printf(“%d”, x);
x++;
} // End if
else if ((x % 7 == 0) ||
(x % y == 1))
{
printf(“%d”, y);
x = x + 2;
} // End else
printf(“n”);
} // End while

0
1
2

printf("End of listn");
return 0;
} // End functionZ

3

int functionZ(int y)
{
int x = 0;

4
6
12

7

13

9
10

15
16
18
20
21

70

Loop Testing - General
•
•

A white-box testing technique that focuses exclusively on the validity
of loop constructs
Four different classes of loops exist
–
–
–
–

•

Simple loops
Nested loops
Concatenated loops
Unstructured loops

Testing occurs by varying the loop boundary values
– Examples:
for (i = 0; i < MAX_INDEX; i++)
while (currentTemp >= MINIMUM_TEMPERATURE)
71

Testing of Simple Loops
1)
2)
3)
4)
5)

Skip the loop entirely
Only one pass through the loop
Two passes through the loop
m passes through the loop, where m < n
n –1, n, n + 1 passes through the loop

‘n’ is the maximum number of allowable passes through the loop

72

Testing of Nested Loops
1)
2)

3)

4)

Start at the innermost loop; set all other loops to minimum values
Conduct simple loop tests for the innermost loop while holding the
outer loops at their minimum iteration parameter values; add other
tests for out-of-range or excluded values
Work outward, conducting tests for the next loop, but keeping all
other outer loops at minimum values and other nested loops to
“typical” values
Continue until all loops have been tested

73

Testing of Concatenated Loops
•
•

For independent loops, use the same approach as for simple loops
Otherwise, use the approach applied for nested loops

74

Testing of Unstructured Loops
•
•

Redesign the code to reflect the use of structured programming
practices
Depending on the resultant design, apply testing for simple loops,
nested loops, or concatenated loops

75

Black-box Testing
•
•
•
•
•

Complements white-box testing by uncovering different classes of
errors
Focuses on the functional requirements and the information domain of
the software
Used during the later stages of testing after white box testing has been
performed
The tester identifies a set of input conditions that will fully exercise all
functional requirements for a program
The test cases satisfy the following:
– Reduce, by a count greater than one, the number of additional test cases
that must be designed to achieve reasonable testing
– Tell us something about the presence or absence of classes of errors,
rather than an error associated only with the specific task at hand

77

Black-box Testing Categories
•
•
•
•
•

Incorrect or missing functions
Interface errors
Errors in data structures or external data base access
Behavior or performance errors
Initialization and termination errors

78

Questions answered by
Black-box Testing
•
•
•
•
•
•
•

How is functional validity tested?
How are system behavior and performance tested?
What classes of input will make good test cases?
Is the system particularly sensitive to certain input values?
How are the boundary values of a data class isolated?
What data rates and data volume can the system tolerate?
What effect will specific combinations of data have on system
operation?

79

Equivalence Partitioning
•
•
•
•
•

A black-box testing method that divides the input domain of a
program into classes of data from which test cases are derived
An ideal test case single-handedly uncovers a complete class of errors,
thereby reducing the total number of test cases that must be developed
Test case design is based on an evaluation of equivalence classes for
an input condition
An equivalence class represents a set of valid or invalid states for
input conditions
From each equivalence class, test cases are selected so that the largest
number of attributes of an equivalence class are exercise at once

80

Guidelines for Defining
Equivalence Classes
•

If an input condition specifies a range, one valid and two invalid equivalence
classes are defined
– Input range: 1 – 10

•

If an input condition requires a specific value, one valid and two invalid
equivalence classes are defined
– Input value: 250

•

Eq classes: {250}, {x < 250}, {x > 250}

If an input condition specifies a member of a set, one valid and one invalid
equivalence class are defined
– Input set: {-2.5, 7.3, 8.4}

•

Eq classes: {1..10}, {x < 1}, {x > 10}

Eq classes: {-2.5, 7.3, 8.4}, {any other x}

If an input condition is a Boolean value, one valid and one invalid class are
define
– Input: {true condition}

Eq classes: {true condition}, {false condition}

81

Boundary Value Analysis
•
•

A greater number of errors occur at the boundaries of the input domain
rather than in the "center"
Boundary value analysis is a test case design method that
complements equivalence partitioning
– It selects test cases at the edges of a class
– It derives test cases from both the input domain and output domain

82

Guidelines for
Boundary Value Analysis
•
•

•
•

1. If an input condition specifies a range bounded by values a and b,
test cases should be designed with values a and b as well as values just
above and just below a and b
2. If an input condition specifies a number of values, test case should
be developed that exercise the minimum and maximum numbers.
Values just above and just below the minimum and maximum are also
tested
Apply guidelines 1 and 2 to output conditions; produce output that
reflects the minimum and the maximum values expected; also test the
values just below and just above
If internal program data structures have prescribed boundaries (e.g., an
array), design a test case to exercise the data structure at its minimum
and maximum boundaries

83

Object-Oriented Testing Methods

Introduction
• It is necessary to test an object-oriented system at a variety of different levels
• The goal is to uncover errors that may occur as classes collaborate with one
another and subsystems communicate across architectural layers
– Testing begins "in the small" on methods within a class and on collaboration
between classes
– As class integration occurs, use-based testing and fault-based testing are applied
– Finally, use cases are used to uncover errors during the software validation phase

• Conventional test case design is driven by an input-process-output view of
software
• Object-oriented testing focuses on designing appropriate sequences of
methods to exercise the states of a class

85

Testing Implications for
•
•
•
•

Because attributes and methods are encapsulated in a class, testing
methods from outside of a class is generally unproductive
Testing requires reporting on the state of an object, yet encapsulation
can make this information somewhat difficult to obtain
Built-in methods should be provided to report the values of class
attributes in order to get a snapshot of the state of an object
Inheritance requires retesting of each new context of usage for a class
– If a subclass is used in an entirely different context than the super class,
the super class test cases will have little applicability and a new set of
tests must be designed

86

Applicability of Conventional
Testing Methods
•

White-box testing can be applied to the operations defined in a class
– Basis path testing and loop testing can help ensure that every statement in
an method has been tested

•

Black-box testing methods are also appropriate
– Use cases can provide useful input in the design of black-box tests

87

Fault-based Testing
•
•

The objective in fault-based testing is to design tests that have a high
likelihood of uncovering plausible faults
Fault-based testing begins with the analysis model
– The tester looks for plausible faults (i.e., aspects of the implementation of
the system that may result in defects)
– To determine whether these faults exist, test cases are designed to exercise
the design or code

•

If the analysis and design models can provide insight into what is
likely to go wrong, then fault-based testing can find a significant
number of errors

88

Fault-based Testing
(continued)
•
•

Integration testing looks for plausible faults in method calls or
message connections (i.e., client/server exchange)
Three types of faults are encountered in this context
– Unexpected result
– Wrong method or message used
– Incorrect invocation

•
•
•

The behavior of a method must be examined to determine the
occurrence of plausible faults as methods are invoked
Testing should exercise the attributes of an object to determine
whether proper values occur for distinct types of object behavior
The focus of integration testing is to determine whether errors exist in
the calling code, not the called code

89

Fault-based Testing
vs. Scenario-based Testing
•

Fault-based testing misses two main types of errors
– Incorrect specification: subsystem doesn't do what the user wants
– Interactions among subsystems: behavior of one subsystem creates
circumstances that cause another subsystem to fail

•

A solution to this problem is scenario-based testing
– It concentrates on what the user does, not what the product does
– This means capturing the tasks (via use cases) that the user has to
perform, then applying them as tests
– Scenario-based testing tends to exercise multiple subsystems in a single
test

90

Random Order Testing
(at the Class Level)
•

•
•

Certain methods in a class may constitute a minimum behavioral life
history of an object (e.g., open, seek, read, close); consequently, they
may have implicit order dependencies or expectations designed into
them
Using the methods for a class, a variety of method sequences are
generated randomly and then executed
The goal is to detect these order dependencies or expectations and make
appropriate adjustments to the design of the methods

91

Partition Testing (at the Class Level)
• Similar to equivalence partitioning for conventional software
• Methods are grouped based on one of three partitioning approaches
• State-based partitioning categorizes class methods based on their ability to
change the state of the class
– Tests are designed in a way that exercise methods that change state and those that
do not change state

• Attribute-based partitioning categorizes class methods based on the attributes
that they use
– Methods are partitioned into those that read an attribute, modify an attribute, or do
not reference the attribute at all

• Category-based partitioning categorizes class methods based on the generic
function that each performs
– Example categories are initialization methods, computational methods, and
termination methods

92

Multiple Class Testing
•
•

Class collaboration testing can be accomplished by applying random
testing, partition testing, scenario-based testing and behavioral
testing
The following sequence of steps can be used to generate multiple
class random test cases
1)
2)
3)
4)

For each client class, use the list of class methods to generate a series of
random test sequences; use these methods to send messages to server
classes
For each message that is generated, determine the collaborator class and
the corresponding method in the server object
For each method in the server object (invoked by messages from the
client object), determine the messages that it transmits
For each of these messages, determine the next level of methods that are
invoked and incorporate these into the test sequence

93

Tests Derived from
Behavior Models
• The state diagram for a class can be used to derive a sequence of tests that
will exercise the dynamic behavior of the class and the classes that
collaborate with it
• The test cases should be designed to achieve coverage of all states
– Method sequences should cause the object to transition through all allowable
states

• More test cases should be derived to ensure that all behaviors for the class
have been exercised based on the behavior life history of the object
• The state diagram can be traversed in a "breadth-first" approach by
exercising only a single transition at a time
– When a new transition is to be tested, only previously tested transitions are
used

94

Unit iv-testing-pune-university-sres-coe

More Related Content

What's hot

Viewers also liked

Similar to Unit iv-testing-pune-university-sres-coe

Recently uploaded

In this document

Unit iv-testing-pune-university-sres-coe