Introduction to White box testing

Introduction to White box
Testing
Aliaa Monier

Thank you all very much for coming today
&
hope you all have a pleasant journey through
white box testing

Let me introduce myself ………….

Agenda
•Introduction to white box testing
•White box testing techniques
•Junit 101
•Demo : Creating structured test cases using Eclipse IDE , Java, Junit and
Maven.
•Calculating code coverage using ECLEmma
•Automating White box testing

Introduction to White box Testing

Testing Techniques
Testing
Static Dynamic
Review Static analysis
Black-box
testing
Dynamic
analysis
Experience-
based testing
White box
testing
Defect-based
testing

What is white box testing?
White Box Testing is the testing of a software
solution's internal coding and infrastructure:
•By writing a set of white-box test cases that
exercise the paths in the code.
White box testing technique is used to:
•Help identify which line of code is actually
executed and which is not, which may indicate
that there is either missing logic or a typo.

White box testing is also
known as……
• Structure-based testing technique
• Glass-box testing technique

Why is it called White box testing ?
Because here the testers require
knowledge of how the software
is implemented.

How to encounter a bug in white box testing?
It involves testing a series of predefined inputs against expected outputs , so when
a specific input does not result in the expected output, you have encountered a bug.

White box testing Vs. Black box testing
In Black box testing, we test the software
from a user’s point of view.
In White box testing, we evaluates the
code and internal structure of the program.

White box testing Vs.
Unit testing
Unit testing is a part of white box testing.

White box testing Vs. Automation testing
In Automation testing, we automate the manual testing of the system functionality.
In White box testing, we evaluates the code and internal structure of the program.

What are the advantages of white box testing?
•It reveals errors in "hidden" code.
•It helps in optimizing the code by removing the extra
lines of code, which can bring in hidden defects.
Research has shown that with all of black box testing techniques , maybe as
much as 70 percent of all of the code in the system might never have been
executed once!
Not even once!

•In-depth knowledge about the programming language is necessary to perform
white box testing.
•It is not realistic to be able to test every single existing condition of the application
and some conditions will be untested.
•The tests focus on the software as it exists, and missing functionality may not be
discovered
What are the disadvantages of white box testing?

Why to use White box testing?
It is used to design structured test cases
with the aim of increasing the test
coverage.
Test coverage measurement
It is used to assess the amount of
testing performed by the structured test
cases.
It serves two purposes:
Structured test cases design

List Some white box testing techniques….
Techniques are used to design
structured test cases

What is test coverage
Test coverage measures the amount of testing
performed by a set of tests.
Coverage = (Number of coverage items
exercised/Total number of coverage items) * 100

What are the steps to perform
white box testing?
Understand the
source code
Agree on the
Benchmarks
Create and execute
the test cases
Preparing the
final report

What is the average sufficient
code coverage?

What are the levels of White box testing?
It can be used at all levels of testing:
•Component testing
•Component integration testing
•System and acceptance testing
•Regression testing

•A unit is a software component that cannot be subdivided into other components.
•White box test cases are written to examine whether the unit is coded correctly.
•Unit testing is important for ensuring the code is solid before it is integrated with
other code.
•Once the code is integrated into the code base, the cause of an observed failure
is more difficult to find.
At the level of component testing:

•Integration testing validates the interaction between software components.
•White box test cases are written to exercise the interfaces between the various
units.
At the level of integrated components:

•White box testing during regression testing is the use of recycled white box test
cases at the unit and integration testing levels
At the level of system testing:
•Ex : White box test cases are written to exercise the coverage of menu options.
At the level of regression testing:

What will be covered during today’s session?
White box testing at the level of Component testing
(Unit testing)

What do you verify in White Box Testing ?
White box testing involves the testing of the software code for the following:
Internal security holes
Broken or poorly structured paths in the source code
The functionality of conditional loops
Testing of each statement, condition and function
Data Flow analysis :The flow of specific inputs through the code
Dynamic Analysis : Memory leakages , wild pointers ….
Mutation testing

Internal security holes
Broken or poorly structured paths in the source code
The functionality of conditional loops
Testing of each statement, condition and function
Data Flow analysis :The flow of specific inputs through the code
Dynamic Analysis : Memory leakages , wild pointers ….
Mutation testing
What will be covered during today’s session?

So , we will cover White box testing
at the level of units
using
control flow analysis techniques

Control-flow testing is done through
control-flow graphs, to better understand
what a code module does.
Control-flow graphs give us a visual
representation of the structure of the code.
The Algorithm is:
•First convert a section of the code into
a control graph.
•Then analyze the possible paths through
the graph.

1 - Building Control-Flow Graphs:
1 - The process block
2 - The junction point
3 - A decision point
Control-flow testing
“Individual lines of code where no decisions are made
do not affect the control-flow and thus can be ignored.”
Main building blocks:

•It consists of a node (bubble or circle) with one path leading to it and one path
leading from it.
•It represents a chunk of code that executes sequentially.
•No decisions are made inside of it.
•Whether the process block has one line or a million lines of code,
we only need one test to execute it completely.
The process block

•It consists of a node (bubble or circle) with any number of different paths
leading into it and only one path leading out.
•No decisions are made in this block
The junction point

•It consists of a node with one input and two or more possible outputs.
•A decision to which way to go is made, and control-flow continues out that
path while ignoring all of the other possible choices.
A decision point

1 #include <stdio.h>
2 main ()
3 {
4 int i, n, f;
5 printf (“n = “);
6 scanf (“%d”, &n);
7 if (n < 0) {
8 printf (“Invalid: %dn”, n);
9 n= -1;
10 } else {
11 f = 1;
12 for (i = 1; i <= n; i++) {
13 f *= i;
14 }
15 printf(“d! = %dn”, n, f);
16 }
17 return n;
18 }
1 - Building Control-Flow Graphs – Example:
if
for
Line 4,5,6
Line 7,10
Line 11Line 8,9
Line 15Line 17
Line 12 Line 13

1.Statement coverage.
1.Decision (or branch) coverage.
1.Loop coverage.
1.Condition coverage.
1.Decision/condition coverage.
1.Modified condition/decision coverage.
1.Multiple-condition coverage.
1.Path coverage.
2 - analyze the possible paths through the graph:
Techniques:

Statement Coverage
•Statement is the line of code or instruction for the computer to understand and
act accordingly.
•“Statement Coverage”, is the method of validating that each line of code is
executed at least once.
•To achieve statement coverage, we pick test data that force the thread of
execution to go through each line of code that the system contains.

To calculate the level of coverage:
•We divide the number of code statements that are executed by the number of
code statements in the entire system.
•If the quotient is equal to one, we have statement coverage.
•Statement-level coverage is considered the least effective of all control-flow
techniques.
Statement Coverage

Statement Coverage
2 main ()
3 {
4 int i, n, f;
5 printf (“n = “);
6 scanf (“%d”, &n);
7 if (n < 0) {
9 n= -1;
10 } else {
11 f = 1;
12 for (i = 1; i <= n; i++) {
13 f *= i;
14 }
15 printf(“d! = %dn”, n, f);
16 }
17 return n;
18 }
if
for
Line 4,5,6
Line 7,10
Line 11Line 8,9
Line 15Line 17
Line 12 Line 13

To achieve statement coverage:
Test Values:
•n < 0 (blue arrows)
•n > 0 (green arrows)
Statement Coverage

1 z = 0;
2 if (a > b) then
3 z = 12;
4 Rep = 72 / z;
Statement Coverage
if
Line 1
Line 2
Line 3
Line 4
Exercise:
Test 1: Green arrows
a = 3, b = 2 >> Rep = 6
Test 2: Blue arrows
a =2, b = 3 >> Rep =? So , do we still another level of coverage?

•“Branch” in programming language is like the “IF statements”. If statement
has two branches: true and false.
•So in Branch coverage (also called Decision coverage), we validate that each
branch is executed at least once.
•To get to the decision level of coverage, every decision made by the code
must be tested both ways, TRUE and FALSE.
•Decision coverage guarantees statement coverage in the same code.
Decision Coverage (branch coverage)

2 main ()
3 {
4 int i, n, f;
5 printf (“n = “);
6 scanf (“%d”, &n);
7 if (n < 0) {
9 n= -1;
10 } else {
11 f = 1;
12 for (i = 1; i <= n; i++) {
13 f *= i;
14 }
15 printf(“d! = %dn”, n, f);
16 }
17 return n;
18 }
if
for
Line 4,5,6
Line 7,10
Line 11Line 8,9
Line 15Line 17
Line 12 Line 13

To achieve decision coverage:
Test Values:
•n < 0 (blue arrows)
•n > 0 (green arrows)
•n == 0 (orange arrows)

The basic minimum number of test cases tends to be two tests:
•Zero times through the loop
•One time through the loop
It’s preferred to test the loop:
•Zero times
•One time
•The maximum number of times it is expected to cycle (if you know how many
times that is likely to be).
“Exhaustive testing is impossible.”
Loop Coverage

We can cover the three point boundary values of the loop variable:
1.If possible, test a value that is one less than the expected minimum value the
loop can take. For example, if we expect to loop with the control variable going
from 0 to 100, try -1 and see what happens.
2.Try the minimum number of iterations—usually zero iterations. Occasionally,
there will be a positive number as the minimum.
3.Try one more than the minimum number.
4. Try to test one less than the maximum value.
5. Try to test the maximum number of loops.
6. Try to test one more than the maximum value.
Loop Coverage

•Condition coverage considers how a decision is made.
•Each decision predicate is made up of one or more simple “atomic” conditions,
each of which evaluates to a discrete Boolean value.
•These are logically combined to determine the final outcome of the decision.
•Each atomic condition must be evaluated both ways by the test cases to achieve
this level of coverage.
•
• "A and B"
Condition Coverage

•Decision Condition coverage specifies that testing must achieve condition coverage
and also requires that decision coverage also be satisfied.
•
•
“A and B”
Decision Condition Coverage

Assuming N unique atomic conditions, MC/DC can usually be achieved in N+1
unique test cases.
MC/DC achieves Decision Condition coverage, but then requires the following
also be fulfilled:
•At least one test where the decision outcome would change if the atomic
condition X were TRUE
•At least one test where the decision outcome would change if the atomic
condition X were FALSE
•Each different atomic condition has tests that meet requirements 1 and 2.
•
•
Modified Condition/Decision Coverage (MC/DC)

Modified Condition/Decision Coverage (MC/DC)
(A or B) and C

•It might be required to test all possible combinations of values that a decision
may contain.
•The number of required tests is dependent on the number of atomic conditions in
the decision statement and can be determined by calculating where n is the
number of uncoupled atomic conditions
•
•
Multiple Condition Testing

Multiple Condition Testing
(A or B) and C

•Path testing consists of identifying paths through the code and then creating tests
to cover them.
•Path coverage tests all the paths of the program. This is a comprehensive
technique which ensures that all the paths of the program are traversed at least
once.
• Path Coverage is even more powerful than Branch coverage.
•In any non-trivial system, however, the number of test cases could become
excessively large due to the nature of looping structures.
Path Coverage

•Pick the first path as the simplest, functionally sensible path from entry to exit.
•Pick each additional path as a small variation on the previous path. Try to
change only one branch in the path that is different for each successive test.
•Favor short paths over long paths when possible.
•Favor paths that make functional sense over ones that do not.
•Pick paths that don't make functional sense only when required for coverage.
•Use intuition when choosing paths (i.e., which paths are most likely to be
executed).
By setting aside the issue of indefinite looping
Path Coverage >> Basic Path Coverage

Cyclomatic Compexity
C = #R + 1
C = 4 + 1
C = 5
Or
C = #E - #N + 2
C = 9 - 6 + 2
C = 5

Cyclomatic complexity is the minimum
number of test cases covering the graph
Creating sufficient tests to cover all paths
(disregarding loops) guarantees that both
statement and branch coverage is achieved.
Basis Paths
•ABF
•ABCF
•ABCDEF (1)
•ABCDEF (2)
•ABCDEEF
Basis Tests
•-5 , 2 >> -1
•2 , -5 >> -1
•16 , 8 >> 8
•4 , 8 >> 4
•20 , 8 >> 4

1. Create a directed control-flow graph for this code.
2. Calculate the cyclomatic complexity.
3. List the basis tests that could be run.

C = # Regions + 1
C = 3 + 1
C = 4
C = #E - #N + 2
C = 7 - 5 + 2
C = 4
C = # decisions + 1
C = 3 + 1
C = 4
1. ABE
2. ABCE
3. ABCD(D1)CE
4. ABCD(D2)CE
Count the number of places where decisions
are made and add 1
Test Cases:

•Class Under Test
•Unit Test class
•Unit Test class – Code Explanation
•The most popular assert methods
•Annotations
•Demo
Maven
Class Under Test
Junit Class
Running Junit test case
•Test to Fail
•Different ways to run JUnit classes
•How to write a good JUnit test class
Junit 101

package MavenDemo.WhiteBoxTesting;
public class Factorial
{
public int CalcFactorial(int n){
int i , f;
if(n <0)
{
n = -1;
}
else
{
f = 1;
for(i = 1 ; i <= n ; i++)
{
f *= i;
}
n = f;
}
return n;
}
}
•-1 ! >> -1
•0 ! >> 1
•3 ! >> 6
Junit 101 - Class Under Test
Test Cases :

import static org.junit.Assert.*;
import org.junit.Test;
public class FactorialTest {
Factorial testObject = new Factorial();
@Test
public void CalcFactOfNLessThan0() {
int expResult = -1;
int actResult = testObject.CalcFactorial(-1);
assertEquals(expResult , actResult);
}
@Test
public void CalcFactOfNEqual0() {
int expResult = 1;
int actResult = testObject.CalcFactorial(0);
}
@Test
public void CalcFactOfNGreaterThan0() {
int expResult = 6;
int actResult = testObject.CalcFactorial(3);
}
Junit 101 - Unit Test class

JUnit test cases should be in the same package of the classes under test

This will import all the assertion statements. These assertion will be used in the test to
verify test logics.
This will import the @Test annotation of Junits, this annotation tells the Junit framework
that this method is a test method.

public class FactorialTest
The test class must be public.

We created a Class variable called testObject of the type Factorial. It is because we
have to test Factorial class.
Factorial testObject = new Factorial();

@Test
•The ‘@Test‘ annotation.
•All the tests in Junit4 has a @Test annotation that should be put on before the test
method.
•This helps the Junit framework to identify tests inside a test class.

public void CalcFactOfNLessThan0()
•Here we have created a method inside the class called CalcFactOfNLessThan0().
•This method will test the add functionality of Factorial class ,and will cover the test
case of N < 0
•The method should be public , returns void and with no arguments.

assertEquals(expected,actual)
•Asserts mean verifying the values.
•Assert can be of Object type, Boolean type, Integer type or any data type.
•It simply verify the actual value with the expected value.
•An assert method is silent when its proposition succeeds but throws an exception if the
proposition fails.
assertEquals(1 , testObject.CalcFactorial(0));
•0! = 1.
•Expected value is 1.
•Actual value is the output from the functional call testObject.CalcFactorial(0)

assertXXX method What it’s used for
assertEquals("message", A, B) Asserts the equality of objects A and B.
When an assertion fails, a message is displayed
assertArrayEquals("message", A, B) Asserts the equality of the A and B arrays.
assertTrue("message", A) Asserts that the A condition is true.
assertFalse("message", A) Asserts that the A condition is false.
assertNotNull("message", A) Asserts that the A object isn’t null.
assertNull("message", A) Asserts that the A object is null.
Fail(“message”) Fail a test with a message
Junit 101 - The most popular assert methods

annotation description
@Test The Test annotation tells JUnit that the public void method to which it is attached
can be run as a test case.
@Before When writing tests, it is common to find that several tests need similar objects
created before they can run.
@After If you allocate external resources in a Before method you need to release them
after the test runs.
@BeforeClass Sometimes several tests need to share computationally expensive setup (like
logging into a database)
@AfterClass If you allocate expensive external resources in a BeforeClass method you need
to release them after all the tests in the class have run.
@Ignore Sometimes you want to temporarily disable a test or a group of tests.
Junit 101 - Annotations

Creating a new Maven Project
Junit 101 - Demo

Creating a new java class – Class Under Test
Junit 101 - Demo

Creating a new Junit Class
Junit 101 - Demo

Running a JUnit Test Class
Junit 101 - Demo

Test to Fail
Junit 101 - Demo
package MavenDemo.NegativeTesting;
public class Negative {
public int calcNegative(int a , int b)
{
int z = 0;
int ret;
if(a>b)
{
z = 12;
}
ret = 72 / z;
return ret;
}
}
if
Line 1
Line 2
Line 3
Line 4
Test 1: Green arrows
a = 3, b = 2 >> Rep = 6
Test 2: Blue arrows
a =2, b = 3 >> Rep =?

package MavenDemo.NegativeTesting;
public class NegativeTest {
Negative obj = new Negative();
@Test
public void CalcTest() {
int expResult = 6;
int actResult = obj.calcNegative(3, 2);
}
@Test
public void CalcTest1() {
int expResult = 12;
int actResult = obj.calcNegative(2, 3);
}
}
Test to Fail
Junit 101 - Demo

Test to Fail
Junit 101 - Demo
Code coverage with Test 1
Junit result after adding Test 2

Test Class (Test case) : A class that contains one or more tests represented by methods
annotated with @Test.
-When we run a JUnit test case one by one
Suite : A group of test classes.
- When we run JUnit test cases all together.
Junit 101 - Different ways to run JUnit classes

Test Suite

import org.junit.runner.RunWith;
import org.junit.runners.Suite;
import org.junit.runners.Suite.SuiteClasses;
@RunWith(Suite.class)
@SuiteClasses({ FactorialTest.class, FactorialTest2.class })
public class AllTests {
}
Test Suite

JUnit test suite should be in the same package of the classes under test
Test Suite

This will import :
@RunWith
@SuiteClasses
import org.junit.runner.RunWith;
import org.junit.runners.Suite;
import org.junit.runners.Suite.SuiteClasses;
Test Suite

@SuiteClasses({ FactorialTest.class,
FactorialTest2.class })
public class AllTests {
}
- We declared an empty class AllTests, but annotated it with :
@SuiteClasses({})
Test Suite

•@RunWith : annotation tells the Junit runner that the this class (AllTests) will not use the
default Junit runner but will like to delegate the run to the class mentioned in the parameter
of @RunWith annotation which is called Suite.
Suite is a class in the Junit framework which helps us run build test by adding multiple test
classes.
•@SuiteClasses({FactorialTest.class, FactorialTest2.class }) :
The Suite class should add Junit test classes named FactorialTest andFactorialTest2.
These two tests are grouped together by this statement and while running tests will be
executed together.
Test Suite

Junit 101 - How to write good JUnit test cases
•Always Write Isolated Test Cases (Order of execution)
•Test One Thing Only in One Test Case
•Use a Single Assert Method per Test Case
•Use a Naming Convention for Test Cases
•Use the Arrange-Act-Assert Style or Given-When-Then Style
•Use Descriptive Messages in Assert Methods
•Measure Code Coverage to Find Missing Test Cases

Feature Atlassian Clover Cobertura JaCoCo
Statement coverage
Branch coverage
MC/DC coverage
Path Coverage
Global coverage
HTML Report
PDF Report
JUnit Framework Support
TestNG Framework
Support
Eclipse IDE Integration
Maven build tool
Integration
Free
JaCoCo
Code Coverage - Coverage tools comparisons

EclEmma is a free Java code coverage tool for Eclipse based on the JaCoCo code
coverage library
Code Coverage - EclEmma
How to install:

How to generate code coverage:

Code coverage report:

EclEmma highights covered code with green background

Tools are for automating test cases generation:
Tools are Programming Language specific:
•PEX (.Net)
•KLEE (C)
•AgitarOne (Java)
Tools for automating white-box testing

AutomaticJUnitCreator – For Java

AutomaticJUnitCreator – How to use >> Just Double Click the Jar file

AutomaticJUnitCreator – How to use >> browse for your source code file
and click Finish

AutomaticJUnitCreator – How to use >> source code is added and test
cases are generated.

AutomaticJUnitCreator – How to use >> Simple control flow graph is
generated.

AutomaticJUnitCreator – How to use >> Double click the test case to set the
expected and actual values.

AutomaticJUnitCreator – How to use >> Set the expected and actual values
and click Done button.

AutomaticJUnitCreator – How to use >> From main window , click export test.

Relying only on black box testing is not sufficient for maximum test coverage.
We need to have combination of both black box and white box testing
techniques to cover maximum defects.
Summary

References
ISTQB Technical Test Analyst syllabus
ISTQB Technical Test Analyst – Book Part3
Udemy – Junit training
Webinar – whitebox testing matrix
http://www.guru99.com/white-box-testing.html
https://sites.google.com/site/markussprunck/blog-1/unit-testing-best-practices
https://msdn.microsoft.com/en-us/library/dd537628.aspx
http://www.codeproject.com/Articles/5019/Advanced-Unit-Testing-Part-I-Overview
http://www.cafeaulait.org/slides/albany/codecoverage/Measuring_JUnit_Code_Coverage.html
https://github.com/junit-team/junit4/wiki/Getting-started
https://www.javacodegeeks.com/2014/11/junit-tutorial-unit-testing.html
http://javarevisited.blogspot.com.eg/2014/08/top-5-books-to-learn-unit-testing-junit-tdd-Java-programmers.html
http://crunchify.com/what-is-the-best-code-coverage-plugin-you-should-use-in-eclipse-ide/
https://confluence.atlassian.com/display/CLOVER/Comparison+of+code+coverage+tools
http://eclemma.org/
https://dzone.com/articles/7-popular-unit-test-naming
http://www.codeaffine.com/2014/03/17/getting-junit-test-names-right/
http://www.codeproject.com/Articles/31141/Getting-started-with-automated-white-box-testing-a
https://www.udemy.com/junit-tutorial-for-beginners-with-java-examples/learn/v4/content
https://www.udacity.com/course/viewer#!/c-433598568/c-ud805/l-3670758553/m-3665498599
http://www.qatestingtools.com/whitebox-testing
http://www.agitar.com/solutions/products/agitarone.html
http://read.pudn.com/downloads162/sourcecode/java/739580/install.pdf
http://www.agitar.com/downloads/demos/junit_gen/junitGen.html
http://www.agitar.com/solutions/products/agitarone_demos.html
http://rbcs-us.com/resources/category/code-coverage
http://rbcs-us.com/resources/category/coverage

References
http://istqbexamcertification.com/what-is-structure-based-technique-in-software-testing/
http://istqbexamcertification.com/what-is-white-box-or-structure-based-or-structural-testing-techniques/
http://www.tutorialspoint.com/software_testing_dictionary/white_box_testing.htm
http://stackoverflow.com/questions/15032219/java-whitebox-testing
http://www.slideshare.net/op_khuong_nguyen/test-design-techniques-structured-and-experiencedbased-
techniques
https://www.udacity.com/course/viewer#!/c-cs258/l-48508434/m-48735047
https://en.wikipedia.org/wiki/White-box_testing
http://agile.csc.ncsu.edu/SEMaterials/WhiteBox.pdf
http://www.drdobbs.com/tools/white-box-testing/184404030?pgno=1
http://www.softwaretestinghelp.com/white-box-testing-techniques-with-example/
http://www.qatestingtools.com/whitebox-testing
http://ecomputernotes.com/software-engineering/testing-techniques
https://en.wikipedia.org/wiki/White-box_testing
https://www.st.cs.uni-saarland.de/edu/automatedtestingverification12/slides/04-StructuralDataFlowTesting.pdf
http://www.eecs.yorku.ca/course_archive/2006-07/W/4313/Slides/Module07-PathTestingCoverage.pdf
https://www.cs.drexel.edu/~spiros/teaching/CS576/slides/4.data-testing.pdf
http://www.testingbrain.com/whitebox/white-box-testing.html
http://www.guru99.com/white-box-testing.html
http://agile.csc.ncsu.edu/SEMaterials/WhiteBox.pdf
https://www.istqb.guru/how-to-calculate-statement-branchdecision-and-path-coverage-for-istqb-exam-purpose/
https://www.youtube.com/watch?v=3bJcvBLJViQ
http://www.softwaretestinghelp.com/white-box-testing/
http://www.softwaretestinghelp.com/white-box-testing-techniques-with-example/
http://programmers.stackexchange.com/questions/27491/black-box-or-white-box-testing-which-do-you-do-first

Introduction to White box testing

More Related Content

What's hot

Similar to Introduction to White box testing

Introduction to White box testing

Editor's Notes