Exploratory Testing and Automated Testing: Create a Healthy Relationship

T24

Special
Topics

10/6/16
15:00

Exploratory
Testing
and
Automated

Testing:
Create
a
Healthy
Relationship

Presented
by:

Dawn
Jardine

Razorfish

Brought
to
you
by:

350
Corporate
Way,
Suite
400,
Orange
Park,
FL
32073

888-‐-‐-‐268-‐-‐-‐8770
·∙·∙
904-‐-‐-‐278-‐-‐-‐0524
-‐
info@techwell.com
-‐
http://www.starwest.techwell.com/

Dawn
Jardine

An
associate
quality
assurance
manager
for
Razorfish
in
Toronto,
Dawn
Jardine
has

worked
in
several
roles
in
her
career.
After
years
in
the
teaching
profession,
Dawn

changed
direction
and
entered
the
software
field
as
a
front-‐end
web
developer
for

an
eLearning
software
company.
Dawn
expanded
her
skills
and
expertise
to
include

instructional
design,
eventually
joining
the
QA
testing
team.
With
more
than
fifteen

years
of
experience,
she
has
participated
in
the
launch
of
several
eLearning
and

eCommerce
projects,
including
Canadian
Police
Knowledge
Network,
Canadian
Tire

Corporation,
Walmart
Canada,
and
American
Express
Membership
Rewards.

Passionate
about
quality
process
and
women
in
technology,
Dawn
can
be
found
on

Twitter
@dawnjardine.

9/21/16
1
Exploratory &
Automation Testing:
Create a Healthy Relationship
Dawn Jardine
2
Introduction

9/21/16
2
3
Toronto, Canada
4
1
2
3
4
The Role of Test Automation
Exploratory Testing and Its Role
The Mutual Relationship
How We Make it Work
Agenda

9/21/16
3
5
The Role of
Test Automation
6
Why use automation?

9/21/16
4
7
Automation can provide better coverage
•  for a large code base
•  when a live site is in maintenance mode
•  when it supplements manual testing
8
Automation can find defects earlier
•  finds defects introduced during sprint development
•  supports process of continuous test-and-fix in lower life cycle
•  reduces the cost of rework

9/21/16
5
9
Automation speeds delivery to client
•  continuous integration and delivery during sprint
•  achieve 100% regression test case execution
•  release small and release often
10
When do you
implement
automation?

9/21/16
6
11
The decision to use automation
•  current code base is growing
•  addition of new features and enhancements are increasing
•  completion time of manual test case execution is expanding
•  releases are occurring frequently
12
What do
you automate?

9/21/16
7
13
Candidates for automated tests
•  repeated tests within sprint development cycles
•  regression tests that are executed pre-release
•  sanity tests that are being performed during releases to
production
14
How do you implement automation?

9/21/16
8
15
Planning for automation
•  choose a valuable platform
•  involve your current QA team
•  hire QA developer(s)
•  keep the process agile
16
Exploratory Testing
and its Role

9/21/16
9
17
Where does exploratory testing fit?
18
“Insanity is doing the same
thing over and over again
and expecting different
results”
~Albert Einstein

9/21/16
10
19 James Bach, Exploratory Testing Explained
“An interactive process of
simultaneous learning,
test design, and test
execution.”
20
Does the QA team need specialized skills?
•  experience is an asset but not a
requirement
•  critical thinking and problem solving
•  autonomy and independence
•  creativity

9/21/16
11
21
Can you “explore” if you already know the way?
•  YES!
•  it’s not User Acceptance testing
•  there are many ways of navigating from Point A to Point B
22
The Mutual Relationship

9/21/16
12
23
“Assumptions are the
termites of relationships”
~Henry Winkler
24
Build on the strengths of your team
•  trust
•  communication and collaboration
•  empowerment
•  confidence

9/21/16
13
25
How We Make it Work
The challenges and successes
26
The story
•  online retail e-commerce solution in production
•  automation in place for pre-release regression
•  an established agile process in development and delivery
•  QA test coverage was still the bottleneck

9/21/16
14
27
Management approval
for change
•  respect and value of the QA team
•  don’t be afraid to innovate!
•  plan for smarter test coverage
28
Shifting the mindset
of the QA team
•  information sessions about exploratory testing
•  review of existing manual regression tests
•  created user stories for high level scenarios

9/21/16
15
29
Be creative, be organized and be brave!
•  mind-mapping sessions for the high level scenarios
•  manual and automation team participation
•  absence of test steps
•  traceability by recording the results
•  organized a trial run for the regression cycle
30
Gain stakeholders
support
•  business wants return on
investment
•  transparency is key
•  continually inform
•  provide metrics

9/21/16
16
31
Sample of metrics for status reports
Manual
Exploratory Tests
Desktop Browsers OS/Devices
Total
Desktop
Chrome
FF
IE11
MAC Safari
IOS8 Chrome
IOS9 Safari
IOS8 Safari
Android Chrome
#of Test Cases 91 9 15 10 12 14 9 12 10
Pass 91 9 15 10 12 14 9 12 10
Fail 0
Blocked 0 0 0 0 0 0 0 0 0
Unexecuted 0 0 0 0 0 0 0 0 0
Executed 91 9 15 10 12 14 9 12 10
Percentage ExecuHon
Complete 100% 100% 100% 100% 100% 100% 100% 100% 100%
32
Continued collaboration
•  ongoing audits of current and
future release work
•  review recorded results from
exploratory tests
•  Repeat, Repeat, Repeat!

9/21/16
17
33
Benefits of testing style
•  the QA team’s confidence level grows
•  the project management can stick to release timelines
•  the business benefits by staying competitive
34
May 2014
•  261 manually executed
step-by-step test cases
•  No automation in place
•  6 days x 4 QAs
January 2016
•  49 Exploratory Test cases
•  1140 total automated tests
covering 4 browsers
•  3 days x 2 QAs with focus
on Mobile
vs
Rewards & Results

9/21/16
18
35
What’s next?
•  automate the mobile web and mobile app regression tests
•  streamline current exploratory tests
•  change the format of our sprint functional tests
36
Conclusion

Exploratory Testing and Automated Testing: Create a Healthy Relationship

More Related Content

What's hot

Viewers also liked

Similar to Exploratory Testing and Automated Testing: Create a Healthy Relationship

More from TechWell

Recently uploaded

Exploratory Testing and Automated Testing: Create a Healthy Relationship