Integrate
Test Activities
in
Agile Projects
|
Integrate
Test Activities in
Agile Projects
Rik Marselis
23 July 2014
Live webinar
2
Based on the EuroSTAR eBook by Leo van der Aalst & Rik Marselis
|
Agile is always QD !!
Agile = Quality Development
Agile = Quick Development
Agile = Quick & Dirty
Agile = Quite a Disaster
What is your QD ???
3
|
Does the Agile
Manifesto have good
footholds for testing?
4
|
Does the Agile Manifesto have good
footholds for testing?
5
|
This is what the Scrum Guide states about
quality and testing
6
|
Benefits of a structured approach to
testing in Agile projects
• Insight into and advice about risks related to the quality
of the system under test
• Defects found and tracked at an early stage
• Insight into root cause of defects
• Defects prevented
• Test products, such as test cases, can be reused
• The test process is transparent and controllable
• The automation of testing tasks (e.g. test specification
or test execution) can be pulled earlier into the project.
7
|
What are the roles of
the Product Owner
and the Scrum Master
in testing?
8
|
An Agile team consists of “developers”
9
So specialized testers are no longer needed?
Development
team
Scrum
master
Product
Owner
Deve-
loper
Scrum guide:
• Development teams are cross-functional,
with all of the skills necessary
• No sub-teams regardless of domains like
business analysis or testing
Testing  one of the skills
Scrum
team
|
What is the role of the Product Owner?
(related to the quality risks)
Set the criteria for the
level of quality
that is needed to reach a
fit for purpose business solution
at an
acceptable level of risk
10Focus on Quality and Risk
|
What is the role of the Product Owner?
Accept the result (or not)
This may need acceptance testing
11
|
What is the role of the Scrum Master in
quality control and testing?
Scrum guide:
• The Scrum Master helps the development team to create
high-value products
 assists the team in achieving the right quality level
• A Scrum Master can detect incomplete transparency by
inspecting the artifacts,
sensing patterns,
listening closely to what is being said, and
detecting differences between expected and real results
 this sounds like quality control, doesn’t it?
By applying early quality measures the Scrum team makes
sure to minimize rework.
12
|
Which
team member(s)
perform(s) the
testing activities
and when?
13
|
Which team member(s) perform(s) the
testing activities and when?
14
Development
team
Scrum
master
Product
Owner
Deve-
loper
Every team member takes
her/his share of testing tasks
(and the Product Owner and
Scrum Master also take part to
a certain extent)
It is advisable to have a
professional tester in the team
to support the other members
with quality & testing tasks,
e.g. risk analysis, reviewing,
create test cases, etc.
|
Collaboration…
15
|
Model Based Testing
In Agile projects time pressure is felt even more than in
tradional IT
So creating test cases must be done as easy as possible
Let the analysis/design-specialist of the team create
“models” (e.g. flowcharts, UML, etc.)
Use a Model Based Testing tool to derive test cases.
(and sometimes the tester first has to translate a “white-board-
model” to a useable format)
16
|
Exploratory Testing
Creating test cases in an automated way is great
BUT…
In order to deal with the challenges of real life
Testing also must be a creative activity
SO …
Exploring is essential.
Various ways of Exploratory testing are known,
As a minimum: Use charters and logs.
And remember Exploratory Testing is NOT Ad Hoc Testing
!
17
|
Collaboration across the application
lifecycle: DevOps
18
Everybody
works
together on
all activities
of the
Application
Lifecycle:
No more
silo’s!!
|
Support for Agile testing needed?
 When every Agile team has no more than one tester
how can you guarantee all necessary skills are
available?
 Organize support, for example on subjects like:
Test Automation (execution, test case creation, …)
Non-functional testing (security, performance, …)
Test Management (reporting, defect management,…)
 In larger organizations a central support team can be
created, with e.g. the “test manager new style” and the
test tooling expert
 In smaller organizations ad hoc support can be
obtained from external entities
19
|
How to deal with the
test strategy?
20
|
The activities of testing
21Traditionally these were phases, but they don’t need to be fully sequential
Pr S E C
I
Ctrl
Pl
Planning Setting up and maintaining Infrastructure
Preparation Specification Execution Completion
Control
|
Testing activities in Scrum
22
Source: TMap NEXT in Scrum
|
How to deal with the test strategy?
Traditional approach:
Based on the Test Strategy, allocate Test
Units and Test Techniques
In Agile/Scrum:
Define the “project test strategy” in the
project planning (sprint 0)
Define the “sprint test strategy” in the
sprint planning.
Be aware that the strategy must be adapted to
changed circumstances whenever necessary.
And, very important:
- Regression testing strategy and
- Test automation strategy
23
A test unit is a
collection of
processes,
transactions
and/or functions
that are tested
collectively.
So this may be
anything from a
tiny function to a
complete
business process
depending on
your goal.
|
Focus on Quality and Risk
24
Overview
testing
activities of
TMap NEXT
in Scrum
Evaluate
backlog items
and
communicate
obscurities with
product owner
Determine product
risk of each backlog
item and record it on
the (story) card
(input for planning
poker, assigning
story points)
|
Test Strategy: Early quality measures
“This requirement was not totally correct” is quite
disturbing in tradional IT, because unplanned rework has
to be done.
In Agile IT having the right quality of artifacts is even more
important, since the difference between achieving your
sprint goals or not is in having adequate quality.
So you need early quality measures:
• A clear and good Definition of Done
• Reviews of intermediate- and end-products
• Checklists to help not to forget important things
• Standards to help work in an efficient manner
25
|
Regression testing
In every sprint the number of products increases. You
must make sure products of previous sprints are still OK.
Thus  regression testing
In sprint “n”:
• Define test cases for new and changed products
• Manually execute these test cases
• Keep track of test cases that must be in regression test
In sprint “n+1”
• Automate the regression test cases
• Execute the complete regression test set
26
Programmers usually are very good in programming test tools.
At the start of sprint “n+1” they can very wel do a regression test task
|
Do we still distinguish
test levels?
&
How to go about with
end-to-end-testing?
27
|
Separate test-sprints?
28No separate testing !!!
Development
Testing
Scrum guide:
No sub-teams
|
Test levels are no longer “levels”
In the dutch language test levels are called “testsoort”
which translates to “test variety”
And this exactly says what we still need in Agile:
Various scopes of testing
The pitfall of Agile teams is to focus on unit testing only.
Or to focus on functional testing only.
Keep in mind the objectives of all test levels when defining
and executing the tests in an Agile team.
29
|
But how to deal with
End-to-end testing?
30
System 1
Business process (complete cycle from customer to customer)
System 2 System 3 System 4 System 5
Existing
unchanged
Existing
uncha nged Scope
E2Etes
t
|
Combine lifecycle models pragmatically
31
Demand
Supply
e.g. E2E-test
|
About sustainability…
32
Conclusion: You can’t sprint all the time
Agile principle:
Agile processes promote sustainable development.
The sponsors, developers, and users should be able
to maintain a constant pace indefinitely.
(prevent having a team-burnout; implement the
Agile principles in a proper way!)
How long does the 100 meter sprint take?
100 meter sprint  +10 seconds
How long does it take to sprint a marathon?
42 kilometer sprinting  +70 minutes
What is the world record for a marathon?
Marathon record  +120 minutes
|
For effective and sustainable testing:
 Integrate all activities including testing
(for as far as possible)
 Testing is primarily a task for all team members
(not necessarily a separate function)
 Remember to still have your different views on testing
(test levels / test varieties)
 Early quality is more important than ever
(do reviews to get it right first time!)
33Rik Marselis, 23 July 2014
|
Questions ?  TestHuddle.com
34For questions & answers: join us at www.testhuddle.com
www.eurostarconferences.com www.ict-books.com
|
Rik Marselis
35
Management Consultant Quality & Testing at
Over 30 years IT experience, of which 15 years quality & testing
Advisor, process-improver & coach at many organizations
Prince2 Practitioner, CMMI and CISA
Trainer for various courses, e.g. Agile testing
TMap, TPI en ISTQB accredited
Research  Author of various books & articles
Fellow of SogetiLabs, Speaker at various conferences
Chairman (association of testers, 1600 members)
@rikmarselis

Integrate Test Activities in Agile

  • 1.
  • 2.
    | Integrate Test Activities in AgileProjects Rik Marselis 23 July 2014 Live webinar 2 Based on the EuroSTAR eBook by Leo van der Aalst & Rik Marselis
  • 3.
    | Agile is alwaysQD !! Agile = Quality Development Agile = Quick Development Agile = Quick & Dirty Agile = Quite a Disaster What is your QD ??? 3
  • 4.
    | Does the Agile Manifestohave good footholds for testing? 4
  • 5.
    | Does the AgileManifesto have good footholds for testing? 5
  • 6.
    | This is whatthe Scrum Guide states about quality and testing 6
  • 7.
    | Benefits of astructured approach to testing in Agile projects • Insight into and advice about risks related to the quality of the system under test • Defects found and tracked at an early stage • Insight into root cause of defects • Defects prevented • Test products, such as test cases, can be reused • The test process is transparent and controllable • The automation of testing tasks (e.g. test specification or test execution) can be pulled earlier into the project. 7
  • 8.
    | What are theroles of the Product Owner and the Scrum Master in testing? 8
  • 9.
    | An Agile teamconsists of “developers” 9 So specialized testers are no longer needed? Development team Scrum master Product Owner Deve- loper Scrum guide: • Development teams are cross-functional, with all of the skills necessary • No sub-teams regardless of domains like business analysis or testing Testing  one of the skills Scrum team
  • 10.
    | What is therole of the Product Owner? (related to the quality risks) Set the criteria for the level of quality that is needed to reach a fit for purpose business solution at an acceptable level of risk 10Focus on Quality and Risk
  • 11.
    | What is therole of the Product Owner? Accept the result (or not) This may need acceptance testing 11
  • 12.
    | What is therole of the Scrum Master in quality control and testing? Scrum guide: • The Scrum Master helps the development team to create high-value products  assists the team in achieving the right quality level • A Scrum Master can detect incomplete transparency by inspecting the artifacts, sensing patterns, listening closely to what is being said, and detecting differences between expected and real results  this sounds like quality control, doesn’t it? By applying early quality measures the Scrum team makes sure to minimize rework. 12
  • 13.
  • 14.
    | Which team member(s)perform(s) the testing activities and when? 14 Development team Scrum master Product Owner Deve- loper Every team member takes her/his share of testing tasks (and the Product Owner and Scrum Master also take part to a certain extent) It is advisable to have a professional tester in the team to support the other members with quality & testing tasks, e.g. risk analysis, reviewing, create test cases, etc.
  • 15.
  • 16.
    | Model Based Testing InAgile projects time pressure is felt even more than in tradional IT So creating test cases must be done as easy as possible Let the analysis/design-specialist of the team create “models” (e.g. flowcharts, UML, etc.) Use a Model Based Testing tool to derive test cases. (and sometimes the tester first has to translate a “white-board- model” to a useable format) 16
  • 17.
    | Exploratory Testing Creating testcases in an automated way is great BUT… In order to deal with the challenges of real life Testing also must be a creative activity SO … Exploring is essential. Various ways of Exploratory testing are known, As a minimum: Use charters and logs. And remember Exploratory Testing is NOT Ad Hoc Testing ! 17
  • 18.
    | Collaboration across theapplication lifecycle: DevOps 18 Everybody works together on all activities of the Application Lifecycle: No more silo’s!!
  • 19.
    | Support for Agiletesting needed?  When every Agile team has no more than one tester how can you guarantee all necessary skills are available?  Organize support, for example on subjects like: Test Automation (execution, test case creation, …) Non-functional testing (security, performance, …) Test Management (reporting, defect management,…)  In larger organizations a central support team can be created, with e.g. the “test manager new style” and the test tooling expert  In smaller organizations ad hoc support can be obtained from external entities 19
  • 20.
    | How to dealwith the test strategy? 20
  • 21.
    | The activities oftesting 21Traditionally these were phases, but they don’t need to be fully sequential Pr S E C I Ctrl Pl Planning Setting up and maintaining Infrastructure Preparation Specification Execution Completion Control
  • 22.
    | Testing activities inScrum 22 Source: TMap NEXT in Scrum
  • 23.
    | How to dealwith the test strategy? Traditional approach: Based on the Test Strategy, allocate Test Units and Test Techniques In Agile/Scrum: Define the “project test strategy” in the project planning (sprint 0) Define the “sprint test strategy” in the sprint planning. Be aware that the strategy must be adapted to changed circumstances whenever necessary. And, very important: - Regression testing strategy and - Test automation strategy 23 A test unit is a collection of processes, transactions and/or functions that are tested collectively. So this may be anything from a tiny function to a complete business process depending on your goal.
  • 24.
    | Focus on Qualityand Risk 24 Overview testing activities of TMap NEXT in Scrum Evaluate backlog items and communicate obscurities with product owner Determine product risk of each backlog item and record it on the (story) card (input for planning poker, assigning story points)
  • 25.
    | Test Strategy: Earlyquality measures “This requirement was not totally correct” is quite disturbing in tradional IT, because unplanned rework has to be done. In Agile IT having the right quality of artifacts is even more important, since the difference between achieving your sprint goals or not is in having adequate quality. So you need early quality measures: • A clear and good Definition of Done • Reviews of intermediate- and end-products • Checklists to help not to forget important things • Standards to help work in an efficient manner 25
  • 26.
    | Regression testing In everysprint the number of products increases. You must make sure products of previous sprints are still OK. Thus  regression testing In sprint “n”: • Define test cases for new and changed products • Manually execute these test cases • Keep track of test cases that must be in regression test In sprint “n+1” • Automate the regression test cases • Execute the complete regression test set 26 Programmers usually are very good in programming test tools. At the start of sprint “n+1” they can very wel do a regression test task
  • 27.
    | Do we stilldistinguish test levels? & How to go about with end-to-end-testing? 27
  • 28.
    | Separate test-sprints? 28No separatetesting !!! Development Testing Scrum guide: No sub-teams
  • 29.
    | Test levels areno longer “levels” In the dutch language test levels are called “testsoort” which translates to “test variety” And this exactly says what we still need in Agile: Various scopes of testing The pitfall of Agile teams is to focus on unit testing only. Or to focus on functional testing only. Keep in mind the objectives of all test levels when defining and executing the tests in an Agile team. 29
  • 30.
    | But how todeal with End-to-end testing? 30 System 1 Business process (complete cycle from customer to customer) System 2 System 3 System 4 System 5 Existing unchanged Existing uncha nged Scope E2Etes t
  • 31.
    | Combine lifecycle modelspragmatically 31 Demand Supply e.g. E2E-test
  • 32.
    | About sustainability… 32 Conclusion: Youcan’t sprint all the time Agile principle: Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely. (prevent having a team-burnout; implement the Agile principles in a proper way!) How long does the 100 meter sprint take? 100 meter sprint  +10 seconds How long does it take to sprint a marathon? 42 kilometer sprinting  +70 minutes What is the world record for a marathon? Marathon record  +120 minutes
  • 33.
    | For effective andsustainable testing:  Integrate all activities including testing (for as far as possible)  Testing is primarily a task for all team members (not necessarily a separate function)  Remember to still have your different views on testing (test levels / test varieties)  Early quality is more important than ever (do reviews to get it right first time!) 33Rik Marselis, 23 July 2014
  • 34.
    | Questions ? TestHuddle.com 34For questions & answers: join us at www.testhuddle.com www.eurostarconferences.com www.ict-books.com
  • 35.
    | Rik Marselis 35 Management ConsultantQuality & Testing at Over 30 years IT experience, of which 15 years quality & testing Advisor, process-improver & coach at many organizations Prince2 Practitioner, CMMI and CISA Trainer for various courses, e.g. Agile testing TMap, TPI en ISTQB accredited Research  Author of various books & articles Fellow of SogetiLabs, Speaker at various conferences Chairman (association of testers, 1600 members) @rikmarselis

Editor's Notes

  • #2 Integrate in Agile Process, or maybe better: in Agile teams Webinar Details Presenter: Rik Marselis Time: 2PM (BST) Date: Wednesday 23rd July 2014 In traditional testing a tester spreads the testing activities over various phases. To be able to effectively test in Agile projects the test activities should be properly integrated in the Agile approach. Leo van der Aalst and Rik Marselis published an EuroSTAR eBook on this topic. They start with a vision on Agile testing and, based on that, describe how to effectively integrate test activities in the Scrum approach. In this webinar Rik will go into subjects like "Does the Agile Manifesto have good footholds for testing?", "What is the role of the Product Owner and the Scrum Master", "Which team member(s) perform(s) the testing activities and when?", "How to deal with the test strategy?", "Do we still distinguish test levels" and "how to go about with end-to-end testing?". Very important is the automation of testing activities, not just of the execution but also the specification of test cases. During the webinar Rik demonstrates that the knowledge and expertise of the well-known test methodology TMap is a very solid basis for integrating test activities in Scrum events of an Agile project. Also Rik will mention the link with ISTQB based on the new Agile Extension and the collaboration across the application lifecycle as advocated by DevOps. Key Takeaways: 1) Be adaptive 2) Use a risk-based approach 3) Testing activities must be automated as much as possible
  • #3 Leo van der Aalst bedanken Or better: in Agile teams?
  • #6 <<<replace this text with instructions for trainer>>>
  • #13 het gaat om inpasbaarheid van het proces en is daarmee dus een taak van de SM
  • #20 Vaak is er wel support over hoe Agile te werken maar niet over hoe Agile te testen.
  • #24 Scrum is founded on empirical process control theory, or empiricism. Empiricism asserts that knowledge comes from experience and making decisions based on what is known. laat dit ook voor de teststrategie gelden, dus niet een vast voorgedefinieerd plan maar een plan/strategie dat zich aanpast aan de omstandigheden.
  • #27 Automating test cases in Sprint “n” is still an option, find the optimum of not wasting time on double work and not wasting time on automating one-off tests.
  • #35 <<<replace this text with instructions for trainer>>>