Testit 2017 - Exploratory Testing for Everyone

Testit 2017 - Exploratory Testing for Everyone

• 2.
  © King.com Ltd2017 – Commercially confidential Exploratory Testing for Everyone Testit 2017 Page 2
• 3.
  © King.com Ltd2017 – Commercially confidential Introduction Johan Hoberg • 13+ years of testing • 10 years at Sony Mobile • 3+ years at King • Things that make me happy • Testing • Games • Other stuff (Family, etc.) Page 3
• 4.
  © King.com Ltd2017 – Commercially confidential What is this talk about? • TL;DR • We replaced all our scripted testing with exploratory testing • The results were very positive in a number of ways • I will go into some more detail exactly what this meant to us and how it affected the team • At the end I will go through why this is an important part of a much bigger puzzle Page 4
• 5.
  © King.com Ltd2017 – Commercially confidential Page 5 Background Game TeamDevelopers, artist, product owner, Scrum master, QA Game Maintain & Develop Content Release New content every 2 weeks Test Scripted release regression test Scripted test cases for new features
• 6.
  © King.com Ltd2017 – Commercially confidential Page 6 Scripted Release Regression test 3-4 people doing all testing 200+ Test cases ”stuck with testing” No commitment No ownership
• 7.
  © King.com Ltd2017 – Commercially confidential Scripted Feature Test • For new features we wrote scripted test cases, which we then performed once the feature was integrated • Some of these tests were then transferred to the regression test suite • The actual scripted test cases had little value except for adding to the regression suite, since our test specialist ran exploratory testing anyway, even if creating them and discussing around the was valuable Page 7
• 8.
  © King.com Ltd2017 – Commercially confidential Why Change? • We found most release stopping bugs with the scripted regression tests (although in general not many new bugs) • But no one liked it, and it took quite a long time to finish the tests • So we wanted to have at least the same coverage as before, but more fun in less time • We also wanted to stop creating a lot of new feature test cases that we then never really used Page 8
• 9.
  © King.com Ltd2017 – Commercially confidential The Change • As a test lead I took the decision to try out using exploratory testing for one release instead of the old test suite • We looked at which broader areas the test cases covered, and created test missions for those instead • Everyone in the team participated in testing together for 1 hour • We stopped writing test cases for new features and instead worked on the design documents with the rest of the team Page 9 Live OPS CANVAS Gameplay MOBILE Purchase & Sales MOBILE
• 10.
  © King.com Ltd2017 – Commercially confidential The Result • Same amount of release stopping bugs found, but more minor bugs • Everyone in the team takes more ownership of quality and testing, not only during the regression tests, and no one complains anymore • What previously took a whole day is done in one hour because everyone participates and we tested based on risk, instead of running “everything” • No unnecessary test artifacts are created or maintained for new features, and the design documents now reflected any risks we saw Page 10
• 11.
  © King.com Ltd2017 – Commercially confidential What it all comes down to • Motivation • Risk-based testing • Not producing waste • Common ownership of quality and test • Everyone testing to the extent of their abilities Page 11
• 12.
  © King.com Ltd2017 – Commercially confidential Bonus Slide: How it all fits together Page 12 Agile & Scrum Intrinsic Motivation Ownership of Quality High Quality Complex Products Mandate & Trust
• 13.
  © King.com Ltd2017 – Commercially confidential Bonus Slide: Intrinsic Motivation and Scrum Self-Determination Theory • Autonomy • Competence • Relatedness Drive • Mastery • Autonomy • Purpose Page 13
• 14.
  © King.com Ltd2017 – Commercially confidential Bonus Slide: Test Competence and Cynefine • I believe strongly that • anyone can perform “obvious” tests • people with system knowledge can perform “complicated” tests • and someone with test competence can perform “complex” tests • Complex • Cause and effect can only be deduced in retrospect • probe–sense–respond Page 14
• 16.
  Thank you!