How to structure testing within the Scrum Framework

How to structure testing within the Scrum Framework

• 1.
  + Testing & Scrum Howto structure testing within the Scrum Framework?
• 2.
  + Introduction  This presentationoutlines my interpretation of how to work with testing within the Scrum Framework
• 3.
  + Scrum
• 4.
  + Generalizing Specialists(or Specializing Generalists)[12]
• 5.
  + Key Message:Everyone is a Tester  Everyone in the Scrum Team must contribute to the testing effort  Each person has a different skillset that must be used efficiently in the testing effort  So everyone in the team is a tester – but certain people have a skillset that is optimal for finding those complex problems that other people might miss
• 6.
  + Quality Definition • “Qualityis value to some person”[1] • Working software is primary measure of progress [2] • The Scrum Team owns the quality of the product [3] • The Definition of Done should define a level of quality of the output that the Development Team delivers
• 7.
  + Acceptance Criteria [4] “Acceptance Criteria are the conditions that a software product must satisfy to be accepted by a user, customer, or in the case of system level functionality, the consuming system.”  “Acceptance Criteria are a set of statements, each with a clear pass/fail result, that specify both functional and non-functional requirements, and are applicable at the Epic, Feature, and Story Level. Acceptance criteria constitute our “Definition of Done”, and by done I mean well done.”  The Given/When/Then format is helpful way to specify acceptance criteria:  Given some precondition When I do some action Then I expect some result
• 8.
  + Acceptance Criteria &Testers  Writing Acceptance Criteria has a lot in common with testing  It requires much of the same skill set to be able to write good Acceptance Criteria as it does to perform testing  Even if the Product Owner is responsible for the Acceptance Criteria, a tester can add value by contributing with his/her expertise
• 9.
  + Agile Test Quadrants[7]
• 10.
  + Key Message:Testing is infused into everything  Scrum is founded on Empirical Process Control [3]  Empiricism asserts that knowledge comes from experience and making decisions based on what is known [3]  Testing is not something you just do at the end of a sprint – it is infused into basically every activity
• 11.
  + Test Ownership Scrum Team Outsideof Scrum Team Isolated Tests Contract/Collaboration Tests Integration Tests System Tests Equipment & Competence Specific Tests  We need to identify who is responsible for performing what types of tests  There are only two important parties for the Scrum Team  The Team  Outside of the Team  Clear ownership is Key  This is just my way of grouping different types of tests, it is not the only way – find a way that works for you
• 12.
  Test Type Definitions Contract / Collaboration / Integration tests  Integrated Tests are a Scam by J.B. Rainsberger [8]  Contract Test:  http://thecodewhisperer.tumblr.com/post/1325859246/in-brief-contract-tests  “Contract Tests explain how a class should extend a superclass or implement an interface, so that I don’t have to read a bunch of prose to figure out how to do that.”  “…a test class that can run the same set of tests for two different implementations of the same interface.”[11]  Collaboration Tests: Also known as interaction tests, as opposed to state-based tests  “Does the client talk to the next layer correctly?”  “Testing interactions means you're verifying that the code under test calls certain methods properly.”  http://googletesting.blogspot.se/2013/03/testing-on-toilet-testing-state-vs.html  Integration tests: http://en.wikipedia.org/wiki/Integration_testing  “An integrated test is any (low level) test where when it fails you cannot pinpoint exactly what went wrong.“  Individual software modules are combined and tested as a group
• 13.
  Test Type Definitions System Test  Tests that span across features and teams due to the complexity of the product  If product and/or feature complexity is very low, then integration tests may be enough to sufficiently cover the test space  Equipment Specific Tests  Tests that require specific hardware or software that is not available to the development teams  If all tools are available to the Scrum Team(s), then this category disappears  Competence Specific Tests  Tests that require specific competence that is not available in the development teams  Examples: Localization, Tracking, Cross promotions, Cross missions, tests that require understanding of multiple King games, etc.  If all competences are available to the Scrum Team(s), then this category disappears
• 14.
  Key Message: Complexityis the main separator  Using the Cynefin framework [9]  Simple tests can be done by anyone  Complicated tests are well suited for someone with a good understanding of the system  Complex tests are well suited for someone with a good testing skillset and a good understanding of the system  Chaotic tests are … difficult?  There is a place for someone with a strong testing skillset both in the Scrum Team, and outside of the Scrum Team
• 15.
  Key Message: Youcan place some testing outside of the Scum Team if you have multiple teams  There is a point in placing some testing outside of the Scrum Team  System test that spans over multiple features and teams  Tests that require equipment not readily available to all teams  Tests that require competence not readily available to all teams  However these test teams must work in parallel with all the Scrum Teams, and this creates some interesting coordination problems
• 16.
  Test Automation  Thisway of working obviously favors automated checks, but in no way mandates it  The Scrum Team should make the decision what they want to automate and what they want to test manually  Even if you have automated checks that cover isolated components, communication between components, and how multiple components work together in a group, you will still need to perform manual testing if you have a user interface
• 17.
  + Testing & Checking “Checkingis the process of making evaluations by applying algorithmic decision rules to specific observations of a product.” [5]  Preferably all manual testing should be actual testing and not checking  Checking detailed Acceptance Criteria that are hard to automate could be an exception  Designing and prototyping automated checks is a test activity  Running automated regression checks is a check activity
• 18.
  + (Exploratory) Testing [6] “Testing is the process of evaluating a product by learning about it through exploration and experimentation, which includes: questioning, study, modeling, observation and inference, output checking, etc.”  All testing is exploratory, even scripted testing, if you are doing it responsibly
• 19.
  + Regression Testing  Everyoneis responsible for covering their own test areas during regression test activities  This motivates programmers and testers to design the system properly with testability in mind, and to automate efficiently  During all test activities you must always have a risk-based approach [10]  You cannot cover the entire test space every time and still be efficient  This drives the need to reduce system complexity and design the system with this in mind
• 20.
  + Conclusion  Testing isan integral part of the Scrum Framework  Everyone should contribute  But there is still a place for a strong testing skillset, driven by the complexity of the product  Test Ownership should be clear and it is possible to place some testing outside of the Scrum Team
• 21.
  References [1] Definition ofQuality Weinberg, Gerald M. (1992), Quality Software Management: Volume 1, Systems Thinking, New York, NY: Dorset House Publishing, p. 7 [2]Agile Manifesto Principles http://agilemanifesto.org/principles.html [3] The Scrum Guide http://www.scrumguides.org/docs/scrumguide/v1/scrum-guide-us.pdf [4] Acceptance Criteria http://www.leadingagile.com/2014/09/acceptance-criteria/ [5] Testing and Checking http://www.satisfice.com/blog/archives/856 [6] Exploratory Testing 3.0 http://www.satisfice.com/blog/archives/1509 [7] Agile Testing Quadrants http://www.developsense.com/presentations/2014-06-Dublin-RSTAgileTesting.pdf [8]Integration Tests are a Scam https://vimeo.com/80533536 [9]Cynefin http://en.wikipedia.org/wiki/Cynefin [10] Heuristic Risk-Based Testing http://www.satisfice.com/articles/hrbt.pdf [11]Contract Tests: An Example http://blog.thecodewhisperer.com/2011/07/07/contract-tests-an-example/ [12]To combine … or not http://angryweasel.com/blog/to-combine-or-not/