Continuous testing & devops with @petemar5hall

Testing software in high
frequency delivery
environments.
Continuous
Testing &
DevOps

- Lead consultant & trainer at Lean
Software Services
- 16+ years working in software
development, based in London &
Madrid
Peter Marshall
MAIL
info@leansoftwareservices.com
TWITTER
@petemar5hall
LINKEDIN
https://www.linkedin.com/in/petedmarshall

Training
Coaching
Consulting
Madrid
Barcelona
London
Copenhagen
- Testing
- DevOps
- Continuous
Delivery
- Automation
- Virtualisation

Gain insight into how continuous testing, DevOps, and organisational
structure can alter the way we think about testing.
Understand how DevOps practices contribute to the test effort.
Objectives

Creating
environments
deploying to
environments
testing
environments
looking at
requirements
pair
programming/
testing
creating
test data
writing
automated
checks
running
through
regression scripts
what else does a
tester do?
triage
mitigating
delays

Who is testing in an agile environment?
What about doing continuous delivery?
Who has a devops culture / team / role?

Sources: Applied Software Measurement: Global Analysis of
Productivity and Quality by Capers Jones, & IBM The Future of Testing
Most forms of testing ﬁnd fewer than 30% of all bugs
almost 40% of testers have seen an increase in the frequency of release
Productivity and quality are directly coupled: projects with
high quality have high productivity and vice versa

How can devops
and continuous
testing help?

the test and QA
headache is
compounded
further by the need
to release faster
and more often
Amazon is on record as making changes
to production every 11.6 seconds on
average in May of 2011.
Facebook releases to production twice a
day.
Many Google services see releases
multiple times a week, and almost
everything in Google is developed on
mainline.
Etsy push unto 70 changes a day to its
monolithic software!

the presence of a
tester can sometimes
introduce more risk
than they mitigate..

Characteristics of
high frequency
delivery
environments

Key Concepts
…continuous
testing…
DevOps

Continuous Testing
…not just about test
automation! (ignore
wikipedia)
Actually includes, but not limited to:
Automated testing
Automated management of production and non
production environments
Automated application feedback through
monitoring
Automated evaluation of business objectives
Evaluating product concepts
Testing
engaging in XP practices
organisational change

The DevOps movement is built
around a group of people who
believe that the application of a
combination of appropriate
technology and attitude can
revolutionize the world of software
development and delivery.
Stephen Nelson-Smith - 2010
its about:
Automating building, testing, and
deployment
Bringing consistency to the way we
use and create technical assets
Providing teams with the tools
required to achieve technical
excellence
DevOps

vs.
more facts..
high performing organizations ship code 30 times faster (and
complete these deployments 8,000 times faster), have 50%
fewer failed deployments, and restore service 12 times faster
than their peers. The longer they had been employing these
practices, the better they performed.
source: ThoughtWorks, 2014

automate the
build
Reduce general coding errors, create a single source of
truth, ensure basic component integration

vs.script your configuration!
say NO to hand crafted conﬁguration ﬁles!

automate the
infrastructure
Remove conﬁguration issues by having a repeatable
mechanism for creating environments.

vs.automate the testing
continuous testing does away
with the concept of
regression testing.
..an effective software delivery process
requires no regression testing stages. It
becomes integral to everything we do..

automate deployment
say NO to sneaker net
say NO manually to copying ﬁles
say NO to manually pulling and pushing servers in and out of load balancers
deploying software is one of the
most mission critical aspects of
what we do, let a machine do it!

monitor
One of the greatest and
overlooked tools in the arsenal of
the tester is monitoring.
Realtime access to
application logs
Realtime access to
error logs
Realtime access to
infrastructure logs

“A good, easily accessible
event & error monitoring
system is essential for anyone
releasing software. Build an
early warning system into your
environment!”

“Understanding your systems
steady state allows you to react
quickly to any deviation”

Build Agent & Devel
The delivery pipeline
Pre commit tests
Code review
PO/UX review
Branch / Merge
Build
Unit tests
Contract tests
Component tests
Coverage
Static analysis
Component
performance
End to end
component tests
NFR tests
Exploratory Test
performance testing
Story sign off /
Demos
NFRs
Analytics
Monitoring
Production tests
canary or blue /
green release
CI Environments
Developer
Local
Pre-Production Production
Virtualised Full stack environments
Commit Auto-Deploy Promote Release
Build Pipeline:
There can be multiple pipelines, usually per system, should execute in minutes
Commit:
The commit can include changes to the infrastructure, database, or application code.
Outputs:
Software, infrastructure and conﬁguration
Environment manifests
Release notes

Create a
single view
of quality

smaller,
more
frequent
deliveries

Thank
you!
TWITTER
@petemar5hall
Agile Consultant & Technical
Architect @planday
Peter Marshall
ﬁveMAIL
info@leansoftwareservices.com
LINKEDIN
https://www.linkedin.com/in/
petedmarshall

Continuous testing & devops with @petemar5hall

More Related Content

What's hot

Viewers also liked

Similar to Continuous testing & devops with @petemar5hall

Recently uploaded

Continuous testing & devops with @petemar5hall