Testing and DevOps
      Culture
LB Denker
Cynical Things I’ve
       Learned


“The two virtues of a good developer are
laziness and hubris”

“You can’t tell a developer to do anything”
Things I’ve Learned

Nearly every “Best Practice” was developed
to address a particular situation.

Identify interdependent relationships and
respect them.

The code is not everything.
My Past


Rochester Institute of Technology

Universal Instruments Corporation

Google, Inc.

Etsy, Inc.
RIT Computing Majors
Information Technology

New Media

Computer Science

Software Engineering

Computer Engineering

Computational Mathematics
RIT

Computational Mathematics, BS

Computer Science, MS



Teaching Assistant in Mathematics

Teaching Assistant in Computer Science
Software Engineering I
        @RIT
 Team of Four to Five (4-5)

   Team Coordinator

   Development Coordinator

   Requirements Coordinator

   Test Coordinator*

   Configuration/QA Coordinator
Software Engineering I
        @RIT
 Team of Four to Five (4-5)

   Team Coordinator       Product Manager
   Development Coordinator Technical Lead

   Requirements Coordinator
                          Project Manager
   Test Coordinator*

   Configuration/QA Coordinator Sys Admin
Software Engineering I
        @RIT
 Team of Four to Five (4-5)

   Team Coordinator       Product Manager
   Development Coordinator Technical Lead

   Requirements Coordinator
                          Project Manager
   Test Coordinator*
                             Release Eng
   Configuration/QA Coordinator
Software Engineering I
        @RIT
 Team of Four to Five (4-5)

   Team Coordinator       Product Manager
   Development Coordinator Technical Lead

   Requirements Coordinator
                          Project Manager
   Test Coordinator*

   Configuration/QA Coordinator        QA
Software Engineering I
        @RIT
 Team of Four to Five (4-5)

   Team Coordinator       Product Manager
   Development Coordinator Technical Lead

   Requirements Coordinator
                          Project Manager
   Test Coordinator*

   Configuration/QA Coordinator Operations
Universal Instruments
     Corporation
   I’m a real Software Engineer!
Universal Instruments
     Corporation

Surface Mount
Machines

Optimizer

Line Balancer
Universal Instruments
     Corporation

Software Engineers

  Machine Level (C and Assembly)

  Software (MFC C++ or C#)

    Red, Blue, and Purple
Universal Instruments
     Corporation

Software Engineers

System Administrators

QA Analysts

SCM Team
Silos
Software Engineers

  ... wrote code

  ... completed ClearQuest tickets

  ... waited for QA to test nightly builds

  ... revisited code when QA opened a ticket

  ... played Counter-Strike at lunch
Silos

System Administrators

  ... maintained the nightly build servers

  ... kicked the nightly build in the morning
  when it failed during the night

  ... maintained staff workstations

  ... provide lots of reasons why not
Silos
QA Analysts

  ... waited for nightly builds

  ... verified completed tickets

  ... manually tested

  ... sometimes used WinRunner

  ... opened tickets for bugs
Silos

SCM Team

  ... decided stream layout in ClearCase

  ... dictated whether:
  <major>.<minor>.<maintenance>.<patch>

  ... burned CDs

  ... pre-loaded machines
DevOps
Why DevOps?


Embrace the interdependence between
Developers and other IT Professionals

Continuous Deployment
Continuous Deployment

 Pro
   Safety Net
 Implementation Difficulties
   Ops giving more access to Developers
   Lack of confidence
 Limitation
   Deploy Medium
One Button Deploy
Mostly Automated
One Button Deploy
Continuous What?!?

Continuous...   Unceremoniously...   Initiated By...


                    Push to
Deployment                              Anyone
                   Production

                  Apply Quality
Integration                             Anyone
                    Process

                 Release a New
  Delivery                                ???
                    Feature
Continuous Delivery
Continuous Deployment
+ Continuous Integration
    Continuous Delivery
Continuous Deployment
+ Continuous Integration
    Continuous Delivery
   lac y
Fal
Continuous Integration
TDD
Test Driven Development
Waterfall
The Assembly Line
TDD vs. Waterfall
Developer Testing
  TDD is a Design Process
Inside Out vs. Outside In


  Implement First    Test First

  Easy to Code       Easy to Test

  Difficult to Test   Easy to Code
“Tests should always be treated
like every other consumer of the
       subject under test.”
Testing Pyramid
Confidence
in System




            Testing Pyramid
Detailed,
Confidence                      Durable,
in System                     Coverage,
                                Faster




            Testing Pyramid
PHPUnit Groups


@small — default

@medium

@large
Four Tenets of Testing

Need to Know How Much is Being Covered

Need to Know What is Being Tested

Need to Be Able to Detect Test Runner
Errors

Need to Have Actionable Test Results
(Ownership)
More PHPUnit Groups

@group cache
      for tests that depend on a cache, like memcache


@group database
      for tests that depend on a database


@group network
      for tests that depend on a network


@group flaky
      for tests that fail without code changes
Pre-Commit Process


Lint Check

Code Reviews

Code Sniffer

Automated Tests
a·gil·i·ty [uh-jil-i-tee]
            



noun
1. the power of moving quickly and easily;
nimbleness: exercises demanding agility.
2. the ability to think and draw conclusions
quickly; intellectual acuity.
Agile Manifesto
Individuals and interactions
     over processes and tools
Working software
    over comprehensive documentation
Customer collaboration
    over contract negotiation
Responding to change
    over following a plan
Things I’ve Learned

Nearly every “Best Practice” was developed
to address a particular situation.

Identify interdependent relationships and
respect them.

The code is not everything.
Things I’ve Learned

Nearly every “Best Practice” was developed
to address a particular situation.

Identify interdependent relationships and
respect them.

The code is not everything.

Nothing is ever complete, finished, done.
Non-Technical
 Suggested Reading

The Goal: A Process of On-Going
Improvement by Eliyahu M. Goldratt

Switch: How to Change Things When
Change is Hard by Chip and Dan Heath
Disclaimers



Everything is based on my experience

Things have likely changed since
Credit


UIC Genesis 2 Picture:                      http://www3.uic.com/wcms/images2.nsf/
(GraphicLib)/Genesis_GC120Q_new_cvr_lg.jpg/$File/Genesis_GC120Q_new_cvr_lg.jpg


DevOps Picture:                http://en.wikipedia.org/w/index.php?
title=File:Devops.svg&page=1


One Button Deploy Picture:                           http://www.flickr.com/photos/
kellan/2422461496/sizes/m/in/photostream/

Testing and DevOps Culture: Lessons Learned

  • 1.
  • 2.
  • 3.
    Cynical Things I’ve Learned “The two virtues of a good developer are laziness and hubris” “You can’t tell a developer to do anything”
  • 4.
    Things I’ve Learned Nearlyevery “Best Practice” was developed to address a particular situation. Identify interdependent relationships and respect them. The code is not everything.
  • 5.
    My Past Rochester Instituteof Technology Universal Instruments Corporation Google, Inc. Etsy, Inc.
  • 6.
    RIT Computing Majors InformationTechnology New Media Computer Science Software Engineering Computer Engineering Computational Mathematics
  • 7.
    RIT Computational Mathematics, BS ComputerScience, MS Teaching Assistant in Mathematics Teaching Assistant in Computer Science
  • 8.
    Software Engineering I @RIT Team of Four to Five (4-5) Team Coordinator Development Coordinator Requirements Coordinator Test Coordinator* Configuration/QA Coordinator
  • 9.
    Software Engineering I @RIT Team of Four to Five (4-5) Team Coordinator Product Manager Development Coordinator Technical Lead Requirements Coordinator Project Manager Test Coordinator* Configuration/QA Coordinator Sys Admin
  • 10.
    Software Engineering I @RIT Team of Four to Five (4-5) Team Coordinator Product Manager Development Coordinator Technical Lead Requirements Coordinator Project Manager Test Coordinator* Release Eng Configuration/QA Coordinator
  • 11.
    Software Engineering I @RIT Team of Four to Five (4-5) Team Coordinator Product Manager Development Coordinator Technical Lead Requirements Coordinator Project Manager Test Coordinator* Configuration/QA Coordinator QA
  • 12.
    Software Engineering I @RIT Team of Four to Five (4-5) Team Coordinator Product Manager Development Coordinator Technical Lead Requirements Coordinator Project Manager Test Coordinator* Configuration/QA Coordinator Operations
  • 13.
    Universal Instruments Corporation I’m a real Software Engineer!
  • 14.
    Universal Instruments Corporation Surface Mount Machines Optimizer Line Balancer
  • 15.
    Universal Instruments Corporation Software Engineers Machine Level (C and Assembly) Software (MFC C++ or C#) Red, Blue, and Purple
  • 16.
    Universal Instruments Corporation Software Engineers System Administrators QA Analysts SCM Team
  • 17.
    Silos Software Engineers ... wrote code ... completed ClearQuest tickets ... waited for QA to test nightly builds ... revisited code when QA opened a ticket ... played Counter-Strike at lunch
  • 18.
    Silos System Administrators ... maintained the nightly build servers ... kicked the nightly build in the morning when it failed during the night ... maintained staff workstations ... provide lots of reasons why not
  • 19.
    Silos QA Analysts ... waited for nightly builds ... verified completed tickets ... manually tested ... sometimes used WinRunner ... opened tickets for bugs
  • 20.
    Silos SCM Team ... decided stream layout in ClearCase ... dictated whether: <major>.<minor>.<maintenance>.<patch> ... burned CDs ... pre-loaded machines
  • 21.
  • 22.
    Why DevOps? Embrace theinterdependence between Developers and other IT Professionals Continuous Deployment
  • 23.
    Continuous Deployment Pro Safety Net Implementation Difficulties Ops giving more access to Developers Lack of confidence Limitation Deploy Medium
  • 24.
  • 25.
  • 26.
    Continuous What?!? Continuous... Unceremoniously... Initiated By... Push to Deployment Anyone Production Apply Quality Integration Anyone Process Release a New Delivery ??? Feature
  • 27.
  • 28.
    Continuous Deployment + ContinuousIntegration Continuous Delivery
  • 29.
    Continuous Deployment + ContinuousIntegration Continuous Delivery lac y Fal
  • 30.
  • 31.
  • 32.
  • 33.
  • 34.
    Developer Testing TDD is a Design Process
  • 35.
    Inside Out vs.Outside In Implement First Test First Easy to Code Easy to Test Difficult to Test Easy to Code
  • 36.
    “Tests should alwaysbe treated like every other consumer of the subject under test.”
  • 37.
  • 38.
    Confidence in System Testing Pyramid
  • 39.
    Detailed, Confidence Durable, in System Coverage, Faster Testing Pyramid
  • 40.
    PHPUnit Groups @small —default @medium @large
  • 41.
    Four Tenets ofTesting Need to Know How Much is Being Covered Need to Know What is Being Tested Need to Be Able to Detect Test Runner Errors Need to Have Actionable Test Results (Ownership)
  • 42.
    More PHPUnit Groups @groupcache for tests that depend on a cache, like memcache @group database for tests that depend on a database @group network for tests that depend on a network @group flaky for tests that fail without code changes
  • 43.
    Pre-Commit Process Lint Check CodeReviews Code Sniffer Automated Tests
  • 44.
    a·gil·i·ty [uh-jil-i-tee]   noun 1. the power of moving quickly and easily; nimbleness: exercises demanding agility. 2. the ability to think and draw conclusions quickly; intellectual acuity.
  • 45.
    Agile Manifesto Individuals andinteractions over processes and tools Working software over comprehensive documentation Customer collaboration over contract negotiation Responding to change over following a plan
  • 46.
    Things I’ve Learned Nearlyevery “Best Practice” was developed to address a particular situation. Identify interdependent relationships and respect them. The code is not everything.
  • 47.
    Things I’ve Learned Nearlyevery “Best Practice” was developed to address a particular situation. Identify interdependent relationships and respect them. The code is not everything. Nothing is ever complete, finished, done.
  • 48.
    Non-Technical Suggested Reading TheGoal: A Process of On-Going Improvement by Eliyahu M. Goldratt Switch: How to Change Things When Change is Hard by Chip and Dan Heath
  • 49.
    Disclaimers Everything is basedon my experience Things have likely changed since
  • 50.
    Credit UIC Genesis 2Picture: http://www3.uic.com/wcms/images2.nsf/ (GraphicLib)/Genesis_GC120Q_new_cvr_lg.jpg/$File/Genesis_GC120Q_new_cvr_lg.jpg DevOps Picture: http://en.wikipedia.org/w/index.php? title=File:Devops.svg&page=1 One Button Deploy Picture: http://www.flickr.com/photos/ kellan/2422461496/sizes/m/in/photostream/