Agile Methods in Research © 2008 Telefónica Investigación y Desarrollo, S.A. Unipersonal Xavier Amatriain Rubio Gemma Hornos Cirera TELEFÓNICA I+D Date: June 2008
01 The Problem 02 Agile Response 03 Scrum   04   eXtreme Programming   05 Agile Scientific Research 06   Conclusions Index
Agile Methods  The Problem
The Problem  Business Environment 01 Fast Moving Market Boost Innovation Forge the Future Workforce Culture 2001  2002  2003  2004  2005  2006  2007
The Problem  Technology  Environment 01 Changes Requirements Scope Technology New Paradigm Technology  Requirements known unknown stable unstable simple chaos complexity Requirements Cost Date Cost Date Features Plan Driven Value Driven FIX ESTIMATE TRADITIONAL   AGILE
The Problem  Summary The market demands and expects  innovative high-quality software  that meets its needs  and soon 01
Agile Methods  The Response
Agile Response Market Needs EMBRACING   CHANGE 02 Traditional Agile Time Cost  of  Change
What is software development? Mental Creative  Human Agile Response Is a process of knowledge transformation Software  Problem Need Software Developer 02 Traceability Knowledge Transformation
What is Agile? Agile processes are  iterative processes  that use specific project management and software engineering practices to sustain the  delivery of new software functionality  every one to four weeks 02
Agile Manifesto That is, while there is value in the items on the left, we value the items on the right more. We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value: 02 More Important Important individuals and interactions processes and tools working software comprehensive documentation customer collaboration contract negotiation responding to change following a plan www.agilemanifesto.org
Agile Manifesto That is, while there is value in the items on the left, we value the items on the right more. We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value: 02 More Important Important individuals and interactions processes and tools working software comprehensive documentation customer collaboration contract negotiation responding to change following a plan www.agilemanifesto.org AGILE :    Only code  Creating and responding to change
Agile Methods Scrum eXtreme Programming (XP) Crystal Feature Driven Development Dynamic Systems Development Method Adaptive Software Development Lean Development Pragmatic Programming 02 Inclusive Rules Generative  Rules
Iterative vs Waterfall Soft-NOT-aware Software Software Software Paperware Focus In Executable Software You can't know everything at the beginning You learn as you work WATERFALL AGILE 02 Requirements Analysis&Design Implementation Testing Deployment Architecture  Requirements … Iteration1 Iteration2 Iteration3 Release1 Release2 Release3
Iterative vs. Waterfall Soft-NOT-aware Software Software Software Paperware Focus In Executable Software You can't know everything at the beginning You learn as you work WATERFALL AGILE Adaptive  Management Predictive  Management what method fits me best? 02 Requirements Analysis&Design Implementation Testing Deployment Architecture  Requirements … Iteration1 Iteration2 Iteration3 Release1 Release2 Release3
Why Iterative Development? knowledge Risk Iterations help acquiring knowledge Of both the problem and the solution Estimation Error Short term is easier to predict and estimate rather than long term Time Iterations 02
What makes a development process  Agile ? Each  iteration  delivers  working software . The  phases  in each iteration are  nearly  concurrent . Team uses specific  engineering practices  to keep the  code base fresh and flexible . Teams are  self-managing . Lean principles and techniques  eliminate waste  whenever possible. 02
Why adopt Agile processes? Reduced  time-to-market. Increased innovation.  Business alignment. Increased quality. Reduces product and process waste. High visibility and control over the project progress. Empowered, motivated and self organizing Teams. 02
What kind of change is required to adopt Agile? Larger teams must be broken into smaller ones. Functional silos have to be broken down, or at least weakened. Specialists have to pick up new skills. Teams must learn to self-manage, and managers must learn to let them. Routine activities have to be automated. 02
Where is Agile most suitable? Projects with some degree of uncertainty around requirements or technology. Projects that aren’t too big or too small. Projects with buy-in from team members, business stakeholders, and management. Collocated project teams and  business customers. Projects using modern languages  and tools. 02
How widely are Agile processes used? 02
How to use The State of Agile Development. Julio 2007- Version One 72% Pilot experience in Telefónica I+D 02 Scrum  Scrum/  XP  Custom/  DSDM   XP Hybrid  Other Hybrid 37% 23% 12% 9% 5%
Agile Methods  Scrum
SCRUM: Planning Iterations Source mountaingoatsoftware.com 03
SCRUM: Planning Iterations  Iteration 1 Iteration 2 Iteration 3 Product Backlog Assessment &  Planning Next Use Cases Implemented Risk Addressed 03
SCRUM: Framework 3 ROLES Product Owner Team Scrum Master 3  BACKLOGS Product Backlog Sprint Backlog Impediments Backlog 4  MEETINGS Sprint Planning Daily Meeting Sprint Review Retrospective EMPIRICAL PROCESS  CONTROL Transparency  Inspect Adapt 03
SCRUM: Workflow 03 Sprint Planning (I) Selected  Product Backlog Sprint Planning (II) Increment Sprint Backlog Retrospective Sprint Review Estimation Meeting Vision SPRINT  0 Impediments Backlog Daily Meeting ¿Hecho? ¿Pendiente?  ¿Trabas?   SPRINT Product Backlog
SCRUM: Sprint Backlog PRIORITIY What does  DONE  mean? 03
SCRUM: Reports 03 Sprint Burndown Chart Remaining  Time days
Agile Methods  eXtreme Programming
XP.  Values & Practices 04 Simple Design Test-Driven Development Pair Programming Refactoring COMMUNICATION SIMPLICITY COURAGE FEEDBACK Stub out code Watch test fail Refactor Repeat Get test to pass
XP.  Values & Practices 04 COMMUNICATION SIMPLICITY COURAGE FEEDBACK Simple Design Test-Driven Development Pair Programming Refactoring Open Workspace Collective Ownership Continuous Integration Metaphor Sustainable Pace Coding Standard Acceptance Tests One Team Iterations Small Releases Release Plan Retrospective User Stories On–site Customer
XP: Planning/Feedback Loop 04 Code Pair Programming Unit Test Pair Negotiation Stand Up Meeting Acceptance Test Iteration Plan Release Plan
XP: Employment Practices The State of Agile Development. July 2007- Version One 04 Agile Practices
Agile Methods  Agile Scientific Research
Agile Scientific Research The Scientific Process 05
Agile Scientific Research The Scientific Process What does the scientific method share with Agile Methodologies? Iterative Fast-changing requirements Test-driven (Hypothesis-driven) Pair-programming (Peer review) Simple Design (Occam´s Razor) … 05
Agile Scientific Research Planning iterations Iteration 0: build up general hypothesis List of “stories” -> things we would like to say in our final article Optionally, write the article itself in this first iteration Every 1-2 weeks come up with a list of prioritized tasks List all possible tasks (ideally 1-2 day workload) Measure interest of the task towards final goal Measure cost in terms of predicted hours of work List them in order of priority value (= interest – cost) For ties, value more those that have less cost (Occam´s Razor) In the iteration planning re-evaluate general hypothesis 05
Agile Scientific Research Test Driven Development Plan stories and tasks by writing the tests first Use test as an “executable hypothesis” “ If I can prove hypothesis H then it should happen that tests t1 and t2 should pass” Maintain collection of tests as a record of experiments Use the red-green-refactor cycle Refactor to refine your theory while still being consistent to experimental data 05
Agile Methods  Conclusions
Conclusions Scientific Research is an iterative process with fast changing requirements It can be managed as an Agile project Many of the Agile practices are applicable Agile can help improve focus, performance, and help researchers accomplish goals in a timely fashion To deliver  high quality ,  running ,  tested   stories  that  meet the business  need in a  predictable ,  efficient  and  collaborative  manner — on time, on budget! 06
© 2008 Telefónica Investigación y Desarrollo, S.A. Unipersonal

Agile Science

  • 1.
    Agile Methods inResearch © 2008 Telefónica Investigación y Desarrollo, S.A. Unipersonal Xavier Amatriain Rubio Gemma Hornos Cirera TELEFÓNICA I+D Date: June 2008
  • 2.
    01 The Problem02 Agile Response 03 Scrum 04 eXtreme Programming 05 Agile Scientific Research 06 Conclusions Index
  • 3.
    Agile Methods The Problem
  • 4.
    The Problem Business Environment 01 Fast Moving Market Boost Innovation Forge the Future Workforce Culture 2001 2002 2003 2004 2005 2006 2007
  • 5.
    The Problem Technology Environment 01 Changes Requirements Scope Technology New Paradigm Technology Requirements known unknown stable unstable simple chaos complexity Requirements Cost Date Cost Date Features Plan Driven Value Driven FIX ESTIMATE TRADITIONAL AGILE
  • 6.
    The Problem Summary The market demands and expects innovative high-quality software that meets its needs and soon 01
  • 7.
    Agile Methods The Response
  • 8.
    Agile Response MarketNeeds EMBRACING CHANGE 02 Traditional Agile Time Cost of Change
  • 9.
    What is softwaredevelopment? Mental Creative Human Agile Response Is a process of knowledge transformation Software Problem Need Software Developer 02 Traceability Knowledge Transformation
  • 10.
    What is Agile?Agile processes are iterative processes that use specific project management and software engineering practices to sustain the delivery of new software functionality every one to four weeks 02
  • 11.
    Agile Manifesto Thatis, while there is value in the items on the left, we value the items on the right more. We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value: 02 More Important Important individuals and interactions processes and tools working software comprehensive documentation customer collaboration contract negotiation responding to change following a plan www.agilemanifesto.org
  • 12.
    Agile Manifesto Thatis, while there is value in the items on the left, we value the items on the right more. We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value: 02 More Important Important individuals and interactions processes and tools working software comprehensive documentation customer collaboration contract negotiation responding to change following a plan www.agilemanifesto.org AGILE :  Only code  Creating and responding to change
  • 13.
    Agile Methods ScrumeXtreme Programming (XP) Crystal Feature Driven Development Dynamic Systems Development Method Adaptive Software Development Lean Development Pragmatic Programming 02 Inclusive Rules Generative Rules
  • 14.
    Iterative vs WaterfallSoft-NOT-aware Software Software Software Paperware Focus In Executable Software You can't know everything at the beginning You learn as you work WATERFALL AGILE 02 Requirements Analysis&Design Implementation Testing Deployment Architecture Requirements … Iteration1 Iteration2 Iteration3 Release1 Release2 Release3
  • 15.
    Iterative vs. WaterfallSoft-NOT-aware Software Software Software Paperware Focus In Executable Software You can't know everything at the beginning You learn as you work WATERFALL AGILE Adaptive Management Predictive Management what method fits me best? 02 Requirements Analysis&Design Implementation Testing Deployment Architecture Requirements … Iteration1 Iteration2 Iteration3 Release1 Release2 Release3
  • 16.
    Why Iterative Development?knowledge Risk Iterations help acquiring knowledge Of both the problem and the solution Estimation Error Short term is easier to predict and estimate rather than long term Time Iterations 02
  • 17.
    What makes adevelopment process Agile ? Each iteration delivers working software . The phases in each iteration are nearly concurrent . Team uses specific engineering practices to keep the code base fresh and flexible . Teams are self-managing . Lean principles and techniques eliminate waste whenever possible. 02
  • 18.
    Why adopt Agileprocesses? Reduced time-to-market. Increased innovation. Business alignment. Increased quality. Reduces product and process waste. High visibility and control over the project progress. Empowered, motivated and self organizing Teams. 02
  • 19.
    What kind ofchange is required to adopt Agile? Larger teams must be broken into smaller ones. Functional silos have to be broken down, or at least weakened. Specialists have to pick up new skills. Teams must learn to self-manage, and managers must learn to let them. Routine activities have to be automated. 02
  • 20.
    Where is Agilemost suitable? Projects with some degree of uncertainty around requirements or technology. Projects that aren’t too big or too small. Projects with buy-in from team members, business stakeholders, and management. Collocated project teams and business customers. Projects using modern languages and tools. 02
  • 21.
    How widely areAgile processes used? 02
  • 22.
    How to useThe State of Agile Development. Julio 2007- Version One 72% Pilot experience in Telefónica I+D 02 Scrum Scrum/ XP Custom/ DSDM XP Hybrid Other Hybrid 37% 23% 12% 9% 5%
  • 23.
  • 24.
    SCRUM: Planning IterationsSource mountaingoatsoftware.com 03
  • 25.
    SCRUM: Planning Iterations Iteration 1 Iteration 2 Iteration 3 Product Backlog Assessment & Planning Next Use Cases Implemented Risk Addressed 03
  • 26.
    SCRUM: Framework 3ROLES Product Owner Team Scrum Master 3 BACKLOGS Product Backlog Sprint Backlog Impediments Backlog 4 MEETINGS Sprint Planning Daily Meeting Sprint Review Retrospective EMPIRICAL PROCESS CONTROL Transparency Inspect Adapt 03
  • 27.
    SCRUM: Workflow 03Sprint Planning (I) Selected Product Backlog Sprint Planning (II) Increment Sprint Backlog Retrospective Sprint Review Estimation Meeting Vision SPRINT 0 Impediments Backlog Daily Meeting ¿Hecho? ¿Pendiente? ¿Trabas? SPRINT Product Backlog
  • 28.
    SCRUM: Sprint BacklogPRIORITIY What does DONE mean? 03
  • 29.
    SCRUM: Reports 03Sprint Burndown Chart Remaining Time days
  • 30.
    Agile Methods eXtreme Programming
  • 31.
    XP. Values& Practices 04 Simple Design Test-Driven Development Pair Programming Refactoring COMMUNICATION SIMPLICITY COURAGE FEEDBACK Stub out code Watch test fail Refactor Repeat Get test to pass
  • 32.
    XP. Values& Practices 04 COMMUNICATION SIMPLICITY COURAGE FEEDBACK Simple Design Test-Driven Development Pair Programming Refactoring Open Workspace Collective Ownership Continuous Integration Metaphor Sustainable Pace Coding Standard Acceptance Tests One Team Iterations Small Releases Release Plan Retrospective User Stories On–site Customer
  • 33.
    XP: Planning/Feedback Loop04 Code Pair Programming Unit Test Pair Negotiation Stand Up Meeting Acceptance Test Iteration Plan Release Plan
  • 34.
    XP: Employment PracticesThe State of Agile Development. July 2007- Version One 04 Agile Practices
  • 35.
    Agile Methods Agile Scientific Research
  • 36.
    Agile Scientific ResearchThe Scientific Process 05
  • 37.
    Agile Scientific ResearchThe Scientific Process What does the scientific method share with Agile Methodologies? Iterative Fast-changing requirements Test-driven (Hypothesis-driven) Pair-programming (Peer review) Simple Design (Occam´s Razor) … 05
  • 38.
    Agile Scientific ResearchPlanning iterations Iteration 0: build up general hypothesis List of “stories” -> things we would like to say in our final article Optionally, write the article itself in this first iteration Every 1-2 weeks come up with a list of prioritized tasks List all possible tasks (ideally 1-2 day workload) Measure interest of the task towards final goal Measure cost in terms of predicted hours of work List them in order of priority value (= interest – cost) For ties, value more those that have less cost (Occam´s Razor) In the iteration planning re-evaluate general hypothesis 05
  • 39.
    Agile Scientific ResearchTest Driven Development Plan stories and tasks by writing the tests first Use test as an “executable hypothesis” “ If I can prove hypothesis H then it should happen that tests t1 and t2 should pass” Maintain collection of tests as a record of experiments Use the red-green-refactor cycle Refactor to refine your theory while still being consistent to experimental data 05
  • 40.
    Agile Methods Conclusions
  • 41.
    Conclusions Scientific Researchis an iterative process with fast changing requirements It can be managed as an Agile project Many of the Agile practices are applicable Agile can help improve focus, performance, and help researchers accomplish goals in a timely fashion To deliver high quality , running , tested stories that meet the business need in a predictable , efficient and collaborative manner — on time, on budget! 06
  • 42.
    © 2008 TelefónicaInvestigación y Desarrollo, S.A. Unipersonal