Downloaded 80 times
More Related Content
Similar to Agile metrics - Agile KC Meeting 9/26/13
Agile metrics - Agile KC Meeting 9/26/13
- 1. Agile Metrics Craig Rudman VPServices, Agile Coach
- 2. Purpose • Greater alignmentof strategy to execution • Improve the flow of work • Improve product quality • Support continuous improvement efforts • Improve the quality of life • Increase trust Discuss ways to go about measuring work and workflow
- 3. Roadmap Context • What canwe measure? • What should we measure? • What principles inform agile metrics? Systems Thinking • Who is your customer? • The nature of demand • Value creation Agile at Scale • Portfolio • Program • Team
- 4. Why Metrics? • Increasevisibility of work • Increase alignment of strategy to execution • Improve trust • Improved product quality • Improved workflow • Improved performance • Identify and solve problems Metrics give us insight into the health of our organization, our people, processes, and tools. We use metrics to help us adapt and improve
- 5. We can quantifythose things we can see We can qualify the things we can't see What can we measure? Quantify • Effort • Time • System Components • Tests • Events • Data Qualify • Quality • Perceptions • Feelings • Attitudes
- 6. We should measurethe things we value What should we measure? If your goal is… • Improving visibility of work • Better alignment • Increasing trust • Improving quality • Improving process • Solving problems then you should measure… → work-in-process → context → commitments → breakage and rework → flow → root causes
- 7. The Manifesto forAgile Software Development We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value: • Individuals and interactions over processes and tools • Working software over comprehensive documentation • Customer collaboration over contract negotiation • Responding to change over following a plan That is, while there is value in the items on the right, we value the items on the left more.
- 8. The Twelve Principlesof Agile Software Development • Our highest priority is to satisfy the customer through early and continuous delivery of valuable software. • Welcome changing requirements, even late in development. Agile processes harness change for the customer's competitive advantage. • Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale. • Business people and developers must work together daily throughout the project. • Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done. • The most efficient and effective method of conveying information to and within a development team is face-to-face conversation. • Working software is the primary measure of progress. • Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely. • Continuous attention to technical excellence and good design enhances agility. • Simplicity--the art of maximizing the amount of work not done--is essential. • The best architectures, requirements, and designs emerge from self-organizing teams. • At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.
- 9. Agile Metrics SupportAgile Principles If motivated by these principles… Measure for these outcomes… Our highest priority is to satisfy the customer through early and continuous delivery of valuable software. Customer satisfaction Cycle time and throughput Product value realized Welcome changing requirements, even late in development. Agile processes harness change for the customer's competitive advantage. Changeability of requirements Value of change Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale. Frequency of delivery Product stability Business people and developers must work together daily throughout the project. Frequency/quality of collaboration Delays Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done. Alignment of work to motivated people Quality of team support Release predictability The most efficient and effective method of conveying information to and within a development team is face-to-face conversation. Efficiency of communication Effectiveness of communication Working software is the primary measure of progress. Rate of product adoption Frequency of product usage Failure demand Breakage Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely. Burnout, attrition, turnover, morale Stability of cadence and velocity Stability of team organization Continuous attention to technical excellence and good design enhances agility. Skills growth Increasing rates of reuse Design/code consistency and quality Test coverage/automation Simplicity--the art of maximizing the amount of work not done--is essential. Low complexity Short work queues The best architectures, requirements, and designs emerge from self-organizing teams. Internal locus of control High feature/story acceptance rates At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly. Consistency of cadence Effectiveness of retrospectives Progress towards improvement goals
- 10. Exercise: Five Votes •Read through the agile principles • Think about where you would like to see improvement • Cast your five votes accordingly – You can put them all on one item – You can distribute them across up to five items • Working as a group, we will discuss a set of metrics to support the highest-priority item
- 11. Customers Suppliers Assets/Expenses Profit/Loss Cash Flow MarketShare Customer Satisfaction Employee Satisfaction Shareholder Benefit Society Benefit Knowledge Creation Product/Service Quality Time to Market Demand Revenue Resources Take a Systems View Demand Value Revenue Metrics help us understand and manage these things The primary metric for agile is whether or not value is being created. This is determined empirically, by demonstration, at the end of every single iteration. Your business is a value-creation system
- 12. The Big BlueCompany Information Services Data Management 21 Application Development 218 Infrastructure/Ops 77 Services 94 Claims (206) Prov. & Gov't Relations (162) Sales & Marketing (146) Finance (106) All Others (155) Customer Service (227) Membership (129) Who is your customer? Demand Value Revenue
- 13. Mandates Preventive Maintenance How much workcan you manage? Initiatives Production Support Enhancement Requests Defects Rework Stuff and Nonsense
- 14. 74% 18% 8% How much workcan you manage? Production Support Enhancement Requests Defects Rework Stuff and Nonsense Mandates Preventive Maintenance Initiatives 209,320 49,788 22,150 Total capacity = 281,258 hrs
- 15. Types of Demand •What is the nature of customer demand? – How much of it is failure demand? – How much of it is mandates? – How much of it is value demand? • If demand exceeds capacity, what are your options? “All time spent handling failure demand is waste.”* *Mary Poppendieck
- 16. Portfolio Management • FailureDemand (from 74% to 50%) – Defects – Rework – Missing Features • Value Demand (from 8% to 20%) – Preventive Maintenance – New Features or Services – New Capabilities • Mandates (from 18% to 30%) – Regulatory compliance 50% 30% 20% Failure Demand Mandates Value Demand Constrain effort to align with strategy
- 17. It’s All AboutChoices • Failure Demand – Take a long-term view – Perform root-cause analysis to find the source of greatest demand – Fix it – Repeat the process • Value Demand – Rank initiatives according to highest value – Estimate the size of the effort – Find the highest value with the least effort – Build that thing with the least delay – Repeat the process • Mandates – Fix the due date – Prioritize features using a MoSCoW model (Must, Should, Could, Won't) – Evaluate design options and favor the simplest approach, even if sub-optimal – Optimize workflow to maximize throughput
- 18. How we builditWhat we build Things We Can Measure Performance • Throughput • Cycle Time • Effort • Schedule • Inventory – Work Not Started – Work In Progress Value • ROI • Revenue • Cost • Cash Flow • Market Share • Growth • Quality
- 19. Value Goals • Increaserevenue • Increase market share • Increase customer satisfaction • Reduce cost • Manage risk
- 20. Features Deliver Value… •Improve customer satisfaction for both members and providers • Increase the number of claims processed per employee • Reduce administrative costs as a percentage of revenue Add claim status inquiry to web Add additional conditions to claims adjudication so that fewer claims "PEND" Reduce dependencies between systems by introducing service layer to architecture
- 21. … But OnlyWhen In Production • How long will you have to wait before the feature makes it into production? • How long will the feature have to be in use before it pays for development and begins to add value? Ideation Design Planning Coding Testing Staging Release Effort incurs development costs Time waiting for new features represents lost opportunity costs
- 22. Cost/Benefit Analysis ofFeatures • Begin by establishing the value proposition • Evaluate your solution options • Calculate development effort and cost of each • Calculate the cost of delay • Track adoption and use of the feature to measure value
- 23. Example: Claim Statuson the Web Every day, Customer Service handles some number of phone calls with certain percentage of those calls are inquiring into claim status. That leads us to the following analysis: • Assume 2400 phone calls each day on average • Assume 20% of them on average (480) are claim status inquiries • Assume 10 minutes per call on average • Assume $50 per hour is the average fully-loaded cost of a CSR • 4800 minutes or 80 hours per day on claim status inquiry equates to $4,000 per day • If there are 253 work days in a year, that equals $1,012,000 spent responding to claim status inquiry each year Note: the example shown is purely fictional and for illustration purposes only.
- 24. Evaluating Options Three solutionsare considered: 1. Do nothing 2. Develop a feature with fully-committed teams in four months at a cost of $255,000 3. Develop the same feature with partially- committed teams in sixteen months at a cost of $338,000
- 25. $- $20,000 $40,000 $60,000 $80,000 $100,000 $120,000 $140,000 $160,000 $180,000 Option 1 Option 2 Option3 Cost of Delay Total 3-year cost: 1. Do Nothing: $3,040,000 2. Dedicated team: $977,000 3. Shared team: $2,070,000 $1,093,400
- 26. Performance Goals • Increasethroughput • Reduce cycle time • Improve quality • Eliminate waste • Improve predictability
- 27. Better Processes ImprovePerformance • Increase throughput • Decrease schedule variance • Shorten feature cycle time Increase frequency of releases Negotiate scope Reduce WIP to move smaller batches through development
- 28. Goal: Improve Throughput •Features per quarter • Stories per sprint • Issues per day • Value per month Throughput is the rate at which a system achieves its goal in units of output over time
- 29. High Variability ReducesFlow Symptoms – Flow becomes unpredictable – Bottlenecks occur – Cycle time increases – Throughput goes down Response – Reduce WIP – Increase slack – Swarm around bottlenecks Knowledge work inherently has a high degree of variability
- 30. Flow Design Capacity: the maximumamount of work that a system is capable of completing over time Cycle Time: the amount of time that work spends in the system Utilization: the amount of capacity being used at a given time (%) Effective Capacity: the actual amount of work that a system is capable of completing over time WIP: the amount of work in that is in flight at a given time Slack: the amount of capacity not being used at a given time (%) Throughput: the average rate at which completed work exits the system Constraint: anything that limits throughput
- 31. Collecting Flow Data •Model value stream • Record transitions • Calculate cycle time • Calculate WIP Feature Ready Dev/Test Add'l Test Deploy Cycle Time (days) 1 2/1/13 2/2/13 2/2/13 2/3/13 2 2 2/1/13 2/2/13 2/3/13 2/3/13 2 3 2/1/13 2/3/13 2/3/13 2/5/13 4 4 2/1/13 2/3/13 2/4/13 2/5/13 4 5 2/1/13 2/3/13 2/6/13 2/7/13 6 6 2/1/13 2/4/13 2/9/13 2/9/13 8 7 2/1/13 2/4/13 2/9/13 2/9/13 8 8 2/1/13 2/6/13 2/9/13 2/10/13 9 Date Ready Dev/Test Add'l Test Deployed Total WIP 2/1/13 10 0 0 0 10 2/2/13 8 1 1 0 10 2/3/13 5 2 1 2 8 2/4/13 7 3 2 2 12 2/5/13 7 3 0 4 10 2/6/13 6 3 1 4 10 2/7/13 8 3 0 5 11 2/8/13 9 4 0 5 13 2/9/13 10 2 2 7 14 2/10/13 9 1 3 8 13
- 32. Cumulative Flow Diagram 0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 12 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 Ready Dev Test Deployed WIP (9 items) Throughput (1.25 items/day) Cycle Time (10 days)
- 33. Agile Teams • Team:a group of people linked together with common purpose • Backlog: Work identified, but not yet started • Sprint: a time-boxed interval of time during which work is completed • Story: a narrative about the feature to be developed with clear completion criteria • Story Points: an estimate of the size of a story compared to other stories • Velocity: team throughput, as measured by story points per sprint
- 34. Velocity Measures • Burndown:total work remaining • Burnup: total work completed • Scope Change: additions or subtractions of work from the backlog • Glidepath: estimated time remaining until the backlog is depleted based on available data Sprint Pts Completed Burnup Burndown Scope Changes Glidepath 0 750 900 1 40 40 790 80 915 2 90 130 700 850 3 10 140 790 100 784 4 75 215 765 50 719 5 120 335 565 -80 654 6 32 367 533 589 7 60 427 473 523 8 95 522 378 458 9 393 10 328 11 262 12 197 13 132 14 67 15 1 16 0
- 35. 0 100 200 300 400 500 600 700 800 900 1000 1 2 34 5 6 7 8 9 10 11 12 13 14 15 16 Burnup Burndown Glidepath Velocity Reporting Velocity = 65 pts/sprint
- 36. Velocity Pros andCons Used Well • Helps teams make commitments that they can keep • Helps estimate the depth of a backlog • Helps product owners know how much runway to prepare • Helps teams reflect on their efficiency and identify ways to improve • Accurately reflects the sustainable pace of a team • Emphasizes the importance of “done” Used Poorly • Misunderstood as a measure of team effectiveness • Misused as a measure of individual performance • Invites meaningless or damaging comparisons between teams • Leads to Inflated Velocity Syndrome (IVS), a misguided attempt to improve throughput by focusing on utilization rather than outcomes.
- 37. Commitment • Scope isnegotiable • Teams are self-governing • Deadlines matter • Done means done Agility is rooted in commitment
- 38. Commitment Variance 0 20 40 60 80 100 120 140 1 23 4 5 6 7 8 9 Plan Actual Range (-20%) Range (+20%)
- 39. A Good Metric •Supports your goals • Fits your process • Is fact-based • Difficult to manipulate
- 40. Inspect and Adapt Portfolio •Inspect: Demand (planned and unplanned) • Adapt: Control allocation of effort to demand categories Program • Inspect: Workflow; feature acceptance rates; value attainment • Adapt: Reduce WIP; negotiate feature scope and priority; remove bottlenecks; increase slack; smaller, more frequent releases Team • Inspect: Team throughput, commitment variance, quality • Adapt: Negotiate story scope and priority, remove impediments, regular retrospectives, continuous improvement