Metrics for Evaluating Change Management Outcomes

Last month, our AI tool adoption rate reached 62.5% among 40 engineers. But that number only tells part of the story. When I shared our change management approach and experimentation framework in previous posts, many of you asked: "How do you actually measure success?" The answer? We have built a comprehensive tracking system that focuses on encouragement, rather than enforcement. 1. Make it visible everywhere. We keep AI adoption front-of-mind through: Bi-weekly NPS surveys (54.5 current score) Monthly Community of Practice meetings Active Slack channel for sharing wins and learnings Real-time usage dashboards are shared team-wide The key insight: visibility drives curiosity, which in turn drives adoption. 2. Track both tools AND outcomes. We monitor two distinct categories: - Agentic Development tools (Copilot, Claude, Cursor) - Conversational AI (ChatGPT, Gemini, Claude) But here's what most teams miss—we also track work outcomes by tagging Jira tickets as "agentic_success" or "agentic_failure." This connects tool usage to actual impact. 3. Focus on insights, not enforcement. Our bi-weekly surveys don't just ask "did you use AI?" They capture: - Which specific tools do teams prefer - Key insights from their experiments - Barriers preventing adoption - Success stories worth sharing The result? 4.8M+ tokens used, 678% growth month-over-month, and most importantly—engineers actively sharing what works. Remember: this isn't about forcing adoption through metrics. It's about creating transparency that encourages experimentation. The dashboard becomes a conversation starter, not a performance review. What metrics have you found most valuable for tracking innovation adoption in your teams? P.S. Links to the change management and experimentation posts in the comments for those catching up on the series. #AIAdoption #EngineeringLeadership #TechTransformation #AgileMetrics

Most change initiatives are measured by one number: Adoption. Did people start using the new system? Did they attend the training? Did they log in? But just because something was adopted doesn’t mean the change worked. Adoption tells you if people used it. It doesn’t tell you how well they’re using it or whether it made anything better. To really measure change success, you need to go deeper: – Is behavior actually different? Are people making decisions in a new way? Are old habits starting to fade? – Is performance improving? Has the change helped teams deliver better results, faster service, fewer errors, or stronger collaboration? – Is the change sustainable? Are people still using the new way of working 3, 6, 12 months later or did things quietly go back to how they were? – Do people understand why the change matters? Real change sticks when people connect it to their purpose, not just their process. Success isn’t just about launch day. It’s about what happens after, when the excitement fades and the real work begins.

KPIs tell you what happened. KCIs tell you what's happening. It's hard to defend investments in change and transformation activities because we're only tracking Key Performance Indicators—the lagging results of change. By the time KPIs appear, it's too late to defend our investments if we're getting it right, and it's too late to course-correct if we got it wrong. Smart change leaders track Key Change Indicators alongside KPIs. Employee adoption rates, collaboration patterns, and engagement scores are the early warning system for transformation success. The best part? This data already exists in your Microsoft Graph, Slack logs, and LMS systems. We just need to surface KCIs and start using data to track and prove the value of strategy activation investments. #strategyactivation #changeleadership