Evaluating Customer Experience Initiatives

Over the last year, I’ve seen many people fall into the same trap: They launch an AI-powered agent (chatbot, assistant, support tool, etc.)… But only track surface-level KPIs — like response time or number of users. That’s not enough. To create AI systems that actually deliver value, we need 𝗵𝗼𝗹𝗶𝘀𝘁𝗶𝗰, 𝗵𝘂𝗺𝗮𝗻-𝗰𝗲𝗻𝘁𝗿𝗶𝗰 𝗺𝗲𝘁𝗿𝗶𝗰𝘀 that reflect: • User trust • Task success • Business impact • Experience quality    This infographic highlights 15 𝘦𝘴𝘴𝘦𝘯𝘵𝘪𝘢𝘭 dimensions to consider: ↳ 𝗥𝗲𝘀𝗽𝗼𝗻𝘀𝗲 𝗔𝗰𝗰𝘂𝗿𝗮𝗰𝘆 — Are your AI answers actually useful and correct? ↳ 𝗧𝗮𝘀𝗸 𝗖𝗼𝗺𝗽𝗹𝗲𝘁𝗶𝗼𝗻 𝗥𝗮𝘁𝗲 — Can the agent complete full workflows, not just answer trivia? ↳ 𝗟𝗮𝘁𝗲𝗻𝗰𝘆 — Response speed still matters, especially in production. ↳ 𝗨𝘀𝗲𝗿 𝗘𝗻𝗴𝗮𝗴𝗲𝗺𝗲𝗻𝘁 — How often are users returning or interacting meaningfully? ↳ 𝗦𝘂𝗰𝗰𝗲𝘀𝘀 𝗥𝗮𝘁𝗲 — Did the user achieve their goal? This is your north star. ↳ 𝗘𝗿𝗿𝗼𝗿 𝗥𝗮𝘁𝗲 — Irrelevant or wrong responses? That’s friction. ↳ 𝗦𝗲𝘀𝘀𝗶𝗼𝗻 𝗗𝘂𝗿𝗮𝘁𝗶𝗼𝗻 — Longer isn’t always better — it depends on the goal. ↳ 𝗨𝘀𝗲𝗿 𝗥𝗲𝘁𝗲𝗻𝘁𝗶𝗼𝗻 — Are users coming back 𝘢𝘧𝘵𝘦𝘳 the first experience? ↳ 𝗖𝗼𝘀𝘁 𝗽𝗲𝗿 𝗜𝗻𝘁𝗲𝗿𝗮𝗰𝘁𝗶𝗼𝗻 — Especially critical at scale. Budget-wise agents win. ↳ 𝗖𝗼𝗻𝘃𝗲𝗿𝘀𝗮𝘁𝗶𝗼𝗻 𝗗𝗲𝗽𝘁𝗵 — Can the agent handle follow-ups and multi-turn dialogue? ↳ 𝗨𝘀𝗲𝗿 𝗦𝗮𝘁𝗶𝘀𝗳𝗮𝗰𝘁𝗶𝗼𝗻 𝗦𝗰𝗼𝗿𝗲 — Feedback from actual users is gold. ↳ 𝗖𝗼𝗻𝘁𝗲𝘅𝘁𝘂𝗮𝗹 𝗨𝗻𝗱𝗲𝗿𝘀𝘁𝗮𝗻𝗱𝗶𝗻𝗴 — Can your AI 𝘳𝘦𝘮𝘦𝘮𝘣𝘦𝘳 𝘢𝘯𝘥 𝘳𝘦𝘧𝘦𝘳 to earlier inputs? ↳ 𝗦𝗰𝗮𝗹𝗮𝗯𝗶𝗹𝗶𝘁𝘆 — Can it handle volume 𝘸𝘪𝘵𝘩𝘰𝘶𝘵 degrading performance? ↳ 𝗞𝗻𝗼𝘄𝗹𝗲𝗱𝗴𝗲 𝗥𝗲𝘁𝗿𝗶𝗲𝘃𝗮𝗹 𝗘𝗳𝗳𝗶𝗰𝗶𝗲𝗻𝗰𝘆 — This is key for RAG-based agents. ↳ 𝗔𝗱𝗮𝗽𝘁𝗮𝗯𝗶𝗹𝗶𝘁𝘆 𝗦𝗰𝗼𝗿𝗲 — Is your AI learning and improving over time? If you're building or managing AI agents — bookmark this. Whether it's a support bot, GenAI assistant, or a multi-agent system — these are the metrics that will shape real-world success. 𝗗𝗶𝗱 𝗜 𝗺𝗶𝘀𝘀 𝗮𝗻𝘆 𝗰𝗿𝗶𝘁𝗶𝗰𝗮𝗹 𝗼𝗻𝗲𝘀 𝘆𝗼𝘂 𝘂𝘀𝗲 𝗶𝗻 𝘆𝗼𝘂𝗿 𝗽𝗿𝗼𝗷𝗲𝗰𝘁𝘀? Let’s make this list even stronger — drop your thoughts 👇

Getting the right feedback will transform your job as a PM. More scalability, better user engagement, and growth. But most PMs don’t know how to do it right. Here’s the Feedback Engine I’ve used to ship highly engaging products at unicorns & large organizations: — Right feedback can literally transform your product and company. At Apollo, we launched a contact enrichment feature. Feedback showed users loved its accuracy, but... They needed bulk processing. We shipped it and had a 40% increase in user engagement. Here’s how to get it right: — 𝗦𝘁𝗮𝗴𝗲 𝟭: 𝗖𝗼𝗹𝗹𝗲𝗰𝘁 𝗙𝗲𝗲𝗱𝗯𝗮𝗰𝗸 Most PMs get this wrong. They collect feedback randomly with no system or strategy. But remember: your output is only as good as your input. And if your input is messy, it will only lead you astray. Here’s how to collect feedback strategically: → Diversify your sources: customer interviews, support tickets, sales calls, social media & community forums, etc. → Be systematic: track feedback across channels consistently. → Close the loop: confirm your understanding with users to avoid misinterpretation. — 𝗦𝘁𝗮𝗴𝗲 𝟮: 𝗔𝗻𝗮𝗹𝘆𝘇𝗲 𝗜𝗻𝘀𝗶𝗴𝗵𝘁𝘀 Analyzing feedback is like building the foundation of a skyscraper. If it’s shaky, your decisions will crumble. So don’t rush through it. Dive deep to identify patterns that will guide your actions in the right direction. Here’s how: Aggregate feedback → pull data from all sources into one place. Spot themes → look for recurring pain points, feature requests, or frustrations. Quantify impact → how often does an issue occur? Map risks → classify issues by severity and potential business impact. — 𝗦𝘁𝗮𝗴𝗲 𝟯: 𝗔𝗰𝘁 𝗼𝗻 𝗖𝗵𝗮𝗻𝗴𝗲𝘀 Now comes the exciting part: turning insights into action. Execution here can make or break everything. Do it right, and you’ll ship features users love. Mess it up, and you’ll waste time, effort, and resources. Here’s how to execute effectively: Prioritize ruthlessly → focus on high-impact, low-effort changes first. Assign ownership → make sure every action has a responsible owner. Set validation loops → build mechanisms to test and validate changes. Stay agile → be ready to pivot if feedback reveals new priorities. — 𝗦𝘁𝗮𝗴𝗲 𝟰: 𝗠𝗲𝗮𝘀𝘂𝗿𝗲 𝗜𝗺𝗽𝗮𝗰𝘁 What can’t be measured, can’t be improved. If your metrics don’t move, something went wrong. Either the feedback was flawed, or your solution didn’t land. Here’s how to measure: → Set KPIs for success, like user engagement, adoption rates, or risk reduction. → Track metrics post-launch to catch issues early. → Iterate quickly and keep on improving on feedback. — In a nutshell... It creates a cycle that drives growth and reduces risk: → Collect feedback strategically. → Analyze it deeply for actionable insights. → Act on it with precision. → Measure its impact and iterate. — P.S. How do you collect and implement feedback?

Surveys can serve an important purpose. We should use them to fill holes in our understanding of the customer experience or build better models with the customer data we have. As surveys tell you what customers explicitly choose to share, you should not be using them to measure the experience. Surveys are also inherently reactive, surface level, and increasingly ignored by customers who are overwhelmed by feedback requests. This is fact. There’s a different way. Some CX leaders understand that the most critical insights come from sources customers don’t even realize they’re providing from the “exhaust” of every day life with your brand. Real-time digital behavior, social listening, conversational analytics, and predictive modeling deliver insights that surveys alone never will. Voice and sentiment analytics, for example, go beyond simply reading customer comments. They reveal how customers genuinely feel by analyzing tone, frustration, or intent embedded within interactions. Behavioral analytics, meanwhile, uncover friction points by tracking real customer actions across websites or apps, highlighting issues users might never explicitly complain about. Predictive analytics are also becoming essential for modern CX strategies. They anticipate customer needs, allowing businesses to proactively address potential churn, rather than merely reacting after the fact. The capability can also help you maximize revenue in the experiences you are delivering (a use case not discussed often enough). The most forward-looking CX teams today are blending traditional feedback with these deeper, proactive techniques, creating a comprehensive view of their customers. If you’re just beginning to move beyond a survey-only approach, prioritizing these more advanced methods will help ensure your insights are not only deeper but actionable in real time. Surveys aren’t dead (much to my chagrin), but relying solely on them means leaving crucial insights behind. While many enterprises have moved beyond surveys, the majority are still overly reliant on them. And when you get to mid-market or small businesses? The survey slapping gets exponentially worse. Now is the time to start looking beyond the questionnaire and your Likert scales. The email survey is slowly becoming digital dust. And the capabilities to get you there are readily available. How are you evolving your customer listening strategy beyond traditional surveys? #customerexperience #cxstrategy #customerinsights #surveys

CSAT measurement must be more than just a score. Many companies prioritize their Net Promoter Score (NPS) as a measure of Customer Satisfaction (CSAT). But do these methods truly give us a complete understanding? In reality, surveys are not always accurate. Bias can influence the results, ratings may be misinterpreted, and there's a chance that we didn't even ask the right questions. While a basic survey can indicate problems, the true value lies in comprehending the reasons behind those scores and identifying effective solutions to improve them. Here’s a better way to look at CSAT: 1. Start with Actions, Not Just Scores: Observable behaviors like repeat purchases, referrals, and product usage often tell a more accurate story than a survey score alone. 2. Analyze Digital Signals & Employee Feedback: Look for objective measures that consumers are happy with what you offer (website micro-conversions like page depth, time on site, product views and cart adds). And don’t forget your team! Happy employees = Happy customers. 3. Understand the Voice of the Customer (VoC): Utilize AI tools to examine customer feedback, interactions with customer support, and comments on social media platforms in order to stay updated on the current attitudes towards your brand. 4. Make It a Closed Loop: Gathering feedback is only the beginning. Use it to drive change. Your customers need to know you’re listening — and *acting*. Think of your CSAT score as a signal that something happened in your customer relationships. But to truly improve your business, you must pinpoint the reasons behind those scores and use that information to guide improvements. Don’t settle for simply knowing that something happened, find an answer for why it happened. Art+Science Analytics Institute | University of Notre Dame | University of Notre Dame - Mendoza College of Business | University of Illinois Urbana-Champaign | University of Chicago | D'Amore-McKim School of Business at Northeastern University | ELVTR | Grow with Google - Data Analytics #Analytics #DataStorytelling

If you're a UX researcher working with open-ended surveys, interviews, or usability session notes, you probably know the challenge: qualitative data is rich - but messy. Traditional coding is time-consuming, sentiment tools feel shallow, and it's easy to miss the deeper patterns hiding in user feedback. These days, we're seeing new ways to scale thematic analysis without losing nuance. These aren’t just tweaks to old methods - they offer genuinely better ways to understand what users are saying and feeling. Emotion-based sentiment analysis moves past generic “positive” or “negative” tags. It surfaces real emotional signals (like frustration, confusion, delight, or relief) that help explain user behaviors such as feature abandonment or repeated errors. Theme co-occurrence heatmaps go beyond listing top issues and show how problems cluster together, helping you trace root causes and map out entire UX pain chains. Topic modeling, especially using LDA, automatically identifies recurring themes without needing predefined categories - perfect for processing hundreds of open-ended survey responses fast. And MDS (multidimensional scaling) lets you visualize how similar or different users are in how they think or speak, making it easy to spot shared mindsets, outliers, or cohort patterns. These methods are a game-changer. They don’t replace deep research, they make it faster, clearer, and more actionable. I’ve been building these into my own workflow using R, and they’ve made a big difference in how I approach qualitative data. If you're working in UX research or service design and want to level up your analysis, these are worth trying.

CX Should Be Measured Like a P&L—Not a Sentiment Score We keep measuring Customer Experience with smiley faces, stars, and survey scores. But here’s the reality: If you can’t tie CX to revenue, retention, or cost savings—it’s not strategic. Too many CX teams report on sentiment. Fewer can show the business impact of improving the experience. Want a seat at the executive table? Start thinking like a P&L owner: ✅ Reduce onboarding friction → Faster time-to-revenue ✅ Improve digital containment → Lower cost-to-serve ✅ Decrease churn triggers → Higher customer lifetime value This is how you move from “nice to have” to business critical. Sentiment is a signal. Value is the outcome. 💬 How are you measuring CX in your org? Can you show the CFO how experience drives ROI?

𝗧𝗵𝗲 𝘁𝗿𝘂𝘁𝗵 𝗮𝗯𝗼𝘂𝘁 𝗩𝗼𝗶𝗰𝗲 𝗼𝗳 𝗖𝘂𝘀𝘁𝗼𝗺𝗲𝗿? It’s broken. Not because customers stopped speaking, but because brands stopped listening like it mattered. Surveys. Scores. Dashboards. 𝗧𝗵𝗮𝘁’𝘀 𝗻𝗼𝘁 𝗹𝗶𝘀𝘁𝗲𝗻𝗶𝗻𝗴. That’s forced interaction. The modern customer isn’t waiting to be surveyed. They’re 𝘭𝘦𝘢𝘷𝘪𝘯𝘨 𝘴𝘪𝘨𝘯𝘢𝘭𝘴 𝘦𝘷𝘦𝘳𝘺𝘸𝘩𝘦𝘳𝘦 - in chats, returns, reviews, support tickets, SMS threads, order cancellations, product reconfigurations, social media, dark social (Reddit, Discord, etc) But most “VoC programs” are still stuck chasing NPS trends while the business burns. Modern Voice of Customer = 𝗘𝘅𝗽𝗲𝗿𝗶𝗲𝗻𝗰𝗲 𝗦𝗶𝗴𝗻𝗮𝗹 𝗜𝗻𝘁𝗲𝗹𝗹𝗶𝗴𝗲𝗻𝗰𝗲 It’s not about asking questions. It’s about 𝗮𝗿𝗰𝗵𝗶𝘁𝗲𝗰𝘁𝗶𝗻𝗴 𝗮 𝘀𝘆𝘀𝘁𝗲𝗺 that 𝘢𝘣𝘴𝘰𝘳𝘣𝘴 𝘴𝘪𝘨𝘯𝘢𝘭, connects it to business outcomes, and triggers action. What You Should Be Measuring Instead: ✅ % 𝗔𝗰𝘁𝗶𝗼𝗻𝗮𝗯𝗹𝗲 𝗦𝗶𝗴𝗻𝗮𝗹𝘀 𝗜𝗱𝗲𝗻𝘁𝗶𝗳𝗶𝗲𝗱 - How much of your incoming feedback actually maps to a real friction point, journey stage, or operational failure? ✅ % 𝗦𝗶𝗴𝗻𝗮𝗹𝘀 𝗧𝗶𝗲𝗱 𝘁𝗼 𝗕𝘂𝘀𝗶𝗻𝗲𝘀𝘀 𝗜𝗺𝗽𝗮𝗰𝘁 - How many of those signals correlate with churn, CLV drop, conversion loss, or increased cost-to-serve? ✅ 𝗖𝗼𝗻𝘁𝗲𝘅𝘁𝘂𝗮𝗹 𝗦𝗲𝗻𝘁𝗶𝗺𝗲𝗻𝘁 (𝗡𝗼𝘁 𝗝𝘂𝘀𝘁 𝗮 𝗦𝗰𝗼𝗿𝗲) - Not “61% negative.” But: “61% 𝘯𝘦𝘨𝘢𝘵𝘪𝘷𝘦 𝘴𝘦𝘯𝘵𝘪𝘮𝘦𝘯𝘵 𝘢𝘳𝘰𝘶𝘯𝘥 𝘥𝘦𝘭𝘪𝘷𝘦𝘳𝘺 𝘴𝘱𝘦𝘦𝘥 𝘵𝘳𝘢𝘯𝘴𝘱𝘢𝘳𝘦𝘯𝘤𝘺.” “78% 𝘱𝘰𝘴𝘪𝘵𝘪𝘷𝘦 𝘴𝘦𝘯𝘵𝘪𝘮𝘦𝘯𝘵 𝘰𝘯 𝘱𝘰𝘴𝘵-𝘱𝘶𝘳𝘤𝘩𝘢𝘴𝘦 𝘴𝘶𝘱𝘱𝘰𝘳𝘵.” That tells a story. That’s signal intelligence. ✅ 𝗦𝗶𝗴𝗻𝗮𝗹 𝗩𝗲𝗹𝗼𝗰𝗶𝘁𝘆 - What’s emerging fast? What’s fading out? Velocity = your 𝘦𝘢𝘳𝘭𝘺 𝘸𝘢𝘳𝘯𝘪𝘯𝘨 𝘳𝘢𝘥𝘢𝘳. ✅ 𝗙𝗿𝗶𝗰𝘁𝗶𝗼𝗻 𝗙𝗮𝘁𝗶𝗴𝘂𝗲 𝗦𝗰𝗼𝗿𝗲 How often is the same friction mentioned with no resolution? High friction fatigue = 𝗹𝗼𝘀𝘁 𝘁𝗿𝘂𝘀𝘁. Your brand becomes a broken record and customers stop playing. CX isn't a function of feedback. It’s a function of 𝘀𝗶𝗴𝗻𝗮𝗹 𝗽𝗲𝗿𝗳𝗼𝗿𝗺𝗮𝗻𝗰𝗲. You don’t need another dashboard. You need a listening architecture that fuels performance. That’s Experience Signal Intelligence. #UnfckYourCX #ExperiencePerformanceSystem  #ExperienceDesign #SignalIntelligence #CLV #VoC #NPS #surveys

As CX programs are being cut, it’s becoming clear that those focused solely on survey scores are at risk. To truly drive value, B2B CX programs must tie their efforts to financial outcomes—a critical connection many programs miss. One simple but powerful metric to consider is order velocity—the frequency of customer orders, regardless of size or type. By combining the order data with good survey questions, you can track how improved customer experiences lead to faster order velocity. While it’s not the final financial metric, it gives you an early indication of CX impact. Order velocity works especially well in industries with less frequent transactions, like B2B insurance. For example, if brokers typically average six policies yearly, an improved experience should lead to more orders the following year. If not, it could signal that your surveys aren’t targeting the right issues or that other factors, like pricing, are having a larger impact. Remember, there’s often a delay between shifts in customer attitudes and changes in behavior. In industries like health insurance, a boost in CX scores during mid-year could drive more orders by Q4. In manufacturing, the timeline might vary—tactical orders may rise quickly, while long-term sales like turbines could take years to reflect the change. For a more holistic view, pair order velocity with client-specific metrics like margin per client or number of categories ordered. Order velocity is relatively easy to track and is a great entry point for deeper insights. Reporting on this invites questions from leadership—and when the right questions are asked, it paves the way for gathering more valuable data. #CX #CXROI #Customerexperience

The MOST critical metric you can use to measure customer satisfaction: (This changed everything for my company) We had a daily deal site with 2 million users. Sounds great, right? But about 18 months in we had a massive problem: → Customer satisfaction was TANKING (we were in the daily-deals business, largest Groupon competitor) Why? Our customers weren't getting the same experience as full-paying customers. They were treated as “coupon buyers”, so they: - Had long wait-times - Didn't get the same food - Got given the cr*ppy tables at the back They went for the full service and they got very low-quality service. And it was KILLING our business model. We tried everything - customer service calls, merchant meetings, forums. Nothing worked. Then I learned about NPS (Net Promoter Score) at EO and MIT Masters. It was an ABSOLUTE revelation. NPS isn't a boring survey asking "How happy are you with our service?" It's way more powerful. It asks, on a simple scale of 0-10: → "How likely are you to recommend this service to a friend or colleague?" 10-9 → Promoters (Nice!) 8-7 → Passive (no need to do anything) 6-0 → Detractors (fix this NOW) It’s such a simple shift on our end and so easy to respond on the customer end: “Hey, would you recommend me or not, out of 10?” “Hm, 7.” “Ok, thank you” — that’s it. Simple reframe, massive impact. We implemented it immediately. But here's the real gold: → We contacted everyone (one-on-one customer service) who used our service and provided a NPS score. They scored us less than 6? - Give them gift cards - Interview them to make them feel heard - Do ANYTHING to flip detractors into promoters Because if they’re scoring you less than 6, they’re actually HARMING your business. These are going to be like e-brakes in your company. NPS became our most important metric, integrated into everything we did. The results? - Improved customer satisfaction - Increased repeat business and customer LTV - Lower CAC (because happy customers = free marketing) - Higher AOV (people were willing to spend more) But it's not just about the numbers. It's about understanding WHY people aren't recommending you and fixing it fast. (Another great feature is that people can also add comments to get some real feedback, but just using the number is POWERFUL). If you're not using NPS, stop what you're doing and implement it tonight. Seriously. And if you are already using it? Double down on those 0-6 scores. Turning your detractors into promoters is where the real growth potential lies. Remember: in business, what gets measured gets managed. And NPS is the ultimate measure of how satisfied your customers REALLY are. So, what's your score? — Found value in this? Repost ♻️ to share to your network and follow Ignacio Carcavallo for more like this!