How Generative AI Boosts Agent Performance

AI isn't just a tool; it's becoming a teammate. A major field experiment with 776 professionals at Procter & Gamble, led by researchers from Harvard, Wharton, and Warwick, revealed something remarkable: Generative AI can replicate and even outperform human teamwork. Read the recently published paper here: In a real-world new product development challenge, professionals were assigned to one of four conditions: 1. Control Individuals without AI 2. Human Team R&D + Commercial without AI (+0.24 SD) 3. Individual + AI Working alone with GPT-4 (+0.37 SD) 4. AI-Augmented Team Human team + GPT-4 (+0.39 SD) Key findings: ⭐ Individuals with AI matched the output quality of traditional teams, with 16% less time spent. ⭐ AI helped non-experts perform like seasoned product developers. ⭐ It flattened functional silos: R&D and Commercial employees produced more balanced, cross-functional solutions. ⭐ It made work feel better: AI users reported higher excitement and energy and lower anxiety, even more so than many working in human-only teams. What does this mean for organizations? 💡 Rethink team structures. One AI-empowered individual can do the work of two and do it faster. 💡 Democratize expertise. AI is a boundary-spanning engine that reduces reliance on deep specialization. 💡 Invest in AI fluency. Prompting and AI collaboration skills are the new competitive edge. 💡 Double down on innovation. AI + team = highest chance of top-tier breakthrough ideas. This is not just productivity software. This is a redefinition of how work happens. AI is no longer the intern or the assistant. It’s showing up as a cybernetic teammate, enhancing performance, dissolving silos, and lifting morale. The future of work isn’t human vs. AI. The next step is human + AI + new ways of collaborating. Are you ready?

I have been thinking about the possible impacts of generative AI on tech-enabled services for schools. I am particularly interested in the applications in places where talent shortages are a barrier to scale quality services - i.e., tutoring, coaching, career counseling, etc. and I came across this insightful paper by MIT Sloan associate professor Danielle Li, MIT Sloan PhD candidate Lindsey Raymond, and Stanford University professor Erik Brynjolfsson, titled "Generative AI at Work." There are so many nuggets worth exploring in the paper, but below are a few that stand out. Summary: The paper studied the impact of a chat-based generative AI support tool across 5K+ customer support agents and 3M+ chat-based conversations. The AI support tool was meant to augment and not outright replace the contact center employees. The model was trained using historical data from the company's highest-performing workers, and it only offered prompts if it was "sufficiently confident" in its answers, which reduced the number of incorrect responses. In addition, workers weren't required to use the recommendations. Key Takeaways -The customer support workers in the "treatment" group only followed the AI recommendations ~30-40% of the time, which is consistent with the industry average for generative AI tools -Overall, workers using the generative AI model increased the number of customer chats resolved per hour by 13.8%, and requests to speak to a manager declined by 25%. Additionally, transfers to other departments tended to happen earlier in the conversation, which suggests that the AI model was able to help workers better match a customer's problem to the right business unit for a solution -Productivity gains were highest among workers with the least experience, who resolved 35% more chats per hour when they used the generative model. Productivity was flat for workers with the most skills and experience. -New workers using the AI tool were able to reach the same level of productivity in 2 months that typically took 8-10 months for workers not using the tool - showing solid signs of the ability to use AI to progress up the learning/experience curve rapidly -The use of the AI tool leads to reduced turnover rates. The strongest reductions in attrition were seen among newer agents, those with less than 6 months of experience. https://lnkd.in/gvfFmv-w  #k12 #edtech #k12design #k12schools #k12education #edtechchat

🧠 Is Generative AI Just Cool, or Does It Really Have an Impact? That's the big debate in tech circles these days. A study led by researchers from Stanford University, MIT, and the National Bureau of Economic Research (NBER) sheds light on this question by examining the real-world impact of deploying generative AI in a customer support environment. Their analysis offers empirical evidence on how AI tools, specifically those based on OpenAI's GPT models, are transforming customer service operations at a Fortune 500 software company. The researchers employed a mix of methodologies: a randomized control trial (RCT) and a staggered rollout, encompassing around 5,000 agents over several months. By analyzing 3 million customer-agent interactions, the study assessed metrics such as resolutions per hour, handle time, resolution rates, and customer satisfaction (Net Promoter Score). To understand the AI's impact over time, dynamic difference-in-differences regression models were used. Here is what they found: 1. 𝐒𝐢𝐠𝐧𝐢𝐟𝐢𝐜𝐚𝐧𝐭 𝐁𝐨𝐨𝐬𝐭 𝐢𝐧 𝐏𝐫𝐨𝐝𝐮𝐜𝐭𝐢𝐯𝐢𝐭𝐲: The AI tool led to a 13.8% increase in the number of customer queries resolved per hour, particularly benefiting less experienced agents. 2. 𝐍𝐚𝐫𝐫𝐨𝐰𝐢𝐧𝐠 𝐭𝐡𝐞 𝐏𝐞𝐫𝐟𝐨𝐫𝐦𝐚𝐧𝐜𝐞 𝐆𝐚𝐩: AI tools accelerated the learning curve for newer agents, allowing them to reach the performance levels of seasoned employees more quickly. 3. 𝐈𝐦𝐩𝐫𝐨𝐯𝐞𝐝 𝐂𝐮𝐬𝐭𝐨𝐦𝐞𝐫 𝐒𝐚𝐭𝐢𝐬𝐟𝐚𝐜𝐭𝐢𝐨𝐧: The AI deployment resulted in higher customer satisfaction scores (as shown by improved Net Promoter Scores) while maintaining stable employee sentiment. 4. 𝐋𝐨𝐰𝐞𝐫 𝐀𝐭𝐭𝐫𝐢𝐭𝐢𝐨𝐧 𝐑𝐚𝐭𝐞𝐬: Interestingly, the AI support led to reduced attrition rates, especially among new hires with less than six months of experience. 5. 𝐎𝐩𝐭𝐢𝐦𝐢𝐳𝐞𝐝 𝐖𝐨𝐫𝐤𝐟𝐥𝐨𝐰𝐬: The AI system reduced the need for escalations to managers, improving vertical efficiency. However, its impact on horizontal workflows, like transfers between agents, showed mixed results, suggesting more refinement is needed in AI integration. 6. 𝐂𝐮𝐬𝐭𝐨𝐦𝐢𝐳𝐞𝐝 𝐀𝐈 𝐌𝐚𝐭𝐭𝐞𝐫𝐬: The software wasn’t off-the-shelf; it was a custom-built solution tailored to the company’s needs using the GPT family of language models. This emphasizes the importance of context-specific AI applications for effective outcomes. For leaders, managers, and AI practitioners, these insights are invaluable—highlighting not just the potential of AI, but also the nuanced ways it reshapes workflows, impacts employee dynamics, and transforms customer experiences.So, does generative AI really make a difference? According to this study, the answer is a resounding yes—but it depends on how thoughtfully it is deployed. Link 🔗 to the paper: https://lnkd.in/ejhUfufz

Just out: Quantifying the impact of #genAI on job performance, by Erik Brynjolfsson & team: "Access to AI assistance increases worker productivity, as measured by issues resolved per hour, by 15% on average, with substantial heterogeneity across workers. The effects vary significantly across different agents. Less experienced and lower-skilled workers improve both the speed and quality of their output, while the most experienced and highest-skilled workers see small gains in speed and small declines in quality. We also find evidence that AI assistance facilitates worker learning and improves English fluency, particularly among international agents. While AI systems improve with more training data, we find that the gains from AI adoption are largest for moderately rare problems, where human agents have less baseline experience but the system still has adequate training data. Finally, we provide evidence that AI assistance improves the experience of work along several dimensions: customers are more polite and less likely to ask to speak to a manager." Open access: https://lnkd.in/d4UecpnQ