Importance of User Trust in Data Products

𝔼𝕍𝔸𝕃 field note (2 of 3): Finding the benchmarks that matter for your own use cases is one of the biggest contributors to AI success. Let's dive in. AI adoption hinges on two foundational pillars: quality and trust. Like the dual nature of a superhero, quality and trust play distinct but interconnected roles in ensuring the success of AI systems. This duality underscores the importance of rigorous evaluation. Benchmarks, whether automated or human-centric, are the tools that allow us to measure and enhance quality while systematically building trust. By identifying the benchmarks that matter for your specific use case, you can ensure your AI system not only performs at its peak but also inspires confidence in its users. 🦸♂️ Quality is the superpower—think Superman—able to deliver remarkable feats like reasoning and understanding across modalities to deliver innovative capabilities. Evaluating quality involves tools like controllability frameworks to ensure predictable behavior, performance metrics to set clear expectations, and methods like automated benchmarks and human evaluations to measure capabilities. Techniques such as red-teaming further stress-test the system to identify blind spots. 👓 But trust is the alter ego—Clark Kent—the steady, dependable force that puts the superpower into the right place at the right time, and ensures these powers are used wisely and responsibly. Building trust requires measures that ensure systems are helpful (meeting user needs), harmless (avoiding unintended harm), and fair (mitigating bias). Transparency through explainability and robust verification processes further solidifies user confidence by revealing where a system excels—and where it isn’t ready yet. For AI systems, one cannot thrive without the other. A system with exceptional quality but no trust risks indifference or rejection - a collective "shrug" from your users. Conversely, all the trust in the world without quality reduces the potential to deliver real value. To ensure success, prioritize benchmarks that align with your use case, continuously measure both quality and trust, and adapt your evaluation as your system evolves. You can get started today: map use case requirements to benchmark types, identify critical metrics (accuracy, latency, bias), set minimum performance thresholds (aka: exit criteria), and choose complementary benchmarks (for better coverage of failure modes, and to avoid over-fitting to a single number). By doing so, you can build AI systems that not only perform but also earn the trust of their users—unlocking long-term value.

*To build trust in complexity, offer small choices and fast feedback* I strongly believe product simplicity and predictability are a superpower. They give the user a sense of control, which is a gift when the world feels so complicated. But some things are legitimately complex. What gives the user a sense of control when predictability is hard to come by? My take: Giving the user a chance to *participate* in the process by laying out steps, enabling them to make specific choices, and offering a clear feedback loop on each small decision. This may make the flow longer, but it gives users a chance to viscerally understand what’s happening. A while ago I got an alarming privacy notification on an important account. I was immediately worried. But the product’s recovery flow calmed me down. Why? It: 1. Laid out all the steps I’d go through, giving me a clear roadmap for what to do. 2. Channeled my anxiety into actions, even if they were small. There were prompts like “Check whether password is compromised? Yes / No”. Is that a necessary prompt? Who would say “no”? But in the moment, the ability to participate in the process of securing my account gave me a sense of control. 3. Gave me fast feedback on each choice by turning each step green on completion. By the end of the list, I felt a sense of relief. Realistically, that product could have taken all those actions without my input. But getting to participate in each step gave me a sense of control. I saw the same thing with a new AI tool my team was working on. Our temptation was to take user input up front and come back with a solution. But our customers didn’t yet trust the magic black box of AI recommendations. Instead, what helped was inserting feedback steps explaining what we were considering and offering the user a chance to change direction at each step. It added friction, but it built trust faster. Then over time, we could remove those interim feedback steps and automatically make decisions. Compare that to a customer service page where you type a question into a contact form and get a message that says, “Thanks, we’ll take care of it.” You don’t really get an understanding of the overall process, a chance to make smaller decisions, or feedback on whether you made the right choices. I’m always stressed about whether I did it right! This applies to people too. When I’m building a relationship with a new manager or peer, I try to frequently outline what I’m doing and give them a chance to redirect. After a few weeks, we know each others’ style and I can stop. Action is the best antidote to fear. Especially when someone is stressed out and longing for control, it helps to ground them in a clear step-by-step process, give them a chance to participate in solving their problem, and letting them know the impact of each choice. That naturally creates some relief, and helps them channel their concern into action. (For regular updates, check out amivora.substack.com!)

AI products like Cursor, Bolt and Replit are shattering growth records not because they're "AI agents". Or because they've got impossibly small teams (although that's cool to see 👀). It's because they've mastered the user experience around AI, somehow balancing pro-like capabilities with B2C-like UI. This is product-led growth on steroids. Yaakov Carno tried the most viral AI products he could get his hands on. Here are the surprising patterns he found: (Don't miss the full breakdown in today's bonus Growth Unhinged: https://lnkd.in/ehk3rUTa) 1. Their AI doesn't feel like a black box. Pro-tips from the best: - Show step-by-step visibility into AI processes - Let users ask, “Why did AI do that?” - Use visual explanations to build trust. 2. Users don’t need better AI—they need better ways to talk to it. Pro-tips from the best: - Offer pre-built prompt templates to guide users. - Provide multiple interaction modes (guided, manual, hybrid). - Let AI suggest better inputs ("enhance prompt") before executing an action. 3. The AI works with you, not just for you. Pro-tips from the best: - Design AI tools to be interactive, not just output-driven. - Provide different modes for different types of collaboration. - Let users refine and iterate on AI results easily. 4. Let users see (& edit) the outcome before it's irreversible. Pro-tips from the best: - Allow users to test AI features before full commitment (many let you use it without even creating an account). - Provide preview or undo options before executing AI changes. - Offer exploratory onboarding experiences to build trust. 5. The AI weaves into your workflow, it doesn't interrupt it. Pro-tips from the best: - Provide simple accept/reject mechanisms for AI suggestions. - Design seamless transitions between AI interactions. - Prioritize the user’s context to avoid workflow disruptions. -- The TL;DR: Having "AI" isn’t the differentiator anymore—great UX is. Pardon the Sunday interruption & hope you enjoyed this post as much as I did 🙏 #ai #genai #ux #plg

74% of business executives trust AI advice more than their colleagues, friends, or even family. Yes, you read that right. AI has officially become the most trusted voice in the room, according to recent research by SAP. That’s not just a tech trend — that’s a human trust shift. And we should be paying attention. What can we learn from this? 🔹 AI is no longer a sidekick. It’s a decision-maker, an advisor, and in some cases… the new gut instinct. 🔹 But trust in AI is only good if the AI is worth trusting. Blind trust in black-box systems is as dangerous as blind trust in bad leaders. So here’s what we should do next: ✅ Question the AI you trust Would you take strategic advice from someone you’ve never questioned? Then don’t do it with AI. Check its data, test its reasoning, and simulate failure. Trust must be earned — even by algorithms. ✅ Make AI explain itself Trust grows with transparency. Build “trust dashboards” that show confidence scores, data sources, and risk levels. No more “just because it said so.” ✅ Use AI to enhance leadership, not replace it Smart executives will use AI as a mirror — for self-awareness, productivity, communication. Imagine an AI coach that preps your meetings, flags bias in decisions, or tracks leadership tone. That’s where we’re headed. ✅ Rebuild human trust, too This stat isn’t just about AI. It’s a signal that many execs don’t feel heard, supported, or challenged by those around them. Let’s fix that. 💬 And finally — trust in AI should look a lot like trust in people: Consistency, Transparency, Context, Integrity, and Feedback. If your AI doesn’t act like a good teammate, it doesn’t deserve to be trusted like one. What do you think? 👇 Are we trusting AI too much… or not enough? #SAPAmbassador #AI #Leadership #Trust #DigitalTransformation #AgenticAI #FutureOfWork #ArtificialIntelligence #EnterpriseAI #AIethics #DecisionMaking

You can’t democratize what you can’t trust. For months, the primary conceit of enterprise AI has been that it would create access. Data scientists could create pipelines like data engineers. Stakeholders could query the data like scientists. Everyone from the CEO to the intern could spin up dashboards and programs and customer comms in seconds. But is that actually a good thing? What if your greatest new superpower was actually your achilles heal in disguise? Data + AI trust is THE prerequisite for a safe and successful AI agent. If you can’t trust the underlying data, system, code, and model responses that comprise the system, you can’t trust the agent it’s powering. For the last 12 months, executives have been pressuring their teams to adopt more comprehensive AI strategies. But before any organization can give free access to data and AI resources, they need rigorous tooling and processes in place to protect its integrity end-to-end. That means leveraging automated and AI-enabled solutions to scale monitoring and resolutions, and measure adherence to standards and SLAs over time. AI-readiness is the first step to AI-adoption. You can't put the cart before the AI horse.

The Personalization-Privacy Paradox: AI in customer experience is most effective when it personalizes interactions based on vast amounts of data. It anticipates needs, tailors recommendations, and enhances satisfaction by learning individual preferences. The more data it has, the better it gets. But here’s the paradox: the same customers who crave personalized experiences can also be deeply concerned about their privacy. AI thrives on data, but customers resist sharing it. We want hyper-relevant interactions without feeling surveilled. As AI improves, this tension only increases. AI systems can offer deep personalization while simultaneously eroding the very trust needed for customers to willingly share their data. This paradox is particularly problematic because both extremes seem necessary: AI needs data for personalization, but excessive data collection can backfire, leading to customer distrust, dissatisfaction, or even churn. So how do we fix it? Be transparent. Tell people exactly what you’re using their data for—and why it benefits them. Let the customer choose. Give control over what’s personalized (and what’s not). Show the value. Make personalization a perk, not a tradeoff. Personalization shouldn’t feel like surveillance. It should feel like service. You can make this invisible too. Give the customer “nudges” to move them down the happy path through experience orchestration. Trust is the real unlock. Everything else is just prediction. #cx #ai #privacy #trust #personalization

Generative AI is transforming industries, but as adoption grows, so does the need for trust and reliability. Evaluation frameworks ensure that generative AI models perform as intended—not just in controlled environments, but in the real world. Key Insights from GCP Blog : Scalable Evaluation - new batch evaluation API allows you to assess large datasets efficiently, making it easier to validate model performance at scale. Customizable Autoraters - Benchmark automated raters against human judgments to build confidence in your evaluation process and highlight areas for improvement. Agentic Workflow Assessment - For AI agents, evaluate not just the final output, but also the reasoning process, tool usage, and decision trajectory. Continuous Monitoring - Implement ongoing evaluation to detect performance drift and ensure models remain reliable as data and user needs evolve. - Key Security Considerations: - Data Privacy: Ensure models do not leak sensitive information and comply with data protection regulations - Bias and Fairness: Regularly test for unintended bias and implement mitigation strategies[3]. - Access Controls:Restrict model access and implement audit trails to track usage and changes. - Adversarial Testing:Simulate attacks to identify vulnerabilities and strengthen model robustness **My Perspective: ** I see robust evaluation and security as the twin pillars of trustworthy AI. #Agent Evaluation is Evolving : Modern AI agent evaluation goes beyond simple output checks. It now includes programmatic assertions, embedding-based similarity scoring, and grading the reasoning path—ensuring agents not only answer correctly but also think logically and adapt to edge cases. Automated evaluation frameworks, augmented by human-in-the-loop reviewers, bring both scale and nuance to the process. - Security is a Lifecycle Concern: Leading frameworks like OWASP Top 10 for LLMs, Google’s Secure AI Framework (SAIF), and NIST’s AI Risk Management Framework emphasize security by design—from initial development through deployment and ongoing monitoring. Customizing AI architecture, hardening models against adversarial attacks, and prioritizing input sanitization are now standard best practices. - Continuous Improvement: The best teams integrate evaluation and security into every stage of the AI lifecycle, using continuous monitoring, anomaly detection, and regular threat modeling to stay ahead of risks and maintain high performance. - Benchmarking and Transparency: Standardized benchmarks and clear evaluation criteria not only drive innovation but also foster transparency and reproducibility—key factors for building trust with users and stakeholders. Check GCP blog post here: [How to Evaluate Your Gen AI at Every Stage](https://lnkd.in/gDkfzBs8) How are you ensuring your AI solutions are both reliable and secure?

✳ Bridging Ethics and Operations in AI Systems✳ Governance for AI systems needs to balance operational goals with ethical considerations. #ISO5339 and #ISO24368 provide practical tools for embedding ethics into the development and management of AI systems. ➡Connecting ISO5339 to Ethical Operations  ISO5339 offers detailed guidance for integrating ethical principles into AI workflows. It focuses on creating systems that are responsive to the people and communities they affect. 1. Engaging Stakeholders  Stakeholders impacted by AI systems often bring perspectives that developers may overlook. ISO5339 emphasizes working with users, affected communities, and industry partners to uncover potential risks and ensure systems are designed with real-world impact in mind. 2. Ensuring Transparency  AI systems must be explainable to maintain trust. ISO5339 recommends designing systems that can communicate how decisions are made in a way that non-technical users can understand. This is especially critical in areas where decisions directly affect lives, such as healthcare or hiring. 3. Evaluating Bias  Bias in AI systems often arises from incomplete data or unintended algorithmic behaviors. ISO5339 supports ongoing evaluations to identify and address these issues during development and deployment, reducing the likelihood of harm. ➡Expanding on Ethics with ISO24368  ISO24368 provides a broader view of the societal and ethical challenges of AI, offering additional guidance for long-term accountability and fairness. ✅Fairness: AI systems can unintentionally reinforce existing inequalities. ISO24368 emphasizes assessing decisions to prevent discriminatory impacts and to align outcomes with social expectations.  ✅Transparency: Systems that operate without clarity risk losing user trust. ISO24368 highlights the importance of creating processes where decision-making paths are fully traceable and understandable.  ✅Human Accountability: Decisions made by AI should remain subject to human review. ISO24368 stresses the need for mechanisms that allow organizations to take responsibility for outcomes and override decisions when necessary. ➡Applying These Standards in Practice  Ethical considerations cannot be separated from operational processes. ISO24368 encourages organizations to incorporate ethical reviews and risk assessments at each stage of the AI lifecycle. ISO5339 focuses on embedding these principles during system design, ensuring that ethics is part of both the foundation and the long-term management of AI systems. ➡Lessons from #EthicalMachines  In "Ethical Machines", Reid Blackman, Ph.D. highlights the importance of making ethics practical. He argues for actionable frameworks that ensure AI systems are designed to meet societal expectations and business goals. Blackman’s focus on stakeholder input, decision transparency, and accountability closely aligns with the goals of ISO5339 and ISO24368, providing a clear way forward for organizations.

I spent this morning deleting my data and account from 23andMe. Going through this process, these thoughts went through my head 👇 As someone who's spent my entire career at the intersection of data science and business, the recent 23andMe bankruptcy has ignited a firestorm of thoughts about our industry's approach to personal data as it relates to the new wave of AI technology. 23andMe, once hailed as a trailblazer in genetic testing, has now become a cautionary tale, facing bankruptcy not only due to market challenges but also a significant data breach 2 years ago that shattered user trust. This raises an uncomfortable questions we have yet to tackle. Who will be the arbiter of the data? What will they do with the data? How can I ensure my data will be taken care of 5 years ago, I said had stated in Harvard Business Review that "As we advance in AI, we must remember: safeguarding data is safeguarding our future." Companies are racing to harness the power of AI, yet many neglect the foundational responsibility of safeguarding the data they rely on. This oversight could lead to devastating consequences—not just for the companies involved but for individuals whose private information is at stake. Here are some hard truths we need to confront: - Privacy is Not Optional: Data breaches are becoming the norm, yet many organizations still treat data protection as a compliance checkbox rather than a core value. - Lack of Transparency: Consumers are in the dark about how their data is used and shared. This secrecy breeds distrust and could have long-lasting ramifications for the industry. - Ethical AI is an Oxymoron: If we continue to prioritize profits over ethics, we risk developing AI systems that exploit user data, creating a dangerous precedent for future innovations. - The fallout from 23andMe's experience should serve as a wake-up call for all of us in tech. If we don’t take action now, we risk losing the hard-earned trust of consumers forever. What’s your take? Are we doing enough to secure data in the age of AI, or are we sleepwalking into a privacy crisis? Let’s ignite this critical conversation! 👇 #data #datascience #ai #fintech

A Design Road Map for an Ethical Generative AI How to Monetize Ethics and Operationalize Values What if the next competitive edge in GenAI isn’t speed, but quality? As GenAI floods the enterprise, companies face a stark choice: automate everything and risk trust, or design with people and values at the center. Ethics will be the single most important strategic asset. Don’t take my word for it: A McKinsey study found that companies scoring highest on trust and transparency outperform their industry peers by up to 30% in long-term value creation.[1] Gartner predicts that by 2026, 30% of major organizations will require vendors to demonstrate ethical AI use as part of procurement.[2] Deloitte reports that consumers are 2.5x more likely to remain loyal to brands that act in alignment with their stated values.[3] It’s clear: Trust scales. Ethics compounds. Values convert. So how do we build AI systems around those principles? Here’s a practical, open-source roadmap to do just that: 1. Design for Ambiguity The best AI doesn’t pretend every question has a single answer. It invites exploration, not conclusions. That’s not weakness—it’s wisdom. 2. Show Your Values Expose the logic behind your systems. Let users see how outcomes are generated. Transparency isn’t just ethical—it’s the foundation of brand trust. 3. Stop Guessing. Start Reflecting. Don’t design AI to guess what users want. Design it to help them figure out what matters to them. Prediction is easy. Clarity is rare. 4. Lead With Ethics While others optimize for speed, you can win on something deeper: clarity, trust, and long-term loyalty. Ethical systems don’t break under scrutiny—they get stronger. 5. Turn Users Into Co-Creators Every value-aligned interaction is training data. Slower? Maybe. But smarter, more adaptive, and more human. That’s the kind of intelligence we should be scaling. The myth is that ethics slows you down. The truth? It makes you unstoppable. Imagine how what it would be like to have a staunch and loyal employee and customer base, an eco-system of shared values? That's the greatest moat of all time ******************************************************************************** The trick with technology is to avoid spreading darkness at the speed of light Stephen Klein is the Founder & CEO of Curiouser.AI, the only values-based Generative AI platform, strategic coach, and advisory designed to augment individual and organizational imagination and intelligence. He also teaches AI ethics and entrepreneurship at UC Berkeley. To learn more or sign up: www.curiouser.ai or connect on Hubble https://lnkd.in/gphSPv_e Footnotes [1] McKinsey & Company. “The Business Case for AI Ethics.” 2023. [2] Gartner. “Top Strategic Technology Trends for 2024.” 2023. [3] Deloitte Digital. “Trust as a Differentiator.” 2022.