The Impact of Algorithms on User Experience

AI systems don’t just reflect the world as it is - they reinforce the world as it's been. When the values baked into those systems are misaligned with the needs and expectations of users, the result isn’t just friction. It’s harm: biased decisions, opaque reasoning, and experiences that erode trust. UX researchers are on the front lines of this problem. Every time we study how someone interacts with a model, interprets its output, or changes behavior based on an algorithmic suggestion, we’re touching alignment work - whether we call it that or not. Most of the time, this problem doesn’t look like sci-fi. It looks like users getting contradictory answers, not knowing why a decision was made, or being nudged toward actions that don’t reflect their intent. It looks like a chatbot that responds confidently but wrongly. Or a recommender system that spirals into unhealthy loops. And while engineers focus on model architecture or loss functions, UX researchers can focus on what happens in the real world: how users experience, interpret, and adapt to AI. We can start by noticing when the model’s behavior clashes with human expectations. Is the system optimizing for the right thing? Are the objectives actually helpful from a user’s point of view? If not, we can bring evidence - qualitative and quantitative - that something’s off. That might mean surfacing hidden tradeoffs, like when a system prioritizes engagement over well-being, or efficiency over transparency. Interpretability is also a UX challenge. Opaque AI decisions can’t be debugged by users. Use methods that support explainability. Techniques like SHAP, LIME, and counterfactual examples can help trace how decisions are made. But that’s just the technical side. UX researchers should test whether these explanations feel clear, sufficient, or trustworthy to real users. Include interpretability in usability testing, not just model evaluation. Transparency without understanding is just noise. Likewise, fairness isn’t just a statistical property. We can run stratified analyses on how different demographic groups experience an AI system: are there discrepancies in satisfaction, error rates, or task success? If so, UX researchers can dig deeper into why - and co-design solutions with affected users. There’s no one method that solves alignment, but we already have a lot of tools that help: cognitive walkthroughs with fairness in mind, longitudinal interviews that surface shifting mental models, participatory methods that give users a voice in shaping how systems behave. If you’re doing UX research on AI products, you’re already part of this conversation. The key is to frame our work not just as “understanding users,” but as shaping how systems treat people. Alignment isn’t someone else’s job - it’s ours too.

Work on designing AI-first assistant and agent experiences has been eye opening. AI UX is both fundamentally the same and widely different, especially for vertical use cases. There are clear and emerging patterns that will likely continue to scale: 1. Comfort will start with proactive intelligence and hyper personalization. The biggest expectation customers have of AI is that it’s smart and it knows them based on their data. Personalization will become a key entry point where a recommendation kicks off a “thread” of inquiry. Personalization should only get better with “memory”. Imagine a pattern where an assistant or an agent notifies you of an anamoly, advice that’s specific to your business, or an area to dig deeper into relative to peers. 2. There are two clear sets of UX patterns that will emerge: assistant-like experiences and transformative experiences. Assistant-like experiences will sound familiar by now. Agents will complete a task partially either based on input or automation and the user confirms their action. You see this today with experiences like deep search. Transformative experiences will often start by human request and will then become background experiences that are long running. Transformative experiences, in particular, will require associated patterns like audit trails, failure notifications, etc. 3. We will start designing for agents as much as we design for humans. Modularity and building in smaller chunks becomes even more important. With architecture like MCP, the way you think of the world in smaller tools becomes a default. Understanding the human JTBD will remain core but you’ll end up building experiences in pieces to enable agents to pick and choose what parts to execute in what permutation of user asks. 4. It’ll become even more important to design and document existing standard operating procedures. One way to think about this is a more enhanced more articulated version of a customer journey. You need to teach agents the way not just what you know. Service design will become an even more important field. 5. There will be even less tolerance for complexity. Anything that feels like paperwork, extra clicks, or filler copy will be unacceptable; the new baseline is instant, crystal‑clear, outcome‑focused guidance. No experience, no input, no setting should start from zero. Just to name a few. The underlying piece is that this will all depend on the culture design teams, in particular, embrace as part of this transition. What I often hear is that design teams are already leading the way in adoption of AI. The role of Design in a world where prototyping is far more rapid and tools evolve so quickly will become even more important. It’ll change in many ways (some of it is by going back to basics) but will remain super important nonetheless. Most of the above will sound familiar on the surface but there’s so much that changes in the details of how we work. Exciting times.

AI changes how we measure UX. We’ve been thinking and iterating on how we track user experiences with AI. In our open Glare framework, we use a mix of attitudinal, behavioral, and performance metrics. AI tools open the door to customizing metrics based on how people use each experience. I’d love to hear who else is exploring this. To measure UX in AI tools, it helps to follow the user journey and match the right metrics to each step. Here's a simple way to break it down: 1. Before using the tool Start by understanding what users expect and how confident they feel. This gives you a sense of their goals and trust levels. 2. While prompting  Track how easily users explain what they want. Look at how much effort it takes and whether the first result is useful. 3. While refining the output Measure how smoothly users improve or adjust the results. Count retries, check how well they understand the output, and watch for moments when the tool really surprises or delights them. 4. After seeing the results Check if the result is actually helpful. Time-to-value and satisfaction ratings show whether the tool delivered on its promise. 5. After the session ends See what users do next. Do they leave, return, or keep using it? This helps you understand the lasting value of the experience. We need sharper ways to measure how people use AI. Clicks can’t tell the whole story. But getting this data is not easy. What matters is whether the experience builds trust, sparks creativity, and delivers something users feel good about. These are the signals that show us if the tool is working, not just technically, but emotionally and practically. How are you thinking about this? #productdesign #uxmetrics #productdiscovery #uxresearch

89% of AI interfaces contain hidden biases. Are you unknowingly designing one? (Learning from Amazon, TikTok, and LinkedIn's missteps) AI is transforming UX. Yet most teams overlook these ethical dangers: → Discriminatory algorithms → Manipulative dark patterns → Privacy vulnerabilities 3 real-world cautionary tales - 1️⃣ Amazon's Sexist Hiring Tool → Trained on male-dominated resumes → produced biased results → Lesson: Biased training data creates unethical AI 2️⃣ LinkedIn's "Skills Endorsement" Trap → AI encouraged random endorsements → diminished genuine expertise → Lesson: Don't sacrifice trust for engagement 3️⃣ TikTok's Addictive Scroll Algorithm → Promoted extreme content to increase watch time → Lesson: Exploiting psychology for engagement is ethically wrong The Solution? Bias audits: Use IBM's Fairness 360 toolkit (free) Consent layers: Give users AI tracking opt-out options (like ChatGPT's incognito mode) Transparency reports: Disclose your AI's training data (see Adobe's 2024 model) PS: If your AI can't explain its decisions to a 5th grader, you've missed the mark. ⚖️