Building Trust in Data Through Contextual Checks

If you think data is product/user data is unbiased, think again. This sniff test will keep you on the right track 👃 You can’t blindly trust numbers without context. That’s where Julie Acosta, Dir. of eCom Analytics @ NOBULL and formerly AutoZone, discussed a crucial tool recently on LaunchPod: The Sniff Test. Here’s how you can use the sniff test to check data against customer behavior and business objectives. – 1. What is the "Sniff Test"? The sniff test asks a simple but powerful question: "Does this conclusion align with what I know about the customer or business?" It’s a check to ensure findings make sense before being shared. Takeaway: Numbers can tell any story you want. The sniff test keeps them honest by pairing quantitative insights with real-world context. – 2. Why the Sniff Test Matters Data alone often misses the bigger picture: 🖇️Correlation ≠ Causation: Metrics may move together without being related. 🌋Anomalies Skew Insights: Outliers can distort trends. 🔑Context is Key: Data models often fail to capture intent or behavior. Julie shared an example from AutoZone: her team worried about high bounce rates on store pages. Further analysis revealed customers were simply finding store hours or addresses quickly then bouncing— it wasn't a problem, it was a success. – 3. How to Apply the Sniff Test 🛒Start with the Customer Ask, “Does the data align with customer intent?” At NOBULL, this means designing faster, friction-free experiences for shoppers. 🤨Gut-Check Models Challenge outputs that don’t align with expectations. “You can't tell me that we're going to be doing worse next year than we are this year, investing double marketing dollars,” says Julie. 🧠Combine Data with Context Spend time in the real world. At AutoZone, Julie learned how in-store interactions could highlight online friction points. ✍️Simplify the Story Leadership doesn’t need the weeds. “Smooth out anomalies and focus on actionable insights,” Julie advises. 🙋Ask ‘Why?’ Relentlessly Data reveals what happened, but finding the why takes curiosity and detective work. – 4. When to Use the Sniff Test The sniff test is critical when: 🧑💼Presenting to Leadership: Ensure insights are clear, concise, and actionable. 🔬Validating Experiments: Combine data with customer feedback to confirm results. 📈Interpreting Metrics: Avoid overreacting to misleading metrics like bounce rates or time on site. – ~~Key Takeaways~~ ❓ Data shows the what, but you need to uncover the why. 🛍️ Always ask: Does this align with customer behavior and business goals? ⚗️ Refine models if they don’t pass the sniff test. 📽️ Use qualitative context to make quantitative insights actionable. 🎯 The sniff test ensures you deliver results that are both accurate and impactful. Numbers alone can’t solve every problem—but paired with intuition, they become a powerful tool. Are you running your data through the sniff test? How’s this work for you?

The only way to prevent data quality issues is by helping data consumers and producers communicate effectively BEFORE breaking changes are deployed. To do that, we must first acknowledge the reality of modern software engineering: 1. Data producers don’t know who is using their data and for what 2. Data producers don’t want to cause damage to others through their changes 3. Data producers do not want to be slowed down unnecessarily Next, we must acknowledge the reality of modern data engineering: 1. Data engineers can’t be a part of every conversation for every feature (there are too many) 2. Not every change is a breaking change 3. A significant number of data quality issues CAN be prevented if data engineers are involved in the conversation What these six points imply is the following: If data producers, data consumers, and data engineers are all made aware that something will break before a change has deployed, it can resolve data quality through better communication without slowing anyone down while also building more awareness across the engineering organization. We are not talking about more meaningless alerts. The most essential piece of this puzzle is CONTEXT, communicated at the right time and place. Data producers: Should understand when they are making a breaking change, who they are impacting, and the cost to the business Data engineers: Should understand when a contract is about to be violated, the offending pull request, and the data producer making the change Data consumers: Should understand that their asset is about to be broken, how to plan for the change, or escalate if necessary The data contract is the technical mechanism to provide this context to each stakeholder in the data supply chain, facilitated through checks in the CI/CD workflow of source systems. These checks can be created by data engineers and data platform teams, just as security teams create similar checks to ensure Eng teams follow best practices! Data consumers can subscribe to contracts, just as software engineers can subscribe to GitHub repositories in order to be informed if something changes. But instead of being alerted on an arbitrary code change in a language they don’t know, they are alerted on breaking changes to the metadata which can be easily understood by all data practitioners. Data quality CAN be solved, but it won’t happen through better data pipelines or computationally efficient storage. It will happen by aligning the incentives of data producers and consumers through more effective communication. Good luck! #dataengineering

Why would your users distrust flawless systems? Recent data shows 40% of leaders identify explainability as a major GenAI adoption risk, yet only 17% are actually addressing it. This gap determines whether humans accept or override AI-driven insights. As founders building AI-powered solutions, we face a counterintuitive truth: technically superior models often deliver worse business outcomes because skeptical users simply ignore them. The most successful implementations reveal that interpretability isn't about exposing mathematical gradients—it's about delivering stakeholder-specific narratives that build confidence. Three practical strategies separate winning AI products from those gathering dust: 1️⃣ Progressive disclosure layers Different stakeholders need different explanations. Your dashboard should let users drill from plain-language assessments to increasingly technical evidence. 2️⃣ Simulatability tests Can your users predict what your system will do next in familiar scenarios? When users can anticipate AI behavior with >80% accuracy, trust metrics improve dramatically. Run regular "prediction exercises" with early users to identify where your system's logic feels alien. 3️⃣ Auditable memory systems Every autonomous step should log its chain-of-thought in domain language. These records serve multiple purposes: incident investigation, training data, and regulatory compliance. They become invaluable when problems occur, providing immediate visibility into decision paths. For early-stage companies, these trust-building mechanisms are more than luxuries. They accelerate adoption. When selling to enterprises or regulated industries, they're table stakes. The fastest-growing AI companies don't just build better algorithms - they build better trust interfaces. While resources may be constrained, embedding these principles early costs far less than retrofitting them after hitting an adoption ceiling. Small teams can implement "minimum viable trust" versions of these strategies with focused effort. Building AI products is fundamentally about creating trust interfaces, not just algorithmic performance. #startups #founders #growth #ai

Safe in the Lab, Risky in Reality?- Rethinking #AI Evaluation 🔺A safe AI model in the lab can fail in the wild. 🔺Trust isn’t built on benchmarks, but on behavior. 🔺Real-world AI needs real-world oversight. 🔺It’s time to measure what truly matters. The paper by University of Michigan AI Laboratory calls for a new approach, one that’s adaptive, real-world, and people-centered. It offers clear steps to make AI safer, fairer, and more accountable. 🔸Why Evaluate AI Systems in the Wild? ➝Lab results don’t reflect real use. ➝Exposes bugs, bias, and safety gaps. ➝Builds trust and accountability. ➝Supports safer, smarter scaling. 🔸What is Being Evaluated? ➝In-the-lab evaluation • Tested in controlled setups. • Focuses on metrics like accuracy. • Misses real-world messiness. ➝Human capability-specific evaluation • Measures how AI supports people. • Tailored to user roles. • Focuses on trust and usability. ➝In-the-wild evaluation • Runs in real settings. • Captures real-world effects. • Adapts with changing use. 🔸Evaluation Principles ➝Holistic: Beyond just performance. ➝Continuous: Never one-and-done. ➝Contextual: Tailored to the setting. ➝Transparent: Clear methods and limitations. ➝Actionable: Results must inform improvements. 🔸Evaluation Dimensions ➝Performance: Is it accurate and fair? ➝Impact: What’s the social cost? ➝Usability: Can people use it well? ➝Governance: Who’s watching it? ➝Adaptation: Can it keep up? 🔸Who Evaluates and How? ➝Benchmark-based: • Standardized tests. Comparable, but lacks context. ➝Human-centered: • Involves real users, impact and ethics. ➝Tradeoffs: • Automated -fast, limited. • Human -deep, resource-heavy. ➝Stakeholder Roles: • Developers -system tuning. • Users -real-world insight. • Auditors -accountability. 🔸Operationalizing Evaluation ➝Start with goals and context. ➝Combine data and lived experience. ➝Include all key voices. ➝Be transparent and traceable. 🔸Practical Systems Evaluation ➝ML Training • Evaluate models, data flows, and feedback loops. • Check for drift, transparency, and labeling quality. ➝Deployed GenAI • Test for prompt issues, hallucinations, and harm. • Assess across users and contexts. ➝Sustainability • Monitor energy use and carbon impact. ➝Data vs Model • Good data beats complex models. • Check how data affects fairness and accuracy. 🔸Examples of Evaluation in the Wild ➝Healthcare: Tracked outcomes and safety. ➝Hiring: Checked bias after launch. ➝Public Safety: Monitored community impact. ➝Education: Measured learning and feedback. Bottomline Real world AI demands real-world accountability. Evaluation must be continuous, collaborative, and ethical. Dr. Martha Boeckenfeld|Dr. Ram Kumar G,|Sam Boboev |Victor Yaromin| Julian Gordon|Saleh ALhammad |Sudin Baraokar |Dr. Tinoo Nandkishore Ubale,|Tony Craddock |Sara Simmonds|Helen Yu|ChandraKumar R Pillai| JOY CASE |Sarvex Jatasra|Vikram Pandya|Prasanna Lohar #ArtificialIntelligence #EthicalAI #AIEvaluation

Part 1 Emic Evaluator: Is Data Collected by Local People Enough? In the last five years, I have received multiple requests from outside organizations awarded research grants or consultancy assignments, asking me to collect data on their behalf. Out of the six such instances I can recall, only one invited me to be part of the data analysis and report writing process. This often results in reports lacking crucial contextual details. Conversely, local organizations sometimes try to cut costs by asking me to send enumerators into the field without involving me in data analysis. I argue that analyzing data without firsthand context leads to incomplete findings. So why is context important? Enhanced Understanding - Immersion in the context provides a deeper understanding of the environment, culture, and circumstances influencing the data, which helps in interpreting the data more accurately. For example, reading an interview transcript might not fully convey the degree of happiness and gratitude someone feels. In one evaluation, I visited a young woman who had benefited from a youth empowerment program and received an interlocking sewing machine. She asked me to visit her store last because she wanted me to officially launch her machine, which was significant to her. She had decorated it with a ribbon and balloons and asked me to cut the ribbon. When I inquired why she went to such lengths, she explained, “It means a lot to me. I am the only one with an interlocking machine in this village. Before, my clients had to go to town for this service. Now, I do everything, and my business has improved.” Contextual Accuracy -Understanding the context ensures that findings are reported accurately and are relevant to the specific setting, helping to avoid misinterpretations that might arise from analyzing data in isolation. In a study I have been involved with for two years in Kenya, I found that some issues in the reports didn’t make sense until I visited the area and took transect walks to observe the activities and hear the beneficiaries' reports. This improved the accuracy of my reporting. Validity and Reliability - Experiencing the context can help identify potential biases and confounding variables, enhancing the validity and reliability of the research findings. Nonverbal cues during interviews can convey additional information and clarify whether a “yes” is genuine. For example, when a stakeholder was asked if she felt meaningfully involved in a program, her nonverbal response was telling. She laughed, leaned back, remained silent for a while, and then said, “Ok, let me just say yes.” Her hesitation indicated that her affirmative answer was likely driven by social desirability rather than truth. Social desirability is the tendency by respondents to answer questions in a manner that will be viewed favourable.

When It Comes to Data Access, Most Companies Start with Roles and End with Regret Data Access control isn’t just about who someone is. it’s about why, where, and how they’re using the data. That’s the essence of ABAC (Attribute-Based Access Control). Unlike RBAC (Role-Based Access Control), which uses static roles, or ACLs (Access Control Lists), which don’t scale well, ABAC adds context: combining metadata about the data with attributes about the user. In one road safety program - I was part of- crash data, hazard reports, and trauma records had to be shared across transport, police, and trauma centres. RBAC gave engineers and analysts blanket access, but ignored region or purpose. ACLs became unmanageable as more local councils and agencies joined. With ABAC, we tagged data like: region = central ; data_type = trauma; sensitivity = high And user attributes like: user.region = central; user.purpose = injury-trend-analysis Now, a traffic officer saw only road hazards from their area. A trauma analyst studied injury trends, without accessing identifiable data outside their remit ABAC is about context: who’s asking, why they need it, and what they should see. I believe that this context-aware access scales trust, not just control; and that’s the level of precision modern data governance demands. Diagrams by Piethein Strengholt (from "Data Management at Scale")

𝗖𝗼𝗻𝘁𝗲𝘅𝘁𝘂𝗮𝗹𝗶𝘇𝗲𝗱 𝗗𝗮𝘁𝗮 𝗶𝗻 𝗜𝗻𝗱𝘂𝘀𝘁𝗿𝗶𝗮𝗹 𝗗𝗮𝘁𝗮𝗢𝗽𝘀: 𝗪𝗵𝗮𝘁’𝘀 𝗠𝘂𝘀𝘁-𝗛𝗮𝘃𝗲 𝘃𝘀 𝗡𝗶𝗰𝗲-𝘁𝗼-𝗛𝗮𝘃𝗲 In the past, having access to data was a competitive edge. Today, it's just the starting point. Real impact — from AI models that perform reliably, to cognitive manufacturing systems that adapt in real time — depends on data with context. It’s not about collecting more; it’s about connecting the right data to the right meaning. Here's how we break down the journey of contextualization across four critical phases 𝟭. 𝗗𝗮𝘁𝗮 𝗙𝗼𝘂𝗻𝗱𝗮𝘁𝗶𝗼𝗻 (𝗠𝘂𝘀𝘁-𝗛𝗮𝘃𝗲) Start with the basics: make your data trustworthy and usable. 𝗗𝗮𝘁𝗮 𝗮𝗰𝗾𝘂𝗶𝘀𝗶𝘁𝗶𝗼𝗻 & 𝘀𝘁𝗿𝘂𝗰𝘁𝘂𝗿𝗶𝗻𝗴 Collect from sensors, logs, systems Clean and standardize across formats Add timestamps, labels, and source identifiers 𝗥𝗲𝘀𝘂𝗹𝘁 :Without this, nothing else works. 𝟮. 𝗢𝗽𝗲𝗿𝗮𝘁𝗶𝗼𝗻𝗮𝗹 𝗜𝗻𝘁𝗲𝗹𝗹𝗶𝗴𝗲𝗻𝗰𝗲 (𝗠𝘂𝘀𝘁-𝗛𝗮𝘃𝗲) Next, turn raw data into situational awareness. 𝗖𝗼𝗻𝘁𝗲𝘅𝘁𝘂𝗮𝗹 𝗺𝗮𝗽𝗽𝗶𝗻𝗴 Tag assets and processes Align data with time and location Link events, users, workflows 𝗥𝗲𝘀𝘂𝗹𝘁 : Now your systems can understand “what is happening” and “where.” If you stop at just the 𝗠𝘂𝘀𝘁-𝗛𝗮𝘃𝗲 𝗹𝗮𝘆𝗲𝗿𝘀, you’ll build a system that works — but not one that learns, adapts, or competes effectively. 𝗗𝗼𝗺𝗮𝗶𝗻 𝗞𝗻𝗼𝘄𝗹𝗲𝗱𝗴𝗲 (𝗩𝗮𝗹𝘂𝗲 𝗔𝗺𝗽𝗹𝗶𝗳𝗶𝗲𝗿𝘀) Build deeper meaning through domain-specific insights. 𝗗𝗼𝗺𝗮𝗶𝗻 𝗶𝗻𝘁𝗲𝗹𝗹𝗶𝗴𝗲𝗻𝗰𝗲 Apply business rules, thresholds, and taxonomies Integrate procedures, documentation, and logs Leverage knowledge graphs or digital twins 𝗥𝗲𝘀𝘂𝗹𝘁 : You now move from “what’s happening” to “why it matters.” 𝟰. 𝗦𝘁𝗿𝗮𝘁𝗲𝗴𝗶𝗰 𝗔𝘄𝗮𝗿𝗲𝗻𝗲𝘀𝘀 (𝗩𝗮𝗹𝘂𝗲 𝗔𝗺𝗽𝗹𝗶𝗳𝗶𝗲𝗿𝘀) Finally, enable cross-functional decision support and foresight. 𝗜𝗻𝗱𝘂𝘀𝘁𝗿𝘆 𝗶𝗻𝘁𝗲𝗹𝗹𝗶𝗴𝗲𝗻𝗰𝗲 Monitor market trends and external conditions Correlate across systems and domains 𝗥𝗲𝘀𝘂𝗹𝘁 : At this stage, data guides not just operations — it drives strategy. The real ROI comes when you move from reaction to intelligent, proactive decision-making. 𝗧𝗵𝗲 𝗘𝗻𝗮𝗯𝗹𝗲𝗿? 𝗦𝗲𝗺𝗮𝗻𝘁𝗶𝗰 𝗠𝗼𝗱𝗲𝗹𝘀 & 𝗞𝗻𝗼𝘄𝗹𝗲𝗱𝗴𝗲 𝗚𝗿𝗮𝗽𝗵𝘀 These technologies unify data across systems, preserve relationships, and make data interpretable — for both humans and machines. They’re key to scaling contextualization across complex environments. 𝗧𝗵𝗲 𝗧𝗮𝗸𝗲𝗮𝘄𝗮𝘆: Stop thinking in terms of “more data.” Start thinking in terms of smarter, contextualized data — delivered to the right place, at the right time, in the right format. Ref : https://lnkd.in/ddq8eN7c

Data won’t stay clean. The job is to keep trust intact when it doesn’t. Too many data quality efforts focus on tests and tools, but miss the bigger picture: trust, ownership, and how we respond when things break. Data teams spend endless hours writing tests and setting up monitoring for the issues they know about today. But tomorrow, priorities shift. Data structures change. New sources get added. Old assumptions break. You build the perfect system for catching yesterday’s mistakes and still trip over what comes next. So what’s the fix? 👇 𝗙𝗼𝗰𝘂𝘀 𝗼𝗻 𝗯𝘂𝗶𝗹𝗱𝗶𝗻𝗴 𝗿𝗲𝘀𝗶𝗹𝗶𝗲𝗻𝘁 𝘄𝗮𝘆𝘀 𝗼𝗳 𝘄𝗼𝗿𝗸𝗶𝗻𝗴. These are the habits and processes that protect trust when things go wrong and get stronger over time. That looks like: 1.) Proactive, transparent communication. When something breaks, don’t quietly patch and move on. Tell impacted teams what happened, what you’re doing about it, and when it’ll be fixed.   2.) Involving accountable owners and enabling self-monitoring. If data quality issues start at the source, the source owner should have the tools and visibility to catch it, not rely on a downstream fire drill.   3.) Reinforcing trust through consistency. People notice patterns. When issues arise, show them the same thing every time: a clear plan, fast action, and lessons learned. Striving for perfect data doesn’t build trust. Resilient teams do.

Data reflects power, priorities, and perspective. Its context matters. Data is shaped by (and shapes) the beliefs, values, and priorities of the people and society who create it. That’s why it’s so important to get to know your data before diving into analysis or creating visualizations. Knowing where your data comes from, how it was generated, and acknowledging its limitations can help you assess its quality and relevance, as well as prevent its misuse and misinterpretation. Here are a few key questions (and follow-ups) you should always ask of your data: 1/ Who is included and who is excluded from these data?  Whose voices, lives, and experiences are missing? Who might be harmed by the publication of these data? 2/ What kind of data were collected? What type of analyses can the data be used for? What questions could be answered by this data? 3/ Where did the data come from?  Who collected it?  Can this data be trusted? 4/ Why was the data collected?  Were the data collected for administrative purposes, compliance, monitoring, or original research? 5/ When was the data collected? Is it still relevant? Will it be updated? 6/ How was the data generated? Was the data gathered from an existing source? Was the data collected electronically or manually? Data are not as objective or neutral as we would like to believe. I encourage anyone who works with data to embrace this context, critically examine it, and create space for meaningful reflection, thoughtful analysis, and more accurate (re)presentations. -- 🔔 Follow me for more data + design content. 💬 Visit my blog for more time-saving data tips and tricks >>>> www.anyamemensah.com/blog 👥 Need help turning your messy data into streamlined systems and stories that stick? Contact me today >>>> https://lnkd.in/ewin337 -- Alt Text: A social media carousel detailing a few key questions you should always ask of your data before diving into analysis or visualization. Page 1: Title Page – Context Matters Page 2: Title: Data in context. Contents: Image of a slide with six circle icons, each depicting a different question to ask of your data. From left to right: who, what, where, why, when, and how. Page 3: Title: Data in context. The first icon (who) is highlighted, and the slide text reads: Who is included and who is excluded from these data? Page 4: Title: Data in context. The second icon (what) is highlighted, and the slide text reads: What kind of data were collected? Page 5: Title: Data in context. The third icon (where) is highlighted, and the slide text reads: Where did the data come from? Page 6: Title: Data in context. The third icon (why) is highlighted, and the slide text reads: Why was the data collected? Page 7: Title: Data in context. The third icon (when) is highlighted, and the slide text reads: When was the data collected? Page 8: Title: Data in context. The third icon (how) is highlighted, and the slide text reads: How was the data generated?

This number is technically correct. So why doesn’t anyone trust it? This was one of the hardest lessons to learn early in my analytics career: Data accuracy ≠ data trust. You can build the cleanest model. You can double-check the SQL, audit the joins, QA the filters. And still… stakeholders say: “That number feels off.” “I don’t think that’s right.” “Let me check in Excel and get back to you.” Here’s what’s often really happening: 🔄 They don’t understand where the number is coming from. If they can’t trace it, they can’t trust it. Exposing calculation steps or using drill-throughs can help. 📊 The metric name isn’t aligned to what they think it means. You might call it Net Revenue. They think it’s Net Revenue after refunds. Boom, there is misalignment. 📆 They forgot the filters they asked for. “Why are we only looking at this year?” → “Because you asked for YTD only, remember?” Keep context visible. Always. 🧠 They’re comparing your number to what they expected, not what’s correct. And unfortunately, expectations are rarely documented. 🤝 You weren’t part of the business process that generates the data. So when something looks odd, they assume it’s a reporting issue, not a process or input issue. Here’s the kicker: Sometimes, being accurate isn’t enough. You also need to be understandable, explainable, and collaborative. That’s when trust happens. Have you ever been 100% confident in a metric, only to spend more time defending it than building it? #PowerBI #AnalyticsLife #DataTrust #DAX #SQL #DataQuality #DataStorytelling