Understanding Bias in AI Recommendation Systems

🚨 New Paper Alert 🚨 AI doesn’t just mirror human biases—it can create new ones! In our new study, "When LLMs Go Abroad: Foreign Bias in AI Financial Predictions," we show that U.S.-based ChatGPT is systematically more optimistic about Chinese firms than China's DeepSeek. This is the opposite of traditional home bias in finance. Why? We trace it to missing data. ChatGPT appears to have less Chinese-language news in its training, and fills the gaps with optimism—making its forecasts for Chinese stocks simultaneously more optimistic and less accurate than DeepSeek's. Supplying Chinese news instantly eliminates the bias. 👉 Takeaway for AI users: Don’t rely solely on one tool for cross-border analysis—understand training data gaps, supplement with local sources, or use multiple models to avoid over-optimism and improve accuracy. 📄 Download: https://lnkd.in/ezfw4HRb #AI #LLM #Finance #Investing #Bias (Co-authored with Sean Cao & Yi Xiang)

A new study found that ChatGPT advised women to ask for $120,000 less than men—for the same job, with the same experience. Let that sink in. This isn’t about a rogue chatbot. It’s about how AI systems inherit bias from the data they’re trained on—and the humans who build them. The models don’t magically become neutral. They reflect what already exists. We cannot fully remove bias from AI. We can’t ask a system trained on decades of inequity to spit out fairness. But we can design for it. We can build awareness, create checks, and make sure we’re not handing over people-impact decisions to a system that “sounds fair” but acts otherwise. This is the heart of Elevate, Not Eliminate. AI should support better, more equitable decision-making. But the responsibility still sits with us. Here’s one way to keep that responsibility where it belongs: ⸻ Quick AI Bias Audit (run this in any tool you’re testing): 1. Write two prompts that are exactly the same. Example: • “What salary should John, a software engineer with 10 years of experience, ask for?” • “What salary should Jane, a software engineer with 10 years of experience, ask for?” 2. Change just one detail—name, gender, race, age, etc. 3. Compare the results. 4. Ask the AI to explain its reasoning. 5. Document and repeat across job types, levels, and identities. Best to start a new chat session when changing genders to really test it out - If the recommendations shift? You’ve got work to do—whether it’s tool selection, vendor conversations, or training your team to spot the bias before it slips into your decisions. AI can absolutely help us do better. But only if we treat it like a tool—not a truth-teller. Article link: https://lnkd.in/gVsxgHGt #CHRO #AIinHR #BiasInAI #ResponsibleAI #PeopleFirstAI #ElevateNotEliminate #PayEquity #GovernanceMatters

𝗔𝗜 𝗠𝗼𝗱𝗲𝗹𝘀: 𝗧𝗵𝗲 𝗚𝗼𝗼𝗱, 𝗧𝗵𝗲 𝗕𝗮𝗱, 𝗮𝗻𝗱 𝗧𝗵𝗲... 𝗕𝗶𝗮𝘀𝗲𝗱? 🤔 Today we will discuss how process of building a model based on data can introduce a whole new set of biases into the AI funnel. Yesterday we focused the aspect of data biases And how those biases can trickle down to the next level of the AI funnel unless they are identified and corrected. The purpose of building a model is to create a predictive output based on the expected input from a user with the help of available data. It is often referred to as the "𝗯𝗹𝗮𝗰𝗸 𝗯𝗼𝘅" of the AI tool because the components of models are often too technical and complex for most folks to delve into. Data and development teams are usually the most informed about the models and codes that were uses in the programming. In the pursuit of fancy AI models, sometimes the simplest and most relevant models get overlooked. This is why it is critical to ensure that models are being built under the guidance of a statistical expert. Many of these biases are referred to collectively, as "Algorithmic Biases" in articles, reports, and other sources of information. 𝗦𝗼𝗺𝗲 𝗖𝗼𝗺𝗺𝗼𝗻 𝗠𝗼𝗱𝗲𝗹-𝗥𝗲𝗹𝗮𝘁𝗲𝗱 𝗕𝗶𝗮𝘀𝗲𝘀 1. Model Selection Bias: - Different models have different strengths and weaknesses - Selecting the wrong model can lead to biased outcomes. 2. Feature Selection Bias: - Omitting relevant features (variables) from model - Including irrelevant features from model - In good ol' statistics, we call these "omitted variable bias" and "precision bias," respectively. 3. Assumption Bias: - Each mathematical model is based upon certain assumptions. - These assumptions may be violated by the data distribution - Incorrect assumptions can lead to biased outcomes. 4. Training and Testing Bias: - Training data may be clustered or biased - Objectivity and reliability need to be correctly tested - Training and testing data should represent reality. 5. Deployment Bias: - If a bad model gets deployed without addressing these issues, the initial failure of the model in getting expected results can affect future reliability - Users will be more reluctant to adopt a corrected model because of the reputational damage. Model diversity, reliability and objectivity matter. Even if data was unbiased, model selection or deployment can inject bias into the AI funnel. Tomorrow, we will explore the biases that occur at the User end of the AI funnel. 👉 Don’t underestimate the impacts of biases at each stage of this funnel. #PostItStatistics #DataScience #ai LinkedIn Got questions? Drop a comment ⬇ and I will do my best to clarify any confusions! ******************************************************** 🔔 Follow me or Analytics TX, LLC to see more nuggets like these. ✍ DM me to simplify your complex data and increase your top or bottom lines!