Download to read offline
![17
● In the 90s, “I was breaking all of [my classmates’] facial-recognition software
because apparently all the pictures they were taking were of people with
significantly less melanin than I have” (Charles Isbell)
● In 2015, the “gorilla mistake” in Google Photos
● 25 years, not the same learning model at all, but the same root cause
Face recognition from the 90s to nowadays](https://image.slidesharecdn.com/slidesdevfest-191007161637/75/When-the-AIs-failures-send-us-back-to-our-own-societal-biases-17-2048.jpg)
![17
● In the 90s, “I was breaking all of [my classmates’] facial-recognition software
because apparently all the pictures they were taking were of people with
significantly less melanin than I have” (Charles Isbell)
● In 2015, the “gorilla mistake” in Google Photos
● 25 years, not the same learning model at all, but the same root cause
Face recognition from the 90s to nowadays](https://crownmelresort.com/image.slidesharecdn.com/slidesdevfest-191007161637/75/When-the-AIs-failures-send-us-back-to-our-own-societal-biases-17-2048.jpg)
![[DSC Europe 23] Shahab Anbarjafari - Generative AI: Impact of Responsible AI](https://cdn.slidesharecdn.com/ss_thumbnails/shahabanbarjafariresponsibleai-231128234708-209fc7a4-thumbnail.jpg?width=640&height=640&fit=bounds)
When the AIs failures send us back to our own societal biases
- 2. 2 The good andthe evil
- 3. 3 The good andthe evil
- 4. 4 Mélanie DUCOFFE PhD., ResearchData Scientist @mducoffe melanie.ducoffe@airbus.com Deep Learning, AI explainability About me !
- 5. 5 Clément DUFFAU PhD., LeadDevOps @clement0210 clement.duffau@stack-labs.com Automation, verification & validation, assurance quality, safety About me !
- 6. 6 Summary Artificial Intelligence andethics Data selection biases In production biases Train interpretation biases
- 7. 7 Job posts gatheredinto a single position: prone to many biases. Data biases and Model biases What do you think of combining : architect + dev+ testing + maintenance ? The nature of AI into the work market Data Scientist Data Analyst Data Engineer Machine Learning Scientist Machine Learning Engineer Statistician Find, clean and organize data for companies Transform and manipulate large datasets to suit the analysis of companies Perform batch processing or real time processing on gathered/stored data Research new data approaches and algorithms Create data funnels and deliver software solutions Interpret, analyze and report statistical information
- 8. 8 ● Bias (instatistics) : “the difference between the expectation of the sample estimator and the true population value, which reduces the representativeness of the estimator by systematically distorting it” ● “The big takeaway is that we don’t know what we don’t know,” (Alice Popejoy) What do you mean about biases ?
- 9.
- 10.
- 11. 11 ● MIT studydecision-making of self-driving car on killing scenario ● Scenario with ○ high/low level of education ○ young/old ○ male/female ○ pets ○ traffic signal respect http://moralmachine.mit.edu/ The Moral Machine
- 12. 12 ● Results “rank”you on ○ number of death ○ law respect ○ gender,age,health, ... ● Need ethical choices at the government, insurance ? manufacturer ? passengers ? The Moral Machine
- 13. 13 ● 3 differentclusters ● Cultural differences ○ USA/Europe kill the oldest vs Japon saves the oldest ○ Colombia saves high educated people vs Finland doesn’t matter ○ South america/France saves women The Moral Machine experiment, Edmond Awad et al. (2018), Nature The Moral Machine
- 14. 14 Ethics in clinicalinvestigations ● Technical committee to review scientific foundations and safety (in France, ANSM) ● Ethics committee on animal AND humans (in France, CPP) ○ Composed by medical professional and citizen ○ Review application form on ■ benefit/risk ■ information quality ■ resources to conduct the study ■ patient recruitment process ■ Patient consent modality ○ Follow the experiment process
- 15. 15 Applying ethics committeein AI at Google ● Advanced Technology External Advisory Council (ATEAC) ○ Ensure the white paper principles for AI at Google ■ Be socially beneficial ■ Avoid creating or reinforcing unfair bias ■ Be built and tested for safety ■ Be accountable to people ■ Incorporate privacy design principles ■ Uphold high standards of scientific excellence ■ Be made available for uses that accord with these principles ● Dissolved 1 week after creation ○ Ethics of some attendees were discussed … ○ Questioning themselves of the needs to represent every part of the society
- 16.
- 17. 17 ● In the90s, “I was breaking all of [my classmates’] facial-recognition software because apparently all the pictures they were taking were of people with significantly less melanin than I have” (Charles Isbell) ● In 2015, the “gorilla mistake” in Google Photos ● 25 years, not the same learning model at all, but the same root cause Face recognition from the 90s to nowadays
- 18. 18 ● In 2014,Amazon develops an IA to “find” key success factor and hire these people ● Train on their 10 last years hired people ● 89% of the engineering workforce is male ● At the end, you have sexist recruitment AI ! ● Representative data inside the company by not inside the society … AI recruitment by Amazon
- 19. 19 Biases in theFeatures Outcome outputFeatures Model Diploma Gender Hobbies Nationality University ... HIRED ? Model Outcome output Features ● Lack of an appropriate set of features ● Lack of an appropriate dataset ● Imbalanced dataset or bias in the output ● Unawareness: remove sensitive features from your data
- 20. 20 Tutorial: your firstbias detector Survival of passengers on the Titanic ● Decision Tree ● Leaves = class labels ● Nodes = splitting/conjunctions of features ● Important nodes = less deep, lot of observations ● Many Framework coexist: FairML, “What If”, IBM Bias Assessment ● ‘No universal solution’: combine them Summarize: Your chance of survival were good iff: - you were a woman - you were a young boy with few siblings
- 21. 21 ● Bad advicepointed out by IBM in internal in 2017 ○ ex : Suggest a cancer patient with severe bleeding be given a drug that could cause the bleeding to worsen ● Started off by using real patient data ● Fed it with hypothetical data ● “Synthetic cases allow you to treat and train Watson on a variety of patient variables and conditions that might not be present in random patient samples, but are important to treatment recommendations” (Edward Barbani) ● Pointed out the difficulties to collect representative data in medical Unsafe medical recommendations by IBM Watson
- 22. 22 Data Augmentation ● Historicallyfor images: rotation, flipping, adding noise… ● Object detection models : performance loss on corrupted images ● CNNs generalize poorly to novel distortion types, despite being trained on a variety of other distortions
- 23. 23 Generating new datawith GANS source: Medium, “Generative Adversarial with Maths” by Madhu Sanjeevi Discriminator Generator Generator
- 24. 24 Understanding biases andGeneralization in Deep Generative Models Can we learn the distribution of data with GANS ? What are the inductive biases of Deep Generative models ? ● Unbiased and consistent density estimation impossible ● Inductive biases ● Similar cognitive bias as humans: numerosity ● Weber’s law: relative change (ratio) True data Generated data
- 25. 25 GAN-based Data AugmentationPerpetuates biases
- 26.
- 27.
- 28. 28 Group Fairness ● DemographicParity ○ ex : gender independent ● Equal odds ○ ex : take into account the reality statistics in one side ● Equal opportunity ○ ex : take into account the reality statistics in both side
- 29.
- 30.
- 31. 31 Black box models:non linear decision functions DOG CAT WHY
- 32. 32 LIME: Zooming ! createneighbors and deduce a linear classifier DOG CAT DOG CAT
- 33. 33 Is it working? Examples of Explanations are usually cherry-picked !
- 34. 34 Individual Fairness forExplainability The fact that 2 resulting saliency maps are different is fundamentally due to the network itself being fragile to such perturbations
- 35.
- 36.
- 37. 37 ● Experienced lanerecognition algorithm ● Doesn’t expect that 3 stickers break the robustness ● Hacking the AI with thinks that doesn’t matter humans Tesla autonomous driving hacked with stickers
- 38.
- 39. 39 Hacking tutorial: designyour own adversarial example CAT DOG CAT DOG if B is predicted as a cat, retry to cross the decision boundary with B ! Linear Model NON Linear Model A A B B
- 40. 40 Hacking Tutorial: Idon’t have access to the model This is not a car ! This is not a car !
- 41. 41 Defending against adversarialattacks ?! SUCCESS FAILURES 2013 Discovery of adversarial examples 2015 Fast Gradient Sign 2015 DeepFool 2016 Carlini Wagner 2019 Unforeseen Attack 2013 Adversarial Training BRUTE FORCE: training with adv 2015 Defensive Distillation output probabilities rather than hard decisions 2016 JPEG compression against adv
- 42. 42 Why is ithard to defend against adversarial examples? ● Adaptative: block one type of attacks but leaves vulnerability open to an attacker who knows the defense being used ● Hard to defend because hard to understand the theory ?
- 43. 43 Adversarial Examples arenot Bugs, they are features BIRD DOG CAT “Bird” “Dog” “Cat” create the label-target adversarial dataset DOG CAT BIRD “Dog” “Cat” “Bird”
- 44.
- 45. 45 Conclusion Data Biases Model Biases Explanation Prod Biases Adv Examples If no onecares, it is highly likely that the next person who suffers from biased treatment is one of us
- 46. 46 What about inthe future ? ● Art project exposed racial bias in the biggest image dataset ImageNet in Sept. 2019 ● ImageNet will remove 600,000 images ● Number of publications per year with the name Imagenet in the abstract…
- 47.