Artificial
Intelligence
Dr. Kim Kyllesbech Larsen,
Group Technology, Deutsche Telekom.
Digital Readiness Campus
Mönchengladbach, Germany.
September 5th, 2016
2
“People worry that Computers
will get too smart and take over the world,
but the real problem is that
they're too stupid and
they've already taken over the world.”
Pedro Domingos,
Author of “The Master Algorithm”
Quiz 
Who are they referring to?
A. People
or
B. Computers
“An illustration of a Winograd Schema”
3
TERMINATOR
RAIN
MAN
GOD(like)
4
AI Related
Do note that AI is not
mentioned explicitly!
5
Reference: (*) Interesting discussion with refs https://www.quora.com/Roughly-what-processing-power-does-the-human-
brain-equate-to ,
6
Reference: (*) Interesting discussion with refs https://www.quora.com/Roughly-what-processing-power-does-the-human-
brain-equate-to ,
7
1 0
Reference:David Silver etal.,“Mastering thegameofGo with deepneural networksand treesearch”, Nature (2016).
Machine Man
GO
NARROW AI
Weak AI
Today ~100% of use cases
Ex Machina (2015)
GENERAL AI
Strong AI
(WIP:-)
9
Future
Work
10
45+ percent of
American jobs are at high risk of
being taken over by
A.I. & Robotics
within the Next Decade or two.
Carl BenediktFrey& MichaelA. Osborne,“TheFutureof Employment”, Oxford Martin Programme on
TechnologyandEmployment(2013).
By 2025ish! This is only from exploiting
Narrow AI
“low-hanging fruits”
(i.e., God-like AIs not considered)
11
“The town councilors refused to give the demonstrators
a permit because they feared violence.”
Question
Who feared violence?
AI’s are not good at common sense!
Common sense reasoning – the Winograd Schema Challenge.
http://spectrum.ieee.org/automaton/robotics/artificial-intelligence/winograd-schema-
challenge-results-ai-common-sense-still-a-problem-for-now
 Answer A: the town councilors
 Answer B: the demonstrators
“A better Turing Test”
12
Chatbots …
http://www.fastcompany.com/3059439/why-the-new-chatbot-invasion-is-so-different-from-its-predecessors
http://www.meemim.com/2016/08/17/chat-bots-new-craze-unlikely-live-hype/, https://chatbotsmagazine.com/when-is-a-i-
really-useful-in-chatbots-7a3e64c41aa8#.moewpofei
http://www.kurzweilai.net/ask-ray-response-to-announcement-of-chatbot-eugene-goostman-passing-the-turing-test
TINKA: T-Mobiles Interaktive Neue Kommunikations-Assistentin
IPSoft
13
Chatbots … “Good” Bots.
Illustrations
Source: https://www.tkxel.com/blog/future-of-chatbots/
14
Chatbots ... “Evil” Bad Bots!
When you learn from the worst! (Illustration)
Must reads: Roman V. Yampolskiy, “Taxonomy of Pathways to Dangerous AI”;
http://arxiv.org/pdf/1511.03246v2.pdf, When IBM Watson went to the Dark Side; http://www.ibtimes.com/ibms-
watson-gets-swear-filter-after-learning-urban-dictionary-1007734, Microsoft Tay goes very Dark Side;
http://www.techrepublic.com/article/why-microsofts-tay-ai-bot-went-wrong/
15
Baby-bot
Mainly Id driven
No SuperEgo
Mature-bot
Well balanced
Ego
Reinforcement learning …
How to raise your bot to be a model “citizen”.
Many many
Conversations
Learn what is Good
& Bad behaviour
The Perfect
Customer
Experience
Agent
Teach you Bot not only what is
Good/Right, but also what is Bad/Wrong.
16
Augmented Humans.
Its not about competition but collaboration!
http://biomech.media.mit.edu/#/
The Future of Brain Implants:
http://www.wsj.com/articles/SB1000142405
2702304914904579435592981780528
http://www.thefoodrush.com/blog/rob
ot-chef-replace-us-kitchen/
http://www.computationallegalstudies.com/
http://www.wired.com/2014/06/ai-healthcare/
https://www.rt.com/uk/
346920-robot-worker-
artificial-intelligence/
17
Supervised Learning.
A.I.
Learning
Principles.
"10,000-Hour Rule“
The key to achieving world class
expertise in any skill, is, to a large
extent, a matter of practicing the
correct way, for a total of around
10,000 hours.
Malcolm Gladwell, “Outliers” (2008).
Unsupervised Learning.
The A.I. “Holy Grail”
The Default Approach
18
BIG DATA required!
The AI quality relates directly to the amount of data available …
19
Supervised learning.
Customer satisfaction illustration.
Id Age Postpaid $Spend
Network
Out
Network
In
Objective
UX Index
Main
location
…
Actual
Happiness
123…76 18 0 $10 10 2 -5 (lat, lon) … -2
235…96 38 0 $25 10 15 +2 (lat, lon) … +2
578..00 28 1 $40 15 25 +4 (lat, lon) … +5
.
.
.
710…13 49 1 $40 8 5 +1 (lat, lon) … -1
Starting point! I have known data & known outcome (e.g., happiness).
Parameter of
interest
Predicted
Happiness
-2
-1
+5
+2
Features or Classifiers
Treat customer as Happy –
However this is False (positive)
Treat customer as Un-Happy –
However this is False (Negative)
20
Teaching the A.I.
“Clean”
Data
“Relevant”
Features
Your Model
Quality
Requirements
(of Model)
Some Math 
PRODUCT
MODEL
New Data
Creation Process.
Actual Result Predicted Result
21
A bunch of undefined pictures
(Training data)
Find Structure
/ Similarity
(Model)
Category F
Category C
PRODUCT
MODEL
Unsupervised learning.
Category C
False Positive
New Picture
22
AI Quality Metrics.
You can be very accurate and precisely wrong!
PREDICTED OUTCOME
POSITIVE NEGATIVE
ACTUAL
OUTCOME
POSITIVE
TRUE
POSITIVE
FALSE
NEGATIVE
NEGATIVE
FALSE
POSITIVE
TRUE
NEGATIVE
PRECISION = #TRUE POSITIVES
#TRUE POSITIVES + #FALSE POSITIVES
RECALL = #TRUE POSITIVES
#TRUE POSITIVES + #FALSE NEGATIVES
Can be very
costly
Can be very
costly
ACCURACY = #TRUE POSITIVES + #TRUE NEGATIVES
ALL DATA POINTS
How often a model
predicted positive
event is correct.
How confident we
can be that model
predicts correctly a
positive event
23
Highly accurate & precisely wrong!
Example of how (not) to find the terrorist?
1 person in 15,000 is a terrorist (e.g., out of all Muslims)
My (naive) model: All people are non-terrorist.
TRUE FALSE
TRUE 0 1
FALSE 0 14,999 ACCURACY = 99.99%
PRECISION = 0%
RECALL = 0%
BOOM!
False comfort!
Predicted
Actual
It is possible to be perfectly accurate and precisely wrong!
24
Precision, Recall & the Cost
of being wrong and right!
20 Million Muslims in EU.
Expect < 1,300 active (Muslim) terrorists in EU*
(note: 5,000+ Europeans estimated to have travelled to Iraq & Syria by end of 2015)
Assume a more sophisticated model** gives the following:
TRUE FALSE
TRUE 1.0 thsd 0.3 thsd
FALSE 9.0 thsd 19,978 thsd ACCURACY = 99.99%
PRECISION = 10%
RECALL = 77%
Still BOOM!
Costly police work
and time spend on
the wrong people.
Predicted
Actual
(*) 687 (suspects) were arrested for terrorism related-offences in 2015 (source: Europol TE-SAT 2016 report).
(**) e.g., Bayesian machine learning models, DNNs, social network analysis (e.g., social physics).
25
Anomaly Detection.
Identifying the black sheep.
Fraud Detection.
DDOS Attacks.
Imminent Hardware Failure.
Virus Propagation.
(RT) System Anomaly Detection.
Examples.
Un-supervised learning models are
(usually) used for anomaly detection.
Look for Precision & Recall
(accuracy tend to be a overrated)
Appropriate Learning Models.
26
Alexa
https://www.fastcompany.com/3058721/app-economy/the-real-reasons-that-amazons-alexa-may-become-the-go-
to-ai-for-the-home; https://www.technologyreview.com/s/601654/amazon-working-on-making-alexa-recognize-your-
emotions/
Alexa AI
lives in
the cloud
o Voice is the Main User Interface.
o Automatic speech recognition (ASR).
o Natural Language Understanding (NLU).
o Contextual (to a degree).
o Improving by using (e.g., optimizing).
o (to come - memory of past conversations).
o (to come - affective feedback).
o (to come - conversational).
User makes
a request
Audio stream is sent up
to Alexa
Alexa pass back a Audio
response (or textual/graphical)
Alexa – the AI for your home.
Imagining customer care in your home whenever!
27
Huge amount of Data!
Need for “Clean” Data!
Model!
(can be up to 80 – 90% of work)
Quality Goals!
(the more data, the higher quality should result
& the less complexity required … in general)
Lots of Computing!
(to built the model, normally as heavy not to run it)
(some) Prerequisites for good A.I.’s
(need to beat “Flipping a Coin”
or Majority “Vote”)
It
Starts
Here!
70%
Train
30%
Test
Have
Fun!
DON’T BE PERFECTLY ACCURATE
AND
PRECISELY WRONG!

Artificial intelligence - Digital Readiness.

  • 1.
    Artificial Intelligence Dr. Kim KyllesbechLarsen, Group Technology, Deutsche Telekom. Digital Readiness Campus Mönchengladbach, Germany. September 5th, 2016
  • 2.
    2 “People worry thatComputers will get too smart and take over the world, but the real problem is that they're too stupid and they've already taken over the world.” Pedro Domingos, Author of “The Master Algorithm” Quiz  Who are they referring to? A. People or B. Computers “An illustration of a Winograd Schema”
  • 3.
  • 4.
    4 AI Related Do notethat AI is not mentioned explicitly!
  • 5.
    5 Reference: (*) Interestingdiscussion with refs https://www.quora.com/Roughly-what-processing-power-does-the-human- brain-equate-to ,
  • 6.
    6 Reference: (*) Interestingdiscussion with refs https://www.quora.com/Roughly-what-processing-power-does-the-human- brain-equate-to ,
  • 7.
    7 1 0 Reference:David Silveretal.,“Mastering thegameofGo with deepneural networksand treesearch”, Nature (2016). Machine Man GO
  • 8.
    NARROW AI Weak AI Today~100% of use cases Ex Machina (2015) GENERAL AI Strong AI (WIP:-)
  • 9.
  • 10.
    10 45+ percent of Americanjobs are at high risk of being taken over by A.I. & Robotics within the Next Decade or two. Carl BenediktFrey& MichaelA. Osborne,“TheFutureof Employment”, Oxford Martin Programme on TechnologyandEmployment(2013). By 2025ish! This is only from exploiting Narrow AI “low-hanging fruits” (i.e., God-like AIs not considered)
  • 11.
    11 “The town councilorsrefused to give the demonstrators a permit because they feared violence.” Question Who feared violence? AI’s are not good at common sense! Common sense reasoning – the Winograd Schema Challenge. http://spectrum.ieee.org/automaton/robotics/artificial-intelligence/winograd-schema- challenge-results-ai-common-sense-still-a-problem-for-now  Answer A: the town councilors  Answer B: the demonstrators “A better Turing Test”
  • 12.
  • 13.
    13 Chatbots … “Good”Bots. Illustrations Source: https://www.tkxel.com/blog/future-of-chatbots/
  • 14.
    14 Chatbots ... “Evil”Bad Bots! When you learn from the worst! (Illustration) Must reads: Roman V. Yampolskiy, “Taxonomy of Pathways to Dangerous AI”; http://arxiv.org/pdf/1511.03246v2.pdf, When IBM Watson went to the Dark Side; http://www.ibtimes.com/ibms- watson-gets-swear-filter-after-learning-urban-dictionary-1007734, Microsoft Tay goes very Dark Side; http://www.techrepublic.com/article/why-microsofts-tay-ai-bot-went-wrong/
  • 15.
    15 Baby-bot Mainly Id driven NoSuperEgo Mature-bot Well balanced Ego Reinforcement learning … How to raise your bot to be a model “citizen”. Many many Conversations Learn what is Good & Bad behaviour The Perfect Customer Experience Agent Teach you Bot not only what is Good/Right, but also what is Bad/Wrong.
  • 16.
    16 Augmented Humans. Its notabout competition but collaboration! http://biomech.media.mit.edu/#/ The Future of Brain Implants: http://www.wsj.com/articles/SB1000142405 2702304914904579435592981780528 http://www.thefoodrush.com/blog/rob ot-chef-replace-us-kitchen/ http://www.computationallegalstudies.com/ http://www.wired.com/2014/06/ai-healthcare/ https://www.rt.com/uk/ 346920-robot-worker- artificial-intelligence/
  • 17.
    17 Supervised Learning. A.I. Learning Principles. "10,000-Hour Rule“ Thekey to achieving world class expertise in any skill, is, to a large extent, a matter of practicing the correct way, for a total of around 10,000 hours. Malcolm Gladwell, “Outliers” (2008). Unsupervised Learning. The A.I. “Holy Grail” The Default Approach
  • 18.
    18 BIG DATA required! TheAI quality relates directly to the amount of data available …
  • 19.
    19 Supervised learning. Customer satisfactionillustration. Id Age Postpaid $Spend Network Out Network In Objective UX Index Main location … Actual Happiness 123…76 18 0 $10 10 2 -5 (lat, lon) … -2 235…96 38 0 $25 10 15 +2 (lat, lon) … +2 578..00 28 1 $40 15 25 +4 (lat, lon) … +5 . . . 710…13 49 1 $40 8 5 +1 (lat, lon) … -1 Starting point! I have known data & known outcome (e.g., happiness). Parameter of interest Predicted Happiness -2 -1 +5 +2 Features or Classifiers Treat customer as Happy – However this is False (positive) Treat customer as Un-Happy – However this is False (Negative)
  • 20.
    20 Teaching the A.I. “Clean” Data “Relevant” Features YourModel Quality Requirements (of Model) Some Math  PRODUCT MODEL New Data Creation Process. Actual Result Predicted Result
  • 21.
    21 A bunch ofundefined pictures (Training data) Find Structure / Similarity (Model) Category F Category C PRODUCT MODEL Unsupervised learning. Category C False Positive New Picture
  • 22.
    22 AI Quality Metrics. Youcan be very accurate and precisely wrong! PREDICTED OUTCOME POSITIVE NEGATIVE ACTUAL OUTCOME POSITIVE TRUE POSITIVE FALSE NEGATIVE NEGATIVE FALSE POSITIVE TRUE NEGATIVE PRECISION = #TRUE POSITIVES #TRUE POSITIVES + #FALSE POSITIVES RECALL = #TRUE POSITIVES #TRUE POSITIVES + #FALSE NEGATIVES Can be very costly Can be very costly ACCURACY = #TRUE POSITIVES + #TRUE NEGATIVES ALL DATA POINTS How often a model predicted positive event is correct. How confident we can be that model predicts correctly a positive event
  • 23.
    23 Highly accurate &precisely wrong! Example of how (not) to find the terrorist? 1 person in 15,000 is a terrorist (e.g., out of all Muslims) My (naive) model: All people are non-terrorist. TRUE FALSE TRUE 0 1 FALSE 0 14,999 ACCURACY = 99.99% PRECISION = 0% RECALL = 0% BOOM! False comfort! Predicted Actual It is possible to be perfectly accurate and precisely wrong!
  • 24.
    24 Precision, Recall &the Cost of being wrong and right! 20 Million Muslims in EU. Expect < 1,300 active (Muslim) terrorists in EU* (note: 5,000+ Europeans estimated to have travelled to Iraq & Syria by end of 2015) Assume a more sophisticated model** gives the following: TRUE FALSE TRUE 1.0 thsd 0.3 thsd FALSE 9.0 thsd 19,978 thsd ACCURACY = 99.99% PRECISION = 10% RECALL = 77% Still BOOM! Costly police work and time spend on the wrong people. Predicted Actual (*) 687 (suspects) were arrested for terrorism related-offences in 2015 (source: Europol TE-SAT 2016 report). (**) e.g., Bayesian machine learning models, DNNs, social network analysis (e.g., social physics).
  • 25.
    25 Anomaly Detection. Identifying theblack sheep. Fraud Detection. DDOS Attacks. Imminent Hardware Failure. Virus Propagation. (RT) System Anomaly Detection. Examples. Un-supervised learning models are (usually) used for anomaly detection. Look for Precision & Recall (accuracy tend to be a overrated) Appropriate Learning Models.
  • 26.
    26 Alexa https://www.fastcompany.com/3058721/app-economy/the-real-reasons-that-amazons-alexa-may-become-the-go- to-ai-for-the-home; https://www.technologyreview.com/s/601654/amazon-working-on-making-alexa-recognize-your- emotions/ Alexa AI livesin the cloud o Voice is the Main User Interface. o Automatic speech recognition (ASR). o Natural Language Understanding (NLU). o Contextual (to a degree). o Improving by using (e.g., optimizing). o (to come - memory of past conversations). o (to come - affective feedback). o (to come - conversational). User makes a request Audio stream is sent up to Alexa Alexa pass back a Audio response (or textual/graphical) Alexa – the AI for your home. Imagining customer care in your home whenever!
  • 27.
    27 Huge amount ofData! Need for “Clean” Data! Model! (can be up to 80 – 90% of work) Quality Goals! (the more data, the higher quality should result & the less complexity required … in general) Lots of Computing! (to built the model, normally as heavy not to run it) (some) Prerequisites for good A.I.’s (need to beat “Flipping a Coin” or Majority “Vote”) It Starts Here! 70% Train 30% Test
  • 28.
    Have Fun! DON’T BE PERFECTLYACCURATE AND PRECISELY WRONG!