Costas Busch - LSU 1
A Universal Turing Machine
Costas Busch - LSU 2
Turing Machines are “hardwired”
they execute
only one program
A limitation of Turing Machines:
Real Computers are re-programmable
Costas Busch - LSU 3
Solution: Universal Turing Machine
• Reprogrammable machine
• Simulates any other Turing Machine
Attributes:
Costas Busch - LSU 4
Universal Turing Machine
simulates any Turing Machine M
Input of Universal Turing Machine:
Description of transitions of M
Input string of M
Costas Busch - LSU 5
Universal
Turing
Machine
M
Description of
Tape Contents of
M
State of M
Three tapes
Tape 2
Tape 3
Tape 1
Costas Busch - LSU 6
We describe Turing machine
as a string of symbols:
We encode as a string of symbols
M
M
Description of M
Tape 1
Costas Busch - LSU 7
Alphabet Encoding
Symbols: a b c d 
Encoding: 1 11 111 1111
Costas Busch - LSU 8
State Encoding
States: 1q 2q 3q 4q 
Encoding: 1 11 111 1111
Head Move Encoding
Move:
Encoding:
L R
1 11
Costas Busch - LSU 9
Transition Encoding
Transition: ),,(),( 21 Lbqaq 
Encoding: 10110110101
separator
Costas Busch - LSU 10
Turing Machine Encoding
Transitions:
),,(),( 21 Lbqaq 
Encoding:
10110110101
),,(),( 32 Rcqbq 
110111011110101100
separator
Costas Busch - LSU 11
Tape 1 contents of Universal Turing Machine:
binary encoding
of the simulated machine M
1100011101111010100110110110101
Tape 1
Costas Busch - LSU 12
A Turing Machine is described
with a binary string of 0’s and 1’s
The set of Turing machines
forms a language:
each string of this language is
the binary encoding of a Turing Machine
Therefore:
Costas Busch - LSU 13
Language of Turing Machines
L = { 010100101,
00100100101111,
111010011110010101,
…… }
(Turing Machine 1)
(Turing Machine 2)
……
Costas Busch - LSU 14
Countable Sets
Costas Busch - LSU 15
Infinite sets are either: Countable
or
Uncountable
Costas Busch - LSU 16
Countable set:
There is a one to one correspondence (injection)
of
elements of the set
to
Positive integers (1,2,3,…)
Every element of the set is mapped to a positive number
such that no two elements are mapped to same number
Costas Busch - LSU 17
Example:
Even integers:
(positive)
,6,4,2,0
The set of even integers
is countable
Positive integers:
Correspondence:
,4,3,2,1
n2 corresponds to 1n
Costas Busch - LSU 18
Example:The set of rational numbers
is countable
Rational numbers: ,
8
7
,
4
3
,
2
1
Costas Busch - LSU 19
Naïve Approach
Rational numbers: ,
3
1
,
2
1
,
1
1
Positive integers:
Correspondence:
,3,2,1
Doesn’t work:
we will never count
numbers with nominator 2:
,
3
2
,
2
2
,
1
2
Nominator 1
Costas Busch - LSU 20
Better Approach
1
1
2
1
3
1
4
1
1
2
2
2
3
2
1
3
2
3
1
4




Costas Busch - LSU 21
1
1
2
1
3
1
4
1
1
2
2
2
3
2
1
3
2
3
1
4




Costas Busch - LSU 22
1
1
2
1
3
1
4
1
1
2
2
2
3
2
1
3
2
3
1
4




Costas Busch - LSU 23
1
1
2
1
3
1
4
1
1
2
2
2
3
2
1
3
2
3
1
4




Costas Busch - LSU 24
1
1
2
1
3
1
4
1
1
2
2
2
3
2
1
3
2
3
1
4




Costas Busch - LSU 25
1
1
2
1
3
1
4
1
1
2
2
2
3
2
1
3
2
3
1
4




Costas Busch - LSU 26
Rational Numbers: ,
2
2
,
3
1
,
1
2
,
2
1
,
1
1
Correspondence:
Positive Integers: ,5,4,3,2,1
Costas Busch - LSU 27
We proved:
the set of rational numbers is countable
by describing an enumeration procedure
(enumerator)
for the correspondence to natural numbers
Costas Busch - LSU 28
Definition
An enumerator for is a Turing Machine
that generates (prints on tape)
all the strings of one by one
Let be a set of strings (Language)S
S
S
and
each string is generated in finite time
Costas Busch - LSU 29
Enumerator
Machine for
,,, 321 sss
Ssss ,,, 321
Finite time: ,,, 321 ttt
strings
S
output
(on tape)
Costas Busch - LSU 30
Enumerator Machine
Configuration
Time 0  
0q
Time
sq
1x 1s#1t
prints 1s
Costas Busch - LSU 31
Time
sq
3x 3s#3t
Time
sq
2x 2s#2t
prints 2s
prints 3s
Costas Busch - LSU 32
If for a set there is an enumerator,
then the set is countable
Observation:
The enumerator describes the
correspondence of to natural numbers
S
S
Costas Busch - LSU 33
Example: The set of strings
is countable

 },,{ cbaS
We will describe an enumerator for
Approach:
S
Costas Busch - LSU 34
Naive enumerator:
Produce the strings in lexicographic order:
a
aa
aaa
......
Doesn’t work:
strings starting with
will never be produced
b
aaaa
1s
2s

Costas Busch - LSU 35
Better procedure:
1. Produce all strings of length 1
2. Produce all strings of length 2
3. Produce all strings of length 3
4. Produce all strings of length 4
Proper Order
(Canonical Order)
……
Costas Busch - LSU 36
Produce strings in
Proper Order:
aa
ab
ac
ba
bb
bc
ca
cb
cc
aaa
aab
aac
......
length 2
length 3
length 1
a
b
c
1s
2s

Costas Busch - LSU 37
Theorem: The set of all Turing Machines
is countable
Proof:
Find an enumeration procedure
for the set of Turing Machine strings
Any Turing Machine can be encoded
with a binary string of 0’s and 1’s
Costas Busch - LSU 38
1. Generate the next binary string
of 0’s and 1’s in proper order
2. Check if the string describes a
Turing Machine
if YES: print string on output tape
if NO: ignore string
Enumerator:
Repeat
Costas Busch - LSU 39
0
1
00
01

10110110101 10110110101
00110110101

101101001010101101

Binary strings Turing Machines
1s
2s 101101001010101101
End of Proof
ignore
ignore
ignore
Costas Busch - LSU 40
Uncountable Sets
Costas Busch - LSU 41
We will prove that there is a language
which is not accepted by any Turing machine
L
Technique:
Turing machines are countable
Languages are uncountable
(there are more languages than Turing Machines)
Costas Busch - LSU 42
Theorem:
If is an infinite countable set, then
the powerset of is uncountable.
S
2 S
S
The powerset contains all possible subsets of
Example:
S
2 S
}},{},{},{,{2},{ bababaS S

Costas Busch - LSU 43
Proof:
Since is countable, we can list its
elements in some order
S
},,,{ 321 sssS 
Elements of S
Costas Busch - LSU 44
Elements of the powerset have the form:
},{ 31 ss
},,,{ 10975 ssss
……

S
2
They are subsets of S
Costas Busch - LSU 45
We encode each subset of
with a binary string of 0’s and 1’s
1s 2s 3s 4s 
1 0 0 0}{ 1s
Subset of
Binary encoding
0 1 1 0},{ 32 ss
1 0 1 1},,{ 431 sss



S
S
Costas Busch - LSU 46
Every infinite binary string corresponds
to a subset of :
10 Example: 0111 0
Corresponds to:
S
ssss 2},,,,{ 6541 
S
Costas Busch - LSU 47
Let’s assume (for contradiction)
that the powerset is countable
Then: we can list the elements of the
powerset in some order
},,,{2 321 tttS

S
2
Subsets of S
Costas Busch - LSU 48
1 0 0 0 0
1 1 0 0 0
1 1 0 1 0
1 1 0 0 1
Powerset
element
Binary encoding example
1t
2t
3t
4t




 
Costas Busch - LSU 49
1 0 0 0 0
1 1 0 0 0
1 1 0 1 0
1 1 0 0 1
1t
2t
3t
4t




Binary string: 0011
(birary complement of diagonal)
the binary string whose bits
are the complement of the diagonal
t
t
Costas Busch - LSU 50
0011tThe binary string
corresponds
to a subset of :
S
sst 2},,{ 43  S
Costas Busch - LSU 51
1 0 0 0 0
1 1 0 0 0
1 1 0 1 0
1 1 0 0 1
1t
2t
3t
4t




Question:
0011
the binary string whose bits
are the complement of the diagonal
t
?1tt  NO: differ in 1st bit
t
Costas Busch - LSU 52
1 0 0 0 0
1 1 0 0 0
1 1 0 1 0
1 1 0 0 1
1t
2t
3t
4t




Question:
0011
the binary string whose bits
are the complement of the diagonal
t
?2tt  NO: differ in 2nd bit
t
Costas Busch - LSU 53
1 0 0 0 0
1 1 0 0 0
1 1 0 1 0
1 1 0 0 1
1t
2t
3t
4t




Question:
0011
the binary string whose bits
are the complement of the diagonal
t
?3tt  NO: differ in 3rd bit
t
Costas Busch - LSU 54
Thus: for every i
However,
since they differ in the th bit
Contradiction!!!
itt 
i
S
ttt 2 for some i
i
S
2Therefore the powerset is uncountable
End of proof
Costas Busch - LSU 55
An Application: Languages
The set of all strings:
},,,,,,,,,{},{ *
aabaaabbbaabaababaS 
infinite and countable
Consider Alphabet : },{ baA 
because we can enumerate
the strings in proper order
Costas Busch - LSU 56
The set of all strings:
},,,,,,,,,{},{ *
aabaaabbbaabaababaS 
infinite and countable
Any language is a subset of :
},,{ aababaaL 
S
Consider Alphabet : },{ baA 
Costas Busch - LSU 57
Consider Alphabet : },{ baA 
The set of all Strings:
},,,,,,,,,{},{ **
aabaaabbbaabaababaAS 
infinite and countable
The powerset of contains all languages:
}},,,{},...,,{},,{},{},{,{2 aababaabaabaaS

uncountable
S
Costas Busch - LSU 58
Turing machines: 1M 2M 3M 
Languages accepted
By Turing Machines: 1L 2L 3L 
accepts
Denote: },,,{ 321 LLLX 
countable
Note:
S
X 2
countable
countable
 *
},{ baS 
Consider Alphabet : },{ baA 
Costas Busch - LSU 59
X countable
S
2 uncountable
Languages accepted
by Turing machines:
All possible languages:
Therefore:
S
X 2
 SS
XX 2getwe,2since 
Costas Busch - LSU 60
There is a language not accepted
by any Turing Machine:
Conclusion:
L
S
X 2 XLL S
 and2
Costas Busch - LSU 61
Turing-Acceptable
Languages
Non Turing-Acceptable Languages
L
Costas Busch - LSU 62
Note that: },,,{ 321 LLLX 
is a multi-set (elements may repeat)
since a language may be accepted
by more than one Turing machine
However, if we remove the repeated elements,
the resulting set is again countable since every element
still corresponds to a positive integer

Universal turing coastus

  • 1.
    Costas Busch -LSU 1 A Universal Turing Machine
  • 2.
    Costas Busch -LSU 2 Turing Machines are “hardwired” they execute only one program A limitation of Turing Machines: Real Computers are re-programmable
  • 3.
    Costas Busch -LSU 3 Solution: Universal Turing Machine • Reprogrammable machine • Simulates any other Turing Machine Attributes:
  • 4.
    Costas Busch -LSU 4 Universal Turing Machine simulates any Turing Machine M Input of Universal Turing Machine: Description of transitions of M Input string of M
  • 5.
    Costas Busch -LSU 5 Universal Turing Machine M Description of Tape Contents of M State of M Three tapes Tape 2 Tape 3 Tape 1
  • 6.
    Costas Busch -LSU 6 We describe Turing machine as a string of symbols: We encode as a string of symbols M M Description of M Tape 1
  • 7.
    Costas Busch -LSU 7 Alphabet Encoding Symbols: a b c d  Encoding: 1 11 111 1111
  • 8.
    Costas Busch -LSU 8 State Encoding States: 1q 2q 3q 4q  Encoding: 1 11 111 1111 Head Move Encoding Move: Encoding: L R 1 11
  • 9.
    Costas Busch -LSU 9 Transition Encoding Transition: ),,(),( 21 Lbqaq  Encoding: 10110110101 separator
  • 10.
    Costas Busch -LSU 10 Turing Machine Encoding Transitions: ),,(),( 21 Lbqaq  Encoding: 10110110101 ),,(),( 32 Rcqbq  110111011110101100 separator
  • 11.
    Costas Busch -LSU 11 Tape 1 contents of Universal Turing Machine: binary encoding of the simulated machine M 1100011101111010100110110110101 Tape 1
  • 12.
    Costas Busch -LSU 12 A Turing Machine is described with a binary string of 0’s and 1’s The set of Turing machines forms a language: each string of this language is the binary encoding of a Turing Machine Therefore:
  • 13.
    Costas Busch -LSU 13 Language of Turing Machines L = { 010100101, 00100100101111, 111010011110010101, …… } (Turing Machine 1) (Turing Machine 2) ……
  • 14.
    Costas Busch -LSU 14 Countable Sets
  • 15.
    Costas Busch -LSU 15 Infinite sets are either: Countable or Uncountable
  • 16.
    Costas Busch -LSU 16 Countable set: There is a one to one correspondence (injection) of elements of the set to Positive integers (1,2,3,…) Every element of the set is mapped to a positive number such that no two elements are mapped to same number
  • 17.
    Costas Busch -LSU 17 Example: Even integers: (positive) ,6,4,2,0 The set of even integers is countable Positive integers: Correspondence: ,4,3,2,1 n2 corresponds to 1n
  • 18.
    Costas Busch -LSU 18 Example:The set of rational numbers is countable Rational numbers: , 8 7 , 4 3 , 2 1
  • 19.
    Costas Busch -LSU 19 Naïve Approach Rational numbers: , 3 1 , 2 1 , 1 1 Positive integers: Correspondence: ,3,2,1 Doesn’t work: we will never count numbers with nominator 2: , 3 2 , 2 2 , 1 2 Nominator 1
  • 20.
    Costas Busch -LSU 20 Better Approach 1 1 2 1 3 1 4 1 1 2 2 2 3 2 1 3 2 3 1 4    
  • 21.
    Costas Busch -LSU 21 1 1 2 1 3 1 4 1 1 2 2 2 3 2 1 3 2 3 1 4    
  • 22.
    Costas Busch -LSU 22 1 1 2 1 3 1 4 1 1 2 2 2 3 2 1 3 2 3 1 4    
  • 23.
    Costas Busch -LSU 23 1 1 2 1 3 1 4 1 1 2 2 2 3 2 1 3 2 3 1 4    
  • 24.
    Costas Busch -LSU 24 1 1 2 1 3 1 4 1 1 2 2 2 3 2 1 3 2 3 1 4    
  • 25.
    Costas Busch -LSU 25 1 1 2 1 3 1 4 1 1 2 2 2 3 2 1 3 2 3 1 4    
  • 26.
    Costas Busch -LSU 26 Rational Numbers: , 2 2 , 3 1 , 1 2 , 2 1 , 1 1 Correspondence: Positive Integers: ,5,4,3,2,1
  • 27.
    Costas Busch -LSU 27 We proved: the set of rational numbers is countable by describing an enumeration procedure (enumerator) for the correspondence to natural numbers
  • 28.
    Costas Busch -LSU 28 Definition An enumerator for is a Turing Machine that generates (prints on tape) all the strings of one by one Let be a set of strings (Language)S S S and each string is generated in finite time
  • 29.
    Costas Busch -LSU 29 Enumerator Machine for ,,, 321 sss Ssss ,,, 321 Finite time: ,,, 321 ttt strings S output (on tape)
  • 30.
    Costas Busch -LSU 30 Enumerator Machine Configuration Time 0   0q Time sq 1x 1s#1t prints 1s
  • 31.
    Costas Busch -LSU 31 Time sq 3x 3s#3t Time sq 2x 2s#2t prints 2s prints 3s
  • 32.
    Costas Busch -LSU 32 If for a set there is an enumerator, then the set is countable Observation: The enumerator describes the correspondence of to natural numbers S S
  • 33.
    Costas Busch -LSU 33 Example: The set of strings is countable   },,{ cbaS We will describe an enumerator for Approach: S
  • 34.
    Costas Busch -LSU 34 Naive enumerator: Produce the strings in lexicographic order: a aa aaa ...... Doesn’t work: strings starting with will never be produced b aaaa 1s 2s 
  • 35.
    Costas Busch -LSU 35 Better procedure: 1. Produce all strings of length 1 2. Produce all strings of length 2 3. Produce all strings of length 3 4. Produce all strings of length 4 Proper Order (Canonical Order) ……
  • 36.
    Costas Busch -LSU 36 Produce strings in Proper Order: aa ab ac ba bb bc ca cb cc aaa aab aac ...... length 2 length 3 length 1 a b c 1s 2s 
  • 37.
    Costas Busch -LSU 37 Theorem: The set of all Turing Machines is countable Proof: Find an enumeration procedure for the set of Turing Machine strings Any Turing Machine can be encoded with a binary string of 0’s and 1’s
  • 38.
    Costas Busch -LSU 38 1. Generate the next binary string of 0’s and 1’s in proper order 2. Check if the string describes a Turing Machine if YES: print string on output tape if NO: ignore string Enumerator: Repeat
  • 39.
    Costas Busch -LSU 39 0 1 00 01  10110110101 10110110101 00110110101  101101001010101101  Binary strings Turing Machines 1s 2s 101101001010101101 End of Proof ignore ignore ignore
  • 40.
    Costas Busch -LSU 40 Uncountable Sets
  • 41.
    Costas Busch -LSU 41 We will prove that there is a language which is not accepted by any Turing machine L Technique: Turing machines are countable Languages are uncountable (there are more languages than Turing Machines)
  • 42.
    Costas Busch -LSU 42 Theorem: If is an infinite countable set, then the powerset of is uncountable. S 2 S S The powerset contains all possible subsets of Example: S 2 S }},{},{},{,{2},{ bababaS S 
  • 43.
    Costas Busch -LSU 43 Proof: Since is countable, we can list its elements in some order S },,,{ 321 sssS  Elements of S
  • 44.
    Costas Busch -LSU 44 Elements of the powerset have the form: },{ 31 ss },,,{ 10975 ssss ……  S 2 They are subsets of S
  • 45.
    Costas Busch -LSU 45 We encode each subset of with a binary string of 0’s and 1’s 1s 2s 3s 4s  1 0 0 0}{ 1s Subset of Binary encoding 0 1 1 0},{ 32 ss 1 0 1 1},,{ 431 sss    S S
  • 46.
    Costas Busch -LSU 46 Every infinite binary string corresponds to a subset of : 10 Example: 0111 0 Corresponds to: S ssss 2},,,,{ 6541  S
  • 47.
    Costas Busch -LSU 47 Let’s assume (for contradiction) that the powerset is countable Then: we can list the elements of the powerset in some order },,,{2 321 tttS  S 2 Subsets of S
  • 48.
    Costas Busch -LSU 48 1 0 0 0 0 1 1 0 0 0 1 1 0 1 0 1 1 0 0 1 Powerset element Binary encoding example 1t 2t 3t 4t      
  • 49.
    Costas Busch -LSU 49 1 0 0 0 0 1 1 0 0 0 1 1 0 1 0 1 1 0 0 1 1t 2t 3t 4t     Binary string: 0011 (birary complement of diagonal) the binary string whose bits are the complement of the diagonal t t
  • 50.
    Costas Busch -LSU 50 0011tThe binary string corresponds to a subset of : S sst 2},,{ 43  S
  • 51.
    Costas Busch -LSU 51 1 0 0 0 0 1 1 0 0 0 1 1 0 1 0 1 1 0 0 1 1t 2t 3t 4t     Question: 0011 the binary string whose bits are the complement of the diagonal t ?1tt  NO: differ in 1st bit t
  • 52.
    Costas Busch -LSU 52 1 0 0 0 0 1 1 0 0 0 1 1 0 1 0 1 1 0 0 1 1t 2t 3t 4t     Question: 0011 the binary string whose bits are the complement of the diagonal t ?2tt  NO: differ in 2nd bit t
  • 53.
    Costas Busch -LSU 53 1 0 0 0 0 1 1 0 0 0 1 1 0 1 0 1 1 0 0 1 1t 2t 3t 4t     Question: 0011 the binary string whose bits are the complement of the diagonal t ?3tt  NO: differ in 3rd bit t
  • 54.
    Costas Busch -LSU 54 Thus: for every i However, since they differ in the th bit Contradiction!!! itt  i S ttt 2 for some i i S 2Therefore the powerset is uncountable End of proof
  • 55.
    Costas Busch -LSU 55 An Application: Languages The set of all strings: },,,,,,,,,{},{ * aabaaabbbaabaababaS  infinite and countable Consider Alphabet : },{ baA  because we can enumerate the strings in proper order
  • 56.
    Costas Busch -LSU 56 The set of all strings: },,,,,,,,,{},{ * aabaaabbbaabaababaS  infinite and countable Any language is a subset of : },,{ aababaaL  S Consider Alphabet : },{ baA 
  • 57.
    Costas Busch -LSU 57 Consider Alphabet : },{ baA  The set of all Strings: },,,,,,,,,{},{ ** aabaaabbbaabaababaAS  infinite and countable The powerset of contains all languages: }},,,{},...,,{},,{},{},{,{2 aababaabaabaaS  uncountable S
  • 58.
    Costas Busch -LSU 58 Turing machines: 1M 2M 3M  Languages accepted By Turing Machines: 1L 2L 3L  accepts Denote: },,,{ 321 LLLX  countable Note: S X 2 countable countable  * },{ baS  Consider Alphabet : },{ baA 
  • 59.
    Costas Busch -LSU 59 X countable S 2 uncountable Languages accepted by Turing machines: All possible languages: Therefore: S X 2  SS XX 2getwe,2since 
  • 60.
    Costas Busch -LSU 60 There is a language not accepted by any Turing Machine: Conclusion: L S X 2 XLL S  and2
  • 61.
    Costas Busch -LSU 61 Turing-Acceptable Languages Non Turing-Acceptable Languages L
  • 62.
    Costas Busch -LSU 62 Note that: },,,{ 321 LLLX  is a multi-set (elements may repeat) since a language may be accepted by more than one Turing machine However, if we remove the repeated elements, the resulting set is again countable since every element still corresponds to a positive integer