Data Structures and algorithms using dynamicProgramming

DYNAMIC PROGRAMMING
MARZIEH ESKANDARI
NJIT
1/125

DEFINITION
 Dynamic Programming refers to
simplifying a complicated problem
by breaking it down into simpler
sub-problems in a recursive
manner.
2/125

DYNAMIC PROGRAMING: STEPS
1. Define a couple of restricted small subproblems and a “Table” for
their solutions
2. Find a Recursive Relation between the main solution and solutions
of smaller problems
3. use an inductive method for constructing the main solution by
using the solutions of smaller problems
3/125

EXAMPLE 1: FIBONACCI’S SERIES
int fib(int n)
{
if(n==1 || n==2) return 1;
return fib(n-1)+fib(n-2);
}
 F(1)=F(2)=1, F(n)=F(n-1)+F(n-2)
4/125

EXAMPLE 1: FIBONACCI’S SERIES
fib(5)
fib(4) fib(3)
fib(3) fib(2) fib(2) fib(1)
fib(2) fib(1)
5/125

RECURSIVE ALGORITHM: TIME COMPLEXITY
𝑇 𝑛 = 𝑂 1 + 𝑇 𝑛 − 2 + 𝑇(𝑛 − 1) ≤ 𝑐 + 2𝑇 𝑛 − 1
≤ 𝑐 + 2𝑐 + 22𝑇 𝑛 − 2 ≤ 𝑐 + 2𝑐 + 22𝑐 + 23𝑇 𝑛 − 3
≤ 𝑐 + 2𝑐 + 22𝑐 + 23𝑐 + 24𝑇 𝑛 − 4
≤ ⋯ ≤ 𝑐 + 2𝑐 + 22𝑐 + 23𝑐 + 24𝑐 + ⋯ + 2𝑘−1𝑐 + 2𝑘𝑇 𝑛 − 𝑘 𝑘 = 𝑛 − 1
= 𝑐 + 2𝑐 + 22𝑐 + 23𝑐 + 24𝑐 + ⋯ + 2𝑛−2𝑐 + 2𝑛−1𝑇 1
= 𝑐
𝑖=0
𝑛−1
2𝑖 = 𝑐
2𝑛 − 1
2 − 1
= 𝑐 2𝑛 − 1 = 𝑂(2𝑛)
6/125

FIBONACCI’S SERIES: DP APPROACH
fib(int n)
{
F[0]=F[1]=1;
for(i=2;i<=n;i++)
F[i]=F[i-1]+F[i-2];
}
1 - - - - -
1
F
1 2 - - - -
1
F
1 2 3 - - -
1
F
1 2 3 5 - -
1
F
fib(5)=?
- - - - - -
-
F
𝑂(𝑛)
7/125

BINOMIAL COEFFICIENTS/ PASCAL TRIANGLE
8/125
𝑥 + 𝑦 𝑛 =
𝑟=0
𝑛
𝑛
𝑟
𝑥𝑟𝑦𝑛−𝑟
𝑛
𝑟
=
𝑛 − 1
𝑟 − 1
+
𝑛 − 1
𝑟
0 ≤ r ≤ n
𝑛
0
= 1
𝑟
𝑟
= 1
n=0 1
n=1 1 1
n=2 1 2 1
n=3 1 3 3 1
n=4 1 4 6 4 1
n=5 1 5 10 10 5 1

EXAMPLE 2: BINOMIAL COEFFICIENTS
int BC(int n, int r)
{
if(r==0 || n==r) return 1;
return BC(n-1,r)+BC(n-1,r-1);
}
 n>=r: C(r,r)=C(n,0)=1, C(n,r)=C(n-1,r-1)+C(n-1,r)
9/125

EXAMPLE 2: BINOMIAL COEFFICIENT
C(4,2)
C(3,2) C(3,1)
C(2,2) C(2,1) C(2,1) C(2,0)
C(1,1) C(1,0) C(1,1) C(1,0)
10/125

RECURSIVE ALGORITHM: TIME COMPLEXITY
𝑇 𝑛, 𝑟 = 𝑂 1 + 𝑇 𝑛 − 1, 𝑟 + 𝑇 𝑛 − 1, 𝑟 − 1 ≤ 𝑂 1 + 2𝑇 𝑛 − 1, 𝑟
≤ 𝑐 + 2𝑇(𝑛 − 1, 𝑟) ≤ 𝑐 + 2𝑐 + 22𝑇 𝑛 − 2, 𝑟 ≤ 𝑐 + 2𝑐 + 22𝑐 + 23𝑇 𝑛 − 3, 𝑟
≤ 𝑐 + 2𝑐 + 22𝑐 + 23𝑐 + 24𝑇 𝑛 − 4, 𝑟
≤ ⋯ ≤ 𝑐 + 2𝑐 + 22𝑐 + 23𝑐 + 24𝑐 + ⋯ + 2𝑘−1𝑐 + 2𝑘𝑇 𝑛 − 𝑘, 𝑟 𝑘 = 𝑛 − 𝑟
= 𝑐 + 2𝑐 + 22𝑐 + 23𝑐 + 24𝑐 + ⋯ + 2𝑛−𝑟−1𝑐 + 2𝑛−𝑟𝑇 𝑟, 𝑟
= 𝑐
𝑖=0
𝑛−𝑟−1
2𝑖 = 𝑐
2𝑛−𝑟 − 1
2 − 1
= 𝑐 2𝑛−𝑟 − 1 = 𝑂(2𝑛−𝑟)
11/125

0
1
2
3
4
0 1 2 3
4
12/125
𝑀 𝑖 𝑗 =
𝑖
𝑗
, 𝑗 ≤ 𝑖

0
1
2
3
4
0 1 2 3
4
0
0
1
0
1
1
2
0
2
1
2
2
3
0
3
1
3
2
3
3
4
0
4
1
4
2
4
3
4
4
13/125
𝑀 𝑖 𝑗 =
𝑖
𝑗
, 𝑗 ≤ 𝑖

{
for(i=0;i<=n;i++)
for(j=0;j<=r;j++)
if(i>=j)
{
if(j==0 || i==j) M[i][j]=1;
else
M[i][j]=M[i-1][j]+M[i-1][j-1];
}
return M[n][r];
}
0
1
2
3
4
0 1 2 3
4
1
1 1
1 1
1 1
1 1
14/125
𝑀 𝑖 𝑗 =
𝑖
𝑗
, 𝑗 ≤ 𝑖

{
for(i=0;i<=n;i++)
for(j=0;j<=r;j++)
if(i>=j)
{
if(j==0 || i==j) M[i][j]=1;
else
M[i][j]=M[i-1][j]+M[i-1][j-1];
}
return M[n][r];
}
0
1
2
3
4
0 1 2 3
4
1
1 1
1 1
1 1
1 1
M[4][2]=? 15/125
𝑀 𝑖 𝑗 =
𝑖
𝑗
, 𝑗 ≤ 𝑖

0
1
2
3
4
0 1 2 3
4
1
1 1
1 1
1 1
1 1
M[2][1]=?
16/125

0
1
2
3
4
0 1 2 3
4
1
1 1
1 2 1
1 1
1 1
M[2][1]=?
17/125

0
1
2
3
4
0 1 2 3
4
1
1 1
1 2 1
1 1
1 1
M[3][1]=?
18/125

0
1
2
3
4
0 1 2 3
4
1
1 1
1 2 1
1 3 1
1 1
M[3][1]=?
19/125

0
1
2
3
4
0 1 2 3
4
1
1 1
1 2 1
1 3 1
1 1
M[3][2]=?
20/125

0
1
2
3
4
0 1 2 3
4
1
1 1
1 2 1
1 3 3 1
1 1
M[3][2]=?
21/125

0
1
2
3
4
0 1 2 3
4
1
1 1
1 2 1
1 3 3 1
1 1
M[4][1]=?
22/125

0
1
2
3
4
0 1 2 3
4
1
1 1
1 2 1
1 3 3 1
1 4 1
M[4][1]=?
23/125

0
1
2
3
4
0 1 2 3
4
1
1 1
1 2 1
1 3 3 1
1 4 1
M[4][2]=?
24/125

0
1
2
3
4
0 1 2 3
4
1
1 1
1 2 1
1 3 3 1
1 4 6 1
M[4][2]=?
25/125

BINOMIAL COEFFICIENT: TIME COMPLEXITY
{
for(i=0;i<=n;i++)
for(j=0;j<=r;j++)
if(i>=j)
{
if(j==0 || i==j) M[i][j]=1;
else
M[i][j]=M[i-1][j]+M[i-1][j-1];
}
return M[n][r];
}
26/125
𝑂(𝑛𝑟)

EXAMPLE 3: FLOYD’S METHOD
 Given a weighted directed graph, find the shortest path between every pair of
nodes.
A D
C
E
G
B
1
1
5
9
3
2 4 2
3
3
A to B
A to C
A to D
A to E
B to A
B to C
B to D
B to E
C to A
C to B
C to D
C to E
D to A
D to B
D to C
D to E
E to A
E to B
E to C
E to D
27/125

A D
C
E
G
B
1
1
5
9
3
2 4 2
3
3
A-B
A-D-C
A-D
A-D-E
B-D-E-A
B-C
B-D
B-D-E
C-D-E-A
C-D-E-A-B
C-D
C-D-E
D-E-A
D-E-A-B
D-C
D-E
E-A
E-A-B
E-A-D-C
E-A-D
28/125
nodes.

A D
C
E
G
B
1
1
5
9
3
2 4 2
3
3
Lengths of all 20 shortest paths
29/125
nodes.

A D
C
E
G
B
1
1
5
9
3
2 4 2
3
3
A to B: 1
A to C: 3
A to D: 1
A to E: 4
B to A: 8
B to C: 3
B to D:2
B to E: 5
C to A: 10
C to B: 11
C to D: 4
C to E: 7
D to A: 6
D to B: 7
D to C: 2
D to E: 3
E to A: 3
E to B: 4
E to C: 6
E to D: 4
30/125
nodes.

GRAPH: REPRESENTATION
A D
C
E
G
B
1
1
5
9
3
2 4 2
3
3
A B C D E
A 0 1
8
1 5
B 9 0 3 2
8
C
8
8
0 4
8
D
8
8
2 0 3
E 3
8
8
8
0
W
31/125
W[i][j]=weight of edge from vertex i to j

SHORTEST PATHS: REPRESENTATION
A D
C
E
G
B
1
1
5
9
3
2 4 2
3
3
A B C D E
A 0 1 3 1 4
B 8 0 3 2 5
C 10 11 0 4 7
D 6 7 2 0 3
E 3 4 6 4 0
D
32/125
D[i][j]=length of shortest path from vertex i to j

OPTIMAL SUBSTRUCTURE
A
B
C
SP(A,B)=P1+P2
If P1 is not the shortest path between A and B,
let P3 is the shortest path between A and C.
So we have: |P3|+|P2|<|P1|+|P2|=|SP(A,B)|
That is a contradiction.
P1 P2
P3
33/125

DEFINING SUBPROBLEMS
 Find the shortest path between all vertices when the graph is
restricted to the first k vertices ({v1,v2,v3,…,vk}) and start and target
vertices.
v1 v4
v3
v5
v2
1
1
5
9
3
2 4 2
3
3
k=3
34/125

 Find the shortest path between all vertices when the graph is
restricted to the first k vertices ({v1,v2,v3,…,vk}) and start and target
vertices.
v1
v3
v2
1 9
3
k=3
35/125

SUBPROBLEMS: EXAMPLE
v1
v3
v2
1
3
K=3
9
d(v1,v2)=1
d(v1,v3)=4
36/125
d(vi,vj): length of shortest path from vi to vj

v4
2 4 2
v1
v3
v2
1
3
K=3
9
d(v1,v2)=1
d(v1,v3)=4
d(v1,v4)=1
1
37/125

v5
5
3
v1
v3
v2
1
3
K=3
9
d(v1,v2)=1
d(v1,v3)=4
d(v1,v4)=1
d(v1,v5)=5
38/125

v1
v3
v2
1
3
K=3
9
d(v1,v2)=1
d(v1,v3)=4
d(v1,v4)=1
d(v1,v5)=5
d(v2,v1)=9
d(v2,v3)=3
39/125

v4
2 4 2
d(v1,v4)=1
1
v1
v3
v2
1
3
K=3
9
d(v1,v2)=1
d(v1,v3)=4
d(v2,v4)=2
d(v1,v5)=5
d(v2,v1)=9
d(v2,v3)=3
40/125

d(v1,v4)=1
v1
v3
v2
1
3
K=3
9
d(v1,v2)=1
d(v1,v3)=4
d(v2,v4)=2
d(v1,v5)=5
d(v2,v1)=9
d(v2,v3)=3
v5
5
3
d(v2,v5)=14
41/125

d(v1,v4)=1
v1
v3
v2
1
3
K=3
9
d(v1,v2)=1
d(v1,v3)=4
d(v2,v4)=2
d(v1,v5)=5
d(v2,v1)=9
d(v2,v3)=3
d(v2,v5)=14
d(v3,v1)=∞
d(v3,v2)=∞
42/125

v4
2 4 2
1
d(v1,v4)=1
v1
v3
v2
1
3
K=3
9
d(v1,v2)=1
d(v1,v3)=4
d(v2,v4)=2
d(v1,v5)=5
d(v2,v1)=9
d(v2,v3)=3
d(v2,v5)=14
d(v3,v1)=∞
d(v3,v2)=∞
d(v3,v4)=4
43/125

v5
5
3
d(v1,v4)=1
v1
v3
v2
1
3
K=3
9
d(v1,v2)=1
d(v1,v3)=4
d(v2,v4)=2
d(v1,v5)=5
d(v2,v1)=9
d(v2,v3)=3
d(v2,v5)=14
d(v3,v1)=∞
d(v3,v2)=∞
d(v3,v4)=4
d(v3,v5)=∞
44/125

d(v1,v4)=1
v1
v3
v2
1
3
k=3
9
d(v1,v2)=1
d(v1,v3)=4
d(v2,v4)=2
d(v1,v5)=5
d(v2,v1)=9
d(v2,v3)=3
d(v2,v5)=14
d(v3,v1)=∞
d(v3,v2)=∞
d(v3,v4)=4
d(v3,v5)=∞
d(v4,v1)=∞
d(v4,v2)=∞
d(v4,v3)=2
d(v4,v5)=3
d(v5,v1)=3
d(v5,v2)=4
d(v5,v3)=7
d(v5,v4)=4
45/125

SUBPROBLEMS: REPRESENTATION
v1 v4
v3
v5
v2
1
1
5
9
3
2 4 2
3
3
v1 v2 v3 v4 v5
v1 0 1 4 1 5
v2 9 0 3 2 14
v3 ∞ ∞ 0 4 ∞
v4 ∞ ∞ 2 0 3
v5 3 4 7 4 0
D
(3)
D [i][j]=length of shortest path between vi and vj in
the graph restricted to vertices {v1,v2,…,vk}
(k)
46/125

NOTATIONS
D =W
(0)
D =D
(n)
D
(k)
D
(k+1)
D =W
(0)
D
(1)
D
(2)
D
(3)
… D =D
(n)
47/125

CALCULATING DS
vi vj
D [i][j]
(k+1)
48/125

CALCULATING DS
vi vj
D [i][j]
(k+1)
vi vj
vi vj
Vk+1
49/125

CALCULATING DS
vi vj
D [i][j]
(k+1)
vi vj
vi vj
Vk+1
D(k)[i][j]
D [i][k+1]+D [k+1][j]
(k) (k)
D [i][k+1]+D [k+1][j],
(k) (k)
D [i][j]=min{
(k+1)
D [i][j]}
(k) 50/125

SHORTEST PATH: EXAMPLE
v1 v4
v3
G
v2
5
15
50
15
5
15 5
v1 v2 v3 v4
v1 0 5 ∞ ∞
v2 50 0 15 5
v3 30 ∞ 0 15
v4 15 ∞ 5 0
W
30
51/125

v1 v2 v3 v4
v1 0 5 ∞ ∞
v2 50 0 15 5
v3 30 35 0 15
v4 15 20 5 0
D
D =W
(0) (1)
v1 v2 v3 v4
v1 0 5 ∞ ∞
v2 50 0 15 5
v3 30 ∞ 0 15
v4 15 ∞ 5 0
52/125

v1 v2 v3 v4
v1 0 5 20 10
v2 45 0 15 5
v3 30 35 0 15
v4 15 20 5 0
D
D
(2) (3)
v1 v2 v3 v4
v1 0 5 20 10
v2 50 0 15 5
v3 30 35 0 15
v4 15 20 5 0
53/125

D =D
(4)
v1 v2 v3 v4
v1 0 5 15 10
v2 20 0 10 5
v3 30 35 0 15
v4 15 20 5 0
54/125

FLOYD’S METHOD: ALGORITHM
Floyd(int W[][], int n)
{
D=W;//initializing D0
for(k=1;k<=n;k++) //calculating Dk
for(i=1;i<=n;i++)
for(j=1;j<=n;j++)
D[i][j]=min{D[i][j],D[i][k]+D[k][j]};
return D;
}
𝑂(𝑛3
)
55/125

FLOYD’S METHOD: ALGORITHM
Floyd(int W[][], int n)
{
D=W;//initializing D0
for(k=1;k<=n;k++) //calculating Dk
for(i=1;i<=n;i++)
for(j=1;j<=n;j++)
if(D[i][k]+D[k][j]<D[i][j])
D[i][j]=D[i][k]+D[k][j];
return D;
}
𝑂(𝑛3
)
56/125

SHORTEST PATHS!
 Rows and columns of P: v1 to vn
 P[i][j]=r: shortest path from vi to vj passes through vr
 P[i][j]=0: edge vivj is the shortest path from vi to vj
1. How can I calculate P?
2. How can I use P for finding shortest paths?
57/125

SHORTEST PATHS!
P[i][j]=r: shortest path from vi to vj passes through vr
P[i][j]=0: edge vivj is the shortest path from vi to vj
v1 v2 v3 v4
v1 0 0 4 2
v2 4 0 4 0
v3 0 1 0 0
v4 0 1 0 0
P
58/125

SHORTEST PATHS!
v1 v2 v3 v4
v1 0 0 4 2
v2 4 0 4 0
v3 0 1 0 0
v4 0 1 0 0
P
SP(v1,v3)=?? v1 v3
59/125

SHORTEST PATHS!
v1 v2 v3 v4
v1 0 0 4 2
v2 4 0 4 0
v3 0 1 0 0
v4 0 1 0 0
P
SP(v1,v3)=??
P[v1][v3]=4
v1 v3
v4
60/125

SHORTEST PATHS!
v1 v2 v3 v4
v1 0 0 4 2
v2 4 0 4 0
v3 0 1 0 0
v4 0 1 0 0
P
SP(v1,v3)=??
P[v1][v3]=4
1. SP(v1,v4)=?
2. SP(v4,v3)=?
v1 v3
v4
61/125

SHORTEST PATHS!
v1 v2 v3 v4
v1 0 0 4 2
v2 4 0 4 0
v3 0 1 0 0
v4 0 1 0 0
P
SP(v1,v3)=??
P[v1][v3]=4
1. P[v1][v4]=?
2. P[v4][v3]=?
v1 v3
v4
62/125

SHORTEST PATHS!
v1 v2 v3 v4
v1 0 0 4 2
v2 4 0 4 0
v3 0 1 0 0
v4 0 1 0 0
P
SP(v1,v3)=??
P[v1][v3]=4
1. P[v1][v4]=2
2. P[v4][v3]=0
v1 v3
v4
63/125

SHORTEST PATHS!
v1 v2 v3 v4
v1 0 0 4 2
v2 4 0 4 0
v3 0 1 0 0
v4 0 1 0 0
P
SP(v1,v3)=??
P[v1][v3]=4
1. P[v1][v4]=2
2. P[v4][v3]=0
v1
v2
64/125
v4
v3

SHORTEST PATHS!
v1 v2 v3 v4
v1 0 0 4 2
v2 4 0 4 0
v3 0 1 0 0
v4 0 1 0 0
P
SP(v1,v3)=??
P[v1][v3]=4
1. P[v1][v4]=2
1.1. P[v1][v2]=0
1.2. P[v2][v4]=0
2. P[v4][v3]=0
v1
v2
65/125
v3
v4

SHORTEST PATHS!
v1 v2 v3 v4
v1 0 0 4 2
v2 4 0 4 0
v3 0 1 0 0
v4 0 1 0 0
P
SP(v1,v3)=??
P[v1][v3]=4
1. P[v1][v4]=2
1.1. P[v1][v2]=0
1.2. P[v2][v4]=0
2. P[v4][v3]=0
v2
66/125
v3
v4
v1

CALCULATING P
vi vj
D [i][j]
(k+1)
vi vj
vi vj
Vk+1
P[i][j]=k+1
D [i][k+1]+D [k+1][j] : P[i][j]=k+1
(k) (k)
If D [i][j]=
(k+1) 67/125

FLOYD’S ALGORITHM
Floyd(int W[][], int n){
D=W;
P=0;
for(k=1;k<=n;k++)
for(i=1;i<=n;i++)
for(j=1;j<=n;j++)
if(D[i][k]+D[k][j]<D[i][j])
{
D[i][j]=D[i][k]+D[k][j];
P[i][j]=k+1;
}
return D;
}
68/125

EXAMPLE 4: MATRIX CHAIN MULTIPLICATION
 Given a sequence of matrices, find the most efficient way to multiply these
matrices together.
 Decide in which order to perform the multiplications to minimize the number of
all regular multiplications.
A: 13x5
B: 5x89
AxB=
(13x89)x5 multiplications
=
89
13
69/125
x
i j
i
j

EXAMPLE 4: MATRIX CHAIN MULTIPLICATION
 Given a sequence of matrices, find the most efficient way to multiply these
matrices together. Decide in which order to perform the multiplications to
minimize the number of all scaler multiplications.
A: 13x5
B: 5x89
C:89x3
D: 3x34
(((AB)C)D): (13x5x89)+(13x89x3)+(13x3x34)=10582
((AB)(CD)): (13x5x89)+(3x89x34)+(13x89x34)=54201
((A(BC))D): (5x89x3)+(13x5x3)+(13x3x34)=2856
(A((BC)D)): (5x89x3)+(5x34x3)+(13x5x34)=4055
(A(B(CD))): (89x3x34)+(5x89x34)+(13x5x34)=26418

70/125

MCM: BRUTE FORCE APPROACH
 The number of all orders in which we parenthesize the product of n
matrices is the Catalan number:
1
𝑛
2𝑛 − 2
𝑛 − 1
= 𝑂(𝑛!)
71/125

MCM: NOTATIONS
A =
di
di-1
i
A =A A A … A A
1 2 3 n-1 n
A : x
i di-1 di
A : x
d0 dn
A : x
1 d0 d1
A : x
2 d1 d2
A : x
3 d2 d3
A : x
n-1 dn-2 d
n-1
A : x
n dn-1 dn
…
A A
i i+1
Number of multiplications of isd d
i+1
di-1 i 72/125

ORDERS! / MINIMUM NUMBER OF MULTIPLICATIONS!
((((A1 A2 )(A3 A4)) A5)A6 )(((A7 A8 ) A9 )(A10 (A11 A12 )))
73/125
A1 A2 A3 A4 A5A6 A7 A8 A9A10 A11 A12

 Find the most efficient way to multiply the following matrices
together:
 For i>=j, M[i][j]=0
 M[1][n]=Optimum number of multiplications.
A x A x … x A
i i+1 j
M[i][j]=Minimum number of multiplications
in product Ai Ai+1 … Aj , i<j
74/125

CALCULATING M[I][J] A A … A
i i+1 j
d d
j
di-1 i
M[i+1][j]+
d d
j
di-1 i+1
M[i][i+1]+M[i+2][j]+
d d
j
di-1 i+2
M[i][i+2]+M[i+3][j]+
d d
j
di-1 k
M[i][k]+M[k+1][j]+
…
…
d d
j
di-1 j-3
M[i][j-3]+M[j-2][j]+
d d
j
di-1 j-2
M[i][j-2]+M[j-1][j]+
d d
j
di-1 j-1
M[i][j-1]+
(A … A )(A … A )
i i+2 i+3 j
(A … A )(A … A )
i j-3 j-2 j
(A A )(A … A )
i i+1 i+2 j
A (A … A )
i i+1 j
…
(A … A )(A … A )
i k k+1 j
…
(A … A )(A A )
i j-2 j-1 j
(A … A ) A
i j-1 j
MIN
75/125

CALCULATING M[I][J]
A A … A
i i+1 j
(A … A )(A … A )
i k k+1 j
d d
j
di-1 k
M[i][j]= MIN { M[i][k]+M[k+1][j] + }
i<=k<j
MIN
M[i][i]= 0 76/125

CALCULATING M
1
2
3
4
:
n
1 2 3 4
…
n
M[i][j]=?
A A … A
i i+1 j
77/125

CALCULATING M
M[i][j]=?
A A … A
i i+1 j
i<=j 1
2
3
4
:
n
1 2 3 4
…
n
78/125

CALCULATING M
Diagonal 0
Diagonal 1
Diagonal 2
Diagonal n-2
Diagonal n-1
M[i][i]: n
M[i][i+1]: n-1
M[i][i+2]: n-2
M[i][i+d]: n-d
M[i][i+n-2]: 2
M[1][1+n-1]: 1
Diagonal d: M[i][i+d], 1<=i<=n-d M[i][j] lies on Diagonal j-i
1
2
3
4
:
n
1 2 3 4
…
n
Diagonal d
79/125

CALCULATING M
M[i][i]=0
1
2
3
4
:
n
1 2 3 4
…
n
0
0
0
0
0
0
80/125

CALCULATING M
d d
i+1
di-1 k
M[i][i+1]= MIN { M[i][k]+M[k+1][i+1]+ }=
i<=k<i+1
d d
i+1
di-1 i
M[i][i]+M[i+1][i+1]+ i+1
=d d d
i-1 i
d d
2
d0 1
1
2
3
4
:
n
1 2 3 4
…
n
0
0
0
0
0
0
81/125

CALCULATING M
d di+2
di-1 k
M[i][i+2]= MIN { M[i][k]+M[k+1][i+2]+ }=
i<=k<i+2
d di+2
di-1 i
MIN{M[i][i]+M[i+1][i+2]+ d di+2
di-1 i+1
, M[i][i+1]+M[i+2][i+2]+ }
1
2
3
4
:
n
1 2 3 4 … n
0
0
0
0
0
0
82/125

CALCULATING M
d dj
di-1 k
M[i][j]= MIN { M[i][k]+M[k+1][j] + }
i<=k<j
k-i<j-i j-k-1<j-i
1
2
3
4
:
n
1 2 3 4 … n
83/125
Diagonal j-i

MATRIX CHAIN MULTIPLICATION
int MCM(int n, int d[]){
for(i=1;i<=n;i++) M[i][i]=0;// Diagonal 0
for(d=1;d<=n-1;d++) // Diagonals 1 to n-1
for(i=1;i<=n-d;i++)
{
j=i+d;
M[i][j]=Min(M[i][k]+M[k+1][j]+d[i-1]*d[k]*d[j]);
}
return M[1][n];
}
i<=k<=j-1
𝑂(𝑛3)
84/125

ORDERS!
P[i][j]=k: the optimal order is
(A … A )(A … A )
i k k+1 j
85/125

ORDERS!
P[i][i+1]=i
(A … A )(A … A )
i k k+1 j
86/125

ORDERS!
P
A1 A2 A3 A4 A5 A6
P[i][i+1]=i
(A … A )(A … A )
i k k+1 j
1
2
3
4
5
6
1 2 3 4
5
6
1 1 1 1
3 4 5
4 5
5
87/125

ORDERS!
P
A1 A2 A3 A4 A5 A6
P[1][6]=1
(A … A )(A … A )
i k k+1 j
1
2
3
4
5
6
1 2 3 4
5
6
1 1 1 1
3 4 5
4 5
5
A1 (A2 A3 A4 A5 A6)
88/125

ORDERS!
P
A1 A2 A3 A4 A5 A6
P[1][6]=1
P[2][6]=5
P[i][i+1]=i
(A … A )(A … A )
i k k+1 j
1
2
3
4
5
6
1 2 3 4
5
6
1 1 1 1
3 4 5
4 5
5
A1 (A2 A3 A4 A5 A6)
A1 ((A2 A3 A4 A5 ) A6)
89/125

ORDERS!
P
A1 A2 A3 A4 A5 A6
P[1][6]=1
P[2][6]=5
P[2][5]=4
P[i][i+1]=i
(A … A )(A … A )
i k k+1 j
1
2
3
4
5
6
1 2 3 4
5
6
1 1 1 1
3 4 5
4 5
5
A1 (A2 A3 A4 A5 A6)
A1 ((A2 A3 A4 A5 ) A6)
A1 (((A2 A3 A4 )A5 ) A6)
90/125

ORDERS!
P
A1 A2 A3 A4 A5 A6
P[1][6]=1
P[2][6]=5
P[2][5]=4
P[2][4]=3
j>=i+1
(A … A )(A … A )
i k k+1 j
1
2
3
4
5
6
1 2 3 4
5
6
1 1 1 3
3 4 5
4 5
5
A1 (A2 A3 A4 A5 A6)
A1 ((A2 A3 A4 A5 ) A6)
A1 (((A2 A3 A4 )A5 ) A6)
A1 ((((A2 A3 )A4 )A5 ) A6)
91/125

ORDERS!
((((A1 A2 )(A3 A4)) A5)A6 )(((A7 A8 ) A9 )(A10 (A11 A12 )))
1. P[1][12]=6
1.1. P[1][6]=5
1.1.1. P[1][5]=4
1.1.1.1. P[1][4]=2
1.2. P[7][12]=9
1.2.1. P[7][9]=8
1.2.2. P[10][12]=10
92/125

MATRIX CHAIN MULTIPLICATION
int MCM(int n, int d[]){
for(i=1;i<=n;i++) M[i][i]=0;// Diagonal 0
for(i=1;i<=n-d;i++)
{
j=i+d;
M[i][j]=Min(M[i][k]+M[k+1][j]+d[i-1]*d[k]*d[j]);
P[i][j]=min_k;
}
return M[1][n];
}
1<=k<=j-1
𝑂(𝑛3)
93/125

65
40
EXAMPLE 5: OPTIMAL BINARY SEARCH TREE
50
60
60
50
55 70
65 75
75
40
70
40
Depth=2 Depth=3 94/125

BINARY TREE SEARCH
node BTS(int x, node root)
{
p=root;
while (p!=NULL)
{
if(p.key==x) return p;
if(p.key<x ) p=p.left;
else p=p.right;
}
return NULL;
}
𝑂(𝐷𝑒𝑝𝑡ℎ)
95/125
40
50
60
55 70
65 75
Search time for x= depth(x)+1

OPTIMAL BINARY SEARCH TREE
 Given a sorted array key 𝑘1 ≤ 𝑘2 ≤ ⋯ ≤ 𝑘𝑛−1 ≤ 𝑘𝑛
of search keys and an array P[0.. n-1], where P[i] is
the number of searches for 𝑘𝑖.
 Construct a binary search tree of all keys such that
the total cost of all the searches is as small as
possible.
Key Frequenci
es
k1 p1
k2 p2
… …
kn-1 pn-1
kn pn
96/125

EXAMPLE
Key Frequenci
es
k1 12
k2 10
k3 8
k4 20
97/125
k1
k2
k4
k3

EXAMPLE
Key Frequenci
es
k1 12
k2 10
k3 8
k4 20
98/125
12 × 2 + 10 × 1 + 8 × 3 + 20 × 2 = 98
k1
k2
k4
k3
2
1
2
3

 Search time for 𝑘𝑠 = 𝑑𝑒𝑝𝑡ℎ(𝑘𝑠) + 1
 Minimizing search time
𝑠=1
𝑛
𝑝𝑠 𝑑𝑒𝑝𝑡ℎ(𝑘𝑠) + 1
Key Frequenci
es
k1 p1
k2 p2
… …
kn-1 pn-1
kn pn
99/125

NORMALIZATION
Key Frequenci
es
k1 12
k2 10
k3 8
k4 20
100/125
12 + 10 + 8 + 20 = 50
k1
k2
k4
k3
2
1
2
3
/𝟓𝟎
/𝟓𝟎
/𝟓𝟎
/𝟓𝟎

EXAMPLE
Key Frequenci
es
k1 .24
k2 .2
k3 .16
k4 .4
101/125
.24 × 2 + .2 × 1 + .16 × 3 + .4 × 2 = 1.96 = 98/50
k1
k2
k4
k3
2
1
2
3

 𝑑𝑠 = 𝑑𝑒𝑝𝑡ℎ(𝑘𝑠)
 Minimizing average search time
𝐴 𝑇𝑟𝑒𝑒 =
𝑠=1
𝑛
𝑝𝑠 𝑑𝑠 + 1
 pi=1/n: AVL
Key Frequenci
es
k1 p1
k2 p2
… …
kn-1 pn-1
kn pn
𝑠=1
𝑛
𝑝𝑠 = 1 102/125

60
50
40
60
40
50
K P
40 0.
7
50 0.
2
60 0.
1
3(0.7)+2(0.2)+1(0.1)=2.6
40 60
50
2(0.7)+1(0.2)+2(0.1)=1.8
40
50
60
1(0.7)+2(0.2)+3(0.1)=1.4
40
50
60
1(0.7)+3(0.2)+2(0.1)=1.5
2(0.7)+3(0.2)+1(0.1)=2.1
103/125

kr
ki ,k2 … ,kr-1 kr+1 ,kr+2 … ,kj
L
R
A[i][j]=Minimum Average Search Time for a binary tree
of keys: ki, ki+1, …,kj-1, kj : 1<= i <= j <=n
104/125

ki
A[i][i]=pi
105/125

kr
k1 ,k2 … ,kr-1 kr+1 ,kr+2 … ,kn
L
A[1][n]=?
R
106/125

ki , … ,kr-1 kr+1 , … ,kj
L R
kr
T*
if 𝒌𝒓 is the root of optimal tree: 𝑨 𝒊 [𝒋] = 𝑨[𝒊][𝒓 − 𝟏] + 𝑨[𝒓 + 𝟏][𝒋] +
𝒔=𝒊
𝒋
𝒑𝒔
107/125

kr
k1 ,k2 … ,kr-1 kr+1 ,kr+2 … ,kn
L
T
R
𝑘𝑠 < 𝑘𝑟 ∶ 𝑑𝑠 = 1 + 𝑑𝑠
𝐿
𝑎𝑛𝑑 𝑘𝑠 > 𝑘𝑟 ∶ 𝑑𝑠 = 1 + 𝑑𝑠
𝑅
𝑑𝑠
𝐿
= 𝑑𝑒𝑝𝑡ℎ(𝑘𝑠) 𝑖𝑛 𝐿
𝑑𝑠
𝑅 = 𝑑𝑒𝑝𝑡ℎ(𝑘𝑠) 𝑖𝑛 𝑅
108/125

ki , … ,kr-1 kr+1 , … ,kj
L R
𝐴(𝑇) =
𝑠=𝑖
𝑗
𝑝𝑠𝑐𝑠 =
𝑠=𝑖
𝑟−1
𝑝𝑠𝑐𝑠 + 𝑝𝑟𝑐𝑟 +
𝑠=𝑟+1
𝑗
𝑝𝑠𝑐𝑠 =
𝑠=𝑖
𝑟−1
𝑝𝑠(𝑑𝑠+1) + 𝑝𝑟 +
𝑠=𝑟+1
𝑗
𝑝𝑠(𝑑𝑠 + 1)
=
𝑠=𝑖
𝑟−1
𝑝𝑠𝑑𝑠 +
𝑠=𝑖
𝑟−1
𝑝𝑠 + 𝑝𝑟 +
𝑠=𝑟+1
𝑗
𝑝𝑠𝑑𝑠 +
𝑠=𝑟+1
𝑗
𝑝𝑠 =
𝑠=𝑖
𝑟−1
𝑝𝑠𝑑𝑠 +
𝑠=𝑟+1
𝑗
𝑝𝑠𝑑𝑠 +
𝑠=𝑖
𝑗
𝑝𝑠
=
𝑠=𝑖
𝑟−1
𝑝𝑠(𝑑𝑠
𝐿+1) +
𝑠=𝑟+1
𝑗
𝑝𝑠(𝑑𝑠
𝑅 + 1) +
𝑠=𝑖
𝑗
𝑝𝑠 =
𝑠=𝑖
𝑟−1
𝑝𝑠𝑐𝑠
𝐿 +
𝑠=𝑟+1
𝑗
𝑝𝑠𝑐𝑠
𝑅 +
𝑠=𝑖
𝑗
𝑝𝑠
⇒ 𝐴 𝑇 = 𝐴 𝐿 + 𝐴 𝑅 +
𝑠=𝑖
𝑗
𝑝𝑠
kr
T
109/125

CALCULATING A[I][J]
kr
ki
kj
ki+1
ki
kj-1
kj
… …
𝑨[𝒊 + 𝟏][𝒋] +
𝒔=𝒊
𝒋
𝒑𝒔 𝑨 𝒊 𝒊 + 𝑨[𝒊 + 𝟐][𝒋] +
𝒔=𝒊
𝒋
𝒑𝒔 𝑨[𝒊][𝒓 − 𝟏] + 𝑨[𝒓 + 𝟏][𝒋] +
𝒔=𝒊
𝒋
𝒑𝒔
𝑨[𝒊][𝒋 − 𝟏] +
𝒔=𝒊
𝒋
𝒑𝒔
𝑨 𝒊 𝒋 − 𝟐 + 𝑨[𝒋][𝒋] +
𝒔=𝒊
𝒋
𝒑𝒔
𝑨 𝒊 [𝒋] = 𝑴𝑰𝑵{ 𝑨[𝒊][𝒓 − 𝟏] + 𝑨[𝒓 + 𝟏][𝒋] +
𝒔=𝒊
𝒋
𝒑𝒔}
𝒊 ≤ 𝒓 ≤ 𝒋
110/125

REPRESENTING A
1
2
3
:
n
n+
1
0 1 2 … n-1
n
𝟏 ≤ 𝒊 ≤ 𝒋 ≤ 𝒏: 𝑨 𝒊 [𝒋] = 𝑴𝑰𝑵{ 𝑨[𝒊][𝒓 − 𝟏] + 𝑨[𝒓 + 𝟏][𝒋] +
𝒔=𝒊
𝒋
𝒑𝒔}
𝒊 ≤ 𝒓 ≤ 𝒋
111/125

REPRESENTING A
1
2
3
:
n
n+
1
0 1 2 … n-1
n
𝒔=𝒊
𝒋
𝒑𝒔}
𝒊 ≤ 𝒓 ≤ 𝒋
𝒊 = 𝟏 → 𝒓 − 𝟏 = 𝟎
112/125

REPRESENTING A
1
2
3
:
n
n+
1
0 1 2 … n-1
n
𝒔=𝒊
𝒋
𝒑𝒔}
𝒊 ≤ 𝒓 ≤ 𝒋
𝒋 = 𝒏 → 𝒓 + 𝟏 = 𝒏 + 𝟏
113/125
𝒊 = 𝟏 → 𝒓 − 𝟏 = 𝟎

REPRESENTING A
1
2
3
:
n
n+
1
0 1 2 … n-1
n
Diagonal 0
Diagonal 1
Diagonal d
Diagonal n-2
Diagonal n-1
Diagonal -1
A[i][i-1]: n+1
A[i][i]: n
A[i][i+1]: n-1
A[i][i+d]: n-d
A[i][i+n-2]: 2
A[1][1+n-1]: 1
Diagonal d: A[i][i+d], 1<=i<=n-d A[i][j] lies on Diagonal j-i
114/125

REPRESENTING A
𝒔=𝒊
𝒋
𝒑𝒔}
𝒊 ≤ 𝒓 ≤ 𝒋
𝒓 − 𝟏 − 𝒊 ≤ 𝒋 − 𝒊 − 𝟏 𝒋 − 𝒓 − 𝟏 ≤ 𝒋 − 𝒊 − 𝟏
𝑶𝒏 𝒅𝒊𝒂𝒈𝒐𝒏𝒂𝒍 𝒋 − 𝒊
115/125

REPRESENTING A
1
2
3
:
n
n+
1
0 1 2 … n-1
n
0
0
0
0
0
0
116/125

REPRESENTING A
1
2
3
:
n
n+
1
0 1 2 … n-1
n
0 p1
0 p2
0 …
0 …
0 pn
0
117/125

int OBST(int n, int P[]){
for(i=1;i<=n+1;i++) A[i][i-1]=0;// Diagonal -1
for(i=1;i<=n;i++) A[i][i]=P[i];// Diagonal 0
for(i=1;i<=n-d;i++)
{
j=i+d;
A[i][j]=Min(A[i][r-1]+A[r+1][j]+P[i]+…+P[j]);
}
return A[1][n];
}
1<=r<=j
𝑂(𝑛3)
118/125

CONSTRUCTING OPTIMAL TREE
R[i][j]=r: kr is the root in the optimal Tree of ki … kj
R[i][i-1]=0, R[i][i]=i
119/125

R[i][i]=i
P
1
2
3
4
5
6
0 1 2 3
4
5
1 1 1 1 4
2 3 4 5
3 4 5
4 5
5 120/125

R[i][i-1]=0, R[i][i]=i
P
1
2
3
4
5
6
0 1 2 3
4
5
1 1 1 1 4
2 3 4 5
3 4 5
4 5
5
R[1][5]=4
k4
k1 … k3 k5
121/125

R[i][i-1]=0, R[i][i]=i
P
1
2
3
4
5
6
0 1 2 3
4
5
1 1 1 1 4
2 3 4 5
3 4 5
4 5
5
R[1][3]=1
k4
k2 , k3
k5
k1
122/125

R[i][i-1]=0, R[i][i]=i
P
1
2
3
4
5
6
0 1 2 3
4
5
1 1 1 1 4
2 3 4 5
3 4 5
4 5
5
R[2][3]=3
k4
k5
k1
k3
k2
123/125

int OBST(int n, int P[]){
for(i=1;i<=n+1;i++) {A[i][i-1]=0;R[i][i-1]=0;}// Diagonal -1
for(i=1;i<=n;i++) {A[i][i]=P[i];R[i][i]=i;}// Diagonal 0
for(i=1;i<=n-d;i++)
{
j=i+d;
A[i][j]=Min(A[i][r-1]+A[r+1][j]+P[i]+…+P[j]);
R[i][j]=min_r;
}
return A[1][n];
}
1<=r<=j
𝑂(𝑛3)
124/125

ANY QUESTION??
HAVE A NICE DAY!
125/125

Data Structures and algorithms using dynamicProgramming

More Related Content

Similar to Data Structures and algorithms using dynamicProgramming

Recently uploaded

Data Structures and algorithms using dynamicProgramming