Chapter 26 aoa

Chapter 26
All-Pairs Shortest Paths
Dr. Muhammad Hanif Durad
Department of Computer and Information Sciences
Pakistan Institute Engineering and Applied Sciences
hanif@pieas.edu.pk
Some slides have bee adapted with thanks from some other lectures
available on Internet. It made my life easier, thanks to Allah
Almighty .

Lecture Outline (1/2)
 All-Pairs Shortest Path
 Extending SSP
 The Floyd-Warshall algorithm
 The structure of a shortest path
 A recursive solution
 Computing weights
 Constructing Shortest Path
 Analysis
 Transitive closure of a directed graph
22. All Pairs Shortest Path.ppt, P-8

Lecture Outline (2/2)
 Johnson’s algorithm for sparse graphs
 Idea
 Preserving Shortest Paths by Reweighting
 Producing nonnegative weights by
reweighting
 Algorithm
 Complexity

All-Pairs Shortest Path Problem
 Generalization of the single source shortest-path problem.
 Simple solution: run the shortest path algorithm for each
vertex  complexity is
 O(|E|.|V|.|V|) = O(|V|4) for Bellman-Ford
 O(|E|.lg |V|.|V|) = O(|V|3 lg |V|) for Dijsktra.
 Can we do better? Intuitively it would seem so, since
there is a lot of repeated work  exploit the optimal sub-
path property.
 We indeed can do better: O(|V|3).
Dr. Hanif Durad 4
lect16.ppt, P-2/39
See next slide for
details

All pair shortest Path Problem
Dr. Hanif Durad 5
 The easiest way!
 Iterate Dijkstra’s and Bellman-Ford |V| times!
 Dijkstra:
 O(VlgV + E) --> O(V2lgV + VE)
 Bellman-Ford:
 O(VE) ------> O(V2E)
 Faster-All-Pairs-Shortest-Paths:
 O(V3lgV)
O(V3)
O(V4)
On dense
graph
22. All Pairs Shortest Path.ppt
Edges (?)=V2

All-shortest paths: example (1)
2
31
5 4
6
7
2-4
1
8
3 4
-5
Dr. Hanif Durad 6

7
2
31
5 4
6
7
2-4
1
8
3 4
-5
0
1
-3
-4 2
2
31
5 4
6
7
2-4
1
8
3 4
-5
)42310( 
1 2 3 4 5
dist )1543(
1 2 3 4 5
pred
1

8
2
31
5 4
6
7
2-4
1
8
3 4
-5
0
3 -4
1-1
2
31
5 4
6
7
2-4
1
8
3 4
-5
)11403( 
1 2 3 4 5
dist )1244( 
1 2 3 4 5
pred

All shortest-paths: representation
We will use matrices to represent the graph, the
shortest path lengths, and the predecessor sub-graphs.
 Edge matrix: entry (i,j) in adjacency matrix W is the
weight of the edge between vertex i and vertex j.
 Shortest-path lengths matrix: entry (i,j) in L is the
shortest path length between vertex i and vertex j.
 Predecessor matrix: entry (i,j) in Π is the predecessor
of j on some shortest path from i (null when i = j or
when there is no path).
Dr. Hanif Durad 9

10
All-shortest paths: definitions
Ejiji
Ejiji
ji
jiwwij










),(and
),(and
if
),(
0
Edge matrix: entry (i,j) in adjacency matrix W
is the weight of the edge between vertex i and vertex j.
Shortest-paths graph: the graphs Gπ,i = (Vπ,i, Eπ,i)
are the shortest-path graphs rooted at vertex I, where:
}{}:{, inullVjV iji  
}}{:),{( ,, iVjjE iiji   

Printing Shortest Path from
vertex i to vertex j
Dr. Hanif Durad 11
PRINT-ALL-PAIRS-SHORTEST-PATH(, ,i j)
1 if i j
2 then print i
3 else if ij NIL
4 then print “no path from” i to j “exists”
5 else PRINT-ALL-SHORTEST-PATH( , , )i ij
6 Print j

Example: edge matrix
2
31
5 4
6
7
2
-4 1
8
3 4
-5






















06
052
04
710
4830
W
1 2 3 4 5
1
2
3
4
5
Dr. Hanif Durad 12

13
Example: shortest-paths matrix
2
31
5 4
6
7
2
-4 1
8
3 4
-5




















06158
20512
35047
11403
42310
L
1 2 3 4 5
1
2
3
4
5
Dr. Hanif Durad

Example: predecessor matrix
2
31
5 4
6
7
2
-4 1
8
3 4
-5

















null
null
null
null
null
5434
1434
1234
1244
1543
1 2 3 4 5
1
2
3
4
5
Dr. Hanif Durad 14

The structure of a shortest path
1. All sub-paths of a shortest path are shortest paths.
Let p = <v1, .. vk> be the shortest path from v1 to vk.
The sub-path between vi and vj, where 1 ≤ i,j ≤ k, pij
= <vi, .. vj> is a shortest path.
2. The shortest path from vertex i to vertex j with at
most m edges is either:
 the shortest path with at most (m-1) edges (no
improvement)
 the shortest path consisting of a shortest path within the (m-
1) vertices + the weight of the edge from a vertex within the
(m-1) vertices to an extra vertex m.

16
Recursive solution to all-shortest
paths
}){min,min( )1(
1
)1()(
kj
m
ik
nk
m
ij
m
ij wlll  


}{min )1(
1
kj
m
ik
nk
wl  








ji
ji
lij
0)0(
Let l(m)
ij be the minimum weight of any path from
vertex i to vertex j that has at most m edges.
When m=0:
For m ≥ 1, l(m)
ij is the minimum of l(m–1)
ij and the
shortest path going through the vertices neighbors:

17
All-shortest-paths: solution
 Let W=(wij) be the edge weight matrix and
L=(lij) the all-shortest shortest path matrix
computed so far, both n×n.
 Compute a series of matrices L(1), L(2), …,
L(n–1) where for m = 1,…,n–1, L(m) = (l(m)
ij) is
the matrix with the all-shortest-path lengths
with at most m edges. Initially, L(1) = W, and
L(n–1) containts the actual shortest-paths.
 Basic step: compute L(m) from L(m–1) and W.

18
Algorithm for extending all-shortest
paths by one edge: from L(m-1) to L(m)
EXTEND-SHORTEST-PATHS(L=(lij),W)
n  rows[L]
Let L’ =(l’ij) be an n×n matrix.
for i  1 to n do
for j  1 to n do
l’ij  ∞
for k  1 to n do
l’ij  min(l’ij, lik + wkj)
return L’
Complexity: Θ(|V|3)

19
This is exactly as matrix
multiplication!
Matrix-Multiply(A,B)
n  rows[A]
Let C =(cij) be an n×n matrix.
for i  1 to n do
for j  1 to n do
cij  0 (l’ij  ∞)
for k  1 to n do
cij  cij+ aik.bkj (l’ij  min(l’ij, lij + wkj))
return L’





min
)(
)1(
cl
bw
al
m
m

Paths with at most two edges
20






















0618
20512
11504
71403
42830






















06
052
04
710
4830
)2(
L






















06
052
04
710
4830
)1(
L





















06
052
04
710
4830
2
31
5 4
6
7
2-4
1
8
3 4
-5

Paths with at most three edges






















0618
20512
11504
71403
42830
)2(
L
21





















06
052
04
710
4830




















06158
20512
115047
11403
42330
2
31
5 4
6
7
2-4
1
8
3 4
-5






















0618
20512
11504
71403
42830
)3(
L




















06158
20512
115047
11403
42330

Paths with at most four edges




















06158
20512
115047
11403
42330
)3(
L




















06158
20512
35047
11403
42310
22





















06
052
04
710
4830
2
31
5 4
6
7
2-4
1
8
3 4
-5




















06158
20512
115047
11403
42330
)4(
L

Paths with at most five edges




















06158
20512
35047
11403
42310
)4(
L




















06158
20512
35047
11403
42310





















06
052
04
710
4830
2
31
5 4
6
7
2-4
1
8
3 4
-5




















06158
20512
35047
11403
42310
)5(
L

24
All-shortest Paths Algorithm
SLOW-ALL-PAIRS-SHORTEST-PATHS(W)
1. n  rows[W]
2. L(1)  W
3. for m  2 to n–1 do
4. L(m)  Extend-Shortest-Paths(L(m–1),W)
5. return L(m)

25
Improved All-shortest Paths
Algorithm
 The goal is to compute the final L(n–1), not all the L(m)
 We can avoid computing most L(m) as follows:
L(1) = W
L(2) = W.W
L(4) = W4 = W2.W2
…        )12()12()2()2( )1lg()1lg()1lg()1lg(
.  

nnnn
WWWL
Since the final product is equal to L(n–1) 
12 )1lg(

nn
only |lg(n–1)| iterations are necessary!
repeated squaring

26
Faster-All-Shortest Paths
Algorithm
FASTER-ALL-PAIRS-SHORTEST-PATHS(W)
1. n  rows[W]
2. L(1)  W
3. m 1
4. while m < n–1 do
5. L(2m)  Extend-Shortest-Paths(L(m),L(m))
6. m 2m
7. return L(m)
Complexity: Θ(|V|3 lg (|V|))

Check Whether it works?
Dr. Hanif Durad 27






















06
052
04
710
4830
)1(
L




















06158
20512
35047
11403
42310
)4(
L






















0618
20512
11504
71403
42830
)2(
L
35
(4
4
7
) [ 4 0511] 11 min(11, 4,4 7,0 11,5 2,11 0)
2
0
l
 
 
 
             
 
 
 
 
YES

Assignment 1
 Problems 25.1-3, 25.1-8 25.1-10 Page
627628
 C++ coding of
 EXTEND-SHORTEST-PATHS,
 SLOW-ALL-PAIRS-SHORTEST-PATHS and
 FASTER-ALL-PAIRS-SHORTEST-PATHS
 Last Date
 Thursday -29 June 2010
Dr. Hanif Durad 28

All pair shortest Path Problem
Dr. Hanif Durad 29
 The easiest way!
 Iterate Dijkstra’s and Bellman-Ford |V| times!
 Dijkstra:
 O(VlgV + E) --> O(V2lgV + VE)
 Bellman-Ford:
 O(VE) ------> O(V2E)
 Faster-All-Pairs-Shortest-Paths:
 O(V3lgV)
O(V3)
O(V4)
On dense
graph
22. All Pairs Shortest Path.ppt

31
Floyd-Warshall algorithm
 Assumes there are no negative-weight cycles.
 Uses a different characterization of the
structure of the shortest path. It exploits the
properties of the intermediate vertices of the
shortest path.
 Runs in O(|V|3).

32
Structure of the shortest path (1)
 An intermediate vertex vi of a simple path p=<v1,..,vk> is
any vertex other than v1 or vk.
 Let V={1,2,…,n} and let K={1,2,…,k} be a subset for k
≤ n. For any pair of vertices i,j in V, consider all paths
from i to j whose intermediate vertices are drawn from
K. Let p be the minimum-weight path among them.
i
k1
j
k2
p

33
1. k is not an intermediate vertex of path p:
All vertices of path p are in the set {1,2,…,k–1}
 a shortest path from i to j with all intermediate
vertices in {1,2,…,k–1} is also a shortest path with all
intermediate vertices in {1,2,…,k}.
2. k is an intermediate vertex of path p:
Break p into two pieces: p1 from i to k and p2 from k to
j. Path p1 is a shortest path from i to k and path p2 is a
shortest path from k to j with all intermediate vertices
in {1,2,…,k–1}.

34
i
k j
p2
all intermediate vertices in
{1,2,…,k–1}
{1,2,…,k}
p1
{1,2,…,k–1}

35
Recursive definition





 
1if),min(
0if
)1()1()1(
)(
kddd
kw
d k
kj
k
ik
k
ij
ijk
ij
Let d(k)
ij be the weight of a shortest path from
vertex i to vertex j for which all intermediate vertices
are in the set {1,2,…,k}. Then:
The matrices D(k) = (d(k)
ij) will keep the intermediate
solutions.

Further Explanation
Dr. Hanif Durad 36
Sl14.pdf

The Floyd-Warshall Algorithm
 Computing the shortest-path weights bottom up
Time complexity (n3)
O(1)
Dr. Hanif Durad 37
ital25.ppt

Example: Floyd-Warshall (1/7)
38






















06
052
04
710
4830
)0(
D






















06
20552
04
710
4830
)1(
D
2
31
5 4
6
7
2-4
1
8
3 4
-5
K={1}
Improvements in d42 and d45
via vertex 1

Dr. Hanif Durad 39
2
3
4
5
2
3
4
5
1
4
8
3
2


4  2: 5 4  5:  2

1
4
40






















06
20552
11504
710
44830
)2(
D






















06
20552
04
710
4830
)1(
D
2
3
5 4
6
7
2-4
1
8
3
-5
K={1,2}
Improvements in d14, d34 and d35
via vertex 2 and then 1

Dr. Hanif Durad
1
3
4
5
1
3
4
5
2
7

1  4: 4 3  4: 5
1  5: 10 3  5: 11
1
3
4
5
! Rejected

3
42






















06
20552
11504
710
44830
)2(
D






















06
20512
11504
710
44830
)3(
D
2
1
5 4
6
7
2-4
1
8
3 4
-5
K={1,2,3}
Improvements in d42
via vertex 3, then 2 and then 1

6
43




















06158
20512
35047
11403
44130
)4(
D






















06
20512
11504
710
4830
)3(
D
2
31
5 4
7
2-4
1
8
3 4
-5
K={1,2,3,4}
Improvements in d13, d14, , d14 ,
d21 , d23 , d25 , d31 , d35 , d51 , d52
and d53
via vertex 4, then 3, then 2 and then 1

6
44




















06158
20512
35047
11403
44130
)4(
D




















06158
20512
35047
11403
42310
)5(
D
2
31
5 4
7
2-4
1
8
3 4
-5
K={1,2,3,4,5}
Improvements in d12, d13 and d14
via vertex 5, then 4, then 3, then 2
and then 1

Extracting Shortest Paths from π
 Use PRINT-ALL-PAIRS-SHORTEST-PATH
Algorithm described earlier
Dr. Hanif Durad 45

46
Transitive Closure (1/3)
 Given a directed graph G=(V,E) with vertices V =
{1,2,…,n} determine for every pair of vertices (i,j) if
there is a path between them.
 The transitive closure graph of G, G*=(V,E*) is such
that E* = {(i,j): if there is a path i and j}.
 Represent E* as a binary matrix and perform logical
binary operations AND (/) and OR (/) instead of min
and + in the Floyd-Warshall algorithm.

47
Transitive Closure (2/3)
( )
( 1) ( 1) ( 1)
0 if 0 and ( , )
1 if 0 and ( , )
( ) for 0
k
ij
k k k
ij ik kj
k i j E
t k i j E
t t t k  
  

  
   
The definition of the transitive closure is:
The matrices T(k) indicate if there is a path with
at most k edges between s and i.

48
Transitive Closure Algorithm (3/3)
TRANSITIVE-CLOSURE(G)
1 | [ ]|n V G
2 for i 1 to n
3 do for j 1 to n
4 do if i j or ( , ) ( )i j E G
5 then tij
( )0
1
6 else tij
( )0
0
7 for k 1 to n
8 do for i 1 to n
9 do for j 1 to n
10 do t t t tij
k
ij
k
ik
k
kj
k( ) ( ) ( ) ( )
( )    1 1 1
11 return T n( )

Example
Dr. Hanif Durad 49
1
4 3
2
T( )0
1 0 0 0
0 1 1 1
0 1 1 0
1 0 1 1













T( )1
1 0 0 0
0 1 1 1
0 1 1 0
1 0 1 1













T( )2
1 0 0 0
0 1 1 1
0 1 1 1
1 0 1 1













T( )3
1 0 0 0
0 1 1 1
0 1 1 1
1 1 1 1













T( )4
1 0 0 0
1 1 1 1
1 1 1 1
1 1 1 1














Assignment 2
 Problems 25.2-6, 25.2-7, 25.2-8 Page
627628
 C++ coding of
 Floyd-Warshall
 Transitive closure
 Last Date
Dr. Hanif Durad 50

25.3 Johnson’s algorithm for
sparse graphs
 Makes clever use of Bellman-Ford and Dijkstra to do
All-Pairs-Shortest-Paths efficiently on sparse graphs.
 Motivation
 By running Dijkstra |V| times, we could do APSP in time
 O(V2lgV +VElgV) (Modified Dijkstra), or
 O (V2lgV +VE) (Fibonacci Dijkstra).
 This beats O (V3) (Floyd-Warshall) when the graph is
sparse.
 Problem: negative edge weights.
Dr. Hanif Durad 52

The Basic Idea
 Reweight the edges so that:
1. No edge weight is negative.
2. Shortest paths are preserved. (A shortest path in the
original graph is still one in the new, reweighted
graph.)
 An obvious attempt: subtract the minimum weight from
all the edge weights. E.g. if the minimum weight is -2:
 -2 - -2 = 0
 3 - -2 = 5 etc.
8a-ShortestPathsMore.ppt
Dr. Hanif Durad 53

Counter Example
 Subtracting the minimum weight from every
weight doesn’t work.
 Consider:
 Paths with more edges are unfairly penalized.
-2 -1
-2
0 1
0
Dr. Hanif Durad 54

Johnson’s Insight
 Add a vertex s to the original graph G, with
edges of weight 0 to each vertex in G:
 Assign new weights ŵ to each edge as follows:
ŵ(u, v) = w(u, v) + d(s, u) - d(s, v)
s 0
0
0
Dr. Hanif Durad 55

Question 1
 Are all the ŵ’s non-negative? Yes:
 Otherwise, s u  v would be
shorter than the shortest path
from s to v.
s
u
v
w(u, v)
0),(),(),(
),(),(),(


vsusvuw
vsvuwus
dd
dd
:Rewriting
),(ˆ vuw
),(),(),( vsvuwus dd  bemust
Dr. Hanif Durad 56

Question 2
 Does the reweighting preserve shortest paths? Yes
 Consider any path
kvvvp ,,, 21 
Dr. Hanif Durad 57

Lemma 25.1 (Reweighting doesn’t change shortest paths)
Given a weighted directed graph G = (V, E) with weight function
w E R:  , let h V R:  be any function mapping vertices to real
numbers. For each edge ( , )u v E , define
( , ) ( , ) ( ) ( )w u v w u v h u h v   .
Let P v v vk 0 1, ,..., be a path from vertex v0 to vk . Then
w P v vk( ) ( , ) d 0 if and only if ( ) ( , )w P v vk d 0 .
Also, G has a negative-weight cycle using weight function w iff G has
a negative weight cycle using weight function w.
Preserving Shortest Paths by
Reweighting (1/4)
25_All-Pairs Shortest Paths.ppt
Dr. Hanif Durad 58

Proof.
 ( ) ( ) ( ) ( )w P w P h v h vk  0
( ) ( , )
( ( , ) ( ) ( ))
( , ) ( ) ( )
( ) ( ) ( )
w P w v v
w v v h v h v
w v v h v h v
w P h v h v
i i
i
k
i i
i
k
i i
i i
i
k
k
k
 
   
   
  







1
1
1
1
1
1
1
0
0
Reweighting (2/4)
Dr. Hanif Durad 59

 w P v vk( ) ( , ) d 0 implies ( ) ( , )w P v vk d 0 .
Suppose there is a shorter path P' from v0 to vk using the weight
function w. Then ˆ ˆ( ') ( )w P w P .
Then
w P h v h v w P
w P w P h v h v
w P w P
k
k
( ' ) ( ) ( ) ( ' )
( ) ( ) ( ) ( )
( ' ) ( ).
  
   
 
0
0
We get a contradiction!
Reweighting (3/4)
Dr. Hanif Durad 60

 G has a negative-weight cycle using w iff G has a
negative-weight cycle using w.
Consider any cycle 0 1, ,..., kc v v v  with v vk0  . Then
0
ˆ( ) ( ) ( ) ( ) ( )kw c w c h v h v w c    .
and thus c has negative weight using w if and only if it has
negative weight using
Preserving shortest paths by
reweighting (4/4)
Dr. Hanif Durad 61

Question 3
 How do we compute the d(s, v)’s?
 Use Bellman-Ford.
 This also tells us if we have a negative-weight
cycle.
Dr. Hanif Durad 62

Reweighting
Single
source
Johnson’s Algorithm
Dr. Hanif Durad 63

64
Johnson’s algorithm example
-4
-5
3
7
4
2
8
6
1
1
2
3
4
5
Fig. 25.1
Dr. Hanif Durad

3
7
4
2
8
-4
6
1
-5
s
0
0
0
0
0
Add a point s, and
assigning a weight
from s 0 to every
point edge.
2
1
3
4
5
Fig. 25.6 (a)
Dr. Hanif Durad 65

0
-1
0 -5
-4 0
4
10
0
2
13
0
2
0
s
5
1
0
4
0
Implementation of
the Bellman-Ford
algorithm, by
starting from s the
shortest distance
of each point.
2
1 3
4
5
Fig. 25.6 (b)
Dr. Hanif Durad 66

0
-1
0 -5
-4 0
4
10
0
2
13
2
0
0
After reweighting
1
2
3
45
Fig. 25.6 Extra
Dr. Hanif Durad 67

Implementation of the
Dijkstra |V| times
Shortest-paths tree=--
a/b= Calculated in
Reweighted/In Original
Graph
2/1
0/0 2/-3
0/-4 2/0
4
10
0
2
13
0
2
0 0
1
2
1
4
45
Fig. 25.6 (c)
Dr. Hanif Durad 68

0/0
2/3 0/-4
2/-1 0/1
4
10
0
2
13
0
2
0
0
2
1 3
45
Fig. 25.6 (d)
Dr. Hanif Durad 69

0/4
2/7 0/0
2/3 0/5
4
10
0
2
13
0
2
0
0
1
2
3
45
Fig. 25.6 (e)
Dr. Hanif Durad 70

0/-1
2/2 0/-5
2/-2 0/0
4
10
0
2
13
0
2
0
0
1
2
3
45
Fig. 25.6 (f)
Dr. Hanif Durad 71

2/5
4/8 2/1
0/0 2/6
4
10
0
2
13
0
2
0
0
1
2
3
45
Fig. 25.6 (g)
Dr. Hanif Durad 72

Johnson’s Algorithm Complexity
1. Find a vertex labeling h such that ŵ(u, v) ≥ 0 for all
(u, v)  E by using Bellman-Ford to solve the
difference constraints
h(v) – h(u) ≤ w(u, v),
or determine that a negative-weight cycle exists.
• Time = O(VE).
2. Run Dijkstra’s algorithm from each vertex using ŵ.
• Time = O(VE + V2 lg V).
3. Reweight each shortest-path length ŵ(p) to produce
the shortest-path lengths w(p) of the original graph.
• Time = O(V2).
Total time = O(VE + V2 lg V).
lecture19.ppt
Dr. Hanif Durad 73

Assignment 3
 Section Problem 25.3-6
 Chapter Problems 25.1
 C++ coding of
 Johnson’s Algorithm
 Last Date
Dr. Hanif Durad 74

75
Other Graph Algorithms (1/3)
 Many more interesting problems, including
network flow, graph isomorphism, coloring,
partition, etc.
 Problems can be classified by the type of solution.
 Easy problems: polynomial-time solutions O(f (n))
where f (n) is a polynomial function of degree at
most k.
 Hard problems: exponential-time solutions O(f (n))
where f (n) is an exponential function, usually 2n.
lect16.ppt,

76
Easy Graph Problems (2/3)
 Network flow – maximum flow problem
 Maximum bipartite matching
 Planarity testing and plane embedding.

77
Hard Graph Problems (3/3)
 Graph and sub-graph isomorphism.
 Largest clique, Independent set
 Vertex Tour (Traveling Salesman problem)
 Graph partition
 Vertex coloring
However, not all is lost!
 Good heuristics that perform well in most cases
 Polynomial-time Approximation algorithms

Chapter 26 aoa

More Related Content

What's hot

Similar to Chapter 26 aoa

More from Hanif Durad

Recently uploaded

Chapter 26 aoa