Mixed Integer Programming (MIP) is a type of mathematical optimization BranchAndBound.ppt

1
Branch and Bound
Searching Strategies

2
The personnel assignment
problem

A linearly ordered set of persons P={P1
, P2
, …, P
n
} where P1
<P2
<…<Pn

A partially ordered set of jobs J={J1
, J2
, …, Jn
}

Suppose that Pi
and Pj
are assigned to jobs f(Pi
) a
nd f(Pj
) respectively. If f(Pi
)  f(Pj
), then Pi
 Pj
. C
ost Cij
is the cost of assigning Pi
to Jj
. We want to
find a feasible assignment with the min. cost. i.e.

Minimize i,j
Cij
Xij
,
where Xij
= 1 if Pi
is assigned to Jj
and Xij
= 0 othe
rwise.

3
 E.g.
Fig. 6-21 A Partial Ordering of Jobs
 After topological sorting, one of the following
topologically sorted sequences will be generated:
 One of feasible assignments:
P1
→J1
, P2
→J2
, P3
→J3
, P4
→J4
J1
J2
↓ ↘ ↓
J3
J4
J1
, J2
, J3
, J4
J1
, J2
, J4
, J3
J1
, J3
, J2
, J4
J2
, J1
, J3
, J4
J2
, J1
, J4
J3
problem

4
problem
 Cost matrix:
Table 6-1 A Cost Matrix for a Personnel Assignment
Problem
Jobs
Persons
1 2 3 4
1 29 19 17 12
2 32 30 26 28
3 3 21 7 9
4 18 13 10 15

5
problem
 Reduced cost matrix:
subtract a constant from each row and each
column respectively such that each row and each
column contains at least one zero.
A Reduced Cost Matrix
Jobs
Persons
1 2 3 4
1 17 4 5 0 (-1
2)
2 6 1 0 2 (-2
6)
3 0 15 4 6 (-3)
4 8 0 0 5 (-1
0)
(-3)

6
problem
 Total cost subtracted: 12+26+3+10+3 = 54
 This is a lower bound of our solution.

7
problem
 Solution tree:
3
1 2
2
4
2
3 4
4 3
1
3 4
4 3
1
2
3
4
Person
Assigned
0

8
Steps to generate solution tress
 Step 1: Add the original partial ordering into stack.
 Step 2: Remove the top partial ordering O from st
ack. If O contains only one element E, then stop
(with the element E being an element in the topolo
gically sorted sequence). Otherwise, add the parti
al orderings into the stack each of which is derived
by removing an element E not preceded by any ot
her element in the partial ordering O. (Every E is a
n element in the topologically sorted sequence.)
 Step 3: Goto Step 1.

9
problem
 Bounding of subsolutions:

10
The traveling salesperson
optimization problem
 Given a graph, the TSP Optimization
problem is to find a tour, starting
from any vertex, visiting every other
vertex and returning to the starting
vertex, with minimal cost.
 It is NP-complete
 We try to avoid n! exhaustive search
by the branch-and-bound technique.

11
 E.g. A Cost Matrix for a Traveling Salesperson Problem.
j
i
1 2 3 4 5 6 7
1 ∞ 3 93 13 33 9 57
2 4 ∞ 77 42 21 16 34
3 45 17 ∞ 36 16 28 25
4 39 90 80 ∞ 56 7 91
5 28 46 88 33 ∞ 25 57
6 3 88 18 46 92 ∞ 7
7 44 26 33 27 84 39 ∞

12
The basic idea
 There is a way to split the solution space
(branch)
 There is a way to predict a lower bound for
a class of solutions. There is also a way to
find a upper bound of an optimal solution.
If the lower bound of a solution exceeds
the upper bound, this solution cannot be
optimal and thus we should terminate the
branching associated with this solution.

13
Splitting
 We split a solution into two groups:
 One group including a particular arc
 The other excluding the arc
 Each splitting incurs a lower bound
and we shall traverse the searching
tree with the “lower” lower bound.

14

Reduced cost matrix:
A Reduced Cost Matrix.
j
i
1 2 3 4 5 6 7
1 ∞ 0 90 10 30 6 54 (-3)
2 0 ∞ 73 38 17 12 30 (-4)
3 29 1 ∞ 20 0 12 9 (-16)
4 32 83 73 ∞ 49 0 84 (-7)
5 3 21 63 8 ∞ 0 32 (-25)
6 0 85 15 43 89 ∞ 4 (-3)
7 18 0 7 1 58 13 ∞ (-26)
reduced:84

15
Table 6-5 Another Reduced Cost Matrix.
j
i
1 2 3 4 5 6 7
1 ∞ 0 83 9 30 6 50
2 0 ∞ 66 37 17 12 26
3 29 1 ∞ 19 0 12 5
4 32 83 66 ∞ 49 0 80
5 3 21 56 7 ∞ 0 28
6 0 85 8 42 89 ∞ 0
7 18 0 0 0 58 13 ∞
(-7) (-1) (-4)

16
Lower bound
 The total cost of 84+12=96 is
subtracted. Thus, we know the lower
bound of this TSP problem is 96.

17
 Total cost reduced: 84+7+1+4 = 96 (lower bound)
decision tree:
The Highest Level of a Decision Tree.
 If we use arc 3-5 to split, the difference on the
lower bounds is 17+1 = 18.

18
For the right subtree
j
i
1 2 3 4 5 7
1 ∞ 0 83 9 30 50
2 0 ∞ 66 37 17 26
3 29 1 ∞ 19 0 5
5 3 21 56 7 ∞ 28
6 0 85 8 ∞ 89 0
7 18 0 0 0 58 ∞
A Reduced Cost Matrix if Arc 4-6 is Included.
(We must set c6-4 to be )

19
For the left subtree

The cost matrix for all solution with arc 4-6:
A Reduced Cost Matrix for that in Table 6-6.
 Total cost reduced: 96+3 = 99 (new lower bound)
j
i
1 2 3 4 5 7
1 ∞ 0 83 9 30 50
2 0 ∞ 66 37 17 26
3 29 1 ∞ 19 0 5
5 0 18 53 4 ∞ 25 (-
3)
6 0 85 8 ∞ 89 0
7 18 0 0 0 58 ∞

20
Upper bound
 We follow the best-first search scheme
and can obtain a feasible solution with c
ost C.
 C serves as an upper bound and many
branchings may be terminated if their lo
wer bounds exceed C.

21
Fig 6-26 A Branch-and-Bound Solution of a Traveling
Salesperson Problem.
1 2
3
5
6
7
4

22
Preventing an arc
 In general, if paths i1-i2-…-im and j1-j2-…-j
n have already been included and a path
from im to j1 is to be added, then path fro
m jn to i1 must be prevented
(by assigning the cost of jn to i1 to be )

23
The 0/1 knapsack problem
 Positive integer P1
, P2
, …, Pn
(profit)
W1
, W2
, …, Wn
(weight)
M (capacity)
m
a
x
im
iz
e PX
i i
i
n


1
s
u
b
je
c
tto W
X M
i i
i
n



1
X
i =0o
r1
,i=
1
,…
,n
.
T
h
ep
r
o
b
le
m
ism
o
d
if
ie
d
:
m
in
im
iz
e 


PX
i i
i
n
1

24
Fig. 6-27 The Branching Mechanism in the Branch-and-
Bound Strategy to Solve 0/1 Knapsack Problem.

25
How to find the upper
bound?
 Ans: by quickly finding a feasible solutio
n: starting from the smallest available i,
scanning towards the largest i’s until M i
s exceeded. The upper bound can be c
alculated.

26
 E.g. n = 6, M = 34
 A feasible solution: X1
= 1, X2
= 1, X3
= 0, X4
= 0,
X5
= 0, X6
= 0
-(P1
+P2
) = -16 (upper bound)
Any solution higher than -16 can not be an optimal solution.
i 1 2 3 4 5 6
Pi
6 10 4 5 6 4
Wi
10 19 8 10 12 8
(Pi
/Wi
 Pi+1
/Wi+1
)

27
How to find the lower bound?
 Ans: by relaxing our restriction from Xi
= 0 or 1 to
0  Xi
 1 (knapsack problem)
Let 

PX
i i
i
n
1
be an optim
al solution for 0/1
knapsack problem and  

PX
i
i
n
i
1
be an optim
al
solutionfor knapsackproblem
. LetY=

PX
i i
i
n
1
,
Y’
=  

PX
i
i
n
i
1
.
 Y’ 
Y

28
The knapsack problem
 We can use the greedy method to find an optimal
solution for knapsack problem.

For example, for the state of X1=1 and X2=1, we have
X1
= 1, X2
=1, X3
= (34-6-10)/8=5/8, X4
= 0, X5
= 0, X6
=0
-(P1
+P2
+5/8P3
) = -18.5 (lower bound)
-18 is our lower bound. (only consider integers)

29
How to expand the tree?
 By the best-first search scheme
 That is, by expanding the node with
the best lower bound. If two nodes
have the same lower bounds, expand
the node with the lower upper
bound.

30
0/1 Knapsack Problem Solved by Branch-and-Bound Strategy

31
 Node 2 is terminated because its lower
bound is equal to the upper bound of
node 14.
 Nodes 16, 18 and others are terminated
because the local lower bound is equal
to the local upper bound.
(lower bound  optimal solution 
upper bound)

32
The A*
algorithm
 Used to solve optimization problems.
 Using the best-first strategy.
 If a feasible solution (goal node) is obtained, then it is
optimal and we can stop.
 Estimated cost function of node n : f(n)
f(n) = g(n) + h(n)
g(n): cost from root to node n.
h(n): estimated cost from node n to a goal node.
h*
(n): “real” cost from node n to a goal node.
f*(n): “real” cost of node n
h(n)  h*(n)
 f(n) = g(n) + h(n)  g(n)+h*
(n) = f*
(n)

33
Reasoning
 Let t be the selected goal node. Let n de
note a node already expanded.
 f(t)f(n)
 f(t) f(n) f*(n)
 But one of f*(n)’s must be the optimal va
lue (cost). Let f*(s) denote the value.
 We have f(t) f*(s) …………(a)

34
Reasoning
 Since t is a goal node, we have h(t)=0.
Thus, f(t)=g(t)+h(t)=g(t).
 By Eq. (a), we have f(t)=g(t)f*(s)
 Yet, f(t)=g(t) is the value of a feasible so
lution. Consequently, g(t) cannot be sm
aller than f*(s) by definition. This means
that g(t)=f*(s).

35
The A*
algorithm
 Stop iff the selected node is also a goal node
Stop iff the selected node is also a goal node
 E.g.
Fig. 6-36 A Graph to Illustrate A*
Algorithm.

36
The A*
algorithm
 Step 1.
g(A)=2 h(A)=min{2,3}=2 f(A)=2+2=4
g(B)=4 h(B)=min{2}=2 f(B)=4+2=6
g(C)=3 h(C)=min{2,2}=2 f(C)= 3+2=5

37
The A*
algorithm
 Step 2. Expand A
g(D)=2+2=4 h(D)=min{3,1}=1 f(D)=4+1=5
g(E)=2+3=5 h(E)=min{2,2}=2 f(E)=5+2=7

38
The A*
algorithm
 Step 3. Expand C
g(F)=3+2=5 h(F)=min{3,1}=1 f(F)=5+1=6
g(G) =3+2=5 h(G)=min{5}=5 f(G) =5+5=10

39
The A*
algorithm
 Step 4. Expand D
g(H)=2+2+1=5 h(H)=min{5}=5 f(H)=5+5=10
g(I)=2+2+3=7 h(I)=0 f(I)=7+0=7

40
The A*
algorithm
 Step 5. Expand B
g(J)=4+2=6 h(J)=min{5}=5 f(J)=6+5=11

41
The A*
algorithm
 Step 6. Expand F
g(K)=3+2+1=6 h(K)=min{5}=5 f(K)=6+5=11
g(L)=3+2+3=8 h(L)=0 f(L)=8+0=8
f(n)

f*(n)
=
I is a goal node !

42
The A* Algorithm
 Can be considered as a special type
of branch-and-bound algorithm.

Mixed Integer Programming (MIP) is a type of mathematical optimization BranchAndBound.ppt

More Related Content

Similar to Mixed Integer Programming (MIP) is a type of mathematical optimization BranchAndBound.ppt

Recently uploaded

Mixed Integer Programming (MIP) is a type of mathematical optimization BranchAndBound.ppt