3. Recursion and Recurrences.ppt detail about recursive learning

Design and Analysis of Algorithms
Dr. Naveed Ejaz

Recursion
• Basic problem solving technique is to
divide a problem into smaller subproblems
• These subproblems may also be divided
into smaller subproblems
• When the subproblems are small enough
to solve directly the process stops
• A recursive algorithm is a problem
solution that has been expressed in terms
of two or more easier to solve
subproblems

Recursion occurs when a function/procedure calls itself.
Many algorithms can be best described in terms of recursion.
Example: Factorial function
The product of the positive integers from 1 to n inclusive is
called "n factorial", usually denoted by n!:
n! = 1 * 2 * 3 .... (n-2) * (n-1) * n
Recursion
Recursion

Recursive Definition
of the Factorial Function
n! =
1, if n = 0
n * (n-1)! if n > 0
5! = 5 * 4!
4! = 4 * 3!
3! = 3 * 2!
2! = 2 * 1!
1! = 1 * 0!
= 5 * 24 = 120
= 4 * 3! = 4 * 6 = 24
= 3 * 2! = 3 * 2 = 6
= 2 * 1! = 2 * 1 = 2
= 1 * 0! = 1

The Fibonacci numbers are a series of numbers as follows:
fib(1) = 1
fib(2) = 1
fib(3) = 2
fib(4) = 3
fib(5) = 5
...
fib(n) =
1, n <= 2
fib(n-1) + fib(n-2), n > 2
of the Fibonacci Numbers
fib(3) = 1 + 1 = 2
fib(4) = 2 + 1 = 3
fib(5) = 2 + 3 = 5

int BadFactorial(n){
int x = BadFactorial(n-1);
if (n == 1)
return 1;
else
return n*x;
}
What is the value of BadFactorial(2)?
We must make sure that recursion eventually stops, otherwise
it runs forever:

Using Recursion Properly
Using Recursion Properly
For correct recursion we need two parts:
1. One (ore more) base cases that are not recursive, i.e. we
can directly give a solution:
if (n==1)
return 1;
2. One (or more) recursive cases that operate on smaller
problems that get closer to the base case(s)
return n * factorial(n-1);
The base case(s) should always be checked before the recursive
calls.

Counting Digits
• Recursive definition
digits(n) = 1 if (–9 <= n <= 9)
1 + digits(n/10) otherwise
• Example
digits(321) =
1 + digits(321/10) = 1 +digits(32) =
1 + [1 + digits(32/10)] = 1 + [1 + digits(3)] =
1 + [1 + (1)] =
3

Counting Digits in C++
int numberofDigits(int n) {
if ((-10 < n) && (n < 10))
return 1;
else
return 1 +
numberofDigits(n/10);
}

Evaluating Exponents Recursively
int power(int k, int n) {
// raise k to the power n
if (n == 0)
return 1;
else
return k * power(k, n – 1);
}

Divide and Conquer
• Using this method each recursive
subproblem is about one-half the size of
the original problem
• If we could define power so that each
subproblem was based on computing kn/2
instead of kn – 1
we could use the divide and
conquer principle
• Recursive divide and conquer algorithms
are often more efficient than iterative
algorithms

Evaluating Exponents Using Divide
and Conquer
int power(int k, int n) {
// raise k to the power n
if (n == 0)
return 1;
else{
int t = power(k, n/2);
if ((n % 2) == 0)
return t * t;
else
return k * t * t;
}

Disadvantages
• May run slower.
• Compilers
• Inefficient Code
• May use more space.

Advantages
• More natural.
• Easier to prove correct.
• Easier to analyze.
• More flexible.

Recurrences
• The expression:
is a recurrence.
• Recurrence: an equation that describes a
function in terms of its value on smaller
functions

















1
2
2
1
)
(
n
cn
n
T
n
c
n
T

Recurrence Examples








0
0
)
1
(
0
)
(
n
n
n
s
c
n
s








0
)
1
(
0
0
)
(
n
n
s
n
n
n
s

















1
2
2
1
)
(
n
c
n
T
n
c
n
T

















1
1
)
(
n
cn
b
n
aT
n
c
n
T

Methods to solve recurrence
• Substitution method
• Iteration method
• Recursion tree method
• Master Theorem

Solving Recurrences
• The substitution method
• A.k.a. “making a good guess method”
• Guess the form of the answer, then use
mathematical induction to find the constants
and show that the solution works
• Example:
• T(n) = 2T(n/2) + n  T(n) = O(n lg n)

Solving Recurrences
• Another option is “iteration method”
• Expand the recurrence
• Work some algebra to express as a summation
• Evaluate the summation
• We will show several examples

• s(n) =
c + s(n-1)
c + c + s(n-2)
2c + s(n-2)
2c + c + s(n-3)
3c + s(n-3)
…
kc + s(n-k) = ck + s(n-k)








0
)
1
(
0
0
)
(
n
n
s
c
n
n
s

• So far for n >= k we have
• s(n) = ck + s(n-k)
• What if k = n?
• s(n) = cn + s(0) = cn








0
)
1
(
0
0
)
(
n
n
s
c
n
n
s

• s(n) = ck + s(n-k)
• What if k = n?
• s(n) = cn + s(0) = cn
• So
• Thus in general
• s(n) = cn








0
)
1
(
0
0
)
(
n
n
s
c
n
n
s








0
)
1
(
0
0
)
(
n
n
s
c
n
n
s

• s(n)
= n + s(n-1)
= n + n-1 + s(n-2)
= n + n-1 + n-2 + s(n-3)
= n + n-1 + n-2 + n-3 + s(n-4)
= …
= n + n-1 + n-2 + n-3 + … + n-(k-1) + s(n-k)








0
)
1
(
0
0
)
(
n
n
s
n
n
n
s

• s(n)
= n + s(n-1)
= n + n-1 + s(n-2)
= n + n-1 + n-2 + s(n-3)
= n + n-1 + n-2 + n-3 + s(n-4)
= …
= n + n-1 + n-2 + n-3 + … + n-(k-1) + s(n-k)








0
)
1
(
0
0
)
(
n
n
s
n
n
n
s
)
(
1
k
n
s
i
n
k
n
i














0
)
1
(
0
0
)
(
n
n
s
n
n
n
s
)
(
1
k
n
s
i
n
k
n
i






• What if k = n?








0
)
1
(
0
0
)
(
n
n
s
n
n
n
s
)
(
1
k
n
s
i
n
k
n
i






• What if k = n?








0
)
1
(
0
0
)
(
n
n
s
n
n
n
s
)
(
1
k
n
s
i
n
k
n
i





2
1
0
)
0
(
1
1




 
 

n
n
i
s
i
n
i
n
i

• What if k = n?
• Thus in general








0
)
1
(
0
0
)
(
n
n
s
n
n
n
s
)
(
1
k
n
s
i
n
k
n
i





2
1
0
)
0
(
1
1




 
 

n
n
i
s
i
n
i
n
i
2
1
)
(


n
n
n
s

Divide and Conquer
The divide-and-conquer paradigm
What is a paradigm? One that serves as a pattern or model.
• Divide the problem into a number of subproblems
• Conquer the subproblems by solving them
recursively. If small enough, just solve directly
without recursion
• Combine the solutions of the subproblems into the
solution for the original problem

Let T(n) be the running time on a problem of size n.
If problem is small enough, then solution takes
constant time (this is a boundary condition, which
will be assumed for all analyses)
It takes time to divide the problem into sub-
problems at the beginning. Denote this work by
D(n).
Analyzing Divide-and-Conquer
Algorithms

It takes time to combine the solutions at the end. Denote this
by C(n).
If we divide the problem into ‘a’ problems, each of which is
‘1/b’ times the original size (n), and since the time to solve
a problem with input size ‘n/b’ is T(n/b), and this is done ‘a’
times, the total time is:
T(n) = (1), n small (or n  c )
T(n) = aT(n/b) + D(n) + C(n)
Analyzing Divide-and-Conquer
Algorithms

Example
Function(int number)
if number<=1
then return;
else
Function(number/2)
Function(number/2)
for(i=1 to number )
Display i;
It follows that
T(n) = 2 T(n/2)+ cn
number of sub-
problems
Sub-problem size
work done
dividing and
combining

Recursion Tree for Algorithm
cn
T(n/2) T(n/2)

cn
cn/2
T(n/4) T(n/4)
cn/2
T(n/4) T(n/4)

cn
cn/2
T(n/4) T(n/4)
cn/2
T(n/4) T(n/4)
Eventually, the input size (the argument of T) goes to 1, so...

cn
cn/2
T(n/4) T(n/4)
cn/2
T(n/4) T(n/4)
T(1) T(1) T(1) T(1) T(1) T(1)
.............................
n sub problems of size 1, but T(1) = c
by boundary condition

cn
cn/2
T(n/4) T(n/4)
cn/2
T(n/4) T(n/4)
c c c c c c
.............................
n subproblems of size 1

level nodes/ cost/
level level
0 20
= 1 cn
1 21
= 2 cn
2 22
= 4 cn
. .
. .
. .
N-1 2N-1
=n cn
Since 2N-1
= n,
N-1 = lg(n)
levels = N = 1+lg(n)
T(n) = total cost = (levels)(cost/level)
T(n) = cn [1+lg(n)] = O( n lg(n))
n nodes at level N-1

The Master Theorem
• Given: a divide and conquer algorithm
• An algorithm that divides the problem of size n into
“a” subproblems, each of size “n/b”
• Let the cost of each stage (i.e., the work to divide
the problem + combine solved subproblems) be
described by the function f(n)
• Then, the Master Theorem gives us a cookbook for the
algorithm’s running time:

The Master Theorem
• if T(n) = aT(n/b) + f(n) then
 
 
 
 
 
 






































1
0
large
for
)
(
)
/
(
AND
)
(
)
(
)
(
)
(
log
)
(
log
log
log
log
log
c
n
n
cf
b
n
af
n
n
f
n
n
f
n
O
n
f
n
f
n
n
n
n
T
a
a
a
a
a
b
b
b
b
b




Using The Master Method
• T(n) = 9T(n/3) + n
• a=9, b=3, f(n) = n
• nlogb a
= nlog3 9
= (n2
)
• Since f(n) = O(nlog3 9 - 
), where =1, case 1
applies:
• Thus the solution is T(n) = (n2
)
   




 a
a b
b
n
O
n
f
n
n
T log
log
)
(
when
)
(

3. Recursion and Recurrences.ppt detail about recursive learning

More Related Content

Similar to 3. Recursion and Recurrences.ppt detail about recursive learning

Recently uploaded

3. Recursion and Recurrences.ppt detail about recursive learning