Algorithms
AVL Tree
Balanced binary tree
● The disadvantage of a binary search tree is that its height can
be as large as N-1
● This means that the time needed to perform insertion and
deletion and many other operations can be O(N) in the worst
case
● We want a tree with small height
● A binary tree with N node has height at least (log N)
● Thus, our goal is to keep the height of a binary search tree
O(log N)
● Such trees are called balanced binary search trees. Examples
are AVL tree, red-black tree.
Binary Search Tree - Best Time
● All BST operations are O(h), where d is tree
depth
● minimum d is for a binary tree
with N nodes
■ What is the best case tree?
■ What is the worst case tree?
● So, best case running time of BST operations
is O(log N)
2h log N
Binary Search Tree - Worst Time
● Worst case running time is O(N)
■ What happens when you Insert elements in
ascending order?
○ Insert: 2, 4, 6, 8, 10, 12 into an empty BST
■ Problem: Lack of “balance”:
○ compare depths of left and right subtree
■ Unbalanced degenerate tree
Balanced and unbalanced BST
4
2 5
1 3
1
5
2
4
3
7
6
4
2 6
5 71 3
Is this “balanced”?
Approaches to balancing trees
● Don't balance
■ May end up with some nodes very deep
● Strict balance
■ The tree must always be balanced perfectly
● Pretty good balance
■ Only allow a little out of balance
● Adjust on access
■ Self-adjusting
Balancing Binary Search Trees
● Many algorithms exist for keeping binary
search trees balanced
■ Adelson-Velskii and Landis (AVL) trees
(height-balanced trees)
■ Splay trees and other self-adjusting trees
■ B-trees and other multiway search trees
AVL Tree is…
● Named after Adelson-Velskii and Landis
● the first dynamically balanced trees to be
propose
● Binary search tree with balance condition in
which the sub-trees of each node can differ by
at most 1 in their height
Definition of a balanced tree
● Ensure the depth = O(log N)
● Take O(log N) time for searching, insertion,
and deletion
● Every node must have left & right sub-trees of
the same height
An AVL tree has the following
properties:
1. Sub-trees of each
node can differ by
at most 1 in their
height
2. Every sub-trees is
an AVL tree
AVL tree?
YES
Each left sub-tree has
height 1 greater than each
right sub-tree
NO
Left sub-tree has height 3,
but right sub-tree has height
1
AVL tree
Height of a node
● The height of a leaf is 1. The height of a null
pointer is zero.
● The height of an internal node is the maximum
height of its children plus 1
Note that this definition of height is different from the one we
defined previously (we defined the height of a leaf as zero
previously).
AVL Trees
10
5
3
20
2
1 3
10
5
3
20
1
43
5
AVL Trees
12
8 16
4 10
2 6
14
AVL Tree
-1
0
00
0
0
-1
00
1
-2
1
-10
0
AVL Tree
AVL Tree
0 Not an AVL Tree
Height of an AVL Tree
● Fact: The height of an AVL tree storing n keys is O(log n).
● Proof: Let us bound n(h): the minimum number of internal nodes
of an AVL tree of height h.
● We easily see that n(1) = 1 and n(2) = 2
● For n > 2, an AVL tree of height h contains the root node, one AVL
subtree of height n-1 and another of height n-2.
● That is, n(h) = 1 + n(h-1) + n(h-2)
● Knowing n(h-1) > n(h-2), we get n(h) > 2n(h-2). So
n(h) > 2n(h-2), n(h) > 4n(h-4), n(h) > 8n(n-6), … (by induction),
n(h) > 2in(h-2i)
● Solving the base case we get: n(h) > 2 h/2-1
● Taking logarithms: h < 2log n(h) +2
● Thus the height of an AVL tree is O(log n)
3
4 n(1)
n(2)
AVL - Good but not Perfect
Balance
● AVL trees are height-balanced binary search
trees
● Balance factor of a node
■ height(left subtree) - height(right subtree)
● An AVL tree has balance factor calculated at
every node
■ For every node, heights of left and right subtree
can differ by no more than 1
■ Store current heights in each node
Height of an AVL Tree
● N(h) = minimum number of nodes in an AVL
tree of height h.
● Basis
■ N(0) = 1, N(1) = 2
● Induction
■ N(h) = N(h-1) + N(h-2) + 1
● Solution (recall Fibonacci analysis)
■ N(h) > h ( 1.62)
h-1
h-2
h
Height of an AVL Tree
● N(h) > h ( 1.62)
● Suppose we have n nodes in an AVL tree of
height h.
■ n > N(h) (because N(h) was the minimum)
■ n > h hence log n > h (relatively well balanced
tree!!)
■ h < 1.44 log2n (i.e., Find takes O(log n))
Insertion
Insert 6
Imbalance at 8
Perform rotation with 7
Deletion
Delete 4
Imbalance at 3
Perform rotation with 2
Imbalance at 5
Perform rotation with 8
Key Points
● AVL tree remain balanced by applying
rotations, therefore it guarantees O(log N)
search time in a dynamic environment
● Tree can be re-balanced in at most O(log N)
time
Searching AVL Trees
● Searching an AVL tree is exactly the same as
searching a regular binary tree
■ all descendants to the right of a node are greater
than the node
■ all descendants to the left of a node are less than
the node
Inserting in AVL Tree
● Insertion is similar to regular binary tree
■ keep going left (or right) in the tree until a null
child is reached
■ insert a new node in this position
○ an inserted node is always a leaf to start with
● Major difference from binary tree
■ must check if any of the sub-trees in the tree have
become too unbalanced
○ search from inserted node to root looking for any node
with a balance factor of 2
Inserting in AVL Tree
● A few points about tree inserts
■ the insert will be done recursively
■ the insert call will return true if the height of the
sub-tree has changed
○ since we are doing an insert, the height of the sub-tree
can only increase
■ if insert() returns true, balance factor of current
node needs to be adjusted
○ balance factor = height(right) – height(left)
 left sub-tree increases, balance factor decreases by 1
 right sub-tree increases, balance factor increases by 1
■ if balance factor equals 2 for any node, the sub-
tree must be rebalanced
Inserting in AVL Tree
M(-1)
insert(V)E(1)
J(0)
P(0)
M(0)
E(1)
J(0)
P(1)
V(0)
M(-1)
insert(L)E(1)
J(0)
P(0)
M(-2)
E(-2)
J(1)
P(0)
L(0)
This tree needs to be fixed!
Re-Balancing a Tree
● To check if a tree needs to be rebalanced
■ start at the parent of the inserted node and journey
up the tree to the root
○ if a node’s balance factor becomes 2 need to do a
rotation in the sub-tree rooted at the node
○ once sub-tree has been re-balanced, guaranteed that the
rest of the tree is balanced as well
 can just return false from the insert() method
■ 4 possible cases for re-balancing
○ only 2 of them need to be considered
 other 2 are identical but in the opposite direction
Let the node that needs rebalancing be .
There are 4 cases:
Outside Cases (require single rotation) :
1. Insertion into left subtree of left child of .
2. Insertion into right subtree of right child of .
Inside Cases (require double rotation) :
3. Insertion into right subtree of left child of .
4. Insertion into left subtree of right child of .
The rebalancing is performed through four separate
rotation algorithms.
Insertions in AVL Trees
j
k
X Y
Z
Consider a valid
AVL subtree
AVL Insertion: Outside Case
h
h
h
j
k
X
Y
Z
Inserting into X
destroys the AVL
property at node j
AVL Insertion: Outside Case
h
h+1 h
j
k
X
Y
Z
Do a “right rotation”
AVL Insertion: Outside Case
h
h+1 h
j
k
X
Y
Z
Do a “right rotation”
Single right rotation
h
h+1 h
j
k
X Y Z
“Right rotation” done!
(“Left rotation” is mirror
symmetric)
Outside Case Completed
AVL property has been restored!
h
h+1
h
j
k
X Y
Z
AVL Insertion: Inside Case
Consider a valid
AVL subtree
h
hh
Inserting into Y
destroys the
AVL property
at node j
j
k
X
Y
Z
AVL Insertion: Inside Case
Does “right rotation”
restore balance?
h
h+1h
j
k
X
Y
Z
“Right rotation”
does not restore
balance… now k is
out of balance
AVL Insertion: Inside Case
h
h+1
h
Consider the structure
of subtree Y… j
k
X
Y
Z
AVL Insertion: Inside Case
h
h+1h
j
k
X
V
Z
W
i
Y = node i and
subtrees V and W
AVL Insertion: Inside Case
h
h+1h
h or h-1
j
k
X
V
Z
W
i
AVL Insertion: Inside Case
We will do a left-right
“double rotation” . . .
j
k
X V
Z
W
i
Double rotation : first rotation
left rotation complete
j
k
X V
Z
W
i
Double rotation : second
rotation
Now do a right rotation
jk
X V ZW
i
Double rotation : second
rotation
right rotation complete
Balance has been
restored
hh h or h-1
AVL Trees Example
AVL Trees Example
AVL Trees Example
AVL Trees Example
AVL Trees Example
AVL Trees Example
5/22/2012
3
Example
● Insert 3 into the AVL tree
11
8 20
4 16 27
8
8
11
4 20
3 16 27
5/22/2012
Example
● Insert 5 into the AVL tree
5
11
8 20
4 16 27 8
11
5 20
4 16 27
8
5/22/2012
AVL Trees: Exercise
● Insertion order:
■ 10, 85, 15, 70, 20, 60, 30, 50, 65, 80, 90, 40, 5, 55
5/22/2012
Deletion X in AVL Trees
● Deletion:
■ Case 1: if X is a leaf, delete X
■ Case 2: if X has 1 child, use it to replace X
■ Case 3: if X has 2 children, replace X with its
inorder predecessor (and recursively delete it)
● Rebalancing
5/22/2012
Delete 55 (case 1)
60
20 70
10 40 65 85
5 15 30 50 80 90
55
5/22/2012
Delete 55 (case 1)
60
20 70
10 40 65 85
5 15 30 50 80 90
55
5/22/2012
Delete 50 (case 2)
60
20 70
10 40 65 85
5 15 30 50 80 90
55
5/22/2012
Delete 50 (case 2)
60
20 70
10 40 65 85
5 15 30
50 80 90
55
5/22/2012
Delete 60 (case 3)
60
20 70
10 40 65 85
5 15 30 50 80 90
55
prev
5/22/2012
Delete 60 (case 3)
55
20 70
10 40 65 85
5 15 30 50 80 90
5/22/2012
Delete 55 (case 3)
55
20 70
10 40 65 85
5 15 30 50 80 90
prev
5/22/2012
Delete 55 (case 3)
50
20 70
10 40 65 85
5 15 30 80 90
5/22/2012
Delete 50 (case 3)
50
20 70
10 40 65 85
5 15 30 80 90
prev
5/22/2012
Delete 50 (case 3)
40
20 70
10 30 65 85
5 15 80 90
5/22/2012
Delete 40 (case 3)
40
20 70
10 30 65 85
5 15 80 90
prev
5/22/2012
Delete 40 : Rebalancing
30
20 70
10 65 85
5 15 80 90
Case ?
5/22/2012
Delete 40: after rebalancing
30
7010
20 65 855
15 80 90
Single rotation is preferred!
5/22/2012
AVL Tree: analysis
● The depth of AVL Trees is at most logarithmic.
● So, all of the operations on AVL trees are also
logarithmic.
● The worst-case height is at most 44 percent
more than the minimum possible for binary
trees.

AVL Tree

  • 1.
  • 2.
    Balanced binary tree ●The disadvantage of a binary search tree is that its height can be as large as N-1 ● This means that the time needed to perform insertion and deletion and many other operations can be O(N) in the worst case ● We want a tree with small height ● A binary tree with N node has height at least (log N) ● Thus, our goal is to keep the height of a binary search tree O(log N) ● Such trees are called balanced binary search trees. Examples are AVL tree, red-black tree.
  • 3.
    Binary Search Tree- Best Time ● All BST operations are O(h), where d is tree depth ● minimum d is for a binary tree with N nodes ■ What is the best case tree? ■ What is the worst case tree? ● So, best case running time of BST operations is O(log N) 2h log N
  • 4.
    Binary Search Tree- Worst Time ● Worst case running time is O(N) ■ What happens when you Insert elements in ascending order? ○ Insert: 2, 4, 6, 8, 10, 12 into an empty BST ■ Problem: Lack of “balance”: ○ compare depths of left and right subtree ■ Unbalanced degenerate tree
  • 5.
    Balanced and unbalancedBST 4 2 5 1 3 1 5 2 4 3 7 6 4 2 6 5 71 3 Is this “balanced”?
  • 6.
    Approaches to balancingtrees ● Don't balance ■ May end up with some nodes very deep ● Strict balance ■ The tree must always be balanced perfectly ● Pretty good balance ■ Only allow a little out of balance ● Adjust on access ■ Self-adjusting
  • 7.
    Balancing Binary SearchTrees ● Many algorithms exist for keeping binary search trees balanced ■ Adelson-Velskii and Landis (AVL) trees (height-balanced trees) ■ Splay trees and other self-adjusting trees ■ B-trees and other multiway search trees
  • 8.
    AVL Tree is… ●Named after Adelson-Velskii and Landis ● the first dynamically balanced trees to be propose ● Binary search tree with balance condition in which the sub-trees of each node can differ by at most 1 in their height
  • 9.
    Definition of abalanced tree ● Ensure the depth = O(log N) ● Take O(log N) time for searching, insertion, and deletion ● Every node must have left & right sub-trees of the same height
  • 10.
    An AVL treehas the following properties: 1. Sub-trees of each node can differ by at most 1 in their height 2. Every sub-trees is an AVL tree
  • 11.
    AVL tree? YES Each leftsub-tree has height 1 greater than each right sub-tree NO Left sub-tree has height 3, but right sub-tree has height 1
  • 12.
    AVL tree Height ofa node ● The height of a leaf is 1. The height of a null pointer is zero. ● The height of an internal node is the maximum height of its children plus 1 Note that this definition of height is different from the one we defined previously (we defined the height of a leaf as zero previously).
  • 13.
  • 14.
  • 15.
  • 16.
    Height of anAVL Tree ● Fact: The height of an AVL tree storing n keys is O(log n). ● Proof: Let us bound n(h): the minimum number of internal nodes of an AVL tree of height h. ● We easily see that n(1) = 1 and n(2) = 2 ● For n > 2, an AVL tree of height h contains the root node, one AVL subtree of height n-1 and another of height n-2. ● That is, n(h) = 1 + n(h-1) + n(h-2) ● Knowing n(h-1) > n(h-2), we get n(h) > 2n(h-2). So n(h) > 2n(h-2), n(h) > 4n(h-4), n(h) > 8n(n-6), … (by induction), n(h) > 2in(h-2i) ● Solving the base case we get: n(h) > 2 h/2-1 ● Taking logarithms: h < 2log n(h) +2 ● Thus the height of an AVL tree is O(log n) 3 4 n(1) n(2)
  • 17.
    AVL - Goodbut not Perfect Balance ● AVL trees are height-balanced binary search trees ● Balance factor of a node ■ height(left subtree) - height(right subtree) ● An AVL tree has balance factor calculated at every node ■ For every node, heights of left and right subtree can differ by no more than 1 ■ Store current heights in each node
  • 18.
    Height of anAVL Tree ● N(h) = minimum number of nodes in an AVL tree of height h. ● Basis ■ N(0) = 1, N(1) = 2 ● Induction ■ N(h) = N(h-1) + N(h-2) + 1 ● Solution (recall Fibonacci analysis) ■ N(h) > h ( 1.62) h-1 h-2 h
  • 19.
    Height of anAVL Tree ● N(h) > h ( 1.62) ● Suppose we have n nodes in an AVL tree of height h. ■ n > N(h) (because N(h) was the minimum) ■ n > h hence log n > h (relatively well balanced tree!!) ■ h < 1.44 log2n (i.e., Find takes O(log n))
  • 20.
    Insertion Insert 6 Imbalance at8 Perform rotation with 7
  • 21.
    Deletion Delete 4 Imbalance at3 Perform rotation with 2 Imbalance at 5 Perform rotation with 8
  • 22.
    Key Points ● AVLtree remain balanced by applying rotations, therefore it guarantees O(log N) search time in a dynamic environment ● Tree can be re-balanced in at most O(log N) time
  • 23.
    Searching AVL Trees ●Searching an AVL tree is exactly the same as searching a regular binary tree ■ all descendants to the right of a node are greater than the node ■ all descendants to the left of a node are less than the node
  • 24.
    Inserting in AVLTree ● Insertion is similar to regular binary tree ■ keep going left (or right) in the tree until a null child is reached ■ insert a new node in this position ○ an inserted node is always a leaf to start with ● Major difference from binary tree ■ must check if any of the sub-trees in the tree have become too unbalanced ○ search from inserted node to root looking for any node with a balance factor of 2
  • 25.
    Inserting in AVLTree ● A few points about tree inserts ■ the insert will be done recursively ■ the insert call will return true if the height of the sub-tree has changed ○ since we are doing an insert, the height of the sub-tree can only increase ■ if insert() returns true, balance factor of current node needs to be adjusted ○ balance factor = height(right) – height(left)  left sub-tree increases, balance factor decreases by 1  right sub-tree increases, balance factor increases by 1 ■ if balance factor equals 2 for any node, the sub- tree must be rebalanced
  • 26.
    Inserting in AVLTree M(-1) insert(V)E(1) J(0) P(0) M(0) E(1) J(0) P(1) V(0) M(-1) insert(L)E(1) J(0) P(0) M(-2) E(-2) J(1) P(0) L(0) This tree needs to be fixed!
  • 27.
    Re-Balancing a Tree ●To check if a tree needs to be rebalanced ■ start at the parent of the inserted node and journey up the tree to the root ○ if a node’s balance factor becomes 2 need to do a rotation in the sub-tree rooted at the node ○ once sub-tree has been re-balanced, guaranteed that the rest of the tree is balanced as well  can just return false from the insert() method ■ 4 possible cases for re-balancing ○ only 2 of them need to be considered  other 2 are identical but in the opposite direction
  • 28.
    Let the nodethat needs rebalancing be . There are 4 cases: Outside Cases (require single rotation) : 1. Insertion into left subtree of left child of . 2. Insertion into right subtree of right child of . Inside Cases (require double rotation) : 3. Insertion into right subtree of left child of . 4. Insertion into left subtree of right child of . The rebalancing is performed through four separate rotation algorithms. Insertions in AVL Trees
  • 29.
    j k X Y Z Consider avalid AVL subtree AVL Insertion: Outside Case h h h
  • 30.
    j k X Y Z Inserting into X destroysthe AVL property at node j AVL Insertion: Outside Case h h+1 h
  • 31.
    j k X Y Z Do a “rightrotation” AVL Insertion: Outside Case h h+1 h
  • 32.
    j k X Y Z Do a “rightrotation” Single right rotation h h+1 h
  • 33.
    j k X Y Z “Rightrotation” done! (“Left rotation” is mirror symmetric) Outside Case Completed AVL property has been restored! h h+1 h
  • 34.
    j k X Y Z AVL Insertion:Inside Case Consider a valid AVL subtree h hh
  • 35.
    Inserting into Y destroysthe AVL property at node j j k X Y Z AVL Insertion: Inside Case Does “right rotation” restore balance? h h+1h
  • 36.
    j k X Y Z “Right rotation” does notrestore balance… now k is out of balance AVL Insertion: Inside Case h h+1 h
  • 37.
    Consider the structure ofsubtree Y… j k X Y Z AVL Insertion: Inside Case h h+1h
  • 38.
    j k X V Z W i Y = nodei and subtrees V and W AVL Insertion: Inside Case h h+1h h or h-1
  • 39.
    j k X V Z W i AVL Insertion: InsideCase We will do a left-right “double rotation” . . .
  • 40.
    j k X V Z W i Double rotation: first rotation left rotation complete
  • 41.
    j k X V Z W i Double rotation: second rotation Now do a right rotation
  • 42.
    jk X V ZW i Doublerotation : second rotation right rotation complete Balance has been restored hh h or h-1
  • 43.
  • 44.
  • 45.
  • 46.
  • 47.
  • 48.
  • 49.
    5/22/2012 3 Example ● Insert 3into the AVL tree 11 8 20 4 16 27 8 8 11 4 20 3 16 27
  • 50.
    5/22/2012 Example ● Insert 5into the AVL tree 5 11 8 20 4 16 27 8 11 5 20 4 16 27 8
  • 51.
    5/22/2012 AVL Trees: Exercise ●Insertion order: ■ 10, 85, 15, 70, 20, 60, 30, 50, 65, 80, 90, 40, 5, 55
  • 52.
    5/22/2012 Deletion X inAVL Trees ● Deletion: ■ Case 1: if X is a leaf, delete X ■ Case 2: if X has 1 child, use it to replace X ■ Case 3: if X has 2 children, replace X with its inorder predecessor (and recursively delete it) ● Rebalancing
  • 53.
    5/22/2012 Delete 55 (case1) 60 20 70 10 40 65 85 5 15 30 50 80 90 55
  • 54.
    5/22/2012 Delete 55 (case1) 60 20 70 10 40 65 85 5 15 30 50 80 90 55
  • 55.
    5/22/2012 Delete 50 (case2) 60 20 70 10 40 65 85 5 15 30 50 80 90 55
  • 56.
    5/22/2012 Delete 50 (case2) 60 20 70 10 40 65 85 5 15 30 50 80 90 55
  • 57.
    5/22/2012 Delete 60 (case3) 60 20 70 10 40 65 85 5 15 30 50 80 90 55 prev
  • 58.
    5/22/2012 Delete 60 (case3) 55 20 70 10 40 65 85 5 15 30 50 80 90
  • 59.
    5/22/2012 Delete 55 (case3) 55 20 70 10 40 65 85 5 15 30 50 80 90 prev
  • 60.
    5/22/2012 Delete 55 (case3) 50 20 70 10 40 65 85 5 15 30 80 90
  • 61.
    5/22/2012 Delete 50 (case3) 50 20 70 10 40 65 85 5 15 30 80 90 prev
  • 62.
    5/22/2012 Delete 50 (case3) 40 20 70 10 30 65 85 5 15 80 90
  • 63.
    5/22/2012 Delete 40 (case3) 40 20 70 10 30 65 85 5 15 80 90 prev
  • 64.
    5/22/2012 Delete 40 :Rebalancing 30 20 70 10 65 85 5 15 80 90 Case ?
  • 65.
    5/22/2012 Delete 40: afterrebalancing 30 7010 20 65 855 15 80 90 Single rotation is preferred!
  • 66.
    5/22/2012 AVL Tree: analysis ●The depth of AVL Trees is at most logarithmic. ● So, all of the operations on AVL trees are also logarithmic. ● The worst-case height is at most 44 percent more than the minimum possible for binary trees.