Sorting Algorithms

Sorting
Prepared by: Afaq Mansoor Khan
BSSE III- Group A
Session 2017-21
IMSciences, Peshawar.

Last Lecture Summary
• Recursion and Types, Space and Time Complexity
• Introduction to Sorting Algorithms
• Bubble Sort Algorithm, Algorithm Analysis

Objectives Overview
• Selection Sort,
• Insertion Sort,
• Algorithms Analysis
• Merge Sort Algorithm,
• Merge Sort Analysis

Selection Sort
• It is specifically an in-place comparison sort
• Noted for its simplicity,
• It has performance advantages over more complicated
algorithms in certain situations, particularly where
auxiliary memory is limited
• The algorithm
▫ finds the minimum value,
▫ swaps it with the value in the first position, and
▫ repeats these steps for the remainder of the list
• It does no more than n swaps, and thus is useful where
swapping is very expensive

Sorting an Array of Integers
• The picture shows an
array of six integers
that we want to sort
from smallest to
largest
0
10
20
30
40
50
60
70
[1] [2] [3] [4] [5] [6][0] [1] [2] [3] [4] [5]

0
10
20
30
40
50
60
70
[1] [2] [3] [4] [5] [6]
The Selection Sort Algorithm
• Start by finding the
smallest entry.
0
10
20
30
40
50
60
70
[1] [2] [3] [4] [5] [6][0] [1] [2] [3] [4] [5]

0
10
20
30
40
50
60
70
[1] [2] [3] [4] [5] [6]
• Start by finding the
smallest entry.
• Swap the smallest
entry with the first
entry.
0
10
20
30
40
50
60
70
[1] [2] [3] [4] [5] [6][0] [1] [2] [3] [4] [5]

0
10
20
30
40
50
60
70
[1] [2] [3] [4] [5] [6]
• Start by finding
the smallest
entry.
• Swap the smallest
entry with the
first entry.
0
10
20
30
40
50
60
70
[1] [2] [3] [4] [5] [6][0] [1] [2] [3] [4] [5]

0
10
20
30
40
50
60
70
[1] [2] [3] [4] [5] [6]
• Part of the array
is now sorted.
0
10
20
30
40
50
60
70
[1] [2] [3] [4] [5] [6]
Sorted side Unsorted side
[0] [1] [2] [3] [4] [5]

0
10
20
30
40
50
60
70
[1] [2] [3] [4] [5] [6]
0
10
20
30
40
50
60
70
[1] [2] [3] [4] [5] [6]
• Find the smallest
element in the
unsorted side.
[0] [1] [2] [3] [4] [5]

0
10
20
30
40
50
60
70
[1] [2] [3] [4] [5] [6]
0
10
20
30
40
50
60
70
[1] [2] [3] [4] [5] [6]
• Find the smallest
element in the
unsorted side.
• Swap with the front
of the unsorted
side.
[0] [1] [2] [3] [4] [5]

0
10
20
30
40
50
60
70
[1] [2] [3] [4] [5] [6]
0
10
20
30
40
50
60
70
[1] [2] [3] [4] [5] [6]
• We have
increased the size
of the sorted side
by one element.
[0] [1] [2] [3] [4] [5]

0
10
20
30
40
50
60
70
[1] [2] [3] [4] [5] [6]
0
10
20
30
40
50
60
70
[1] [2] [3] [4] [5] [6]
• The process
continues...
Smallest
from
unsorted
[0] [1] [2] [3] [4] [5]

0
10
20
30
40
50
60
70
[1] [2] [3] [4] [5] [6]
0
10
20
30
40
50
60
70
[1] [2] [3] [4] [5] [6]
• The process
continues...
[0] [1] [2] [3] [4] [5]

0
10
20
30
40
50
60
70
[1] [2] [3] [4] [5] [6]
0
10
20
30
40
50
60
70
[1] [2] [3] [4] [5] [6]
• The process
continues...
Sorted side
is bigger
[0] [1] [2] [3] [4] [5]

0
10
20
30
40
50
60
70
[1] [2] [3] [4] [5] [6]
0
10
20
30
40
50
60
70
[1] [2] [3] [4] [5] [6]
• The process keeps
adding one more
number to the
sorted side.
• The sorted side has
the smallest
numbers, arranged
from small to large.
[0] [1] [2] [3] [4] [5]

0
10
20
30
40
50
60
70
[1] [2] [3] [4] [5] [6]
0
10
20
30
40
50
60
70
[1] [2] [3] [4] [5] [6]
• We can stop when the
unsorted side has just
one number, since
that number must be
the largest number.
[0] [1] [2] [3] [4] [5]

0
10
20
30
40
50
60
70
[1] [2] [3] [4] [5] [6]
The Selection Sort Algorithm
• The array is now
sorted.
• We repeatedly
selected the smallest
element, and moved
this element to the
front of the unsorted
side.
[0] [1] [2] [3] [4] [5]

Selection Sort – Pseudocode
Input: An array A[1..n] of n elements.
Output: A[1..n] sorted in descending order
1. for i  1 to n - 1
2. min  i
3. for j  i + 1 to n {Find the i th smallest element.}
4. if A[j] < A[min] then
5. min  j
6. end for
7. if min  i then interchange A[i] and A[min]
8. end for

Selection Sort – Implementation

Complexity of Selection Sort
• An in-place comparison sort
• O(n2) complexity, making it inefficient on large lists,
and generally performs worse than the similar
insertion sort.
• Selection sort is not difficult to analyze compared to
other sorting algorithms since none of the loops
depend on the data in the array

• Selecting the lowest element requires scanning all n
elements (this takes n − 1 comparisons) and then
swapping it into the first position
• Finding the next lowest element requires scanning the
remaining n − 1 elements and so on,
• for (n − 1) + (n − 2) + ... + 2 + 1 = n(n − 1) / 2 ∈ O(n2)
comparisons
• Each of these scans requires one swap for n − 1
elements (the final element is already in place).

• Worst case performance
• Best case performance
• Average case performance
• Worst case space complexity Total:
• Worst case space complexity auxiliary:
• Where n is the number of elements being sorted 25

Insertion Sort
• Insertion sort is not as slow as bubble sort, and it is
easy to understand.
• Insertion sort keeps making the left side of the array
sorted until the whole array is sorted.
• Real life example:
▫ Insertion sort works the same way as arranging your
hand when playing cards.
▫ To sort the cards in your hand you extract a card, shift
the remaining cards, and then insert the extracted card
in the correct place.

0
10
20
30
40
50
60
70
[1] [2] [3] [4] [5] [6]
The Insertion Sort Algorithm
• Views the array as
having two sides
• a sorted side and
• an unsorted side.
[0] [1] [2] [3] [4] [5]

0
10
20
30
40
50
60
70
[1] [2] [3] [4] [5] [6]
• The sorted side
starts with just the
first element, which
is not necessarily
the smallest
element.
0
10
20
30
40
50
60
70
[1] [2] [3] [4] [5] [6][0] [1] [2] [3] [4] [5]

0
10
20
30
40
50
60
70
[1] [2] [3] [4] [5] [6]
• The sorted side
grows by taking
the front element
from the
unsorted side...
0
10
20
30
40
50
60
70
[1] [2] [3] [4] [5] [6][0] [1] [2] [3] [4] [5]

0
10
20
30
40
50
60
70
[1] [2] [3] [4] [5] [6]
• ...and inserting it
in the place that
keeps the sorted
side arranged
from small to
large.
0
10
20
30
40
50
60
70
[1] [2] [3] [4] [5] [6][0] [1] [2] [3] [4] [5]

0
10
20
30
40
50
60
70
[1] [2] [3] [4] [5] [6]
• In this example,
the new element
goes in front of
the element that
was already in the
sorted side.
0
10
20
30
40
50
60
70
[1] [2] [3] [4] [5] [6][0] [1] [2] [3] [4] [5]

0
10
20
30
40
50
60
70
[1] [2] [3] [4] [5] [6]
• Sometimes we
are lucky and the
new inserted
item doesn't
need to move at
all.
0
10
20
30
40
50
60
70
[1] [2] [3] [4] [5] [6][0] [1] [2] [3] [4] [5]

0
10
20
30
40
50
60
70
[1] [2] [3] [4] [5] [6]
• Sometimes we
are lucky twice in
a row.
0
10
20
30
40
50
60
70
[1] [2] [3] [4] [5] [6][0] [1] [2] [3] [4] [5]

0
10
20
30
40
50
60
70
[1] [2] [3] [4] [5] [6]
Copy the new
element to a
separate location.
0
10
20
30
40
50
60
70
[1] [2] [3] [4] [5] [6]
3] [4] [5] [6]
[0] [1] [2] [3] [4] [5]

0
10
20
30
40
50
60
70
[1] [2] [3] [4] [5] [6]
Shift elements in
the sorted side,
creating an open
space for the new
element.
0
10
20
30
40
50
60
70
[1] [2] [3] [4] [5] [6]
3] [4] [5] [6]
[0] [1] [2] [3] [4] [5]

0
10
20
30
40
50
60
70
[1] [2] [3] [4] [5] [6]
0
10
20
30
40
50
60
70
[1] [2] [3] [4] [5] [6]
Shift elements in
the sorted side,
creating an open
space for the new
element.
3] [4] [5] [6]
0
10
20
30
40
50
60
70
[1] [2] [3] [4] [5] [6][0] [1] [2] [3] [4] [5]

0
10
20
30
40
50
60
70
[1] [2] [3] [4] [5] [6]
0
10
20
30
40
50
60
70
[1] [2] [3] [4] [5] [6]
Continue shifting
elements...
3] [4] [5] [6]
0
10
20
30
40
50
60
70
[1] [2] [3] [4] [5] [6][0] [1] [2] [3] [4] [5]

0
10
20
30
40
50
60
70
[1] [2] [3] [4] [5] [6]
0
10
20
30
40
50
60
70
[1] [2] [3] [4] [5] [6]
...until you reach
the location for
the new element.
3] [4] [5] [6]
0
10
20
30
40
50
60
70
[1] [2] [3] [4] [5] [6][0] [1] [2] [3] [4] [5]

0
10
20
30
40
50
60
70
[1] [2] [3] [4] [5] [6]
0
10
20
30
40
50
60
70
[1] [2] [3] [4] [5] [6]
Copy the new
element back into
the array, at the
correct location.
3] [4] [5] [6]
[0] [1] [2] [3] [4] [5]

0
10
20
30
40
50
60
70
[1] [2] [3] [4] [5] [6]
3] [4] [5] [6]
• The last element
must also be
inserted. Start by
copying it...
[0] [1] [2] [3] [4] [5]

0
10
20
30
40
50
60
70
[1] [2] [3] [4] [5] [6]
0
10
20
30
40
50
60
70
[1] [2] [3] [4] [5] [6]
How many shifts
will occur before
we copy this
element back into
the array?
3] [4] [5] [6]
[0] [1] [2] [3] [4] [5]

0
10
20
30
40
50
60
70
[1] [2] [3] [4] [5] [6]
0
10
20
30
40
50
60
70
[1] [2] [3] [4] [5] [6]
3] [4] [5] [6]
• Four items are
shifted.
[0] [1] [2] [3] [4] [5]

0
10
20
30
40
50
60
70
[1] [2] [3] [4] [5] [6]
3] [4] [5] [6]
• Four items are
shifted.
•And then the
element is copied
back into the array.
[0] [1] [2] [3] [4] [5]
The Insertion Sort Algorithm

Insertion Sort Example
• To sort an array with k elements, Insertion sort
requires k – 1 passes.
• Example:

Insertion Sort - Algorithm
For i = 2 to n do the following
a. set NextElement = x[i] and
x[0] = nextElement
b. set j = i
c. While nextElement < x[j – 1] do following
set x[j] equal to x[j – 1]
decrement j by 1
End wile
d. set x[j] equal to nextElement
End for

Insertion Sort - Pseudocode
Input: An array A[1..n] of n elements.
Output: A[1..n] sorted in nondecreasing order.
1. for i  2 to n
2. x  A[i]
3. j  i - 1
4. while (j >0) and (A[j] > x)
5. A[j + 1]  A[j]
6. j  j - 1
7. end while
8. A[j + 1]  x
9. end for

Insertion Sort - Implementation
void InsertionSort(int list[], int size){
int i,j,k,temp;
for(i=1;i < size;i++) {
temp=list[i];
j=i;
while((j > 0)&&(temp < list[j-1]) {
list[j]=list[j-1];
j--;
} // end while
list[j]=temp;
} // end for loop
} // end function

Complexity of Insertion Sort
• Let a0, ..., an-1 be the sequence to be sorted. At the
beginning and after each iteration of the algorithm
the sequence consists of two parts: the first part
a0, ..., ai-1 is already sorted, the second part
ai, ..., an-1 is still unsorted (i in 0, ..., n).
• The worst case occurs when in every step the proper
position for the element that is inserted is found at
the beginning of the sorted part of the sequence.

The minimum # of element comparisons (best case) occurs when
the array is already sorted in nondecreasing order. In this case,
the # of element comparisons is exactly n - 1, as each element
A[i], 2 ≤ i ≤ n, is compared with A[i - 1] only.
The maximum # of element comparisons (Worst case) occurs if the
array is already sorted in decreasing order and all elements are
distinct. In this case, the number is
n n-1
∑ (i – 1) = ∑ (i – 1) = n(n-1)/2
i =2 i =1
This is because each element A[i], 2 ≤ i ≤ n is
compared with each entry in subarray A[1 .. i-1]
 Pros: Relatively simple and easy to implement.
Cons: Inefficient for large lists.

• In the insertion sort algorithm (n – 1) times the loop will
execute for comparisons and interchanging the numbers
• The inner while loop iterates maximum of ((n – 1) × (n –
1))/2 times to compute the sorting
• Best Case
▫ occurs when the array A is in sorted order and the outer for
loop will iterate for (n – 1) times
▫ And the inner while loop will not execute because the given
array is a sorted array
i.e. f(n)=O(n)

• Average Case
▫ On the average case there will be approximately
(n – 1)/2 comparisons in the inner while loop
▫ Hence the average case
f (n) = (n – 1)/2 + ...... + 2/2 +1/2
= n (n – 1)/4
= O(n2)
• Worst Case
▫ The worst case occurs when the array A is in reverse
order and the inner while loop must use the
maximum number (n – 1) of comparisons
f(n) = (n – 1) + ....... 2 + 1
= (n (n – 1))/2
= O(n2)

• Best case: O(n). It occurs when the data is in sorted
order. After making one pass through the data and
making no insertions, insertion sort exits.
• Average case: θ(n2) since there is a wide variation
with the running time.
• Worst case: O(n2) if the numbers were sorted in
reverse order.

• Best case performance
• Average case performance
• Worst case performance
• Worst case space complexity auxiliary
• Where n is the number of elements being sorted
  n2/2 comparisons and exchanges
55

Comparison Bubble and Insertion Sort
• Bubble sort is asymptotically equivalent in running
time O(n2) to insertion sort in the worst case
• But the two algorithms differ greatly in the number
of swaps necessary
• Experimental results have also shown that insertion
sort performs considerably better even on random
lists.
• For these reasons many modern algorithm textbooks
avoid using the bubble sort algorithm in favor of
insertion sort.

Comparison Bubble and Insertion Sort
• Bubble sort also interacts poorly with modern CPU
hardware. It requires
▫ at least twice as many writes as insertion sort,
▫ twice as many cache misses, and
▫ asymptotically more branch mispredictions.
• Experiments of sorting strings in Java show bubble
sort to be
▫ roughly 5 times slower than insertion sort and
▫ 40% slower than selection sort

Comparison of Sorts
Bubble Selection Insertion
Best Case
Average Case
Worst Case
Space
complexity

Merge Sort
• Like QuickSort, Merge Sort is a Divide and Conquer
algorithm. It divides input array in two halves, calls
itself for the two halves and then merges the two
sorted halves. The merge() function is used for
merging two halves. The merge(arr, l, m, r) is key
process that assumes that arr[l..m] and arr[m+1..r]
are sorted and merges the two sorted sub-arrays into
one.

Merge Sort - Implementation
• void MergeSort(int LIST[], int lo, int hi)
• {
• int mid;
• if(lo < hi)
• {
• mid = (lo + hi)/ 2;
• MergeSort(LIST, lo, mid);
• MergeSort(LIST, mid+1, hi);
• ListMerger(LIST, lo, mid, hi);
• }
• }
• void ListMerger(int List[], int lo, int mid, int hi)
• {
• int TempList[hi-lo+1];
//temporary merger array
• int i = lo, j = mid + 1; //i is for left-hand,j is for right-hand
• int k = 0;
//k is for the temporary array
• while(i <= mid && j <=hi)
• {
• if(List[i] <= List[j])
• TempList[k++] = List[i++];
• else
• TempList[k++] = List[j++];
• }

Merge Sort - Implementation
• //remaining elements of left-half
• while(i <= mid)
• TempList[k++] = List[i++];
• //remaining elements of right-half
• while(j <= hi)
• TempList[k++] = List[j++];
• //copy the mergered temporary List to the original List
• for(k = 0, i = lo; i <= hi; ++i, ++k)
• List[i] = TempList[k];
•
• }

Performance
• Time Complexity: Sorting arrays on different machines.
Merge Sort is a recursive algorithm and time complexity
can be expressed as following recurrence relation.
T(n) = 2T(n/2) + O(n)
The above recurrence can be solved either using
Recurrence Tree method or Master method. It falls in
case II of Master Method and solution of the recurrence
is O(nlogn).
Time complexity of Merge Sort is O(nlogn) in all 3 cases
(worst, average and best) as merge sort always divides
the array in two halves and take linear time to merge two
halves.

Performance
• Auxiliary Space: O(n)
• Algorithmic Paradigm: Divide and Conquer
• Sorting In Place: No in a typical implementation
• Stable: Yes

Summary
• Selection Sort,
• Insertion Sort,
• Algorithms Analysis
• Merge Sort Algorithm,
• Merge Sort Analysis

References
• https://www.geeksforgeeks.org/sorting-
algorithms/
• https://brilliant.org/wiki/sorting-algorithms/
• https://betterexplained.com/articles/sorting-
algorithms/
• https://codeburst.io/algorithms-i-searching-
and-sorting-algorithms-56497dbaef20

Sorting Algorithms

More Related Content

What's hot

Similar to Sorting Algorithms

More from Afaq Mansoor Khan

Recently uploaded

Sorting Algorithms