Review
Binary search
We did a binary search iteratively, but we can do it recursively as well:
There are two base cases: when we find the item, or when the search space is reduced to 0 (indicating that the item is not found).
The recursive case compares the value of the target to the value at the current midpoint, and then reduces the size of the search space (by recursively searching either the left or right sides).
This looks like
template<typename T>
int binary_search(const vector<T>& data,
const T& target,
int low = 0,
int high = data.size()-1)
{
if(low > high)
return -1;
int mid = low + (high - low) / 2; // Why did I do this?
if(data.at(mid) == target)
return mid;
else if(data.at(mid) < target) // Search right
return binary_search(data, mid+1, high, target);
else if(data.at(mid) > target) // Search left
return binary_search(data, low, mid-1, target);
}
Sorting algorithms
A sorting algorithm is a function that takes a sequence of items and somehow
constructs a permutation of them, such that they are ordered in some fashion.
Usually, we want things to be ordered according to the normal comparison
operators, so that if a < b
then a comes before b in the final permutation.
Still, there are a few things we have to make sure we get right:
Obviously we can’t lose any elements through the process.
There may be duplicates in the original, if so, there should be an equal number of duplicates in the output.
For convenience, we allow sorting an empty sequence (which, when sorted, results in yet another empty sequence)
There are some terms associated with sorting that it’s important to be aware of:
Stability – when the input sequence has element which compare as equal but which are distinct (e.g., employees with identical names but otherwise different people) the question arises as to whether, in the output sequence, they occur in the same order as in the original. E.g., if employee “John Smith” #123 came before “John Smith” #234 in the original sequence, then we say that a sort is stable if #123 is before #234 in the result.
Stability is important when sorting a sequence on multiple criteria. E.g., if we sort based on first name, then based on last name, an unstable sort won’t give us the result we want: the first name entries will be all mixed up.
Adaptability – some input sequences are already partially (or completely) sorted; an adaptable sorting algorithm will run faster (in big-O terms) on partially sorted inputs. The optimal runtime for a completely sorted input is \(O(n)\), the time that it takes to verify that the input is already sorted.
In-Place – an in-place sorting algorithm is one that needs no extra space (i.e., it’s space complexity is \(O(1)\)) to sort. Some algorithms cannot be used in place, and have to construct a separate output sequence of the same size as the input, to write the results into.
Selection sort
To selection sort a list of items, we first find the smallest item in the entire list, and put it at the beginning.
Then we find the smallest item in everything after the first item, and put it second.
Continue until there’s nothing left unsorted.
Effectively, selection sort splits the list into the sorted region at the beginning, and the unsorted region at the end. The sorted region grows, while the unsorted region shrinks.
Selection sort is not stable.
Iteratively, this looks like this:
template<typename T>
void selection_sort(vector<T>& data) {
for(int i = 0; i < data.size()-1; ++i) {
// Find smallest
int s = i;
for(int j = i+1; j < data.size(); ++j)
if(data[j] < data[s])
s = j;
// Swap it into place
std::swap(data[i], data[s]);
}
}
Let’s trace through this on a small example to get a feel for how it works.
How can we implement this recursively? In addition to passing the vector, we’ll
pass an int
argument i
, specifying where the sorting should start. It is
assumed that data[0]...data[i-1]
is already sorted. Thus, we just remove
the outer loop and replace it with an extra parameter and recursion (this
pattern will apply to the other sorting algorithms as well).
Let’s analyze the recursion:
The base case is when
i == data.size()-1
. That means there’s only 1 element, and a 1-element list is always sorted.The recursive case is when
i < data.size()-1
. In that case, we recursively break it down by- Finding the minimum of the region, and placing it at the beginning.
- Recursively selection-sorting data starting at
i+1
template<typename T>
void selection_sort(vector<T>& data, int i = 0)
{
if(i == data.size() - 1)
return;
else {
// Find smallest
int s = i;
for(int j = i+1; j < data.size(); ++j)
if(data[j] < data[s])
s = j;
// Swap it into place
std::swap(data[i], data[s]);
// Recursively sort the remainder
selection_sort(data, i + 1);
}
}
Let’s draw the recursion tree for this. We won’t trace through the loop, we’ll just assume (for now) that it works correctly.
DIAGRAM
Stability of selection sort
Is selection sort stable? In order to determine this, we need to examine a hypothetical situation: we are sorting a sequence consisting of number-letter pairs, by their number parts:
... 1a ... 1b ...
Selection sort is stable if, whenever we have the above situation (element
1a
occurs before 1b
), in the output sequence we will have
... 1a, 1b ...
There are two situations to consider:
Suppose we have already sorted all the elements less than 1:
-3d, -2a, -1t, 0q, XXX ... 1a ... 1b ...
Our search for the smallest remaining element will find the first
1
element (because the comparison used is<
and not<=
) and use that for the next element:-3d, -2a, -1t, 0q, 1a ... XXX ... 1b ...
Thus, in this case,
1a
is still before1b
.The second case is less obvious: what if
1a
occurs earlier in the sequence, such that it would be swapped out? E.g., suppose we have-3d, -2a, 1a, ... 1b ... -1t ...
When we place
-1t
in the correct location, we will swap it with1a
, producing:-3d, -2a, -1t, ... 1b ... 1a ...
Later, when we try to find the smallest 1, we will find not
1a
, but1b
.
Thus, selection sort is not stable. It can be made stable if we do an insert instead of a swap, but this is obviously more expensive, as it requires moving all the following elements (unless we are using a linked list!).
Double-selection sort
Double selection sort works by finding both the smallest and largest elements in the unsorted region and then swapping both into place: smallest at the beginning, largest at the end. Thus, it sorts the array in half the time of the normal selection sort, because it places two elements per pass instead of one.
template<typename T>
void selection_sort(vector<T>& data) {
for(int start = 0, end = data.size()-1; start < end; ++start, --end) {
// Find smallest, largest
int s = start, l = start;
for(int j = start + 1; j <= end; ++j) {
if(data[j] < data[s])
s = j;
if(data[j] > data[l])
l = j;
}
// Swap them into place
std::swap(data[start], data[s]);
std::swap(data[end], data[l]);
}
}
However, there’s a problem with this implementation. Consider the following setup:
{ 10, 5, 1, 3, 7 }
----------------------
0 1 2 3 4
In this setup, the first pass will end with s = 2
(index of the smallest
element) and l = 0
(index of the largest) and then we will do
std::swap(data[start], data[s]);
--> { 1, 5, 10, 3, 7 }
followed by
std::swap(data[end], data[l]);
--> { 7, 5, 10, 3, 1 }
Something went wrong! The largest and smallest elements are in the wrong places.
This problem occurs when the largest element is in the start
position. When
this occurs, the first swap moves the largest element, so we have to update
l
accordingly:
// Swap them into place
std::swap(data[start], data[s]);
if(l == start)
l = s;
std::swap(data[end], data[l]);
Now the swaps work correctly.
Gnome sort
Gnome sort is a simple-to-implement sort which does not require a nested loop, but is somewhat difficult to analyze. It works by maintaining a current position within the sequence, which is initially 1 (not 0). At each step, we do the following: Compare the element at the current position with the previous element (at position - 1):
If they are in order, increase the current position by 1.
If they are out of order, swap them and decrease the current position by 1.
If the current position
==
the length of the sequence, stop, we are done.
This has a straightforward translation into code:
void gnome_sort(vector<int>& v) {
int pos = 1;
while(pos < v.size()) {
if(pos == 0 || v[pos] >= v[pos - 1])
++pos;
else {
std::swap(v[pos], v[pos - 1]);
--pos;
}
}
}
Gnome sort is quadratic, but is adaptive: if the list is almost sorted, the “gnome” will tend to walk straight from the beginning to the end of the sequence.
Gnome sort is stable.
Bubble sort
The idea is to compare adjacent elements in the
input (e.g., a[i]
and a[i+1]
) and swap them if they are out of order.
We start at the beginning of the input and walk through it, swapping our way
to the end. After one such pass, the largest element will have “bubbled” up
to the last element of the array. So then we can make another pass, but
we skip the last element this time. Whereas selection sort made smaller and
smaller passes starting from the front, bubble sort makes smaller and smaller
passes from the end.
Bubble sort is stable.
An implementation looks something like this:
template<typename T>
void bubble_sort(vector<T>& data) {
for(int j = 1; j < data.size(); ++j)
for(int i = 0; i < data.size() - j; ++i)
if(data[i] > data[i + 1])
swap(data[i], data[i + 1]);
}
As implemented, this function is of order \(O(n^2)\) in both the best and worst cases; nested loop, with the same “triangular” structure we saw before. We can actually implement a simple optimization: in the inner loop, if we perform no swaps, then the input is sorted, and we can stop. If we check for this, it may allow us to exit early.
template<typename T>
void bubble_sort(vector<T>& data) {
for(int j = 1; j < data.size(); ++j) {
bool sorted = true;
for(int i = 0; i < data.size() - j; ++i)
if(data[i] > data[i + 1]) {
swap(data[i], data[i + 1]);
sorted = false;
}
if(sorted)
return;
}
}
Now, what are the best and worst cases?
The best case is when the input is already sorted. In this case, we make one pass through the vector (to check whether it is sorted) but don’t swap anything, and then we return. So the best case run time efficiency is \(O(n)\).
The worst case is an array that is sorted in descending order. Every element will have to “bubble” the full distance to its proper place, so we’ll never exit early (due to
sorted
) and the condition in theif
statement will be true every time, so we’ll perform as many swaps as necessary. If we work out the sum, we get$$\sum_{j = 1}^n (n - j) = n - \sum_{j = 1}^n j$$where the second sum expands to \(\frac{n(n+1)}{2}\), giving us an order of \(O(n)\).
Recursive bubble sort
As with selection sort, we can make a recursive version of bubble sort by replacing the outer loop with an extra parameter and recursion:
Base case:
j == data.size()
, in which case there’s nothing left to sort.Recursive case:
j < data.size()
, in which case we do “bubble pass” and then recursively sort the remainder.
template<typename T>
void bubble_sort(vector<T>& data, int j = 1)
{
if(j == data.size())
return;
else {
bool sorted = true;
for(int i = 0; i < data.size() - j; ++i)
if(data[i] > data[i + 1]) {
swap(data[i], data[i + 1]);
sorted = false;
}
if(sorted)
return;
bubble_sort(data, j + 1);
}
}
Insertion sort
Suppose we split the input into two “sections”, sorted and unsorted. The sorted section is initially empty. For each element in the unsorted section, we insert it into the sorted section, in its proper, sorted, position. Because this is an array/vector, inserting an element requires shifting all the elements after it up one step. There’s a simplification we can apply: the steps of finding the proper sorted position and inserting/shifting it there can be combined into a single loop. We take the element and swap it with the previous element, until either it is in the correct position, or it has reached the beginning of the vector.
Insertion sort is stable.
template<typename T>
void insertion_sort(vector<T>& data)
{
for(int i = 1; i < data.size(); ++i)
for(int j = i; i > 0 && data[j] < data[j-1]; --j)
std::swap(data[j], data[j-1]);
}
In terms of number of lines, insertion sort is probably the simplest \(O(n^2)\) sorting algorithm.
What are the best and worst cases for this algorithm?
The best case is if the condition on the inner loop is false every time, so the inner loop doesn’t run at all. This happens if the input is already sorted. If the inner loop doesn’t run, then we only do \(O(n)\) work, making insertion sort adaptive.
If the inner loop has to run all the way to the end (if the vector is reverse-sorted) then we have the familiar “triangular” complexity which is \(O(n^2)\).
We can write a recursive version through a pretty straightforward removal of the outer loop:
template<typename T>
void insertion_sort(vector<T>& data, int i = 1)
{
if(i == data.size())
return;
else {
for(int j = i; i > 0 && data[j] < data[j-1]; --j)
std::swap(data[j], data[j-1]);
insertion_sort(data, i+1);
}
}