Review

We did a binary search iteratively, but we can do it recursively as well:

This looks like

template<typename T>
int binary_search(const vector<T>& data, 
                  const T& target,
                  int low = 0, 
                  int high = data.size()-1) 
{

    if(low > high)
        return -1;

    int mid = low + (high - low) / 2; // Why did I do this?

    if(data.at(mid) == target)
        return mid;
    else if(data.at(mid) < target) // Search right
        return binary_search(data, mid+1, high, target);
    else if(data.at(mid) > target) // Search left
        return binary_search(data, low, mid-1, target);
}

Sorting algorithms

A sorting algorithm is a function that takes a sequence of items and somehow constructs a permutation of them, such that they are ordered in some fashion. Usually, we want things to be ordered according to the normal comparison operators, so that if a < b then a comes before b in the final permutation. Still, there are a few things we have to make sure we get right:

There are some terms associated with sorting that it’s important to be aware of:

Selection sort

Effectively, selection sort splits the list into the sorted region at the beginning, and the unsorted region at the end. The sorted region grows, while the unsorted region shrinks.

Selection sort is not stable.

Iteratively, this looks like this:

template<typename T>
void selection_sort(vector<T>& data) {
    for(int i = 0; i < data.size()-1; ++i) {

        // Find smallest
        int s = i;
        for(int j = i+1; j < data.size(); ++j)
            if(data[j] < data[s]) 
                s = j;

        // Swap it into place
        std::swap(data[i], data[s]);
    }
}

Let’s trace through this on a small example to get a feel for how it works.

How can we implement this recursively? In addition to passing the vector, we’ll pass an int argument i, specifying where the sorting should start. It is assumed that data[0]...data[i-1] is already sorted. Thus, we just remove the outer loop and replace it with an extra parameter and recursion (this pattern will apply to the other sorting algorithms as well).

Let’s analyze the recursion:

template<typename T>
void selection_sort(vector<T>& data, int i = 0) 
{
    if(i == data.size() - 1)
        return;
    else {
        // Find smallest
        int s = i;
        for(int j = i+1; j < data.size(); ++j)
            if(data[j] < data[s]) 
                s = j;

        // Swap it into place
        std::swap(data[i], data[s]);

        // Recursively sort the remainder
        selection_sort(data, i + 1);
    }
}

Let’s draw the recursion tree for this. We won’t trace through the loop, we’ll just assume (for now) that it works correctly.

DIAGRAM

Stability of selection sort

Is selection sort stable? In order to determine this, we need to examine a hypothetical situation: we are sorting a sequence consisting of number-letter pairs, by their number parts:

 ...  1a ... 1b ...  

Selection sort is stable if, whenever we have the above situation (element 1a occurs before 1b), in the output sequence we will have

 ... 1a, 1b ...

There are two situations to consider:

Thus, selection sort is not stable. It can be made stable if we do an insert instead of a swap, but this is obviously more expensive, as it requires moving all the following elements (unless we are using a linked list!).

Double-selection sort

Double selection sort works by finding both the smallest and largest elements in the unsorted region and then swapping both into place: smallest at the beginning, largest at the end. Thus, it sorts the array in half the time of the normal selection sort, because it places two elements per pass instead of one.

template<typename T>
void selection_sort(vector<T>& data) {
    for(int start = 0, end = data.size()-1; start < end; ++start, --end) {

        // Find smallest, largest
        int s = start, l = start;
        for(int j = start + 1; j <= end; ++j) {
            if(data[j] < data[s]) 
                s = j;
            if(data[j] > data[l])
                l = j;
        }

        // Swap them into place
        std::swap(data[start], data[s]);
        std::swap(data[end], data[l]);
    }
}

However, there’s a problem with this implementation. Consider the following setup:

{ 10,  5,  1,  3,  7 }
----------------------
   0   1   2   3   4

In this setup, the first pass will end with s = 2 (index of the smallest element) and l = 0 (index of the largest) and then we will do

std::swap(data[start], data[s]);

--> { 1,  5,  10,  3,  7 }

followed by

std::swap(data[end], data[l]);

--> { 7,  5,  10,  3,  1 }

Something went wrong! The largest and smallest elements are in the wrong places. This problem occurs when the largest element is in the start position. When this occurs, the first swap moves the largest element, so we have to update l accordingly:

// Swap them into place
std::swap(data[start], data[s]);
if(l == start)
    l = s;
std::swap(data[end], data[l]);

Now the swaps work correctly.

Gnome sort

Gnome sort is a simple-to-implement sort which does not require a nested loop, but is somewhat difficult to analyze. It works by maintaining a current position within the sequence, which is initially 1 (not 0). At each step, we do the following: Compare the element at the current position with the previous element (at position - 1):

This has a straightforward translation into code:

void gnome_sort(vector<int>& v) {
    int pos = 1;
    while(pos < v.size()) {
        if(pos == 0 || v[pos] >= v[pos - 1])
            ++pos;
        else {
            std::swap(v[pos], v[pos - 1]);
            --pos;
        }
    }
}

Gnome sort is quadratic, but is adaptive: if the list is almost sorted, the “gnome” will tend to walk straight from the beginning to the end of the sequence.

Gnome sort is stable.

Bubble sort

The idea is to compare adjacent elements in the input (e.g., a[i] and a[i+1]) and swap them if they are out of order. We start at the beginning of the input and walk through it, swapping our way to the end. After one such pass, the largest element will have “bubbled” up to the last element of the array. So then we can make another pass, but we skip the last element this time. Whereas selection sort made smaller and smaller passes starting from the front, bubble sort makes smaller and smaller passes from the end.

Bubble sort is stable.

An implementation looks something like this:

template<typename T>
void bubble_sort(vector<T>& data) {

    for(int j = 1; j < data.size(); ++j)
        for(int i = 0; i < data.size() - j; ++i)
            if(data[i] > data[i + 1]) 
                swap(data[i], data[i + 1]);
}

As implemented, this function is of order \(O(n^2)\) in both the best and worst cases; nested loop, with the same “triangular” structure we saw before. We can actually implement a simple optimization: in the inner loop, if we perform no swaps, then the input is sorted, and we can stop. If we check for this, it may allow us to exit early.

template<typename T>
void bubble_sort(vector<T>& data) {

    for(int j = 1; j < data.size(); ++j) {
        bool sorted = true;
        for(int i = 0; i < data.size() - j; ++i)
            if(data[i] > data[i + 1]) {
                swap(data[i], data[i + 1]);
                sorted = false;
            }

        if(sorted)
            return;
    }
}

Now, what are the best and worst cases?

Recursive bubble sort

As with selection sort, we can make a recursive version of bubble sort by replacing the outer loop with an extra parameter and recursion:

template<typename T>
void bubble_sort(vector<T>& data, int j = 1) 
{
    if(j == data.size())
        return;
    else {
        bool sorted = true;
        for(int i = 0; i < data.size() - j; ++i)
            if(data[i] > data[i + 1]) {
                swap(data[i], data[i + 1]);
                sorted = false;
            }

        if(sorted)
            return;

        bubble_sort(data, j + 1);
    }
}

Insertion sort

Suppose we split the input into two “sections”, sorted and unsorted. The sorted section is initially empty. For each element in the unsorted section, we insert it into the sorted section, in its proper, sorted, position. Because this is an array/vector, inserting an element requires shifting all the elements after it up one step. There’s a simplification we can apply: the steps of finding the proper sorted position and inserting/shifting it there can be combined into a single loop. We take the element and swap it with the previous element, until either it is in the correct position, or it has reached the beginning of the vector.

Insertion sort is stable.

template<typename T>
void insertion_sort(vector<T>& data) 
{
    for(int i = 1; i < data.size(); ++i)        
        for(int j = i; i > 0 && data[j] < data[j-1]; --j)
            std::swap(data[j], data[j-1]);

}

In terms of number of lines, insertion sort is probably the simplest \(O(n^2)\) sorting algorithm.

What are the best and worst cases for this algorithm?

We can write a recursive version through a pretty straightforward removal of the outer loop:

template<typename T>
void insertion_sort(vector<T>& data, int i = 1) 
{
    if(i == data.size())
        return;
    else {    
        for(int j = i; i > 0 && data[j] < data[j-1]; --j)
            std::swap(data[j], data[j-1]);

        insertion_sort(data, i+1);
    }    
}