Selection Sort Analysis

When we analyze algorithms, we make a couple of assumptions:

1. That each line takes a constant amount of time
2. The time for an iteration of an outer loop is some constant c₁ + time for the inner loop
3. Each iteration of the inner loop takes some constant c₂ time

The key thing we have to figure out is the number of iterations of the inner loop.

i inner loop iterations 0 n-1 1 n-2 2 n-3 ... n-2 1

So the total time is:
[c₁ + (n-1)c₂] + [c₁ + (n-2)c₂] + [c₁ + (n-3)c₂] + ... + [c₁ + (1)c₂]
We have n-1 terms, each of the form c₁ + (...)c₂.
Total time = (n-1)c₁ + [(n-1) + (n-2) + (n-3) + ... + 1]c₂.
What is 1 + 2 + 3 + ... + (n-1)?

Fact:

1 + 2 + 3 + ... + n = = n(n + 1) / 2 .
This is known as an arithmetic series.
So, 1 + 2 + 3 + ... + n-1 = ((n-1) * n) / 2 = (n² - n) / 2 = O(n²)
Therefore, total time is (n-1)c₁ + c₂ * O(n²) = O(n²)
This result is independent of the input characteristics.

Insertion Sort

Works like sorting a hand of cards - you place A[i] where it belongs in A[0..i] so that A[0..i] is sorted.

At iteration i, A[0..i-1] is the original A[0..i-1], but sorted. We place A[i] where it belongs in A[0..i], so that now A[0..i] becomes sorted.

See insertionSort.cpp.

Analysis

Outer loop runs i = 1 to n - 1 ==> O(n)
Inner loop runs between 0 and i times ==> O(n)
Total is O(n) * O(n) = O(n²)

But remember that here O(n²) is only an upper bound. Can the running time be less?

Best Case

Inner loop runs 0 times ==> O(n) total time for the algorithm. This occurs when the array is already sorted. The closer to sorted the input is, the faster insertion sort runs. Times when input might be "almost" sorted could include the way checks you write arrive at your bank. Some may take longer to get back than others, but they will mostly arrive in numerical order (assuming, of course, that you write them in numerical order)

Average Case

Inner loop runs i/2 times. This reduces the worst case time by 1/2, but it's still O(n²). This is an example of how the average case often isn't much better than the worst case.