Last time, we saw that the time for a linear search is d₁n + d₂, and that time for a binary search is b₁log₂n + b₂

We say that

• Linear search "grows like" n: O(n)
• Binary search "grows like" log₂n: O(log₂n)

You can think of O(n) as "order n", but what it really means is:

There exists some constants n₀, c > 0 such that for every n > n₀, actual time for n <= cn. (i.e., cn is an upper bound for sufficiently large values of n.

This definition can be generalized to any function f(n). Use O-notation to give an upper bound on the rate of growth of the running time. Typically, we want as slow a rate of growth as possible.

We usually focus on the worst case.

• Worst case time is the upper bound on actual time for any input.
• Gives a guarantee that the algorithm never takes longer.
• We don't need to make an educated guess and just hope it never gets much worse.

Why not focus on the average case?

• What is the average case? Take the example of searching.
  • We could usually find it early, for example in a video store when the most frequently requested videos are listed first.
  • We might usually find it halfway, like in a linear search with all values equally likely.
  • We might usually not find it, as if we were a typical Radio Shack store in which all they have are recordings like "Sound Effects of Turnips".
• The average case is often about as bad as the worst case. (But not always. Quicksort's average case is significantly better than the worst case.)

Sorting

Input: an array A[0..n-1].
Output: Rearrange the elements of A so that A[0] <= A[1] <= A[2] <= A[3] <= ... <= A[n-2] <= A[n-1].

In practice, we typically have satellite data that travels with the sort keys. For example, think about the final grades in this course. On December 10th, I will sort them by the key, which is the % of total points you received. The satellite data is your name. Those of you in the latter parts of the alphabet should certainly be happy to hear that the two items travel together. Otherwise, you would get stuck with all the worst grades in the class.

Sorting Algorithms

Random

Randomly rearrange the elements in A and check to see if they are in order. Free clue: It's a stupid algorithm. Nobody in their right mind would use it. There are n! ways to rearrange the elements of A, and there's no guarantee that we'd even stumble upon the sorted order in n! attempts.

Selection Sort

In selection sort, the idea is to scan through A, find the smallest element, and swap it with whatever happens to be in the place it should occupy. We then repeat this until we have picked the n-1 smallest elements. (At that point, we know the one left must be the largest one anyway.) Another way of stating it is that we maintain the loop invariant that in the array, A[0..i-1] holds the i smallest elements in sorted order, and that A[i..n-1] holds the largest n-i elements, but in an unknown order. Here's a brief example:

The basic idea is the following:

for (i = 0; i < n-1; i++) {
  smallest = index of least element in A[i..n-1];
  swap(A[i], A[smallest]);
  }

And to find the index of the least element in A[i..n-1]:

smallest = i;
for (j = i+1; j < n; j++)
  if (A[j] < A[smallest])
    smallest = j;

Putting these two pieces together, we get the heart of the code (see selectionSort.cpp for the whole program).

for (i = 0; i < n-1; i++) {
  smallest = i;
  for (j = i+1; j < n; j++)
    if (A[j] < A[smallest])
      smallest = j;
  swap(A[i], A[smallest]);
  }