Lecture 24, November 18, 1996

Binary Search

In linearSearch4.cpp we saw how to find x using a sentinel placed on the end of a sorted array. And each time through the loop we eliminated just 1 possible index. But you know how to eliminate half of the remaining indices each time.

See Demo binarySearch.cpp

In this algorithm we maintain an invariant: If x is anywhere in A[0..n-1], then it is somewhere in A[low..high].
The invariant is initially true since low == 0, high == n - 1.
Maintained in each recursive call.
If low > high, subarray A[low..high] is empty, and therefore x is not found.

Analysis

Let s_i = interval size in the ith recursive call. Therefore, when we start out, s₁ = n.

To compute s_i+1 from s_i:

If s_i is odd, s_i+1 = (s_i-1) / 2 < s_i / 2.
If s_i is even, s_i+1 = s_i / 2 or s_i / 2 - 1.
Either way, s_i+1 <= s_i / 2.

So, now we have:

	s₁ = n
	s₂ <= s₁/2 = n/2
	s₃ <= s₂/2 = n/4
	s₄ <= s₃/2 = n/8
	.
	.
	s_i <= n/2^i-1

Now how many recursive calls k until s_k < 1, i.e., s_k = 0? We want n / 2^k-1 < 1. If we take the log of both sides we get log₂ n < log₂ 2^k-1 = k - 1. Therefore k > log₂ n + 1.

What integer value does k have? First, suppose that log₂ n is an integer. Because k must be strictly greater than log₂ n + 1, we choose k = log₂ n + 2. Now suppose that log₂ n is not an integer. Let q be the least integer greater than log₂ n. Then we choose k = q + 1.

Each recursive call makes <= 3 comparisons. Therefore, binary search makes <= 3 log₂n + 6 comparisons, even if x is not found.

Recall that log₂n grows much slower than n:

	n		     log₂n
	-		----------------
	1		       0
	1,000              approx 10
	1,000,000          approx 20
	1,000,000,000      approx 30

So it's better to have a log₂ n term than an n term.

More abstract analysis

We have been counting the number of comparisons. Is that enough? Is that too much? A lot goes on besides comparisons. How do we count it all?

We don't count it exactly. We just understand how the running time grows with the input size.

Consider linearSearch0. The running time is c₁n + c₂ for some constants c₁, c₂ > 0. This is because each loop iteration takes c₁ time, and there's a time cost of c₂ at the beginning and end.

How about linearSearch1? The time is d₁n + d₂ for some constants d₁, d₂ > 0 in the worst case.

We use the same sort of analysis for all the linear search functions in the worst case. In the worst case, each has to check all n input values, so the time is some linear function of n.

And what about the time for binary search? We make at most log₂n + 2 recursive calls. And it takes constant amount of time per recursive call plus the time for the calls that it makes. There's also some time to set up the first call. Therefore, the running time of binary search is b₁log₂n + b₂ for some constants b₁, b₂ > 0.

The Rate of Growth of Running Time

I will have much more to say on this in the future, but for now consider the rules for figuring the running time as:

Consider only the leading term
d₁n vs. b₁log₂n
since lower-order terms become insignificant for large values of n.
Ignore coefficients because constant factors are less important than the rate of growth for large inputs.

Timing Example

Consider a machine that will run a linear search. It can execute 100,000,000 instructions per second and when translation is done by the compiler it takes 2n instructions to search through n values. Also, consider another machine that will run binary search. It can execute only 1,000,000 instructions per second (it's 100 times slower) and when translation is done by the compiler it takes 25 log₂n instructions to search through n values. Lastly, assume that the size of our input array is 10,000,000 elements.

Linear Search Time = (2 * 10,000,000) / 100,000,000 = 0.2 secs
Binary Search Time = (25 log₂ 10,000,000) / 1,000,000 = 0.0006 secs

Even with a huge handicap, binary search turns out to be 333 times faster. So what is the moral of the story? First, constants are not as important as the order of growth of the time. Second, algorithms are a technology. You can have fast hardware and good compilers, but they are almost worthless without good algorithms.

To Index Previous Next