COMPUTING SCIENCE 210
Algorithm Analysis and Data Structures

Shell Sort

In 1959, D.L. Shell published "A High-Speed Sorting Procedure" [CACM 2(7):30-32] that was based on insertion sort, but that used multiple passes across the data, each pass sorting elements that were separated from each other by a fixed distance (or increment), gradually reducing the distance between the elements being compared until the last pass used an increment of 1 (like regular insertion sort). Thus, this sorting algorithm is also called diminishing increment sort.

For example, the first pass might sort elements that were 7 locations apart; the second pass, elements separated at a distance of 3; and the final pass, at a distance of 1. (A final pass with an increment of 1 is always needed to ensure that all elements are correctly sorted.)

Shell reasoned that a fast sorting algorithm would start out by moving out-of-order elements over large distances, then gradually move them by smaller distances, until they were in the correct place.

Sorting elements that are at some fixed distance from each other requires only a small revision of standard insertion sort — the introduction of a parameter to specify the increment to be used for the current pass. This increment then determines the step size for the loop that traverses the data and which elements are compared with each other.

ShellSort.java

Robert Sedgewick has done extensive analysis of Shell sort and suggests choosing increments from the sequence

1391376, 463792, 198768, 86961, 33936, 13776, 4592, 1968, 861, 336, 112, 48, 21, 7, 3, 1

Sedgewick also provides a very compact version of Shell sort (in C) which, like the algorithm above, is based on insertion sort, but not so obviously:

shell.c

Analysis of Shell sort is difficult, but its time complexity for some sequences of increments is known:

• For the sequence

  1 8 23 77 281 1073 4193 16577 ... 4^j+1+3*2^j+1

  the running time of Shellsort is O(n^4/3).

• For the sequence

  1 2 3 4 6 9 8 12 18 27 16 24 36 54 81 ...

  (which is constructing by starting at 1, appending 2 and 3 times each increment in the sequence), the running time of Shellsort is O(n log²(n)). This is the best known asymptotic performance of any increment sequence, but is not competitive in practice because it uses too many increments.

• For the sequence

  1 3 7 15 31 63 127 ... 2^j-1

  the running time of Shellsort is Θ(n^3/2), which is Θ(n √n).

The O(n^3/2) running time is achieved by most sequences of increments that approximate a geometric sequence, with the best running times (lowest constant factors) for growth rates between about 2 and 4.

• Knuth recommended the sequence

  1 4 13 40 121 364 1093

  because it is easy to calculate: each term is one greater than 3 times the previous term, so the diminishing increments can be calculated as

  incr = incr / 3

  (using integer arithmetic).

• Robert Kruse suggested using the formula

  incr = incr / 3 + 1

  beginning at n / 3 + 1. I have found this to be the fastest sequence of increments in my limited testing.

• Gonnet and Baeza-Yates suggest using

  incr = (5 * incr - 1) / 11

  for incr >= 5, based on empirical studies showing that a growth rate of 11/5 was best for geometric increment sequences.

Shell sort is a good sorting algorithm for sorting problems of moderate size, because it avoids Quicksort's O(n²) worst-case performance, yet sorts an array in place.

As Robert Kruse remarks in summary, "Never sell Shell sort short."