Bucket Sort

Intro

Sorting in expected O(n) time into buckets.

Bucket sort or Bin sort is generalized to Radix sort.

However Bucket sort can also be generalized with linked list to not only sort into buckets but sort continuously and have O(n) time. The radix sort is discrete.

Implementation

Assuming a random distribution and a known interval length, dividing the interval into n containers for n points we can take advantage of calculating which container a point lies in because this calculation is constant in time.

Let $v_{i}$ be a vector of n elements.
$x_{0}$ be the least and $x_{1}$ be the greatest element.
$f (x) = ⌊ \frac{x - x_{0}}{x_{1} - x_{0}} n ⌋$

Let $c_{i}$ be a counter of $n$ intervals.
$for i = 0 .. n - 1$
$++ c_{f (v_{i})}$

Let $cp_{i}$ be the start position of the interval.
$s = 0$ $cp_{0} = 0$
$for i = 0 .. n - 1$
$s += c_{i - 1}$
$cp_{i} = s$

Now iterate through v inserting which bucket the counter points to. As an element is inserted that counter is increased. So all the elements are inserted into the correct buckets.

I implemented this in a crude fashion - see C++ nbody in the cell.h file. It has been designed to sit in cache so I avoided STL.

Bucket Sort with Linked Lists

This is like a hash table. In fact it is a hash table with the key as the bucket. All you do is order on insertion the element into the bucket which is O(k).

But here is where it gets good. If O(k) is bounded in time then it is constant in time relative to O(n). In practice the maximum bucket length is small. In this situation you can then expect O(n) insertion time into the buckets.

For those of you who have not heard of hashed arrays they are the fasted data structure by far because they have constant access and retrieval time which is magic for algorithms. Their weakness is that the data is unordered.

To say just how good they are I had this school assignment with a dictionary that took around 25 minutes of execution time with balanced trees, the same execution took 30 seconds with a hashed data structure with the same code. The moral of the story is this - it does not matter how badly you wrote your code if it executes in constant time.

Back to the topic, the bucket sort with linked lists is easy to implement and can sort in O(n) time on continuous data.

For unfriendly data you could add more buckets and change the index function.

I implemented this and experimentally verified the O(n) relationship. See C++ bucket.

Hybrid Bucket Sort

The problem with the Bucket sort is that if the data is unfriendly it becomes a O(n^2) instead of O(n). Now in practices increasing the bucket length is really good for most applications and you can change the hash function yourself. But to the theoretical mathematician and computer scientist there is a problem - an algorithm which can become O(n^2) is unacceptable.

There are situations where O(n^2) can be made acceptable by reordering the data in a random way, unfortunately this does nothing for the bucket sort as the sorter's complexity is independent of the data it sorts.

However combining a bucket sort with another sorter which uses a balanced tree for O(nlogn) is possible. So when the data starts behaving in an unfriendly way use the second default sorter.

The worst case was detected by having a maximum number of links in a bucket. When this was exceeded the whole bucket was inserted into the default sorter. As this is itself a linear operation the sorting complexity is not changed, but is the sum of a linear insert and the complexity of inserting into the default sorter.

This strategy gives the expected complexity of the sorter as O(n) and the worst case complexity as O(nlogn).

Here is the maths in the worst case. Let k be the maximum number of links. Assume that inserting into a bucket is O(k) but since k is much smaller than n and independent of n then O(k)=O(1).
Iterating though n data elements and inserting in either the bucket list or the default sorter expressed with complexities as an equation gives
O(n)(O(k)+O(logn))
=O(n)(O(1)+O(logn))
=O(nlogn)

See C++ bucket.

Hash Tables

These generally use buckets too as this solves the collision problem and memory is cheap.

They should run in O(n) time. On my old P3 O(n) insertion and access of integers up to around 2,000,000 integers observed. Where I am making the hash table bucket size equal to the data size that the table will hold.

They can also be used as maps. For example have a string and a counter as a data type with the hash on the string. Inserting string and counter pairs into the hash table, updating a counter could retrieve the same data string with an empty counter and then alter the counter with the retrieved pointer.

Hash tables depend on hash functions. If you have some idea of your applications data distribution you could build a hash function to suit it. This is more critical for the bucket sort which hashes and sorts too. Generally people have really good hash functions with bit operators. There is heaps on the net.

Integer Sorting

I am concerned with sorting unique integer sequences.

Here is an algorithm for sorting integers in a range.

Suppose that I wish to sort k integers with an array of n length. Further suppose that each integer i satisfies 0 < i < n. So the array is marked 0 if the integer is not present, else 1.

Sorting then consists of iterating over k and assigning a 1 to the corresponding array element.

While sorting is linear O(n) for small k O(klog(k)) < O(n). So a general O(klog(k)) sorter could be more effective in these situations.

Alternatively use the bucket sort with linked lists and insert linearly in the correct order. Since the linked lists are small insertion is constant in time.

If the distribution is not random then make it random. For example I have a data structure with a lot of consecutive integers which represent points. These are just some of the points in a global points container. By re labeling the points by means of a random shuffle of all the points in the global container, the data structure now has a random distribution of integers to the points and this is suitable for the bucket sort with linked lists.