ASA058
the K-Means Problem

ASA058 is a C library which seeks solutions of the K-Means problem, by David Sparks.

ASA058 is Applied Statistics Algorithm 58. Source code for many Applied Statistics Algorithms is available through STATLIB.

In the K-Means problem, a set of N points X(I) in M-dimensions is given. The goal is to arrange these points into K clusters, with each cluster having a representative point Z(J), usually chosen as the centroid of the points in the cluster. The energy of each cluster is

        E(J) = Sum ( all points X(I) in cluster J ) || X(I) - Z(J) ||^2
      

For a given set of clusters, the total energy is then simply the sum of the cluster energies E(J). The goal is to choose the clusters in such a way that the total energy is minimized. Usually, a point X(I) goes into the cluster with the closest representative point Z(J). So to define the clusters, it's enough simply to specify the locations of the cluster representatives.

This is actually a fairly hard problem. Most algorithms do reasonably well, but cannot guarantee that the best solution has been found. It is very common for algorithms to get stuck at a solution which is merely a "local minimum". For such a local minimum, every slight rearrangement of the solution makes the energy go up; however a major rearrangement would result in a big drop in energy.

A simple algorithm for the problem is known as "H-Means". It alternates between two procedures:

• Using the given cluster centers, assign each point to the cluster with the nearest center;
• Using the given cluster assignments, replace each cluster center by the centroid or average of the points in the cluster.

A more sophisticated algorithm, known as "K-Means", takes advantage of the fact that it is possible to quickly determine the decrease in energy caused by moving a point from its current cluster to another. It repeats the following procedure:

• For each point, move it to another cluster if that would lower the energy. If you move a point, immediately update the cluster centers of the two affected clusters.

Note: the original reference lists the input variable F as an integer workspace array. However, F is used in the CLUSTR routine exclusively as a real array. Even in single precision, this causes the routine to compute incorrect results (try it, please!); in double precision it also causes memory overwrites. The code presented here has corrected this mistake.

Licensing:

The computer code and data files described and made available on this web page are distributed under the GNU LGPL license.

Related Data and Programs:

asa058, a C library which implements the K-means data clustering algorithm, by David Sparks. This is a version of Applied Statistics Algorithm 58.

Source Code:

• asa058_test.c
• asa058_test_input.txt coordinates of 100 points in 2D.
• asa058_test.txt the output file.