Principal Curves and Surfaces

• Abstract

  Principal, saddle, and minor surfaces are critical surfaces of various dimensions that underlie functions which represent data density over some suitable space; this could be color or other feature distribution over space for images or probability density for arbitrary features in a statistical machine learning setting. We present a generalization of the local maximum, minimum, or saddle point concept as the fundamental definition of such surfaces in an attempt to frame the problem of manifold learning (and related problems with various names) as a geometrical structure identification problem.

• Description

  Principal, saddle, and minor surfaces are critical surfaces of various dimensions that underlie functions which represent data density over some suitable space; this could be color or other feature distribution over space for images or probability density for arbitrary features in a statistical machine learning setting. We present a generalization of the local maximum, minimum, or saddle point concept as the fundamental definition of such surfaces in an attempt to frame the problem of manifold learning (and related problems with various names) as a geometrical structure identification problem. Our definition of the principal surface of a function is as follows: a point is an element of the principal surface if that point is a local maximum of the function in the orthogonal subspace at that point. This modifies Hastie\'s definition of self-consistent principal surfaces by replacing the mean-in-the-orthogonal-subspace with local maximum. The advantages are: (i) the definition is local and is only concerned with local derivatives of the function, (ii) it is trivial to define other critical surfaces such as saddle and minor surfaces, which generalize critical points (0-dimensional critical surfaces) to any dimensionality, (iii) since given the function (for instance the pdf) we can calculate the gradient, Hessian, etc at any point, we can know whether a point is on this principal surface or not without solving for the whole surface (as in manifold learning for instance). This latter property is very useful in designing manifold learning algorithms as well as dimension reduction techniques. The definition also easily handles bifurcations, graphs, trees that underlie the structure; most manifold learning algorithms assume a globally smooth manifold and fail to handle singularities easily or rigorously.

• Team Members