
Dimension Reduction and Feature Selection
Linear and nonlinear dimension reduction methods can allow highdimensional data to be visualized in 2 or 3 dimensions,
as well as create more compact features for a variety of purposes.



NonNegative Matrix Factorization, Convexity and Isometry
Nikolaos Vasiloglou, Alexander Gray, and David Anderson
SIAM Data Mining (SDM) 2009
A nonnegative matrix factorization (NMF) method which also preserves distances like a manifold learning method, based on semidefinite programming.
We also show how NMF can be performed as a convex optimization for the first time.
[pdf]
Abstract:
Traditional NonNegative Matrix Factorization (NMF)
is a successful algorithm for decomposing datasets into
basis functions that have reasonable interpretation. One
problem of NMF is that the original Euclidean distances
are not preserved. Isometric embedding (IE) is a manifold
learning technique that traces the intrinsic dimensionality
of a data set, while preserving local distances. In this
paper we propose a hybrid method of NMF and IE, IsoNMF.
IsoNMf combines the advantages of both NMF and Isometric
Embedding and it gives a much more compact spectrum
compared to the original NMF.
@Inproceedings{vasiloglou2009isonmf,
Author = "Nikolaos Vasiloglou and Alexander Gray and David Anderson",
Title = "{NonNegative Matrix Factorization, Convexity and Isometry}",
Booktitle = "{SIAM International Conference on Data Mining (SDM)}",
Year = "2009"
}

See also
QUICSVD: Fast SVD Using Cosine Trees
We have developed a method for fast approximate singular value decomposition and demonstrated it for principal component analysis.
[see full entry here]
FuncICA: ICA for Time Series
We have developed a variant of independent component analysis for functional data such as time series.
[see full entry here]
In preparation
Volumetric Manifold Learning
We have developed a new approach to manifold learning based on ideas from Riemannian geometry.
Decision Trees for Density Estimation
We have developed the analog of classification or regression trees, for density estimation, which yields automatic feature selection among other benefits.


Scalable Semidefinite Manifold Learning
Nikolaos Vasiloglou, Alexander Gray, and David Anderson
Machine Learning and Signal Processing (MLSP) 2009
Fast algorithms for maximum variance unfolding and related manifold learning methods. We've demonstrated up to 1 million points recently.
[pdf]
Abstract:
Maximum Variance Unfolding (MVU) is among the state of
the art Manifold Learning (ML) algorithms and experimentally
proven to be the best method to unfold a manifold to its
intrinsic dimension. Unfortunately it doesn't scale to more
than a few hundred points. A non convex formulation of
MVU made it possible to scale up to a few thousand points
with the risk of getting trapped in local minima. In this
paper we demonstrate techniques based on the dualtree
algorithm and LBFGS that allow MVU to scale up to 100,000
points. We also present a new variant called Maximum Fur
thest Neighbor Unfolding (MFNU) which performs even
better than MVU in terms of avoiding local minima.
@Inproceedings{vasiloglou2009mvu,
Author = "Nikolaos Vasiloglou and Alexander Gray and David Anderson",
Title = "{Scalable Semidefinite Manifold Learning}",
Booktitle = "{IEEE International Workshop on Machine Learning For Signal Processing (MLSP)}",
Year = "2009"
}


Learning Dissimilarities by Ranking: From SDP to QP
Hua Ouyang and Alexander Gray
International Conference on Machine Learning (ICML) 2008
The first manifold learning method to preserve the ordering of the distances rather than
their numerical values. The result can often work where the standard approach cannot.
[pdf]
Abstract:
We consider the problem of learning
dissimilarities between points via formulations
which preserve a specified ordering between
points rather than the numerical values of
the dissimilarities. Dissimilarity ranking
(dranking) learns from instances like "A is more
similar to B than C is to D" or "The
distance between E and F is larger than that
between G and H". Three formulations of
dranking problems are presented and new
algorithms are presented for two of them, one
by semidefinite programming (SDP) and one
by quadratic programming (QP). Among the
novel capabilities of these approaches are
outofsample prediction and scalability to large
problems.
@Inproceedings{ouyang2008metric,
author = "Hua Ouyang and Alexander Gray",
title = "{Learning Dissimilarities by Ranking: From SDP to QP}",
booktitle = "Proceedings of the Twentyfifth International Conference on Machine Learning",
year = "2008"
}

