Supervised Learning Classification, regression, and relatives comprise the most widely-used type of statistical task. fastlab Big Ideas People Stuff Isometric Separation Maps Nikolaos Vasiloglou, Alexander Gray, and David Anderson Machine Learning and Signal Processing (MLSP) 2009 An approach to learning the kernel for support vector machines, which can guarantee linear separability in the kernel space. [pdf] Abstract: Maximum Variance Unfolding (MVU) and its variants have been very successful in embedding data-manifolds in lower dimensional spaces, often revealing the true intrinsic dimension. In this paper we show how to also incorporate supervised class information into an MVU-like method without breaking its convexity. We call this method the Isometric Separation Map and we show that the resulting kernel matrix can be used as a binary/multiclass Support Vector Machine-like method in a semi-supervised (transductive) framework. We also show that the method always finds a kernel matrix that linearly separates the training data exactly without projecting them in infinite dimensional spaces. In traditional SVMs we choose a kernel and hope that the data become linearly separable in the kernel space. In this paper we show how the hyperplane can be chosen ad hoc and the kernel is trained so that data are always linearly separable. Comparisons with Large Margin SVMs show comparable performance. @Inproceedings{vasiloglou2009ism, Author = "Nikolaos Vasiloglou and Alexander G. Gray and David Anderson", Title = "{Learning Isometric Separation Maps}", Booktitle = "IEEE International Workshop on Machine Learning For Signal Processing (MLSP)", Year = "2009" } New Algorithms for Efficient High-Dimensional Nonparametric Classification Ting Liu, Andrew W. Moore, Alexander G. Gray Journal of Machine Learning Research (JMLR), 2006 Fast exact algorithms for nearest-neighbor classification (and related problems), exploiting the fact that solving this does not strictly require finding the nearest neighbors and demonstrating speedups in higher dimensionalities than typical for nearest neighbor search. [pdf] Abstract: This paper is about non-approximate acceleration of high-dimensional nonparametric operations such as k nearest neighbor classifiers. We attempt to exploit the fact that even if we want exact answers to nonparametric queries, we usually do not need to explicitly find the data points close to the query, but merely need to answer questions about the properties of that set of data points. This offers a small amount of computational leeway, and we investigate how much that leeway can be exploited. This is applicable to many algorithms in nonparametric statistics, memory-based learning and kernel-based learning. But for clarity, this paper concentrates on pure k-NN classification. We introduce new ball-tree algorithms that on real-world data sets give accelerations from 2-fold to 100-fold compared against highly optimized traditional ball-tree-based k-NN. These results include data sets with up to 10^6 dimensions and 10^5 records, and demonstrate non-trivial speed-ups while giving exact answers. @article{liu2006nnclsf, Author = "Ting Liu and Andrew W. Moore and Alexander G. Gray", Title = "{New Algorithms for Efficient High Dimensional Nonparametric Classification}", journal = "Journal of Machine Learning Research (JMLR)", volume = "7", pages = "1135--1158", year = "2006" } Isee elsewhere Optimization of Kernel Machines We are developing efficient optimization methods for regularized kernel machines, including support vector machines. [see webpage here] Iin progress Feature-Selecting Support Vector Machines We are exploring efficient methods for learning L0 and L1 sparse support vector machines.