Practical Exploration of Randomized Features For Classification Tasks

Abstract

In the field of machine learning, kernel methods have risen to become a very popular tool to enable learning algorithms to detect very general types of relationships. Kernels do this by implicitly lifting the data into a feature space of higher dimension, and then taking an inner product. Despite this power, because of the implicit mapping that is performed, kernel methods usually scale poorly with the size of the input [1]. To address this problem we employ a method, due to Rahimi and Recht [7], called random Fourier features that computes random projections of the input data into a feature space that approximates the kernel, thereby making the runtime linear in the input. This method allows us to analyze datasets that are too large for kernel methods. We explore the applications of this method on two datasets, MNIST and SVHN, both of which have sizes on the order of 105.