Scaling Bayesian Optimization for High-Dimensional Iterative Experimental Design

Abstract

Bayesian optimization (BO) is an intelligent search technique for optimizing expensive nonlinear black-box objective functions. It is tempting to apply BO to iterative experimental design in the physical sciences. But these scenarios often have high dimensionality, presenting two problems: first, the large amount of time the algorithm takes to generate suggestions for subsequent experiments, and, second, the prohibitively large number of expensive experiments needed to thoroughly search the parameter space. We present a new approach to mitigate both issues with Bayesian optimization for high-dimensional problems. The proposed solution involves changing the prior model of the black-box objective function and developing a local optimization approach for the acquisition function. We evaluate these solutions on high-dimensional optimization tasks and on a computational analogy to a biological experimental design task: CRISPR/Cas9 guide RNA sequence optimization. In categorical spaces, the random forests prior model leads to fast convergence, whereas in continuous spaces, the Gaussian process prior performs best. However, random forests generate suggestions much more quickly than Gaussian processes. Local optimization (LO) improves performance across the board in exchange for a small constant time increase. When gradient information is available, gradient descent methods with momentum accentuate this performance improvement.