A Memory-Augmented Neural Network Model of Abstract Rule Learning

Abstract

The ability to extrapolate knowledge from familiar to novel domains is a defining feature of human intelligence. Contemporary neural network techniques, however, are primarily limited to interpolation among data in their training experience. In this work, we focus on neural networks’ capacity for arbitrary role-filler binding, the ability to associate abstract “roles” to context-specific “fillers,” which is a capacity that many have argued is an important mechanism underlying the ability to extrapolate. Using a simplified version of Raven’s Progressive Matrices, a hallmark test of human intelligence, we introduce a sequential formulation of a visual problem-solving task that requires this form of binding. Further, we introduce the Arbitrary Binding Network, a recurrent neural network model augmented with an external memory, and empirically demonstrate that it successfully learns the underlying abstract rule structure of our task and perfectly generalizes this rule structure to novel fillers.