Deep Orientation

Brain-Like Orientation Invariance in Deep Nets

Colin Conwell (Harvard University) , George Alvarez (Harvard University)
January 5, 2020

Orientation Invariance

… in the Human Visual System

The ability to recognize objects despite substantial variation in the position, size, lighting and orientation in which those objects appear is a defining characteristic of biological visual intelligence often referred to simply as invariance or invariant representation. Orientation invariance – defined here as the representation of a stimulus that does not significantly vary as the stimulus is viewed from different angles – emerges abruptly in the human visual cortical information processing cascade (Morgan & Alvarez, VSS2014). In V3, for example, we observe little to no invariance: the difference between the neural patterns elicited by a given stimulus and the same stimulus rotated to 90 degrees is as large the difference between the neural patterns elicited by two different stimuli. In LOC, on the other hand, we observe strong invariance, with no statistically significant differences across the neural activity elicited by the same stimuli across any rotations.

… in Deep Neural Networks

Deep neural network models (computer vision algorithms defined by distributed computations in depth) have in the past been shown to capture the representational geometry of neural responses to different objects, but it’s still unclear whether they show the same types of invariance we observe in different parts of the human visual system. The question we ask, then, is: where, if anywhere, does orientation invariance emerge in deep neural networks (DNNs)?

fMRI Data

Participants in the fMRI study were shown a series of 8 stimuli at 5 different degrees of rotation (0, 45,90,135,180) – examples of which you can explore in the carousel below.