Spatial Vision: Single Neurons and Perception
W. S. Geisler and D. G. Albrecht

The topic of spatial vision concerns the fundamental mechanisms within the eye and the brain that analyze and represent the distribution of light across the visual field, with the ultimate goal of understanding how these mechanisms contribute to object recognition and scene interpretation in general.

A wealth of psychophysical and physiological research supports the view that stimulus selectivity plays a fundamental role in spatial vision. Psychophysical studies have provided evidence that the human visual system is selective along a number of stimulus dimensions including orientation, size, position, wavelength (color), speed of motion, direction of motion, and binocular disparity. These studies have shown that there are mechanisms ("channels") selective to different regions along each of these stimulus dimensions. Similarly, neurophysiological and anatomical studies have demonstrated that neurons in the visual pathway are selective along a number of stimulus dimensions and that this selectivity increases from the retina to the primary visual cortex. For example, photoreceptors are selective along a few stimulus dimensions (spatial position, wavelength, temporal frequency), whereas cortical neurons are selective along many stimulus dimensions (spatial position, wavelength, temporal frequency, orientation, spatial frequency, direction of motion, disparity, etc.). Concomitant with this increase in stimulus selectivity, there is an increase in the heterogeneity; that is, there is an increase in the complexity and diversity of the cells along all of the stimulus dimensions. Thus, for example, the intensity response functions of cones are all very similar from cell to cell, whereas the contrast response functions of cortical neurons are quite different from cell to cell.

A number of different explanations have been proposed for this emergence of stimulus selectivity along the visual pathway. One hypothesis is that this progressive selectivity is part of a hierarchical process, ultimately leading to single neurons which respond uniquely to specific real world objects (Barlow, 1972; Barlow, 1995): "The Neuron Doctrine." A second hypothesis is that this selectivity reflects a low redundancy code which is well matched to the statistics of natural images (Barlow, 1961; Barlow, 1989; Field, 1987; Olshausen & Field, 1997): "Sparse Coding."

An alternative hypothesis is that this selectivity is a critical step in segregating objects from their context: "Object Segregation." Objects of interest within the natural environment are generally located within a very complex context of other objects. In order to recognize an object of interest, the parts of the object must be separated from the parts of other objects. For example, to recognize a longhorn bull behind a barbed wire fence, it is necessary to separate the image features that define the wire fence from those that define the bull. Fortunately, context is much less of a problem on a local scale; it is relatively easy to identify the orientation, position, and color of local image contours. The selectivity of visual cortical neurons permits recognition of these local image properties, thus allowing subsequent grouping mechanisms to bind together the contours that define the fence separately from those that define the bull.

These three different explanations for stimulus selectivity are not necessarily incompatible. Object segregation or sparse coding could be a first step in producing single neurons tuned to real world objects. On the other hand, the processing following object segregation or sparse coding could be highly distributed. Further, having neurons matched to the statistics of the natural environment must surely be advantageous for both sparse coding and object segregation, given the constraint of limited resources. However, the goals of sparse coding and object segregation are quite different; hence, the specific selectivities, and how they are implemented, could well be different. It is important to keep all three explanations in mind, given that one's theoretical viewpoint can substantively influence the direction of future research.

In this chapter we will rely upon a wealth of psychophysical and physiological research to develop the topic of spatial vision with two themes in mind. The first theme concerns how stimulus selectivity develops along the visual pathway. The second theme concerns how the anatomical and physiological mechanisms of stimulus selectivity contribute to visual performance, and ultimately, object recognition and scene interpretation.