Conceptual Basis for Why There May Be No Visual Lag for Moving Objects

According to some estimates, neural transmission delays in the visual system should cause an observer to be conscious of physical events that occurred close to 100 msec in the past (DeValois and De Valois 1991). If these neural delays translated directly into visual delays for moving objects, then, for example, a car moving at 30 mph should appear 4.4 feet "behind" in its trajectory. How, then, are ball players able to catch balls traveling at speeds much greater than 30 mph? (It is known that ball players are able to intercept balls that require timing precision of 10-15 msec; Lee and Young 1985). The traditional answer is that the motor plans set up by the observer take these various delays into account, but recently a new possibility has been raised. Several researchers (Ramachandran and Anstis 1990; De Valois and De Valois 1991; P. Cavanagh 1995, personal communication) have suggested that there might be a mechanism within the visual system which compensates for these delays. Thus, a moving object's current actual location and its visual location may be closer to one another than one might expect on the basis of neural delays and pictures like figure 13.1. According to this view, the compensatory motor plans are necessary for overcoming delays due to movement of body parts per se, and that visual delays do not depend on motor plans for

Figure 13.1

A scene depicting a car traveling from left to right. The grids schematically represent the photoreceptor layer and the primary visual cortex. The vertical lines represent neural activity. If perception occurs on the basis of cortical information, then the perceived position of the moving object should lag its actual position.

Figure 13.1

A scene depicting a car traveling from left to right. The grids schematically represent the photoreceptor layer and the primary visual cortex. The vertical lines represent neural activity. If perception occurs on the basis of cortical information, then the perceived position of the moving object should lag its actual position.

compensation. This thesis is based on an illusion in which moving elements visible within a window cause the window to appear shifted in the direction of the motion of the elements, that is, in the direction that would most likely be occupied by the moving elements in the immediate future (figure 13.2).

This interpretation of the "movement-based positional bias," however, seems at odds with some other findings. First, the displacement effect has been found to increase rapidly with increased eccentricity, being relatively weak or absent in the fovea (De Valois and De Valois 1991). In interceptive behavior, for which compensation of visual delays would be crucial, the animal attempts to foveate the moving target of interest. Clearly, there are significant delays from the fovea to the cortex, and a mechanism that compensates for peripheral delays, but not foveal delays, is not very plausible. Second, the positional bias occurs only if target (window) boundaries are equiluminous with the background or, more generally, if sharp luminance-based edges are missing from the stimulus. These conditions are rarely, if ever, satisfied by moving objects in the natural environment.

A fundamental goal of the visual system, unraveled by measurements of retinal ganglion cell responses by Kuffler (1952), is to determine which part of the retina has been stimulated in order to ascertain the visual field location of the object responsible for the stimulation. Might a similar principle hold for objects whose retinal image is in motion? Let us entertain the possibility that this principle holds, and that there is no lag in the visual position of moving objects. The proposal is that the correct instantaneous position of the moving object is determined by the visual system (on the basis of the past trajectory and speed of the object) and given directly in the visual percept of that object (Nijhawan 1994). This implies that the instantaneous visual position of moving objects is already correct, so no further compensation need be postulated.

Figure 13.2

Schematic depiction of the dynamic stimuli used in the study. The patches themselves were stationary within which elements moved. The actual stimulus was (1) a moving sinusoidal grating within a gaussian envelope (DeValois and DeValois 1991) or (2) kinetic edges created by moving random dots in a field of static dots (Ramachandran and Anstis 1990).

Figure 13.2

Schematic depiction of the dynamic stimuli used in the study. The patches themselves were stationary within which elements moved. The actual stimulus was (1) a moving sinusoidal grating within a gaussian envelope (DeValois and DeValois 1991) or (2) kinetic edges created by moving random dots in a field of static dots (Ramachandran and Anstis 1990).

Let us first tackle a few questions about this "no-lag" conjecture: Is there any conceptual basis for why there may be no lag for a moving item's visual location relative to its actual location? Suppose we do indeed find no lag in the visual position of moving objects. What relevance would this fact have for the study of vision? If a type of "compensation" does occur, what are the possible underlying neural mechanisms that might be responsible for this? Let us consider the first two questions first and defer the consideration of the third to a later section.

The question that must now be faced head-on concerns the meaning of the statement: There is a mismatch between an object's actual location and its perceived location. In this case "actual location'' can have at least two different meanings. One meaning becomes apparent when an

Figure 13.3

A wedge prism bends the path of light such that the visible object is displaced toward the wedge.

Figure 13.3

A wedge prism bends the path of light such that the visible object is displaced toward the wedge.

observer views a stationary object, say localized directly in front, and then closes his or her eyes before reaching for it. If the reach is successful— that is, if the reach first produces a tactile sensation at the observer's hand, followed by some benefit (e.g., nourishment), then we might say that the object's actual position and its visual position are in agreement. One easy way of causing this mapping between vision and pro-prioception to temporarily breakdown is to place a wedge prism between the object and the observer's eye (figure 13.3). If the observer now tries to reach for the object with eyes closed, his or her reach will miss the object. We might say this is because the object's actual position and its visual position no longer agree.

A second meaning of "actual location,'' more relevant to the current topic, is the one associated with the direction in which the light from a given object travels to the eye. Consider light entering the eye at a certain angle from a point source. If the point is seen in the direction of the path of light, then one might say that the point is visible in its "actual location." One common physical situation where the two meanings of the term "actual location" produce different answers is that of the transmission of light from a distant object, such as the sun, to the eye. Neural delays in this case are unimportant because the angle of the sun changes too slowly. It takes light about eight minutes to travel from the sun to Earth; thus one might say that the visual location of the sun is different from its actual location due to Earth's rotation. Therefore, the sun is not seen in its actual location if one considers its material aspects, such as its mass, as representing the term "actual." At the same time one may argue that the sun is visible in its actual position because it is perceived in the direction along which its rays enter the eye, and not in some other direction. Note the association between the material location of a nearby object and the tactile or proprioceptive sensation it has the potential to produce (Cutting and Vishton 1995), and that between the optical location of an object and the retinal image it projects.

Let us call the meaning associated with a given object's material aspects the object's actual (proprioceptive) location, and that involving the path of light, the object's actual (retinal) location. Which of these two meanings of "actual location'' does one imply when speaking of neural transmission delays causing a moving object to appear in a position that lags relative to its actual instantaneous position? We believe that if the implied lag were between a moving object's actual (proprioceptive) location and its visual location, then the problem would not be so serious, because there are numerous situations (e.g., figure 13.3) where this can occur. Furthermore, an observer can adapt (within about two weeks for extreme prism distortions, such as retinal image inversion) to a dissociation between the visual and the proprioceptive locations of objects and learn to get around in the world (Stratton 1896, 1897). Examples of dissociation between an object's actual (retinal) location and its visual location are, however, not easy to find, and there are good indications that this aspect of vision is not modifiable (Harris 1963). Anatomical considerations also suggest that the relationship between a stimulated retinal location (local sign) and the direction in which the stimulus is visually localized is not modifiable (Kaufman 1974). Moreover, findings with congenitally blind children (Schlodtmann 1902) support an innate determination of the connection between the retinal local sign and perceived visual direction. Thus, the connection between the locus of the retina stimulated by an object and the direction in which the object is visible seems innately determined and unmodifiable.

Now consider the implied visual lag for moving objects. In this case not only should the perceived position of the object and the stimulated retinal position disagree, but this disagreement should survive despite the observer's years of experience with the visual environment. Visual lag for moving objects would mean, for example, that an object moving rightward past a vertical line would, during its trajectory close to the line, be visible to the left of the line even after its retinal image has crossed over the retinal image of the line (figure 13.4). It seems simpler to assume that the theory of local sign (Lotze 1971; Rock 1975: 159-183) holds for stationary as well as moving objects, with the "no lag'' conjecture a direct consequence of this.

Many experiments have been devoted to the study of adaptations that occur in situations of potential conflict between sensory data, for example, that existing between sound and vision (e.g., ventriloquist effect) or between vision and proprioception. However, the implied dissociation between a moving object's instantaneous visual position and its retinal image position, or the lack there of, has not received adequate attention. This is particularly surprising because the crux of the unresolved debate in the 1960s surrounding the issues of adaptation was between scientists who believed that adaptation

Figure 13.4

Top view of an object moving rightward past a stationary vertical line. The eyeball is stationary. At the instant depicted, the retinal image of the moving object has already crossed the retinal image of the line while the visible moving object has not yet crossed the visible line.

Figure 13.4

Top view of an object moving rightward past a stationary vertical line. The eyeball is stationary. At the instant depicted, the retinal image of the moving object has already crossed the retinal image of the line while the visible moving object has not yet crossed the visible line.

produces a remapping between the retinal image position and the perceived position of objects, and those who opposed such a view (see Harris 1965 for a review).

Suppose it is found that there is no disagreement between a moving object's instantaneous visual location and the local sign. How would this fact be important? First, this would suggest that there is some "corrective" mechanism within the visual system which compensates for the neural transmission delays and establishes a one-to-one correspondence between a moving object's perceived location and its instantaneous local sign. This would certainly have implications for the design of the visual system. Second, this would suggest that smooth motion can essentially be used as an objective "clock'' relative to which other visual events can be timed. For example, it suggests a way of measuring visual delays in the processing of discrete events, such as flashes, which cannot be done with methods like response time measurement. Finally, one could empirically study the consequences of overlap (or lack there of) of two stimuli (one moving and one flashed) in some "high level" representation of space (e.g., VI), while at the same time the two retinal images producing those representations do (or do not) overlap.

Experimental Support for the "No-Lag" Conjecture for Moving Objects

Consider the visual delays involved in the perception of a single flash of white light projected to the observer's fovea. The neural signals generated by the flash must first travel to the ganglion cells and then to the lateral geniculate nucleus on their way to the visual cortex. Suppose, in addition, we now present a moving object that passes the location of the flash at the exact instant of the flash. What observers report in this case is a location mismatch between the two objects, with the flashed object in a lagging position relative to the moving object. One form of this experiment employs a single white rod moving from left to right and two light sources. One of the lights, which illuminates only one section of the rod, is a stroboscopic source capable of producing a 1 microsec flash at a predetermined time. The rest of the rod is continuously illuminated by the other light source, which may be an ordinary lightbulb. A given section of the rod is illuminated by one and only one source. The observer fixates a given location as the moving rod (with some sections continuously illuminated) approaches this location. When the rod is in the neighborhood of the fixation position, the other source briefly illuminates the rest of the rod. What observers see is a rod "broken" into segments, with the flashed segments lagging behind the continuously illuminated segments (figure 13.5a).

In one method of quantitatively measuring this phenomenon, which for simplicity we call the "flash-lag" effect, the observer cancels the effect by positioning the flashed segments physically "ahead" of the continuous segments (Nijhawan 1994). Using rotary motion, it has been found that the flash-lag effect increases monotonically with the speed of the moving item (figure 13.5b). This phenomenon may be explained as follows: The flashed segment of the bar is perceived in the correct location, although after some delay, due to transmission time of neural signals. The continuously visible segments of the moving bar are also seen in the correct location ("no-lag" conjecture). Thus, the flashed segment becomes visible after the moving segments (and their percepts) have passed the location of the flashed segment.

In experiments measuring response time to a flash, it is generally believed that the flash must first produce a visual percept; only then can the observer respond to it (however, see Goodale et al. 1991). However, using response time to measure the time it takes for the flash to become visible is like trying to measure the distance to a mountain surface, on the basis of echoes, without first knowing the speed of sound. In the flash-lag experiment the visual processing delay of the flashed segment is given by the magnitude of the flash-lag effect divided by the object velocity.

There are some additional noteworthy observations concerning the flash-lag phenomenon, of which we would like to mention two. The first has to do with contours that are visible to observers but are not given directly in the retinal image. There is considerable interest in contours that are visible although the conventional features, such as luminance discontinuities, are missing. Among the well-known examples is the Kanizsa triangle (figure 13.6a). We have found that the flash-lag phenomenon produces vivid contours that are not given in the retinal image, and yet these contours are unlike the illusory contours in that observers cannot distinguish them from "real" (luminance-based) contours. The flash-lag-based contours may, in fact, belong to a previously unknown category of contours.

Several experiments have produced these contours, of which the following is a good example. Observers viewed a white ring moving on a gray background while maintaining fixation. A brief flash of a disk "filled" the center of the ring when the ring was in the vicinity of the fixation point. Surprisingly, the perception of a "spurious" edge of a white disk against a gray background accompanied the report of the flash-lag effect (figure 13.6b). This "spurious" edge is treated by the visual system as if it were a real edge (Nijhawan manuscript under review; also see section "Flash-Lag Effect and Visual Attention''), a point that distinguishes it from illusory contours. It would be of interest also to see what

Angular Velocity (rpm)

15 25 35 45 Angular Velocity (rpm) b

Figure 13.5

(a) A snapshot of the stimulus at the instant of the strobe. (b) Data for two observers at increasing speeds of rotary motion. The vertical axis represents the perceived angle between the flashed and the moving segments.

Angular Velocity (rpm)

15 25 35 45 Angular Velocity (rpm) b

Figure 13.5

(a) A snapshot of the stimulus at the instant of the strobe. (b) Data for two observers at increasing speeds of rotary motion. The vertical axis represents the perceived angle between the flashed and the moving segments.

Figure 13.6

(a) The Kanizsa triangle, (b) A snapshot of the ring-disk display at the instant of the flash of the disk (dashed line) centered on the ring. The percept shows contours of the disk visible against the gray background.

Figure 13.6

(a) The Kanizsa triangle, (b) A snapshot of the ring-disk display at the instant of the flash of the disk (dashed line) centered on the ring. The percept shows contours of the disk visible against the gray background.

characteristics these contours may share with illusory contours. For example, do these contours, like the illusory contours, produce greater activation in area 18 of the monkey cortex than in area 17 (von der Heydt et al. 1984)? Does something special about illusory contours produce the asymmetric responsiveness of neurons between areas 17 and 18, or might the difference reflect a general difference between responses to contours given directly in the image and to those that are not?

A second observation concerns color vision. The connection between space and color vision has been evident since Newton's demonstration with the prism splitting white light into its components. The prism disperses different wavelengths over space (and the retinal surface), each wavelength triggering its own sensation of color. But is there a connection between a neural representation of visual space and color? As first suggested by Hecht (1928), this question is important for theories of color perception. He demonstrated a "binocular fusion of yellow'' with an experiment in which a red disk stimulated only one eye while a green disk stimulated the corresponding retinal region of only the other eye. In these conditions it is not obvious whether observers will see a single fused image or not, but fortunately they do. The fused red-plus-green disks produced the percept of a yellow disk, which is what results when the red and green disks are superimposed (additively mixed) on the retinal surface. Hecht's conclusion was that since neither disk alone can activate a "yellow" mechanism within the retina, the percept of yellow must be a result of cortical processes which are activated by both the disks together, that is, activation of binocular cortical cells stimulated by both disks. This outcome supports the Young-Helmholtz-Maxwell theory of color vision (however, see Hurvich and Jameson 1951).

It would be of interest to see if the reverse is also true, that is, red and green stimuli superimposed on the retinal surface appearing separated in space, and failing to "mix" into yellow. The flash-lag effect was employed to study this issue (Nijhawan 1997). Observers viewed a green bar revolving against a dark background, while a horizontal red line was flashed such that it was

Stimulus: ^ Moving Green bar with flashed Red line

Moving Green bar's "previous location

Cut out: moving bar's location behind the screen prior to Ihe flash of red line s

Slit

Opaque screen with a slit used in brief exposure condition

Percept

Figure 13.7

The stimulus and the percept are shown at the left. Downward arrows represent the green bar's motion direction. The right side of the figure schematically shows the opaque screen used in the brief exposure condition. At the instant depicted, the downward-moving green bar is invisible to the observer, being occluded by the screen.

optically superimposed on the bar (figure 13.7). There were two viewing conditions: extended exposure and brief exposure. For the brief exposure condition the green bar was briefly flashed such that it elicited no motion signal. (In this condition the bar was exposed only through a narrow slit within an opaque screen that occluded the rest of the bar's motion trajectory.) In this condition observers adjusted the color of the flashed line (by adjusting its intensity) until it appeared yellow. In the extended exposure condition, observers viewed this same stimulus, but now the screen was removed and the bar's motion was visible. In this condition the flashed line appeared to lag the green bar, and against the dark background its color appeared reddish. Thus, despite equal quantal absorption at the photoreceptors, the color of the flashed line was perceived as more greenish in the brief exposure condition and more reddish in the extended exposure condition. This finding adds to Hecht's analysis and suggests that retinal superimposition of red and green is neither necessary nor sufficient for producing the sensation of "yellow.''

Was this article helpful?

0 0

Post a comment