Current Issue	Past Issues	About YSM	Subscriptions	Advertisements	Contact Us

77.4 - Summer 2004

From the editor Articles News Departmental notes In Memoriam Where are they now? Undergrad profile Book reviews Opinions Web search 25, 50, 100 years ago Puzzle corner

Search YSM Articles

Advanced Search

> Summer 2004 > Articles

By Brandon Schneider

Like a complex algebraic equation, constructing an image in the brain from a visual stimulus is a process affected by many unknowns. The six brown squares on these two blocks are the same shade, but depending on each square’s visual context, our brain interprets its color to be very different from the others’. This is why, no matter how hard we try, we cannot extract the “lighter” brown square from its shadowed context, and thus, it appears lighter. (Credit: R. Beau Lotto)

Seeing is not like capturing an image on film. The information entering the eye must be processed by the brain before a representation of the world can be created. Visual illusions show that people may perceive something different from what really exists in the world.

“You see [ambiguous displays] in two different ways, but what is coming into your eyes is constant,” says Marvin Chun, professor of psychology and neuroscience. “They flip back and forth, because the brain is actively trying to interpret what it is seeing.”

Ambiguity can result from the theoretically impossible task the brain must accomplish: reconstructing a three-dimensional representation from a two-dimensional image formed from light concentrated on the retina. “It is like an indeterminate algebraic equation,” Chun explained. “There are too many unknowns, and in theory, there are an infinite number of objects that can give rise to any given 2-D image on the retina.”

Yet even with this seeming deficiency of information, the amount of stimuli hitting the retina at any given moment is almost too much to process. “If you were to break the visual input into the individual bits of information needed to fully characterize it, it would be absolutely overwhelming,” said Brian Scholl, assistant professor of psychology. Paradoxically, the visual system must deal with input that is simultaneously too sparse and too rich.

Since the brain only has a finite amount of computing power, it must be highly selective about what information gets processed in what ways. This selectivity is implemented via the operation of attention. But what does that mean for things we do not attend to? Does the brain not process them at all? Are they processed outside of our awareness? Research conducted in the labs of Chun and Scholl show that we do in fact subconsciously process such images. Areas of the brain are still stimulated even by images that subjects report not seeing.

The phenomenon of visual attention tunnels one’s perception into such a restricted scope that extremely salient objects can go unnoticed. This effect is called “inattentional blindness.” A series of experiments conducted by Scholl and Steven Most, a postdoctoral researcher in Chun’s lab, have explored the phenomenon.

In these experiments, subjects are seated before a computer screen that shows a number of black and white shapes. All of the shapes move in random trajectories, and the subjects are given a “dummy” task which requires attending to only some of the shapes — for example, keeping track of four of them as they move, or counting the number of times that only the black shapes touch a line dividing the screen in half. The critical part of the experiment lies in the fourth trial when a bright red cross appears and travels for five seconds horizontally across the screen. When questioned afterward, about thirty percent of the subjects fail to recall the cross. They have no idea it existed.

“This should shock you,” said Scholl. “The cross has a completely novel color, a completely novel shape, a completely novel type of trajectory, is fully visible, literally right under your nose for five seconds, and people don’t see it.” Though seemingly impossible, this is actually a very strong phenomenon. Scholl recalls a subject who, in the middle of being debriefed, exclaimed, “Wait a minute… this is the experiment, isn’t it? You’re trying to convince me that there was a cross there!”

Scholl, Rachel Sussman TC ’04, and Nicholaus Nole GRD ’08 extended the same paradigm to study how inattentional blindness might affect cell-phone users who talk while driving. Although cell-phone users and control subjects do not differ significantly in their ability to track the shapes in the attention-demanding “dummy” task, the percentage of subjects who reported seeing the red cross plummets from seventy percent for non-cell-phone users to ten percent for cell-phone users.

“You know to look over your shoulder when you are merging, because you know you can’t see something there [without doing so],” Scholl noted, “but no one tries to spend extra effort trying to make sure you see what is right in front of your eyes. You just assume you would automatically see it. This research shows that it is not true.” Bicycles, people, deer – things that are not expected to be on the road might be harder to see when drivers do not pay close attention.

While these experiments have shown that the selectivity of attention can cause people to be unaware of salient objects, “motion-induced blindness” experiments seem to show an opposite trend: people can perceive an object that is not physically present. A yellow dot is presented along with a rotating grid of blue crosses. By focusing on a white circle in the center of the grid, the subjects see the yellow dot pass in and out of awareness — it disappears and reappears even though nothing changes on the screen.

The reason for this failure of awareness is not fully known, but Scholl and colleagues have been exploiting this phenomenon to study other aspects of perception. The beauty of this phenomenon is that it allows experimenters to manipulate the dot outside of people’s awareness and test what they perceive. “This is a great tool which we can use to explore all the types of processing that occur even when we are not aware of it,” said Scholl.

One such experiment tests what people perceive if the dot is removed while it is outside of awareness. In an article under review, Scholl and postdoctoral researcher Stephen Mitroff noted the paradoxical question: “We are essentially asking whether observers can see the disappearance of an object which they cannot see.”

In fact, they can. Ninety-five percent of subjects witnessed the disappearance of the dot while it was out of awareness. “You see a brief flash of the dot,” described Scholl. At the moment the flash is perceived, however, the dot has already disappeared. One is seeing something that is no longer physically present. It seems, therefore, that the brain continues to process the dot even outside of awareness.

In further experiments probing this idea, subjects watch two dots move in and out of conscious awareness. When separated, they tend to move independently; when connected by a line into a dumbbell, they move together. But what if the connecting line is removed after the dumbbell disappears from awareness while subjects cannot see it? Will the dots still come back together, or will the visual system reorganize the display into separate dots even outside awareness?

The latter is indeed what occurs. Dots that are separated outside of awareness come back into attention separately. “The fact that you can actually detect this reliably shows that you are continuing to process [the shapes] even outside of awareness,” said Scholl.

Chun studies whether unperceived objects are still registered and processed outside of consciousness by using neuroimaging techniques. In a paper collaborated with graduate student Do-Joon Yi and Dr. René Marois of Vanderbilt University in the February 2004 edition of Neuron, Chun describes findings showing that parts of the brain are stimulated by images that subjects do not realize they had seen.

In his study, Chun employed the “attentional blink” paradigm, in which subjects search for two images presented in a series of scrambled distracter images. The subjects are instructed to look for a face and remember what it looks like and then to detect whether an indoor or outdoor scene, if any, is presented. Since the slides flash by at ten per second, perception of the scene is severely impaired – about eighty percent do not see it, that is, they are not aware that it flashed before them. Because the mind is busy processing the face, it does not have enough capacity to process the scene as well.

To determine whether the brain actually processes the missed scene, Chun used functional magnetic resonance imaging (fMRI) which detects increased blood flow to active areas of the brain. Compared to when scenes were not presented at all, Chun and colleagues found that the brain showed significantly more activity when scenes were presented, even when people failed to “see” the scene. Naturally, this activity is lower than that for consciously perceived faces, but missed scenes nonetheless stimulate the brain. Therefore, even when people think that they do not see something, it does not go unnoticed by the brain.

Visual perception is a complex system of which we are just scraping the surface. Recent research on “statistical summary representations” is uncovering a capability that we never knew we had. “Though we normally think of visual perception only in terms of representing individual objects and relations between them,” Scholl stated, “we may also automatically represent scenes in terms of their average properties.”

The set of seemingly random shapes becomes three easily visible B’s when a web of black is added.  The visual system assumes lines to continue beyond occluders, making the figure with extra stimuli easier to interpret. (Credit: Marvin Chun)

When given two groups of circles of various sizes, for example, subjects can quickly judge which group has the larger average size, down to differences of only six percent in average diameter. Such an ability would be ecologically advantageous: a predator deciding which herd to attack would benefit by being able to judge which has the largest average size. It cannot know which individual animal it will kill, but it can know which group is more robust on average. “It is something the brain may be doing all the time as a way of dealing with the world that you might never have noticed you were even capable of doing,” noted Scholl. “You don’t have to think about it; it’s more like a visual reflex.”

Understanding the method by which the brain does this poses some interesting questions. The only accurate way to compute the average is to sum up all the areas and divide by the number of objects. Does the brain calculate, according to the formula Area = pr2? Does it count up bits of area of the circles? To test whether the brain determines average area by direct measurement or by indirect geometric computation, subjects were asked to compare a group of circles and a group of semicircles as if they were full circles. Again, subjects were able to compare average area across such heterogenous sets with surprising accuracy, even though the displays were each presented for less than a third of a second.

Since the subject did not seem to be counting bits of area, Sussman and Scholl turned to the calculation hypothesis. Subjects were shown two sets of objects simultaneously. Each item in one set was a circle, and each item in the other set was a group of three short line-segments. Subjects had to quickly imagine the circles that could be circumscribed around each group of three lines, and then compare the average size of those imagined circles with the actual set of circles. Subjects were able to do this successfully when the three lines were each arranged to as to highlight the radius of the imaginary circles (as in the Mercedes-Benz logo), but not when they were organized differently in ways which did not emphasize the radius (e.g., as chords of the imaginary circles).

“This shows us that extracting just a few measurements of an object turns out to be a very useful way to represent it,” said Scholl. “You can represent and recognize [shapes] by measuring only a few salient axes.” It seems that using these axes to compute area or volume is a task that the brain is always performing.

Through the research of Chun and Scholl, we are coming to understand that conscious awareness of a stimulus is not necessary for it to be processed by the brain. An entire world of computation takes place outside of consciousness, unbeknownst to people and only now slowly being unlocked by scientists.

Science Links

Present your research to New Haven high school students (Synergy Outreach) Astronomy Picture of the Day (NASA) BBC Science & Nature (BBC) Learn how our lifestyles can change the way our genes work, examine a yet-to-be-broken code on a sculpture called Kryptos, see preserved dinosaur blood vessels (NOVA ScienceNow) Watch Health Videos: Weight-loss surgery to caffeine on the brain (CNN Health News) Science, Health, Environment & Technology (ScienceDaily)

Copyright 2013 Yale Scientific Publications, Inc. - Disclaimer