Was the dress gold and white or blue and black?

Remember the famous black and blue dress? Or...was it gold and white? Some saw the dress as black and blue; others swore it was gold and white. (A small minority saw it as brown and blue.) The resulting debate over its true colors went viral, prompting millions of tweets and causing a brief Internet sensation.

A superficial controversy, to be sure, yet one that underscores serious scientific questions in neuroscience that are related to perception, and the ability of human vision to distinguish surface colors under different lighting conditions.

And it gets to the heart of Laurence T. Maloney's research.

Maloney focuses on applications of mathematical models to understanding human behavior, including work in the physics and mathematics of color vision, as well as applying statistical decision theory to perceptual judgments and the planning of movement.

Currently, he and his colleagues are studying color perception and surface material perception in complex, three-dimensional scenes, kin recognition, and human performance in perceptual and movement tasks.

"You are gathering information every instant," says Maloney, a professor of psychology and neural science at New York University. "You have to decide at every instant whether you have enough information to act--or do you need further information? This is something we are doing every second of our lives. It's a fundamental thing we do with information, to decide if we've got enough."

This applies to virtually every action, whether it's staring at a photo of a famous dress to decide what color it is, or something as simple as looking across the table at a glass of wine and deciding how far away it is before you move your hand to grab it. There are many cues to depth, and together they limit how good an estimate of depth your visual system can provide.

"You can reach for it based on the cues you have, but if you are mistaken you may end up spilling wine on the table cloth and your fellow diners, " Maloney says. "Alternatively, you could add another cue, by moving your head back and forth to get more information by viewing the same object from different perspectives. The cost to you of seeking information is the time spent gathering information and the fact that, well, you look silly."

This process, which neuroscientists call "cue combination," occurs in a fraction of a second as people constantly process information in the environment and decide what to do with it. This National Science Foundation (NSF)--funded scientist is trying to better understand what happens in the brain as information flows from perception to action.

One of Maloney's NSF-funded projects studied the impact of light on visual color perception, research that relates to the "dress" discussion in the sense that peoples' conclusions about its color depend heavily on how and where they see the lighting.

"What I would suggest is happening is that you are gathering information unconsciously as to where the lighting is," he says. "The information in the picture is ambiguous. People arrive at different interpretations of the lighting in the scene and how light flows...to the dress and eventually the eye.

"If you see the dress in shadow against a bright background, you will see it as gold and white. If you see the light as coming round behind you, you will see the dress as blue and black."

More importantly, however, insights gained from studying how the brain figures out color signals in the environment, and how the state of the light affects these perceptions, potentially could help better understand the progression of certain degenerative eye disorders, such as glaucoma and retinitis pigmentosa. NSF supported this work with $324,060 awarded in 2011.

More recently, Maloney received a fellowship from the John Simon Guggenheim Memorial Foundation, which annually supports a diverse group of scholars, artists, and scientists chosen on the basis of prior achievement and exceptional promise. He plans to write a book with working title "The Statistical Brain" detailing how information flows in the brain--or at least what scientists know about it now.

In the NSF-supported color and light studies, his experiments "take place in little virtual environments," using computer graphics, he says. Subjects watch "floating balls and cubes illuminated in ways that the experimenters control," he says. "In the course of trying to figure out the colors of things, they are effectively estimating the illumination. I show you a square. I ask you: does it look bluish or yellowish? You tell me bluish, then I make it yellower. If you say yellow, then I make it bluer." Ultimately, "you will reach the point where you are saying blue or yellow in equal numbers. This gives us a reference point, a surface you consider to be colorless," he explains.

He repeats the exercise with red and green, "until you have a square whether you are equally likely to call blue or yellow or red or green, and it is neutral in appearance. Now the question is: how does your neutral square change when I change the lighting? How does your choice change? We want to show that people are sensitive to light in the environment, and we can measure the light shining on each part of the scene."

The work essentially traces a signal as it moves through the brain.

"Color is a nice, measurable thing," he says. "It's a marvelous signal to trace through the visual system. It's just complicated enough that we can actually keep track of what is happening to it, to figure out how the brain works out the colors we seek out in the environment, and how it is affected by the state of the light."

In applying his findings to eye diseases, researchers could determine whether--and how--individuals compensate and change their eye movements over the course of the illness, he says.

"In effect, we are studying how people relearn to use their vision as it progressively decays," he says. "The key thing is information--and the plasticity of the system. You have information, and you are a biological organism. Parts of our body degrade with time. But in many cases we compensate marvelously. We want to see how we spontaneously reorganize the brain to maximize the information we get from the world."

As for the dress, Maloney himself was in the minority.

"I saw it as brown and blue," he says. "But then I have very few illusions."