Science

Lessons From the Dress

Vision is fundamentally ambiguous—but usually not this ambiguous.

How can two people look at the same photo and see different things?

Photo illustration by Slate

If you are just encountering The Dress for the first time, you might first want to click here to see what all the fuss was about.

The brain lives in a bony shell. The completely light-tight nature of the skull renders this home a place of complete darkness. So the brain relies on the eyes to supply an image of the outside world, but there are many processing steps between the translation of light energy into electrical impulses that happens in the eye and the neural activity that corresponds to a conscious perception of the outside world. In other words, the brain is playing a game of telephone and—contrary to popular belief—our perception corresponds to the brain’s best guess of what is going on in the outside world, not necessarily to the way things actually are. This has been recognized for at least 150 years, since the time of Hermann von Helmholtz. This week, it was recognized by masses of people on the Internet, who have been debating furiously over what should be a simple question: What color is this dress?

Many parts of the brain contribute to any given perception, and it should not be surprising that different people can reconstruct the outside world in different ways. This is true for many perceptual qualities, including form and motion. While this guessing game is going on all the time, it is possible to demonstrate it clearly by generating impoverished stimulus displays that are consistent with different, mutually exclusive interpretations. That means the brain will not necessarily commit to one interpretation, but will switch back and forth. These are known as ambiguous or bi-stable stimuli, and they illustrate the point that the brain is ultimately only guessing when perceiving the world. It usually just has more information to disambiguate the interpretation.

Rubin’s vase: A classical example of figure/ground segmentation. The image is fundamentally ambiguous. People perceive a vase or faces, but not both at the same time.

Wikimedia Commons

The fact that the brain is constantly constructing a model of what the world really looks like is also true for color vision. The fundamental challenge in the perception of color is to identify an object despite changing illumination conditions—how bright or dim the ambient lighting is. The mixture of wavelengths that reaches our eye will be interpreted by the brain as color, but which part is due to the reflectance of the object and which part is due to its surrounding illumination?

A bi-stable motion stimulus. Do you see the dots moving from left to right or up and down?

Pascal Wallisch

This is an inherently ambiguous situation, so the brain has to make a decision about whether to take the appearance of an object at face value or to discount part of the information as stemming from the illumination. The brain is not primarily interested in the correct representation of hues, but rather the identification of objects in light of dramatically varying conditions. (For example, there is a predominance of long wavelengths in the early morning and late afternoon, versus more short wavelengths at noon.) The brain instead strives for “color constancy”—recognizing the same object as having the same color no matter what the time of day—and it is doing a pretty good job at that. But in this trade-off toward discounting specific wavelengths, something has to give, and that is that we are bad at estimating the absolute hue of objects. For instance, a white surface illuminated by red light will objectively look reddish. The same white surface illuminated by blue light will objectively look blueish. In order to recognize both as the same white surface, the subjective perception needs to discount the color of the light source.

So it should not be surprising that inference of hue can be dramatically influenced by context. The same shade of gray can look almost black on a bright background but almost white on a dark one.

Lightness illusions are common. The gray bar in the middle is all one shade.

Wikimedia Commons.

Note that this is not a bug, but a feature. It is a necessary trade-off in the quest to achieve a stable appearance of the same object, regardless of context.

So far, so good. Now where does the dress come in? The latest sensation to sweep social media has sharply divided observers. Some see the dress as gold on white, others as black on blue.

Here it is again.

Photo illustration by Slate

As I’ve noted before, this kind of divergence of interpretation might be rather common with complex stimuli. The importance of the dress stimulus is the extent to which subjective interpretation differs between people. To my knowledge, this is by far the most extreme such stimulus in the color domain. Of course one has to allow for the fact that not everyone’s monitor will be calibrated in the same way and viewing angles might differ, but this doesn’t account for the different subjective experience of people viewing the exact same image on the same monitor from the same position. And of course the reason why the “true colors” of the dress are in dispute in the first place is the color constancy phenomenon we alluded to above. This was likely a black/blue dress that was photographed with poor white balance, giving it an ambiguous appearance. But that doesn’t change the fact that some people sincerely perceive it as white/gold.

That the interpretation of the color values itself depends on context can readily be seen if the context is taken away. In the image below, some stripes were extracted from the original image without altering it in any other way. The “white/blue” stripe can now be identified as light blue and the “gold/black” stripe as brown.

Dress stripes, out of context: One now unambiguously looks like blue, the other like brown.

Pascal Wallisch

But why the difference in interpretation? That is where things get interesting. If the ambiguity derives from color constancy (and it looks like it does), the most plausible explanation is that people differ in their interpretation of what the illumination source is. Those who interpret the dress as illuminated by a blue light (corresponding to a bright setting) will discount for this and see it as white/gold. Those who interpret the illumination as reddish (in a dimmer setting) will tend to see it as black/blue. Interestingly, the image itself does allow for both interpretations: The illumination looks blueish in the top of the image, but yellowish/reddish in the bottom. On a more fundamental level, a blue/black dress illuminated by a white light source might be indistinguishable from a white/gold one with a blueish shadow falling onto it.

But if this is the case, one should be able to consciously override this interpretation once it is pointed out, but—for many people—this does not seem to be the case, in contrast to most other such ambiguous displays, such as the duck-rabbit. People are usually able to willfully control what they see.

Inherently ambiguous stimuli. Interpretations switch, they don’t blend. But people can consciously override interpretations once they are pointed out.

Pascal Wallisch

This persistence raises several intriguing possibilities. For instance, it has been recognized for quite a while that the human “retinal mosaic”—the distribution of short-, medium- and long-wavelength cones in humans—is radically different between observers, but this seems to only have a minute impact on the actual perception of color. Perhaps it is the case that differences in retinal mosaic can account for difference in the perception of this kind of ambiguous “dress” stimulus. Moreover, there is another type of context to consider and that is temporal context. We don’t just perceive visual stimuli naively, but we perceive them in the context of what we have encountered before, in part because not all stimuli are equally likely. This learned expectation is known as a “prior.” It is quite conceivable that some people (early birds versus night owls) have a different prior based on what kind of illumination conditions they encounter more frequently. Or there could be a complex interaction between the two.

While we must confess that we currently do not know why some people consistently see the dress one way, others consistently in another way, and some switch, it is remarkable that the switching happens on very long timescales. Usually, switching is fast, for instance in the Rubin’s vase stimulus above. This delay could be particular to color vision. There are no shortcuts other than to do research into the underlying reason that accounts for this striking difference in subjective perception.

Meanwhile, one lesson that we can take from all of this is that it is wise to assume a position of epistemic humility. Just because we see something in a certain way doesn’t mean that everyone else will see it in the same way. Moreover, it doesn’t mean that our perception necessarily corresponds to anything in the real world. A situation like this calls for the hedging of one’s bets, and that means to keep an open mind. Something to remember next time you disagree with someone.

Who is right? Does the question even make sense? Or is it important to see the bigger picture?

Source unknown

A version of this article originally appeared in Pascal’s Pensées.