Dark energy and I came of age together. In 1998, two competing groups of astronomers discovered that the expansion of the universe was accelerating; the same year, I graduated from college with a degree in physics and began work on a Ph.D. Though much of my research chased down other paths, I kept returning to dark energy. Call it a 16-year on-and-off-again dysfunctional relationship with no end in sight.
Of course, I'm very much not alone in trying to understand dark energy: It stands as one of the central problems in modern cosmology. Even the name “dark energy” is a placeholder for our ignorance, representing the fact of cosmic acceleration without indicating its identity. Astronomers have measured the acceleration rate and determined that dark energy constitutes more than two-thirds of the energy content of the cosmos, but its identity has defeated theoretical physicists.
Despite dark energy’s magnitude, astronomers didn't know about its existence until recently because its effects are subtle. It doesn't noticeably affect the planets in the solar system or the motion of stars in the galaxy. That subtlety enshrouds it in mystery: Scientists are busily determining what dark energy does, and they have yet to reach any consensus on what dark energy is.
Because dark energy doesn't correspond easily to anything in the standard toolkit of physics, researchers have been free to be creative. The result is a wealth of ideas, some that are potentially interesting and others that are frankly nuts. Some string theorists propose that our observable universe is the result of a vast set of parallel universes, each with a different, random amount of dark energy. Other physicists think our cosmos is interacting with a parallel universe, and the force between the two drives cosmic acceleration. Still others suspect that dark energy is a sign that our currently accepted theory of gravity—Einstein's general theory of relativity—is incomplete for the largest distances.
Despite the crazy proliferation of solutions, scientists are not completely in the dark on dark energy. Observations of supernova explosions in distant galaxies seem to indicate that cosmic acceleration hasn't changed in strength over time, and it seems to act at the same rate in all parts of the cosmos. While there's still some wiggle room for small changes over long time spans and distances, the observed constancy of dark energy already rules out some explanations as to its identity.
Dark energy behaves very differently from both ordinary and dark matter. (Dark matter is arguably a much larger mystery, but despite the similar names, they don’t seem to have anything to do with each other.) Matter mutually attracts via gravity, forming galaxies and other structures, while dark energy doesn't seem to “clump” in any way. Even though matter drives cosmic expansion simply by possessing energy, its overall effect on expansion is to slow things down.
Dark energy, by contrast, seems to operate on a feedback cycle. The more the universe expands, the more dark energy there seems to be, and the more dark energy there is, the more it drives expansion. That characteristic indicates dark energy is not created by any known type of particle. Its influence grows as the universe ages, while matter's effects dissipate. If trends continue, the galaxies we see today will grow so far apart as to be invisible from one another in the far future. That would bring an end to astronomy as we know it. Assuming our species (or evolutionary offspring) can survive beyond the end of Earth, any descendants in the distant future would only be able to see vast expanses of blackness beyond the outskirts of the galaxy.
So if dark energy isn't anything normal, what is it?
The most popular candidate is vacuum energy: the energy from quantum fluctuations in empty space. This idea was first kicked around in the 1970s and '80s, long before cosmic acceleration was discovered. Physicists recognized that the same stew of quantum processes that determine the properties of electrons and other particles would grant energy to empty space. From general relativity, any energy has a gravitational effect. In this case, the energy would serve to accelerate cosmic expansion.
The effect of vacuum energy can easily be incorporated into Einstein's theory as something known as a “cosmological constant,” an extra gravitational factor not created by matter. The problem: The amount of energy predicted by quantum theory is much larger than the observed amount of dark energy, something researchers knew even before the discovery of cosmic acceleration.
That doesn't rule out vacuum energy completely, partly because we don't have a quantum theory of gravity that would tell us how to calculate the cosmological constant. String theory offers one possible way out, but it has its own problems. Rather than calculating a unique vacuum energy prediction, it provides an almost unimaginably vast number of possible values known as the “string theory landscape.” Most of vacuum states don't resemble our observable universe at all, and it's unclear whether there's anything testable in the landscape model.
To see what else dark energy could be, it's useful to review the evidence. Two competing groups of astronomers measured light emitted by white dwarf supernovas (also known as Type Ia supernovas). These are extremely bright and all explode in very similar ways, allowing them to be used to measure very large distances. Using these supernova data, the astronomers determined the rate of expansion of the universe was increasing, a discovery that won them the 2011 Nobel Prize in physics.
Subsequent observations have increased the number of known supernovas and shown that the effect of dark energy hasn't changed over time. That rules out many models for cosmic acceleration, such as “quintessence,” a hypothetical fluid with very odd properties that is dynamic, changing measurably in its effects over the lifetime of the cosmos.
Dark energy has made physicists face the possibility that general relativity breaks down on the scale of the universe. In that case, new rules in gravity could kick in on distances larger than galaxies, driving cosmic expansion without needing any new substance or quantum effects.
The idea isn't crazy. Einstein's theory is very well tested in the solar system and in regions of strong gravity (near binary pulsars, for example). However, it's possible the existence of dark energy is a sign of the failure of general relativity on the largest scales. (Most physicists think general relativity will also require modification on the smallest scales, where quantum rules take effect.)
The observational evidence strongly constrains how we can modify gravity. The evidence for dark energy includes the supernova data previously mentioned, but also the spectrum of light from the universe's early years: the cosmic microwave background. Any modification to gravity must reproduce all the data, and preferably predict something new against which we can test the theory.
Several modified theories of gravity have drawn inspiration from quantum physics. These ideas utilize a different set of equations than general relativity does, the new equations reproduce the known behavior of gravity in the solar system but act differently on larger scales.
Another style of modification to gravity is more subtle. These theories extend the known rules into a higher-dimensional reality. Just like Earth has a surface and interior, so-called braneworld models depict the universe we know as a surface or “brane” (short for membrane), with one or more extra dimensions that we can't access directly. However, gravity reaches into the extra dimensions, and other branes could influence what goes on in our observable cosmos.
In some braneworld models, dark energy is the result of gravitational tension between our universe and a neighboring brane. One version—the cyclic model—predicts that dark energy will dissipate at some point in the future as the branes reach a critical distance from each other, after which point cosmic expansion will reverse. While that point is far in our future, we may be able to spot other signatures of the higher-dimensional reality: The Big Bang in the cyclic model marks the moment when our universe collided with the neighboring brane.
In my opinion, all current explanations for dark energy are unsatisfactory, though like unhappy families they are all unsatisfactory in their own way. The modifications to gravity inspired by quantum physics seem most promising to me, however. That's for the simple reason that we know how those laws work, thanks to decades of research in particle physics. Since we already suspect that general relativity must be modified to accommodate quantum effects (especially where black holes are concerned), perhaps the cosmos is giving us a strong hint about where to look for new ideas.
But that's a hunch on my part. The universe, after all, is the true judge of our ideas. No matter how elegant a theory may be, it must match the observational evidence. As new data come in, theories may stand or fall. Ongoing observations of supernovas and distant galaxies will tell us whether dark energy has always been as it is now, or if it fluctuates; perhaps the cosmos itself will tell us how to change our thinking.