About 68 percent of everything that exists is made of something we have never detected, cannot explain, and which is doing the one thing gravity was never supposed to allow: making the universe expand faster instead of slower.
By Santhosh Vishwamitra
In this file
The Mystery · Discovery · What Scientists Know · Leading Theories · Evidence For Each · Problems With Each · Current Research · What Could Solve It · Where the Mystery Goes Next
The Mystery
For most of the twentieth century, cosmologists argued about one number. They wanted to know how fast the expansion of the universe was slowing down.
Nobody doubted that it must be slowing. Gravity pulls matter together, so the mutual attraction of every galaxy on every other galaxy had to act like a brake on the expansion left over from the Big Bang. The only debate was whether the brake would eventually stop the expansion and reverse it, or whether the universe would coast forever.
Then two teams measured the braking and found there was none. The expansion is not slowing. It is speeding up.
That single observation broke something fundamental. If expansion accelerates, then some influence must be overpowering gravity on the largest scales. Physicists gave that influence a placeholder name, dark energy, and the name is honest in one way. The word "dark" here means the same thing it means in "dark matter." It means we cannot see it. We only see what it does.
The mystery is not merely that we lack an answer. It is that our best guess produces a number so absurdly wrong that it is sometimes called the worst prediction in the history of physics.
Discovery
In 1998, two rival groups announced the same uncomfortable result almost simultaneously. The Supernova Cosmology Project, led by Saul Perlmutter, and the High-z Supernova Search Team, led by Brian Schmidt with Adam Riess, had both been using distant exploding stars as cosmic distance markers.
Both expected to measure deceleration. Both found the opposite. The discovery earned Perlmutter, Schmidt and Riess the 2011 Nobel Prize in Physics.
Established fact, not theory
The acceleration itself is one of the most solid findings in modern cosmology. It is supported by three independent methods, not just supernovae. What remains unknown is the cause, not the effect.
Their tool was a particular kind of stellar explosion called a Type Ia supernova. These explosions all reach nearly the same peak brightness, which makes them reliable markers. If you know how bright something truly is, and you measure how dim it looks, you can calculate how far away it is.
The two teams found that distant supernovae looked dimmer than they should. They were farther away than a decelerating universe would allow. The only clean explanation was that expansion had sped up while their light was in transit.
| Year | Milestone |
|---|
| 1917 | Einstein introduces the cosmological constant into his equations, then later abandons it |
| 1998 | Two supernova teams announce the expansion is accelerating |
| 2003 | WMAP maps the cosmic microwave background and confirms the cosmic energy budget |
| 2011 | Nobel Prize awarded for the acceleration discovery |
| 2013 | Planck satellite refines the composition of the universe to high precision |
| 2024 | DESI reports preliminary hints that dark energy may weaken over time |
What Scientists Know
Three independent measurements all point to the same picture. That convergence is why cosmologists take the phenomenon seriously even without an explanation.
First, the supernovae described above. Second, the cosmic microwave background, the faint afterglow of the early universe, measured in detail by the Planck satellite. Third, baryon acoustic oscillations, frozen ripples in the distribution of galaxies that act like a standard ruler across cosmic time.
Three different tools, three different eras of the universe, one consistent answer. That answer is a strange cosmic budget. Ordinary matter, every star, planet and person, makes up about 5 percent of the universe's total energy. Dark matter, the invisible scaffolding that holds galaxies together, accounts for about 27 percent. The remaining 68 percent is dark energy.
How weak, and why it still wins
If dark energy is vacuum energy, its density is tiny, around six ten-millionths of a joule per cubic metre. That sounds like nothing. It basically is nothing, locally. It only becomes dominant because it does not dilute as space expands, and because there is an almost incomprehensible amount of space. Gravity loses the tug of war not because dark energy is strong, but because it is everywhere and never thins out.
Physicists describe the behaviour of dark energy with a single number called its equation of state, written as w. For a cosmological constant, w equals exactly minus one. Observations currently pin w very close to that value. The trouble is "very close" is not "exactly," and that tiny gap is where entire alternative theories live.
A genuine disagreement
Cosmologists openly disagree over the so-called Hubble tension. Measurements of today's expansion rate from nearby supernovae give about 73 kilometres per second per megaparsec, while the early-universe reading from the cosmic microwave background gives about 67. That gap has survived years of scrutiny and may point to new physics, or to an unknown measurement error. Nobody has resolved it.
Leading Theories
There is no shortage of explanations. What there is, is a shortage of evidence to choose between them.
The simplest idea is Einstein's cosmological constant, a constant energy woven into the fabric of empty space. It fits nearly every observation with a single number.
The second major camp is quintessence, a dynamic field whose strength can change over time, meaning dark energy could weaken or strengthen as the universe ages. A third approach abandons the idea that a hidden energy is the culprit and instead proposes that our theory of gravity, general relativity, simply breaks down across cosmic distances. A fourth group of models, often called early dark energy, was built mainly to repair the Hubble tension.
| Theory | Core idea | Key strength | Biggest weakness |
|---|
| Cosmological constant | Empty space carries a fixed, unchanging energy | Fits almost all data with one parameter | Quantum theory predicts a value 120 orders of magnitude too large |
| Quintessence | A slowly evolving field fills space | Could explain why dark energy seems to change | No field has ever been detected, and it needs careful tuning |
| Modified gravity | General relativity fails at the largest scales | Removes the need for a mysterious energy | Hard to reconcile with solar system tests and gravitational wave observations |
| Early dark energy | A brief burst of energy in the early universe | Could resolve the Hubble tension | Tightly constrained by the cosmic microwave background |
Evidence For Each
The cosmological constant has the strongest hand. It explains the supernova dimming, the microwave background and the galaxy distribution all at once, and every measurement so far has found w sitting close to minus one. Its virtue is almost its problem. It works too well with too little.
Quintessence got a boost of attention from the Dark Energy Spectroscopic Instrument, or DESI. Its preliminary results, published in 2024 and strengthened in 2025, showed intriguing hints that dark energy might be weakening over time rather than staying constant.
It is vital to stress that these are early statistical whispers, not a confirmed discovery. The current data does not yet establish evolving dark energy as a fact, but it provides the first real reason to take dynamic models seriously.
Modified gravity draws its motivation from a pattern in the history of physics. Newton's gravity was replaced by Einstein's when measurements reached new precision. It is not unreasonable to expect general relativity to have its own breaking point.
Problems With Each
Every candidate carries a serious wound.
The cosmological constant problem is the most embarrassing one in theoretical physics. When physicists try to calculate the energy of empty space using quantum field theory, the answer comes out around 10 to the power 120 times larger than what we observe. That is a one followed by 120 zeros. It is not a small mismatch. It suggests something essential is missing from our understanding.
There is also the coincidence problem. Why is dark energy becoming dominant right now, in cosmic terms? For billions of years matter ruled, and for billions more dark energy will rule. We happen to live in the brief transition between the two. That looks either like remarkable luck or like a clue.
Quintessence avoids the frozen constant problem but replaces it with a new one. It requires a field that has never been observed and that must be tuned extraordinarily carefully to behave the way the data demand.
Modified gravity, meanwhile, took a heavy hit in 2017 when two gravitational waves and their light arrived together from a distant collision. That observation showed gravity travels at the speed of light, which quietly ruled out a large family of modified gravity models.
Current Research
The field has moved from discovery mode into precision mode. DESI is mapping the positions of tens of millions of galaxies to trace how expansion has changed over cosmic history. The European Space Agency's Euclid mission is doing the same from space.
The Vera C. Rubin Observatory is scanning the sky repeatedly to catch thousands of supernovae, and the Nancy Grace Roman Space Telescope is designed to measure the expansion history with a precision current instruments cannot reach.
The goal of all these projects is essentially one measurement. Does w stay locked at minus one, or does it drift? A confirmed drift would be the end of the cosmological constant as the full story. A tighter confirmation of minus one would push dynamic theories into a corner.
What Could Solve It
A solution will almost certainly come from one of three directions.
One, a decisive measurement showing dark energy evolves, which would kill the simple constant and favour something like quintessence. Two, a resolution of the Hubble tension that forces the standard model to change. Three, a gravitational wave standard siren measurement, which uses merging neutron stars as independent distance markers and could bypass the calibration chain that underlies today's supernova measurements.
There is also the wild card of theory. If someone derives the observed value of dark energy from first principles and explains why it is small but not zero, that alone could settle the question without any new telescope.
Where the Mystery Goes Next
The popular story frames dark energy as a hunt for a mysterious substance. That framing misses the more interesting point. The acceleration is not mainly a problem about a missing ingredient. It is a symptom of a collision between our two best theories.
General relativity describes gravity and the large universe beautifully. Quantum field theory describes particles and the small universe beautifully. Dark energy is the place where they are forced to share a room, and they do not speak the same language.
The 120 orders of magnitude gap between prediction and observation is not a detail to fix later. It is the clearest signal we have that one of the two theories, or both, is fundamentally incomplete.
There is a second misunderstanding worth clearing up. People imagine the acceleration as a violent tearing force, something dramatic happening to space right now. It is the exact opposite. It is unbelievably gentle.
Over the distance between here and the nearest stars, its effect is essentially zero. It only registers across hundreds of millions of light years, where it has room to accumulate. The universe is not being ripped apart in any local sense. It is quietly drifting apart on scales no human can feel.
What is most worth watching over the next five years is the collision between precision and systematic error. Instruments like DESI, Euclid and Rubin are so sensitive that the limiting factor is no longer the hardware. The bottleneck is our ability to control tiny biases in calibration, in how we model supernovae, and in how we interpret the microwave background.
The Hubble tension may ultimately be a lesson that we reached the edge of what our current methods can cleanly measure. If the field confirms the equation of state equals exactly minus one to absurd precision, dark energy becomes a fixed feature of reality. If it finds evolution, everything reopens.
Either outcome is valuable. The one thing that would be genuinely bad is a result too fuzzy to decide. Readers who want the broader context of how these cosmic questions connect to the rest of science can follow the ongoing coverage at Vishwamitra Chronicles.
Dark energy is less a missing substance than a visible crack between general relativity and quantum theory.
Sources:
1. Riess, A. G., et al. (1998). "Observational Evidence from Supernovae for an Accelerating Universe and a Cosmological Constant." The Astronomical Journal, 116, 1009.
2. Perlmutter, S., et al. (1999). "Measurements of Omega and Lambda from 42 High-Redshift Supernovae." The Astrophysical Journal, 517, 565.
3. Planck Collaboration (2020). "Planck 2018 results. VI. Cosmological parameters." Astronomy & Astrophysics, 641, A6.
4. Weinberg, S. (1989). "The Cosmological Constant Problem." Reviews of Modern Physics, 61, 1.
5. DESI Collaboration (2024). "DESI 2024 VI: Cosmological Constraints from the Measurements of Baryon Acoustic Oscillations." arXiv:2404.03002.
Santhosh Vishwamitra
Web Developer and Founder of Vishwamitra Chronicles, launched in February 2026.
Comments
Post a Comment