Dark energy is said to be a mystery. Google on dark energy mystery and you can find plenty of material saying as much. Such as dark energy: the biggest mystery in the universe by Richard Panek in the Smithsonian magazine. Or dark energy: mystery of the millennium by Thanu Padmanabhan on the arXiv. Dark energy comes above dark matter in 10 greatest unsolved mysteries in physics on IFL science, and since it’s circa 68% of the mass-energy of the universe as opposed to 27% for dark matter, it’s got to be the bigger mystery. I think it could be even bigger than that. The proportions used to be quoted as 73% and 23%:
Dark matter pie image by NASA
I also think dark energy is the bigger mystery because there seems to be some confusion about what it does. We don’t have this issue for dark matter. Everybody knows that dark matter causes flat galactic rotation curves and excess gravitational lensing. For dark energy, things aren’t quite so clear cut. So much so that some don’t know what it is, and some don’t know what it’s not.
Dark energy is not zero-point energy
Dark energy is not zero-point energy, which arguably dates back to Walther Nernst and 1916. You might reasonably claim that dark energy is the same kind of thing as the cosmological constant, which is the “energy density of the vacuum of space”. That’s because the cosmological constant is said to be the “simplest possible form of dark energy”. You might also claim that dark energy is the same kind of thing as gravitational field energy, which as Einstein said “shall act gravitatively in the same way as any other kind of energy”. That’s because the modern approach is to treat dark energy as a component of the stress-energy tensor. But saying it’s the same thing as zero-point energy is comparing apples and oranges, and conflating relativity with quantum physics. See Svend Rugh and Henrik Zinkernagel’s 2002 paper on the quantum vacuum and the cosmological constant problem. They refer to Wolfgang Pauli’s 1920s “café calculation” that zero-point energy would result in a universe that “would not even reach to the moon”, and to Niels Bohr in 1948 saying zero-point energy “would be far too great to conform to the basis of general relativity”. That was the beginning of the vacuum catastrophe. In articles about vacuum energy you can read that “one contribution to the vacuum energy may be from virtual particles which are thought to be particle pairs that blink into existence and then annihilate in a timespan too short to observe”. But that’s just lies-to-children. Space is not some maelstrom of virtual particles popping in and out of existence like magic. Spontaneously, like worms from mud. Virtual particles only exist in the mathematics of the model. They aren’t the same thing as the vacuum fluctuations of the very weak Casimir effect. Hence as Rugh and Zinkernagel point out, photons do not scatter on the vacuum fluctuations of QED. If they did, “astronomy based on the observation of electromagnetic light from distant astrophysical objects would be impossible”. Hence when they say the QED vacuum energy concept “might be an artefact of the formalism with no physical existence independent of material systems”, they’re right. The energy of the gravitational field doesn’t consist of quantum fluctuations. Nor does dark energy.
What’s it supposed to be?
So, if we know what dark energy is not, do we know what it is? Not immediately, because there’s some ambiguity about what it’s supposed to be. Some say dark energy is responsible for the accelerating expansion of the universe. Some say it’s responsible for the expansion of the universe. Some say it’s an unknown form of energy which permeates all of space. The former is what it’s officially supposed to be. In the 2015 symmetry article What is dark energy? by Diana Kwon you can read how the phrase “dark energy” was coined by Michael Turner in 1998. That was in a paper about the supernovae observations that provided evidence of the accelerating expansion of the universe.
Image by Dmitri Pogosian, see his Astronomy 122 course
The Supernova Cosmology Project headed up by Saul Perlmutter and the High-Z Supernova Search Team headed up by Brian Schmidt and Adam Riess were expecting to measure a slowing-down of the expanding universe. Instead they found it was speeding up. Hence you can read that dark energy “is an unknown form of energy which is hypothesized to permeate all of space, tending to accelerate the expansion of the universe”.
All of the above
However you can also find articles that tell you that dark energy is all of the above. Such as the 2016 article Blinded by the Dark (Energy) by Stephen Battersby. He says astronomers discovered that “the speed of the expansion is not slowing, but accelerating”, and that dark energy is responsible. Check. He says dark energy is shrouded in mystery, and “all we know is that it has the peculiar property of pushing outwards”. Check. He refers to Einstein’s cosmological constant, a dark-energy candidate which is “equivalent to giving empty space its own energy”. Check. He perhaps didn’t mean to say it’s all of the above, but I’m confident that he’s correct to do so. Because of the clues in the literature.
Dark energy isn’t just responsible for the accelerating expansion
For example check out The Early History of Dark Energy by Stephen Kent. He says Einstein’s 1917 static model had in modern language, two components, namely baryons and dark energy. Kent effectively says double ditto for Alexander Friedmann’s and Georges Lemaître’s dynamical models, with radiation pressure as an extra for Lemaître. The pressure thing is important. Don’t forget that Erwin Schrödinger’s 1918 paper, translated by Alex Harvey in How Einstein Discovered Dark Energy, featured cosmic pressure. Or that Einstein talked about cosmic pressure in his 1919 letter to David Hilbert. Google on “dark energy” “cosmic pressure” and there’s plenty of papers and articles telling you about it. Such as Lemaître’s 1933 paper Evolution of the Expanding Universe where he talked of a cosmological constant pressure p = – ρc². Or the Nature news article Stellar performance nets physics prize: “the discovery was accepted almost immediately by the astronomical community – in part because the idea of a cosmic pressure pushing the Universe outwards had already been mooted by Albert Einstein”. Or Is Hubble’s Expansion due to Dark Energy? by Ramesh Gupta and Anirudh Pradhan dating from 2010: “Hence, it is concluded that: yes, indeed it is the dark-energy responsible for the Hubble’s expansion too, in-addition to the current on-going acceleration of the universe”. That’s enough to persuade me that dark energy isn’t just responsible for the accelerating expansion of the universe. It’s responsible for the expansion too. The moot point is this: if dark energy is only responsible for the accelerating expansion, some other dark mysterious thing must be responsible for the expansion. And since there’s nothing else on the menu, I’ll take the dark energy please. But I have to say I don’t like the way it’s served up.
Trouble with tension
That’s because there’s a big issue with the “given explanation” of dark energy. It’s associated with a kind of negative pressure, a tension, which counters gravitational attraction. This goes all the way back to Schrödinger’s 1918 paper. Again see the translation within Alex Harvey’s 2012 paper How Einstein Discovered Dark Energy. Schrödinger said “it is desirable to connect the system of solutions with a reasonable physical example. This is possible to a certain degree if one accepts Einstein’s hypothesis for the energy tensor of a continuous compressible fluid”. He gave an expression for the stress-energy tensor, saying “where p and ρ are scalars, the ‘pressure’ and density of the fluid”. He also said “these hypotheses see matter in the large essentially as a compressible fluid of constant density at rest under a constant spatially isotropic tension”. OK he said a fluid rather than a solid, but I get the drift because Silly Putty is a non-Newtonian fluid. And because Paul Halpern explains it in his 2016 book Einstein’s dice and Schrödinger’s cat. Einstein was unhappy with Schrödinger’s proposal, saying “one has to start out from the hypothesis of the existence of a nonobservable negative density in the interstellar spaces”. I think he was right to be unhappy with something that was unphysical, but wrong to reject it entirely instead of seeking to improve on it. Particularly since the negative pressure has now stuck. What Schrödinger said is pretty much what modern cosmology says, as you can deduce from Robert J Lambourne’s Relativity, Gravitation and Cosmology. Negative pressure has stuck, and it sucks.
Negative pressure sucks
Ned Wright gives a valiant explanation in his 2001 physicsFAQ article Is there a nonzero cosmological constant? He uses a vacuum cylinder as an analogy. The piston is in contact with the back face of the cylinder, and when you pull it out you do work and add positive energy. Wright says the vacuum is then trying to pull the piston back into the cylinder, hence it has a negative pressure, because a positive pressure would tend to push the piston out:
Image by Ned Wright, see Vacuum energy density
It’s a nice analogy, but it’s flawed because it’s the positive air pressure that pushes the piston back into the cylinder. Then if you set that aside as a quibble, the analogy is still flawed because vacuum sucks rather than blows, and you’d thus expect a contracting universe. You wouldn’t be the only one either, because Sean Carroll was critical of the negative pressure in his 2013 blog article why does dark energy make the universe accelerate? He said negative pressure aka tension “is more like a stretched string or rubber band, pulling in rather than pushing out”. He used phrases such as “word salad” and “quickly mumble”, and said “what matters, according to this line of fast talk, is the gravitational effect of the negative pressure”.
The wrong approach
I think there is a tension, but that Caroll is broadly correct here because the given explanation is wrong. It essentially says energy is pressure times volume and positive pressure causes gravitational attraction, hence a negative pressure is required to cause a gravitational repulsion. This logic is wrong because it isn’t based on any understanding of how gravity works. The horizontal light beam curves downwards because it’s moving through inhomogeneous space. There’s something akin to a spatial pressure gradient in this space, hence the speed of light is spatially variable, hence light bends downwards like sonar. Matter is similarly affected due to the wave nature of matter, wherein the electron is like light going around and around. But where space is homogeneous the pressure is uniform and the speed of light is constant, so light doesn’t curve and your pencil doesn’t fall down. And where is space homogeneous? At the largest scale, in the universe at large. So there is no overall gravitational field and never was, which is why the universe didn’t collapse when it was small and dense. Yes, there’s something causing the expansion of space, but it isn’t some gravitational repulsion. Some might liken this expansion to the waterfall analogy in reverse, but it’s important to appreciate that a gravitational field is a region of inhomogeneous space. And that gravity alters the motion of light and matter through space, but it doesn’t suck space in. We do not live in some Chicken Little world where the sky is falling in. Or in some opposite scenario wherein the sky is falling up, and the universe is expanding because of some kind of repulsive gravity. The expanding universe is not the opposite of a gravitational field. The universe doesn’t have a pressure gradient, it has a pressure. A cosmic pressure.
The universe is like a black hole in reverse
As Stephen Hawking said, the universe is like a black hole in reverse. Pulling away from a black hole across space can be likened to being in a universe expanding over time. In the former situation you might say space is inhomogeneous across space in a non-linear way, and we can model this as curved spacetime. In the latter situation you might say space is inhomogeneous over time in a non-linear way, and that we can also model this as curved spacetime. But then the cosmological principle is asking too much, because the vertical light beam doesn’t curve in a gravitational field. Demanding isotropy in expanding space is like saying the vertical light beam must curve like the horizontal light beam. Even though the vertical light beam doesn’t curve at all. Instead it goes faster as it ascends, echoing de Sitter’s 1917 paper. But despite all this, Friedmann’s 1922 paper was on the curvature of space, even though space isn’t spacetime. Thus there are some serious implications for the Friedmann model, which posits a universe with positive curvature, negative curvature, or no curvature:
Images by Nick Strobel, see www.astronomynotes.com
The idea is that if the density parameter Ω is greater than the critical density, space is positively curved. If Ω is less than the critical density, space is negatively curved. If Ω is equal to the critical density, space is flat. This is subject to the constraint that if space is negatively curved, it can’t become positively curved, and vice versa. That means two out of three “shapes of the universe” were always going to be wrong. Which perhaps does not auger well for what’s left. Perhaps it isn’t just two out of three that are wrong, perhaps it’s all of the above.
Dark energy isn’t just responsible for expansion
But all is not lost. Because dark energy is more than just something causing expansion, whether it’s accelerating or not. Remember that the cosmological constant is said to be the simplest form of dark energy, which is said to permeate all of space. Then see the 2008 paper Dark Energy and the Accelerating Universe by Joshua Frieman, Michael Turner, and Dragan Huterer: “Zel’dovich (1968) realized that Λ, mathematically equivalent to the stress-energy of empty space – the vacuum – cannot simply be dismissed”. Hence as per the Wikipedia Einstein field equations article, its term in the field equation can be written as part of the stress-energy tensor, and “the terms ‘cosmological constant’ and ‘vacuum energy’ are now used interchangeably in general relativity”. So dark energy is vacuum energy is spatial energy is stress-energy, and stress is directional pressure. There’s more. Again see The Early History of Dark Energy by Stephen Kent, who says the de Sitter metric corresponded to a universe made purely of dark energy. Then check out de Sitter’s 1917 paper On the relativity of inertia. Remarks concerning Einstein’s latest hypothesis. He spoke of the need to suppose that space is filled with matter of sorts. He said “this hypothetical matter I will call the ‘world-matter’”. And that “the world-matter is the three-dimensional space, or at least is inseparable from it”. He’s talking about space itself as some kind of dark energy. He is not alone in saying this sort of thing. See a simple explanation of mysterious space-stretching ‘dark energy?’ It’s in ScienceMag dating from 2017 and was written by Adrian Cho. He says dark energy could be part of space itself, a pressure inherent in the vacuum. Alternatively see the 2016 Dark Energy, Dark Matter article on the NASA Astrophysics website. It says one explanation for dark energy is that it is a property of space, and that Albert Einstein was the first person to realize that empty space is not nothing.
Space is indistinguishable from the gravitational field
In his 1920 Leyden address Einstein spoke of space as a something rather than a nothing. He said: “This space-time variability of the reciprocal relations of the standards of space and time, or perhaps the recognition of the fact that ’empty space’ in its physical relation is neither homogeneous nor isotropic, compelling us to describe its state by ten functions (the gravitational potentials gμν), has. I think, finally disposed of the view that space is physically empty”. The Einstein tensor is describing the state of space. Helge Kragh notes this in his 2011 paper Preludes to dark energy: Zero-point energy and vacuum speculations. He says Einstein “stressed that ‘empty space’ is not empty in the sense of having no physical properties. Quite the contrary, for space was indistinguishable from the gravitational field, which might be thought of as a non-absolute ether”. The idea of space being an ether will be unfamiliar to some, but according to Robert B Laughlin the modern concept of the vacuum of space is a relativistic ether. Hence Julian Schwinger talked of vacuum polarization in 1949. It is somewhat similar to de Sitter’s world-matter.
A field is a state of space
The idea of space being indistinguishable from the gravitational field will also be unfamiliar to some. But see Einstein talking about field theory in 1929. He was talking about the electromagnetic field and the gravitational field, and said it can “scarcely be imagined that empty space has conditions or states of two essentially different kinds”. According to Einstein, a field isn’t something separate from space. It isn’t something that exists in space. It’s a state of space. So the energy of the gravitational field is the energy of space itself. Only if you’re in a place where there’s no gradient in gravitational potential, space is homogeneous and there’s no detectable field, so light doesn’t curve and your pencil doesn’t fall down. But there is a gravitational potential, which you can detect with your optical clocks. The lower the gravitational potential the higher the spatial energy density and the slower the clocks. And as Feynman is alleged to have said, potential is more fundamental than field. It’s interesting stuff.
Space is indistinguishable from energy
As is Kragh’s opening sentence: “according to modern physics and cosmology, the universe expands at an increasing rate as the result of a ‘dark energy’ that characterizes empty space”. Characterizes is such an interesting word, n’est-ce pas? Take another look at Cormac O’Raifeartaigh’s 2017 article Albert Einstein and the origins of modern cosmology. He talks of Gμν = -κTμν, where Gμν describes the geometry of a region of spacetime or simply the state of space, Tμν is the stress-energy tensor, and κ is the Einstein constant. Einstein added his cosmological constant term λgμν to give the expression Gμν + λgμν = -κTμν, because he was giving space some kind of ground state. Only nowadays as Peter Coles said in One Hundred Years of the Cosmological Constant, we tend to think of the cosmological constant as dark energy on the right hand “source” side of the field equations instead of the left. Coles asked which side of the Einstein equations are you on? I’m on both. Not because of the Scholarpedia article on the equivalence of cosmological constant and vacuum energy by Tamara Davis and Brendan Griffen. Because of what Einstein said in 1930, siding with de Sitter’s world-matter. As Stephen Kent said, the de Sitter metric corresponded to a universe made purely of dark energy. I rather think that means dark energy isn’t just characteristic of space. Einstein’s field equation Gμν = -κTμν isn’t just saying a non-uniform energy density causes spacetime curvature. When we’re talking about homogeneous space on the largest scale, overlooking minor details such as galaxies, Gμν = -κTμν isn’t saying the universe has some overall gravitational field. Or some overall antigravitational field. The equals sign doesn’t mean causes. It means equals. It means is.
Space is modelled as an elastic solid
I would venture to say this is the deep fundamental reason why general relativity is related to continuum mechanics. Because Einstein’s elastic space is all about the “dynamic, elastic fabric of reality called space-time”. Which is why the stress-energy tensor, which “describes the density and flux of energy and momentum in spacetime”, has a shear-stress term. See understanding general relativity too by Rafi Moor for more about the stress-energy tensor. But don’t think in terms of matter, think in terms of spatial energy. That shear stress term tells you space is modelled as some kind of elastic solid, and the energy-pressure diagonal tells you it’s modelled as an elastic solid that’s subject to pressure. You start with a block of gin-clear ghostly elastic jelly with grid lines in it. You slide your hypodermic needle into the centre of the block, and inject more jelly. This represents a concentration of energy in the guise of a massive star. It creates a pressure gradient in the surrounding jelly. Stress is directional pressure, and the pressure is outwards. But note that you added jelly to represent energy, and that the jelly also represents space. Space doesn’t just have some kind of innate intrinsic vacuum energy. At some deep fundamental level, energy and space are the same thing. And on the largest scale that thing doesn’t have a pressure gradient. It has a pressure, with vanishing off-diagonal components just like Schrödinger said. As for why Einstein didn’t say it, I don’t know. If only they’d had stress balls back then. Have you ever squeezed down a stress ball then opened your fist? What happens to your world-matter when it’s no longer confined?
CCASA image by Lewis Ronald, see Wikipedia
It expands. This expansion appears to be accelerating, so you might think the cosmic pressure is increasing. However it might not be. Because there’s more than one way to skin Schrödinger’s cat.
Waves run through it
I say that because there’s a very important thing to appreciate about space: waves run through it. If you’ve ever looked at displacement current you’ll know that Maxwell talked about transverse undulations. A ripple in a rubber mat is a transverse undulation. It’s a transverse wave. So are light waves. So are gravitational waves, which you can read about at LIGO. The thing about waves is this: if a seismic wave moves through the ground, the ground waves. If an ocean wave moves through the sea, the sea waves. And if a gravitational wave moves through space, space waves. That’s why the LIGO interferometer measures a change in arm length. Shake a rubber mat and watch the wave run through it, at speed vs = √(G/ρ). It’s a shear wave propagating because of an elastic tension that opposes the stress. It takes two to tango. A wave in space also propagates because of an elastic tension that opposes the stress. See page 5 of New Physics at Low Accelerations (MOND): an Alternative to Dark Matter. Mordehai Milgrom mentions the strength of space. In a way, space has a tensile strength. It’s harder than diamond and stronger than steel, because such things are made of it. That’s why light waves propagate so fast. Space just doesn’t have an innate cosmic pressure, it has a tension too. If it didn’t have both, waves wouldn’t run through it. There’s also something closer to home that has both. Something that’s repeatedly depicted in papers and articles. Helge Kragh has two such depictions in his 2011 paper Preludes to dark energy: Zero-point energy and vacuum speculations. One features de Sitter blowing lambda, the other features a cherub blowing vacuum energy. Kragh says the former featured in the Algemeen Handelsblad in 1930, and the latter was Wolfgang Priester’s illustration from 1984. In both cases, what they’re blowing into, is a balloon.
The balloon analogy
The balloon analogy is often used when talking about the expanding universe. It’s sometimes presented as some kind of explanation for some kind of higher-dimensional curvature, or as some kind of reason why the universe has no centre. See for example Where is the Center of the Universe? written by Brian Ventrudo in 2012. However as Charles Lineweaver and Tamara Davis point out in their 2005 SciAm article Misconceptions about the big bang, “this balloon analogy should not be stretched too far”.
Image from the Charles Lineweaver and Tamara Davis SciAm.article Misconceptions about the Big Bang
That’s because there’s no evidence of any higher dimension, or for any grand-scale spatial curvature. Nevertheless the balloon analogy is useful. Imagine a balloon in vacuum as an analogy for the universe. The pressure of the air inside the balloon is counterbalanced by the tension in the skin, and the result is in a stable equilibrium. You can then make the balloon expand by blowing in more air. You inflate it by increasing the pressure, whereupon the balloon expands until the pressure and tension are again in equilibrium. But since energy is pressure times volume, that would be in breach of conservation of energy. I know of no perpetual motion machines. The ascending photon doesn’t lose energy. The descending photon doesn’t gain it. If you send a 511keV photon into a black hole, the black hole mass increases by 511keV/c², no more. In similar vein I’d be happier if conservation of energy applied to cosmic photons, to the balloon analogy, and to the universe at large.
The bubble-gum balloon
As it happens there is another way to make the balloon bigger. Not by blowing in air to increase the pressure, but by reducing the tension. This sounds impossible until you think bubble-gum. Blow a bubble-gum bubble, and you are inflating a balloon of sorts. But a balloon with a difference. As this balloon expands, the skin gets thinner and weaker, and less able to resist the expansion. So it expands further, so the skin gets weaker, and so on. The pressure drops, the volume increases, and energy is conserved. This fits with the negative pressure. Tension is negative pressure. I rather think the universe isn’t expanding because somebody is blowing cosmological constant into some balloon. But because there’s a cosmic pressure, and the strength of space is getting weaker. I am reminded of what happens to drooping Silly Putty when you stretch it out. It sags, faster and faster.
Space is dark
Silly Putty is made of matter. What’s matter made of? Energy. What’s dark matter made of? Dark energy. Energy is energy, and when dark energy is inhomogeneous it has a mass equivalence and a gravitational effect. Then we call the delta dark energy dark matter. So what’s the true percentage of dark energy? Is it 73% + 23% = 96%? No, because energy is energy, and it’s the one thing we can neither create nor destroy. If it’s propagating we usually call it light, then if it’s going round and round we call it matter. That’s what Einstein was talking about in Nottingham in 1930. He said “the strange conclusion to which we have come is this – that now it appears that space will have to be regarded as a primary thing and that matter is derived from it, so to speak”. Matter is made of energy, as is antimatter. Dark energy is vacuum energy and spatial energy, as is field energy. It’s the self-same thing as gin-clear ghostly elastic space. What’s space made of? Energy. What’s energy made of? Space. Like electricity and magnetism, they are two sides of the same coin. So the true percentage of dark energy isn’t 68%, or 73%, or 96%. It’s 100%. Where is it? It’s all around. It’s hiding in plain sight. You are made of it. It reminds me of Chinatown: She’s my sister. Slap. She’s my daughter. Slap. She’s my sister and my daughter. Look up to the clear night sky. Space is the same stuff as energy, and there’s a lot of it about. And space, of course, is dark.