Cosmic Calorimetry or: How I Learned to Stop Worrying and Love Hydrogen

The next time you are enjoying the sun’s warm rays, think of the tremendous voyage those photons have taken to get to you. Traveling, by definition, at the speed of light they left their point of origin about eight minutes previously in a furious haul to the Earth. Our detectors are the eyes and skin, which are sensitive to visible photons and infrared photons. Our bodies absorb photons at higher and lower energies, but the effects are only noticed after time and most often, unfortunately, for the damage they have done to our biologic makeup. In these sun-filled moments, it is important to remember that the overwhelming majority of the sun’s photons will travel undisturbed for the entire future history of the universe, never meeting a planet, a person, a cell, or even another particle.

The story couldn’t be more different for photons coming from the distant universe, which face the barrier of intergalactic hydrogen. Hydrogen is the most abundant atom in the universe, and it’s also the simplest and most well understood. A photon with energy above 13.6 electronvolts (or 1 Rydberg, named after the Swedish physicist Johannes Rydberg) can tear the sole electron away from a hydrogen atom, and thus completely ionize it. Young massive stars, supernovae and other explosions, and accreting black holes are prodigious producers of these ionizing photons. Photons with less energy or a lot more energy simply don’t wreak such hydrogenic havoc. Below this energy, some photons have the specific energy required to excite the neutral hydrogen atom, jostling its sole electron to higher energy levels as opposed to removing it completely. The electrons are excited to a higher level and eventually decay down, cascading back to the ground state. Those special photons with energies at exactly 3/4 Rydberg produce “Lyman-alpha” excitation—the transition between the ground state and the first excited energy level (and named for the discoverer Theodore Lyman). These wavelengths are beyond our vision but for very distant sources where the expansion of the universe has redshifted them to visible wavelengths. It is precisely this interplay between the hydrogen in the universe and these photons that astronomers can use to figure out the nature of photon producers over cosmic time, as well as the fundamental parameters of our universe, such as its matter content.

The easiest objects to see in the distant universe are quasars powered by gas accretion onto supermassive black holes. The 1965 observation of a quasar at a redshift of two was transformative. Immediately upon seeing the spectrum, the cosmologists of the time recognized something that remained a puzzle for decades after the observation. Jim Gunn and Bruce Peterson noticed that, contrary to their expectations, they could actually detect the UV photons coming from this quasar in the distant universe. We can tell that a photon has been absorbed because this light is “removed” from the spectrum we observe—so instead of light, we see darkness. At that time, as ever, cosmologists were struggling to understand the basic geometry and content of the universe. If prevailing wisdom at the time was correct, these photons should never have arrived at our telescopes on Earth. The great barrier of neutral hydrogen atoms throughout the universe should have absorbed them—all of them. Why was the universe essentially transparent?

At this time, the density of neutral hydrogen was not known, but it was known that this was a key cosmological parameter. If the density of neutral hydrogen was comparable to the smoothed-out density of matter in stars, the universe would be filled with it and, along every direction, photons would encounter this gas and once they reached the special Lyman-alpha transition, the hydrogen atom would absorb it. It was thus immediately clear that the density of these neutral atoms must be extremely low: either there were very few hydrogen atoms in intergalactic space, or almost all of these atoms were completely ionized.

While Gunn and Peterson found that intergalactic space was mostly transparent, as telescopes got bigger and bigger and instruments could take finer and finer spectra, it was realized that there was absorption, but it was concentrated in narrow lines corresponding to concentrations of neutral hydrogen. This pattern of narrow absorption spikes was subsequently called the Lyman-alpha forest owing to its appearance in the observations. For decades, theorists struggled to understand why photons were streaming across the universe unimpeded by neutral hydrogen. Many different ideas for the distribution of neutral hydrogen in the cosmos were put forward. Finally in the 1990s, it became possible to numerically compute the cosmological evolution of the universe accounting for both the gravitational and hydro dynamics of its contents.

These simulations showed that the Lyman-alpha forest could be reproduced remarkably well by the leading theory of structure formation, with absorption arising in filaments created by gravitational collapse—a metastructure we now refer to as the Intergalactic Medium (IGM). In this model, 99.999 percent of the hydrogen is ionized, but the remainder is enough to produce the Lyman-alpha forest. The comparison of simulations and observations further revealed that most of the atoms in the universe are in the gas that produces the Lyman-alpha forest.

The success in explaining the Lyman-alpha forest provided an important confirmation of the leading cosmological scenario in a previously untested regime, and it helped to pin down the total density of hydrogen atoms in the universe. Two decades on, the parameters of the cosmological model are well measured by other observations, especially the precise measurements of temperature fluctuations in the cosmic microwave background. The Lyman-alpha forest now allows us to test our understanding of the IGM and the sources producing ionizing photons, including their evolution to lower redshifts using Hubble Space Telescope observations of UV light. As the universe ages and expands, it cools off the intergalactic hydrogen gas—creating neutral hydrogen and more absorption in the Lyman-alpha forest. At the same time, galaxies and black holes grow and evolve, producing the photons that ionize and heat the IGM, effectively “pruning” the forest of absorption. This balance between heating and cooling as recorded in the Lyman-alpha forest thus provides us with a cosmic calorimeter. At intermediate redshifts, this calorimeter is working perfectly. The ionizing photons we count up from the census of stars and quasars is just right to reproduce the observed properties of the IGM. Our IGM calorimeter seemed to be working fine in the local universe as well—until recently.

New instruments on the Hubble Space Telescope, far more sensitive than their predecessors and specifically designed to measure the local Lyman-alpha forest, have indeed produced extraordinarily accurate measurements of the local IGM. Cosmological surveys have made new measurements of quasar and galaxy activity over cosmic time. Cosmological hydrodynamic calculations are now hundreds of times larger and more physically realistic than the original simulations that first reproduced the IGM. Comparing these new observations with the new simulations should have been a matter of a more detailed balance between the theoretical predictions and the observations. However, when we made the first comparison, we found something was broken, a mismatch so striking that we called it the “photon underproduction crisis.” The amount of ionizing photons that the measurements of neutral hydrogen demand is much, much higher (a whopping 200–400 percent depending on whose accounting you believe) than that we obtain by counting up the photons leaking from galaxies and quasars. How can this be? There are a few possibilities but they fall into three broad categories: stars in galaxies, quasars, and exotica.

First, we have the galaxies and their stars. It is extremely difficult to observe the ionizing photons escaping from galaxies directly in the local universe owing to the low surface brightness of UV photons that are able to emerge from their galactic origins. Where this measurement has been made, in all but a few very rare instances, the fraction of ionizing photons escaping galaxies is thought to be quite low. If observations have underestimated the escaping flux (by a factor of 5) or perhaps missed the types of galaxies that leak most of the ionizing photons, then this would help us understand the crisis. Of course, this would then mean that our observations of the escape fraction are greatly underestimated (thus moving the crisis from one side of the balance sheet to the other).

The second possibility is the quasars. If there are more accreting black holes than we have accounted for, that would be helpful. Some estimates for the number density of these systems are higher than others. The most optimistic of these estimates turns the 400 percent deficit into a less uncomfortable 200 percent, but I personally don’t sleep well with that large a pea under my mattress.

Finally, it is possible that there is something really quite exotic happening and the Lyman-alpha forest is trying to send us a clue. Perhaps there is another source of ionization that we haven’t considered at all. One candidate would be high-energy particles produced from the interaction of powerful jets of accreting black holes with the cosmic microwave background. Another candidate would be the decay of the heretofore unidentified dark-matter particle into low-energy photons or heat. Finally, it’s possible that the triumph of explaining the local Lyman-alpha forest was a mere coincidence and the forest is actually produced from some other process. It would then be hard to understand why things work so well in the young universe. We think these exotic possibilities are unlikely, compared to the more prosaic explanation of inaccurate accounting, but they are tantalizing nonetheless. Once we fully understand the ionizing flux emerging from galaxies and quasars locally, these possibilities will be either far more or less interesting.

Rather than worry that our predictions do not match the observations perfectly, we must remember never to let a good crisis go to waste! The detailed use of the Lyman-alpha forest as a cosmic calorimeter will certainly reveal information about the balance of ionizing photons coming from stars and quasars, and perhaps something entirely new about our universe that we have not anticipated.