NASA Wants Standards for Evaluating Claims of Extraterrestrial Life

The burgeoning field of astrobiology and its search for life beyond Earth has high hurdles aplenty, from hunting for habitable exoplanets to trying to understand life as we don’t know it. All the steps in this research are challenging, but the last step may be the most daunting of all: How to prove to fellow scientists that claims about extraterrestrial life are sound, then how to communicate the findings to the public in a responsible way.

With more such claims expected in the future, NASA’s Chief Scientist Jim Green and his colleagues have become increasingly interested in coming up with community standards for evaluating such findings. As far back as the Viking exploration of Mars in the 1970s, and as recently as last year’s speculations about possible life in the Venus atmosphere, claims of possible extraterrestrial life have been met with strong pushback and contentious debate from other scientists. Some researchers in the field worry that recurring strife puts the credibility of astrobiology at risk, or diminishes the actual scientific importance of some findings because they are ultimately found not to be life detections.

As a result, more than a hundred scientists representing a variety of disciplines are convening (virtually) under NASA auspices this week. They’ll be hashing out issues ranging from how to increase scientific confidence in “biosignature” detections to how best to convey the level of confidence in new discoveries to non-scientists. The hope is that this will lead to formal “best practices” and perhaps even reporting protocols for scientists working in the field.

“The discovery of a potential biosignature in [a planet’s] atmosphere is important, but it’s just the start,” said Green in an interview. “You have to examine potential false positives, whether there are [non-biological] ways to form the chemical, whether the measurement is an artifact of your instrument, whether the environment on the planet is conducive or hostile to life, whether water is present.”

Applying such a chain of reasoning could result in a kind of credibility scale for the general public to turn to when they read about new results in astrobiology. “What we envision is a scale to be worked through by the community; a more formal assessment of confidence they can use to describe how far along in detecting life a finding might be.”

Green and Mary Voytek, senior scientist for astrobiology at NASA, began seriously discussing the need for new life detection reporting guidelines last year, then brought in a number of NASA colleagues. One was Tori Hoehler, a research scientist at NASA Ames who focuses on the habitability of environments beyond Earth and the detectability of any life that may reside there.

Hoehler said the group that organized this week’s workshop had some sense of what might be needed, but wanted input from the larger community. “This may involve changes in how things have traditionally been done—for instance, for a research team to have frank discussions with other scientists in their field before submitting a paper for publication. But there’s a lot of discussion to be had first about biosignatures and how to evaluate and raise confidence in claims being made about them.”

This raising of confidence is essential because any “discovery” will likely be reported in terms of probability. For example, a 90 percent probability might be considered a robust finding that life had indeed been found.

While the idea for a workshop came from NASA officials, it was organized and many of its ideas were developed under the auspices of the agency’s Nexus for Exoplanet System Science (NExSS) initiative and its Network for Life Detection (NfoLD) group. The chairs of the workshop, Victoria Meadows of the University of Washington and Heather Graham of NASA’s Goddard Space Flight Center, are each on the steering committees of these groups, which are sponsored by NASA but include members from many other institutions.

In her opening charge to the workshop attendees, Meadows made the case for why standards of evidence and reporting protocols are important.

“The detection of extraterrestrial life in our solar system and beyond will likely be neither instantaneous nor unambiguous, yet it’s a high stakes scientific achievement that will garner enormous public interest,” she said. The goals of the workshop, she told participants, are to set broad confidence standards separate from the peer review process that already goes into deciding which discoveries and papers are suitable for publication in research journals.

Co-chair Graham introduced the idea of five “Confidence Levels” for evaluating new claims related to extraterrestrial biology. They include rigorous tests of any claimed biosignature, analysis of instrument artifacts, and a deep study of whether the environment was possibly conducive to life. It would also include exacting tests of the “null hypothesis,” which in these cases is to assume that any observed biology, or suggestion of past biology, is due to sampling or experiment error until proven otherwise.

She thought the workshop sessions might become heated—as some discussions of potential life detection findings have—but advised participants to be collegial. “Many of these topics are deeply passionate and personal to us…and we are hoping for active debate,” she told the attendees. But, she added, “Let’s have fun with this. We should really embrace the great excitement of this field.”

The workshop included breakout sessions focused on exoplanets, Mars and icy worlds. Participants discussed how the desired confidence in a particular finding could be increased, via multiple lines of detection using a variety of instruments; through rigorous analysis of whether or not the planet in question could actually support the claimed form of life; and by including in the analysis a broad range of scientists from different disciplines.

The conclusions and recommendations of the four-day conference, which includes participation by scientists in Europe and in Asia, will be synthesized into a white paper that will be presented this fall to the broader astrobiology, planetary science and astronomy communities, then to NASA.

NASA’s Green put the effort in the context of other confidence scales the agency relies on. The 1-to-10 Torino scale, for example, is used to assess the likelihood that an asteroid might hit Earth. NASA also uses a Technological Readiness Level scale to judge whether space instruments or other hardware still in development are mature enough to be included on space missions.

“We need more discipline when it comes to claims of extraterrestrial life, and that may involve a meeting of scientific milestones,” Green said. “They could be of use for ongoing discussions between scientists, and certainly for communications with a public that looks to us for confidence in major findings.”

Marc Kaufman | READ MORE

Marc Kaufman is the founder of the online column Many Worlds (manyworlds.space), and the author of two books about space: First Contact: Scientific Breakthroughs in the Search for Life Beyond Earth (Simon & Schuster) and Mars Up Close: Inside the Curiosity Mission. He also reported on NASA for the Washington Post.