Why we can not exclude the existence of a snowman

How the null hypothesis supports the life of a hairy hominid

Recently, I received a letter from an anthropologist with a comment about a new report published in the journal Proceedings of the Royal Society. The subject of the report was the snow man - more precisely, the genetic analysis of hair, about which different people throughout many have declared that they belong to a giant hairy unknown primate.



An international team of scientists led by geneticist Brian Sykes of Oxford University did not find any evidence that the DNA of these hairs belong to mysterious primates. Instead, for the most part, they belonged to completely unwelcome mammals - porcupines, raccoons and cows.



My correspondent summed up his opinion succinctly: "Well, and then!"



This new work will not remain in history as one of the greatest scientific works of all time. It does not change our understanding of the natural world and of ourselves. But it demonstrates the counterintuitive way of working of modern science.



People often believe that the job of scientists is to prove that the hypothesis is true - the existence of electrons, or the possibility of a drug to treat cancer. But scientists often do the opposite thing: they decide to disprove the hypothesis.



Scientists took many decades to develop this method, but one day of the early 1920s is particularly prominent in this story. At the agricultural research station in England, three scientists took a break for tea. Statistician Ronald Fisher poured a cup and offered it to his colleague Muriel Bristol .



Bristol refused. She preferred the taste of milk tea when she first poured milk into a cup.



“Nonsense,” replied, as they say, Fisher. "Sure there is no difference."



But Bristol stood her ground. She claimed she could notice the difference.



The third scientist, who was present at the conversation, William Roach, suggested to conduct an experiment. (This may also have been a scientific flirtation: Roach and Bristol got married in 1923). But how to verify the Bristol statement? The simplest action for Fisher and Roach was to pour a cup of tea so that she would not see it, give it a try and ask to guess how it was prepared.



But if Bristol guesses, it will not be proof of its high sensitivity. With a 50% chance of a correct answer, she can just guess it by chance.



A few years later, in his 1935 book, The Design of Experiments, Fisher described how to verify this statement. Instead of trying to prove that Bristol could feel the difference between different cups of tea, he tried to reject the hypothesis that her choice was random. “This hypothesis can be called the null hypothesis ,” wrote Fisher. - The null hypothesis is never proven, but it can be refuted in the course of the experiment. We can say that any experiment exists only to give facts a chance to refute the null hypothesis. ”



Fisher figured out a way to reject the null hypothesis — that Bristol makes a random choice. He will prepare eight cups of tea, in four of which he will pour milk in the first place, and in the rest - in the second. He mixes the cups randomly and offers Bristol to try the tea one by one. Then he will divide them into two groups — cups into which, in her opinion, milk was poured first, will fall into one group, and the rest into the other.



Bristol, allegedly, brilliantly passed the experiment, and correctly identified all eight cups. Thanks to the scheme of the Fisher experiment, the chances that it would correctly divide eight cups into two groups at random were small. There were 70 different ways to divide eight cups into two groups of four, which meant that Bristol could determine the correct sorting of cups by accident only in 1 case out of 70.



Fisher’s check could not completely eliminate the possibility that Bristol was guessing. It simply meant that the probability of this is small. He could reduce her even more, forcing Bristol to drink more tea, but he could never reduce it to zero.



Since absolute evidence cannot be found, Fisher preferred a practical approach when performing experiments. In the lab where they worked with Bristol, Fisher had to analyze the data accumulated over the decades to determine if this information could give a hint about various details, such as the best recipe for fertilizing cereals. Scientists could use this data to develop larger experiments with ever increasing accuracy of results. Fisher believed that there was no point in developing an experiment, which could take centuries to obtain the results of. At some point, he thought, scientists just have to finish the work.



He believed that a meaningful error was the value of 5%. If we assume that the null hypothesis is true, and find that the chances for the appearance of data do not exceed 5%, it can be safely rejected. In the case of Bristol, the chances were quite within this framework, they were about 1.4%.



Thanks mostly to Fisher, the null hypothesis has become an important tool for scientific discoveries. Null hypothesis tests can be found in all areas of science, from psychology to virology and cosmology. And scientists use a five percent error after Fisher.



Which brings us back to the snowman.



For years, the existence of Bigfoot, Bigfoot, Yeti, and other mysterious creatures offered various dubious evidence. The existence of none of them has been proven, but it has not been unambiguously refuted.



People have claimed for decades that they have met hairy humanoids. They offered grainy photographs, ambiguous impressions of footprints, mysterious shreds of hair. In recent years, they have even tried to extract DNA from hair, but scientists have rejected such genetic studies because they did not use the standard security measures used in such research.



Supporters of Bigfoot constantly claim that professional scientists deliberately ignore irrefutable evidence. The problem is that these supporters do not use a scientific approach to the issue of Bigfoot. So two years ago, Sykes and his colleagues decided to conduct a scientific study of this hair with an "anomalous primate." This included the creation of a null hypothesis, which must be tried to refute.



They developed the following null hypothesis: the hair that supposedly belonged to the snowman (bigfoot, yeti, or other local species of the creature) actually does not belong to an unknown primate, but to a known mammal. They extracted DNA fragments from 30 different hair samples and were able to isolate the same short section from each sample. They then compared it with the corresponding DNA regions obtained from a variety of living mammals.



The results were clear: the scientists found an exact match with known mammals for all 30 samples.



Does this mean that Sykes and his colleagues have refuted the existence of Bigfoot? Not. It simply means that Sykes, unlike Fisher and his tea, could not refute the null hypothesis. But the question of the existence of Bigfoot remains open, and always will be.



This is not to say that there were no surprises in Sykes research. Two samples of hair from the Himalayas coincided with DNA extracted from a 40,000-year-old fossil polar bear. Strangely, their DNA does not coincide with polar bears living today.



In the report, Sykes and colleagues offer a version of how this could happen. It is possible that the ancient polar bears and brown bears interbred with each other, and some of the bears living in the Himalayas still carry a piece of the DNA of an ancient polar bear.



Some skeptics offered an alternative explanation. It is possible that DNA, similar to what a polar bear had, was taken from a living mammal, say, a brown bear, which has undergone a couple of mutations that created a false similarity to the DNA of an ancient polar bear.



These skeptics, in fact, created a null hypothesis. And you can refute it in a simple way. Scientists need to find more DNA of these mysterious bears. If other sections of DNA also coincide with sections of ancient polar bears, then this null hypothesis can be rejected.



This is how science advances, from one null hypothesis to another.



Karl Zimmer is a columnist with The New York Times, author of 12 books, including A Planet of Viruses.