WHILE DOING THE READING that led to my earlier post this morning, looking at things from Scientific American, scientific journals, etc. I came across something at Retraction Watch about a study allegedly concerning religion published in Nature which has now been retracted, sort of. The original study was one I remembered because as it says in the piece:
A widely-touted 2019 study in Nature which argued that large societies gave rise to belief in fire-and-brimstone gods — and not the other way around — has been retracted by the authors after their reanalysis of the data in the wake of criticism diluted the strength of their conclusions.
The article, “Complex societies precede moralizing gods throughout world history,” came from a group of scholars in the United Kingdom, the United States and elsewhere, and was led by Harvey Whitehouse, an anthropologist and the the director of the Centre for the Study of Social Cohesion at the University of Oxford.
The study prompted a significant amount of interest on social media and in the global press, according to Altmetric, with articles in Scientific American, Yahoo! News, PBS, El Pais and many other publications worldwide. As Scientific American put it, Whitehouse’s group found that the advent of moralizing gods did not lead to the formation of complex societies. [Note, you can see the links to the popular press coverage at the link.]
And the media-social media coverage - they're always quick to cover anything that could discredit religion - led to the certainly anticipated and expected comment thread babblage by the God haters, the Christian bashers are the primary audience for this kind of audience-pleasing science. And that popularity isn't found only on the popular level but among other scientists one expects from allied fields:
The paper, which has been cited 49 times, according to Clarivate Analytics’ Web of Science — giving it a Highly Cited Paper designation among papers of the same age.
Well, typical for the social sciences, entirely expectable for anyone of any experience of them, their methodology was dodgy and so their results were:
[The paper] also met with skepticism. Immediately after publication, a group led by Bret Beheim, of the Max Planck Institute for Evolutionary Anthropology, submitted a response to the journal questioning the validity of the results. Beheim and his colleagues posted a preprint of their argument on PsyArXiv in early May 2019. Nature accepted the article on Feb. 18, 2021, roughly 21 months later.
The core challenge from Beheim et al. is rather that the original paper treats absence of evidence for moralizing gods as evidence of their absence. It changes all N/A (not available) values to absent. In other words, the original paper considers that if there is no information on the presence of moralizing gods in a certain region at a certain time, there is no moralizing god.
After reanalyzing their results in light of the challenge, Whitehouse’s group agreed that the findings weren’t quite as robust as they’d initially claimed.
I have to wonder what would happen to a student taking an introduction to statistics course at any of the universities that employ these researchers who made that kind of an obvious error in their analysis of and misapplication of data. I wonder if some statistics teacher catching such an error would write the old rule of thumb popularized by the Milton Berle of such aphorisms, Carl Sagan, that absence of evidence is not evidence of absence. I would imagine a lot of those guys who did that had said that, themselves.
What might be most telling in this is not only did that most august of science journals, Nature accept the original obviously flawed study (you do have to wonder about the review process that did not catch the flaws) but in how they dragged their feet in publishing the critique of it once that was pre-published.
Beheim told us his group was surprised by the long delay between the submission of their article and publication:
[It] took nearly two years from submission to acceptance, with only clarifying revisions made to the analysis. Our three referees were excellent, but the journal was not forthcoming about anything for over a year. Thank god we posted a preprint at submission.
I look forward to reading the … revised argument. The objection I had to the original analysis wasn’t so much their conclusion, but that they weren’t transparent about the amount of missing data involved. If our critique can help them nail that problem down, I think the field will progress.
It being anthropology I wouldn't count on that. Anthropology as in all the sciences allegedly dealing with human minds is never going to actually practice rigorous science because rigorous science cannot be done on the object of its supposed study. It can, though, make up stuff that is a simulation of science, in which case their imitation should be held to a higher standard of pretense than Nature let them get away with. As he admitted, even the anthropologist who led the criticism wasn't bothered by the result but in the way they got to it.
Nature's excuse for the long delay in allowing such a frequently cited, much followed in pop-sci followage is pretty telling in a way that made me decide to write about this.
Matters Arising submissions that meet Nature‘s initial selection criteria are sent to the authors of the original paper for a formal response. The comments and formal response may then be sent to independent referees for peer review, as was the case in this instance. As with primary research papers, the period between submission and acceptance for Matters Arising can vary, as it incorporates peer review, careful consideration and evaluation by the editors of the issues raised, and author revisions. In addition, the consideration of the Matters Arising in this case was associated with the retraction of the original paper, which added complexity to the process.
I strongly suspect that the comparative speed and obvious lack of adequate publishing of the original paper was due to its ideological usefulness for the dominating ideology of Nature and of British-English language scientific establishment. A part of that general establishment such as gave it all of those popular-press citations. All of which are supposedly cleansed out by the rigorous application of the scientific method. Though, clearly, that gets out far less than 100% of that kind of pollution when it's of use to that ideology.
No comments:
Post a Comment