And the Gileadites took the passages of Jordan before the Ephraimites: and it was so, that when those Ephraimites which were escaped said, Let me go over; that the men of Gilead said unto him, Art thou an Ephraimite? If he said, Nay; Then said they unto him, Say now Shibboleth: and he said Sibboleth: for he could not frame to pronounce it right. Then they took him, and slew him at the passages of Jordan: and there fell at that time of the Ephraimites forty and two thousand.
—Judges 12:5-6
There’s a lot of bad science in the world, and sometimes it can be tricky to recognize it. Consider the case of The Dr. Oz Show, much criticized by scientific skeptics, and ultimately exposed by a Congressional investigation headed by Senator McCaskill. How is the viewer to know whether Dr. Oz is now staying the course on his revamped and more accurate show? There’s a temptation sometimes to turn to a trick with Biblical pedigree, a special word or phrase that only (or never) get used in real science (as the Ephraimites couldn’t pronounce the “sh” in shibboleth). It would be an elegant solution to a complex problem and has obvious appeal.
Sometimes it works. The antievolution “equal time” laws of the early ’80s, for instance, often used the term “evidences,” rather than treating “evidence” as a collective noun. They defined “creation-science” as “the scientific evidences for creation and inferences from those scientific evidences,” while “evolution-science“ referred to “the scientific evidences for evolution and inferences from those scientific evidences.” That’s not what scientists usually do, but it’s a common feature of religious apologetics (carrying forward a usage common in the 19th century), clear evidence of the religious origin of the bills, as Jay Topkis observed in his oral arguments against the laws before the US Supreme Court. He told the Court:
Now, this bill was of course drafted by a theologian, or somebody versed in apologetics. There's an amusing bit of evidence on that subject in the very language of the bill. The bill keeps using...the Act keeps using the term “evidences” in the plural. We lawyers never speak of "evidences" in the plural. We speak of “evidence,” the singular. And I got nagged by it, and I looked it up the other day. And of course the only dictionary reference to “evidences” is to Christian apologetics: the evidences for Christianity. This is a matter of theological disputation.
Of course, in the aftermath, creationists stopped saying “evidences” quite so much.
Other times, the hunt for a shibboleth goes astray. Ed Brayton calls out a meme circulating in some atheist circles, showing various editions of the Bible and asserting “only a lie has different versions.” Methinks that’s a standard likely to backfire. Darwin’s On the Origin of Species went through 6 editions, and is available now from multiple publishers in different versions. As with any good science, Darwin’s argument changed in light of new evidence, becoming stronger as evidence grew, and addressing counterarguments and new scholarship. Those multiple versions are evidence of the author’s honesty and willingness to engage evidence (new versions of the Bible often reflect similar changes in scholarship on ancient texts and ancient language use).
Abby Hafer took a different approach in a recent paper for The American Biology Teacher. She counted references to “data” and various cognates of the term “argue” (“argument,” “arguing,” etc.) in real science and pseudoscience publications. She found that works purporting to be scientific papers favoring intelligent design creationism had fewer uses of words like “data” and more uses of forms of “argu-” than a group of papers from a Smithsonian research facility. This is offered as evidence that creationists rely more on argument than on data. But can we be sure that people who say “data” have data, or that people who don’t say “argue” aren’t arguing? (I know for a fact that scientists argue, as well they should; argumentation is part of the scientific process.) The comparison works for those two bodies of text, but I wouldn’t recommend this as a general test.
Two researchers took a further step down that path recently by comparing the text of a climate change consensus statement and a climate change denial ripoff of the statement. They found (quoting the paywalled journal article), the legitimate climate scientists “used more conservative (i.e., more cautious, less explicit) language to present their claims compared to” the deniers. The scientists were more formal and more tentative, and by various measures less readable. They conclude, “the language style used by climate change skeptics [sic] suggests that the arguments put forth by these groups warrant skepticism in that they are relatively less focused upon the propagation of evidence and more intent on discrediting the opposing perspective.”
Discussing that paper, my colleague Emily Schoerning suggested these four signs of a good scientific argument:
a lack of emotion, a scarcity of absolute statements, tentativeness in conclusions, and supportive references to many external documents. Scientific arguments generally spend more time building up than tearing town, and direct their use of resources accordingly.
Except for the last of the four, these aren’t necessarily appealing language characteristics.
Indeed, I would not want tentative, unreadable, formal, milquetoast writing to be seen as a sign of scientific credibility. There’s some unreadable dreck in the denialist literature, and while deniers are generally overconfident, scientists do not sacrifice credibility by expressing justified confidence or by making their discoveries accessible. Good writing is a valuable skill which scientists should cultivate, not something they should think they need to avoid to seem credible.
So where does that leave a search for pseudoscience shibboleths? Alas, probably doomed. What separates good science from bad is not some linguistic tic, but the approach to evidence. With training and practice, we can pick out logical fallacies, and look carefully at whether the evidence offered really justifies the conclusions drawn or might equally support some other conclusion. We can evaluate how someone uses evidence: does it seek to test claims, pursuing counterexamples, or simply to confirm presuppositions? But often we have to rely on experts to tell us whether a new finding is plausible or not. That’s why Dr. Oz’s promotion of bogus supplements was so damaging. It’s why measures of scientific consensus matter so much on contentious issues, and why scientists need to be active participants in public discussions. Even the smartest scientist is a non-specialist on something, and needs guidance to separate some kinds of science from pseudoscience. The more good scientists we have in the public square, the harder it will be for nonsense to go unaddressed.
