As You Likert

Do you know those questionnaires where you are presented with a statement and asked whether you (1) strongly agree, (2) agree, (3) neither agree nor disagree, (4) disagree, or (5) strongly disagree? The responses themselves are supposed to be symmetrical and balanced around the neutral position (whether or not it is included). In the trade, these statements together with the ranges are called Likert items, and the sum of the responses is called a Likert scale, after the psychologist Rensis Likert (1903–1946)—which is pronounced LICK-ert, not LIKE-ert, making a mockery of the Shakespearian title here. (I also considered “Some Likert Hot.”) Likert is credited with developing the apparatus on the strength of his paper “A Technique for the Measurement of Attitudes,” published in Archives of Psychology in 1932. And yet, as I recently discovered to my surprise, a couple of sociologists were using such a questionnaire in 1923—and doing so in order to give a hard time to a leading creationist.

In 1923, Stuart A. Rice (1889–1969) and Malcolm M. Willey (1897–1974) were both young instructors in sociology at Dartmouth College in Hanover, New Hampshire, both having recently received their doctoral degrees at Columbia University. When they learned that William Jennings Bryan (above) was coming to their college to give a talk on “Science vs. Evolution,” they thought that it would be interesting to ascertain what the effect, if any, would be on the students—especially because, as they wrote in the January 13, 1924, issue of The New York Times, “Dartmouth is one college in the country where every undergraduate is required to take a course in evolution and where a department of evolution has a place of equal importance with the departments devoted to the classics, the social sciences[,] and the rest.” They prepared a questionnaire to administer to 136 sociology students, all of whom had taken the required course in evolution in their first year at the college. The sole question was a Likert item avant la lettre.

With reference to the doctrine that man evolved from lower animal forms, in harmony with general principles of organic evolution:

1. I reject the doctrine completely.
2. While I do not reject it completely, I do not believe that the evidence favors it.
3. I am undecided whether to accept or reject it.
4. While I do not accept it completely, I believe that the evidence favors it.
5. I accept the doctrine completely.

Rice and Willey administered the questionnaire to the students twice, before Bryan spoke and again after Bryan spoke. Two “significant facts” were thereby revealed, they wrote in the Times: that “an overwhelming number of these undergraduates were believers in the doctrine of organic evolution” and that “these beliefs were substantially unchanged as a result of Mr. Bryan’s address,” although they acknowledged that insofar as there was change, it was away from acceptance of evolution:

Understandably, Rice and Willey didn’t inflict a detailed statistical analysis of their data on the readers of the Times, instead devoting a considerable amount of space to the comments submitted with the questionnaire. “A skillful, masterly oration,” opined a representative student, “given by a prejudiced man, who did not have, or did not show that he has, a knowledge of what he was talking about. Such an argument would only convince the uneducated.” 

But when Willey and Rice (reversing the order of authorship) published their paper “William Jennings Bryan as a Social Force” in the Journal of Social Forces in the same year, there wasn’t any detailed statistical analysis to speak of there, either—which is surprising, considering that Rice was a future mover and shaker in the world of statistics, serving as president of the American Statistical Association in 1933. But they provided further data, including data from a group of first-year students who hadn’t taken the evolution course yet but attended Bryan’s talk, unmentioned in the Times article. It isn’t surprising that Wiley and Rice observe, “Partly, it may be presumed, as a result of greater maturity, but in greater part due to their familiarity with the principles of evolution acquired in the compulsory course the percentage of students accepting the doctrine without qualifications was four times greater in the sample representing the upper classes than in the freshman group.”

The Mead Project, which seeks to revitalize research on work of the sociologist George Herbert Mead and collect documents related to his work, says of Willey and Rice’s paper, “This is the first example we have been able to identify of the ‘Likert-style’ response.” Probably part of the reason that Likert is remembered for it, however, is that he thought about it in general terms, carefully articulating the rationale for it in his work, thus helping scientists to apply and develop the method, while Rice and Willey apparently just plunged ahead with their questionnaire at Dartmouth College without reflecting on the underlying methodological issues. I sympathize, actually: I tinkered for a while trying to apply modern statistical tests to Willey and Rice’s data, but my statistical skills are (to put it mildly) rusty, and there were more pressing demands on the time of NCSE’s statistics maven Josh Rosenau. But if you’re not scared by two-tailed t tests, contingency table analysis, or analysis of variance, and have time to kill, drop me a line…