Teaching the "E-Word" in Tennessee

Introduction

The publication of Charles Darwin's On the Origin of Species can be reasonably argued as marking the beginning of modern biological science. Since 2009 is the sesquicentennial of this momentous event, we thought it would be appropriate to report on how well some fundamental ideas of science in general, and evolution in particular, are understood by today's university students enrolled in introductory and advanced biological anthropology courses. For the past decade, we have administered a survey during the first day of class designed to gauge how well these students understand science and evolution. Additionally, the survey has the dual purpose of facilitating and guiding the ensuing conversation on the subject after the answer-sheets are anonymously completed and returned.

In this paper we will describe the survey instrument that is administered and discuss the information it intends to elicit. We will then present results derived from quantitative and textual analyses of these data collected from a decade of completed surveys. Finally, we will discuss our occasionally surprising and counterintuitive findings, speculate about their causes, and suggest some remediation that may serve to dispel common misconceptions about science and evolution among university students.

The Survey Instrument: Its Origin and Rationale

The "Science and Evolution" survey (Figure 1) was developed by the senior author to structure and stimulate a discussion with his students during the first day of his introductory physical anthropology class at the University of Tennessee. He felt that this was necessary because he had noticed that some common misconceptions and misunderstandings concerning evolution were consistently being voiced by his students' questions in class, as well as in private conversations outside of class. He decided that directly addressing these issues at the beginning of the semester would benefit all students by encouraging them to share their thoughts on a subject that engendered strong feelings and was generally perceived as controversial and unsettled.

The survey provides four statements whose truth or falsity the student is asked to evaluate. Additionally, the survey includes a request for the student to define "evolution"in their own words.They are asked to supply their responses anonymously on a provided answer-sheet that also collects demographic information from the respondents including age, sex, class standing, home town and state, and how much biology education they had received prior to taking this class.

The first statement "Science has proven that evolution is true" is intended to elicit a discussion concerning the limitations of empirical science. Many students misperceive science as a mechanism by which universal truths are proven. It is important to convey, however, that all empirical scientific statements are tentative and must have the capacity to be falsified if adequate counter-evidence is brought to bear. The discussion emphasizes the dynamic nature of science and contrasts scientific versus dogmatic statements. Interestingly we have noticed, after years of these discussions, that students often answer this question correctly (False) for the wrong reasons and incorrectly (True) for the right ones.Many students with anti-evolutionary sentiments respond that this statement is false simply because they cannot accept evolution (often on religious grounds) while not objecting to the mischaracterization of science's ability to prove an idea true. Conversely, students who more vigorously yet naively accept the evidence for evolution argue that this statement is correct while similarly not realizing that empirical science is not in the business of "proof" or "truth".

The theme of science's capabilities and limitations is further developed during the conversation stimulated by the second statement: "Evolution is not science, it is just a theory."The phrase "just a theory" is used as a springboard to highlight the difference between the common usage of the word theory (equating it with a guess or a hunch) versus its scientific meaning as a powerful explanation that has withstood the test of time. We discuss the difference between fact and theory in the anti-evolutionary context that often has tried to mandate that evolution is not taught as "fact" but only as "a theory."We emphasize to our students that good science teachers do exactly that:evolution (as a scientific theory) explains the facts of the fossil record, comparative anatomy, genetic distance and so on.

We then focus on two commonalities that are shared by all scientific theories (including evolution): natural causation and falsifiability. We emphasize that explanations in science can only invoke natural causes whose evidence may be detected by our senses or our instrumentation. Therefore, anything in our observable universe is potentially open to scientific investigation. Nevertheless, anti-evolutionists often contend that evolution fails as a scientific theory on the second criterion of falsifiability since it is so allencompassing that no evidence can be brought to bear that would cause it to be abandoned.Therefore, these anti-evolutionists claim that evolution is a dogma that has been elevated to the status of a secular state religion. However, we counter these claims with our students by asking them if they can imagine any evidence that could potentially be uncovered tomorrow, next year or in the coming decades that would cause us to discard our current understanding of human evolution. Inevitably, one student provides exactly that kind of challenge: what if a fully human skeleton were to be found in direct association with dinosaurs? We joke that we would be out of a job if that discovery were made but we stress that is the kind of naturally-occurring evidence that has the potential to overturn human evolutionary theory. At this point, we also share the dictum that "extraordinary claims require extraordinary evidence" exemplified by the anti-evolutionary contention of human and dinosaur footprints preserved together at the Paluxy River in Texas (see p 39 of this issue of RNCSE). Paleontologists subsequently demonstrated that the "human" tracks were made by three-toed dinosaurs and today these claims of Texan human/dinosaur contemporaneity have been disavowed by most anti-evolutionists.

The third statement "Creationism is a legitimate scientific alternative to evolution" is included to illustrate the fundamental differences between scientific and non-scientific (including religious) explanations. In this discussion, we focus on the Judeo-Christian creation story and ask our students "What are the first five words of Genesis?"They respond: "In the beginning, God created ..."and acknowledge that since this is an invocation of supernatural causation, it is not a scientific explanation. We then press them to elaborate and most often they come to the realization that the presence or absence of God cannot be tested.This discussion demonstrates to our students that "scientific creationism" as well as its current manifestation as "intelligent design"both fail as science on the dual criteria of natural causation and falsifiability. At the end of this conversation, however, we clearly state that creationism is absolutely a legitimate religious alternative to evolution, while reaffirming that it is simply not a scientific one.

The last statement that we ask them to evaluate is likely the most important from the students' perspective: "If you believe in evolution you cannot believe in God." We kick-start this discussion by having them ask us if we believe in evolution.The surprise is palpable in the room when we answer that we do not, because belief implies taking something on faith. Rather, we accept evolution based on the preponderance of supporting evidence that has accumulated over the years. The ensuing dialog reveals the striking diversity of students' personal philosophies that many have constructed allowing a belief in God alongside an acceptance of evolution. These accommodations run the gamut from progressive evolution to deism to theistic evolution. We acknowledge that some religious doctrines are not compatible with an acceptance of evolution, particularly the more fundamentalist denominations of Christianity, Judaism, and Islam that interpret their holy texts as inerrant and historically factual. We note, however, that most of the mainstream organizations representing these religions have publicly stated that they perceive no inherent conflict between their religion and evolution.

We use this opportunity to restate that science is mute on many of the "big" questions: "Is there a God?","Do we have ultimate purpose?","Is there life after death?" and we acknowledge that religion has traditionally been the primary source of guidance on these issues. We then steer the discussion to what we believe to be the root cause of the creationism/evolution conflict in the United States today: the inappropriate intersections forged by some between the religious and scientific cognitive fields (Bunge 1984). We explain that the belief that evolution and God are mutually incompatible is ironically shared by both extremes along the creationism/evolution spectrum. On the one hand, there is the fundamentalist hell-fire and brimstone preacher who claims that those who believe in evolution are going to hell; while on the other there is the atheist evolutionist whose materialist philosophy does not allow God to exist (for example, Dawkins 2006). We support the "nonoverlapping magisteria" (NOMA) principle (see Gould 1997) that explains how these conflicts can be avoided if the boundaries between religious and scientific "ways of knowing" are acknowledged and respected by all. We convey these distinctions by asking our students to describe the kinds of questions appropriately addressed by these two distinct knowledge spheres, and they quickly come to realize that science is best equipped to answer "what?", "when?", "where?" and "how?" while we most often turn to religion to answer "why?" We conclude by refocusing on the atheist evolutionist. First, we reaffirm their right to claim on personal philosophical grounds that there is no God, just as we defend anyone else's right to believe in a supreme being. However, we strenuously object when these scientists use their academic bully-pulpits to contend that their atheistic conclusions are scientifically grounded.

Materials and Methods

The Dataset

The data used here were collected from the survey described above over a ten-year period (1998–2007) to students enrolled in three physical anthropology courses at the University of Tennessee. These courses are titled Human Origins (Anthropology 110, a lowerdivision, introduction to physical anthropology course), Human Paleontology (Anthropology 495, an upper-division majors course) and Paleoanthropology (Anthropology 582, a graduate-level course).The total sample includes 1079 surveys; 860 completed in the introductory class, 176 from Anthropology 495, and the remaining 43 from the graduate-level course. The modal student from this sample was a 19–year-old female sophomore from Tennessee who had completed high-school biology only and was enrolled in Anthropology 110.

Hypothesis Testing

Since we were primarily interested in investigating initial student understanding of science in general and evolution in particular, we focused most of our analytical attention on the data provided by our 860 introductory students. These data were a rich source of information that allowed us to test a number of null hypotheses regarding the student scores on the true/false statements. The null hypotheses stated that student scores should not differ by class standing, age, what year they enrolled in the course,how much biology they had had previously, and geography (including in-state versus out-of-state, urban versus rural, and differences in home region). Secondarily, we wanted to compare the performance of our upper-division and graduate students to each other as well as to the introductory students.

The student definitions of evolution provided an additional source of compelling data. Using these definitions, we were able to test student familiarity with the classic definition of microevolution ("change in gene frequency within a population over time") as well as the more generalized usage ("biological change over time"). We were also able to evaluate the prevalence of misunderstandings and misrepresentations of evolution through analyses of these definitions. For example, we could identify student definitions that invoked progress, supernatural causation and Lamarckian notions of "wants" or "needs." As in the true/false data described above, we compared definitions of evolution between the students enrolled in the introductory, major and graduate courses.

Analytical Procedures

The quantitative true/false data were analyzed in SPSS 15.0 for Windows (version 15.0.1, released November 22, 2006). Descriptive demographic statistics and frequencies of correct answers were first generated then the null hypotheses described above were evaluated by a series of univariate analyses of variance tests.

The evolution definitions were analyzed in SPSS Text Analysis for Surveys 2.1 (released in 2007). Each student definition was input to the program verbatim and evaluated for how well it matched the classic microevolution and general evolution definitions (described above). In addition, each was analyzed to determine if and how many invocations of misrepresentative concepts (such as progress, supernatural causation, and so on) were included in their definitions.

These text analyses were accomplished by instructing the program to compare each student definition of evolution to an "ideal" definition. For example, the first textual analysis evaluated the input definitions in comparison to the classic microevolutionary "change in gene frequency within a population over time." The ideal definition should reflect the five words that are bolded in the previous sentence. Any definition that included all five terms would earn a maximum score of 5. In addition, the program has the flexibility to include user-input synonyms for each relevant term so in the above example "genetic"and "allele" could substitute for "gene" while "generations" and "the ages" would be treated as synonymous with "time". Similarly for the more generalized definition of "biological change over time" (which had a maximum score of three) the program was instructed to accept "life" and "species" for "biological." Finally, to evaluate the definitions that invoked misrepresentative concepts, the program was instructed to identify (up to six) terms that reflected the following: 1) progress (including advance/advancement, better/best, improve/improvement, perfect/perfection), 2) supernatural causation (including God, the Creator), 3) exclusive application of the concept only to humans (no mention of other life), 4) equating evolution with individual growth and development, 5) ultimate origins (of life, the earth, the universe) and 6) Lamarckian notions (including wants or needs).

Results

Quantitative Hypothesis Tests

A maximum score of 4 was achieved if the student responded that all four statements were false. Less than 10% (81/860) of the students in Human Origins, the introductory physical anthropology course, achieved the maximum score, and the average for these students was 2.55 (62.5%). The scores of these students for the first three questions were similar, ranging from 51% to 58% correctly answered, while the frequency of correct scores for Statement #4 ("If you believe in God, you cannot believe in evolution.") was much higher at 92% (Figure 2).

As would be expected, the students enrolled in the upper-division Human Paleontology course scored significantly better (p .001) than the introductory students with an average number correct of 2.99 (=74.8%). Similarly, the graduate students enrolled in Paleoanthropology outscored (p .01) the majors in Human Paleontology with an average score of 3.35 (83.8%; Figure 3).

The demographic diversity present among the introductory students provided a rich source of data to test a number of commonsensical and intuitive ideas concerning which student subsamples better understood science in general and evolution in particular. For example, we presumed that these data would demonstrate that seniors would score better than freshmen, that students who had already completed college-level biology would score better than those who had not, and that students who graduated from urban high school districts in Tennessee would score better than those who graduated from rural high schools. None of these expectations was borne out by the data analysis; the analysis of variance tests would not allow us to reject any of the three null hypotheses of no difference among these groups.

We were similarly unable to reject our other null hypotheses of no difference in performance based on comparisons between younger (17–18 years old) college- age students versus older (20–21 years old), and Tennessee versus non-Tennessee residents. However, as we delved more deeply into the issues of age and geography, significant differences did appear. For example, "non-traditional" college students (defined here as age 30 and older) scored better (2.79 versus 2.61) than their 17–22 year old classmates (p .05). When we looked at these data regionally, students who went to high school in the northeast scored significantly better (3.22 versus 2.64) than those from the southeastern US (p .001).

Evolution Definitions

Student definitions of evolution that included all five relevant terms (or their synonyms) reflecting the classic microevolutionary meaning ("change in gene frequency within a population over time") were exceedingly rare. In fact, of the total number of definitions recorded (over 1000) in the decade of survey collection, only two received the highest score of 5 by incorporating all of the necessary words. Not surprisingly, both of these definitions were provided by graduate students in the Paleoanthropology class including this example from a 38–year-old female from Connecticut who was enrolled in the course in 2003: "change in gene frequencies within a population over time due to differential reproductive and survival rates." The modal score for the entire sample when the student scores were evaluated against the "ideal" microevolutionary definition was in fact 0 (34.4%), meaning that over one-third of the students did not use a single required term or its synonym in their definition.

Nearly 30% of the sample used one term and approximately the same percentage invoked two. However, only 3.7% used three, and 1.2% used four. As would be expected, the upper-division majors enrolled in the Human Paleontology course used more of the required terms in their definitions than the introductory students (1.47 versus 0.91; p .001) while the graduate students taking Paleoanthropology outperformed the Human Paleontology undergraduates (2.26 versus 1.47; p .001; Figure 4).

When the student definitions were evaluated against the more general definition of evolution ("biological change over time"), a much higher percentage of respondents (17.2%) achieved the highest possible score of 3. The following is an example of one of these definitions provided by a 22–year-old male junior from Tennessee enrolled in the introductory class in 2002:"[evolution] is the gradual change of a species over a long long period of time."However, nearly the same percentage of students did not use any of the three required terms (17.9%), while 30% used one and 35% incorporated two of the words or their synonyms into their definitions. Again, as would be expected, the upper-division majors enrolled in the Human Paleontology course used more of the required terms in their definitions than the introductory students (1.87 versus 1.39; p .001) while this time the graduate students taking Paleoanthropology did not significantly out-perform their Human Paleontology counterparts (1.98 versus 1.87; Figure 5).

When the student definitions were evaluated with respect to the six misrepresentative concepts, we were pleased to note that the modal definition did not include any of these ideas (61%) and that the "poorest" of these definitions included a maximum of three misrepresentations (1.4%). Examples of definitions that earned scores of 3 included one from a 21–yearold female junior from Tennessee who enrolled in Anthropology 110 during the summer term of 2006: "The beginning of time. Who or what created the Earth & all humans. My opinion evolution is God who created all things." This definition invoked ultimate origins (of time and earth), supernatural causation and the exclusivity of the concept as applied only to humans. Another score of 3 was earned by a 25–yearold male senior from Tennessee who enrolled in Anthropology 110 during the summer term of 2006: "The natural progression of life from simple to complex (based on species' adaptive needs)." His definition invoked progress, the inevitability of increasing complexity and the Lamarckian notion of adaptive need. Similar to the cross-class results described above, the upper-division majors in the Human Paleontology course used significantly fewer misrepresentative concepts in their evolution definitions than did the introductory students, while the graduate students in Paleoanthropology invoked these concepts even less frequently than the anthropology majors (Figure 6).

Discussion

The results of the true/false statement analyses were both encouraging and troubling regarding student attitudes and preconceptions about science and evolution. We were pleased to note that on average nine of ten students consistently did not believe that their faiths precluded an acceptance of evolution. From discussions with our students it was apparent that many had developed personal accommodations that allowed these ideas a mutual coexistence. These accommodations were very diverse, spanning across the creation/evolution continuum as described by Scott (1999). Statements from mainstream Judeo- Christian denominations affirming the compatibility of their religious tenets and the evidence for evolution (as described in the National Center for Science Education's Voices for Evolution; Sager 2008) probably contributed to this result as well.

We were also encouraged that our expectations of better performance on both the true/false statements and the evolution definitions by the anthropology majors and graduate students were borne out by the data analyses. In every comparison, the majors enrolled in Human Paleontology scored significantly better than the introductory students while the grads in Paleoanthropology outscored the majors in all but one comparison. This result was at least partially due to the fact that a prerequisite for the major Human Paleontology course was the introductory Human Origins class, so an undetermined percentage of majors had completed the survey twice (the first time in Anthropology 110 and the second in Anthropology 495). Therefore, these students had the explicit benefit of previous knowledge derived from their first experience in one of our classes. However, we have no way of ascertaining how many of the respondents had completed the survey multiple times because of the anonymous nature of its administration.

Our most problematic and unexpected results revealed insignificant differences in the number of correct answers to the true/false statements that were provided by certain subsets of students enrolled in our introductory course. We anticipated that seniors would out-perform freshmen, that 21–year-olds would score better than 18–year-olds and that urban high school graduates' scores would exceed those who graduated from rural schools. None of these expectations was verified by our analyses. One possible explanation for the first two findings is that upper-class students taking an introductory course late in their college career may have enrolled simply to satisfy a requirement and were not that motivated to be in the class in the first place. Since the introductory physical anthropology course satisfies a general education natural science credit at the University of Tennessee, this may indicate that these upper-class students did not have much more of a science background than their freshman and sophomore classmates. Most troubling to us was the fact that students who had already completed a college-level biology course did not score significantly better on the true/false statements than their classmates who had only taken high school biology or none at all. This suggested to us that the basic foundations of science and evolution may not be communicated effectively and are not occupying a central role in some college-level biology courses.

Two findings of significant difference within the introductory student cohort were sensible and expected. "Adult learners" (students over the age of 30) scored better than their traditionally college-aged peers (ages 17–22). This may be because the non-traditional students are often more motivated to be in class than their younger counterparts since they generally are paying their own tuition and must balance other demands on their time including work and family. Therefore they are probably better prepared and have more realistic expectations about what the class entails. The second significant finding revealed that students who graduated from high schools in the northeastern US outperformed those from the southeastern portion of our country. This seems to make sense in the context of widely reported national trends in which the urban Northeast is much less resistant to evolution in their public school classrooms than those in the Southeast. This result may also reflect a comparatively greater emphasis on evolution in northeastern state's biology curriculum standards (Lerner 2000; Moore 2001).

The students' evolution definitions were in some respects mildly disappointing, yet often promising and occasionally entertaining. Our textual analyses revealed a telling paucity of familiarity with the classic microevolutionary definition in that none of the over 800 introductory students incorporated all five of the key words into their definition of evolution. However, we were heartened by the finding that over 50% of all students used at least two of the required terms for the more generalized biological definition and that over 60% of all students did not invoke a single misrepresentative concept in their formulations. The most bizarre definition was offered by an 18–year-old male freshman from Tennessee enrolled in Anthropology 110 during the Spring 2002 semester: "Quarks=atoms=molecules=cells=organs=living being. In other words we were spoken into existence." Finally, enjoy this poetic contribution (worthy of Erasmus Darwin!) from a 24–year-old female junior from Tennessee enrolled in Anthropology 110, Fall 2002: "[Evolution] is a study of how human beings came to be from the protozoa swimming in the sea."

Recommendations and Future Work

Our primary recommendation is to encourage instructors of introductory biological anthropology courses to incorporate a discussion of science in general and evolution in particular into an early class period, if not the first of the semester. We have been uniformly pleased at the high levels of enthusiastic response we have had from our students during these conversations and enjoyed the added benefit of addressing these issues right at the beginning of the semester so they do not have the opportunity to fester as the term progresses. One addition we have considered but not (yet) implemented is an "exit interview"to gauge how well these concepts and ideas have been assimilated by our students by the end of the course.

Secondarily, we would encourage instructors of introductory university-level biology courses to integrate evolution as a central theme throughout their courses if they have not already done so. This apparent deficiency was made clear by the subset of introductory physical anthropology students who had completed at least a semester of college-level biology and yet performed no better on the true/false statements than did their classmates with little or no high-school biology. We would expect that neophyte college biology students would benefit from an early, explicit discussion of science and evolution in their classes at least as much as those enrolled in introductory anthropology.

One of our goals for future work with these data is to better understand regional differences in our survey responses. To that end, one of us (ACD) has had the opportunity to administer a large number of surveys to his introductory students at universities in Illinois and Texas. In this paper we have reported insignificant differences in the true/false response rate between the Southeast (primarily represented by Tennessee, of course) and both the Midwest (including Illinois) and Southwest (including Texas). However, those comparisons involved relatively small numbers of students enrolled in the University of Tennessee's introductory physical anthropology courses who graduated from high schools from those regions. These additional data will add hundreds of survey responses from students attending universities after graduating high schools in the Midwest and Southwest and should better allow us to investigate regional differences in student misconceptions about evolution.

The ten years of completed surveys are potentially informative regarding the identification of any temporal trends in these data. Preliminary analyses did not produce significant differences between any of the annual cohorts in their frequencies of correct responses to the true/false statements.However, this result may be biased by the fact that over 75% of the introductory student responses came from two large sections of the Human Origins course taught in 2000 and 2002. More sophisticated quantitative approaches that we intend to employ may reveal statistically meaningful trends over time in the currently available survey data. In addition, the senior author will be teaching another large section of the introductory course during the Fall 2009 semester which will allow large-scale comparisons (class sizes of ~250) between student responses separated by nearly a decade (2000 versus 2009).

Finally, the senior author has collected a number of surveys completed by honors biology students from local high schools that have not been incorporated into these analyses. It may be quite interesting to discern if this advanced subset of high school students share the same misconceptions concerning science and evolution that we have seen among university students enrolled in introductory physical anthropology courses. Whether or not these misunderstandings are as widespread among honors biology high school students as they are among freshman students in introductory anthropology classes at the university, we as college-level instructors will continue to extol the virtues of our survey instrument and especially the discussions it inspires in helping our students dispel some of the most persistently misleading ideas about evolution in America today.

Acknowledgments

We would like to thank Ann Reed and Bob Muenchen, of the University of Tennessee's Statistical Consulting Center, for their assistance in our quantitative and textual data analyses. We thank John Dansby for his excellent work in data entry. We would particularly like to thank the hundreds and hundreds of students who completed these surveys and invited us into their lives, at least for a semester, so that they could learn about the wonders of anthropology.