A review of research instruments assessing levels of student acceptance of evolution

Quasi-continuous scales for classifying levels of student acceptance of evolution

The MATE is a 20-item evolution acceptance questionnaire based on a 5 Likert-scale method which is most widely used in evolution education research (Smith, 2010). The actual research tool is provided in Appendix A. Rutledge and Warden (2000) report a very high value of a reliability coefficient of 0.98 for the MATE. The 20 items measure five different aspects related to acceptance of evolution: the scientific validity of evolution, the acceptance of evolution within the scientific community, creationist perspectives on divine creation, human evolution and the age of the earth (Rutledge and Warden, 2000). Student acceptance is then scored from 20-100 possible points, with 20 being the lowest level of acceptance and 100 being the highest level of acceptance. The corresponding scores and categories for acceptance are 89-100, Very High Acceptance; 77-88, High Acceptance; 65-76 Moderate Acceptance; 53-64, Low Acceptance; and 20-52, Very Low Acceptance (Rutledge, 1996).

However, although wildly used, the MATE is not considered as the best tool for assessing student levels of acceptance for a number of reasons. First, the framework for classifying the five continuous levels of acceptance of evolution based on the corresponding scores fails to offer a clear boundary between those accepting evolution and those rejecting evolution, even though the labels “low” and “very low” acceptance may hint at the tendency of rejection of evolution. This critique is supported by the actual use of the MATE by Donnelly et al. (2009). Although they used the MATE as a tool to classify their student participants as “evolution acceptors” and “evolution rejecters”, they did not rely on the suggested framework of the five continuous levels, but adopted a statistical approach to convert the Likert rating scale data to interval data and designed the breaking point to distinguish the two groups of students by themselves. This statistical complication does not only discourage basic users of statistics, but it does also suggest that the classification into two groups is done in an arbitrary manner (i.e. depending on the breaking point decided by the researchers). Furthermore, the suggested approach does not weight items and there is no guarantee that all items provide the same amount of information in relation to the construct of interest, that is, acceptance.

Similarly, although Hokayem and BouJaoude (2008) also used the MATE, they did not classify levels of student acceptance of evolution solely based on this research tool. In fact, they explained, “this questionnaire [MATE] was just used to gather preliminary information to initiate a discussion about the topic [the theory of evolution] and illustrate any changes in mind later on” (p. 401). They actually classified levels of student acceptance based on interviews. It is true that the MATE has been widely used in educational research focusing on evolution education, but not in the way for which it was originally intended.

The second reason is provided by Smith (2010). From a philosophical perspective, Smith (2010) critiques the MATE for conflating knowledge with acceptance. From an empirical perspective, although the MATE has been shown to have a high Cronbach alpha coefficient value, suggesting that the items measure a single factor, Smith (2010) questions whether that factor is really acceptance. He points out: “what does it mean, for example, when a respondent asserts (“agree”, “strongly agree”, etc.) to the following statement: “Evolutionary theory generates testable predictions with respect to the characteristics of life”? Does the respondent accept the statement as true? Does s/he believe the statement is true? Does s/he accept/believe the statement as valid?” To address this issue, Smith (2010) suggests a possible way to measure acceptance of evolution as well as a classification of levels of acceptance of evolution in a way that is less ambiguous with respect to the distinctions between belief and acceptance, which will be discussed later.