Asia-Pacific Forum on Science Learning and Teaching, Volume 9, Issue 1, Article 1 (June, 2008)
Ling L. Liang, Sufen Chen, Xian Chen, Osman Nafiz Kaya, April Dean Adams, Monica Macklin and Jazlin Ebenezer
Assessing preservice elementary teachers views on the nature of scientific knowledge: A dual-response instrument

Previous Contents Next

Methodology

 Student Understanding of Science and Scientific Inquiry (SUSSI) was developed to evaluate NOS views of preservice teachers.  SUSSI was based on the conceptual framework presented in the NOS literature, and the most current national science education reform documents. The procedure for validating SUSSI consisted of four phases:  (1) Selection of standards- and literature-based NOS items, (2) Pilot test and interviews, (3) Expert review and field test, and (4) Further revision and field test.    

Phase I - Selection of standards- and literature-based NOS items. 

Target ideas germane to NOS were gathered from the national and international Standards documents (e.g., AAAS, 1993; NRC, 1996; McComas & Olson, 1998), and three existing NOS instruments, VOSTS (Aikenhead & Ryan, 1992), VNOS (Lederman, Abd-El-Khalick, Bell, & Schwartz, 2002) and VOSE (Chen, 2006). The target NOS ideas reflected tentativeness, empirical basis, observations and inferences, creativity and imagination, social and cultural embeddedness, scientific theories and laws, and multiple methods of scientific investigations.  The NOS ideas were modified into 58 Likert scale items and 10 open-ended questions. The items and questions were developed based on empirically derived instruments such as VOSTS, VOSE and students' responses on VNOS. Following is one of the questions with five Likert items and an open-ended question.

Question 6.  Do you think that scientists discover scientific theories (e.g. atomic theory) just like gold miners discover gold, or that scientists invent scientific theories somewhat like artists invent sculptures?   

 

A.    Scientists discover theories that are embedded in nature.

B.    Scientists discover theories from experimental facts.

C.    Some scientists may discover theories by chance, but other scientists may invent theories from facts they already know.

D.   Scientific theories were invented by scientists to explain the observed or perceived natural phenomena.

E.    Scientific theories were invented and tested by scientists.

 

Please explain the difference between discovering scientific theories and inventing scientific theories. If you can, please use an example to illustrate your idea.

 

 

Phase II - Pilot test and interviews.   

Between the summer and the fall semester of 2004, SUSSI was pilot-tested with 40 American preservice elementary teachers.   In addition, 20 preservice elementary teachers were interviewed. The interview data were used to further modify certain Likert statements and translation phrases. For instance, in the sample question (No. 6) shown above, participants tended to select the choice of "agree" or "strongly agree" when responding to the statements 6A (Scientists discover theories that are embedded in nature) and 6B (Scientists discover theories from experimental facts).  During the interviews, however, it was found that the preservice teachers did not necessarily all fail to recognize the invented and creative nature of scientific knowledge. Some of them did not read the focus question (or the heading statement) where the term "discover" was defined.  They chose "agree" or "strongly agree" mainly because they were familiar with the phrase "scientific discovery" frequently used in everyday language.  Therefore, in the revised version, the original statements 6A and 6B were replaced with "Scientific theories exist in the natural world and are uncovered through scientific investigations."

Phase III - Expert review and field test.

The revised SUSSI was reviewed by an expert panel of nine international science educators who are currently engaged in either NOS research or teaching. The panel's comments and suggestions for improvement were used to modify the items.  In 2005, the revised SUSSI was administered again to 60 American undergraduate students. The administration time was about 30 - 40 minutes.

To analyze the Likert items, a taxonomy of views about NOS was created based on the existing literature. All 58 Likert items were classified into two groups:  positive or negative items. The statements marked as '+' represented views consistent with the current National and International Science Education Reform documents, whereas the items with '-' signs represented common student naïve understandings of NOS that are not consistent with the Standards documents (Appendix B). For each of the 'positive' Likert items, student responses were assigned with numbers ranging from one to five (from 'strongly disagree= 1' to 'strongly agree=5'). The scores were assigned in a reverse order for each 'negative' Likert item.  Meanwhile, a scoring guide for independently analyzing students' constructed responses to the open-ended questions in the SUSSI was also developed. The rubric was used to analyze the consistency between the students' responses to the Likert items and their constructed responses (see Table 1 for example). Student responses to each Likert item were rated as "Consistent" (C) or "Not Consistent" (NC) with constructed responses to each associated open-ended question.  A code "NA" was assigned when student-constructed responses did not address any content related to the examined Likert item. Likert items in SUSSI that were identified as "Not Consistent" were removed and/or modified.  Finally, the overall structure of the SUSSI and certain items were modified to enhance clarity and readability (Appendix A).

Phase IV - Further revision and field test. 

The current SUSSI targeted six NOS themes:  Observations and Inferences, Tentative Nature of Scientific Theories, Scientific Laws vs. Theories, Social and Cultural Influence on Science, Imagination and Creativity in Scientific Investigations, and Methodology in Scientific Investigations. Each theme consists of four Likert items, involving both the most common naïve ideas and informed views, and an open-ended question (Appendix A).

During the data analysis phase, the taxonomy created earlier was used again for classification of the 24 Likert items, and a new scoring guide was developed for analyzing students' constructed responses to the open-ended questions associated with each of the six themes (see Table 1).  Student responses on at least five completed surveys were first coded by three members of the research team, and an average inter-rater reliability higher than 80% was achieved. The coding of the remaining responses was completed by two research team members using the common rubric.    

Table 1: Sample SUSSI Scoring Guide for Evaluation of Constructed Responses

Not Classifiable (NC)

Naïve View (1)

Informed View (3)

There is no response; they state that they do not know; the response does not address the prompt; OR the response cannot be classified based on the rubric descriptions.

 Scientists observations and/or interpretations are the same no matter which scientist observes or interprets because scientists are objective or because observations are facts. Scientists observations and interpretations may be different because of their prior knowledge, personal perspectives, or beliefs.
OR
  The response includes contradictions of basic assumptions concerning the nature of science or self-contradicting statements.  

Sample

This study adopted a convenience sampling technique, and involved 209 preservice elementary teachers who were enrolled at two American universities, one in a rural area and the other in an urban area. The participants were either majoring in elementary education (K-6) or had dual majors in elementary (K-6) and special education (K-12). 

Copyright (C) 2008 HKIEd APFSLT. Volume 9, Issue 1, Article 1 (Jun., 2008). All Rights Reserved.