Effects of storylines embedded within the context-based approach on pre-service primary school teachers’ conceptions of matter and its states

Study Context

The General Chemistry course is a compulsory subject for the divisions of Primary School Teacher Education of Education Faculties at universities in Turkey. All pre-service primary school teachers take this course for two hours per week during their first year. The unit Matter and Its States is included in this course. The content of the unit includes general concepts related to matter and its states. These concepts include common and distinctive properties of matter, heat-matter interaction, pure matter, solution and mixture, melting point and freezing point, evaporation, condensation, boiling, freezing, states of matter, the particulate nature of matter, and phase changes. This content is prepared by Higher Education Council (YOK) and faculties have to teach this content to their pre-service teachers.

Research Design and Participants

A simple experimental design (one group pre-test/post-test design) was chosen for the study and this involved one group that is pre-tested, exposed to a treatment, post-tested and delayed-tested. There are several threats to the internal validity of this design (Robson, 1998); lack of random assignments and a control group limits confidence in assigning causality to an intervention(Trochim, 2001). In particular, because the experimental group is exposed to a specific teaching implementation within a significant amount of time, students in this group are expected to become more successful than the control on post-test (Sadler, 2009). Nonetheless, using only one experimental group without control is plausible and a number of studies employing only one experimental group have been reported in the literature (Çalık et al., 2010; Karslı and Çalık, 2011).

Another threat to validity is, as Trochim (2001) argues, the perception of being involved in an “experiment” (i.e., in this case a teaching intervention) and may result in an apparent improvement in students’ conceptual understanding.

To decrease this validity threat, researchers advocate the use of a delayed-test (Çalık et al., 2010; Karslı and Çalık, 2012). According to them, if gains are due to students involved in an intervention, this cause will probably diminish by the time. Thus, gains after the implementation (in post- and delayed-test) are seen to be evidence of a genuine improvement in conceptual understanding. Because of these reasons, for this study, we preferred using only one experimental group along with a delayed-test.

The study was conducted with 35 pre-service teachers in their first year in the Primary School Teacher Education program of Fatih Faculty of Education at Karadeniz Technical University in Turkey. The implementation lasted for 8 lesson hours (two lessons per week), and each lesson was fifty minutes. The study was conducted during the 2012-2013 spring semester, and the first author taught the students by using context-based approach.

Data Collection Instruments

It is used two instruments and an interview to collect data for the study. The instruments included The States of Matter Achievement Test (SMAT) and the Chemistry Attitude Scale (CAS).

The States of Matter Achievement Test (SMAT) was composed of twenty multiple-choice (items 1-20) and five open-ended (items 21-25) items. Two steps were followed to develop the SMAT. First, the content boundaries were defined and instructional objectives were determined from the general chemistry curriculum. Second, a review of the literature related to students’ was completed. Tests items were constructed by considering the objectives of the curriculum and alternative conceptions identified from the literature. Four of the multiple-choice questions in the test were adapted from the literature (Osborne & Freyberg, 1985), the others were prepared by the researchers. Examples of the items in the SMAT are given in Appendix A (translated from Turkish to English). Table 1 includes the content areas of all the test items.

Table 1. Content areas of the items in the test

Item number		Content area
1, 2, 7, 16, 18, 24		States of matter (solid, liquid, gas)
3, 4, 22		Evaporation
5, 6, 13, 8, 21		Condensation
9, 10, 11, 12, 14, 23		Particulate nature of matter
15, 19, 25		Boiling
17, 20		Temperature curves

The content of the test was validated by three experts in chemistry education. In addition, the test was piloted with fifty pre-service second grade teachers. After the pilot study, the reliability and validity studies of each section of the test were performed in different ways.

For the multiple-choice section (items 1-20), item analysis was conducted and then discrimination index of each question was computed. The discrimination indexes of the questions ranged from 0.76 to 0.35. The reliability coefficient for this section of the test (first 20 items) was found to be 0.83 by using KR-20 formula. In this section, each true response was marked with 3 points, and each wrong one was scored with a zero. Thus, for this section of the SMAT the maximum score was 60 points.

For the open-ended section (items 21-25), the questions were categorized and scored as given in the Table 2. Similar categories have been used in the literature (Abraham et al., 1994). The maximum score for this section was 15 points. To provide inter-rater reliability, the authors independently classified student responses to open-ended items the categories listed in Table 2. The kappa statistic (Fleiss, 1981) was used to assess the extent of agreement among the three raters. The authors coincided around 90% or more of the classifications. Finally, all differences or disagreements were resolved by discussion.

Table 2. Classification of students’ responses to open-ended questions

Categories	Criteria for the classification of student responses	Score
Sound understanding (SU)	Responses that included all components of the validated response.	3 points
Partial understanding (PU)	Responses that included at least one of the components of validated response, but not all the components	2 points
Alternative conception (AC)	Responses that included an alternative conception	1 points
No Answer (NA)	Repeated the question; contained irrelevant information or an unclear response; left the response blank	0 points

To examine the effect of different question types on students’ understanding and whether students’ achievement differed in subtopics, we computed and compared the means and standard deviations of students’ scores for each section of SMAT and for each subtopic in Table 1.

To identify students’ conceptual understanding on the states of matter, the SMAT was given as a pre-test to the sample population six weeks before and two weeks after the treatment (post-test). Then, the SMAT was administered four months after the treatment (delayed-test) to the sample. Enough time elapsed among the three administrations in order to allow the students to forget the test items and to help avoid bias; the post-test was administered 14 weeks after pre-test, and the delayed test was administered 16 weeks after post-test.

The second instrument used in the study was the Chemistry Attitude Scale (CAS), a 15-item chemistry attitude scale (with 10 positive and 5 negative statements) was adapted from Geban et al. (1994). The main adaptation was to use the word “chemistry” instead of “science.” Positive statements in the attitude scale, a five point Likert-type, were marked from Strongly Agree (5 points) to Strongly Disagree (1 point). In contrast, negative statements were scored from Strongly Agree (1 point) to Strongly Disagree (5 points). The total score of the CAS ranged from minimum of 15 points to maximum of 75 points. Its alpha reliability coefficient was found by Geban et al. (1994) was 0.83. For the analysis of the CAS, firstly the total score of each student was computed and then mean score of the entire group was calculated.

The last data collection instrument was interviews with students. Semi-structured interviews were conducted after the intervention with three female and three male pre-service teachers. The students were randomly selected from a pool of male and female participants. The interviews aimed to determine pre-service teachers’ views on teaching based on context-based approach. Each interview lasted 10-15 minutes. All the interviews were audiotaped and transcribed by the researchers. The results of interviews were presented by using direct quotations from the interviews.

Development of Context-Based Teaching Material (CBTM)

The Context-Based Teaching Material (CBTM) was prepared by the researchers and used to teach the states of matter. To develop the CBTM and to determine the coverage and time-schedule, we examined a number of relevant resources, such as the Turkish chemistry textbooks, publications of Salters’ Advanced Chemistry Course, and the general chemistry curriculum of Higher Education Council (YOK). In addition, all the authors are chemists and they have been teaching chemistry at the university level for many years; consequently, this study benefitted from their experiences and teaching materials that they have been using to teach the concepts. The CBTM included the same number of lessons about the nature of matter and its changes that are in the national curriculum, but with a different teaching approach. A group of experts, three experienced chemistry teachers and three chemistry educators, reviewed and validated the content of the CBTM.

The material for eight teaching sessions (8x50 minutes) was designed to engage the students actively in context-based learning. All the eight lesson plans consisted of a storyline and related activities (e.g., pictures, images taken from real events, laboratory work, worksheets, animations, power-point presentations, group and class discussion). Each of the plans started with a storyline and followed with related activities based on the storyline. The lesson plans for the first two lessons are given in Table 3 as an example.

The other plans were prepared in a similar way. The storylines Matter from Empedocles to Dalton (Story 1) were retrieved from Carpi (2005), State of Life and A Difficult Decision (Story 2) was written by the first author, The Ozone Hole (Story 3), Science or Magic? (Story 4) and Flood in the Library (Story 5) were retrieved from LeMay et. al. (1996), Falling and Flashing Plasma (Story 6) was retrieved from URL-1 (2005), and Control of Nature: Cooling the Lava (Story 7) was retrieved from URL-2 (2005). All of the other activities such as worksheets, power-point presentations, were prepared by the researchers. One of the activities (Activity 1) is given in Appendix B.

Table 3. An outline of the teaching design

Lesson Plan	Teacher’s role	Students’ role
First class time	The teacher initially told the story (Matter from Empedocles to Dalton) to enable students to capture the related concepts they needed to make sense.	(S)he tried to capture and find out the related key concepts they needed to make sense by listening the story carefully.
	After completing the storyline, (s)he asked the students to find the key concepts.	(S)he explained the concepts (S)he found.
	The teacher asked the questions “what is the solid, liquid and gas states of matter made of?” and “why can we not see the particles forming matter?” to the students and started a class discussion. Then she asked the students to perform the Activity 1.	They performed the Activity 1 (see Appendix B), entitled “What are all substances mainly made of?”. For this, they work in the groups of 5 to 6 members. They discussed the questions at the end of the worksheet within their group. Then each group presented its results to entire class.
	PowerPoint presentation, entitled “What are substances made of?, was explained by the teacher to the students. The teacher asked a few questions to students about the presentation. She tried to relate the presentation with the results of Activity 1.	(S)he try to answer the teacher’ questions.
Second class time	The teacher initially told the story 2 (State of Life and A Difficult Decision) to enable students to capture the related concepts needed.	(S)he tried to capture and find out the related key concepts they needed to make sense by listening to the storylines carefully.
	After completing the storyline, (s)he asked the students to find the key concepts.	They discussed the concepts covered in the story.
	The teacher encouraged the students to participate the discussion. Then, she asked them to perform Activity 2 (Is each substance compressible?)	They performed Activity 2 in groups and discussed their results.
	The data in the tables created by the groups was combined in one table by the teacher. Then the table was presented with a projection to the entire class. She asked the students to perform Activity 3.	Each group created a table of physical properties (solid, liquid and gas) of substances and presented its table to entire class.
	After Activity 3, she asked the question “what are the differences between solid and liquid and gas particles?”	They carried out Activity 3 in the groups (Do molecules move?)
	She summarized the properties of solids and liquids by using power-point presentation involving daily-life pictures and molecular-level images.	They participated in all class-discussion.

Experimentation Process

The SMAT and the CAS were administered to the sample as a pre-test six weeks before the intervention. Pre-service primary school teachers spent eight teaching periods (8 x 50 minutes) to learn about matter and its changes. The first author of this study taught the students. In the implementation process, the teacher initially told the story to enable students to convey the related concepts needed to understand the topic. After completing the storylines, she asked the pre-service teachers to find the key concepts and to perform different activities such as completing worksheets and creating diagrams. At the end of the lessons, she summarized the concepts through power-point presentations.

The experimentation process was completed in eight hours that lasted for four weeks. Two weeks after the intervention, the SMAT and the CAS were re-administered to the study population as a post-test. To measure how the intervention affected primary pre-service teachers’ long-term memory (retention), the same instruments were given as a delayed-test four months after the intervention.