Effects of storylines embedded within the context-based approach on pre-service primary school teachers’ conceptions of matter and its states

The first research question for this study was to determine whether the storylines embedded within a context-based approach caused a statistically significant improvement in the levels of pre-service teachers’ understanding of matter and its states, or not. After the sections (multiple-choice and open-ended) of the SMAT were examined in detail, mean scores and standard deviations for each of the test applications (pre-, post-, and delayed-test) were computed. The results are presented in Table 4.

Table 4. Mean scores and standard deviations of pre-, post- and delayed-tests

Tests	N	Multiple-Choice Section		Open-Ended Section
Tests	N	Mean	SD	Mean	SD
Pre-test	35	28.89	7.17	8.2	2.89
Post-test	35	46.03	8.39	12.17	2.06
Delayed test	35	45.38	8.58	12.75	1.94

The students gained 17.14 (22.85 out of 100) points on average in the multiple-choice section and 3.97 (24.46 out of 100) points on average in the open-ended section from pre-test to post-test (Table 4). These differences are quite high when compared with the results of other experimental studies in the science education literature (e.g., Demircioğlu et al., 2005; Özmen et al., 2009). After examining mean changes from post-test to delayed test, a decrease of 0.65 points was found in the multiple-choice section, as there was an increase of 0.58 points for the open-ended section. To statistically test the retention and knowledge development of matter and its change, we conducted a one-way ANOVA for repeated measures analysis and LSD test for post hoc comparisons on the scores of pre-, post-, and delayed-tests. The results are summarized in Table 5.

Table 5. The results of one-way ANOVA for repeated measures on the scores of pre-, post-and delayed-test

Sections of SMAT	Source	Sum of square	df	Mean square	F	p	Difference(LSD test)
Multiple-Choice Section	Intervention	6531.43	2	3265.21	239.67	.000	2-1; 3-1*
Multiple-Choice Section	Error	926.57	68	13.63
Open-Ended Section	Intervention	428.59	2	214.29	74.32	.002	2-1; 3-1*
Open-Ended Section	Error	196.07	68	2.88

As seen from Table 5, the effect of the intervention on pre-service teachers’ achievement is statistically significant in both the multiple choice section (F(2;68) = 239.67; p<0.05) and the open-ended section (F(2;68) = 74.32; p<0.05). The LSD post hoc test results showed that while there were statistically significant differences between pre-test and post-test and between the pre-test and delayed-test, there is no significant difference between the post-test and delayed-test in both sections of the survey (Table 5). Since the only independent variable was the intervention, it could be concluded that this difference likely resulted from the context-based learning materials.

To determine if the pre-service teachers’ achievement differed in subtopics, the results of the questions related to each content area listed in Table 1 were computed independently. Means and standard deviations identified from each set of questions are given in Table 6. The results are given out of 100.

Content areas (Item no)	Pre-test		Post-test		Delayed-test
	Mean	SD	Mean	SD	Mean	SD
	%	%	%	%	%	%
States of matter (1.2,7,16,18,24)	44.44	15.83	74.44	22.39	76.98	22.35
Evaporation (3,4,22)	55.24	30.55	86.35	18.09	86.03	18.35
Condensation (5,6,8,13,21)	67.81	26.35	81.14	21.78	80.38	23.82
Particulate nature of matter (9,10,11,12,14,23)	43.65	15.07	78.73	22.40	74.29	18.67
Boiling(15,19,25)	45.71	39.05	67.14	34.18	65.71	33.81
Temperature curves (17,20)	41.43	41.10	72.86	40.84	72.86	42.60

As seen in Table 6, pre-service teachers’ performance in each subtopic had notable increases from the pre-test to the post-test, ranging from 13.33 (condensation) to 35.08 (particulate nature of matter). The pre-test means for condensation (M=67.81) and evaporation (M=55.24) are higher than others (Table 6). It should be noted that pre-service teachers were more familiar with these two concepts (condensation and evaporation) than others. The reason for this could be that students are faced with these two concepts in their daily life more often. Mean changes of each subtopic in Table 6 were compared by using Wilcoxon signed ranks test. The results obtained from the analysis showed that the mean change of each subtopic from pre-test to post-test was statistically significant at the 0.05 level.

As seen from Table 6, while the mean for the states of matter subtopic increased 2.54 points, the others decreased or remained unchanged from post-test to delayed-test. According to Wilcoxon signed ranks test, the mean change from the post-test (74.44%) to delayed-test (76.98%) for states of matter subtopic was found to be significant at the 0.05 level.

These are similar findings reported in the literature that point to positive effects of context-based approach on student achievement in chemistry (Barker & Millar, 1999; 2000; Belt et al., 2005; Demircioğlu et al., 2009). Teaching materials used in these studies were more useful than the traditional approaches in teaching complex chemistry concepts.

The intention of the second research question was to determine how the teaching activities based on the context-based approach influenced pre-service teachers’ retention and long-term memory of new conceptions about matter. In other studies investigating the retention of knowledge, a decrease from post-test to delayed-test is generally observed. However, this study identified a 0.58 point increase in the open ended section (Table 4) and a 2.54 point increase in the states of matter subtopic (Table 6). This finding is promising for science education; it shows that the intervention caused a retentive change in pre-service teachers’ knowledge structures (Banister & Ryan, 2001; Barker & Millar, 2000) and, more importantly, it reveals that some students continued to configure the concepts in their mind even after the treatment (Çalık, 2006). These results contradict with the literature where after an intervention minor decreases over time are often reported in student achievement (e.g., Coştu, 2006; Çalık et al., 2010).

These results are indicators for the effectiveness of such an approach on pre-service teachers’ understanding and their retention of knowledge. The reason for this effectiveness is probably due to the storyline-based discussions, the detailed explanations found in materials, and the relation of daily-life to the materials. These results concur with other studies showing positive effects of both storylines and context-based approach on students’ understanding of chemistry concepts (Barker & Millar, 1999; 2000; Belt et al., 2005; Demircioğlu et al., 2009).

The third research question for the study was to determine whether the storylines embedded within a context-based approach are effective in overcoming pre-service teachers’ alternative conceptions of matter and change, or not. The alternative conceptions concerning the matter and its states were determined by the pre-service teachers’ responses to questions in the SMAT. To focus on the major misconceptions held by students, only alternative conceptions held by 17% or more of the students (i.e., at least 6 students) in the pre-tests were analyzed. To consider each and every alternative conception would not show useful statistical evidence and would distract from the notable findings. The percentages of these alternative conceptions determined in the pre-, post-, and delayed-test are given in the Table 7.

Table 7. Pre-service teachers’ alternative conceptions determined in the pre-test and post-test

Alternative conceptions	Pre-test	Post-test	Delayed-test
Alternative conceptions	%	%	%
Particle size would increase as it changed from liquid state to gas state.	44	5.7	5.7
Solid state of a substance is heavier than its liquid or gas state	48.6	17.1	11.4
Gas is the densest state of substance	22.9	5.7	0
Evaporation does not occur at all temperatures	40	14.3	8.6
The cold transforms the oxygen and hydrogen found in the air into the water	17.1	8.6	14.3
Gases are not fluid	25.7	14.3	5.7
There is water or air between two water molecules	54.3	25.7	31.4
Solid particles begin to vibrate in the melting point	22.9	0	14.3
The boiling point of a substance never changes	31.4	14.3	5.7
The boiling point of liquids decrease at high pressure	25.7	8.6	0
Gases do not have mass	17.1	0	0
Particles of a substance are solid in solid phase, liquid in liquid phase, and gas in gas phase	31.4	5.7	5.7
The cause of the smoke rising from an ice cube taken out of the freezer is evaporation	20	8.6	5.7

The results from the pre-test showed that pre-service teachers held a large number of alternative conceptions before receiving formal instruction on matter and change. After the intervention, they showed progress in eliminating their alternative conceptions and corrected two alternative conceptions completely (Table 7). In addition, the sample completely corrected one more alternative conception in the delayed-test. These results likely reflect the effectiveness of the materials. The ratios of alternative conceptions from pre-test to post-test and delayed test show that storylines embedded within a context-based approach contributed to an acquisition of the scientific conceptions and elimination of alternative conception.

From implementation of the SMAT as a pre-test, thirteen alternative conceptions were identified. While the percentages of students’ alternative conceptions in pre-test ranged from 17.1% to 54.3%, these ratios in the post-test ranged from 0% to 25.7%. And also, these ratios in the delayed test ranged from 0% to 31.4% (Table 6). These results show that the teaching activities based on the context-based approach helped pre-service teachers to overcome their alternative conceptions.

The most common alternative conception encountered in this study is “there is water or air between two water molecules” (see Table 6). Similar alternative conceptions have been reported in the literature. For example, Lee et al. (1993), Novick and Nussbaum (1981), and Osborne and Freyberg (1985) studied with students of different ages (12, 13, 14 ages) and found the same alternative conception. To address this misconception, the CBTM developed for this study included is each substance compressible? (storyline 2), along with activities, discussions, and a power point presentation. While the teacher was telling storyline 2, she showed the presentation consisting of movement and arrangement of solid, liquid and gas particles and discussed differences as the occasion arose. As could be seen in Table 7, such applications were effective in overcoming the alternative conception to a certain extent (54.3% in pre-test, 25.7% in post-test). Although storyline 2 and activity 2 aimed to overcome the pre-service teachers’ alternative conceptions, 25.7% of the pre-service teachers continued to retain the same alternative conceptions. The microscopic nature of the content might have made it difficult for pre-service teachers’ to understand the concept; some might have failed to transfer their macroscopic-level knowledge learned from the materials to the microscopic-level.

Another common alternative conception encountered in the study is that the “solid state of a substance is heavier than its liquid or gas state” (see Table 7). For this belief, pre-service teachers provided different explanations such as “…gap between molecules in gas is more.” “…molecules in the solid are more tightly packed,” and “…number of the particles in gas is lower.” None of the reasons is scientifically acceptable and similar alternative conceptions are also reported in the literature (Coştu, 2006; Demircioğlu, 2008; Lee et al., 1993; Osborne & Cosgrove, 1983; Stavy, 1990). For example, Stavy (1990) reported that students (age 9-15) believed that gas is lighter than the same material in its liquid or solid state or that gas has no weight. Durmuş and Bayraktar (2010) reported the latter alternative conception, as well. The teachers in Turkish schools generally use the following pictures (Figure 2) to visualize the particles in their students’ minds and some textbooks contain the same pictures. Such images may cause students to think that solid phase is heavier.

To overcome this alternative conception, Storyline 1 and Activity 1 were used. When the teacher told Storyline 1, she asked students to find and discuss key concepts in it. She used questions such as “what does matter mean for you?” and “what is matter made of?” Then, pre-service teachers performed Activity 1 (see Appendix B). In this activity, they had the chance to observe that the weight of a piece of ice did not change by its phase. Whole-class discussion followed the activity and Figure 3 was shown to the pre-service teachers to summarize the concept.

As can be seen in Table 7, such applications became effective in overcoming the alternative conception to a certain extent (48.6% in pre-test, 17.1% in post-test). However, this ratio decreased to 11.4% in delayed test showing that some of the sample population reverted to their earlier conception.

When table 7 is examined, it is seen that ratios of the alternative conceptions decreases from pre-test to post-test and delayed test. In some cases, however, there is an increase from post-test to delayed test. Because we used a new teaching approach in the experimental group based on storylines and activities, we can easily say that this approach is effective in addressing understandings and alternative conceptions. This conclusion is supported by a comparison of the pre-test and post-test results.


Solid particles	Liquid particles	Gas particles

Given that student alternative conceptions are specially addressed during instruction, the decreasing the ratios from the pre-test to post-test are not surprising. And we cannot ignore that the post-test shows that a high ratio of students still have some alternative conceptions. For instance, the ratios of some alternative conceptions, “the cold transforms the oxygen and hydrogen found in the air into the water,” “there is water or air between two water molecules,” and “solid particles begin to vibrate in the melting point,” increased from the post-test to delayed test.

One reason for this can be that some pre-service teachers who were taught through traditional teaching methods were unwilling to participate in different teaching methods presented in the current study. Therefore, such students might not attend to the applications sufficiently. Those learning habits might have caused them to retain their alternative conception in post-test or to revert back to them before the delayed test.

The likely reason for this was that some students moved back to previous ideas. It has been argued that students’ original believes are probably changing in the long run, even though they may give correct answers to the test (Rutherford & Ahlgren, 1991).These results show us the well-known reality that alternative conceptions are pervasive, stable, and resistant to change and often held even after the completion of years of formal science instruction (Guzzetti, 2000).

Despite evidence that supports strong commitment to alternative conceptions, our study revealed a change that gives hope to the ability promoting conceptual change. We found it remarkable that the ratios of some alternative conceptions decreased from the post-test to delayed test. These results reveal that the new teaching approach and materials may help students to continue constructing knowledge even after formal instruction. Furthermore, the decreasing ratios from the post-test to delayed test may indicate that the construction of knowledge takes time, continues beyond the time of instruction, and becomes more meaningful for students as time passes. In another words, consolidation of some learning continues over many months.

Another factor under investigation in the present study was pre-service teachers’ attitudes towards chemistry. The statistical significance of the difference between pre-and post-CAS means was determined by using paired-samples t test. The results and descriptive statistics of analysis are given in Table 8. As can be seen from Table 8, mean and standard deviation of attitude scores were 51.09 ± 9.53 before the intervention as they were 57.94 ± 8.74 after the intervention. The pre-service teachers’ mean scores showed an increase of almost 7 points and the effect of the intervention on attitude scores is statistically significant (t(34)=6,93; p<0,05).

Table 8. Summary of the paired-samples t test on scores of pre- and post-CAS

	N	Mean	SD	Mean Difference	df	t	p
Pre-CAS	35	51.09	9.53	6.85	34	6.93	.000
Post-CAS	35	57.94	8.74

The results showed that the context-based approach had a strong effect on attitudes towards chemistry; the pre-service teachers were more motivated to participate in chemistry classes. Context-based courses inform students why they need to know scientific ideas; therefore, this result is expected. The results are consistent with related literature that indicates context-based approaches produced significantly better understanding of science ideas and more positive attitudes (Bennett, 2005; Tsai, 2000; Winther & Volk, 1994; Yager & Weld, 1999). Also, Bennett et al. (2003) reported that context-based approaches motivated students to learn science lessons and stimulated their interest in science. In addition, we observed that pre-service teachers were very willing to discover key concepts in the storylines and to complete the worksheets.

The pre-service teachers participating in the interview said that they have never encountered chemistry teaching based on storylines; therefore, they found this application remarkable and different. Also, the pre-service teachers found concepts presented in a story format more interesting than those presented in expository text. Graesser et al. (1994) also reported that teachers found stories easier to comprehend and remember than other forms of text such as expository text. The following excerpts support this claim:

-“…teaching style of a teacher is very important to love and be interested in lesson. …I learned the states of matter concepts a few years ago, but now I do not remember. …I will remember the concepts you taught with the context-based approach even after 4-5 years (a female pre-service teacher).

-“I think that stories increase the retention. When the concepts are taught though stories, I find myself in the story and would like to participate in class. I do not forget the concepts. ...the story and its heroes are easier to remember than concepts or formula. A person do not easily forgets it, but s(he) forgets a formula or a concept a week after an exam” (a male pre-service teacher).

-“…I loved the stories…”, I wish other chemistry topics also were taught by using this method…” (a female pre-service teacher).

It is understood from interviews that the courses based on context-based approach are more enjoyable and understandable than previous chemistry courses and the stories used in the study make a significant contribution to this process. The following excerpts support this assertion:

-“I start to love chemistry. With this course, everything starts to become more understandable. In the previous way, everything was more abstract and more difficult to understand. In this way, I learned more… (a male pre-service teacher).

-…Stories made concepts more understandable…”(a female pre-service teacher).

-…”In this way, I learned better by seeing and doing experiments…”(a female pre-service teacher).

The twenty-week time period between the post-test and the delayed-test is considered to be adequate to test the retention of knowledge. Despite the elapsed time from the post-test to the delayed-test, most teacher candidates did not forget the concepts they learned during the intervention and did not revert to earlier alternative conceptions. This showed that the storylines, activities, class and group discussions, power-point presentations, and the pictures and animations used in the study were effective for the retention of pre-service teachers’ understanding and for addressing their alternative conceptions.

The final research question for the study was to determine whether correlations between the pre-service teachers’ attitudes towards chemistry and their achievement on the matter and its states were statistically significant. Correlations between scores of the pre- and post-CAS and scores of each section of the SMAT pre- and post-tests are given in Table 9. As seen in Table 9, there were significant correlations levels between pre- and post-test scores for the following:

· multiple choice section of SMAT and pre- and post-test scores of CAS (at 0.01 level);

· pre-test scores of the open-ended section of the SMAT and post-CAS scores (at 0.01 level);

· post-test scores of the open-ended section of SMAT and post-CAS scores (at 0.05 level).

However, there was no significant correlation between pre- and post-test scores of open-ended section of SMAT and pre-CAS scores. From this, it can be said that there was a meaningful correlation between students’ attitudes towards the concepts under investigation and their achievement.

Table 9. Correlations between scores of pre- and post-CAS and scores of each section of pre- and post-SMAT

	Multiple Choice Section of SMAT		Open-Ended Section of SMAT
	Pre	Post	Pre	Post
Pre-CAS	0.46**	0.52**	0.24	0.12
Post-CAS	0.53**	0.66**	0,55**	0,34*

It is already known that as attitude scores increase, learner achievement increases and that there is a significant correlation between attitude and achievement (Coleman, 2009). In the present study, the context-based materials contributed to a significant improvement on pre-service teachers’ attitudes towards chemistry and a meaningful correlation was found between attitude and achievement.