Asia-Pacific Forum on Science Learning and Teaching, Volume 2, Issue 2, Article 2 (Dec., 2001)
Russell TYTLER
Describing and supporting effective science teaching and learning in Australian schools - validation issues
Previous Contents Next
Validation of the SiS Components

There are a number of ways in which the project has gone about validating the SiS Components and in this paper I will describe these different forms of validation and how they relate to the different purposes of the components. Some of these forms of validation involve the Component Mapping process, for which a separate validation issue exists concerning whether it adequately measures science classroom practice. These different validation issues are discussed as responses to a series of questions. The first four questions relate to the extent to which the components comprise a defensible description of effective teaching and learning in science. The fifth question relates to whether they fulfil their purpose as vehicle for supporting improvement in science teaching and learning.

  1. Are the SiS Components plausible as a description of effective science teaching and learning?

    This is an issue of 'face validity', and the evidence is necessarily anecdotal. The SiS Components have achieved wide acceptance within schools, and have been acclaimed by SiS consultants working with them. Some of the university-based consultants have used the Components in their teacher education programs. Some schools are using the Components in other learning areas. One of the dangers in describing effective teaching and learning is that the descriptions can be so general as to be unhelpful. We have continued to develop explicit interpretive descriptions for each of the components, to clarify and sharpen their meaning.

  2. Do the Components align with the literature on teaching and learning in science?

    A comprehensive search of the literature dealing with effective science teaching, student learning, major curriculum reform projects, and teaching standards, was undertaken (Tytler & Waldrip, 2001). The Components were found to align strongly with the literature. Where a component was not well represented in the mainstream science education literature, its inclusion could be justified by other research literatures, or by contemporary concerns such as those associated with the Middle Years of schooling.

  3. Do the SiS Components adequately capture the practice of the teachers from whom they were developed?

    In a 'reverse validation' process, nine of the primary teachers who had been originally interviewed as effective practitioners agreed to undertake the Component Mapping process and also to comment on how well it, and the case descriptions generated from the original interviews, matched their practice. The result vindicated our analysis, with the mean score on each component exceeding 3.2 out of a possible 4 (and exceeding 3.5 on 'meaningful understandings'). These teachers all claimed to be comfortable with the Component Map as allowing them to represent the core of their practice.

  4. Do the components support the generation of a reliable profile of the practice of individual teachers?

    Examination of patterns in teacher highlighting of phrases within the component map, and discussions with SiS Coordinators involved in the interview process, exposed some ambiguities in descriptions, and the existence of separate dimensions in some components. This led first to the separation of the original Component 1 describing the classroom learning environment, into the first two components listed above. These have recently been further separated into three sub-components each. The language has also been tightened. A selected sample of Coordinators, following trials of the modified map, provided confirmation that this clarified the mapping process and led to more reliable descriptions of teacher practice.

  5. Are the Components effective in supporting the change process?

    Part of the evidence for effectiveness is anecdotal, based on numerous presentations by SiS Coordinators in workshops in which they represent significant initiatives their schools have undertaken in terms of the SiS Components. An examination of responses from a review questionnaire showing the pattern of initiatives undertaken by schools confirmed that each of the Components is well represented. Component 1 (engagement) stood out as a core focus overall. The patterns of use were different for primary compared to secondary schools.

Other evidence of the effectiveness of the Components and Component mapping came from the same questionnaire sent to all schools following the development of draft Action Plans. Table 1 shows mean scores for the Component Mapping aspect of the auditing process, and for the SiS Components. For each process coordinators were asked to "rate each of the following aspects of the strategy in terms of its usefulness in framing the direction of the innovation in the school". The results show a strong acceptance of the Components and the Component mapping.

The scoring system was:
Score 4: Of critical importance; 3: Very useful; 2: Somewhat useful; 1: Not very useful

Table 1: Perceived importance of aspects of the SiS Strategy
Aspect of strategy Mean (Primary Schools) Mean (Secondary Schools)
SiS components as part of the support structures 3.3 3.3
Component mapping as part of the auditing process 3.4 3.1

Comments on the mapping process in formal and informal reports from schools, and field notes taken at workshops, support the finding that this is an extremely valuable part of the auditing process, with the following effects:

Coordinators used the mapping process to gain insight into teacher practice which helped them extract common concerns and issues that could feed into the school action plan, but also to shape their response to individual teachers as the project developed. There were a number of stories of the development of new classroom strategies immediately following the mapping process, as recognition by teachers of gaps in practice acted as a catalyst for change.

 

Copyright (C) 2001 HKIEd APFSLT. Volume 2, Issue 2, Article 2 (Dec., 2001)