What teachers assess and its consequences

This study involved a small-scale survey that employed both qualitative and quantitative approaches.

1. Data collection instruments

Data for the research were collected through semi-structured interviews and document (i.e., examination papers) analysis.

1.1. Interviews

To answer the research question and to make the analysis more manageable, a semi-structured interview was used for the study (Robson, 1993).

Twenty-four interviews with teachers were conducted, and all of the interviews were tape-recorded. The interview schedule was mainly guided by the research question. The interviews included two sections. The first section of each interview included questions about the teacher's background information and also assessed the teacher's broad perceptions. This section sought to obtain biographical information, each teacher's perception of their role as a teacher, and a list of professional development activities. The information from this section of the interview was intended to provide background information to generally explain teachers’ reported actions and perceptions of what they assessed.

The second section of the interview included questions related to the kind of learning and skills that teachers assessed. Previous studies have stressed the importance of assessment in targeting higher-order thinking skills (Brown et al., 1997). This is also an official requirement of assessment policy in Turkey. Thus, in this section of the interview, teachers were asked about the tasks that they assigned, the kind of questions they asked in the classroom, and the questions that they asked in their examinations. The goal of this section of the interview was to further to elicit their criteria for the judgment of learning and understanding.

1.2. Documents

All of the teachers interviewed provided examination papers that they had graded. Specifically, six teachers provided two copies of two examinations they had conducted at different times. Thirty examination papers were collected, and there were a total of 370 questions in these papers (230 short answer questions and 140 multiple-choice test questions).

2. Participants

Twenty-four biology teachers from 16 state secondary schools in Ankara, Turkey, participated in the study.

The schools were selected from the Ministry of Education’s directory of state schools. Schools were selected from three districts that have different socio-economic levels. Sixteen schools in total were randomly selected from the directory, including five schools from District 1, six schools from District 2 (the largest district of the three), and five schools from District 3. The schools sampled were all state high schools.

Each school had two to eight biology teachers, for a total of 62 biology teachers across the studied schools. Twenty-four of the teachers volunteered to be interviewed. Each teacher was assured of confidentiality.

The table below summarizes the biographical information of the teachers that participated in the study.

Table I: Teacher characteristics

		*Interview sample*
		Frequency (n=24)	Percentage (%)
*Gender*	Male	11	46
*Gender*	Female	13	54
*Educational qualification*	B.Ed.	21	88
	Master's Degree	3	12
	Ph.D.	0	0
*Years of teaching experience*	6-10	9	37.5
	11-19	9	37.5
	20-25	6	25

The sample included 11 male teachers and 13 female teachers. The teaching experience of the teachers varied from 6 to 25 years. Thus, the majority of the teachers were experienced teachers. Three teachers held a Master's Degree in biology.

3. Data analyses

The interview questions were analyzed using Bloom’s taxonomy of knowledge levels. The author and another expert in this field first analyzed each question separately. Next, the results were compared, and when there was a discrepancy between the results, the item was discussed until an agreement was reached.

The interview data were analyzed following step-by-step guidelines advocated by Krueger and Casey (2000), Miles and Huberman (1994) and Bogdan and Biklen (1992).

After completing the data collection, the raw data were transformed into a readable form for analysis. Tape-recorded interviews were transcribed verbatim as soon as possible and then were entered into a word processing document.

The transcripts were read carefully several times to identify emerging themes, topics or concepts that had the potential to answer the study’s questions (i.e., Bogdan and Biklen, 1992; Miles and Huberman, 1994). These themes, topics, and concepts (hereafter referred to as "codes") were written in the right margin of the paper. This search was continued until no new codes emerged. The two researchers first conducted this analysis separately. Next, the results were compared and discussed. Consensus was reached on all items.

The codes were then grouped together to form broader categories with strong commonalities (i.e. Miles and Huberman, 1994). Two main categories of "cognitive" and "non-cognitive" criteria emerged from the qualitative data analysis. There were also sub-categories. For example, the category of non-cognitive criteria contained codes that categorized students as "respectful," "completes homework," "obeys rules," "polite," and so on.

Throughout this study, each teacher was identified as teacher (T), followed by a number to protect the teacher’s identity.