A study on evaluation of the biology projects submitted to the TUBITAK secondary education research projects contest from the Bursa Region

Document analysis, which is a data collection method for qualitative research, was used for evaluating the biology projects joining the “Research Project Competition for Secondary Education Students” held by TUBITAK-BIDEB (The Science Fellowships and Grant Programmes) based on the criteria of determination of problem, originality and creativity, scientific method, consistency and contribution, usefulness, implementability, literature review, results. Document analysis is a research method used for making valid and reliable inferences out of texts (Krippendorff, 2004). It involves the analysis of written materials that contain information about the phenomenon or phenomena that is/are subject to research.

The Sampling procedure

The projects joining the national “Research Project Competition for Secondary Education Students” regularly held by TUBITAK-BIDEB in Turkey every year are evaluated based on project reports by biology juries in 12 different regional scientific boards cross Turkey. The projects which are found suitable for exhibition through such preliminary evaluation are invited to exhibitions held in regional scientific boards. The students preparing those projects invited to exhibitions are interviewed by juries during exhibitions. At the end of regional exhibitions, regional finalists are determined. The finalists of 12 regions are invited to Turkey Final Competition where the projects are re-evaluated. In this way, the best projects in Turkey are selected by discipline.

In the present study, research universe consists of biology projects joining the above-mentioned competition, and research sample consists of 107 biology projects that applied to the competition in Bursa Region, which is one of the 12 regions, between 2009 and 2012 with an application form and a project report. The biology projects were divided into sub-groups based on subject areas through analysis of the keywords used in project reports (Figure 1).

Research tool

Data sources of the present study are (1) the application forms filled in by project owners during application to the competition and (2) the project reports. The application forms were used for determining the provinces from which most projects were submitted by year and the distribution of such years by year and school type.

The project reports were evaluated through content analysis based on the predetermined criteria (determination of problem, originality and creativity, scientific method, consistency and contribution, usefulness, implementability, literature review, results). Each evaluation criterion was taken as a category, and whether or not a document included the relevant category was investigated via Secondary Education Project Evaluation Chart. The evaluation instrument developed by Güngör et al., (2013) was used in evaluation. This chart consists of 8 main items and 23 sub-items. 8 main items are as follows: The Determination of Problem (DP), Originality and Creativity (OC), Scientific Method (SM), Consistency and Contribution (CC), Usefulness (U), Implementability (I), Literature Review (LR), and Result (R). The items in the chart were answered with the following responses: “Yes” (2), “Partly” (1), “No” (0). The expert opinions and recommendations were taken concerning the Secondary Education Project Evaluation Chart in order to ensure scope validity of the scale.

The projects were evaluated on the chart in accordance with predetermined criteria by two different experts. The qualitative data thus obtained were quantified via content analysis. A normality test was performed on the data obtained from the expert evaluators. The results can be found in Table 1. The SPSS 18.00 was used in the normality analysis of the data obtained after evaluating 107 biology projects, as well as in the compatibility tests between the evaluators. Moreover, the f and % values were calculated for the qualitative data.

Kolmogorov-Smirnov test demonstrated that data did not display a normal distribution (Özdamar, 2011). Thus, Kendall’s tau-c coefficient was used for interpreting the research data.

Table 1. The analysis of normality of the data obtained by expert evaluators

Criteria	N	Mean	SD	Kolmogorov-Smirnov Z	p
a)The determination of problem	214	2.68	0.126	0.150	0.00*
b) Originality and creativity	214	0.75	0.068	0.292	0.00*
c) Scientific method	214	5.47	0.271	0.216	0.00*
d)Consistency and contribution	214	2.20	0.116	0.172	0.00*
e) Usefulness	214	1.64	0.118	0.207	0.00*
f) Implementability	214	0.65	0.042	0.311	0.00*
g) Literature review	214	2.15	0.124	0.162	0.00*
h) Result	214	2.21	0.091	0.221	0.00*

*p<0.05

According to Table 1, the data obtained from the expert evaluators do not fit normal distribution. For continuous data which do not display a normal distribution, the Kendall's tau c cofactor was used for the correlation between the expert evaluators. The consistency values between the expert evaluators were calculated with the Kendall tau c cofactor, based on the total scores from every section. The meanings of Kendall’s tau-c coefficients are as follows:

>0.50 : High-level correlation,

0.36-0.49 : Significant correlation,

0.20-0.5 : Intermediate level correlation,

0.10-0.19 : Low-level correlation,

< 0.10 : No correlation.

There is a high correspondence between the scores of expert evaluators pertaining to the criteria of The Determination of Problem (τ_c= 0.769), Originality and Creativity (τ_c= 0.666), Scientific Method (τ_c= 0.825), Consistency and Contribution (τ_c= 0.799), Usefulness (τ_c=0.693), Implementability (τ_c= 0.510), Literature Review(τ_c= 0.759) and Result (τ_c= 0.898).