A study on the evaluation of science projects of primary school students based on scientific criteria

13922 online project applications were made to the project competition in 2007 18313 were made in 2008, 31866 were made in 2009, 33264 were made in 2010, and 63247 were made in 2011. Of these projects, 959 were considered worthy of being exhibited at regional level in 2007, 902 were in 2008, 1045 were in 2009, 1004 were in 2010, and 1048 were in 2011. 62 science projects ranked among the top 100 in 2007, 62 in 2008, 66 in 2009, 68 in 2010, and 65 in 2011 (MNE, 2007a; MNE, 2008; MNE, 2009; MNE, 2010; MNE, 2011). A total of 454 project applications were made in Bursa region in the project competition titled “This Is My Work”, jointly conducted by MNE and TUBITAK Department of Supporting Scientists in the field of science between 2007 and 2011. At the first stage, 181 of these projects were considered by the science boards composed of faculty members in Bursa region worthy of being exhibited. 20 of the exhibited projects were deemed suitable for being sent to Ankara to be exhibited for country-wide eliminations. 9 of these projects received awards by ranking among the top 50 in Turkey at the exhibition held in Ankara. The projects within the scope of “This Is My Work: Mathematics and Science Project Competition” underwent quite detailed long evaluation processes at different levels in schools, districts, provinces, and regions until this final stage. Certain scientific principles and criteria were not sufficiently and effectively used by relevant people during the evaluation of the projects that went through evaluation boards of different levels. These evaluations need to be revised in accordance with certain criteria. It was determined at our information meetings in Bursa, and the interviews during the exhibition that the projects sent to an upper level competition without paying attention to evaluation criteria had a negative effect on both the teachers guiding the projects and the students preparing the projects. Thus students lost self-confidence to a considerable extent or formed a defense texture based on knowledge without any scientific basis, and mostly exhibited reactive attitudes. Based on the review of the evaluation chart employed by juries composed of teachers and faculty members, it is seen that required attention was not paid to scientific criteria. Based on the re-evaluation of projects through main titles and sub-titles included in the evaluation chart prepared in accordance with scientific criteria, it is realized that there are certain aspects, which are lack of particular qualifications, and cannot be improved or changed. In this regard, this section gives a brief evaluation of the data obtained from findings in accordance with the main titles in the evaluation chart.

Based on the distribution of the project applications by school types, it is seen that Public Schools ranked first (74.9%), which were followed by Private Schools (17.4%) and Science and Arts Centers (7.7%).

Although there are 3522 public primary schools and 88 private primary schools in the Bursa region, there are just 10 Science and Arts Centers. Science and Arts Centers are the institutions affiliated to the MNE that provide special education in order to enable highly or specially talented students attending pre-school education, primary education, and secondary education institutions to be aware of their individual skills and to make best of them by improving themselves. Standing for 0.28% of the schools in our region, Science and Arts Centers have a share of 7.7% in a total of 454 science projects participating in the competition. The fact that 35 projects participated in the competition from such few number of institutions results from the fact that students with superior abilities and high capacities are dense at these institutions, the students at such institutions have adequate motivation, such institutions have enough physical equipment, and family, teacher and school support are sufficiently provided to students at these kinds of institutions.

The fact that the private schools, which have 2.43% of primary school students have a share of 17.4% in all project applications is caused to a considerable extent by that an effective teacher and family support is provided at these schools and the projects are subjected to an elimination beforehand. It is understood that a special effort is made for students to participate in project competitions, which means that students at these schools are more motivated for participating in scientific activities. This situation strengthens the impression that the same scientific support and encouragement, as well as, assistance required for project design and management are not provided at public schools. Although the ratio of public schools in the region is 97.29%, the ratio of participation in scientific research and project activities is just 74.9%. The study conducted by Argon and Yılmaz (2006) where the dimensions having an effect on the educational processes at primary schools were examined demonstrated “administrators” had the highest influence, and “students” and “school environment and parents” had the lowest influence. Non-inclusion of the environment where the school is located in educational processes through school activities causes students to be educated through a process where students are not integrated with the environment. The study conducted by Gür and Batır (2009) determined that the students receiving education at public schools and their parents did not find adequate this free service of the state, thus spent much money for private educational institutions and training centers. This situation shows that there are important problems in the inclusion of students in active educational processes at these educational institutions.

Although private primary schools make up 17.4% of all primary schools located in 12 provinces, it is clear that the students of these schools are prepared by their teachers for participating in these kinds of activities more, and their wishes for participating in scientific activities receive more support by their institutions in comparison to their peers at PSs. The effective reflection of this support in tools, materials, self-confidence, original thinking, and reporting is easily understood from the evaluated projects. In this regard, the students attending PrSs and SACs have similar features. Moreover, it is understood that the students at SACs are prone to project generation, implementation, and finalization, have good facilities, receive support from their families, and their projects are supervised and evaluated by chosen teachers, thus have better conditions in comparison to the students studying at PSs. These results are closely related to project design in the context of the use of scientific methods besides the reflection of outlook on research, motivation and socio-economic level in education (Çeken, 2011). Project design is included in the MNE Science and Technology curriculum under the titles of “project assignments” and “performance assignments” (MNE, 2005). It is seen that the evaluated 454 projects failed to satisfy the related expectations of the Council of Education and Morality, as well as, 2005 Science and Technology curriculum. The teachers taking part in provincial and regional science boards for evaluating the projects submitted to competitions need to acquire realistic and consistent evaluation criteria as well as the qualifications to implement such criteria through an effective in-service training. It is understood that the projects sent from schools were prepared without complying with particular criteria, and that the goal was to increase the number of the projects sent. Although some studies conducted abroad demonstrate that public school students are more successful than private school students (Cutts and Moseley, 2001), the situation in Turkey is contrary to this finding (Köse, 1997; Erdoğan, 2002). It is obvious that the approach adopted by private schools for the selection of students is an important factor in this sense.

There are differences in the distribution of the projects by school types and years. The biggest number of project applications was made in 2007 in which 118 projects (26% of all projects under examination) were submitted. The fewest number of applications were made in 2008 in which 50 projects were submitted. Although it is possible to say that the fact that the projects started to be evaluated via electronic media in 2008 had an effect, the programs and encouragements of the authorities on the projects were not influential over the years, which can be understood from the figures belonging to years. The biggest number of applications from PSs came in 2007 (22%), from PrSs came in 2011 (4.4%), and from SACs came in 2010 (3.5%), which shows that a specific ratio was not ensured in the participation in project competitions and the generalization effort made in this matter failed.

The ranking in project applications based on provinces is as follows: Bursa-Kütahya-Balıkesir for PS; Bursa-Bilecik-Kütahya for PrS; and Bursa-Bilecik-Afyonkarahisar for SAC. We are of the opinion that the officials serving at the Provincial Directorate of National Education at related periods have an effect on this ranking.

Although there is no significant difference between projects by gender, female students predominate (51.8%).

The distribution of the projects participating in competition by sub-fields of Science and Technology is as follows: Physics: 241 (53.8%) – Biology: 125 (27.54%) – Chemistry: 88 (19.38%). This situation indicates that subjects were mostly selected from daily life. Making life easy, providing energy saving or making use of any mechanism for other purposes outweighed in the selection of subjects. Of the projects prepared in the field of physics, 46.5% were about Technological Design, 12% were about Electricity, and 10% were about energy.

When the projects were revised in accordance with evaluation criteria, it was seen that projects were mostly below the desired level or the level expected from related age groups, and the subjects were not processed well and planning was not good in the projects in which advanced level project subjects were selected. Moreover, the fields to which some subjects were close could not be determined by guidance counselors or related jury members until the final stage. In this sense, the incompetence of teachers in the matter of project preparation should be taken into account, especially in physics-related subjects.

The subjects selected for projects should motivate students for studying. Provide them with skills to use tools and equipment, be about real life, pave the way for different studies. Give an opportunity to improve mental and physical abilities, cover desired activities, be freed from useless endeavors, should be worth of the investment made in the tools, equipment and references employed, and ended up with a proper output (Gözüm et al., 2005). The projects in the field of biology are mostly about plants and the environment. The fact that the studies on animals and certain microorganisms are considered harmful for human health and inconvenient in terms of animal rights has an important role in the prominence of the subject of plants. Project designs in the field of biology inspired by daily life can be regarded as a realistic approach. The fact that tools and equipment were easily supplied and plants were good experimental materials caused 25.6% of the submitted projects to be about Plant Physiology and Development, which was followed by Ecology (22.4%), Human Anatomy and Physiology (12%), Biochemistry (11.20%), and Microbiology and Teaching Materials (9.6%). Total ratio of the biology-related subjects mentioned under the aforesaid six sub-titles is 90.4%. The fact that the projects about Entomology, Inheritance, Teaching Materials, Soil Biology, Vitamins, Cell, and certain animals constituted 9.6% of all projects shows that some project subjects were selected completely independently from the Science and Technology curriculum. This situation should be regarded in the sense that it requires an additional good preparation for research subjects. On the other hand, the diversity of subjects in biology provides participants with important conveniences in project design.

23.6% of the project applications are about Organic Chemistry, and 8% are about Biochemistry. This indicates that some projects failed to go beyond the field of chemistry in terms of content. The fewness of the number of the projects dealing with Chemical Reactions (10.3%) and Acids and Bases (7.9%) demonstrates that chemistry subjects should be taken into consideration in the project preparation process and that the selected subjects should be put under the microscope. It is understood that there is an important deficiency in this field.

The fact that physics, biology, and chemistry subjects included in Science and Technology curriculum are not prepared with the aim of teaching and solving the issues that students are interested in or regard as problems in their daily lives not only impedes the generalization of research culture, but also leads to certain misperceptions and mistakes about the field. The project works that are not based on scientific terms, concepts, and approaches do not only cause loss of time and effort for non-interesting subjects, but also make teaching difficult by causing a misunderstanding of many concepts. This kind of learning difficulty should not be fed by project works.

Since it is necessary to conduct and finalize projects in coherence, the experts, teachers, and officials who are to take part in the selection of competition projects must act sensitively and carefully. This is why; it is required to ensure the use of the points such as originality, problem selection, hypothesis, consistency, scientific contribution, implementability, social benefit, and general effect as criteria in the evaluation of projects, and to teach that project preparation, implementation, and finalization refers to a process that must be conducted within the framework of scientific research methods and techniques. The selectors not having the above-mentioned competence should not be assigned. A separate teaching or recall is needed for each one of these criteria.

Supporting the place of a well-prepared project in the current structure of science and technology in a justifiable manner and through literature studies, reasoning over the hypotheses of it, and determining the conceptual and theoretical framework of it well increase the originality of the project. It should be noted that the methods chosen should be compatible with the purpose, support the suggested solution approach, and contain necessary variables. There should be appropriate infrastructure facilities for the project to be carried out. In addition, the project should be prone to obtaining broad results that can be employed in different fields, and have a capability to generate solutions to the problems of society. The achievement of all these expectations depends on the sufficiency of the evaluation criteria that are explicitly or implicitly used in the project evaluations. In this regard, an attempt was made to constitute a set of project evaluation criteria in the present study. The correspondence of the data collected through these criteria with the results obtained in the competition processes was examined. 23 sub-titles included in the project evaluation chart were scrutinized by two expert researchers and one consultant. The results were evaluated as “yes”, “no”, and “partly”. In addition, an attempt was made to make the criteria more effective and useful by examining whether there was any difference between the opinions of experts who evaluated sub-titles independently from one another. Since the study was conducted via survey method, new criteria that would cover the presentations and allow the evaluation of these kinds of titles were included. Since it was thought that a 5-point likert type evaluation would be more useful, the criteria were finalized to include The Determination of Problem, Scientific Method, Originality, Consistency, Contribution, Usefulness, Implementability, Source Usage, Result, and Presentation. The set of criteria came to consist of 10 main items and 33 sub-items. In fact, this approach is based on the intention for enabling the criteria to be used more effectively and more sensitively when evaluating projects.

The fact that researchers were not able to give the answer of “yes” to any one of the sub-titles during evaluations evidences that criteria were not fulfilled during project preparation. The answer, “partly” was given only to two sub-titles. 17 of 23 sub-titles were answered “no” by experts. This shows that our region was below the expected level concerning project design in accordance with competition conditions.

It is understood from the table 11 that a partial success was achieved in such criteria as the determination of problem (54.6%), the planning of method (47.1%), and the consistency between the purpose and solution of problem (73.1%). This partial success indicates that the effort made in the matter of the MNE project and performance assignments was not sufficiently understood and effectively implemented at educational institutions and organizations. If what was prescribed in the related regulations and other instructions had been performed, it would have been possible for these values to be found higher.

It goes without saying that originality has a special place in projects. If a project is based on research culture and problem-solving demands, it is needed to meticulously abide by a scientific research method, carefully plan such method, duly manage the process, and prepare result report. In consideration of the originality values of 454 Science and Technology projects under examination, it is seen that those with the highest originality values only have an originality value of 5%, which points to another basic deficiency of the projects under examination.

Establishing a hypothesis is one of the most important stages of scientific research. Based on the evaluations made by experts in accordance with project evaluation criteria, it has seen that 94.4% of the projects are unsuccessful in making a prediction for the solution of the problem and determining the way to be followed for reaching solution. If school administrators and other institutions and organizations make an effort in the matter of project management and support students and teachers, more successful results can be obtained (Özer and Özkan, 2012).

It was determined that the methods employed in most of the projects (78.5%) did not contain the variables necessary for testing the project hypotheses and that the experimental processes required by the projects were not carried out (59%). It was observed that teachers and students were incompetent about scientific process skills despite high-level expectations in the MNE legislation.

Another problem is about proper data collection. It was understood that there were significant deficiencies in both recording the collected data (77.6%) and analyzing such data (76.5%). However, it is not technically difficult to eliminate this deficiency.

It was seen that although there was a partial consistency between the purposes and the solutions of the projects, there was no consistency between the problems and sub-problems (83%). It was found that there was no search for a new method or for implementing a known method in another field within the scope of the projects under examination (88.8%). This situation evidently resulted from the deficiency in effective construction and association of scientific methods.

The points of usefulness and implementability were not sufficiently taken into account. It was determined that majority of 454 projects did not have any feature to be used in different scientific and technical fields, did not contribute to economy and society, and did not have any capability to put forward any broad and usable result related to their fields.

It is remarkable that the level of use of the sources reached through literature review within project report was quite low (0.7%). The fact that none of the projects associated the reached sources with project subjects indicates an important issue that must be focused on. This is obviously a very important deficiency. It means that the existing knowledge base about the project subject was not reached or effectively used. Accordingly, there is a deficiency about reaching the information, using the information, and associating the information with project outputs, which results from lack of mental preliminary preparation about the subject. It is thought that reminding our teachers of the fact that it is necessary to consider the research subject and knowledge and findings in the literature together through appropriate environments and conditions will make important contributions to advancement. Guidance counselors are required to review term papers and performance assignments based on the Science and Technology Curriculum 2005 of MNE Council of Education and Morality. Knowledge can be reached by doing and experiencing, however it should be noted that past experiences should not be ignored.

It is an expected result that 79.5% of the projects were finalized. Only 2.6% of 454 projects had a difficulty in interpreting cause and effect relationships. Students had a difficulty in putting forward a product.

Kruskal-Wallis test was carried out to determine whether there was any year-dependent significant difference between the criteria of The Determination of Problem, Originality and Creativity, Scientific Method, Consistency and Contribution, Usefulness, Implementability, Literature Review, and Result included, in the Evaluation Chart for Science Projects Event for Primary School Students. The test results showed that there was a significant difference between the criteria by years (p˂0.05). In addition, the mean ranking of the projects showed that they were more successful in The Determination of Problem in 2007 and 2011, in Originality and Creativity in 2009 and 2010, Scientific Method in 2010 and 2011, in Consistency and Contribution in 2009 and 2011, in Usefulness in 2009 and 2011, in Implementability in 2009 and 2011, in Literature Review in 2007 and 2011, and in Result in 2009 and 2011. This situation points to the fact that no regular and meaningful advancement was achieved in the course of time.

It was found that there was a significant difference between The Determination of Problem, Originality and Creativity, Scientific Method, and Literature Review in physics, chemistry, and biology by subject areas (p˂0.05). The mean ranking of the projects demonstrated that the projects about physics were more successful in The Determination of Problem, the projects about biology were more successful in Scientific Method, and the projects about chemistry were more successful in Originality and Creativity, Consistency and Contribution, Usefulness, Implementability, Literature Review, and Result.

Mann-Whitney U test was performed to determine whether there was any significant difference between the statuses of being invited to the exhibition of the projects making an application. It was seen that there was a significant difference in favor of the projects invited to the exhibition in terms of 8 criteria (p˂0.05). This is an indicator of the fact that there is a consistency between the results obtained through our evaluation criteria, and the previous evaluations.

Kruskal-Wallis test was carried out to investigate whether there was any significant difference between the scores obtained through project evaluation chart by provinces. There was no significant difference (p>0.05) between provinces in any criteria other than the scientific method employed (p=0.165). The mean ranking of the projects demonstrated that the projects were more successful in The Determination of Problem in Kocaeli, in Originality and Creativity and Implementability in Eskişehir, in Scientific Method in Sakarya, in Consistency and Contribution and Usefulness in Yalova, in Literature Review in Çanakkale, and in Result in Balıkesir.

Although projects and performance assignments and information concerning the assessment and evaluations of such works had a wide coverage in the legislation of the MNE as well as in the directives of the General Directorate of Primary Education besides related regulations, these kinds of expectations and requirements did not make sufficient impact at schools (MNE, 2011). The evaluation of projects and performance assignments via a grading key requires preparing criteria accordingly. The prepared grading keys should be shared with students and teachers. Since projects can be carried out individually or in groups, a long period is required sometimes. The process should be supported by such skills as curiousness, research, and communication. Within the framework of project design, students should acquire knowledge by doing and experiencing or examining. The chart suggested by the Council of Education and Morality to be used in the evaluation of project works was limited to motivation, planning, data collection, report writing, and presentation. These criteria are far from competency for evaluating the competition projects. They are mostly aimed at evaluating the term papers. Considering this deficiency, the project evaluation and scoring system was re-arranged. Through this kind of an arrangement, the chart may be turned into a scoring chart that can be used under a single title by assigning a separate score for each sub-title. However, we are of the opinion that total scores to be assigned to criteria in the projects aimed at generalization of competition and research culture should concentrate on originality, implementability, and presentation.