Asia-Pacific Forum on Science Learning and Teaching, Volume 17, Issue 2, Article 8 (Dec., 2016) Nadi SUPRAPTO What should educational reform in Indonesia look like? - Learning from the PISA science scores of East-Asian countries and Singapore Previous Contents Next

Findings

1. Triggering from PISA 2012

For PISA 2012, the main science results from OECD (2014a) can be summarized as follows: 1) the mean in science for OECD countries increased to 501 points; 2) Shanghai-China, Hong Kong-China, Singapore, Japan and Finland are the top five performers in science in PISA 2012; 3) Between 2006 and 2012, and between 2009 and 2012, Italy, Poland and Qatar, and, Estonia, Israel and Singapore, respectively, increased their shares of top performers and simultaneously reduced their shares of low performers in science, and 4) across OECD countries, 8% of students are top performers in science (Level 5 or 6). These students can identify, explain and apply scientific knowledge and knowledge about science in a variety of complex life situations. Table 1 shows the rankings of several countries participating in PISA 2012. As mentioned in the introduction, Indonesia’s performance in science was ranked near the bottom. In contrast, some countries from East Asia and Singapore earned the top ranking.

Table 1. The highest and lowest ranking countries for performance in science (PISA 2012)

No	OECD average (Including Mathematics, Reading, and Science)*		Science
			Country rank	Mean scores	Annual change in score points
1	Shanghai-China**	587.67	Shanghai-China**	580	1.8
2	Singapore***	555.33	Hong Kong**	555	2.1
3	Hong Kong**	553.67	Singapore***	551	3.3
4	Taiwan**	535.33	Japan**	547	2.6
5	Korea**	542.67	Finland	545	-3.0
6	Macao**	522.67	Estonia	541	1.5
7	Japan**	540.33	Korea**	538	2.6
8	Liechtenstein	525.33	Vietnam***	528	-
9	Switzerland	518.33	Poland	526	4.6
10	Netherland	518.67	Liechtenstein	525	0.4
11	Estonia	526.00	Canada	525	-0.5
12	Finland	529.33	Germany	524	1.4
13	Canada	522.00	Taiwan**	523	-1.5
...	...
	OECD average	494	OECD average	501	0.5
...	...
56	Costa Rica	425.67	Montenegro	410	-0.3
57	Albania	365.00	Jordan	409	-2.1
58	Brazil	402.00	Argentina	406	2.4
59	Argentina	396.67	Brazil	405	2.3
60	Tunisia	396.67	Colombia	399	1.8
61	Jordan	398.00	Tunisia	398	2.2
62	Colombia	392.67	Albania	397	2.2
63	Qatar	382.67	Qatar	384	5.4
64	Indonesia***	384.33	Indonesia***	382	-1.9
65	Peru	375.00	Peru	373	1.3

(Reorganized from OECD, 2014b)
*mean was counted from PISA data; the PISA 2012 survey focused on mathematics, with reading, science and problem-solving minor areas of assessment
** East Asia countries
*** Southeast Asia countries

2. Trends of Science Content and Science Scores in PISA Science Content

Science Content

Sadler and Zeidler (2009) acknowledged several features of the science portion of PISA as an innovative international test: a variety of items, contextualized prompts, emphasis on scientific processes and the use of scientific evidence. The test also equipped students in decision making in real-life S&T situations. They concluded that the science PISA framework has much in common with socio-scientific issues (SSI) in science education. In addition, Fensham (2013) described the content of the science PISA by starting from scientific literacy as a representation of vision toward science knowledge. There were three cognitive and three affective scientific competencies: identifying scientific issues, explaining phenomena scientifically and using scientific evidence and interest in science, support for science and responsibility towards resources and environments (Fensham, 2013; OECD, 2013a).

Science Scores

Figure 1 depicts the trends of PISA science scores from some countries which have unique trends from 2006 until 2012. The first trend is an accelerating or positively parabolic trend as performed by Macao-China. We see a similar trend in the steadily changing or linear positive results of Hong Kong, Korea, Japan, etc. The trends of Indonesia and several other countries showed no changes over the three periods of the test.

Figure 1. Curvilinear trajectories of average science performance across PISA assessments
Rate of acceleration or deceleration in performance (quadratic term) (OECD, 2014a)

3. Expectation to PISA 2015

The 2015 PISA places science as a major domain; meanwhile, reading, mathematics, and collaborative problem solving are viewed as minor domains. Financial literacy is an additional domain that will be assessed this year. CMEC (2015) stated that approximately 70 countries/economies have been participating, including Indonesia. Contextual questionnaires also are administered to students and school principals. The assessment is entirely computer-based. The results of PISA 2015 will be published in December 2016. Indonesia has big expectations towards the results and hopes to either improve the ranking or increase the scores in science, mathematics, reading, and literacy.

Table 2. Four aspects will be assessed in science PISA 2015

Dimension	Description
Contexts	Personal, local, national and global issues, both current and historical, which demand some understanding of science and technology.
Knowledge	An understanding of the major facts, concepts and explanatory theories that form the basis of scientific knowledge. Such knowledge includes both knowledge of the natural world and technological artifacts (content knowledge), knowledge of how such ideas are produced (procedural knowledge) and an understanding of the underlying rationale for these procedures and the justification for their use (epistemic-knowledge).
Competencies	The ability to explain phenomena scientifically, evaluate and design scientific inquiry, and interpret data and evidence scientifically.
Attitudes	A set of attitudes towards science indicated by an interest in science and technology; valuing of scientific approaches to inquiry, where appropriate, and a perception and awareness of environmental issues.

Source: OECD (2013b)

In particular, according to science assessment of PISA 2015, scientific literacy may be characterized as consisting of four interrelated aspects, as shown in Table 2. In addition, Figure 2 depicts the framework of scientific literacy assessment more diagrammatically. In terms of context, the student should be able to recognize life situations involving science and technology by thinking personally, locally, and globally. Regarding competencies, students must be able to identify scientific issues, explain phenomena scientifically, interpret data and use scientific evidence. Turning to attitudes, students must perform well in supporting scientific inquiry, motivation, and act responsibly towards natural resources and environments.

Figure 2. Framework for PISA 2015 Scientific Literacy Assessment (OECD, 2013b)

4. Education System and Assessment in Indonesia

In Indonesia, the education system has undergone radical change in the twenty-first century (Berry, 2011). This reform has been marked by the implementation of school-based management (manajemen berbasis sekolah), which includes reforming the national education objectives, decentralizing management from the government of schools and implementing the 2004 curriculum, KTSP curriculum, and 2013 curriculum. In the past, the Indonesian education system placed a heavy emphasis on cognitive-attainment by students (Muhaimin & Ali, 2001). The new curriculum aims at promoting students’ ability in applying knowledge in real life situations and calls for teachers’ to use classroom-based assessment to support learning. A widespread feeling is that continuous professional growth of teachers, strong school management, and leadership are the keys to the successful implementation of the reforms (Raihani, 2007).

In particular, assessment reform is very important because it is influenced by the globalization process. In line with Broadfoot (2007), assessment reform serves as a response to new social pressure. In the case of PISA results, all of the education components in Indonesia are under social pressure.

5. Assessment Reform around the World

Over the past few years, new approaches to assessment have emerged in a number countries. In their assessment policies, many countries have highlighted the need to promote learner autonomy, a key element of educational success. Assessment has been mainly used for selection and accountability purposes in the eastern and western worlds, including Indonesia. As people have become aware of the problems of high-stakes examinations, assessment can be used as a tool to support learning and enhance teaching. Most countries have relied on an education reform with high emphasis on assessment as part of their learning agendas. These countries, thus, have sought to reduce excessive use of tests and examinations, and have used assessment to understand and support learning, while using student information to improve teaching.

The international comparison of results has done little to help establish an assessment for learning about culture. Some countries hold schools and teachers accountable for the performance of their students in the standardized inter-school comparative tests. Consequently, although many teachers acknowledge the significance of formative assessment in student learning, teaching is still very much test-oriented. To help students achieve better results, it has become common practice to design tests simulate high-stakes external examination and to teach conventional types of knowledge and competence. Some countries have reported success in their assessment reforms, such as Shanghai-China (Tan, 2013), Singapore (Ng, 2008), and Hong Kong (Yung, 2006; Darling-Hammond, 2010).  The important thing for these countries is shifts in perception about learning that are commonly and internationally labeled as the need for “lifelong learning”, “learning-to-learn” and “whole-person development” (Berry, 2011).

6. Education and Assessment Reform for Science in Shanghai-China

Zhao (2011) analyzed assessment reform in China by promoting the Assessment Indicator System for Sustainable Cities. The functions of this system are: 1) allows the city to systematically analyzed and further determine key issues that need to be resolved; 2) enables decision makers to focus on key issues and prioritize areas related to sustainable development; 3) can instruct policy and decision makers to let them better understand the framework for sustainable city development clearly; 4) can simplify and improve the understanding of sustainable city development among all the groups of society, promote their understanding of related plans and actions and take active measures and actions in a cooperative manner; 5) can show the status and executive effect of policies on sustainable city construction development and enable people to understand the progress of developments at any time; and 6) serves as a control tool and precautionary method used by decision makers and managers.

Moreover, China Digital Time- CDT (2013) reported that Shanghai (one of the biggest city in China) has been named the world champion of PISA for the second time. Shanghai’s students were ranked the highest globally with an average score of 613 in 2012 (up from 600 in the previous PISA test in 2010). Actually, we can learn many things from Shanghai. For example, in Shanghai we see chess master teachers, tiger mothers, dragon children, and application of the Kung Fu Panda philosophy to teacher mentoring and collaboration. In other words, the educational success in Shanghai is a combination of structural and sociocultural factors. Stimulated by the motivation to understand how this region become the best in PISA, conversations with principals, teachers, parents, and students is very important.

In particular, Tan (2013) successfully explored and depicted the success of education in Shanghai. According to her, there were four components of education success: Shanghai’s shared vision of education; using standards and the best policy; many resources, hard work, conducting research and teacher development; and, synergies between teaching, learning and thinking. In particular, the introduction of testing and the various mechanics of performance have not caused the teachers to lose their rationale for practice and their relationship to the meaningfulness of what they do. In addition, the balancing between decentralization and centralization and the implementation of autonomy and accountability as part of the school appraisal system have become the key success factor in this city. In Shanghai, testing and examination serve as a means of central control. Moreover, according to Tan (2013),

From Shanghai, we can learn ‘ABC’, A= Anchor yourself on what you’re already good at; B= Borrow new ideas judiciously; C= Continue to improve and excel.

In science class, the Chinese approach is applied to curriculum, pedagogy (teaching-learning), innovation in assessment, and thinking. Regarding thinking, critical thinking, and creative thinking, teachers encourage students to become a great thinker and a hard worker (Tan, 2013; CDT, 2013). Turning to teaching, the “post-tea house teaching” approach (lesson delivery and lesson preparation) and dialogue-style teaching, from “teacher talk to student talk” are used in daily lessons (Tan, 2013).   The following example is part of a student worksheet on a science topic in class:

...1. Since electricity can generate magnetic energy, can magnetic energy generate electricity? (Encourages students’ inferential thinking)
2. Read the textbook, page … on an experiment conducted. Which of the following scenarios will have the magnetic energy generating electricity? (Encourages students to draw their own conclusions)
3. Look at the equipment on the table. [Students are shown pictures of equipment.] Which equipment is able to produce magnetic fields? (Encourages students to apply what they have learned and made their own judgments)
4. Read the textbook, page… Discuss with the rest and share what you have learned about scientific research and the importance of physics. (Encourages students to relate what they have learned to their lives)... (Tan, 2013).

7. Education and Assessment Reform for Science in Hong Kong

In Hong Kong, high-stakes assessments are mandated to involve by a mandated school-based continuous assessment schemes, such as the Teacher Assessment Scheme (TAS) (Yung, 2006). The teachers’ roles in TAS are either as an assessor or a teacher. By implementing TAS, teachers in Hong Kong:

• Alleviate the problem of the over-practicing of rat dissections
• Reduce students’ examination pressure (with regard to practical work)
• Allow a valid assessment of students’ practical abilities
• Enhance teachers’ professionalism and widen their experience 
(Yung, 2006;  p13).

Turning to science education, several lessons that can be drawn from the success of science teachers in Hong Kong can be summarized from a study by Yung (2006):

a. Science teachers do not just teach students to make a living but to enhance active learning

b. Science teachers apply a spiral of teaching, e.g: analogies, illustrations, examples from everyday life in explaining abstract concepts.

c. Science teachers argue that predictive capability is the greatest thing about science

d. Science teachers use mind-training through practical teaching, e.g: conceptual understanding, identify misconceptions, and problem-solving

e. Science teachers conduct practical work by developing thinking habits, scientific attitudes, and social skills

f. Science teachers emphasize the aspects of the nature of science (NOS)

g. Science teachers are aware the students’ role in the process of learning

h. Science teachers’ beliefs underlie classroom practices and assessment practices

i. Science teachers integrate assessment with teaching and learning

8. Education and Assessment Reform for Science in Taiwan

Driskell (2014) argued that Taiwan’s education system has historically been criticized for putting too much pressure on students and focusing too heavily on exams requiring rote memorization rather than the creative application of knowledge.  Recently, reforms have focused on fostering the critical thinking and literacy skills necessary to be internationally competitive on PISA. In response to this need, the Ministry has promoted changes to policy on teachers’ professional development, so that teachers are encouraged to teach reading and develop reading curriculum with more of a focus on critical thinking. Teachers have designed formative diagnostic assessments and lesson plans that begin to incorporate more critical thinking exercises that reflect the higher order skills measured by PISA. More broadly, PISA has changed the extent to which the nation regards teachers as professionals who are expected to improve their own skills.

Specifically, in science education, Tsai, et al (2011) noticed that the role of socio-cultural factors, the epistemology of science (or the nature of science), and educational technology are important issues in the field when exploring students’ science learning and investigating students’ conceptions and learning environments. The development of constructivist pedagogy and evaluation of its effects on academic performance have encouraged students to explore how to promote the practice of higher-order thinking. In terms of Bloom’s Taxonomy, higher-order thinking (HOTs) in the cognitive domain may include modes such as evaluating and creating in comparison with remembering, understanding, applying, and analyzing (Anderson & Krathwohl, 2001). Moreover, in Taiwan, the incorporation of technologies such as databases and online technologies in instructional designs is becoming a trend in science classrooms. Nevertheless, most technology-enhanced instructions are aimed at improving learners’ conceptual understanding and basic process skills.

9. Education and Assessment Reform for Science in South Korea

Darling-Hammond (2010) pointed out that Korea achieved educational success by replacing overcrowded curriculum and emphasizing several features: focusing on deeper understanding of concepts; developing of core competencies: HOTs, self-control, responsibility, independence, creativity, self-directed learning,  problem-solving, and social capital development; addressing the needs of a global knowledge-based economy; and infusing technology. According to Darling-Hammond, “Every Korean school had high-speed internet connections in classroom by 2002 and ICT used at least 10% of every subject”.

In Korea, the Ministry of Education, Science, and Technology has strived to strengthen teachers’ competency, supporting professional development through science in-service programs. According to Slate (2012), there are more than a hundred science centers, which provide students with an in-depth science curriculum and laboratories to support science education. Korean students generally spend quite a lot of time studying concepts and principles rather than engaging in hands-on activities at the secondary level. This is a strategy of obtaining knowledge in a short time and probably contributes to higher math and science test scores. Learning concepts and principles is as important as learning through hands-on activities.

10. Education and Assessment Reform for Science in Japan

In general, the Japanese educational system used top-down orientation and reaffirmed the Japanese approach, “the Meiji reforms”. Based on OECD (2012), several features and main characteristics of the educational system influenced the results of PISA: encouraging competition among schools, balancing of public and private education, growing reliance of private tutoring, implementing universal pre-primary education, autonomy in curricular decisions, a high-quality teaching force (i.e. lesson study), effective school-home communication, setting standards and accountability arrangements, and low levels of differentiation and emphasis on heterogeneous classes. In addition, Japan applied a new philosophy to education, namely Ikiru Chikara or zest for living, which emphasized key competencies: independent thinking and problem solving. Moreover, Japanese peoples have a high commitment to education, which assures that all students can achieve at high levels. In other words, the education still emphasizes values.

11. Education and Assessment Reform for Science in Singapore

In general, Singapore’s success in education has been influenced not only by the advantages of the small scale but also several key features. Based on an integrated planning system, Singapore developed a new framework for Curriculum 2015. According to OECD (2012), the orientation of this curriculum aims for every student to become: a confident person who thinks independently, critically, and communicates effectively; a self-directed learner; an active contributor, with innovation and initiative; a concerned citizen who is informed about the world and local affairs. In addition, close links between policy implementers, researchers and educators and commitment to equity and merit have become the key to success.

Singapore has a strong focus on mathematics, science, and technical skills. Consequently, students are now more engaged in science project work and HOTs questions. Moreover, “teach less and learn more” is now a widely used slogan (Ng, 2008). In line with Quek and Tan (2013), several educational policies have been implemented in Singapore, such as: Thinking Schools Learning Nation (TSLN), Teach Less Learn More (TLLM), IT Master Plans, Innovation and Enterprise (I&E), Project Work (PW), and Integrated Programs (IP). Furthermore, 21st-century skills, civic literacy, global awareness, and cross-cultural skills became priorities. Darling-Hammond (2010) highlighted that Singapore’s students routinely speak English, the language of the test, at home. Moreover, the OECD (2012) also noted the following ten key lessons learned from Singapore:

1. A bold long-term education vision and leadership
2. Alignment of the education system to economic development goals
3. Coherence of the education system
4. Clear goals, rigorous standards, and high-stakes gateways
5. Curriculum, instruction, and assessment to match the standards
6. High-quality teachers and principals
7. Strong central capacity and authority to act
8. Accountable for performance management
9. Believing in meritocratic values, all ethnic student backgrounds and all ranges of ability is the route to advancement
10. Adaptation of proven practices from abroad

Specifically, Singapore is interested in changing the balance in student assessment from the assessment of learning to assessment for learning. The Singapore science curriculum in primary and lower secondary grades focuses on developing the idea of science as a process of inquiry which consists of three domains: knowledge, understanding and application; skills and processes; and ethics and attitudes. By having a useful skill, inquiry projects based on the roles played by science in daily life and society, awaken students' interest in science. Moreover, co-curricular activities such as science fairs, competitions, science in outdoor settings have been designed to generate interest among students. The important thing is, science teachers are selected from the top one-third of their cohort, receive initial training on the national science curricula during their pre-service training and are entitled to 100 hours of professional development each year (OECD, 2012). To sum up, the Singapore government wants students to develop the following traits: a spirit of inquiry and original thinking; a willingness to do something differently; a spirit of character; and a sense of giving back to the community.