Research Developments - Articles online
Supporting international capacity building
ACER may be best known internationally for its involvement in large-scale student assessments, such as the OECD Programme for International Student Assessment (PISA) and the International School Assessment (ISA), but the organisation is also heavily involved in international capacity-building programs. Peter McGuckian explains.
As well as high-profile testing programs, ACER also provides a wide range of tailored services for international clients including consultancies, development of assessment and curriculum materials, and professional development for teachers and education department officials.
ACER provides advice and training to assist policymakers and practitioners to better understand the role of assessment and to develop comprehensive programs that address local needs efficiently and effectively.
In recent years training in educational assessment and evaluation has been provided for participants from Fiji, Malaysia, Indonesia, the Philippines, Hong Kong, Cambodia, Bhutan, Sri Lanka, Malaysia, Singapore and the South Pacific.
Several recent examples saw ACER staff conduct workshops on higher-order thinking in Botswana, exam writing training for staff in Indonesia, and teacher profiling, also in Indonesia.
HOT in Botswana
ACER Senior Research Fellow Dr Jennifer Bryce ran a series of workshops on higherorder thinking skills for the Botswana Examinations Council (BEC) in February 2008.
The purpose of the workshops was to assist the BEC to develop stimulating and challenging examination questions capable of testing students’ higher-order thinking skills.
According to Dr Bryce, higher-order thinking skills (HOT) are essential for young people in school, higher education and the workplace.
“With the development of the internet, with knowledge being much more readily accessible, it’s becoming more and more important to encourage students to use critical reasoning, to problem-solve, to evaluate material, and to be inventive. Obviously students do need to learn facts and their thinking needs to be based on facts, but higher-order thinking involves the understanding of information rather than the mere recall of information,” she says.
Dr Bryce ran two workshops over two weeks. The first week was attended by 35 staff from the BEC responsible for preparing assessment procedures, training teachers to write exam items, and organising the exam writing itself. These officers undertook intensive training in what higher-order thinking involves and how to develop examination items that encourage and test for higher-order thinking.
The officers then assisted Dr Bryce in the second week to train a group of 170 teachers to write exam items. The workshop for this larger group consisted of plenary sessions, practical group sessions in which participants wrote items, and a process the locals called “shredding,” where colleagues’ work is critiqued and refined. Participants worked in subject groups in the key areas of mathematics and sciences, practical subjects, languages, including English, and social studies, including moral and religious education.
Exam item writing in Indonesia
Exam writing workshops were held in Indonesia in February 2008.
Four ACER consultants travelled to Jakarta and Jogyakarta to run training as part of the Indonesian Basic Education Program. The workshops were aimed at developing the skills of trainers chosen by the Indonesian Educational Assessment Centre who will ultimately be training exam writers for the National Testing Program run in the country’s 260 000 schools.
ACER consultants Mark Butler, Helen Lye, Greg Reid and Andrew Hay were involved in the project. The three-day workshops covered mathematics, science, social sciences and English language.
About 20 participants selected by the Educational Assessment Centre attended each of the four workshops. These included staff from the national examining body, university professors, teachers and other civil servants. Following these workshops, these participants will train others in their field.
The workshops provided a background to quality exam writing, including current international best practice, and case studies of testing instruments from other countries, including Australia, and from the Programme for International Student Assessment (PISA). The workshops gave participants a general introduction to writing high-quality items, as well as practice in working with colleagues to write and refine items, and an experience of training as a partnership between presenter and participants.
According to ACER Research Fellow Mark Butler, the workshops are a move towards a consistent, high-quality national testing system. “There are 33 different provinces in Indonesia, and each province currently writes its own tests, but takes items from a national pool. It’s vital that all the items in the pool are of a very high quality,” says Butler.
“The Educational Assessment Centre is interested in developing students’ higherorder thinking, but we feel that if we focus primarily on teaching exam writers to produce high-quality items, these items will naturally address a range of skills. The workshops focused on the principles underpinning quality item writing, followed by more subject-specific issues,” he says.
Teacher profiling in Indonesia
An ongoing project to profile Indonesia’s 2.7 million teachers in 260 000 schools will result in more informed decisions regarding educational finances and staffing, says ACER Senior Research Fellow Dr Julie Kos.
The project is part of the Australia– Indonesia Partnership for Reconstruction and Development, which was created following the 2004 tsunami to fund reconstruction and development in Indonesia. Under this partnership, Australia provides extensive assistance for basic education in Indonesia with the aims of contributing to the long-term goal of Indonesian education reform, and increasing levels of educational attainment in disadvantaged areas, leading to longerterm employment and income-generating prospects.
The project aims to collect and analyse reliable, up-to-date data about the teachers, including age, gender, qualifications, length of service, teaching load, and professional development history. This information is being used by the Indonesian Ministry of National Education to develop and implement policies across the education sector.
ACER’s role is to assist the Ministry to refine its collection methods, data cleaning and analysis, and reporting. Dr Kos, who has been working on the project for 12 months, says the information will allow resources and programs to be targeted more effectively.
“The data have shown us that in Indonesia, as in Australia, the ageing population is a problem, so we need to focus on getting some younger teachers. It has confirmed that many teachers do not have a tertiary degree, and that quite a few did not complete high school. The lack of training is a huge problem, so the Ministry has implemented a certification process, and will commence the task of upgrading qualifications in the near future,” she says.
“We will also be running a similar data collection project with the Indonesian Ministry of Religious Affairs, which is responsible for the religious school sector, ” she says.
ACER’s work on data collection will be ongoing, and the Ministry is currently creating a national data collection team, of which Dr Kos will be a member. At the same time ACER will extend its involvement in Indonesia to include a number of longerterm research projects across the country, as well as student assessments, and training programs to build capacity of staff.
“We are looking at bringing some staff to Australia for training, or potentially sending ACER staff to Indonesia to train groups of teachers in schools,” says Dr Kos.
These projects are just a few examples of ACER’s international work. The key areas of ACER’s current international work are: the collection, interpretation and reporting of student achievement at the system, school and classroom levels; the development of exam and selection materials for use internationally; system-level examination reform; and curriculum framework reform. This work is focused on capacity building to support the efforts of others – especially classroom teachers, school leaders, education consultants, regional and district staff, system leaders, parents, caregivers and learners themselves – to improve educational outcomes.
