ACER eNews


ACER’s international role in PISA ACER has a dual role in PISA. In addition to implementing PISA in Australia and writing the national report, ACER leads an international consortium of research organisations and educational institutions to deliver the International PISA project on behalf of the Organisation for Economic Cooperation and Development (OECD). Before the 14,000 Australian students sat down to tackle the 2009 PISA assessments, each item had been through a rigorous process of development and trial to ensure it could be understood by students from a wide range of language and cultural backgrounds and was based on relevant, everyday situations. All items were then translated into nearly 50 languages to meet the language needs of the 65 participating OECD member countries and partner economies. Item development is just part of the work undertaken by the ACER-led consortium that conducts PISA around the world. Internationally, ACER’s work on PISA includes: Leading the development and devising the methodology of procedures required to implement the PISA survey in all participating countries. Developing and implementing sampling procedures and assisting with monitoring sampling outcomes across participating countries. Leading the development of all assessment instruments in Reading, Mathematics, Science, Problem Solving, Computer-based testing, background and contextual questionnaires; Developing purpose-built software to assist in sampling and data capture; and Analysing and assisting the OECD to prepare the international report. The OECD established PISA as a means of assessing and comparing education systems worldwide after discovering it had a number of economic measures of its member countries but no measures of educational achievement. By testing the skills and knowledge of 15-year-olds in three core subject areas, PISA determines how capable students are at applying their skills and knowledge to real-life problems and situations, and whether they can analyse, reason and communicate their ideas effectively. By assessing students at the age when they are nearing the end of compulsory schooling, PISA ascertains whether students are prepared for the challenges of life as young adults. Since 2000, PISA has been conducted every three years. The continuous cycle of PISA allows for longitudinal trends in educational performance to be monitored. PISA includes items to assess reading, mathematical and scientific literacy accompanied by stimulus material, which may include text, diagrams or images, and is followed by a questionnaire that asks students about their attitudes and background. This questionnaire seeks to gain information about students’ home and school environment to identify possible influences on school achievement. The questions used in PISA are carefully developed and selected. Expert groups are consulted, ideas are discussed among participating countries and advice is sought from various boards and groups. Every item included is rated by each country in terms of potential cultural, gender or other biases, as well as the relevance to 15-year-olds’ familiarity and level of interest. The questions are tested in field trials in participating countries before they are considered to be part of the main PISA study. When PISA is conducted every three years a nationally representative sample of 15-year-olds is chosen from each participating country. The sample is carefully designed to ensure that PISA provides an accurate snapshot of educational achievement of the full 15-year-old population. Once the tests are completed a team of specially trained markers checks them. The data is then sent back to the PISA consortium and collated with that of other countries to create an international database. The database created provides a multitude of information that may eventually be used to inform educational policy in participating countries. Information from the attitude and background questionnaire is used to analyse potential influences on students’ performance across and within countries. The data collected by PISA helps to show the successes of schools in some countries, and the challenges faced by others. It allows countries to compare best practices and to further develop an educational reform agenda appropriate for their particular school system. This article is based on an overview of PISA in a new publication, which describes projects from ACER’s growing range of international work. The first edition of International Developments can be read online at http://www.acer.edu.au/international/ More information about PISA from an international perspective can be found on the OECD PISA website « Back to eNews
Copyright © Australian Council for Educational Research 2013 All rights reserved. Except under the conditions described in the Copyright Act 1968 of Australia and subsequent amendments, no part of this electronic publication may be reproduced, stored in retrieval system or transmitted in any form or by any means, electronic, mechanical, photocopying or otherwise, without written permission. Please address any requests to reproduce information to communications@acer.edu.au Subscribe Unsubscribe Australian Council for Educational Research Private Bag 55, Camberwell, Victoria Australia 3124 Tel: + 61 3 9277 5555 Fax: + 61 3 9277 5500 Web: www.acer.edu.au

ACER has a dual role in PISA. In addition to implementing PISA in Australia and writing the national report, ACER leads an international consortium of research organisations and educational institutions to deliver the International PISA project on behalf of the Organisation for Economic Cooperation and Development (OECD).

Before the 14,000 Australian students sat down to tackle the 2009 PISA assessments, each item had been through a rigorous process of development and trial to ensure it could be understood by students from a wide range of language and cultural backgrounds and was based on relevant, everyday situations. All items were then translated into nearly 50 languages to meet the language needs of the 65 participating OECD member countries and partner economies. Item development is just part of the work undertaken by the ACER-led consortium that conducts PISA around the world.

Internationally, ACER’s work on PISA includes:

Leading the development and devising the methodology of procedures required to implement the PISA survey in all participating countries.
Developing and implementing sampling procedures and assisting with monitoring sampling outcomes across participating countries.
Leading the development of all assessment instruments in Reading, Mathematics, Science, Problem Solving, Computer-based testing, background and contextual questionnaires;
Developing purpose-built software to assist in sampling and data capture; and
Analysing and assisting the OECD to prepare the international report.

The OECD established PISA as a means of assessing and comparing education systems worldwide after discovering it had a number of economic measures of its member countries but no measures of educational achievement. By testing the skills and knowledge of 15-year-olds in three core subject areas, PISA determines how capable students are at applying their skills and knowledge to real-life problems and situations, and whether they can analyse, reason and communicate their ideas effectively. By assessing students at the age when they are nearing the end of compulsory schooling, PISA ascertains whether students are prepared for the challenges of life as young adults.

Since 2000, PISA has been conducted every three years. The continuous cycle of PISA allows for longitudinal trends in educational performance to be monitored.

PISA includes items to assess reading, mathematical and scientific literacy accompanied by stimulus material, which may include text, diagrams or images, and is followed by a questionnaire that asks students about their attitudes and background. This questionnaire seeks to gain information about students’ home and school environment to identify possible influences on school achievement.

The questions used in PISA are carefully developed and selected. Expert groups are consulted, ideas are discussed among participating countries and advice is sought from various boards and groups. Every item included is rated by each country in terms of potential cultural, gender or other biases, as well as the relevance to 15-year-olds’ familiarity and level of interest. The questions are tested in field trials in participating countries before they are considered to be part of the main PISA study.

When PISA is conducted every three years a nationally representative sample of 15-year-olds is chosen from each participating country. The sample is carefully designed to ensure that PISA provides an accurate snapshot of educational achievement of the full 15-year-old population.

Once the tests are completed a team of specially trained markers checks them. The data is then sent back to the PISA consortium and collated with that of other countries to create an international database. The database created provides a multitude of information that may eventually be used to inform educational policy in participating countries. Information from the attitude and background questionnaire is used to analyse potential influences on students’ performance across and within countries.

The data collected by PISA helps to show the successes of schools in some countries, and the challenges faced by others. It allows countries to compare best practices and to further develop an educational reform agenda appropriate for their particular school system.

This article is based on an overview of PISA in a new publication, which describes projects from ACER’s growing range of international work. The first edition of International Developments can be read online at http://www.acer.edu.au/international/

More information about PISA from an international perspective can be found on the OECD PISA website

ACER’s international role in PISA