ACER eNews


Comparing results from PISA and TIMSS In early December 2004, the results of two international studies were released providing the most recent evidence we have on how levels of school achievement in Australia compare with international standards. ACER's chief executive Geoff Masters compares the results. The OECD Programme for International Student Assessment (PISA) surveys reading, mathematical and scientific literacy levels every three years. The Trends in International Mathematics and Science Study (TIMSS) surveys student achievement in mathematics and science every four years. Every 12 years PISA and TIMSS align, and their results are released more or less simultaneously. This was the case in 2004. Both surveys were conducted in 2002/03 in more than 40 countries. In Australia , samples of students were drawn from all states and territories and included government, Catholic and independent schools. As well as providing information about overall levels of achievement, PISA and TIMSS provide details of the performances of girls, boys, students in each State/Territory, Indigenous students, students living in cities/regional/rural areas, students with non-English language backgrounds, and students from various socioeconomic backgrounds. While PISA and TIMSS have much in common, they provide different, but complementary, information about levels of student achievement. PISA looks at 15-year-olds-who in most countries are approaching the end of compulsory schooling-and asks how well they are able to apply basic understandings and skills in reading, mathematics and science to everyday situations. TIMSS, on the other hand, looks at how well Year 4 and Year 8 students have mastered the factual and procedural knowledge taught in school mathematics and science curricula. TIMSS begins with a detailed analysis of Year 4 and Year 8 mathematics and science curricula and then tests curriculum content common across participating countries. So what did we learn? It will take some time to sift through the results in detail, but there are some clear and immediate conclusions. From PISA we learn that Australian 15-year-olds perform well (on average) when it comes to careful reading, logical thinking, and the application of reading skills and mathematical and scientific understandings to everyday problems. In fact, Australian students are among the best in the world on tasks of this kind. In reading literacy we rank fourth among 41 participating countries, behind Finland , Korea and Canada , with only Finland significantly outperforming Australia . In mathematical literacy, we rank eleventh, with only Hong Kong SAR, Finland , Korea and Netherlands significantly outperforming Australia . In scientific literacy, we rank sixth, significantly below Finland , Japan and Korea . The conclusion from PISA is that, on average, Australian 15-year-olds have relatively high levels of reading, mathematical and scientific 'literacy', defined as the ability to apply skills in reading and basic mathematical and scientific principles and processes to everyday problems. PISA also reveals that many students in Australia -as in other countries-complete the compulsory years of school with only minimal levels of reading, mathematical and scientific literacy. For example, among Australian 15-year-olds, 7 per cent of girls and 17 per cent of boys perform at or below the lowest international reading benchmark. Many of these students are able to locate specific detail in a piece of text, but are unable to connect ideas or to draw conclusions from a piece of writing. From TIMSS we learn that Australian students perform less well on tests of mathematical and scientific knowledge . Among the 25 countries testing at Year 4 in 2002/03, Australia ranked 16 th in mathematics and 11 th in science. Countries significantly outperforming Australia in either Year 4 mathematics or science included England , USA , Latvia , Lithuania , Russian Federation , Hungary and Cyprus . Worse, over the past decade, achievement levels in Australia remained largely static while achievement levels in many other countries increased. The result is that some countries which were below or equal to Australia a decade ago in school science achievement (eg, Hong Kong SAR, England ) and school mathematics (eg, England and Hungary ) now outrank us. Among the 46 countries testing at Year 8, Australia ranked 14 th in mathematics and 11 th in science. Countries significantly outperforming Australia in either Year 8 mathematics or science included England , Belgium , Netherlands , Estonia and Hungary . And while our performance in Year 8 science improved over the past decade, half the countries we outscored in Year 8 mathematics in 1994/5 improved to perform at the same level as Australia in 2002/03. What lessons can we learn from these observations? PISA and TIMSS provide information about different aspects of students' mathematics and science learning. PISA assesses careful reading, logical thinking and the application of general mathematical and scientific processes and principles to everyday problems. TIMSS assesses mastery of the factual and procedural knowledge taught in school mathematics and science curricula. While students in some countries-such as Hong Kong SAR and Korea -perform very well in both these areas, students in some other countries perform better in one area than the other. In Australia and New Zealand students perform better (on average) in applying general mathematical and scientific principles and skills to everyday problems than in recalling and using curriculum-based factual and procedural knowledge. As an illustration, Australian high school students significantly outperform students in the United States in the first of these two areas, but perform no better than US students in the second. An obvious question that follows from these observations is whether Australian schools are placing sufficient emphasis on the teaching of factual and procedural knowledge in mathematics and science, particularly at Year 4. While 73 per cent of Year 4 students in Singapore reach the high international mathematics benchmark, only 26 per cent of Australian students reach this benchmark. The corresponding percentages for Year 4 science are 61 per cent and 38 per cent. And, relative to other countries, Australian Year 4 students now perform less well in school mathematics and science than they did a decade ago. During the 1990s, considerable effort went into the reform of curricula for the primary and middle years of schooling in Australia , resulting in new state curriculum and standards frameworks. In the same period, education systems introduced systemwide testing programs to monitor student and school achievement. It is not clear that these efforts have improved levels of mathematics and science performance in Australian primary schools. If Australia is to lift its performance in TIMSS over the next decade, then greater attention will need to be given to the teaching of basic factual and procedural knowledge and the development of teachers' confidence and competence in teaching primary school mathematics and science. The focus of the past decade on what is taught (the curriculum) needs to be accompanied by a greater focus on how subject matter is taught (research-based pedagogy). And testing programs for accountability and monitoring need to be complemented by assessments more capable of diagnosing individuals' learning difficulties and providing guidance to classroom teaching and learning. « Back to eNews
Copyright © Australian Council for Educational Research 2013 All rights reserved. Except under the conditions described in the Copyright Act 1968 of Australia and subsequent amendments, no part of this electronic publication may be reproduced, stored in retrieval system or transmitted in any form or by any means, electronic, mechanical, photocopying or otherwise, without written permission. Please address any requests to reproduce information to communications@acer.edu.au Subscribe Unsubscribe Australian Council for Educational Research Private Bag 55, Camberwell, Victoria Australia 3124 Tel: + 61 3 9277 5555 Fax: + 61 3 9277 5500 Web: www.acer.edu.au

In early December 2004, the results of two international studies were released providing the most recent evidence we have on how levels of school achievement in Australia compare with international standards. ACER's chief executive Geoff Masters compares the results.

The OECD Programme for International Student Assessment (PISA) surveys reading, mathematical and scientific literacy levels every three years. The Trends in International Mathematics and Science Study (TIMSS) surveys student achievement in mathematics and science every four years. Every 12 years PISA and TIMSS align, and their results are released more or less simultaneously. This was the case in 2004.

Both surveys were conducted in 2002/03 in more than 40 countries. In Australia , samples of students were drawn from all states and territories and included government, Catholic and independent schools. As well as providing information about overall levels of achievement, PISA and TIMSS provide details of the performances of girls, boys, students in each State/Territory, Indigenous students, students living in cities/regional/rural areas, students with non-English language backgrounds, and students from various socioeconomic backgrounds.

While PISA and TIMSS have much in common, they provide different, but complementary, information about levels of student achievement.

PISA looks at 15-year-olds-who in most countries are approaching the end of compulsory schooling-and asks how well they are able to apply basic understandings and skills in reading, mathematics and science to everyday situations.

TIMSS, on the other hand, looks at how well Year 4 and Year 8 students have mastered the factual and procedural knowledge taught in school mathematics and science curricula. TIMSS begins with a detailed analysis of Year 4 and Year 8 mathematics and science curricula and then tests curriculum content common across participating countries.

So what did we learn? It will take some time to sift through the results in detail, but there are some clear and immediate conclusions.

From PISA we learn that Australian 15-year-olds perform well (on average) when it comes to careful reading, logical thinking, and the application of reading skills and mathematical and scientific understandings to everyday problems. In fact, Australian students are among the best in the world on tasks of this kind. In reading literacy we rank fourth among 41 participating countries, behind Finland , Korea and Canada , with only Finland significantly outperforming Australia . In mathematical literacy, we rank eleventh, with only Hong Kong SAR, Finland , Korea and Netherlands significantly outperforming Australia . In scientific literacy, we rank sixth, significantly below Finland , Japan and Korea .

The conclusion from PISA is that, on average, Australian 15-year-olds have relatively high levels of reading, mathematical and scientific 'literacy', defined as the ability to apply skills in reading and basic mathematical and scientific principles and processes to everyday problems.

PISA also reveals that many students in Australia -as in other countries-complete the compulsory years of school with only minimal levels of reading, mathematical and scientific literacy. For example, among Australian 15-year-olds, 7 per cent of girls and 17 per cent of boys perform at or below the lowest international reading benchmark. Many of these students are able to locate specific detail in a piece of text, but are unable to connect ideas or to draw conclusions from a piece of writing.

From TIMSS we learn that Australian students perform less well on tests of mathematical and scientific knowledge . Among the 25 countries testing at Year 4 in 2002/03, Australia ranked 16 th in mathematics and 11 th in science. Countries significantly outperforming Australia in either Year 4 mathematics or science included England , USA , Latvia , Lithuania , Russian Federation , Hungary and Cyprus . Worse, over the past decade, achievement levels in Australia remained largely static while achievement levels in many other countries increased. The result is that some countries which were below or equal to Australia a decade ago in school science achievement (eg, Hong Kong SAR, England ) and school mathematics (eg, England and Hungary ) now outrank us.

Among the 46 countries testing at Year 8, Australia ranked 14 th in mathematics and 11 th in science. Countries significantly outperforming Australia in either Year 8 mathematics or science included England , Belgium , Netherlands , Estonia and Hungary . And while our performance in Year 8 science improved over the past decade, half the countries we outscored in Year 8 mathematics in 1994/5 improved to perform at the same level as Australia in 2002/03.

What lessons can we learn from these observations?

PISA and TIMSS provide information about different aspects of students' mathematics and science learning. PISA assesses careful reading, logical thinking and the application of general mathematical and scientific processes and principles to everyday problems. TIMSS assesses mastery of the factual and procedural knowledge taught in school mathematics and science curricula. While students in some countries-such as Hong Kong SAR and Korea -perform very well in both these areas, students in some other countries perform better in one area than the other.

In Australia and New Zealand students perform better (on average) in applying general mathematical and scientific principles and skills to everyday problems than in recalling and using curriculum-based factual and procedural knowledge. As an illustration, Australian high school students significantly outperform students in the United States in the first of these two areas, but perform no better than US students in the second.

An obvious question that follows from these observations is whether Australian schools are placing sufficient emphasis on the teaching of factual and procedural knowledge in mathematics and science, particularly at Year 4. While 73 per cent of Year 4 students in Singapore reach the high international mathematics benchmark, only 26 per cent of Australian students reach this benchmark. The corresponding percentages for Year 4 science are 61 per cent and 38 per cent. And, relative to other countries, Australian Year 4 students now perform less well in school mathematics and science than they did a decade ago.

During the 1990s, considerable effort went into the reform of curricula for the primary and middle years of schooling in Australia , resulting in new state curriculum and standards frameworks. In the same period, education systems introduced systemwide testing programs to monitor student and school achievement. It is not clear that these efforts have improved levels of mathematics and science performance in Australian primary schools.

If Australia is to lift its performance in TIMSS over the next decade, then greater attention will need to be given to the teaching of basic factual and procedural knowledge and the development of teachers' confidence and competence in teaching primary school mathematics and science. The focus of the past decade on what is taught (the curriculum) needs to be accompanied by a greater focus on how subject matter is taught (research-based pedagogy). And testing programs for accountability and monitoring need to be complemented by assessments more capable of diagnosing individuals' learning difficulties and providing guidance to classroom teaching and learning.

Comparing results from PISA and TIMSS