ACER eNews


Students struggle to evaluate credibility on the net A study of students’ ability to evaluate digital texts has revealed that teenagers find it particularly difficult to determine the credibility and trustworthiness of material on the internet. ACER Senior Research Fellow Tom Lumley and ACER Research Director Juliette Mendelovits report on research investigating how well young people deal with information online. How well do young people deal with contradictory and unreliable information online? That’s a question we investigated using data from the first large-scale international assessment of online reading, the digital reading assessment that was part of the 2009 OECD Programme for International Student Assessment (PISA). Our study provides empirical evidence – presented at the 2012 Annual Conference of the American Educational Research Association in Vancouver, Canada, in April – that students are better able to evaluate the relevance of content they are presented with than to make basic evaluations about the credibility and trustworthiness of digital texts. Students’ ability to evaluate digital texts available online is of significance because such content is in many, perhaps most, cases not subjected to the traditional mechanisms that in print publishing exert some control over its authority, reliability, credibility and trustworthiness. To see how well they are able to evaluate digital texts available online, we analysed data from a field trial for the PISA digital reading assessment conducted in 2008 as well as from the PISA main survey in 2009 in terms of two distinct types of critical judgement that are called upon during online reading – predictive judgements and reflective judgements – as identified by Soo Young Rieh in her 2002 paper, ‘Judgement of information quality and cognitive authority in the web’ in the Journal of the American Society for Information Science and Technology. Rieh’s predictive judgements are the kinds of judgements that students make about which site to go to, based on relevance, authenticity and authority. Rieh’s reflective judgements are the kinds of judgements that students must make once a site has been reached, about its authority, reliability, credibility and trustworthiness. We found that tasks requiring students to identify contradictory information appeared to be relatively easier than tasks requiring evaluation. We found that in fact the latter kind of tasks are relatively challenging for 15-year olds, and those demanding the critical appraisal of texts for credibility or trustworthiness are particularly difficult. There is an assumption that students – so-called ‘digital natives’ in our schools – are able to use online resources in their studies that are not only relevant to the tasks they are set, but also are likely to provide trustworthy information. In order to make judgements about the reliability, credibility and trustworthiness of online content, however, students need to have criteria for evaluating information, and need to be taught how to make evaluations. This message is of critical importance to policy makers and teachers. The full paper, ‘How well do young people deal with contradictory and unreliable information online? What the PISA digital reading assessment tells us’ is available from: research.acer.edu.au/pisa/3/ « Back to eNews
Copyright © Australian Council for Educational Research 2013 All rights reserved. Except under the conditions described in the Copyright Act 1968 of Australia and subsequent amendments, no part of this electronic publication may be reproduced, stored in retrieval system or transmitted in any form or by any means, electronic, mechanical, photocopying or otherwise, without written permission. Please address any requests to reproduce information to communications@acer.edu.au Subscribe Unsubscribe Australian Council for Educational Research Private Bag 55, Camberwell, Victoria Australia 3124 Tel: + 61 3 9277 5555 Fax: + 61 3 9277 5500 Web: www.acer.edu.au

A study of students’ ability to evaluate digital texts has revealed that teenagers find it particularly difficult to determine the credibility and trustworthiness of material on the internet. ACER Senior Research Fellow Tom Lumley and ACER Research Director Juliette Mendelovits report on research investigating how well young people deal with information online.

How well do young people deal with contradictory and unreliable information online? That’s a question we investigated using data from the first large-scale international assessment of online reading, the digital reading assessment that was part of the 2009 OECD Programme for International Student Assessment (PISA).

Our study provides empirical evidence – presented at the 2012 Annual Conference of the American Educational Research Association in Vancouver, Canada, in April – that students are better able to evaluate the relevance of content they are presented with than to make basic evaluations about the credibility and trustworthiness of digital texts.

Students’ ability to evaluate digital texts available online is of significance because such content is in many, perhaps most, cases not subjected to the traditional mechanisms that in print publishing exert some control over its authority, reliability, credibility and trustworthiness.

To see how well they are able to evaluate digital texts available online, we analysed data from a field trial for the PISA digital reading assessment conducted in 2008 as well as from the PISA main survey in 2009 in terms of two distinct types of critical judgement that are called upon during online reading – predictive judgements and reflective judgements – as identified by Soo Young Rieh in her 2002 paper, ‘Judgement of information quality and cognitive authority in the web’ in the Journal of the American Society for Information Science and Technology.

Rieh’s predictive judgements are the kinds of judgements that students make about which site to go to, based on relevance, authenticity and authority. Rieh’s reflective judgements are the kinds of judgements that students must make once a site has been reached, about its authority, reliability, credibility and trustworthiness.

We found that tasks requiring students to identify contradictory information appeared to be relatively easier than tasks requiring evaluation. We found that in fact the latter kind of tasks are relatively challenging for 15-year olds, and those demanding the critical appraisal of texts for credibility or trustworthiness are particularly difficult.

There is an assumption that students – so-called ‘digital natives’ in our schools – are able to use online resources in their studies that are not only relevant to the tasks they are set, but also are likely to provide trustworthy information. In order to make judgements about the reliability, credibility and trustworthiness of online content, however, students need to have criteria for evaluating information, and need to be taught how to make evaluations. This message is of critical importance to policy makers and teachers.

The full paper, ‘How well do young people deal with contradictory and unreliable information online? What the PISA digital reading assessment tells us’ is available from: research.acer.edu.au/pisa/3/

Students struggle to evaluate credibility on the net