52 stories, 4 goals, 1 mission
Contact us
19 Prospect Hill Rd Camberwell VIC 3124
T: +61 3 9277 5555
F: +61 3 9277 5500
52 stories
Week 45: eWrite
A new online assessment that automatically marks student writing is providing instant diagnostic feedback for schools, teachers and students.
The online assessment of writing for students in Years 5 to 8 is the result of work by Jocelyn Cook and Martina Bovell in ACER’s Perth office, and Charles Darr and Jenny Whatman at the New Zealand Council for Educational Research (NZCER). ‘The development of eWrite has been a collaborative project, conceptualised, designed and developed by ACER with NZCER,’ says Martina, a Senior Research Fellow in ACER’s Assessment and Reporting research program.
‘Initially, we developed 15 writing tasks for a small-scale pilot study involving students from a range of schools in Western Australia,’ Martina explains. ‘Following qualitative feedback from the test administrators, classroom teachers and pilot markers, we narrowed these down to five tasks: two persuasive writing tasks, and a description, narrative and report writing task. All of these tasks were comprehensively trialled with a large population of Australian and New Zealand students who typed their writing into an online program. Their writing was hand-marked on screen by experienced markers.’
Using the results of this trial, the parameters for machine scoring the writing were developed. ‘A “training set”, consisting of the trial scripts and their set of scores, was used to build scoring models for each of the five tasks,’ Martina explains. ‘Using the enormous analytic power of computing and some smart programming, the machine can then use the algorithms derived from the training set to score new, unseen essays written to the same topics.
‘The work of Daniel Urbach, of ACER’s Psychometrics and Methodology team, was vital to ensuring the validity of the measurement scale that underpins the suite of reports available in eWrite. So was the marking done by the expert human markers in developing the training script scores.
‘We’ve investigated the reliability of eWrite and found the correlations between the scores of the expert human markers and the computer scores are at least as high as the correlations between human markers.’
Mette Hoeyberg, Project Director in Assessment Services, has overseen the rollout of eWrite. The result, says Martina, is an accessible bank of writing tasks and associated reports that are aligned to the Australian Curriculum for English, providing teachers for the first time with a standardised assessment of writing that has direct curriculum reference and is underpinned by an empirically developed measurement scale.
‘eWrite frees up teacher marking time, but more than that, it gives immediate formative and summative feedback,’ says Martina. ‘It also provides consistent and reliable marking within classes and across schools – so it’s a really useful tool to help teachers work towards improving students’ writing.’
Continue reading...
Week 52: State of the art psychometric and statistical analysis
Week 51: Assessing civics and citizenship
Week 50: International research into teaching and learning
GOAL 1
Learners and their needs
every learner engaged in challenging learning opportunities appropriate to their readiness and needs
GOAL 2
The Learning Profession
every learning professional highly skilled, knowledgeable and engaged in excellent practice
GOAL 3
Places of learning
every learning community well resourced and passionately committed to improving outcomes for all learners
GOAL 4
A Learning Society
a society in which every learner experiences success and has an opportunity to achieve their potential
Download
Pursuing Quality and Equity through Evidence
The work of Australian Council for Educational Research
© 2013 Australian Council for Educational Research
ABN: 19 004 398 145
