Research conducted on VCE data over the past ten years illustrates the flawed general statement that ‘girls outperform boys’ at VCE level, and draws attention to the dangers associated with referring to differences between classes as the class/teacher effect or as the teacher effect. These patterns are interpreted quite differently when both the data and the statistical analyses are verified and interpreted in context.

School performance evaluations based on students’ unadjusted (raw) marks favour schools with higher intakes of bright and advantaged students. The learning gains of middle and lower ability students are overlooked and the achievements of students and schools in disadvantaged areas are not valued when the focus is concentrated solely on those achieving the highest marks. With ability-adjusted analyses of school data however, each student who achieves higher marks than similar ability peers is recognised as having performed well, and this is a fairer method of evaluating school performance.

For this project, extensive quantitative and qualitative research was undertaken within more than one hundred schools in 2005, as part of ACER’s Data Interpretation Service. Since 2000, ability-adjusted information for each student, class and subject is provided to participating schools via user-friendly graphs and tables on a CD, to allow flexible access to their data throughout the year. A Professional Learning Seminar is conducted with senior staff, so that the patterns in the data are interpreted both statistically and educationally. Teachers like the detailed visual presentation of both the raw and ability-adjusted data, as they can verify the scores, and integrate relevant background information when interpreting patterns in their results. 

The broad statement ‘Girls are outperforming boys in VCE’ is misleading because of the wide range in marks for boys and girls at each level of ability. The typical pattern is that bright boys generally achieve as well, if not better than bright girls, but more lower-ability boys perform worse than lower-ability girls. The mean gender effect at class and school levels, in both this ‘within-schools’ and ‘across-schools’ VCE research, is accounted for largely by the poor results of some lower ability boys, and not because all boys are performing worse than all girls.

More useful questions in relation to gender are: ‘Which boys are performing better than similar ability boys within each school?’ and ‘What factors in schools influence some low ability boys to perform well, while other low ability boys do poorly? The relevant focus ought be ‘which boys’ and ‘which girls’ when gender differences are quoted in research reports.

Results of this research indicate that the difference between ability-predicted and actually achieved VCE scores can not be solely attributed to the class teacher. In some schools, two classes in the same subject taught by the same teacher had significantly different results, indicating that other un-measured factors were affecting student performance. 

Consequently, the commonly repeated statement that around 30% of the difference between classes is due to differences between teachers was not supported in this research, when interpretations of VCE results were validated in discussions with senior staff in schools. Few researchers have analysed data on which teacher effects are calculated authentically, because appropriate measures of verifiable teacher behaviours explicitly linked to improvement in student achievement are just not available. Until such measures are produced and validated, current claims regarding teacher effect sizes in educational research, whether calculated from multi-level models of students in schools, or even students in classes, remain questionable.

However, this does not mean that teachers, and the quality of teaching, are not vitally important factors influencing student achievement. It is just that research has not yet accurately and independently measured the specific teacher behaviours and attitudes that positively impact student performance across all ability levels yet. Similarly, while research has measured the effects of individual and class ability on student achievement, as well as calculated student gender and year level effects, there have been no measures of other equally important, contributory factors. For example, we have yet to calculate the effects of student motivation and aspirations, time on task, private tutoring, illness and personal trauma, all of which affect student performance. All student and class factors need to be taken into account, as well as subject, school and community factors, before it is possible to speak definitively about the teacher effect. This may never be totally possible, given all the above-mentioned variables that affect student achievement are interdependent, not isolated factors. 

However, in some cases where classes had strong, positive differences between their predicted and achieved VCE scores year after year, colleagues in these schools attributed these results to ‘high-performing’ teachers. Research could be usefully focused within such schools where successful ability-adjusted performance had been validated, so that successful teaching and learning strategies could be shared across schools. Similarly, verified factors that positively influenced boys and girls’ achievements, and those of low, average and high ability students could be identified and promulgated. These variables need to be investigated at student, class, subject and school levels, as well as within regions, systems and at state level.

Current government and system-level monitoring of education still over-emphasises teacher and school effectiveness, but fails to take into account the multi-level structure within which teaching and learning operate, that is, they fail to measure the relevant factors at each significant and interacting levels in education.

Ms Carmel Richardson is a Senior Research Fellow at the Australian Council for Educational Research and presented a paper at the ACER Research Conference. The full conference paper is available on the ACER web site.

« Back to eNews