Assessment of Language Competence (ALC)

Effect of Gender and reading ability on questions in the target language in L2 listening tests

(Mr Luc Le, Dr Anna Filipi, Ms Bernadette Brouwers – Australian Council for Educational Research)

Presented to the Pacific Rim Objective Measurement Symposium 2009 (Hong Kong)

Background

The Assessment of Language Competence certificates (ALC) is an annual, international testing program developed by the Australian Council for Educational Research to test listening and reading comprehension of school age children. The tests are developed for three levels in Chinese, French, German, Indonesian, Italian and Japanese. There is a mixture of target language and English questions in the Level 2 and 3 listening tests. Some teachers raise this as a concern and believe that all questions should only be offered in English.

Aims

Is there any interaction between the item types in the L2 listening tests and gender or student L2 reading ability?

Sample

Data used were the student responses to the ALC 2008 listening and reading tests for Certificate 2. The data included about 17,000 students in six test languages.

Method

Students were grouped by gender and two reading-ability levels (by their L2 reading scores). DIF analysis (Rasch model base) was implemented to explore the variation of the item difficulty estimates by gender and reading ability levels in each of the test languages.

Results

Main findings from this study show that the data fit well to the Rasch model. Neither gender nor L2 reading ability give a significant effect on the question types. No particular favouring direction has been detected for the questions in the target language.

Conclusions

L2 listening tests for native English-speaking students can include both questions in English and in the target language.