OECD Programme for International Student Assessment (PISA)

What do PISA participants do?

Cognitive Assessment Booklets

Participating students each complete a two-hour assessment booklet and a 30-to 40-minute questionnaire. In all, there are usually 13 different assessment booklets[1] that contain questions about one or more of the domains being tested[2]. This results in a total of about seven hours of assessment items rotated through the boolets.

The assessment booklets are assembled according to a complex design so that each booklet is linked through common items to other booklets in a balanced way. All booklets contain items from the major assessment (i.e., in PISA 2009, all booklets contained reading literacy items, in PISA 2012 all booklets will contain mathematical literacy items), with a rotation system ensuring that items from the two minor assessment areas appear equally throughout the 13 booklets. In this way, a broader range of tasks can be undertaken and can be linked to other items using Item Response Theory. This means also that the administration of the test is enhanced because students are unlikely to be doing the same booklet as students around them.

There are five types of question format: multiple choice; complex multiple choice; closed constructed response; open constructed response and short response. In some cases, students select their response from a list or provide a short written response and in other cases students have to write extended answers.

The return of reading literacy as a major assessment in PISA 2009, and the further development of the reading literacy framework, which recognises the ever-increasing use of electronic texts, has provided an avenue to assess the reading of electronic texts. Eighteen countries, including Australia, participated in this optional assessment. A sub-sample of students who participated in the pencil-and-paper assessment also completed an electronic assessment using the IT infrastructure at schools.

Context Questionnaires

The data collected in the Student Questionnaire provides an opportunity to investigate factors that may influence performance and consequently provide further meaning to the achievement scores. The internationally standard Student Questionnaire has sought similar information across the PISA cycles. Data has been collected about students’ family background (including socioeconomic status), aspects of learning (including engagement and approaches to learning), aspects of instruction, and the context of instruction (including instructional time and class size) in the context in the major literacy domain being assessed. Students have also been asked to report on their beliefs and confidence about learning and aspirations in previous PISA cycles.

Countries also have the opportunity to examine specific topics of interest, through international options. Familiarity with Information and Communication Technology (ICT) and Educational Career Paths are two questionnaires that Australia has incorporated into the questionnaire, which have been administered in all PISA cycles.

Information at school-level is collected through a 30-minute School Questionnaire, answered by the principal (or the principal’s designate). The questionnaire seeks descriptive information about the school and information about instructional practices. For example, questions were asked about qualifications of teachers and numbers of staff, teacher morale, school and teacher autonomy, school resources, and school policies and practices such as use of student assessments. Although previous PISA cycles have collected school-level data through a pencil-and-paper questionnaire, in PISA 2009, the school questionnaire was completed online.

[1] This was the case for PISA 2003, PISA 2006 and PISA 2009, whereas there were only 9 assessment booklets in PISA 2000.

[2] PISA 2003 also included a one-off assessment of cross curricular problem solving skills.