An ACER evaluation of the assessment and rating process that measures day care, preschool and outside school hours care services against national quality standards has found that the process is valid and reliable.

Commissioned by the Commonwealth Department of Education, Employment and Workplace Relations in June 2012, the ACER evaluation concentrated on the validity and reliability of the National Quality Standard for Early Childhood Education and Care and School Age Care (NQS) assessment and rating process and the instrumentation used.

A total of 491 services were assessed by authorised officers in all states and territories as part of the evaluation, of which 224 were long day care centres, 137 were preschools, 81 were outside school hours care centres and 49 were family day care services. The services rated are not necessarily representative of the entire population.

The assessment and rating process involves authorised officers visiting each service to determine the extent to which the service meets seven quality areas of the National Quality Standard:

1. Educational program and practice
2. Children’s health and safety
3. Physical environment
4. Staffing arrangements
5. Relationships with children
6. Collaborative partnerships with families and communities
7. Leadership and service management

Across all 491 services assessed, the quality area of Educational program and practice had the lowest percentages of services that received ratings of ‘exceeding’ the NQS (24 per cent) and ‘meeting’ the NQS (33 per cent). Consequently, Educational program and practice had the highest percentage of services rated as ‘working towards’ the NQS (43 per cent). Staffing arrangements had the lowest percentage of services rated as working towards the NQS (12 per cent), and Relationships with children had the highest percentage of services rated as exceeding the NQS (39 per cent).

Based on their quality area ratings, each service then receives an overall rating. Of the 491 services included in the evaluation, almost half (48 per cent) were rated as meeting or exceeding the NQS. Only two services – one preschool and one family day care – were rated as ‘significant improvement required’, representing 0.4 per cent of all services assessed and rated as part of the evaluation.

Results varied by service type. For long day care services, the proportion of services exceeding, meeting and working towards the NQS were 17 per cent, 23 per cent and 60 per cent respectively; for preschools the proportions were 50 per cent, 24 per cent and 25 per cent; for family day care services, 27 per cent, 21 per cent and 52 per cent; and for outside school hours care, 12 per cent, 15 per cent and 73 per cent.

Because of differences in the number of services assessed, the evaluation was unable to determine why there are differences between the ratings achieved by the service types. There was, however, no evidence of bias against any service type or location, nor any bias as a result of variations in the judgements of authorised officers. Overall, the evaluation found the assessment process to be valid and reliable.

Questionnaires were used to obtain comments from assessed services and authorised officers relating to the assessment and rating process. More than 80 per cent of authorised officers who responded to the survey stated that they had no difficulty collecting adequate evidence to assign a rating at any level of the process.

Regardless of the outcome, services were generally satisfied with the assessment and rating process, with 81 per cent of services responding to the online survey reporting that their experience was positive, including more than one-third of these services calling it ‘very positive’.

Evaluation of the assessment and rating process under the National Quality Standard for Early Childhood Education and Care and School Age Care, by a team of ACER researchers led by Sheldon Rothman, is available from research.acer.edu.au/early_childhood_misc/8/

« Back to eNews