Social Workers in Schools evaluations: A use case for IDI but we can do more

A study co-authored by CSDA researchers has identified positive effects for some student groups from the government-funded Social Workers in Schools (SWiS) programme. The paper is the first of two studies recently published by the Ministry of Social Development.

Rhema Vaithianathan, who co-authored the first paper says the studies confirm that impact evaluations are a ‘use case’ for linked data in New Zealand, but there is further to go if we want to realise the potential of the Integrated Data Infrastructure (IDI).

“We are really only starting to realise the strengths and weaknesses of linked data to tell us about program effectiveness. Studies like these ones, finding impact for some groups, confirm that impact evaluation is a use case. However, they also show the limitations when we cannot individually track those who benefited from the program. Here, the results tell us about the positive impact on children who attended schools that offered SWiS, but they do not tell us about the impact on children who actually used the service because we did not have individual client level data (ICLD). In the future, if we can add anonymous ICLD to the research data set, we are likely to be able to deliver much stronger conclusions about program impact. This extension would be the natural next step to how we use the IDI, and would better enable researchers to show that programs are making a real difference.”

The first SWiS paper (“The Impact of Social Workers in Schools: A preliminary investigation using linked administrative data”, February 2017) was co-authored by Nan Jiang, Tim Maloney, Anita Staneva and Rhema Vaithianathan (AUT) and Moira Wilson (MSD).

Using the Integrated Child Dataset (ICD), a precursor to the full Integrated Data Infrastructure (IDI), the researchers said their results were “consistent with SWiS having had positive effects on children’s transition to high school”, with impacts including:

A reduction in the number of days of ‘non-enrolment’ in high school;
An improvement in the proportion of girls with NCEA Level 1 by the end of the year in which they turned 16; and
Reductions in CYF youth justice referrals in the first three years at high school.

In the second paper (“Estimating the impact of Social Workers in Schools using linked administrative data”, February 2018) a second group of researchers applied the methodology devised for the first study to a later expansion of SWiS, and used the newly available IDI. The second study found that, while SWiS did not reduce the rate of three additional outcomes of interest (stand downs/suspensions, CYF notifications and Police apprehensions), it did have the following effects for particular subgroups:

Maori boys enrolled in SWiS base schools showed a drop in Police apprehensions; and
Pacific students in SWiS schools had lower rates of care and protection notifications to CYF.

Contact:

Phone:

Email: