Allegheny Family Screening Tool
Researchers led by Rhema Vaithianathan modelled, designed and supported implementation of this world-first child welfare predictive analytics tool. The Allegheny Family Screening Tool (AFST) uses rich administrative data to generate a screening score for incoming calls alleging child maltreatment and neglect. The score is an additional piece of information that helps call screeners as they decide whether to open an investigation. Allegheny County introduced this decision support tool with the aim of improving accuracy and consistency of call screening decisions. An independent impact evaluation of the tool was completed by researchers at Stanford University in March 2019. The evaluators’ findings included that use of the tool improved the accurate identification of children in need of services and was associated with a modest reduction in racial disparities in case openings.
Cuccaro-Alamin, S., Foust, R., Vaithianathan, R., & Putnam-Hornstein, E. (2017). Risk assessment and decision making in child protective services: Predictive risk modeling in context. Children and Youth Services Review, 79, 291-298.
The authors, including Rhema Vaithianathan, review the literature and provide a context for predictive risk modelling in the current risk assessment paradigm in child protective services. They describe how predictive analytics or predictive risk modelling using linked administrative data may provide a useful complement to current approaches.
Vaithianathan, R., Jiang, N., Maloney, T., Nand, P., & Putnam-Hornstein, E. (2017). Developing predictive risk models to support child maltreatment hotline screening decisions: Allegheny County methodology and implementation. Auckland: Centre for Social Data Analytics.
This methodology document traces the development and implementation of the Allegheny Family Screening Tool (AFST), a child welfare predictive risk-modelling tool built by a research team led by Rhema Vaithianathan. The AFST is intended to support call screening of child maltreatment referrals by Allegheny County Department of Human Services. The methodology document includes an overview of existing practice, methodology and performance metrics of the model and details of the external validation and next steps for the model rebuild.
Chouldechova, A., Putnam-Hornstein, E., Benavides-Prado, D., Fialko, O., & Vaithianathan, R. (2018). A case study of algorithm-assisted decision making in child maltreatment hotline screening decisions. Proceedings of Machine Learning Research, 1-15.
Diana Bendavides-Prado, Rhema Vaithianathan, Oleksandr Fialko (CSDA research fellow 2017-2018), Alexandra Chouldechova and Emily Putnam-Hornstein describe their work on developing, validating, fairness auditing, and deploying a risk prediction model in Allegheny County, PA, USA. They discuss the results of their analysis to-date and highlight critical problems and data bias issues that present challenges for model evaluation and deployment.
Vaithianathan, R., Kulick, E, Putnam-Hornstein, E & Benavides Prado, D. (2019). Allegheny Family Screening Tool: Methodology, Version 2. Centre for Social Data Analytics. Auckland: Centre for Social Data Analytics.
This methodology report describes changes to the Allegheny Family Screening Tool (AFST), building upon and updating the original methodology report, Developing Predictive Risk Models to Support Child Maltreatment Hotline Screening Decisions (March 2017). Modifications implemented include changes to specific predictor fields used in the model itself, the modelling methodology, and County policies concerning the tool’s use,upholding Allegheny County’s ongoing commitment to transparency by continuing to inform the community about changes to the tool and the County’s policies.
Vaithianathan R, Putnam-Hornstein E, Chouldechova A, Benavides-Prado D, Berger R. Hospital Injury Encounters of Children Identified by a Predictive Risk Model for Screening Child Maltreatment Referrals: Evidence From the Allegheny Family Screening Tool. JAMA Pediatr. Published online August 03, 2020. doi:10.1001/jamapediatrics.2020.2770
This paper confirms that children identified as at risk by the Allegheny Family Screening Tool (AFST), a predictive risk modelling tool that supports child protection decisions in Allegheny County (PA, United States), are also at considerably heightened risk of hospitalisation injury. The authors, including Rhema Vaithianathan and Diana Benavides Prado, used a one-off linkage between the child protection system and Pittsburgh Children’s Hospital data to show that children identified as at risk by the AFST are indeed at considerably heightened risk of injury, abuse, and self-harm hospitalisation. The authors also identified a sharply heightened hospitalisation risk for children receiving the highest AFST scores and found that the AFST is particularly sensitive to the risk facing White children, which means the child protection system is potentially under protecting those children.
Key AFST documents including Methodology and Methodology V2, Ethics Report, Frequently Asked Questions and Impact Evaluation (March 2019) along with press coverage are available on the Allegheny Analytics site.
- Child Protection and the Promise of Predictive Analytics, July 7, 2017
- CSDA takes Danish experts behind the scenes of US projects, November 28, 2017
- “What is predictive risk modeling and how can it be applied to child welfare?”, Casey Family Programs Child Safety Convening, May 2018
- Bias Detectives: the researchers striving to make algorithms fair" - CSDA’s work in Nature news, July 2018
- Can an algorithm keep kids safe? Innovation Hub, June 2018
- Analysis of screening tool wins best paper award at international fairness conference, February 15, 2018
- Can an Algorithm Tell When Kids are in Danger?" - our research in the NYT, January 12, 2018
- Allegheny Family Screening Tool evaluation: Improved decision accuracy, reduced disparities, May 3, 2019
Location: Allegheny County – PA, United States
Partner/s: Children’s Data Network (University of Southern California), Carnegie Mellon University, Allegheny County Department of Human Services