A recently released Associated Press review identified a number of concerns about the technology welfare agencies use, including that it could harden racial disparities in the child welfare system.
Attorney Robin Frank is challenging the algorithm that decides which families should be investigated for alleged child neglect, AP reports. The algorithm is similar to the one in Allegheny County, Pennsylvania, which was found
to have a pattern of flagging more Black children for a “mandatory” neglect investigation compared with white children.“A lot of people don’t know that it’s even being used,” Frank said. “Families should have the right to have all of the information in their file.”
Researchers also found that social workers in Allegheny also disagreed with the risk scores the algorithm produced about one-third of the time.
“Workers, whoever they are, shouldn’t be asked to make, in a given year, 14, 15, 16,000 of these kinds of decisions with incredibly imperfect information,” said Erin Dalton, director of the county’s Department of Human Services, and a pioneer in applying the predictive child welfare algorithm.
Typically, incidents of potential neglect are reported to Allegheny County’s child protection hotline before the algorithm calculates the child’s potential risk and assigns a score through an automatic screening process. Social workers are then able to decide whether to investigate.
Critics of the artificial intelligence-run algorithm say it gives the data-driven program mostly collected using the information on poor people an unfair disadvantage in deciding the fate of families in poverty-stricken areas.
Researchers found that when the algorithm determined which families should be screened, it would have recommended two-thirds of Black children to be investigated, compared with about half of all other children reported.
“It’s not decreasing the impact among Black families,” said Logan Stapleton, a researcher at Carnegie Mellon University. “On the point of accuracy and disparity, (the county is) making strong statements that I think are misleading.”