External Evaluation

Sierra Health Foundation launched the Positive Youth Justice Initiative (PYJI) in 2012 with the goal of improving the lives of young people involved with the juvenile justice system, with an initial focus on crossover youth – young people who have experienced documented neglect, abuse and/or trauma, have a history in the child welfare and foster care system, and who currently are engaged in the juvenile justice system. The initiative is supporting California counties in designing and implementing a series of systems reforms through an integrated model that invests in youth, treats trauma, provides wraparound service delivery and strengthens local infrastructure.

An independent, external evaluation team from Resource Development Associates (RDA) was selected to evaluate the initiative’s implementation, addressing the following questions:

  1. What successes and challenges do counties experience in implementing PYJI?
  2. What systems changes occur as a result of PYJI?
  3. To what extent, if any, do the systems changes affect the experiences of youth and their caregivers with the juvenile justice system?

Results in Brief

In January 2016, RDA released a cumulative report examining PYJI implementation over its first two years. Recognizing that systems change is a multi-year process that requires the commitment of many county and community-based stakeholders, the evaluation nonetheless identified several areas of notable progress, including creating or modifying policies and procedures that institutionalize aspects of the PYJI model, developing and implementing tools to support data-driven decision making, bringing on new partners to support collaboration and implementing trauma-informed training, among other achievements.

At the same time, counties experienced some challenges in the systems reform process. These included competing priorities and time constraints, reliance on executive leadership to advance reforms, resistance to change among mid-level and line staff, bureaucratic processes to change policies and procedures, capacity to consistently hold team-based decision making meetings and barriers to cross system data sharing.

While the implementation of PYJI has led to changes that could eventually impact youth, results from youth and caregiver surveys and youth focus groups indicated that it will take more time for youth and their families to feel the impacts in a consistent way. A majority of youth and caregiver respondents reported that they believe their probation officers and teachers want things to go well for them. At the same time, most youth participating in focus groups reported that the quality of their relationships with probation officers, schools and service providers depend largely on individual staff people, rather than on a system-wide approach.

Overall, the external evaluation found that PYJI has encouraged systems and staff to commit to new and ongoing reforms, while also recognizing that it takes time to overhaul traditional practices, shift culture among staff, and for youth to experience the results of those changes.