OP-ED: Evidence-based Practices Aren’t the Only Tool in the Shed

Over the last 15 years, juvenile justice advocates fought hard to convince policymakers and government officials that the best way to help youth succeed and improve public safety is to keep them out of secure confinement. To keep youth out of confinement, we argued, we should place youth in the community and enroll them in evidence-based practices (EBPs) close to home.

Robust evaluations of these programs, like Multi-Systemic Therapy (MST) and Functional Family Therapy (FFT), showed that they are more cost-effective than locking up young people. Blue Prints for Healthy Youth Development and the Washington State Institute for Public Policy made the evaluations of these programs easy to understand and compare to incarceration.

Policymakers and government officials seem to have bought what we were selling: As a former staffer at a juvenile justice agency, I had to help a council member understand why every youth couldn’t and shouldn’t be in an MST program. These policymakers wanted the “gold standard” EBPs, but not every young person qualified to participate in MST.

What’s more, MST and other evidence-based programs couldn’t meet the needs of every young person in the system. Perhaps it’s our own doing that we now hear policymakers, government officials and an array of stakeholders beating the drum to implement evidence-based practices, even at the expense of other promising and innovative services, interventions and programs.

Although they can well serve youth with specific needs, evidence-based practices and programs aren’t perfect, and they aren’t the only programs that can meet a young person’s needs. They can also be very costly to implement and evaluate, demand strict fidelity to the original model and only work for the type of youth for which they were designed.

In other words, they are not meant to be tinkered with to meet the unique needs of a young person, their family, their community or the agency implementing them. This can leave out a lot of young people, perhaps the same young people who still end up in secure confinement or who have been transferred to the adult system.

The problem isn’t simply that EBPs can’t and don’t serve all youth — the problem is, in part, that we think they should. The development and promotion of these programs have been, undoubtedly, an invaluable asset for youth and the field. But our zealous advocacy for EBPs may have resulted in the devaluation of other crucial, flexible approaches to serving the large numbers of youth whose needs are not fully met by EBPs.

EBPs can have a clinical rather than individualized approach. While therapy and medications are clearly helpful, when used in isolation they can have a pathologizing effect that makes youth feel less like an asset and more like damaged goods. Treating youth like an asset is a central feature of positive youth development, an approach widely accepted, but not necessarily evidence-based.

To get around the challenges posed by the restrictiveness of EBPs, especially related to customizing services, youth-serving agencies and organizations have adopted or created programs and practices that are designed to meet the unique needs of young people in the context of their communities, ideally leveraging community resources. When implemented well, these programs are informed by the best research, practices and data about what works to help kids and are designed to meet the unique needs of the agencies that decide to use them too. They are flexible and can be adjusted quickly to meet public safety, political, social or budgetary needs.

Such programs are usually based on fundamental principles of positive youth development, like giving a young person the opportunity to learn and experience new things, develop positive relationships and a sense of belonging, and be treated like an asset. Programs like educational services, counseling, mentoring and job training, among many others, are all widely understood to provide the ingredients for success.

For cash-strapped jurisdictions, programs or policies that are not dubbed evidence-based but use evidence-based principles are a boon. Rather than use a strict, academic evaluation procedure, they seek to use a highly calibrated and robust system of oversight to make sure youth are getting high-quality services that meet their unique needs and can produce positive outcomes. These approaches can help provide the supports and services that help youth succeed in the community, making an out-of-home placement unnecessary. Here are some examples:

• Positive Youth Justice (PYJ) is an approach that draws from positive youth development principles to meet the unique needs of justice-involved youth. PYJ includes six domains: education, work, relationships, creativity, community and health. These domains are not meant to be used in isolation, but rather as a system of supports and services that can include EBPs.
• DC YouthLink in the District of Columbia provides supports and services to young people placed in the community along the five PYJ domains. DC YouthLink relies on a number of programs and practices with evidence-based principles, like mentoring for example. It also uses a robust system of quality assurance to ensure that young people are getting what they need to succeed, which includes going beyond recidivism measures to include positive outcomes like high school graduation, job attainment and positive attachments.
• Wayne County, Michigan has a system of community-based programs and services that are not all evidence-based but still exposed to a system of quality checks. By building funding and service coordination of juvenile justice services into a Care Management Organization (CMO) model, the county and the nonprofit collaborators can tailor the right kind of services to the youth and their family in the context of their community and collect data to track to ensure quality service delivery.
• The Standard Program Evaluation Protocol is a rubric designed by Mark Lipsey and his colleagues at Vanderbilt University to determine the cost-effectiveness of community-based programs. It isn’t meant to be an off-the-shelf guide to evaluation and, for now, it focuses solely on recidivism as the outcome. As a tool, it helps systems look at their whole youth population and determine the most cost-effective interventions that can be effective for youth with different risks and needs, giving the system the ability to ensure that their interventions are working without investing in a traditional evaluation.

Now that we’re a decade out from many of the larger reform efforts, it’s easier to see that evidence-based practices aren’t and shouldn’t be the only tool in the shed. Just as the field helped to advance the credibility of EBPs, so do we also need to call for other types of community-based programs rooted in the knowledge of what works to help kids.

With tens of thousands of youth still locked up — the one intervention that is clearly not evidence-based — we should have the courage, vision and ingenuity to provide customized supports and services to youth in the community rather than relying on one type of intervention.

A hammer is very effective, but only if everything is, in fact, a nail.

Amanda Petteruti is Senior Research Associate at the Justice Policy Institute and former Program Analyst for the District of Columbia’s Department of Youth Rehabilitation Services.