Evaluating the Approach
How do we know if our efforts are working? In medicine, science, technology, and other fields, strategies and techniques are constantly evolving as knowledge derived from research leads to new advances. Criminal justice must take the same approach.
Through IPS, prosecutors are teaming up with researchers to determine the effectiveness of their programs. Some prosecutor’s offices employ in-house researchers or have long-standing relationships with local universities and are ready to hit the ground running with data collection and analysis. Other sites navigate the process of establishing new partnerships with researchers — and find potential long-term collaborators in the process.
A researcher’s work doesn’t start at the end of the project. Researchers may assist prosecutors with project development: they can help develop an action plan, which serves as a concrete tool to solidify project objectives, elucidate activities, and establish a timeline to keep the project on track. They can also provide data at the outset of a project, helping define geographical areas to target interventions, or provide data in a constant feedback loop, helping practitioners shift strategies as they go.
Of course, the primary job of the researcher is to assess how the criminal justice program is working — typically referred to as a process evaluation — and to determine whether project strategies have led to their desired effect, i.e., an outcome evaluation. Learn more about their ongoing efforts below.
Shaping Project Strategies
Through IPS, the Cook County State’s Attorney’s Office (SAO) in Chicago, IL, set out to reduce shootings and other gun crimes. As a first step, the team conducted a hotspot analysis to identify six areas with the highest rates of violent crime in the city. Based on this information, the SAO embedded prosecutors within police departments in the six hotspot precincts. These prosecutors, members of the newly-created Gun Crime Strategies Unit (GCSU), work closely with law enforcement to build intelligence on individuals who disproportionately commit violent crime. The IPS team is also testing an early alert system that immediately notifies the SAO when a high-risk violent offender is arrested. The system enables prosecutors to quickly gather information on offenders prior to their bond hearings, which occur a mere 48 hours after arrest.
The SAO’s research partners at the University of Chicago Crime Lab are now assessing the effects of real-time notifications and close police-prosecutor collaboration. While the Crime Lab is still conducting its final evaluation, preliminary findings have convinced the SAO to sustain the project beyond the IPS grant. Between 2016 and 2018, arrests for enhanced gun offenses quadrupled in one precinct and nearly tripled in another, while gun violence declined faster in the GCSU districts than in the city as a whole.
The District Attorney’s (DA) Office in Chatham County, GA, which covers the greater Savannah metro area, also focused on reducing gun violence. As with Cook County, their research partner — at Georgia Southern University (GSU) — worked closely with the DA’s Office and the Savannah Police Department, analyzing data to identify violent crime hotspots. The office focused on one of these neighborhoods, West Savannah, and hired a dedicated gun violence prosecutor to focus investigative and prosecution efforts against offenders driving crime in that area.
GSU researchers measured the impact of hotspot prosecution by comparing crime rates in West Savannah to two other neighborhoods with similar levels of violence. What they found is that gun-related crime initially increased in the targeted hotspot, and then steadily decreased after that spike for the duration of the grant. It remains unclear if this pattern was due to the effectiveness of the intervention, or a reflection of increased enforcement actions that resulted in a higher number of arrests and prosecutions.
A separate component of the project involved implementing a youth diversion program that offered mentorship and skill-building opportunities to low-level offenders at risk of recidivism. To evaluate the program, GSU researchers used a mixed methods approach, which included fielding a community survey of residents who live in the targeted areas, and a survey of participants and staff. Overall, participants reported that program staff served as positive role models and that they had more hopeful attitudes regarding their futures.
Evaluating Program Processes
For its IPS project, the Circuit Attorney’s Office in St. Louis, MO, created a Young Adult Offender Program (YOP) for nonviolent felony offenders ages 17 to 25. Based on each participant's needs and goals, the YOP team customizes a plan that might include therapy, mentorship, substance abuse counseling, and budgeting, housing, education, employment, and parenting support. Participants can also be connected to health insurance and medical services through CareSTL Health, a comprehensive healthcare center. Once participants successfully complete the program, their guilty pleas are withdrawn and charges dismissed.
Research partners at Saint Louis University and Arizona State University collected and analyzed robust data to assess YOP participants’ progression through the program’s four phases. They tracked participants’ sociodemographic information, their engagement with social and health services, sanctions, and metrics pertaining to participants’ self-worth and self-efficacy. Of the 18 individuals who had been accepted into YOP as of May 30, 2019, two-thirds were active participants or had graduated from the program, while one-third were inactive due to dropping out of the program or receiving new felony charges. Of the 5 participants who completed surveys, two reported an increase in their self-efficacy, while three reported gaining valuable resources from their experience in YOP.
Prosecutors in Nashville, TN also used IPS funds to offer an alternative to incarceration — a human trafficking intervention court. Known as Cherished HEARTS (Healing Enslaved and Repressed Trafficking Survivors), the program set out to provide wraparound, trauma-informed services to survivors of human trafficking who are involved with the criminal justice system in Nashville. An independent researcher is conducting both a process and an outcome evaluation to assess how well the Cherished HEARTS program is addressing the needs of trafficking survivors. The researcher is currently tracking how the program identifies justice-involved victims and the barriers they face in accessing those services, as well as connecting victims with services. As of the final quarter of the grant period, 27 individuals had been admitted into the program and 6 had graduated. Cherished HEARTS is experiencing an increase in demand, and staff have expressed an interest in expanding the program to serve the needs of more survivors in the greater Nashville area.
Measuring Outcomes
The prosecutor’s office in Essex County, NJ (ECPO) aimed to enhance the investigation and prosecution of violent offenders through intensified scrutiny of social media evidence. It created a Special Prosecution Unit to systematically review social media sources for particular cases. To evaluate whether the effort improved criminal justice outcomes, researchers from the Center on Policing at Rutgers University compared cases in the treatment group, all of which fell under the specialized unit’s purview, against cases in three control groups, which were not subject to enhanced social media review.
Useful social media evidence was found in twice as many cases in the treatment group as in the control group. And by the end of the grant, there were more guilty pleas and guilty verdicts at trial in the treatment group than the control group — a small but potentially significant finding. The project also demonstrated how social media investigations can improve public safety beyond case dispositions, including through identification of suspects and corroboration of evidence, increased charges and/or pleas, successful arguments for the detention of dangerous offenders, and prevention of gun violence at local schools. Based on these preliminary findings, the ECPO made the Special Prosecution Unit a permanent fixture of the office.
In Detroit, MI, the Wayne County Prosecutor’s Office set out to reduce violent crime in the city’s 9th district by setting up a one-person grand jury and creating a dedicated team that responds to all nonfatal shootings. To assess whether the IPS team’s strategies have affected crime and case outcomes, researchers from Michigan State University are comparing dynamic trends in gun violence and case closures in the 9th precinct with those elsewhere in the city. Since the one-person grand jury was put on hold due to the COVID-19 pandemic, MSU will account for the pandemic’s effect on crime response. The researchers are also documenting the ways in which the one-person grand jury may streamline cases — for example, by enabling the investigators and prosecutors to gather evidence and make charging decisions more quickly.
Performance Management
The District Attorney’s Office in Cobb County, GA wants to better understand the criminal justice response to sexual violence. How do police and prosecutors make decisions? How can you measure results beyond conviction rates? And what are the challenges to achieving justice in these cases? Building on work done through the Sexual Assault Justice Initiative, a team of prosecutors, advocates, and police officers are entering information from hundreds of sexual violence case files into a shared database. Kennesaw State University researchers will then analyze the data to better understand why victims don’t report their assaults, which factors contribute to case attrition, and how complex cases are resolved in the criminal justice system. Armed with these insights, the multidisciplinary team will strategize how to enhance victim experience, prosecution practices, and case outcomes.
The State’s Attorney’s Office (SAO) for the Fourth Judicial District in Jacksonville, FL, is enhancing its data management system both to reduce gang violence and to evaluate prosecution decision-making. An IT specialist is building a new infrastructure, while a data analyst/evaluation specialist is developing methods to share data with other jurisdictions to enhance intelligence and increase cooperation between law enforcement agencies. Researchers from Florida International University (FIU) will regularly train prosecutors to leverage the SAO’s data trends—for example, diversion program completion rates for various offenses—so they can think beyond individual cases to make better prosecution decisions. The new system will inform the office’s Prosecutorial Performance Indicators, which are being developed in partnership with the FIU researchers and with funding from the MacArthur Foundation. By evaluating prosecutorial efficiency, effectiveness, and fairness on a regular basis, the SAO will be able to strategize on how to enhance its practice.
Resources
Model Response to Sexual Violence for Prosecutors (RSVP Model): Measuring the Impact (PDF)
The RSVP Model is a step-by-step guide for implementing a performance management (PM) system to better understand and improve the prosecution response to sexual violence. The principles, strategies, and tools contained within the guide are beneficial to prosecutors interested in measuring and improving performance in responding to any crime.
Center for Research Partnerships and Program Evaluation (website)
The U.S. Department of Justice Bureau of Justice Assistance's Center for Research Partnerships and Program Evaluation provides visitors with information on practitioner-researcher partnerships, performance measures, logic models, program evaluation, and other important information to devise solutions to improve the quality, quantity, and equity of criminal justice and public safety services.
Innovative Prosecution Solutions Action Plan (PDF)
This template, developed by the IPS TTA partners, is frequently used by IPS grantees to plan project strategies and conceptualize outputs and outcomes.
Related Case Studies
