Generally, compared to traditional non-restorative approaches, restorative justice was found to be more successful at achieving each of its four major goals. In other words, based on the findings of this meta-analysis, restorative justice programs are a more effective method of improving victim-offender satisfaction, increasing offender compliance with restitution, and decreasing the recidivism of offenders when compared to more traditional criminal justice responses (i.e. incarceration, probation, court-ordered restitution). In fact, restorative programs were significantly more effective than these approaches in all four outcomes (with the exclusion of the offender satisfaction outlier).
The positive results of this meta-analysis are tempered, however, by the self-selection bias evident in controlled outcome studies on restorative justice programs. Restorative justice, by its very nature, is a voluntary process. This creates a treatment group of participants (both offenders and victims) who have chosen to participate in the program and may therefore be more motivated than the control group. This concern is elevated by the high rate of attrition within many of the studies in this meta-analysis. McCold and Wachtel (1998), for example, found clear differences in the recidivism rates of restorative justice participants (20%) versus individuals who refused participation in the program (48%) versus the comparison group (35%). In fact, these authors argued that there was no treatment effect on recidivism from participation in restorative justice beyond a self-selection effect.
This is an inherent problem in restorative justice research. It is not possible to truly randomly assign participants to treatment and control conditions. Once an individual is forced to participate in a restorative justice program, most would argue that the program is no longer truly restorative. Given this, we believe that an alternative method of determining the effectiveness of restorative justice is necessary. We recommend administering questionnaires designed to measure participants’ motivation prior to program participation. This would allow researchers to examine the motivation of the control group, restorative justice participants and those who refused participation. A research design such as this would provide a comparison of highly motivated, moderately motivated and unmotivated individuals in each group. If the satisfaction or recidivism rates, for example, were improved in the restorative justice group, and motivation was controlled for in the analysis, we would be more convinced that there is a treatment effect from participation in restorative justice processes.
Notwithstanding the issue of self-selection bias, the results of this meta-analysis, at present, represent the best indicator of the effectiveness of restorative justice practices (i.e. those individuals who choose to participate in restorative justice programs find the process satisfying, tend to display lower recidivism rates and are more likely to adhere to restitution agreements).
While the effects of restorative justice participation on recidivism remains somewhat uncertain due to the self-selection bias, many argue that it is naïve to believe that a time-limited intervention such as a VOM will have a dramatic effect on altering criminal and delinquent behaviour (Umbreit, 1994b). Additional factors, such as anti-social peers, substance abuse and criminogenic communities, which have been linked to criminal behaviour (Hawkins et al., 1998; Lipsey & Derzon, 1998), are not adequately addressed in the restorative process. Andrews and Bonta (1998) have also identified several criminogenic needs that they maintain are imperative to address in the treatment of offenders in order to reduce recidivism effectively. Generally, they identified anti-social attitudes, pro-criminal associates, personality factors, family factors and low levels of educational and employment attainment. In fact, previous meta-analytic work conducted by Dowden (1998) and Andrews et al. (1990) found that “appropriate” correctional treatment (i.e. those programs that adhered to the clinically relevant principles of risk, need and responsivity) displayed an appreciably higher mean effect size (+.26 and +.30, respectively) for recidivism compared to the findings for restorative justice programs (+.07) presented here. In other words, although restorative justice programs may yield reductions in recidivism compared to more traditional criminal justice responses to crime, they did not have nearly as strong an impact on re-offending as psychologically informed treatment.
It has been argued, however, that restorative justice and rehabilitative treatment are rather complementary approaches (Crowe, 1998). The utilization, therefore, of both restorative justice processes and “appropriate treatment” as a comprehensive response to criminal behaviour would be a valuable and theoretically directed experiment. This combination would enable both approaches to capitalize on their strengths and minimize their weaknesses. More specifically, the restorative processes could increase victim-offender satisfaction and restitution compliance while the rehabilitative processes could have a significant impact on recidivism.
Although we did not discover any significant differences in reported outcomes based on model type (i.e.VOM versus conferencing), VOM models did have higher victim satisfaction ratings and offender satisfaction ratings (excluding the outlier). The low number of effect sizes, however, coupled with the lack of a significant difference, does not allow for a firm conclusion. Theoretically, the large number of participants in a conference compared to a VOM could be contributing to the reported lower victim-offender satisfaction ratings. There is concern in the restorative justice literature that coming to a satisfactory agreement in a session is sometimes of concern, particularly when there are numerous individuals from various backgrounds participating in the process (Hooper & Busch, 1996). This possibility is certainly increased in the case of conferences.
Interestingly, we did not discover differences among programs that operate along different entry points to the criminal justice system, with the exception of offender satisfaction. This difference is not convincing, however, because of the outlier issue and the fact that the entry points were difficult to code (more than half of the programs had mixed entry points). Thus, we recommend that researchers more explicitly code the entry point for their program and/or separate the analyses by this variable.
There were several questions that we were unable to answer due to a lack of data reported in the literature. For example, we were interested in exploring whether the characteristics of the facilitator had a significant moderating impact on restorative justice program effectiveness. However, very few, if any of the studies, provided information concerning the education, professional background or training of the facilitators. This is particularly noteworthy as facilitators within restorative justice programs can have a significant impact on the outcome of a session. Support for this assertion may be found within the correctional treatment literature where program staff characteristics and behaviours have been found to have a significant impact on program effectiveness (Dowden & Andrews, under review). There was also rather limited data on additional important variables, such as the criminal history of the offenders (i.e. first-time offenders versus repeat offenders), the specific offences (i.e. minor versus serious offences, property versus violent offences), and the relationship between offenders and victims (i.e. family, neighbour, stranger).
In general, we were unable to provide an adequate explanation for the large range of reported effect sizes in each of the outcomes. It is possible that the significant factors in determining a more successful restorative justice program are those that were not reported in the literature (i.e. facilitator characteristics, offence types, criminal history). To facilitate a better understanding of the effectiveness of restorative justice, we recommend that future studies report outcomes, such as recidivism or satisfaction, separately for groups of offenders using such variables as gender, age, criminal history, offence types and relationship between victim and offender. In addition, we recommend that studies provide more detailed information on the processes used within the restorative justice programs and the facilitators.
Another issue that future studies may wish to explore is the effect that offender compliance with restitution agreements has on victim satisfaction. The restrictions of meta-analytic procedures preclude such an analysis. Morris and Maxwell (1998), however, did report that the reason most frequently reported for victim dissatisfaction in an evaluation of a family group conference program in New Zealand was a failure to receive the appropriate restitution. More empirical research into the restitution conditions (i.e. type of restitution, size of restitution, length of time given to comply) that lead to successful compliance would also be appropriate. Moreover, the same type of analysis could be completed on restitution conditions and victim-offender satisfaction. And finally, there is no research in the literature that examines the longer term effects for victims who participate in a restorative justice process. An examination of whether victims still feel that they have experienced some closure and healing six months or a year after the restorative process would be beneficial.
 For a detailed description of the principles of risk, need and responsivity and their importance in effective correctional treatment, see Andrews and Bonta (1998).