Evaluation of the Aboriginal Justice Strategy

Appendix D: Summary of the Recidivism and Costing Analyses

1. Introduction

As part of an evaluation of the Aboriginal Justice Strategy (AJS), the Department of Justice Canada (the Department) conducted a statistical analysis of the impact of the AJS-funded community-based justice programs on rates of re-offending. This study was previously undertaken by the Department in 2000 Footnote 53, 2007 Footnote 54 and 2011 Footnote 55. These studies were based on a comparison of recidivism rates for offenders who participated in one of the selected AJS programs to a comparison group of offenders who were referred but did not participate. To further the study, a comparative costing analysis was conducted to estimate the impact the AJS programs have had on reducing costs of the mainstream justice system (MJS). The analysis compared the costs of an offender participating in an AJS program versus going through the MJS court process, the only alternative for the majority of AJS program participants.

2. Methodology

2.1 Recidivism Study

The methodology developed for this study was intended to replicate the quasi-experimental approach taken for the previous recidivism studies. To that end, a total of 30 community-based justice programs with a sample size of 2,807 were included in the study. Survival analysis was the statistical approach used to model the likelihood of re-offending. This method is ideally suited to modeling the occurrence and timing of events Footnote 56, particularly in cases such as this study, in which a re-offending event does not occur for all individuals in the sample. This is known as “right-censored” data, as the observation period ends before the re-offending event takes place for some members of the sample. The standard survival regression procedure used for these types of analyses is the Cox Proportional Hazards Model Footnote 57, which was also used in the three previous recidivism studies.

2.2 Comparative Costing Analysis

The impact of reducing recidivism was also measured through a costing analysis. This analysis was undertaken by calculating the costs of AJS programs and the total program spending averaged over the recorded number of clients for each program for one fiscal year (2014-15). For comparative purposes, the costs of the MJS were also calculated for the same fiscal year. Recognizing that the main point of diversion from the MJS for participants of AJS programs is the courts, expenditures as reported in public accounts related to court administration, prosecutions and legal aid were used to determine the costs of the MJS.

2.3 Limitations and Mitigation Strategy

For the recidivism analysis, a main limitation was the lack of a true experimental design, as practical and ethical constraints precluded the random assignment of persons to participant and comparison groups. Pre-existing differences between the participant and comparison groups could lead to differential outcomes with respect to re-offending. However, to mitigate this limitation, a statistical approach Footnote 58 that could control underlying differences between the participant and comparison groups was utilized.

With regards to the costing analysis, although the full costs of the AJS were included in the analysis, only estimates of some court-based (administration of court services, prosecution and legal aid) costs were included through examining public accounts. Additional costs associated with the administration of justice (e.g., police, probation, custody) and those borne by the community were excluded from the analysis. This reduces the ability to understand the full impact of the AJS, although it provides an indication of potential cost savings for one point of contact with the MJS.

3. Recidivism Results

Table 1 below presents some of the key characteristics of the total sample, as well as of the AJS program participant group and the comparison group. In terms of offender characteristics, the majority of offenders in the total sample were men (53.6%), and offenders were most frequently (41.6%) between the ages of 18 and 24 at the time of Program completion, with a mean age of 30. With respect to offence history, prior to their referral to an AJS program, 91% of offenders had never been convicted of a drug offence, 72.5% had never been convicted of a violent offence, and 66.1% had never been convicted for a non-violent offence. The comparison group members, relative to program participants, were slightly less likely to have never been convicted for an offence prior to program referral, with a difference of 3.4% for drug offences, 5.2% for violent offences, and 3.4% for non-violent offences.

Table 1: Characteristics of Offenders in the AJS Program Sample (National Results)
Number of Prior Convictions (%) Program Participants
Comparison Group
Total Sample
0 90.0 93.4 91.0
1-5 9.1 6.3 8.3
6 or more 0.9 0.28 0.7
Mean 0.26 0.12 0.22
0 71.2 76.4 72.5
1-5 20.9 20.0 20.7
6 or more 7.9 3.6 6.8
Mean 1.3 0.73 1.18
0 65.3 68.7 66.1
1-5 23.0 24.3 23.3
6 or more 11.7 7.0 10.5
Mean 2.4 1.3 2.1
Years of Program Completion (%)
2004-2007 18.7 30.0 21.6
2008-2011 53.4 51.8 53.0
2012-2014 27.9 18.2 25.4
Age at Program Completion (%) Table note i
18-24 40.5 44.6 41.6
25-34 29.5 29.3 29.4
35-44 17.9 15.9 17.4
45 and over 12.0 10.2 11.6
Mean 30.0 29.0 30.0
Gender (%)
Male 55.5 48.2 53.6
Female 44.5 51.8 46.4
British Columbia 3.9 6.4 4.6
Manitoba 20.9 43.7 26.7
New Brunswick 8.1 7.0 7.8
Newfoundland and Labrador 0.14 0.0 0.1
Ontario 30.2 14.8 26.2
PEI 0.29 0.0 0.2
Saskatchewan 35.2 27.7 33.3
Yukon 1.0 0.4 1.0
Table note i

For the comparison group, age at program “completion” is deemed to be age at six months following referral to the program.

Return to table note i referrer

Though the data collected for this study considered offenders referred between 2004 and 2012, over half (53%) of offenders in the total sample completed (in the case of program participants) or would have completed (in the case of the comparison group) the AJS program between 2008 and 2011.

Table 2 demonstrates the estimated cumulative recidivism rates for program participants and the comparison group at various points in time after participation in a program. These estimates are based on the percentage of offenders who re-offended during the observation period, by program participation, and are fitted to represent the “average” offender in the total sample. The number of prior convictions and age were all fixed at the total sample means for each variable. For comparative purposes, the table includes the results from the previous two studies, which indicate that the current results are consistent with past results.

Table 2: Percentage of Offenders who have Re-Offended, by Time and Ajs Program Participation
Time After Program Completion Cumulative Percentage of Referrals Who Have Re-Offended Table note ii
2016 Table note iii 2012 2006
Participants Comparison Group Participants Comparison Group Participants Comparison Group
1 year 11.9 20.7 10.9 18.2 10.9 21.8
2 years 17.6 29.8 17.6 28.5 17.6 33.8
3 years 20.4 34.1 22.0 35.1 22.3 41.7
4 years 23 38 24.8 39.1 26.7 48.6
5 years 24.7 40.5 27.2 42.4 29.9 53.2
6 years 26.7 43.2 28.7 44.5 31.3 55.1
7 years 28.1 45.3 30.4 46.7 32.2 57.4
8 years 29.6 47.3 32.0 48.8 32.2 59.2
Table note ii

Results from the 2000 Recidivism Study are not included in the table as the same information was not included in the report.

Return to table note ii referrer

Table note iii

Note: Recidivism rates are fitted from the Cox Proportional Hazards Model and are based on the average characteristics of the national sample. For 2016:

  • number of prior convictions – drug (mean=0.22)
  • number of prior convictions – violent (mean=1.18)
  • number of prior convictions – non-violent (mean=2.1)

Return to table note iii referrer

Rates of re-offending were found to be significantly lower among program participants at every point in time after completing the program. The differences between the likelihood of re-offending for AJS program participants and non-participants are particularly pronounced in the years immediately following the program, but the cumulative effects, even after eight years, remain.

The results derived from the 2016 Cox Regression Survival Model are plotted in Graph 1 on a continuous time scale to graphically show the time path of recidivism among the two groups. An immediate divergence in recidivism rates between the two groups is again evident. The divergence persists for the long term, up to eight years and beyond, which is the maximum limit of time available with this sample.

Graph 1: Percentage of Offenders who have Re-Offended, by Time and AJS Program Participation
Graph 1: Percentage of Offenders who have Re-Offended, by Time and AJS Program Participation, described below
Graph 1: Percentage of Offenders who have Re-Offended, by Time and AJS Program Participation - Text Version

A line chart of the cumulative percentage of the AJS average recidivism rates over time.

Vertical Axis: Cumulative percentage (values: from 0-70)

Horizontal Axis: Time in years after program completion (values from 1-9)

Line 1: Participant Group

Cumulative percentage of average recidivism rates: 1 year: 11.9; 2 years: 17.6; year 3: 20.4; year 4: 23.0; year 5: 24.7; year 6: 26.7; year 8: 29.6.

Line 2: Comparison Group

Cumulative percentage of average recidivism rates: 1 year: 20.7; 2 years: 29.8; year 3: 34.1; year 4: 38.0; year 5: 40.5; year 6: 43.2; year 8: 47.3.

4. Comparative Costing Analysis

4.1 Aboriginal Justice Strategy

The total costs of AJS programs were calculated based on the amounts allocated from the Department and cost-shared contributions from the provinces and territories for the 2014-15 fiscal year, with total allocations of $25,591,255 Footnote 59. The total number of clients referred to an AJS-funded program for 2014-15 was 9,039. The mean of the average cost per participant of the AJS programs in 2014-15 was $2,831.

4.2 Mainstream Justice System

The total MJS court-based costs, including the administration of court services, prosecution and legal aid, was approximately $1,650,268,754 in 2014-15. When costs for the administration of court services, prosecution and legal aid costs are considered together, the average cost per case Footnote 60 going through the MJS in 2014-15 was $4,435.

4.3 Cost Comparison

The total AJS cost per case in 2014-15 was $2,831 and the total MJS cost per case was $4,435. Therefore, AJS provides immediate savings to the MJS in the amount of $1,604 per program participant.

4.4 Present Value and Future Cost Savings

The present value, in 2014 dollars, of the longer-term (eight year) cost savings associated with the AJS were calculated based on the rationale that the differences in the participant and comparison groups' rates of re-offending result in fewer instances of AJS-funded program participants being involved in the MJS in the future, which reduces the amount of required future court, prosecution and legal aid expenditures. To estimate the value of these cost savings, the incremental reduction in the average recidivism rates between program participants and the comparison group were calculated each year for eight years following program participation. The incremental reduction in the recidivism rate each year was then calculated as an average cost savings to the MJS each year over the eight-year period in question, for each AJS program participant. Since the recidivism analysis found the program participants in the study were 8.8% Footnote 61 less likely to re-offend after one year than the comparison group, and the average cost per case in the MJS was estimated at $4,435, the cost savings per program participant, in 2014 dollars, one year later would be:

$4,435 × 8.8% = $390

Finally, the total present value (in 2014) of the eight years of cost savings per participant was calculated using the cost savings per participant per year, and the TBS-accepted real social discount rate for federal cost-benefit analysis of 8% per year.

cost of savings equation, described below
Cost of savings equation - Text Version

The figure shows the present value formula of the cost savings of the AJS Program. The numerator is the future value, the cost savings per program participant to the mainstream justice system each year; divided by the denominator, one plus the social discount rate to the power of the n, the number of years of cost saving. The summation symbol shows n equals eight years as the upper limit of summation, and the social discount rate of i set at 0.08, or eight percent.

Using this formula, the present value of the cost savings per AJS-funded program participant over the eight years following program participation was $660 Footnote 62. When added to the immediate cost savings per participant of $1,604 Footnote 63, there is a total present value of savings of $2,264 Footnote 64 over eight years. As AJS programs served approximately 9,000 participants each year, these cost savings per participant would contribute to much greater total savings.

As an example, Table 3 suggests that in 2014-15, approximately $20.5M in present and future cost savings to the MJS were achieved through AJS programs.

Table 3. Total Savings of the 2014-15 AJS Cohort to the MJS Using Present Value over Eight Years
Immediate cost savings per AJS-funded program participation in 2014-15 $1,604
Cost savings per AJS-funded program participant over the eight years (discounted total PV) $660
Total present value of savings ($1,604 + $660) per AJS-funded program participant $2,263
Total savings of the 2014-15 AJS cohort to the MJS over eight years Table note iv $20,464,296
Table note iv

Calculated by taking the total present value of savings and multiplying it by the number of AJS clients for 2014-15 ($2,264 x 9,039 clients)

Return to table note iv referrer

As the costs of AJS programs, numbers of clients served, and reductions in participant recidivism rates tend to remain relatively steady over time, it is reasonable to assume that the future cost savings incurred each year would be similar.

5. Conclusion

One of the main objectives of the AJS is to contribute to decreasing rates of victimization, crime and incarceration among Indigenous people in communities operating AJS programs. This is expected to be achieved by reducing rates of re-offending. The results of this study indicate that the AJS has been effective in meeting this objective.

The analyses presented in this report were carried out in an effort to assess whether a measurable link could be established between offender participation in AJS programs and their likelihood of re-offending. As a whole, findings derived from these analyses show strong support for the existence of such a link. Offenders who were referred to, but did not participate in, an AJS program were significantly more likely to re-offend than offenders who participated in an AJS program.

When interpreting the findings of this investigation, it is also important to bear in mind that conclusions based on the current data set cannot necessarily be generalized across all programs funded under the AJS because the sample of programs remains somewhat limited. Of the 217 AJS programs funded in 2014-15 across Canada, only 30 programs from eight provinces and territories were considered here. However, results indicate that, at least for these programs, AJS program participation was consistently linked with reduced recidivism and a cost saving to the MJS. These results were consistent over the past three recidivism studies as well.