Evaluation of the Aboriginal Justice Strategy
Appendix D: Summary of the Recidivism and Costing Analyses
1. Introduction
As part of an evaluation of the Aboriginal Justice Strategy (AJS), the Department of Justice Canada (the Department) conducted a statistical analysis of the impact of the AJS-funded community-based justice programs on rates of re-offending. This study was previously undertaken by the Department in 2000 Footnote 53, 2007 Footnote 54 and 2011 Footnote 55. These studies were based on a comparison of recidivism rates for offenders who participated in one of the selected AJS programs to a comparison group of offenders who were referred but did not participate. To further the study, a comparative costing analysis was conducted to estimate the impact the AJS programs have had on reducing costs of the mainstream justice system (MJS). The analysis compared the costs of an offender participating in an AJS program versus going through the MJS court process, the only alternative for the majority of AJS program participants.
2. Methodology
2.1 Recidivism Study
The methodology developed for this study was intended to replicate the quasi-experimental approach taken for the previous recidivism studies. To that end, a total of 30 community-based justice programs with a sample size of 2,807 were included in the study. Survival analysis was the statistical approach used to model the likelihood of re-offending. This method is ideally suited to modeling the occurrence and timing of events Footnote 56, particularly in cases such as this study, in which a re-offending event does not occur for all individuals in the sample. This is known as “right-censored” data, as the observation period ends before the re-offending event takes place for some members of the sample. The standard survival regression procedure used for these types of analyses is the Cox Proportional Hazards Model Footnote 57, which was also used in the three previous recidivism studies.
2.2 Comparative Costing Analysis
The impact of reducing recidivism was also measured through a costing analysis. This analysis was undertaken by calculating the costs of AJS programs and the total program spending averaged over the recorded number of clients for each program for one fiscal year (2014-15). For comparative purposes, the costs of the MJS were also calculated for the same fiscal year. Recognizing that the main point of diversion from the MJS for participants of AJS programs is the courts, expenditures as reported in public accounts related to court administration, prosecutions and legal aid were used to determine the costs of the MJS.
2.3 Limitations and Mitigation Strategy
For the recidivism analysis, a main limitation was the lack of a true experimental design, as practical and ethical constraints precluded the random assignment of persons to participant and comparison groups. Pre-existing differences between the participant and comparison groups could lead to differential outcomes with respect to re-offending. However, to mitigate this limitation, a statistical approach Footnote 58 that could control underlying differences between the participant and comparison groups was utilized.
With regards to the costing analysis, although the full costs of the AJS were included in the analysis, only estimates of some court-based (administration of court services, prosecution and legal aid) costs were included through examining public accounts. Additional costs associated with the administration of justice (e.g., police, probation, custody) and those borne by the community were excluded from the analysis. This reduces the ability to understand the full impact of the AJS, although it provides an indication of potential cost savings for one point of contact with the MJS.
3. Recidivism Results
Table 1 below presents some of the key characteristics of the total sample, as well as of the AJS program participant group and the comparison group. In terms of offender characteristics, the majority of offenders in the total sample were men (53.6%), and offenders were most frequently (41.6%) between the ages of 18 and 24 at the time of Program completion, with a mean age of 30. With respect to offence history, prior to their referral to an AJS program, 91% of offenders had never been convicted of a drug offence, 72.5% had never been convicted of a violent offence, and 66.1% had never been convicted for a non-violent offence. The comparison group members, relative to program participants, were slightly less likely to have never been convicted for an offence prior to program referral, with a difference of 3.4% for drug offences, 5.2% for violent offences, and 3.4% for non-violent offences.
| Number of Prior Convictions (%) | Program Participants (n=2,091) |
Comparison Group (n=716) |
Total Sample (N=2.807) |
|---|---|---|---|
| Drug | |||
| 0 | 90.0 | 93.4 | 91.0 |
| 1-5 | 9.1 | 6.3 | 8.3 |
| 6 or more | 0.9 | 0.28 | 0.7 |
| Mean | 0.26 | 0.12 | 0.22 |
| Violent | |||
| 0 | 71.2 | 76.4 | 72.5 |
| 1-5 | 20.9 | 20.0 | 20.7 |
| 6 or more | 7.9 | 3.6 | 6.8 |
| Mean | 1.3 | 0.73 | 1.18 |
| Non-violent | |||
| 0 | 65.3 | 68.7 | 66.1 |
| 1-5 | 23.0 | 24.3 | 23.3 |
| 6 or more | 11.7 | 7.0 | 10.5 |
| Mean | 2.4 | 1.3 | 2.1 |
| Years of Program Completion (%) | |||
| 2004-2007 | 18.7 | 30.0 | 21.6 |
| 2008-2011 | 53.4 | 51.8 | 53.0 |
| 2012-2014 | 27.9 | 18.2 | 25.4 |
| Age at Program Completion (%) Table note i | |||
| 18-24 | 40.5 | 44.6 | 41.6 |
| 25-34 | 29.5 | 29.3 | 29.4 |
| 35-44 | 17.9 | 15.9 | 17.4 |
| 45 and over | 12.0 | 10.2 | 11.6 |
| Mean | 30.0 | 29.0 | 30.0 |
| Gender (%) | |||
| Male | 55.5 | 48.2 | 53.6 |
| Female | 44.5 | 51.8 | 46.4 |
| Province | |||
| British Columbia | 3.9 | 6.4 | 4.6 |
| Manitoba | 20.9 | 43.7 | 26.7 |
| New Brunswick | 8.1 | 7.0 | 7.8 |
| Newfoundland and Labrador | 0.14 | 0.0 | 0.1 |
| Ontario | 30.2 | 14.8 | 26.2 |
| PEI | 0.29 | 0.0 | 0.2 |
| Saskatchewan | 35.2 | 27.7 | 33.3 |
| Yukon | 1.0 | 0.4 | 1.0 |
- Table note i
-
For the comparison group, age at program “completion” is deemed to be age at six months following referral to the program.
Though the data collected for this study considered offenders referred between 2004 and 2012, over half (53%) of offenders in the total sample completed (in the case of program participants) or would have completed (in the case of the comparison group) the AJS program between 2008 and 2011.
Table 2 demonstrates the estimated cumulative recidivism rates for program participants and the comparison group at various points in time after participation in a program. These estimates are based on the percentage of offenders who re-offended during the observation period, by program participation, and are fitted to represent the “average” offender in the total sample. The number of prior convictions and age were all fixed at the total sample means for each variable. For comparative purposes, the table includes the results from the previous two studies, which indicate that the current results are consistent with past results.
| Time After Program Completion | Cumulative Percentage of Referrals Who Have Re-Offended Table note ii | |||||
|---|---|---|---|---|---|---|
| 2016 Table note iii | 2012 | 2006 | ||||
| Participants | Comparison Group | Participants | Comparison Group | Participants | Comparison Group | |
| 1 year | 11.9 | 20.7 | 10.9 | 18.2 | 10.9 | 21.8 |
| 2 years | 17.6 | 29.8 | 17.6 | 28.5 | 17.6 | 33.8 |
| 3 years | 20.4 | 34.1 | 22.0 | 35.1 | 22.3 | 41.7 |
| 4 years | 23 | 38 | 24.8 | 39.1 | 26.7 | 48.6 |
| 5 years | 24.7 | 40.5 | 27.2 | 42.4 | 29.9 | 53.2 |
| 6 years | 26.7 | 43.2 | 28.7 | 44.5 | 31.3 | 55.1 |
| 7 years | 28.1 | 45.3 | 30.4 | 46.7 | 32.2 | 57.4 |
| 8 years | 29.6 | 47.3 | 32.0 | 48.8 | 32.2 | 59.2 |
- Table note ii
-
Results from the 2000 Recidivism Study are not included in the table as the same information was not included in the report.
- Table note iii
-
Note: Recidivism rates are fitted from the Cox Proportional Hazards Model and are based on the average characteristics of the national sample. For 2016:
- number of prior convictions – drug (mean=0.22)
- number of prior convictions – violent (mean=1.18)
- number of prior convictions – non-violent (mean=2.1)
Rates of re-offending were found to be significantly lower among program participants at every point in time after completing the program. The differences between the likelihood of re-offending for AJS program participants and non-participants are particularly pronounced in the years immediately following the program, but the cumulative effects, even after eight years, remain.
The results derived from the 2016 Cox Regression Survival Model are plotted in Graph 1 on a continuous time scale to graphically show the time path of recidivism among the two groups. An immediate divergence in recidivism rates between the two groups is again evident. The divergence persists for the long term, up to eight years and beyond, which is the maximum limit of time available with this sample.
Graph 1: Percentage of Offenders who have Re-Offended, by Time and AJS Program Participation
Graph 1: Percentage of Offenders who have Re-Offended, by Time and AJS Program Participation - Text Version
A line chart of the cumulative percentage of the AJS average recidivism rates over time.
Vertical Axis: Cumulative percentage (values: from 0-70)
Horizontal Axis: Time in years after program completion (values from 1-9)
Line 1: Participant Group
Cumulative percentage of average recidivism rates: 1 year: 11.9; 2 years: 17.6; year 3: 20.4; year 4: 23.0; year 5: 24.7; year 6: 26.7; year 8: 29.6.
Line 2: Comparison Group
Cumulative percentage of average recidivism rates: 1 year: 20.7; 2 years: 29.8; year 3: 34.1; year 4: 38.0; year 5: 40.5; year 6: 43.2; year 8: 47.3.
4. Comparative Costing Analysis
4.1 Aboriginal Justice Strategy
The total costs of AJS programs were calculated based on the amounts allocated from the Department and cost-shared contributions from the provinces and territories for the 2014-15 fiscal year, with total allocations of $25,591,255 Footnote 59. The total number of clients referred to an AJS-funded program for 2014-15 was 9,039. The mean of the average cost per participant of the AJS programs in 2014-15 was $2,831.
4.2 Mainstream Justice System
The total MJS court-based costs, including the administration of court services, prosecution and legal aid, was approximately $1,650,268,754 in 2014-15. When costs for the administration of court services, prosecution and legal aid costs are considered together, the average cost per case Footnote 60 going through the MJS in 2014-15 was $4,435.
4.3 Cost Comparison
The total AJS cost per case in 2014-15 was $2,831 and the total MJS cost per case was $4,435. Therefore, AJS provides immediate savings to the MJS in the amount of $1,604 per program participant.
4.4 Present Value and Future Cost Savings
The present value, in 2014 dollars, of the longer-term (eight year) cost savings associated with the AJS were calculated based on the rationale that the differences in the participant and comparison groups' rates of re-offending result in fewer instances of AJS-funded program participants being involved in the MJS in the future, which reduces the amount of required future court, prosecution and legal aid expenditures. To estimate the value of these cost savings, the incremental reduction in the average recidivism rates between program participants and the comparison group were calculated each year for eight years following program participation. The incremental reduction in the recidivism rate each year was then calculated as an average cost savings to the MJS each year over the eight-year period in question, for each AJS program participant. Since the recidivism analysis found the program participants in the study were 8.8% Footnote 61 less likely to re-offend after one year than the comparison group, and the average cost per case in the MJS was estimated at $4,435, the cost savings per program participant, in 2014 dollars, one year later would be:
$4,435 × 8.8% = $390
Finally, the total present value (in 2014) of the eight years of cost savings per participant was calculated using the cost savings per participant per year, and the TBS-accepted real social discount rate for federal cost-benefit analysis of 8% per year.
Cost of savings equation - Text Version
The figure shows the present value formula of the cost savings of the AJS Program. The numerator is the future value, the cost savings per program participant to the mainstream justice system each year; divided by the denominator, one plus the social discount rate to the power of the n, the number of years of cost saving. The summation symbol shows n equals eight years as the upper limit of summation, and the social discount rate of i set at 0.08, or eight percent.
Using this formula, the present value of the cost savings per AJS-funded program participant over the eight years following program participation was $660 Footnote 62. When added to the immediate cost savings per participant of $1,604 Footnote 63, there is a total present value of savings of $2,264 Footnote 64 over eight years. As AJS programs served approximately 9,000 participants each year, these cost savings per participant would contribute to much greater total savings.
As an example, Table 3 suggests that in 2014-15, approximately $20.5M in present and future cost savings to the MJS were achieved through AJS programs.
| Immediate cost savings per AJS-funded program participation in 2014-15 | $1,604 |
|---|---|
| Cost savings per AJS-funded program participant over the eight years (discounted total PV) | $660 |
| Total present value of savings ($1,604 + $660) per AJS-funded program participant | $2,263 |
| Total savings of the 2014-15 AJS cohort to the MJS over eight years Table note iv | $20,464,296 |
- Table note iv
-
Calculated by taking the total present value of savings and multiplying it by the number of AJS clients for 2014-15 ($2,264 x 9,039 clients)
As the costs of AJS programs, numbers of clients served, and reductions in participant recidivism rates tend to remain relatively steady over time, it is reasonable to assume that the future cost savings incurred each year would be similar.
5. Conclusion
One of the main objectives of the AJS is to contribute to decreasing rates of victimization, crime and incarceration among Indigenous people in communities operating AJS programs. This is expected to be achieved by reducing rates of re-offending. The results of this study indicate that the AJS has been effective in meeting this objective.
The analyses presented in this report were carried out in an effort to assess whether a measurable link could be established between offender participation in AJS programs and their likelihood of re-offending. As a whole, findings derived from these analyses show strong support for the existence of such a link. Offenders who were referred to, but did not participate in, an AJS program were significantly more likely to re-offend than offenders who participated in an AJS program.
When interpreting the findings of this investigation, it is also important to bear in mind that conclusions based on the current data set cannot necessarily be generalized across all programs funded under the AJS because the sample of programs remains somewhat limited. Of the 217 AJS programs funded in 2014-15 across Canada, only 30 programs from eight provinces and territories were considered here. However, results indicate that, at least for these programs, AJS program participation was consistently linked with reduced recidivism and a cost saving to the MJS. These results were consistent over the past three recidivism studies as well.
- Date modified: