A Meta-Analytic Examination of Drug Treatment Courts: Do They Reduce Recidivism?
3. Results (cont’d)
3.3 Participant Characteristics
In total, the studies examined 17,214 offenders who had successfully completed drug treatment court programs and 14,505 offenders in the control or comparison groups. The mean age of DTC participants recorded within the studies was 28.4 years – 7 studies provided data primarily on youth under 18 years of age. Given that the literature is almost exclusively American, Aboriginal identity was rarely recorded. Instead, when racial information was provided, the data was broken down according to categories such as Black, Hispanic or Caucasian. However, these data were not available often enough to include in the meta-analysis. Table 4 provides the gender breakdown within studies and indicates that most of the participants were male.
| VARIABLE | N (%) |
|---|---|
| Gender (N=66) | |
| All male | 1 ( 1.5%) |
| Mostly male (seventy percent or more) | 31 (47.0%) |
| Mixed/unknown | 32 (48.5%) |
| Mostly female (seventy percent or more) | 1 ( 1.5%) |
| All female | 1 ( 1.5%) |
3.4 Recidivism Results
The 66 DTC programs within this meta-analysis directly measured the effectiveness of treatment on reducing future criminal behaviour. The mean overall ESE was + 0.14 with a 95% confidence interval of + 0.10 to + 0.17. By converting the ESE into a Binomial Effect Size Display, a simple statement can be made:
Generally, 57% of the participants in the drug treatment courts will not be charged with a new criminal offence during the follow-up period compared to 43% of offenders within the control/comparison groups.
This can also be translated into a similar but more general statement:
drug treatment courts reduced recidivism rates by 14% compared to traditional criminal justice system responses.
There is, however, an explicit relationship between statistical significance, effect size and sample size whereby the size of a study increases the level of significance (Rosenthal, 1991). In order to give more weight to studies with larger sample sizes, a weighted ESE was calculated according to the technique described by Hunter, Schmidt and Jackson (1982). The weighted ESE was +0.13, which is relatively similar to the unweighted ESE.
As Figure 1 indicates, the majority of the DTC programs demonstrated a positive impact on recidivism (i.e., programs with an effect size above zero). Only 10 studies indicated a negative impact while 56 studies demonstrated a positive impact. A single-sample t-test indicated that the mean effect size estimate was significantly different from zero (t(df=65)=7.58, p< .001). Therefore, in general, DTCs appear to decrease the likelihood of future criminal behaviour better than traditional justice responses. Heterogeneity analysis, however, indicated that the variance in effect size estimates was also significant (c2 (df=65)=465.3, p< .001). The moderating variables were, therefore, further examined to determine differences in effect based upon program characteristics, participant characteristics and study characteristics.
Figure 1 – Recidivism Effect Size Estimate Distribution
3.5 Moderating Variable Analysis
In order to understand the potential impact of moderating variables, individual ESEs were calculated across a number of different groups such as adults versus youth and first-time offenders versus repeat offenders. Table 5 provides the results of this analysis. Variables that were not amenable to this form of analysis (e.g., substantial amount of missing data, minimal variance) were not included. As well, due to missing data within some studies, the total number of effect size estimates within each moderator analysis is not always equal to the total number of possible ESEs. For example, only 49 of the possible 66 ESEs had information regarding the age of the participants. Finally, the additional treatment components reported in Table 3 (e.g., anger management, academic skills, vocational skills) were not used within the moderator analysis as the information was deemed too unreliable. As indicated previously, it was only recorded when the authors made a direct statement that the component was part of the treatment. In other words, the data is not necessarily an accurate reflection of the programs.
Table 5: Moderating variable analysis
| VARIABLE | Mean ESE (N) | 95% CI |
|---|---|---|
| Youth (less than eighteen years) | + .06 ( 7) | - .12 to + .24 |
| Adults (eighteen years and older) | + .16 (42) | + .11 to + .20 |
| VARIABLE | Mean ESE (N) | 95% CI |
|---|---|---|
| First-time offenders | + .15 (13) | + .09 to + .22 |
| Mixed/unknown | + .13 (22) | + .07 to + .16 |
| Repeat offenders | + .17 (21) | + .08 to + .25 |
| VARIABLE | Mean ESE (N) | 95% CI |
|---|---|---|
| Academic journal | + .14 (33) | + .08 to + .20 |
| Other publication type | + .13 (33) | + .09 to + .17 |
| VARIABLE | Mean ESE (N) | 95% CI |
|---|---|---|
| Less than one year | + .09 ( 8) | + .01 to + .17 |
| One year to two years | + .14 (44) | + .10 to + .19 |
| Greater than two years | + .17 (12) | + .10 to + .23 |
| VARIABLE | Mean ESE (N) | 95% CI |
|---|---|---|
| Forty-five percent attrition or less | + .13 (20) | + .06 to + .21 |
| Greater than forty-five percent attrition | + 14 (24) | + .08 to + .19 |
| VARIABLE | Mean ESE (N) | 95% CI |
|---|---|---|
| Random assignment | + .09 ( 8) | - .01 to + .20 |
| Non-random assignment | + .14 (58) | + .10 to + .18 |
| VARIABLE | Mean ESE (N) | 95% CI |
|---|---|---|
| Justice system | + .13 (39) | + .09 to + .18 |
| Drop-outs/non-graduates | + .31 (10) | + .17 to + .45 |
| Eligible (not participating) | + .11 (23) | + .05 to + .17 |
| VARIABLE | Mean ESE (N) | 95% CI |
|---|---|---|
| Out-patient only | + .11 (24) | + .03 to + .18 |
| Combined (both in-patient and out-patient) | + .13 (30) | + .08 to + .17 |
| VARIABLE | Mean ESE (N) | 95% CI |
|---|---|---|
| Less than one year | + .07 (15) | - .00 to + .15 |
| One year to eighteen months | + .18 (33) | + .13 to + .23 |
| Longer than eighteen months | + .08 ( 6) | + .02 to + .14 |
First, one of the noteworthy findings from the moderator analysis is the difference in the reported effectiveness of DTCs according to age. When comparing youth and adults, the results indicate that DTCs are more effective for adults, although the difference is not statistically significant. However, when examining the mean ESE for youth alone, the 95% confidence interval includes zero, thus diminishing confidence that DTCs are actually effective with youth. Confidence intervals give us a measure of the precision of the mean effect size estimate computed. In this case, the 95% confidence interval implies that 19 times out of 20, the ‘true’ mean will fall within the provided range. If this range includes zero, we cannot be statistically certain that there is an actual effect from DTC participation, as a zero ESE implies no difference between the DTC participants and the control/comparison group. Since there are only 7 ESEs contributing to the youth results, however, additional research is warranted to determine if the DTC model is, in fact, not effective with youth.
Second, the difference in the mean ESE based upon the follow-up length used to measure recidivism is important. Those studies with longer follow-up periods produced larger effects compared to those with shorter follow-up periods. Normally, recidivism rates increase with longer follow-up lengths as offenders have more time at-risk to re-offend and/or come to the attention of police. Not surprisingly, the results of this meta-analysis follow this trend as recidivism rates generally increase within both DTC programs and control/comparison groups as the follow-up length increases. The important difference, however, is that the gap between the two groups increases over time. In other words, as the follow-up time increases, those in the comparison group become even more likely to re-offend compared to the DTC participants. It is therefore likely that the benefits of DTC participation increase with time. As such, longer follow-up periods are particularly important in DTC research to fully understand the impact of participation on recidivism.
Third, it is not surprising that those evaluators who chose to randomly assign offenders into a treatment or control group generated diminished effects compared to those who did not use random assignment. Previous research has demonstrated that as the methodological rigour of a study increases, the reported effects decrease (Latimer, 2001). Further, the 95% confidence interval for the six studies that used random assignment actually included zero and therefore diminishes confidence that DTCs are effective when a random treatment/control design is used to measure effectiveness. However, random assignment is difficult within this context as judges and lawyers often prefer (understandably) to prioritize treatment for those deemed most appropriate. Therefore, in many of the studies that initially tried to implement random assignment procedures, the process was discontinued and replaced with a comparison group design.
The fourth finding from the moderator analysis is the fact that the choice of control/comparison group has a significant impact on ESEs. Studies that used drop-outs or non-completers as a comparison group demonstrated a significantly higher mean ESE (F(df=2)=6.87, p<.01) compared to studies that used a traditional justice system comparison group or offenders who were eligible but did not participate. These results are also not surprising given that drop-outs/non-completers would logically provide a comparison group that are not as motivated as those who completed the program. Those who were eligible for the program but did not participate likely form the most similar group for comparison as they would have met the inclusion criteria (i.e., screened for substance abuse problems, similar criminal histories and offence types). And when this group was used, DTC participation still demonstrated an 11% improvement in recidivism.
The fifth and final finding is the difference in reported effects based upon the length of the DTC program. Programs that provided services for one year to eighteen months demonstrated a significant reduction in recidivism (F(df=2)=3.76, p<.05) compared to shorter or longer programs. In fact, the 95% confidence interval for shorter programs includes zero, which further reduces confidence that programs shorter than one year have a positive impact on recidivism. It is possible that the shorter time frame is not sufficient for the cognitive and behavioural changes to become entrenched while the longer timeframe may induce some form of "treatment fatigue."
- Date modified: