Treating Youth in Conflict with the Law: A New Meta-Analysis

3. Results

3. Results

After a comprehensive search of both published and unpublished literature, a total of 154 documents were selected as meeting the study selection criteria. Numerous other studies were eliminated due to the strict age and offence criteria as well as the reliance on control/comparison groups and/or appropriate statistical tests. Since some studies reported on more than one treatment group, a total of 195 unique treatment programs were examined in this meta-analysis, which generated 332 unique effect sizes.

3.1 Number of unique outcomes

Most of the studies accepted for this meta-analysis reported on the effectiveness of treatment in reducing recidivism. Few studies, however, reported on additional outcomes such as family functioning and substance abuse reduction. Table 3.1 provides the number of effect sizes generated within each outcome category of interest.

Table 3.1 - Number of Effect sizes by Outcome (N=332)
Outcome No. of Effect sizes
Recidivism 176
Academic performance/attendance 28
Psychological well-being 26
Family functioning 19
Employment gains 14
Social skills 14
Anti-social attitudes 12
Substance abuse 11
Anger management 10
Anti-social peer involvement 6
Cognitive skills 6

3.2 Study characteristics

The studies were published between 1964 and 2002 with a median year of 1988. As Table 3.2 indicates, the 'typical' study used in this meta-analysis was conducted in urban America by an independent evaluator using a non-randomized comparison group and published in an academic journal. In total, the 154 studies examined almost 75,000 young offenders - 30,184 in the treatment groups and 44,334 in the control groups. The studies were generally conducted using a follow-up period of one year to measure the outcomes of interest.

Table 3.2 - Study Characteristics (N=154)

Country
VARIABLE N (%)
United States 125 (83%)
Canada 12 ( 8%)
United Kingdom 10 ( 7%)
Australia 4 ( 3%)

Setting
VARIABLE N (%)
Urban 90 (92%)
Rural 8 ( 8%)
Matched comparisons 25 (17%)

Evaluator
VARIABLE N (%)
Independent 118 (80%)
Involved 30 (20%)

Publication Source
VARIABLE N (%)
Academic journal 118 (80%)
Dissertation 12 ( 8%)
Government report 12 ( 8%)
Non-governmental report 6 ( 4%)

Study Design
VARIABLE N (%)
Simple comparison group 64 (42%)
Random assignment 58 (38%)
Pre/post analysis 6 ( 3%)
  • Note: Frequencies do not total 154 in most cases due to missing data.
  • Note: Percentages do not always total 100% due to rounding.

3.3 Participant characteristics

Age was coded according to the central tendency reported in each of the studies. The overall mean age for the entire sample was 15.23 (SD=1.36) years with the vast majority being male offenders (see Table 3.3). With the exception of gender, most studies did not partition the data in order to allow for an examination of unique groups of interest such as violent offenders or sexual offenders. We also attempted to code additional variables, such as education level, familial status, and ethnicity, but information was not sufficiently recorded in the studies.

Table 3.3 - Treatment Participant Characteristics (N=195)

Gender
VARIABLE N (%)
Primarily male offenders 141 (78%)
Mixed 33 (18%)
Primarily female offenders 6 ( 3%)

Criminal History
VARIABLE N (%)
Primarily first-time offenders 24 (19%)
Mixed 77 (60%)
Primarily repeat offenders 27 (21%)

Offence Types
VARIABLE N (%)
Primarily property offenders 9 ( 9%)
Mixed 81 (80%)
Primarily violent offenders 7 ( 7%)
Primarily sexual offenders 4 ( 4%)
  • Note: Frequencies do not total 195 in most cases due to missing data.
  • Note: Percentages do not always total 100% due to rounding.

3.4 Program characteristics

As Table 3.4 indicates, the 'typical' program examined in this meta-analysis was a group-focused program, operating in the community and providing social skills training and targeting the general psychological well-being of the participants. Program length ranged from only one day to 896 days with a median value of 112 days. In order to gain a clearer sense of the actual exposure to the program (dosage), we coded the number of hours of direct program participation. Program dosage ranged from one hour to 1800 hours in treatment with a median of 27 hours of direct exposure.

Table 3.4 - Program Characteristics (N=195)

Type of Response
VARIABLE N (%)
Group-focused 50 (21%)
Institutional/residential 22 (11%)
Individual-focused 21 (10%)
Multi-focused 19 (10%)
Intensive supervision 19 (10%)
Restorative justice 19 (10%)
Mixed/unknown 17 ( 8%)
Family-focused 13 ( 6%)
Wilderness programs 9 ( 3%)
Boot camps 6 (3%)

Setting
VARIABLE N (%)
Community-based 149 (76%)
Custodial 46 (24%)

Treatment Targets
VARIABLE N (%)
Social skills 103 (53%)
Psychological well-being 97 (50%)
Family issues 82 (42%)
Academic skills 76 (39%)
Cognitive skills 74 (38%)
Employment skills 49 (25%)
Substance abuse 34 (17%)
Community improvements 33 (17%)
Antisocial peers 32 (16%)
Leisure/recreation 31 (16%)

Anger management
VARIABLE N (%)
Antisocial attitudes 23 (12%)
Relapse prevention 23 (12%)

Recent research has demonstrated that program integrity, which involves developing manuals, providing staff training and supervision, and monitoring compliance with the program manual, is an important element of successful treatment (Andrews amp; Dowden, under review). Table 3.5 indicates that the vast majority of the treatment programs were not coded as ensuring program integrity. In fact, of the 195 programs, only 11 programs indicated the existence of a program manual, provided staff training and supervision, and monitored program compliance.

Table 3.5 - Program Integrity Indicators (N=195)

Program Manuals
VARIABLE N (%)
Available 29 (15%)
Not indicated 166 (85%)

Staff Training
VARIABLE N (%)
Provided 90 (46%)
Not indicated 105 (54%)

Staff Supervision
VARIABLE N (%)
Provided 43 (22%)
Not indicated 152 (78%)

Program Compliance
VARIABLE N (%)
Monitored 30 (15%)
Not indicated 165 (85%)

Current research has begun to examine the impacts of including more participants in the treatment process such as the victim and the community, particularly in the restorative justice literature. While additional participants were involved in select programs in this meta-analysis, Table 3.6 indicates that most programs did not involve other participants. Some form of family involvement, however, was present in 45% of the programs.

Table 3.6 - Additional Treatment Participants (N=195)
  VARIABLE
  Victim Involvement N (%) Family Involvement N (%) Community Involvement N (%)
Substantial 20 (10%) 43 (22%) 10 ( 5%)
Limited 3 ( 2%) 45 (23%) 15 ( 8%)
None 172 (88%) 107 (55%) 170 (87%)

3.5 Recidivism

Of the 195 programs within this meta-analysis, 176 programs directly measured the effectiveness of treatment on reducing future criminal behaviour. The mean overall effect size generated was + 0.09 with a 95% confidence interval of + 0.06 to + 0.12. This result can be translated into the following statement:

There was a 9% difference in the recidivism rate of youth in the treatment groups compared to youth in the control groups.

It should be noted that this reduction results from exposure to any form of treatmentregardless of the appropriateness. Moreover, a 9% difference theoretically prevented more than 1,300 offenders from re-offending. This corresponds rather closely to the findings of Lipsey (1995) and Dowden and Andrews (1999), who both reported a 10% reduction in recidivism for youth. Figure 3.1 provides a graphical representation of the effect size distribution.

Figure 3.1 Effect Size Distribution (Recidivism)

Figure 3.1 Effect Size Distribution (Recidivism)
[Description]

In general, providing a response to young offenders, in addition to, or in place of, standard system-based responses (e.g., fines, probation, custody) does appear to decrease the likelihood of recidivism. The effect sizes, however, ranged considerably from - 0.44 to + 0.65 with a standard deviation of 0.20. We therefore examined numerous possible moderating variables to explain the variance among programs.

3.5.1 Intervention form

The first area of exploration was the actual form of intervention. Table 3.7 demonstrates that multi-focused (i.e., counselling programs that offer individual, group and family sessions) and family-focused programs were the most effective responses to youth delinquency, followed closely by individual-focused and institutional/residential programs. Wilderness programs and boot camps, which are often described as punitive in nature, actually demonstrated a negative overall effect from treatment. That is, those youth who participated in these programs demonstrated a higher recidivism rate compared to participants in the control group who did not receive those intervention forms.

Table 3.7 - Effect Size by Intervention Form (N=176)
Response Mean ES
Multi-focused (k=19) + 0.16
Family-focused (k=12) + 0.16
Individual-focused (k=20) + 0.13
Institutional/residential (k=22) + 0.13
Restorative justice (k =17) + 0.11
Intensive supervision (k=19) + 0.08
Group-focused (k=40) + 0.05
Mixed/unknown (k=16) + 0.04
Boot camps (k =5) - 0.07
Wilderness programs (k=6) - 0.09

3.5.2 Treatment targets

The next area of exploration centred on the treatment targets, which can be described as the particular 'needs' that programs attempt to address such as family dysfunction, substance abuse, and anger management. In order to account for the negative effects from boot camps and wilderness programs, these programs were excluded from the analysis. Table 3.8 examines the treatment targets when addressed in positive programs. Encouraging and less encouraging targets demonstrated appreciable effects on recidivism (i.e., generated a difference of 4% or more in the recidivism rates of the treatment versus control group), while ambiguous targets did not demonstrate appreciable effects.

Table 3.8 - Effect Size by Treatment Targets (N=165)

Encouraging
Treatment Target Mean ES
Targeted Not Targeted
Anger management (k=20) + 0.20 + 0.09
Anti-social attitudes (k=15) + 0.15 + 0.10
Academic skills (k=67) + 0.13 + 0.08
Cognitive skills (k=57) + 0.13 + 0.09
Social skills (k=87) + 0.12 + 0.08

Ambiguous
Treatment Target Mean ES
Targeted Not Targeted
Anti-social peers ( k= 31) + 0.12 + 0.10
Relapse prevention ( k= 16) + 0.12 + 0.10
Employment skills ( k= 44) + 0.11 + 0.10
Community functioning ( k= 31) + 0.10 + 0.10
Family dysfunction ( k= 76) + 0.09 + 0.11

Less Encouraging
Treatment Target Mean ES
Targeted Not Targeted
Psychological well-being (k=79) + 0.08 + 0.12
Leisure/recreation (k=27) + 0.07 + 0.11
Substance abuse (k=27) + 0.06 + 0.11

In general, positive programs that target anger management, academic skills, anti-social attitudes, cognitive skills and social skills demonstrate improvements in recidivism compared to programs that do not target these needs. On the other hand, programs that target the psychological well-being of youth, leisure/recreation and substance abuse demonstrate diminished effects (i.e., increases in recidivism compared to programs that do not target these specific needs).

Family functioning, academic skills and employment skills were further coded into more specific targets to better understand their impacts on recidivism. Table 3.9 provides these results. Targeting communication and warmth within the family and providing parents with the skills to adequately supervise and monitor their children were related to clear reductions in recidivism while general family therapy and non-specific family targets demonstrated poorer results. Providing specific vocational skills within a trade (e.g., auto-mechanics) demonstrated very positive results compared to general employment skills (e.g., resume writing) and non-specific employment targets. Finally,

involving educators (e.g., teachers, principals, guidance counsellors) directly in the treatment program and targeting school performance and attendance provided substantial reductions in recidivism while non-specific academic targets demonstrated ambiguous results.

Table 3.9 - Effect Size by Specific Treatment Targets (N=165)

Family Functioning
Treatment Target Mean ES
Targeted Not Targeted
Communication/ warmth* (k=18) + 0.20 + 0.09
Monitoring/supervision (k=20) + 0.14 + 0.09
General family therapy ( k= 51) + 0.07 + 0.11
Non-specific family targets ( k= 6) + 0.03 + 0.10

Employment Skills
Treatment Target Mean ES
Targeted Not Targeted
Specific vocational training ( k= 9) + 0.20 + 0.09
General employment skills ( k= 35) + 0.13 + 0.09
Non-specific employment targets ( k= 7) + 0.01 + 0.10

Academic Skills
Treatment Target Mean ES
Targeted Not Targeted
Direct system involvement ( k= 7) + 0.20 + 0.10
Academic performance* ( k= 35 ) + 0.17 + 0.08
School attendance ( k= 18) + 0.15 + 0.09
Non-specific academic targets (k=21) + 0.12 + 0.10

* p < .05

3.5.3 Treatment length and dosage

Both treatment dosage (i.e., number of hours spent in direct treatment) and length (number of total days spent in the treatment program) were rather important in the overall success of positive programs. Firstly, the length of time the youth participated in the program was inversely related to program success. Those programs that were shorter than six months in length generated a mean effect size of + 0.11 (k=67) compared to programs longer than six months which generated a mean effect size of + 0.05 (k=25). Secondly, those programs that had limited treatment exposure during the program were considerably more successful than programs with higher dosages of treatment. Programs that offered 20 hours or less of treatment exposure generated a mean effect size of + 0.11 (k=25, while programs longer than 100 hours essentially displayed a zero mean effect from treatment (ES = + 0.01, k=11). Those programs in the middle (21-100 hours) generated a mean effect size of + 0.04 (k=34). These findings directly contradict the previous work of Lipsey (1995) who reported that programs longer than 26weeksor with a dosage over 100 hourswere more successful.

The relationship between previous criminal history, which has been deemed a form of risk (Andrews et al., 1990; Dowden, 1998), and its relationship to treatment dosage was evaluated. In other words, the impact of providing low and high dosage programs to first-time (low risk) offenders and repeat (high risk) offenders was examined. The results in Table 3.10 indicate that high risk offenders are more suitable for a higher treatment dosage and low risk offenders are more suitable for a lower treatment dosage. While only 22 studies provided sufficient information on both the risk level of the youths and treatment dosage, this finding is consistent with previous meta-analytic work in this area (Andrews et al., 1990; Dowden amp; Andrews, 1999). It still remains important, however, to maintain a six month limit on the program length as both high risk and low risk offenders were more successful when the program was shorter than six months.

Table 3.10 - Risk Level and Treatment Dosage/Length (N=25)

High risk
  Mean ES   Mean ES
Low dosage ( k= 2) + 0.01 Short length ( k= 14) + 0.23
High dosage ( k= 10) + 0.12 Long length ( k= 5) - 0.01

Low Risk
  Mean ES   Mean ES
Low dosage ( k= 4) + 0.16 Short length ( k= 11) + 0.20
High dosage ( k= 6) + 0.11 Long length ( k= 1) + 0.03

3.5.4 Treatment setting

Previous research has demonstrated that community-based treatment is more effective compared to treatment in an institutional setting. The results of this analysis did not replicate this finding. There was no discernable difference in the recidivism rates of participants from community (ES = + 0.10, k=126) or institutional settings (ES = + 0.10, k=39). It appears that the treatment targets and the form of treatment are more important than the treatment setting.

3.5.5 Program integrity

As discussed, program integrity is thought to be an important contributor to program success and is a notion supported by the results of this meta-analysis. Studies were evaluated based on the number of program integrity principles they stated to follow. As Table 3.11 demonstrates, those programs that followed all four program integrity principles (training and staff supervision, produced manuals and measured program compliance) generated a higher mean effect size compared to programs that did not report efforts to ensure program integrity.

Table 3.11 - Effect Size by Program Integrity (N=165)
Program Integrity Criteria Mean ES
Zero or one criteria (k= 115) + 0.09
Two or three criteria (k=39) + 0.12
All four criteria ( k= 11) + 0.15

3.5.6 Age of participants

In general, programs that primarily treated youth under the age of 16 demonstrated higher reductions in recidivism. Those programs that targeted youth 12 to 15 years of age generated a mean effect size of + 0.13 (k=76) while programs that targeted youth 16 and 17 years of age displayed a mean effect size of + .07 (k=69). Previous research into the effectiveness of treatment for young offenders found that younger participants are more open to the treatment process when compared to older participants (Dowden, 1998; Latimer, 2001). The results of this meta-analysis provide further support for these findings.

3.5.7 Other demographic moderators

There were several other variables that could have explained some of the variance in effect sizes such as gender, education level, familial living arrangements, previous criminal history or type of offence committed. None of these variables, however, provided an explanation for the range of effect sizes within this meta-analysis (i.e., did not demonstrate a 4% change in recidivism rates).

3.5.8 Additional program participants

Programs that directly involved the family in the treatment process generally displayed a stronger mean effect compared to those that did not involve the family. Not having the family involved generated a mean effect size of + 0.08 (k=85), while the mean effect size for limited involvement was + 0.11 (k=38) and for substantial involvement it was + 0.13 (k=42). Due to the limited numbers, we grouped substantial and limited community involvement together and found that some form of involvement (ES = + 0.17, k=19) was related to reducing recidivism compared to having no community involvement (ES = + 0.09, k=142). Involving the victim directly in the treatment process (ES = + 0.11, k=20) did not have an appreciable impact on recidivism compared to no victim involvement (ES = + 0.10, k=145).

3.6 Alternative outcomes

While we felt it was important to examine additional outcomes of interest other than recidivism, such as academic performance and family functioning, the total number of studies that met the selection criteria andmeasured such outcomes was quite low. While the overall effects are presented in the following sections, we were also interested in disaggregating the data in order to identify unique program characteristics related to success. The low number of effect sizes, however, made this analysis untenable.

3.6.1 Academic performance/attendance

Maintaining youth in school and increasing their school performance is clearly a laudable goal, not only in terms of reducing the risk of recidivism, but also in terms of improving the overall life chances of youth. There were a total of 28 studies that measured the effectiveness of treatment in improving academic performance/attendance. The mean effect size generated was + 0.09 (SD = .26) with a confidence interval of - 0.01 to + 0.19. Since the confidence interval included zero, this moderates our confidence that there is a discernable effect from treatment. In addition, the effect sizes ranged from - 0.79 to + 0.61 (SD = .26), which is a considerably large discrepancy.

3.6.2 Psychological well-being

Although targeting the psychological well-being of youth did not have a strong impact on reducing recidivism, treatment, in general, does seem to have an impact on improving the psychological well-being of youth. There were 26 programs that directly measured the impact of treatment on such concepts as self-esteem, depression and anxiety. The effect sizes ranged from - 0.19 to + 0.99 with a mean effect size of + 0.24 (N=26). The 95% confidence interval was + 0.14 to + 0.35 which demonstrates a positive impact from treatment.

3.6.3 Substance abuse

Treating substance abuse has long been an important aspect of correctional treatment as drug and alcohol abuse is highly correlated with delinquent behaviour. The results of the recidivism analysis, however, demonstrated that substance abuse was a less encouraging treatment target. The 11 programs that measured their effectiveness in directly reducing substance abuse indicated that, in general, treatment does have a rather positive impact (ES = + 0.15). The effect sizes ranged from zero to + 0.38 with a 95% confidence interval of + 0.06 to + 0.26.

3.6.4 Employment gains

There were 14 treatment programs that examined employment gains, such as acquiring or maintaining a job, following participation in treatment. The overall effect demonstrated that treatment was successful at increasing the likelihood of youth participating in the workforce (ES = + 0.17). The effect sizes ranged from - 0.26 to + 0.80; however, the 95% confidence interval was -0.002 to + 0.34. Since zero falls within this interval, our confidence in these results are somewhat diminished. Interestingly, employment outcomes are directly related to recidivism outcomes. Multivariate analysis was conducted with all of the outcomes in this meta-analysis and the only significant correlation was between employment gains and recidivism gains (r = .87, p < .001). That is to say, maintaining or acquiring a job had a profound impact on reducing the likelihood of engaging in future criminal behaviour.

3.6.5 Anti-social peer involvement

One of the stronger correlates of delinquency is involvement with anti-social peers. There were six individual studies that examined the effectiveness of reducing involvement with anti-social peers, which was operationalised primarily as reducing involvement with criminal gangs. The mean effect size was + 0.07 with a range from

- 0.31 to + 0.63. The 95% confidence interval of - 0.26 to + 0.39, however, coupled with the very small number of effect sizes, reduces confidence that there is a positive effect.

3.6.6 Anti-social attitudes

Anti-social attitudes were found to be a 'more encouraging' treatment target to reduce recidivism and is therefore an important outcome. The results of the 12 studies that measured changes in anti-social attitudes generated a mean effect size of + 0.12 indicating that treatment was somewhat successful at reducing anti-social attitudes and improving respect for authority and the institutions of the criminal justice system. The range was from - 0.12 to + 0.54 with a 95% confidence interval of zero to + 0.24. As this was an encouraging treatment target for reducing recidivism, it should be further explored in order to determine the successful approaches to reducing anti-social attitudes.

3.6.7 Anger management

The most encouraging treatment target identified for reducing recidivism was anger management. The 10 studies that fit our selection criteria and measured improvements in anger generated a mean effect size of + 0.26, which is the highest effect found in this meta-analysis. The effect sizes ranged from - 0.18 to + 0.91 with a 95% confidence interval of + 0.05 to + 0.96. Since anger management strategies are related to reductions in criminal behaviour, it is important for future research to develop a clearer understanding of what types of anger management are most effective.

3.6.8 Social skills

Social skills, such as a sense of social competence, conversational aptitudes and the ability to work well in groups, are thought to be an important contributor to the development of youth. There were 14 programs that targeted and measured improvements in this area. The mean effect size was + 0.18 with a 95% confidence interval of + 0.01 to + 0.34 and a range of - 0.17 to + 0.73. The large variation indicates support for future research in this area to identify effective components of successful treatment programs.

3.6.9 Cognitive skills

There were only six studies that measured the effectiveness of treatment at improving cognitive skills, such as problem solving and effective goal setting, with youth engaged in criminal behaviour. The results, therefore, are somewhat less reliable. The mean effect size produced by the six studies was + 0.22 with a range of + 0.03 to + 0.76 (the only outcome with all positive effects). However, the 95% confidence interval was - 0.07 to + 0.51 which reduces our confidence in the + 0.22 mean effect.

3.6.10 Family Functioning

amily functioning was deemed to be an important contributor to reductions in recidivism among youth in conflict with the law. There was a total of 19 studies that examined the effect of treatment directly on family functioning. The mean effect size generated was + 0.15 with a range of - 0.06 to +0.60 and a 95% confidence interval of + 0.06 to + 0.23. The results demonstrate the potential of treatment to improve communication within families as well as parental performance on such concepts as monitoring and supervision.

Date modified: