Aboriginal Justice Strategy Mid-Term Evaluation, Final Report
The Aboriginal Justice Strategy (AJS) was created as a pilot project in 1991 and has since been renewed three times, in 1996, 2002, and most recently in 2007 for another five years with enhanced funding.
To meet the federal government's commitment to results-based management and accountability, the Department of Justice Canada conducted a mid-term evaluation of the AJS. The evaluation covered the first two years of the renewed AJS, specifically, the 2007/08 and 2008/09 fiscal years. Since the AJS has been in place for almost 20 years, the focus of the evaluation was to assess the implementation of the expansion, as well as outcomes associated with providing existing programs with additional funding. As such, the purposes of the mid-term evaluation were to:
- follow up on progress in addressing the recommendations from the 2007 AJS Summative Evaluation;
- assess the design and implementation of the expanded mandate of the AJS;
- examine results for selected short-term outcomes of existing Community-based Justice Programs; and,
- explore the adequacy of data to support a 2011/12 impact evaluation of the AJS.
1. Program description
The AJS enables Aboriginal communities to have increased involvement in the local administration of justice, providing timely and effective alternatives to mainstream justice processes in appropriate circumstances. Enhanced funding in 2007 allowed the AJS to expand its reach into areas of high need such as urban, northern and off-reserve Aboriginal communities, and to focus on Aboriginal youth.
The objectives of the AJS, as stated in its 2007 Terms and Conditions, are:
- to contribute to decreasing rates of victimization, crime and incarceration among Aboriginal people in Aboriginal communities operating AJS programs;
- to assist Aboriginal people to assume greater responsibility for the administration of justice in their communities;
- to provide better and more timely information about Community-based Justice Programs funded by the AJS; and,
- to reflect and include Aboriginal values within the justice system.
The Community-based Justice Program component of the AJS provides support to culturally relevant Aboriginal justice programs in partnership with Aboriginal communities and provincial and territorial governments. The Capacity Building Fund supports capacity-building efforts in Aboriginal communities in relation to building increased knowledge and skills for the establishment and management of Community-based Justice Programs. The enhanced funding, resulting from the last renewal, allowed the AJS to expand the reach of existing AJS program services, to fund new Community-based Justice Programs and to implement the Capacity Building Fund.
The Department's Aboriginal Justice Directorate (AJD) is responsible for managing the AJS and chairing the Federal/Provincial/Territorial (F/P/T) AJS working group, which consists of representatives from the Department of Justice, provinces and territories.
The methodology used to conduct this evaluation is comprised of four main components:
- 69 key informant interviews with Department of Justice staff, provincial and territorial partners, and funding recipients;
- a review of project files;
- a Web-based survey of 46 police and Crown representatives; and
- a review of documentation.
The evaluation was conducted between November 2008 and January 2010. The file review included all 232 program files for fiscal years 2007/08 and 2008/09. Most interviews with provincial and territorial partners were completed during August and September 2009. Interviews with Department of Justice staff took place in September and October of 2009, while the police/Crown survey was conducted in September 2009. Interviews with funding recipients occurred between November 2009 and January 2010. Given that the focus of the mid-term evaluation is on the first two years of the AJS after the renewal in 2007, the findings presented herein are relevant to that time period. Although many key informants noted recent improvements, these were deemed to be outside the scope of the mid-term evaluation.
3. Program Relevance
The objectives of the AJS, as cited in its 2007 Terms and Conditions, are well aligned with the strategic outcome of the Department of Justice
"to create a fair, relevant and accessible justice system that reflects Canadian values". The AJS provides an alternative to the mainstream justice system that recognizes the cultural values and unique context of Aboriginal communities while operating within the Department's strategic framework. As evidenced by the 1996 Criminal Code amendments pertaining to Aboriginal justice and the ongoing federal strategic mandate of encouraging self-government within Aboriginal communities, the objectives of the AJS are derived from and very closely aligned with federal priorities.
The 2005/2006 Annual Report of the Office of the Correctional Investigator observed the ongoing failure of the criminal justice system to meet the unique needs of the population and reflect the "fundamentally different world view" of Aboriginal peoples. As such, it remains an ongoing federal priority to encourage culturally appropriate alternative justice measures within Aboriginal communities, reflected in the renewal and enhancement of the AJS. A number of federal initiatives such as the Aboriginal Courtwork Program (ACW), First Nations policing and other crime prevention strategies also address these concerns. Furthermore, the AJS provides funding to programs that support alternative court systems built around the "Gladue principle" which requires that the Aboriginal offender's situational context be considered before a ruling or sentence is passed down.
All key informants indicated that the AJS is still relevant to Aboriginal people and communities, as there is a strong continued need for community-based Aboriginal justice programs due to the ongoing demand for alternative justice approaches that reflect traditional Aboriginal cultural values and practices. This ongoing need is also reflected in the fact that over 13,000 individuals benefited from a Community-based Justice Program in 2007/08 and 2008/09. It was also expressed that the programs consider victimization and healing in tandem with addressing offenders in a holistic approach that most closely mirrors Aboriginal culture-based justice traditions. It should be noted that some AJS-funded programs have already integrated program elements around victimization and healing.
4. Design and Delivery
During the transitional period after the announcement of renewal and enhancement of the AJS in 2007, a number of discussions were held between the AJD and its AJS partners regarding the criteria and selection of new community-based justice programs. Preliminary discussions with AJS partners were also held regarding the revision of the Community-Based Funding Guide, including the development of new selection criteria and joint processes for the selection of new programs. Consensus among all AJS regional partners, however, was not reached due to conflicting regional requirements.
Overall, the criteria for selecting new Community-based Justice Programs were perceived to be fair and effective, although there is room to improve the clarity of the assessment and selection criteria used. The Capacity Building Fund application and selection process and criteria could be further explained.
The sunsetting nature of the AJS and the recurring need to secure program funding have created challenges for AJS programs and the communities they serve. Programs are forced to shut down or lay off staff in response to uncertainty about the availability of future funding. This has a negative economic impact in addition to disrupting ongoing community and individual healing.
Initially, informal announcements and selection criteria regarding the Capacity Building Fund were communicated to community-based project funding recipients by AJD Regional Coordinators. There was no formal communication strategy to promote the Capacity Building Fund or application process. The extent of communication varied according to region and amount of available funding. During the second year after renewal, the AJD began to develop a Capacity Building Funding Guide, which would include selection criteria and a communication strategy, both of which were to be completed and implemented by fall 2009.
The reach of the AJS was expanded by 42 funding recipients, from 62 in 2006/07 to 104 in 2008/09, a 68% increase in the number of Community-based Justice Programs funded.
The first two years of the renewal saw high staff turnover in the AJD, in addition to instability of leadership and lack of clarity concerning roles and responsibilities. Staff turnover made it difficult for AJS partners and funding recipients to communicate and collaborate with the AJD, and has created barriers in terms of accessing information and general program development. In addition, there have been instances where positions were vacant for extended periods and existing staff were shuffled around to different positions on an as needed basis. Although the staffing situation was beginning to improve under new leadership in the AJD toward the end of the second year of the renewal, many respondents indicated that dedicated funding for the AJS would likely help to reduce these challenges.
In general, yearly renewal of contribution agreements can make it difficult to distribute the funds to programs in a timely manner, which interferes with the ability of the funding recipients to spend the funds within a few short months, particularly for the Capacity Building Fund, which is currently determined after Community-based Justice Program allocations have been committed. In the first two years after renewal, AJS disbursements were less than the amounts committed: $2.98 million below the total allocated amount in 2007/08; and $2.58 million below the allocated amount in 2008/09.
In 2007/08, the first year of the renewal, a main reason for the discrepancy was the delay in the announcement of the renewal and the allocation of funds to the AJS. The cost-shared nature of the AJS creates a situation whereby funding recipients in provinces/territories with tripartite or flow-through agreements are most affected by delays in the release of funding. In the second year, the late announcement and confirmation of funds was less of an issue.
In both years, the difference between the amount committed and the amount dispersed to funding recipients is the result of a number of factors. Funding recipients may not be able to meet their final reporting requirements during the fiscal year and, with a 10% hold back associated with final reporting, not all funds committed could be dispersed. Once reports and financial statements are submitted, at the end of the fiscal year, they must be reviewed and approved by the Department. The timeliness of this process can be exacerbated by late submission of reports. Some funding communities also under-expended the planned/committed amounts due to cost-efficiencies that may have been realized or to late signing of agreements. Finally, uncertainty around the timing and availability of funds, particularly for the Capacity Building Fund, made it difficult to adequately implement programs and projects as planned in the first two years of the renewal.
The AJD cultivates relationships with its F/P/T partners in order to address AJS priorities, through: the AJS F/P/T working group, AJD Regional Coordinators and provinces/territories working with communities to develop their program proposals and Capacity Building Fund applications.
Roles and responsibilities of the funding recipients are clearly defined and documented in contribution agreements. However, as the Terms of Reference in place for the AJS F/P/T working group do not explicitly state the roles and responsibilities of each group, these remain less 'formally' defined.
Although many AJD staff members were unfamiliar with the role of the Aboriginal Law and Strategic Policy (ALSP) Group due to limited involvement, particularly Regional Coordinators, those who were familiar with the ALSP felt that their role was clear when interviewed in the fall of 2009. However, many feel disconnected from the policy-making process and resultant decisions, and it was also expressed that Regional Coordinators who work directly with the programs are not directly involved with policy decisions. This is problematic, as many issues at the 'ground' or program-level are not necessarily being fully communicated at the policy level.
In 2007/08 and 2008/09, Community-based Justice Program and Capacity Building Fund funding recipients were located across all provinces and territories. In addition, the reach of funded programs often extends beyond the community in which the recipients are located. AJS-funded programs and projects reached more than 400 communities across Canada. Overall, 13,166 Aboriginal youth and adults were served by AJS-funded programs in the two years after the Strategy's renewal. It should be noted that although there has been an increase in the number of communities served, many key informants indicated that there are still many Aboriginal communities and people in Canada that do not have access to AJS alternative justice programming.
Community-based Justice Programs funded in 2007/08 and 2008/09 provided a wide range of services to adults and youth, either prior to or after a charge had been laid, depending on the program. Referrals tend to be first-time offenders accused of minor infractions, although some programs try to assist the reintegration of those who have been incarcerated back into the community. Family, community members and victims are typically invited to participate in the alternative programming. Anecdotal evidence from the interviews suggests that the Community-based Justice Programs are successful, often deterring first-time offenders, particularly youth, from exhibiting similar behaviours in the future. As reported in the 2007 AJS Summative Evaluation, a 2006 recidivism study provided evidence supporting the notion that those who participate in a Community-based Justice Program are less likely to reoffend than those who are referred but do not participate. The success of the programs are reported to be the result of a combination of factors that represent Aboriginal approaches to justice and healing, including fostering understanding, restitution and re-establishing balance in relationships.
Capacity Building Fund projects supported Community-based Justice Programs through implementing the following types of activities:
- organizing forums and conferences, and networking;
- delivering or arranging workshops for staff and/or partners;
- undertaking program expansion activities;
- organizing and implementing skills-building activities for community members (particularly youth); and,
- delivering or arranging training related to program delivery (e.g., circles keeper training) that was not covered through the Community-based Justice Program.
6. Adequacy of Data for Impact Evaluation
Currently, the AJD does not have an electronic standardized reporting database. To address this issue, a working group has been established to develop a set of national data requirements and a data collection system for the AJS. As of fall 2009, the working group had established the objectives and guiding principles of the National Data Requirements project and a draft data model had been designed. The system will not be operational by the year of the impact evaluation; therefore, the data necessary for the evaluation will have to be obtained from other sources. As such, related data required for the impact evaluation will have to be accessed from existing paper reports submitted by funding recipients unless other methods of tracking are implemented prior to the evaluation. As these reports are not always complete and vary in terms of content, the ability to evaluate the impact of the AJS will be a challenge.
In addition, the capacity of communities to implement and conduct data collection, reporting and evaluation varies widely. Reporting was inconsistent in many cases, which could affect the impact evaluation should project files be used as a data source. Smaller communities may need capacity-building assistance to meet data collection requirements for the evaluation.
Finally, as overall program awareness was somewhat low within the police/Crown community, it would be beneficial to conduct a more comprehensive survey and/or telephone interviews with a larger sample size, in addition to expanding the sample population to include other mainstream justice professionals.
- Date modified: