Family Justice Services Western
Final Evaluation

2004-FCY-8E

1.0 INTRODUCTION

This document represents the final evaluation report on the pilot project known as Family Justice Services Western. The project is funded by the Department of Justice Canada, Child-Centered Family Justice Fund—Incentive for Special Projects. It is sponsored by the Department of Justice, Government of Newfoundland and Labrador and managed by a community-based steering committee. This report was prepared by IHRD, in association with Goss Gilroy Incorporated and the Institute for the Advancement of Public Policy.

1.1 Development of Family Law Services in Canada / Newfoundland and Labrador

The Federal-Provincial-Territorial Report on Custody and Access and Child Support (2002) demonstrated there is general acceptance of the need for supportive services in family law matters.[1] The report identified five major service delivery themes for consideration:

  • Public and professional education / information—the FPT Report recommended governments support education for service providers and recipients as core services.
  • Dispute resolution—the FPT Report recommended provision of broad and voluntary dispute resolution services, with informed participants and effective screening for power imbalances.
  • Enforcement of child support.
  • Family legal aid-services—Family legal aid services in Canada are generally restricted to the very poor and often do not cover all family law matters. Some 40 to 80 % of all family law litigants are not represented by lawyers. There is a strong consensus that Legal Aid services are not appropriately available in family law matters.
  • Completion of family court models—the FPT Report recommended the establishment of family courts, with specialized staff that have in-depth training in relevant areas.

The trend emerging across the country is to streamline the court process in family law and make it more accessible. According to the FPT Inventory, 2000,[2] "… All provincial and territorial governments are working to implement effective, out-of-court, dispute resolution mechanisms, and are strongly encouraging separating or divorcing couples to use them." These services are focusing on two primary areas: mediation and parent education.

In Newfoundland and Labrador, publicly funded alternative family law services have been available through the Unified Family Court (UFC) in St. John's since 1978. These services have variably included counselling, education and mediation. However, despite some advocacy for expansion, there has been no extension of these services into other regions of the province prior to this pilot program in the Western Region. Cost would appear to have been the primary obstacle in establishing such services, along with the lower population density outside of St. John's. The UFC delivery catchment area was expanded to include a larger segment of the province, but geographical realities have limited the feasibility of this approach. Travel distances can exceed two hours within the current jurisdiction. Many of the UFC services (e.g. mediation) are located only in St. John's.

Since the inception of FJSW, a second program has been introduced by the federal and provincial governments, again on a pilot basis, to implement family justice services in the Central Region of the province. This program is known as Family Justice Services Central (FJSC). The two levels of government have also recently undertaken a needs analysis for the potential introduction of family law services in the Eastern Region of the province.

The services currently provided by the three programs in place differ significantly in several respects, as the table below illustrates. Several unresolved issues related to the administration and delivery of family law services in the province are also being addressed. These, include:

  • Referrals—all referrals to FJSW come from the Provincial and Supreme Courts, which make automatic referrals prior to court appearances. In FJSC (upon court application) and UFC (at any point), persons can access services through self-referral and other third party means, including, but not limited to, the court.
  • Administration of services—three administrative options appear to be possible: court-based, community or court-annexed, and with Legal Aid (though this option is not being tested at present).
  • Intake—intake was initially a stand-alone service in FJSC, but it is now incorporated into the mediator or counsellor roles in all three programs.
  • Education—both UFC and FJSW offer group education sessions, facilitated by lawyers, mediators and Support Application Social Workers (and the counsellor in the case of FJSW). FJSC is contemplating offering such sessions, but at present is offering only individual sessions with a social worker.
  • Mediation—FJSC and UFC provide mediation services utilizing social workers (UFC) and a person with a business degree (FJSC) with specialized mediation training. FJSC initially provided a mediator who was also a lawyer, but this changed when the original lawyer left the program. Currently the service is restricted to custody and access matters and to formal mediation approaches. In FJSW, the mediators are lawyers and the practice is broader in scope, incorporating a variety of approaches that include, but are not limited to, formal mediation.

Services

Unified Family Court

FJSW

FJSC

Referral

Court, lawyers, general public

Court only

Court, lawyers, general public

Administration

Court-based

Community-based (Court-annexed)

Community-based

Intake

Provided by mediators

Provided by lawyer /mediator

Provided by counsellor

Education

Voluntary group sessions, multi-disciplinary delivery

Court-referred group sessions, multi-disciplinary delivery

Individual sessions—counsellor

 

Mediation

Provided by social workers with approved training from Family Mediation Canada

Provided by lawyer / mediators with mediation training

Provided by mediator with mediation training

Counselling

Not provided

Provided by certified counsellor

Provided by social worker

Child Support

Support Application Social Workers

Support Application Social Workers

Support Application Social Workers

Recalculation

Not provided

Recalculation clerk provides service

Not provided

There is widespread acceptance of the need for alternatives to the existing family law services available in the province, as demonstrated in consultations undertaken in 2001, as part of a national process relating to proposed changes to the Divorce Act.[3] Judges, lawyers, justice officials and community advocates agree that families experience negative consequences in the absence of these services (e.g. higher costs, poorer outcomes, increased stress on parents and children).

1.2 Family Justice Services Western

The idea for Family Justice Services Western (FJSW) arose in the late 1990s from a small group of individuals associated with a community-based mental health service for children—Community Mental Health Initiatives (CMHI) At the time, there were no public mediation or counselling services specifically for families experiencing separation and divorce in that region of the province. This group became aware of funds available under the Department of Justice Canada's Incentive Fund and, in association with the provincial government, successfully applied to pilot a program of family law services, beginning in 2000-2001 and continuing to March 2003.

The program began providing client services in February 2001, under the direction of a steering committee This committee is chaired by an official from the provincial Department of Justice and is composed of representatives from the judiciary, Community Mental Health Initiatives and the provincial Department of Human Resources and Employment. Family Justice Services Western (FJSW) provides education, information, mediation of custody / access and support, counselling, and recalculation services for persons engaged in family law matters in the Western Region of Newfoundland and Labrador. Its staff includes a lawyer / mediator, a counsellor, a Support Applications Social Worker and a recalculation clerk.

Both the Department of Justice Canada and the provincial Department of Justice are interested in FJSW as a possible model for the provision of family law / family justice services. The Department of Justice Canada has an interest in the administrative recalculation of child support component, as this is a new and relatively untested approach. FJSW is an operational representation of the Department of Justice Canada's particular interest in child support reforms, which dates back to 1997, when the federal government included Section 25.1 in the Divorce Act and encouraged provinces and territories to set up recalculation mechanisms. In November 2002, Newfoundland and Labrador became the first province to obtain a designation for a child support service.

1.3 Evaluation Objectives / Approach

This evaluation, as stated in the research contract, has as its objective to "… Monitor and evaluate, and produce a high-quality research report on, the Family Services Western project in Newfoundland with a particular emphasis on and interest in the new administrative child support recalculation service." The evaluation analyzes the project's program design and delivery, staffing, organizational structure, service scope and effectiveness, resources and client satisfaction. The program includes an emphasis on administrative recalculation, as a new and innovative service, and the reader will note a particular focus on this service in the report.

The evaluation used multiple lines of evidence about FJSW, from utilization data to perspectives of community / government stakeholders and direct client input. The evaluation process included a design phase, a review of administrative data, a literature / document review, key informant interviews with stakeholders, a client survey and an interim report. The evaluation findings with respect to the service generally, and recalculation specifically, provide an analysis of this approach and how it may guide the future development of similar services.

1.4 Methodology

The methodology used in the evaluation incorporates a range of approaches and activities. These are described below. In conducting a review of a pilot program of this nature, qualitative and quantitative information need to be combined in order to comprehensively assess the program. The key outcomes to be measured, namely the impact of family law / family justice services on client families and the court system, do not lend themselves to a purely statistical approach. In programs such as FJSW, where provision of innovative services is such a key aspect, the opinions and experiences of key informants and clients are essential components of an evaluation.

Design

This evaluation included a design phase, wherein the following tasks were completed:

  • Site visits—the consultants visited the service site in Corner Brook on two occasions, conducting most of the personal interviews with staff and steering committee members in person. The staff provided a presentation / overview of services. Forms and utilization information were also submitted.
  • Review of written materials (policies, program descriptions, statistics)—the consultants reviewed a comprehensive array of written materials relating to FJSW, including: a rationale and vision, mission, beliefs and goals; a description of its organizational structure; a brief description of its program development; a detailed education program; a domestic violence policy; information for clients and staff on the mediation process and a code of conduct; a public family law guide; detailed information on recalculation; and comprehensive statistics on program utilization.
  • Review and refinement of research questions, information sources and success indicators—research questions were developed for use in evaluating FJSW and another program being implemented in Central Newfoundland (Family Justice Services Central). When these projects were separated, the list of questions for FJSW was redone, with substantial input from FJSW.
  • Development of tools (survey instrument and key informant guides)—the consultants developed a survey instrument to be used with clients, in consultation with the staff of FJSW and federal and provincial justice officials.
  • Design report—a design report was completed, with the refined research questions, instruments, tools and information sources identified.

Administrative Data / Document Review

The consultants received and reviewed several documents from the FJSW program, including:

  • An information pamphlet.
  • Policy documents, including a domestic violence policy.
  • Materials used in client information sessions.
  • Descriptive materials about mediation services.
  • Written program descriptions.
  • Organizational charts.
  • Program statistics.

No formal literature review was called for in the design of this evaluation. It was anticipated that key documents and programs would be identified for the consultants. Relevant documents were reviewed on the suggestion of FJSW and government officials involved in the evaluation. These included:

  • Prior evaluation of Unified Family Court.[4]
  • Provincial rules and legislation with respect to recalculation.[5]
  • A national inventory of government-based services that support the making and enforcement of custody and access decisions.
  • A draft review of Legal Aid services in family law cases in Canada[6].

Key Informant Interviews

Key informants were selected based upon their direct involvement with and knowledge of FJSW, and / or their general position in the government or community. The selection process was conducted in consultation with FJSW and government officials, with efforts made to be inclusive of diverse stakeholders. All those approached agreed to participate and interviews were completed either by phone or in person. Attempts were made to complete all interviews with program staff and steering committee members prior to the client survey, so as to inform its design and to better understand responses. A total of 30 interviews were completed for the evaluation. These included:

  • Provincial government officials (2).
  • Program Steering Committee (5).
  • FJSW staff (4).
  • Judges (4).
  • Court staff (1)—direct discussions were held with Supreme Court staff, but despite numerous attempts, efforts to obtain input from Provincial Court staff were unsuccessful.
  • Private and public lawyers (4).
  • Community agencies (3).
  • Other provincial bodies (7)—Law Society, Legal Aid, Public Legal Information Association of Newfoundland, Canadian Bar Association-Family Law Section and Chief Justices of the Provincial and Supreme Courts.

The protocols used are attached as appendices.

General Survey of Participants

Experienced phone interviewers with a minimum education of a bachelor's degree in the social sciences were identified to conduct the survey. A pre-test was done with the instrument (using five completed survey calls) and minor modifications made as a result. The consultants received a list of 152 clients who had received service through FJSW since its inception and had signed consent forms to be involved in research / evaluation activity. Overall, an estimated 830 clients had used the program since its inception. Using the contact information provided by clients, the research team attempted to complete telephone interviews with as many of the 152 clients as could be reached. At least five attempts were made per contact, at various times of day. A total sample of 86 persons completed the telephone survey, which took approximately one half hour to complete.

The survey sample was distinct in some ways that are important to note. Key characteristics, which may influence the direction of findings to some degree, include:

  • Seventy-one percent (71%) of those interviewed were women. More women gave their consent to be interviewed in the first instance, plus the researchers found the males in the sample more difficult to contact. An analysis of survey results by gender shows that for the most part there were no significant differences based on gender. However, there were some significant differences found on specific questions, including those relating to the program facilitating referral to other services, perceived fairness and safety of the mediation approach, reducing costs and feeling helped in dealing with issues of separation and divorce. Interestingly, in all of these categories, males expressed a more positive response.
  • Given that more than half of the sample obtained (53.5 %) were sole custody mothers, there is obviously some variation in the sample of 150 from the overall population served.
  • There is no guarantee that the survey population is representative of the general population served in FJSW. Assuming that the survey results are consistent with the survey frame of 150 individuals who signed consents, there is 90 % confidence that results are within plus / minus 5.8 % of reported values.

Some survey findings involving a small number of clients, (i.e. those relating to start-to-finish time in various interventions such as mediation, counselling and total time) were clearly inaccurate. These did not emerge in the test of the instrument and so were not uncovered until analysis of the total sample was conducted. They likely represented an interviewer error or a participant interpretation error or both. Such concerns are restricted to these questions. Other survey findings were supported by alternative lines of evidence.

The general client survey instrument used is contained in the appendices.

Survey of Recalculation Clients

There was an attempt in the general client survey to poll recalculation clients about their experiences. However, very small numbers were obtained through this method (n=7). One of the judges with significant involvement provided several possible explanations for these low numbers, including:

  • The timing of the survey may have been prior to formal notification of recalculation outcomes to some survey participants.
  • The region of coverage for recalculation is broader than that served by FJSW generally, so a high percentage of recalculated cases would not have otherwise been clients of FJSW and therefore would not have signed the consents used by the consultants to identify survey participants.

It was determined that a separate telephone survey of recalculation clients would be undertaken. A survey instrument was designed, reviewed and tested by FJSW and government officials. Interviewers, including one of those who conducted the general client survey, were identified and trained with respect to recalculation and the process followed in FJSW.

A list of 130 cases, which represent all the cases that had been recalculated as of January 2003, was forwarded to the consultants by program staff. Of this number, a total of 72 surveys were completed. Findings from this survey are incorporated into the main body of this report, as well as described separately in Appendix 1.

Strengths and Limitations

The major strength of this evaluation lies in its use of multiple lines of evidence. On most important issues, the findings of the general client survey concur with those of other information sources. This is especially important given some of the previously discussed limitations of the survey in terms of its representativeness. Further, the second survey on recalculation provides for a maximum input from clients of this service. Clear administrative data from the program in terms of activity and outputs also made for an ease of analysis.

Specific limitations in this evaluation include:

  • There was no direct observation of services—this was not considered an option in this instance, owing to cost and confidentiality issues.
  • The sample of completed interviews in the general client survey was somewhat smaller than hoped for and not representative of the persons using the service, both in terms of gender and custodial situation.
  • There was no comparison group identified and interviewed with respect to experiences and satisfaction with the existing court-based approaches in the province. Most information on this comparison was obtained from other interviewees (e.g. judges, lawyers).
  • In evaluating impacts of any pilot program, a significant limitation is the fact that the program is being developed and refined as it is being implemented. This evolution means that the service received by later clients may in fact be quite different from those of clients served earlier in the program's implementation. While this needs to be considered as a general limitation, the findings of the survey do not identify such differences to date.

1.5 Reader's Guide to the Report

This report is organized into the following sections: an introduction, a program description, an evaluation of results, and conclusions. Appendices include the findings of two surveys (one on recalculation and one on the service overall), as well as the instruments used for the study. The fieldwork for the study was completed in the fall of 2002, except for the survey of recalculation clients, which was completed in February 2003. Thus the report focuses largely on the period from February 2001 to August 2002, unless otherwise stated.


Date modified: