Research on Compliance with Child Support Orders and Agreements in Prince Edward Island

2003-FCY-1E

6. REVIEW OF THE EFFECTIVENESS OF THE RESEARCH STRATEGY

An important objective of the pilot research in P.E.I. was to assess the viability and cost-effectiveness of research strategies being considered for broad application across the country. It was recognized that circumstances in P.E.I. differ in a number of respects from those in other provinces and territories, particularly with regard to the size of the population (and therefore the MEP caseloads and the numbers of separating parents with children), but also to the nature of the population and the practices employed at the MEPs and, more generally, by the family law and social service agencies. Nevertheless, it is believed that sufficient similarities exist to be able to derive useful lessons from the P.E.I. experience.

In this section, we review each of the research elements employed in P.E.I., in order to assess the quality and usefulness of the information obtained, the cost factors to take into account, the sample size considerations that will likely influence decisions in larger provinces, and issues such as the burden placed on the local MEP and the logistics of conducting the research. The focus is on what took place in P.E.I., but reference is made to the likely implications in other provinces.[36]

6.1 MEP Case File Data Collection

Collection of data from the MEP information system in P.E.I. was intended primarily to capture data on the support payment records of paying parents, in order to develop useful categories of paying parents and then to analyze that information, both in aggregate and on a case by case basis in conjunction with demographic and interview derived information. The case files also contained other information, such as dates of birth, place of residence, age and gender of children, and basic information on the nature of the orders under which child support payments were being made. Information on employment and social service status was available in only some cases, and was not reliable in terms of timing. No income information was in the system.

In P.E.I., the size of the sample (500 cases, about 27 percent of all cases) and the projected cost and timing of extracting the information in an automated fashion meant that it was more practical to collect the information manually. That meant developing a data collection form, and going into several screens in the system for each case. As well, it was necessary in some cases to review hard file ledgers to obtain payment information prior to 1996. For the payment information, researchers had to total annual obligations and payments using a calculator, and determine for each month whether the obligation had been made. In most cases the latter was obvious, but in some cases payment adjustments had been made (for example if payments had been made directly to a recipient without the MEP’s knowledge), and this required that payments be allocated fairly to the appropriate months in order to get an accurate count of payments made or missed.

While this was painstaking work, the sample size was such that it was not inordinately expensive. The information we obtained, particularly on payment records, was both critical to the research as a whole and valuable in itself in characterizing compliance in P.E.I.

The quality of the information, we believe, is high for several reasons. First, the collection process was based on a thorough assessment of the information in the system and close collaboration with MEP staff to ensure that researchers understood how to interpret both standard information and the anomalies that inevitably showed up. In a small but substantial number of cases (no count was kept, but a reasonable estimate would be 75 cases), some inquiry was necessary to ensure that we were interpreting the information correctly. Second, in a few of those cases it was determined that the payment record data required updating. As a result, all cases that showed similar characteristics were reviewed and updated as necessary. In the great majority of cases this meant only that the arrears figures would be more accurate. In some cases the records of monthly obligations paid was affected, most typically when a file should have been closed because a child was no longer living with the custodial parent or was no longer eligible for support and when the recipient had not contacted the MEP to inquire about missed obligations.

The data collection method did result in one limitation that may be alleviated by an automated approach. That is, that payments and obligations and numbers of missed payments were recorded on an annual basis instead of monthly. For most purposes this is not a problem, but it is not possible, for example, to produce a chart depicting the pattern of monthly payments in order to obtain a highly detailed breakdown of compliance patterns. However, this is beyond the immediate requirements of the current research, and should be considered only if the additional cost is minimal.

In retrospect, it would appear that collecting information from the MEP system on employment patterns and interaction with social services and employment insurance may not be cost-effective, except where the MEP information system is a more reliable source than it is in P.E.I. This information was collected routinely in the interviews with paying parents and recipients, and we believe this is a more reliable source.

Sample size for the case file data collection is only a significant issue when data extraction cannot be done automatically. There is no apparent reason why this research element could not include the full population of cases, with the possible exception of very large MEPs, where data file size might be beyond manageable limits (although this is unlikely).

The burden that this element of the research placed on local MEP staff was primarily at the outset, as researchers were helped to understand how the MEP operated and the information system functioned, and to make sense of the information itself. Once that was accomplished, all that was required was periodic verification of case-specific information.

6.2 Interviews with Professionals

The purpose of interviewing professionals who work in various capacities with separating and divorcing parents was to find out what aspects of the legal and social service systems might be influencing parents’ decisions about child support and related matters, such as parenting and visitation. As well, we were interested in what they had to tell us about how their clients appeared to experience the legal process, and the process of separation itself, when children were involved. In P.E.I. we interviewed family law lawyers, judges, court-appointed mediators (who were also social workers assigned to conduct “home studies” to assist the court in deciding about custody arrangements), parenting educators, court workers with responsibilities related to child support, and Maintenance Enforcement staff.

This element of the research, in our view, was important to an understanding of how the legal system and related social services operate in P.E.I., and provided valuable insights into the factors that may be influencing parents’ experiences, the attitudes they adopt and the decisions they make about child support. It was a very inexpensive aspect of the research, and well worth the investment.

In P.E.I. the number of interviews required was limited because of the size of the family law and social service communities. The cost of this aspect would increase in other jurisdictions, depending on a number of factors: the size and diversity of the population of relevant professionals; geographic factors and the potential differences that might be relevant, for example, between major urban, smaller urban and rural areas; and the extent to which a representative sample was desired.

On the latter point, the conclusion drawn from the P.E.I. research is that a representative sample is not necessary. What is required is a sufficient number of interviews with a wide enough range of types of professionals to ensure an understanding of the systems and procedures that parents are faced with and the services that are available to them. As well, it is important to include a sufficient number of interviews to enable researchers to obtain a diversity of perspectives. The object is to gain insight into systemic and other local factors that might influence parents’ attitudes and actions relating to child support (the opinions of individual professionals about what factors might be most important, while of interest, are not critical). We rely primarily on the parents themselves to provide that information.

6.3 Interviews with Parents Registered with the MEP

A primary impetus for the research in P.E.I. was to explore the range of factors that could be influencing compliance and non-compliance with child support orders and agreements. While we recognized that income and employment factors were important, we were also interested in pursuing other possibilities related more to “willingness to pay” as opposed to “ability to pay”. The research in P.E.I. presented an opportunity to link two critical sources of information: the child support payment records of paying parents in the province; and the parents themselves. Lengthy and detailed interviews were conducted with 130 parents, including 51 non-custodial parents (usually fathers) and 79 custodial parents (usually mothers). In 31 cases, we were able to interview both parents from a separation with children. The interviews addressed a wide range of issues, following the families from the time they were together, through the separation process and the legal and other interventions that took place, to the present. Twenty of the interviews were conducted in person in P.E.I., and the remainder were conducted by telephone.

This element of the research was the most expensive. It required substantial work at the outset to establish a representative sampling strategy and to draw the initial sample. Effort from the researchers and the MEP Director and staff was required to develop and send letters to potential respondents describing the study and requesting their participation. Development of the interview guide involved the combined efforts of two members of the research team and a number of Department of Justice Canada research officers for an extended period. The interviews took between one hour and two and a half hours to complete, with some post-interview tidying up of notes as well. Also, for many respondents it was necessary to conduct a search for a current telephone number. As it turned out, a second sample and a second wave of introductory letters were required because we were unable to obtain sufficient numbers of interviews from the first wave. A number of factors contributed to that problem, including the timing (the first wave of interviews was conducted in the summer), the large number of people who could not be reached at the address or telephone number known to the MEP, and a smaller number who declined to participate or who agreed but were then never available for the interview.

Once the interviews were completed, a coding scheme had to be developed for the large number of variables (questions and sub-questions) in the interviews, and the responses for each interview had to be coded and entered into a statistical program (SPSS) for analysis. In addition, responses to many of the questions were analyzed qualitatively because the specific wording of the responses was viewed as being important in understanding the point of view parents were expressing, as well as their attitudes and the sources of those attitudes.

The fact that this element is very labour-intensive is moderated by two considerations. First, the P.E.I. research has laid the groundwork for much of what would be required in other jurisdictions. The interview guides will undoubtedly be modified somewhat based on what we have learned, but the work on those would be substantially reduced. To the extent that the interview guides resemble those used in P.E.I., the coding scheme would be similar. We learned valuable lessons about techniques for locating respondents which would contribute in other jurisdictions. The approach of sending introductory letters appears to have worked well, and a similar approach would likely be appropriate in most jurisdictions.[37] Experience in conducting the interviews and analyzing the results will certainly result in some efficiencies in subsequent studies.

The second consideration is that the interviews, as a research tool, are so critical to the primary purpose of the research. Simply put, there is no other comparable source of information from which to examine the full range of factors influencing compliance. There are certainly limitations to the interviews. We recognize that in some cases the responses obtained may not be accurate, and that some respondents may attempt to use the interview for purposes contrary to ours. We also recognize that our interpretations of the responses will reflect, to some extent, biases of the researchers. Nonetheless, this element of the research was intended to be exploratory. There is little similar research available in Canada or elsewhere, and the questions we are attempting to answer are complex and may well involve a variety of interdependent factors which may at times be contradictory. Thus, as an early step in trying to understand why some paying parents pay child support in full and others don’t, the interviews with parents have been essential. As we have seen in the previous chapter, they have provided us with many insights and have helped to support some of the observations derived from other sources. It would not be possible, without this research tool, to pursue the broad question of why people comply or don’t comply.

However, there are several decisions to be made that can influence the approach taken and the cost involved. The first is whether in-person or telephone interviews are the most cost-effective approach. The conclusion from the P.E.I. experience is that respondents reached by telephone were just as open and willing to talk to us as those approached in person. In a very few cases, prospective respondents said they would not conduct the interview by telephone but said they would have in person. So, there may have been some impact on the sample itself (but very minor), but there did not appear to be an impact in terms of either the depth of information obtained or the quality and nature of the responses to specific questions. One benefit that did derive from the in-person interviews was that it allowed the researchers to gain some insight into the circumstances in which the respondents were living, and it is possible that this influenced the interpretation of responses to some extent.

Another consideration is the sample size required to obtain a sufficient degree of representativeness. Especially in large provinces with a great geographic and cultural diversity of communities, a larger number of interviews would be needed than were conducted in P.E.I. if there was an interest in being able to report on findings at the provincial level. If all that is required is reporting at an aggregate level, and if at least four or five provinces participate, the sample sizes would not have to be substantially larger than the 130 parents interviewed in P.E.I. The cost of individual telephone interviews is not prohibitive.

6.4 Interviews with Parents not Registered with the MEP

This element of the research was included to account for the fact that some, and perhaps many, separating parents never register their child support agreements with the courts or the MEP. There was an interest in finding out whether such non-MEP cases had any tendencies related to compliance and the factors we have been investigating that might differ from MEP cases. It was added after the start of the research, and was intended as a small-scale first step in examining such cases. Names of non-MEP parents were requested from lawyers being interviewed in P.E.I. No attempt was made to randomize the selection of cases. Our hope was to interview about 40 parents, but ultimately only 10 names were provided that resulted in interviews, and these were all with recipient mothers. The low number resulted in part from the difficulty lawyers had in finding appropriate cases in their recent files, and in part because some of the parents referred to us could not be contacted.

This initial effort in P.E.I. made it clear that a more systematic and intensive procedure will be needed to pursue this line of research further. One potential source of non-MEP names is the MEPs themselves, because (at least in some provinces) parents leave the enforcement program and self-regulate their agreements. These cases would have had some contact with the legal system and the MEP, but would not necessarily have had any history of payments through their local MEP.

Another potential source are Family Court records. These, like the MEP sources, would not include cases with no involvement with the legal system at all, but in many cases the involvement could be minimal (for example, formal court approval of a separation or divorce agreement between the parents). The difficulty with court records as a source is that in many jurisdictions they are not computerized, or are not automated in such a way as to make the retrieval of information for research purposes easy. The cost of using court records would likely be high, and there might well be difficulties in maintaining randomness in the selection of cases.

A third potential source are surveys conducted by the Canadian Centre for Justice Statistics or by the Department of Justice Canada that identify separated and divorced parents and have some information on parenting arrangements. To use such sources, the permission of respondents would need to be requested at the time of the original survey, in order for them to be contacted for follow-up research.

6.5 Conclusions Regarding the Research Strategy

An important purpose of the P.E.I. research was to test research methods for the analysis of factors influencing child support compliance as a basis for moving ahead with a larger research project in other provinces. The central elements of the child support compliance research strategy were to use the MEP databases to identify categories of paying parents according to how regularly and fully they comply with child support orders and agreements, and to relate support payment records to information obtained from interviews with paying parents and recipients. With these elements, it was hoped that we would be able to examine the relationships between compliance and a range of factors with potential to influence paying parents’ willingness to pay child support. Interviews with professionals who work with separating parents with children were included to ensure that we understood the legal system and related government-based services that many separating parents encounter. We also wanted to obtain the views of those professionals about the factors influencing compliance, as a way to support and help explain findings from the parent interviews.

It is fair to say that the central elements of the strategy have proven effective, in that the methods we used provided us with the type of information we required at an acceptable cost. We have noted throughout the report that a larger sample of interviews would provide us with a greater ability to determine the strength of some of the relationships we have identified. As well, a larger number of interviews would enable us to examine some of the interactions among the factors we have analyzed, to provide a more complex picture of how paying parents make decisions about paying child support, and perhaps to provide us with some “paying parent profiles” that combine payment records and influencing factors. Finally, we have learned much about the logistics of undertaking this research, and have recognized the need for some modifications in the approach, such as certain changes to the parent interview guides and obtaining the MEP case file data in an automated fashion. However, the methods on the whole appear to be sound and would be reproducible on a larger scale in other provinces.

The interviews with professionals proved useful, but are not viewed as critical to the central question of what factors influence compliance and how. Also, it is not clear from the P.E.I. experience whether it would be necessary to interview a sizeable sample of professionals in all jurisdictions in order to be aware of the importance of different operating environments. Nor is it clear whether operational criteria would be more useful in selecting the sample of professionals. For example, it might be adequate to include in a larger sample some professionals operating in a unified family court and some in the traditional superior/provincial court model, some in major urban centres and some in smaller centres, and some in locations where mediation, parenting education and other services are widely available or even mandatory and others where such services are not readily available. We can conclude that some professional interviews should be included in future research, but that they need not be budgeted for all jurisdictions, at least until a more modest approach based on operational criteria is tested.

It is also the conclusion of the review of the research strategy in P.E.I. that research involving non-MEP parents should not be viewed as a core element of the strategy, but that if sufficient resources are available, consideration should be given to one or more of the suggested alternative approaches in one jurisdiction. The purpose would be to make some progress in an area recognized as important because of the large number of potential families affected, but without taking the focus of this project away from the MEP-based approach, where we know that quality information is accessible.