Victims of Crime Research Digest, Issue No. 4

The 2009 General Social Survey on Victimization in the Territories: Lessons Learned

  • Luke Pelot, Chief, Labour Statistics, Statistics Canada
  • Catherine Allan, Project Manager, General Social Survey, Statistics Canada
  • Jodi-Anne Brzozowski, Chief, General Social Survey, Statistics Canada
  • Patrick St-Cyr, Senior Methodologist, Household Survey, Methods, Statistics Canada

This article is adapted from the report 2009 General Social Survey on Victimization in Yukon, Northwest Territories and Nunavut, Collection and Evaluation Report.

The Statistics Canada General Social Survey (GSS) program, established in 1985, conducts telephone surveys across the ten provinces. The GSS is recognized for its regular collection of cross-sectional data that allows for trend analysis and for its capacity to provide information on specific social policy issues of current or emerging interest.

In 2009, Statistics Canada conducted the victimization cycle of the GSS for the fifth time. The purpose of the survey is to collect information on the nature and extent of criminal victimization as provided by Canadians. In addition, it examines risk factors associated with victimization, reporting rates to the police, and how Canadians perceive crime and the criminal justice system.

The main sample of the 2009 survey was distributed over the ten provinces and a supplementary survey was conducted in the three territories during the fall of 2009. This article provides an overview of the collection strategy in the North, the results of the data quality evaluation, and recommendations on the uses and limitations of the data.

Introduction

For many years, territorial governments have emphasized the importance of including their populations in the General Social Survey (GSS) cycles on self-reported victimization. The territories have historically been limited to official police and court statistics to inform policy decisions related to justice issues. Beyond these sources, there has been little information about the scope and nature of victimization in the North.

Due to the challenges associated with conducting surveys in the North, previous efforts to collect victimization data have yielded modest results. In order to improve the quality of the 2009 GSS data for the territories, innovative methods were developed which included using multiple surveys for the sample files and some face-to-face interviews where telephone coverage was poor.

The development and implementation of the strategy involved input from a wide variety of internal and external partners who provided advice and guidance on methodological issues, content development, collection planning, and monitoring.

Background

Given the territorial governments' high priority need to understand victimization in the territories, efforts were made in previous victimization cycles to pilot the collection of data in the territories. The results were mixed. Collection in Yukon, Nunavut, and the Northwest Territories poses unique challenges, and as a result, obtaining a representative sample in each of the territories is more difficult than in other parts of Canada. Some of these challenges include

  • incomplete telephone coverage (e.g., persons without a land-line telephone);
  • high levels of response burden due to the small populations;
  • language barriers;
  • difficulties reaching small, remote communities because of limited transportation services and weather conditions;
  • high mobility of the population and high turnover in telephone listings; and
  • less reliable telecommunications links in some areas.

In 1999, the GSS collected its first pilot test data in the territories using the Random-Digit Dialing (RDD) method. This is the method which is used to select the GSS sample in the provinces. Following a detailed evaluation of the results, it was recommended that data from this northern pilot test not be released because of bias caused by substantial undercoverage.[1]

A second pilot test was conducted in 2004. Collection was again done by telephone, but to improve the coverage of the survey, the sample was selected from respondents to the 2003 Canadian Community Health Survey. Although the data were releasable with cautionary notes, the global response rates in the territories were lower in each of the territories than the provincial results. In addition, Aboriginal people and individuals living in more remote communities were underrepresented in the sample.[2]

In preparation for the 2009 GSS on victimization, the topic of data collection in the territories was revisited in consultation with focal points[3] from the territorial statistical agencies, Statistics Canada methodologists, subject matter experts, survey operations specialists, and regional offices. It was decided that, based on the experience of previous attempts in the North, further efforts to improve the quality of the data in the territories were required. This would be done through a multi-source listing strategy and some face-to-face interviews.

Overview of Data Collection Strategy

Data collection efforts in the North in 2004 and 1999 revealed that telephone interviews were not sufficient to produce reliable estimates. Telephone penetration rates were lower than in the provinces, particularly in Nunavut, where at least 20% of households did not have a regular land line in 2005.

As RDD had not worked well in the past, it was felt that an area frame[4] would be a better approach to obtaining adequate coverage in combination with an effort to reach households with no telephones. Survey results are prone to bias if this is not done. Furthermore, in addition to telephone interviews, some personal interviews would also be conducted. There was a strong recommendation from Inuit associations that personal interviewing was more appropriate than telephone interviewing for the North. This had also been the experience of the post-censal surveys.

The 2009 strategy for data collection in the territories, therefore, consisted of telephone interviews in areas with acceptable telephone coverage and face-to-face or personal interviews in other areas. In order to support the alternative approach for the data collection, funds were obtained from Statistics Canada, the Department of Justice Canada, and the Policy Research Data Group (PRDG).

Target Population

The target population for the GSS Cycle in the North consists of all residents of the territories aged 15 years and over who are not living in institutions. The survey targeted households, and after the completion of the roster, a single eligible member of each sampled household was randomly selected by the application to complete the questionnaire.

Sample Size

The GSS budget normally provides for approximately 25,000 respondents (completed interviews) in the provinces. For the 2009 GSS, resources for 1,500 respondents (completed interviews) were reallocated from the provinces to the territories. The target of 1,500 was based on results from the previous cycles and with the objective of obtaining good estimates of victimization rates for each territory.

GSS methodologists together with the survey team planned for 80% of the interviews to be carried out by telephone and the remaining 20% to be conducted face-to-face. The latter were planned for communities where telephone coverage was poor and where previous attempts had identified undercoverage of certain subpopulation groups.

Sample Selection

The sample for the victimization survey in the territories was drawn from an area frameof households which had completed the 2007-2008 Canadian Community Health Survey (CCHS) or the 2007-2009 Labour Force Survey (LFS). This decision took into consideration the results of the 1999 Victimization pilot survey in the territories, the 2004 GSS on victimization in the North, and the findings in the report A proposed territorial strategy for households prepared by the Statistics Canada Joint Federal-Territorial Working Group. It was noted in the latter that RDD had not yielded acceptable results, but that the results from second phase surveys by telephone had been positive. The sample unit was the dwelling, as it was for the two source surveys.

It should be noted that due to operational difficulties inherent to remote locales, only the ten largest communities in Nunavut are covered by the CCHS. As the GSS sample was drawn in part from the CCHS, it was also limited to the same communities.

Personal and Telephone Interviews

The most significant change in the 2009 GSS in the territories was the inclusion of personal interviews in addition to telephone interviews. The decision to include personal interviews was based on the recommendation in the report referred to above that personal interviews are the preferred mode of collection in the territories. This decision was also based on the 2004 collection experience.

Input from Partners

Due to the new approach that was being developed for the survey in the North, in addition to the GSS victimization survey team, input and guidance was sought from within and outside of Statistics Canada throughout the development, collection, and production phases. Input from outside Statistics Canada was provided by key federal government policy departments, the Territorial Statistical Focal Points, and justice system stakeholders from the Northwest Territories, Yukon and Nunavut. All were involved in consultations on content development throughout the survey development period.

Collection Time Period

The period between September and December 2009 was identified by the Territorial Focal Points as the best time for collection in the North. Due to weather conditions, it was felt that January and February would not be good months for personal interviews, and given that many potential respondents would be out on the land, the April to June period was also not considered optimal. Collection was therefore conducted from August 31 to December 31, 2009.

Recommendations

The major recommendations from the qualitative analysis of the collection strategy include the following:

  • Based on the GSS experience, surveys in the North should include a personal interview component. This is most notably applicable to Nunavut, which had the lowest response rates, resulting in fewer publishable estimates.
  • Consideration should be given to expanding the collection period in order to maximize the likelihood of making contact with respondents.
  • For future victimization cycles, the Territorial Focal Points should be involved earlier in the consultation and content development phases. This would ensure that the territories' data needs for relevant content are addressed.

Quantitative Analysis of the 2009 GSS in the North

Quantitative analysis of the collection approach, data quality measures, and the sample representativity was carried out. The parameters and data quality measures which were examined included the distribution of telephone and personal interviews, response, non-response and slippage rates, adjustment of weights to improve representativity and calibration. The response rates for the 2009 and 2004 surveys are shown in Table 1.

Table 1: 2004 and 2009 GSS North Response Rates
Territory Responses Response rate (%)
2004 2009 2004 2009
Yukon 450 355 66.1 54.4
NWT 595 536 62.8 54.7
Nunavut* 245 203 44.8 38.7
All 1290 1094 59.3 50.7

* Nunavut's ten largest communities

The following are the findings and recommendations with respect to the data quality and on the uses and limitations of the data.

Findings:

  • Due to the revised sampling plan and changes in the collection approach, this evaluation indicates that the data collected in 2009 is, in general, more representative of the population in the territories than in 2004. This is true despite the lower response rates recorded in 2009.
  • While some population groups were found to be underrepresented in the 2009 sample for the territories, adjustments were made to help correct for these. With these adjustments, there is no identifiable bias in the sample when the three territories are grouped together.
  • With adjustments to account for underrepresentation for certain population groups, there is no evidence of bias in the sample for Yukon or in the sample for the Northwest Territories.
  • Even with adjustments, there is still some evidence of bias in Nunavut's ten largest communities due to the lower response rate and the underrepresentation of the Inuit population.

Recommendations:

  • Due to the major changes in survey collection, sampling methodology, and the quality of the estimates between the 2004 and the 2009 GSS surveys in the North, comparisons between results over the two survey periods should not be made.
  • In view of the different survey collection periods, collection modes, and the underrepresentation of the Inuit population, comparisons between the results from the 2009 GSS in the territories and the 2009 GSS in the provinces should be made with caution.
  • Estimates combining the three territories can be released in accordance with established Statistics Canada quality and releasability guidelines.
  • Estimates for Yukon and the Northwest Territories can be released in accordance with established quality and releasability guidelines.
  • Overall estimates for Nunavut's ten largest communities should be released with caution due to the underrepresentation of the Inuit population.
  • It is further recommended that reports containing analyses of the data from the territories also contain the following statement:

The 2009 GSS on victimization was conducted in Yukon, the Northwest Territories, and Nunavut using a combination of telephone and face-to-face interviews. Compared to other areas in Canada, conducting surveys in the territories poses unique challenges, including incomplete telephone service, language difficulties, high population mobility, and the remoteness of many communities.

Collection in Nunavut's ten largest communities was particularly challenging and resulted in undercoverage of the Inuit population and lower response rates than those in Yukon and the Northwest Territories. As a result, results from Nunavut should be used with caution.

Conclusion

While many of the data collection challenges outlined in this report will continue to exist in Canada's territories, it is hoped that the lessons learned through the experience of the collection of the 2009 GSS will be applied to future victimization surveys and other household surveys in the North.


  • [1] Housing, Family and Social Statistics Division, Testing in the Yukon and the Northwest Territories: Evaluation Repor,unpublished report from the 1999 General Social Survey, Cycle 13 Victimization, Ottawa, Statistics Canada, 2000.
  • [2] Social and Aboriginal Statistics Division, Household Survey Methods Division, Pilot Test in Yukon, Northwest Territories, and Nunavut: Evaluation Report, unpublished report from the 2004 General Social Survey, Cycle 18 Victimization, Ottawa, Statistics Canada, 2005.
  • [3] For Statistics Canada, statistical focal points representing the territories are the most general and comprehensive consultative mechanism with the territorial governments. Members represent the heads of the statistical agencies of their respective territory. While the same mechanism exists with the provinces, the nature and scope of this particular project was limited to the involvement of the territorial focal points.
  • [4] The survey frame provides the means of identifying and contacting the units of the survey population. There are two types of frames: list frames and area frames. A list frame lists all units in the survey population (e.g., a list of all addresses or telephone numbers). RDD is a list frame of telephone numbers. An area frame is a special kind of list frame where the units on the frame are geographical areas from which dwellings are selected.
Date modified: