Consultations on Physician-Assisted Dying - Summary of Results and Key Findings

Executive Summary

The results of the analysis here support the view that the External Panel on Options for a Legislative Response to Carter v. Canada (the Panel) can and should use the results of the consultation using the Workbook or Issue Book methodology as a reflection of the views of Canadians on the issue. Research by definition involves choices and the result of those choices will influence the nature of the findings and the conclusions drawn from them.

Clearly the Issue Book framed the issue for respondents in a unique way that differs from a traditional survey approach and this must be kept in minding in thinking about the questions and the responses. In addition, the needs and interests of the Panel, as reflected in its mandate, drove the consultation process. The results are not an open investigation into the question of physician-assisted dying.

Like all research there are areas of strength and weakness in the design and analysis provided.

• A strength of the process is clearly tied to its breadth of sample and the fact that while all Canadians were welcome to participate (and many did so), the open participation was balanced with a closed survey that was designed to mimic as close as possible the demographic character of Canada (general public sample). The Panel can situate the expert and stakeholder advice it received through the open Issue Book and other submissions within the broader Canadian public perspective.
• An obvious strength is the ability to make clear comparisons of how people view the issue across a range of scenarios with different informational prompts. While the scenarios could have been improved, they clearly function as a way of assessing relative support since the changes to the scenario create a comparative context for individuals. The reporting could have more clearly explained the purpose of the scenario process and the inferences being drawn for readers.
• A drawback is that a small change in the scenario could make different things salient for the individuals and change the distribution of opinion. As a result, responses must always be viewed through the lens of the scenarios. This is important since there is no measure of support for physician-assisted dying as a policy issue outside of the scenarios to contextualize the scenario information.
• The Issue Book contains a considerable amount of information and context for respondents both as part of, and separate from, the scenarios. While this review has not provided an analysis for bias of this content, the Panel should understand that this information (and the absence of other information) provide a critical and possibly controversial context for interpreting the results.

About the author

Richard Jenkins, Ph.d. has more than 15 years of market research experience, which includes working on more than 15 consultations involving workbook like exercises. He has been directly involved in more than 250 surveys. More about him is available in the appendix.

Background and Objectives

The External Panel on Options for a Legislative Response to Carter v. Canada (the Panel) engaged Richard Jenkins of Jenkins Research Inc. to conduct an independent expert review of the Issue Book Online Consultation that was conducted by the Panel. The consultation was designed to bring public input into the Panel’s work process.

The review is to include both the overall methodology and the specific questionnaire/ workbook design used to capture the input of Canadians.

The following specific tasks were conducted:

• Review the questionnaire Issue Book used to capture feedback along with the description of the methodology used overall.
• Review the report based on the public input with particular emphasis on how the information is presented including any qualifications on the data validity and interpretation.^{Footnote 1}
• Speak with key stakeholders involved in the construction of the workbook as necessary.^{Footnote 2}
• Review existing external critiques of the methodology.^{Footnote 3}
• Write a report that reviews the online consultation and identifies the following:
  • Advantages and disadvantages of the research tool used and overall approach; and
  • An opinion on the value and reliability of the data collected and reported.

The approach taken in this expert review takes as its starting point that it is essential to understand three questions when approaching a review of a research program:

• Was the study conducted in a manner consistent with the principles of the research plan? In other words, was the workbook process internally consistent with accepted principles of this form of research?
• Is there any reason to believe that the research was designed to be biased in a particular direction?
• Is the analysis and recommendations supported by the evidence? This includes not only whether the recommendations flow from the actual answers but also whether this type of study can support these recommendations.

It should be noted, that while Richard Jenkins is an expert in survey-based methodologies and consultations, he has only a general knowledge of the issue of physician-assisted dying.

Analysis of the Methodology

The Issue Book Online Consultation conducted by the Panel involved the development of an issue book that was designed to develop an understanding of how Canadians think about the issue of physician-assisted dying.

The best starting point for any discussion of the methodology used to capture public input in this case is to understand that the approach differs significantly from a traditional public opinion surveys. While surveys are usually defined in terms of a set of usually neutral questions designed to capture public views on issues, the workbook approach is designed specifically not to be just a measure of what people think but to go beyond that to better capture the role of information and exposure to arguments.

The root of a workbook process has its origin in ideas about deliberative democracy and the idea of deliberative polling in which a representative sample is brought together and provided information and a chance to express views.^{Footnote 4} While the Issue Book process did not involve a coming together of people for deliberation, it did attempt to do more than ask questions. Canada has a history of work book type consultation processes.

Advantages/ Benefits of a Workbook Process:

• On topics that people have not thought much about, the provision of information can ensure that those completing the consultation are all working from the same-shared perspective and that shared-perspective helps respondents evaluate specific questions more easily than they could in the absence of information.
• It ensures that those who are answering are addressing the specific policy choices that are being considered and not just broad value-based answers.

Disadvantages/ Drawbacks of the Workbook Process:

• The results because they are based on a process with information provided to respondents may not be consistent with other surveys of the population.
• By definition, there is considerable influence of information and structure of the workbook on the results. If in the design, key information or policy options are not represented well the results will be biased as a result.

The approach used clearly differs from traditional surveys but is a proven approach for generating deeper more thoughtful insights into public policy.

Analysis of the Sample

Who participates in the consultation is an important factor in the research but the approach taken for the Panel achieves the best of an open consultation and a controlled public opinion exercise.

Two separate groups or samples of Canadians completed the same instrument and the data is treated separately in analysis.  An open public group includes anyone who learned about the consultation, through a variety of means and self-selected themselves to complete the workbook. A second, controlled sample of the general public was drawn from an online panel and weighted to reflect the population of Canada.

The reason for two separate samples is that the open public is more along the lines of a traditional consultation where anyone interested can speak. This group is more engaged on the topic and will be more vocal in public discourse. The Panel should, as it would with all submissions it receives from interested parties, understand that individuals and groups were motivated to participate.

The general public sample is designed to capture what average Canadians think on the topic. Using an online panel, which in this case was outsourced to Leger Marketing is a reasonable approach. Online panels are a standard market research tool in which participants are recruited, using a combination of methods, to take online surveys as needed by the company. While people self-select themselves into most research panels, they do not do so on the basis of any particular public policy issue (no one knew that they needed to be on the Leger panel in order to be invited to take the Issue Book) so the panel should not have an inherent policy bias. In addition, the data for the general public was weighted so as to ensure that the profile of respondents was a reflection of the Canadian population.

Observations:

• Drawback: Using an online panel, which is not randomly recruited, does not allow for the use of a margin of error or other probability statistics.^{Footnote 5}
• Strength: Online research is a widely used and generally accepted practice for conducting cost effective market and policy research.

The approach to the two samples is consistent with best practices in the industry and should not be a source of significant risk in terms of interpreting the results. While one might prefer a randomized recruitment method, the Panel should feel comfortable in assessing the data from the two samples.

The Issue Book

The structure of the Issue Book includes four sections. After capturing demographic and other profile information about the respondent, a set of scenarios are used to develop an understanding of the conditions under which there is support for physician-assisted dying. The Issue Book then separately considers risks and then safeguards. Building the workbook in this manner makes conceptual sense, especially given the mandate of the Panel.^{Footnote 6}

In evaluating the workbook or any questionnaire, one needs to be conscious that the very act of asking questions will shape the answers and data that emerge. Choices about words, scale points, and language clearly have effects. There is no true opinion that just needs to be measured with the right tool. For this reason, the fair approach to take in evaluating the workbook is not to critique questions in terms of the construction, unless it is clearly egregious in producing bias, but rather to understand how the questions and structure limit or constrain the analysis and the confidence one can place in the results.

• Scenarios: There are four overarching scenarios that cover things from a significant life-threatening illness to a mental health condition. The purpose of the scenarios is to focus on “who should have access to physician-assisted dying” (page 8). Within each scenario, there are additional scenarios that qualify the overarching scenario.
  • It is noteworthy that in the scenarios there is no description of why in the scenario the person should or should not have eligibility. The expectation is that respondents will formulate an explanation based on the information given. For example, in the mental health scenario there is no rationale for why this person might be given the ability to do so or the right not to.
  • There are two strengths of this approach: (1) respondents have a concrete situation to evaluate; (2) it is possible to observe the impact of adding context to the other scenarios on support (for example, the strongly agree for scenario 2 drops from 13% to 6% for the representative sample when information is added that the event was recent).
  • Drawbacks. A drawback of this approach is that the scenarios can be subject to criticism in terms of their construction and the associated information (for example McRuer 2015). There are grounds for suggesting that the scenarios could have been better constructed and less ambiguous.  Each of the scenarios does require a respondent to make assumptions about information not included in the scenario (e.g. when a person is going to have a physician help them die).  There are no perfect ways, however, of constructing scenarios.
  • Choice of scenarios and qualifications. Constructing 4 overarching scenarios, with various other specific scenarios, that are both realistic and which cover off a variety of potential real-life occurrences where physician-assisted dying might be relevant is obviously challenging and opens the process to confusion and criticism. While the scenarios appear to stretch the ideas expressed in the Court decision (based on the information provided to the respondent), there is value in understanding what Canadians are comfortable with and what they are uncomfortable with. A better explanation in the report of why these scenarios were used and the nature of the inferences being made based on how people responded to the overarching and contextual sub-scenarios would improve the report.
• Risks: The section on risks assesses risk in terms of the level of concern with a set of more than 30 different risks for patients, patients with disabilities, persons with mental health conditions, palliative care, and society at large.
  • Strengths: there is a breadth of coverage and the items allow us to have a more nuanced understanding of how Canadians think of the impact of physician-assisted dying.  Since the items are all on the same scale, respondents will be evaluating them as a series of concerns.
  • Drawbacks:
    1. Concern is being used to capture general negative outcomes that could emerge from permitting physician-assisted dying but some of the items are multi-dimensional and we have no way to gauge how big an actual problem the public thinks each of these risks are.
    2. There is a framing of these as risks but they may be more appropriately understood as impacts (e.g. “normalizing societal views on assisted dying as an alternative to other ways of addressing suffering” may occur but depending on position on the issue you may see this as a positive not as an area of concern).
• Safeguards: The final substantive section of the Issue Book covers the importance that Canadians assign to the presence of 22 safeguards that include such things as waiting periods and public awareness campaigns.
  • Strengths: While the number of questions is long, they cover off in a manner that allows respondents to effectively identify important versus unimportant safeguards which are a reflection of both the policies they would want but also the concerns they likely have about physician-assisted suicide.
  • Drawbacks: Many of the questions come with explanatory information that is designed to give respondents extra context for their assessment. It would be reasonable to expect that this content has shaped the answers and this content should be reflected in the report since it is misleading to only include the text of the question. Most of these explanatory statements, in this authors view are designed to explain why the safeguard would be needed so it would be misleading to assume that these are fixed views of Canadians on safeguards. If presented with drawbacks or reasons for not having them, the results might be different.

The questions in the workbook could certainly have been improved to better capture public views on the issues but on balance it is my expert opinion they do not prevent one from drawing reasonable inferences about the state what people care think given the information presented to them. The report, which focuses more on the relative importance of different factors and/ or the impact of information, tends to draw inferences and conclusions at a high level.

It should be noted that there is a risk that other people may interpret the results in a manner that does not recognize the limitations of these question types. As such the report could include a reflection on the best use of the data.

Conclusions

It is worth returning to the three broad questions posed at the beginning of this review.

Was the study conducted in a manner consistent with the principles of the research plan? In other words, was the workbook process internally consistent with accepted principles of this form of research?
Response: Yes. While not everyone agrees that a consultative approach like this would be useful or beneficial, the instrument is available, the samples were created and reported using reasonable standards and the report does not draw inferences that are not supported by the data presented.

Is there any reason to believe that the research was designed to be biased in a particular direction?

Response: There is no clear bias in the overall research plan beyond the bias introduced by appearing to focus on the terms of reference for the Panel. The use of the Supreme Court result to frame the information no doubt contributes to a structure of the Issue Book that might be different if the starting point was some other place.  The nature of the safeguard questions with their emphasis on providing information that explained the purpose of the safeguard may, for example, reflect the focus on the terms of reference.

Is the analysis and recommendations supported by the evidence? This includes not only whether the recommendations flow from the actual answers but also whether this type of study can support these recommendations.

Response: Yes. The overall analysis as presented in the report is consistent with the data presented. It would be helpful to have more analysis and more explicit recognition in the report of the limitations and purposes of the questions as asked and posed. 

Appendix: About Richard Jenkins

Previous experience:

Vice President and Corporate Director of Public Opinion Research, TNS Canadian Facts (2003-2009)

A member of the Senior Management Team for TNS Canadian Facts and on the global sector committee for TNS Global, Dr. Jenkins was responsible for the overall intellectual direction and project management of all public sector (including crown corporations) work.  This work included traditional public opinion research, market research for Crown Corporations and program evaluation work.

Senior Consultant, EKOS Research Associates, 2001-2003

Responsibilities included the project management of three on-going syndicated research products between 2001 and 2003 (Rethinking Government, Rethinking Citizen Engagement, and the Public Security Monitor), in addition to custom quantitative survey research for clients. Tasks included marketing, client liaison, research design, analysis (including advanced segmentation techniques), reporting and presentation of results to senior managers.

Limited Term Assistant Professor and SSHRCC Post-Doctoral Fellow, 1999-2001, Queen’s University (Kingston, Ontario)

Teaching responsibilities included undergraduate and graduate courses in statistics and research methods, elections and public opinion, and Canadian politics. Research and publications focused on the secondary statistical analysis of public opinion data relating to public policy, and Canadian attitude and values.

Honours

Social Science and Humanities Research Council Postdoctoral Fellowship (1999-2001)
Social Science and Humanities Research Council Doctoral Fellowship (1997-1999)
University Graduate Fellowship (1996-1997)

Publications

Goodman, S., D. Hammond, F. Pillo-Blocka, T. Glanville, and R.W.  Jenkins. (2011) “Use of Nutritional Information in Canada: National Trends between 2004 and 2008.” Journal of Nutrition Education and Behavior, 43(5), 356-65.

Rudin-Brown, C.M., Jenkins, R.W., Whitehead, T., & Burns, P.C. (2009). “Could ESC (Electronic Stability Control) change the way we drive?” in Traffic Injury Prevention, 10(4), 340-347.

Jenkins, Richard. 2007. “Voters Lack Commitment: Canada” in Public Opinion Polling in a Globalized World. Carballo and Hjelmar eds. London: Springer. pp 153-173.

Jenkins, R.W.(2005) “A Lack of Commitment: The Key to Voter Turnout” in Canadian Journal of Market Research. 22.1: 15-21.

Jenkins, R.W. (2005) “Mass Media and Third Party Insurgency” in Richard Johnston and Henry Brady eds. Capturing Campaign Effects. University of Michigan Press.

Jenkins, R.W. (2003) “More Bad News: The Uneasy Relationship between the Reform Party and the Media” in The Media and Neo-Populist Movements: Processes of Media Construction of Political Reality, edited by G. Mazzoleni, B. Horsfield & J. Stewart (Praeger).

Graves, F. and Jenkins, R.W. (2002) “Canadian Attitudes Towards Productivity: Balancing Standard of Living and Quality of Life” in Andrew Sharpe, France St-Hilaire and Keith Banting, eds., The Review of Economic Performance and Social Programs 2002 (Montreal: IRPP).

Jenkins, R.W. (2002) “How Campaigns Matter: Priming and Learning as Explanations for Reform’s 1993 Campaign Success” Canadian Journal of Political Science, 35(2).

Jenkins, R.W. and Cutler, F. (2001) “Where One Lives and What One Thinks: Implications of Rural-Urban Opinion Cleavages for Canadian Federalism,” State of the Federation 2001. (Institute of Intergovernmental Affairs: Kingston).

Jenkins, R.W. and Mendelsohn, M. (2001) “The Media and Referendum Campaigns” in M. Mendelsohn & A. Parkin eds., Referendums and Democracy: The Impact of Referendums on Liberal-Democratic Politics. (Palgrave).

Jenkins, R.W. (2001) Book Review of Seeing Spots: A Functional Analysis of Presidential Television Advertisements, 1952-1996 (Westport, CT: Praeger, 1999) in Canadian Journal of Political Science, 34(1).

Jenkins, R.W. (2001) “The Media, Voters, and Election Campaigns: the Reform Party and the 1993 Election” in Joanna Everitt and Brenda O’Neil eds. Political Behaviour: Theory and Practice in a Canadian Context (Toronto: Oxford).

Johnston, R., Fournier, P. and Jenkins, R.W. (2000) “Party Location and Party Support: Unpacking Competing Models” Journal of Politics. 62 (4): 1145-60.

Jenkins, R.W. (1999) “How Much is Too Much? Media Attention and Popular Support for an Insurgent Party,” Political Communication 16 (4): 429-445.

Jenkins, R.W. (1998) “Untangling the Politics of Electoral Boundaries in Canada, 1993-1997,” American Review of Canadian Studies.  28 (4): 517-38.

Jenkins, R.W. (1996) “Public Opinion, the Media and the Political Agenda” in James Guy ed., Expanding Our Political Horizons: A Journal of Readings in Canadian Politics (Toronto: Harcourt Brace).