Archived - Evaluation of the CFIA's Stakeholder Consultation Processes

Management Response

October 2011

Executive Summary

Stakeholder consultation is essential to inform and provide stakeholders with opportunities for input on issues of concern and to include them in the decision-making process. For the CFIA, stakeholder involvement also enables it to fulfil its legislative and regulatory mandate and achieve objectives, develop public policy, design and deliver programs and build public trust. As a regulatory agency, the CFIA is required to consult with stakeholders throughout the regulatory life cycle.

An evaluation of the CFIA's consultation processes was requested by senior management as part of the Agency's regular oversight of its key activities as well as to provide an update on its work respond to Recommendation 47 of the Weatherill report.^Footnote 1 The evaluation assessed the relevance and performance of CFIA's stakeholder consultation mechanisms and processes, focussing on the CFIA's regulatory obligations. The scope of activities included formal consultations in support of proposed regulatory, policy and program changes. The evaluation excluded communications that are primarily for the purpose of education and consultation during emergencies.

The evaluation covered the five year period from 2006/07 to 2010/11 and was conducted over the period of March to July 2011.

Conclusions

Consultation with stakeholders is a good practice for policy development in general, and stakeholder consultation is a mandatory activity for federal regulatory agencies, with a strong foundation in policies and guidelines. The CFIA has a strong history and culture of consultation, with significant skills and expertise.

Reach to stakeholder groups is generally good with recent efforts to increase engagement of consumers. Stakeholders broadly view the CFIA's consultations as open, meaningful and balanced. Stakeholder consultations influence regulatory, policy and program development. In some cases, the lag between consultation and implementation, combined with a lack of feedback, affects goodwill.

Responsibility for the design and conduct of consultation is dispersed within the CFIA. This allows for flexibility in approach, responsiveness and good technical expertise. However, this also results in inconsistent processes, inefficiencies and unevenness in the conduct of consultation activities. The lack of a focal point in the Agency hampers sharing of best practices and the development and sharing of tools.

Relevance

• Consultation with stakeholders is a good practice for policy development in general, and as a regulatory agency, the CFIA is obliged to consult with stakeholders who may be affected by regulations, policies and programs. There is a strong policy basis within the federal government, and the CFIA has recently developed a supporting consultation policy and framework.
• The Agency consults on proposed regulatory or program changes, aligned with Government of Canada and CFIA priorities. There may be additional opportunities to consult on strategic issues of interest to the CFIA. Management of stakeholder expectations is also a factor.
• Overall, the CFIA's consultation policy and conduct of consultations are consistent with government policies and guidelines. Some minor areas of weakness were identified in terms of use of the "Consulting with Canadians" website, providing feedback, and evaluating consultations.

Performance - Outcomes

• Consultation activities have traditionally focused on regulated parties, industry and other government departments and levels of government. International organizations and other governments are regularly engaged. Consumers have not been well represented in the past though there is recent effort to address this. Other groups that may be less well represented include small and medium sized businesses, and Aboriginal peoples.
• Overall, the CFIA is meeting the policy goals of informing stakeholders and gathering input, though there are inconsistencies. Improvements to transparency could be realised with improved feedback and better visibility of consultations. Managing expectations raised through the consultation process is important.
• Input received from stakeholders is considered and influences regulatory and program changes to varying degrees. Industry associations are able to see their influence and appreciate that not all recommendations can be accommodated.
• Stakeholders broadly view the Agency as open, meaningful and balanced. Elapsed time to implementation combined with a lack of feedback impacts goodwill in some instances.
• Roles and responsibilities for consultation are dispersed across branches and are generally well understood. A gap exists in the provision of advice and the development and sharing of tools.
• Generally consultations are completed when the regulatory response is partially developed, though there is a trend towards engaging stakeholders earlier in the regulatory life cycle. Less formal consultation is being conducted post-implementation.

Performance - Efficiency and Economy

• Consultations generally occur in a timely fashion though comment periods are at times too short for associations to consult with their respective members adequately. Limited coordination mechanisms exist within the CFIA, resulting in inefficiencies and relearning; however, this does not appear to be affecting stakeholders significantly. No major overlap or duplication was identified, though some industries are engaged through multiple mechanisms.
• The CFIA uses a range of mechanisms to engage stakeholders. Assessment and sharing of what mechanisms are most effective in which situations ("what works best when") could be improved. Additional capacity and support is required to better use web-based mechanisms. Opportunities exist to create a better coordinated approach to consultation, which may result in efficiencies based on sharing best practices and tools.

Recommendations

Given the strength and breadth of consultation skills and experience within the Agency, a focal point is required in order to operationalize the Consultation Policy and to improve the consistency and efficiency of stakeholder consultations.

1. Support Agency consultation activities by:
   1. Providing coordination, advice and tools within CFIA and a source of expert advice for Agency personnel;
   2. Providing key coordination mechanisms for stakeholder consultation such as, an Agency-wide list of stakeholder organizations (explore Agriconnexions) and a schedule of major consultation activities;
   3. Enhancing mechanisms for consultation, specifically, more consistent use of the Consulting with Canadians website, enhanced web-based tools and improved visibility of consultations on the CFIA external site, and;
   4. Promoting standardization and sharing of best practices through activities such as: development and promotion of a toolkit; creating a resource centre for consultation on the CFIA's intranet; and establishing a community of practice within the CFIA.
2. Develop and implement guidelines for providing feedback to participants of consultation activities.
3. Continue to enhance engagement of consumers and small and medium sized businesses and engage less traditional stakeholder groups, such as Aboriginal Peoples and vulnerable populations as required.

Table of Contents

• Executive Summary
• 1. Introduction and Context
  • 1.1 Rationale and Policy Context
  • 1.2 Target Population: Who are the CFIA's Stakeholders?
  • 1.3 Stakeholder Consultation Mechanisms and Processes
  • 1.4 Governance and Resources
  • 1.5 Logic Model
• 2. Evaluation Methodology
  • 2.1 Evaluation Scope and Framework
  • 2.2 Data Collection
  • 2.3 Data Quality and Limitations of the Evaluation
• 3. Relevance
  • 3.1 Continued Need for the Activity
  • 3.2 Alignment with Government Priorities
  • 3.3 Alignment with Federal Roles and Responsibilities
• 4. Performance
  • 4.1 Achievement of Expected Outcomes
  • 4.2 Demonstration of Efficiency and Economy
• 5. Conclusions and Recommendations
  • 5.1 Conclusions
  • 5.2 Recommendations
• Annex A - Stakeholder Consultations Logic Model
• Annex B - Advisory Committee Members
• Annex C - Evaluation Matrix
• Annex D - Listing of Documents Reviewed
• Annex E - Interview Guides

List of Acronyms and Abbreviations

AAFC

Agriculture and Agri-Foods Canada

AERO

Audit, Evaluation and Risk Oversight

CAR

Consumers Association Roundtable

CCB

Communication and Consultations Branch (CCB)

CDSR

Cabinet Directive on Streamlining Regulations

CFIA

Canadian Food Inspection Agency

CFPF

Canadian Fertilizer Products Forum

Codex

Codex Alimentarius

CSD

Consultation Services Directorate (Health Canada)

DFAIT

Department of Foreign Affairs and International Trade

EC

Environment Canada

FAO

Food and Agriculture Organisation (of the United Nations)

F/P/T

Federal / Provincial / Territorial

FPTFSC

F/P/T Food Safety Committee

FTE

Full Time Equivalent

HC

Health Canada

HPFB

Health Products and Food Branch

IT

Information Technology

NAAHP

National Aquatic Animal Health Program

OCAPI

Office of Consumer and Public Involvement

OECD

Organisation of Economic Co-operation and Development

OIE

World Organisation for Animal Health

PHAC

Public Health Agency of Canada

POR

Public Opinion Research

P/T

Provincial / Territorial Governments

RIAS

Regulatory Impact Analysis Statement

TC

Transport Canada

TORs

Terms of Reference

VCRT

Value Chain Round Tables

WHO

World Health Organisation

WTO

World Trade Organisation

1. Introduction and Context

1.1 Rationale and Policy Context

Stakeholder consultation is a good practice in the development of public policy in general. From the CFIA perspective, stakeholder involvement enables it to fulfil its legislative mandate and achieve objectives, develop public policy, design and deliver programs, and build public trust.^Footnote 2

As a regulatory agency, the CFIA is required to consult with stakeholders throughout the regulatory life cycle.^Footnote 3 In support of this, and of Recommendation 47 of the Weatherill report,^Footnote 4 the CFIA recently developed a Consultation Policy and Framework for the Canadian Food Inspection Agency. The Policy Statement contained in the document states that:

"The Canadian Food Inspection Agency (the CFIA or Agency) will undertake consultation with stakeholders, both regulated and non-regulated, on significant policy and program issues that impact them."

The Regulatory Impact Analysis Statement (RIAS) requires a summary of consultation undertaken for pre-publication in Gazette I. The RIAS Writers Guide^Footnote 5 indicates that:

"Pre-publication is also not a substitute for consultation. The groups that are most affected by the regulation should have been consulted before pre-publication. Pre-publication is intended only as a final check to give all interested parties a last opportunity to become informed about the government's planned course of action."

1.2 Target Population: Who are the CFIA's Stakeholders?

In the context of consultation on regulation, "…[b]roadly defined, stakeholders are individual, groups, or organizations likely to be affected by proposed regulatory changes."^Footnote 6 The CFIA's stakeholders can be grouped as follows:

• Regulated Parties: individual companies and representative industry associations;
• Consumers/Canadians: individual consumers and consumer associations;
• Government Departments/Agencies: other interested federal government departments/agencies, provincial and territorial (P/T) governments and municipalities; and
• International: foreign governments and international organizations.

In addition, the CFIA consults with other experts such as academics and researchers with specific technical expertise.

1.3 Stakeholder Consultation Mechanisms and Processes

The CFIA uses a mix of permanent forums as well as temporary processes to engage stakeholders.

• Regulated Parties: The CFIA regularly consults with industry to identify priorities and challenges in industry sectors, and to gather feedback from industry on various programs and initiatives.
• Consumers/Canadians: The CFIA consults with consumers primarily through the Consumers Association Roundtable or through associations.
• Government Departments and Agencies: Consultation with other government departments and levels may be done informally or through consultative and advisory committees.
• International: The CFIA engages foreign governments and international organizations independently as well as jointly with other federal government departments.
• Other Experts: CFIA collects academic expert advice through a variety of permanent and temporary mechanisms such as holding discussions, commissioning research studies, hosting technical workshops with academia, industry, and agricultural or agronomic experts from Canada and other countries or establishing advisory panels.

1.4 Governance and Resources

CFIA branches engage independently with the Agency's external stakeholders for their own purposes. Currently no single branch has overall responsibility for strategic oversight and coordination of the CFIA's consultation activities with external stakeholders.

All public opinion research (POR) projects are coordinated through the Corporate Communications Directorate, Public Affairs Branch. In addition, Public Affairs Branch has assumed responsibility for consultation with consumers, including the coordination of the Consumers Association Roundtable. In 2007/08, a consultation office existed for a short period within the Corporate Secretariat, but was not sustained.

Due to the spread of consultation activities throughout the Agency, estimates of resources were not possible.

1.5 Logic Model

A logic model was developed in support of the evaluation, based on the current draft CFIA Consultation Policy. It is contained in Annex A.

2. Evaluation Methodology

2.1 Evaluation Scope and Framework

The evaluation assessed the relevance and performance of CFIA's stakeholder consultation mechanisms and processes. The scope of activities included formal consultations in support of proposed regulatory, policy and program changes. The evaluation excluded communications that are primarily for the purpose of education and consultation during emergencies.

The evaluation covered the five year period from 2006/07 to 2010/11, and was conducted between March and July 2011. An advisory committee provided strategic and technical advice to the Head of Evaluation and identified important evaluation issues (see Annex B for members).

The evaluation strategy matrix in Annex C contains the details of evaluation questions, indicators and data sources and collection methods. The issues and questions are summarized here.

2.2 Data Collection

Data collection consisted of the following elements, with further details outlined below:

• Document, file and web review
• Key informant interviews
• Best practices review
• Case studies

Document, File and Web Review

The document, file and web review included items such as (see Annex D for details):

• Consultation policies and guides;
• Internal working documents such as manuals, tools, templates, reports;
• Consultation outputs such as planning documents, postings, reports; and
• Web sites including CFIA's internal and external site, Canada Gazette, other government departments.

Key Informant Interviews

Key informant interviews were conducted internally with CFIA staff and managers as well as with external stakeholders. Interview guides were developed and circulated prior to interviews (see Annex E for the interview guides). Interviews were conducted with a total 21 people within CFIA, 4 with other government departments and 18 with associations. No survey was conducted due to the absence of a central list of stakeholders within the CFIA.

Best Practice Review

A review was conducted of academic and general publications on the approach to stakeholder evaluations in a regulatory environment. The review was not intended to be in-depth, rather the purpose was to identify potential best practices.

In addition, the consultation practices of other federal government departments were reviewed, including:

• Partner Departments: Health Canada, Agriculture and Agri-Food Canada
• Other Regulatory Agencies: Environment Canada, Transport Canada (web review only)

Case Studies

Three case studies, one from each business line, were conducted of completed consultations. The focus of the case studies was to inform the evaluation questions related to reach, program design, efficiency, and success. The mix of case studies included both permanent and temporary activities, and reflected different stakeholder groups. The topics of the case studies were:

2.3 Data Quality and Limitations of the Evaluation

Overall data quality was good, with multiple lines of evidence employed. Consistency of findings across lines of evidence was very good. However, the view of stakeholders was limited to interviews with a sample of industry associations and consumer associations. A broad survey of all stakeholders was planned, but the lack of a central contact list prevented the conduct of the survey. In response, the number of interviews was increased from 5 to 18. Some stakeholder groups were not represented such as provincial and territorial governments, and Aboriginal groups.

3. Relevance

3.1 Continued Need for the Activity

Is there a continued need for stakeholder consultation?

Consultation is a formal process for sharing information, getting public input, analysing the input and using it to inform policy development, and to develop effective solutions. The process helps identify the range of affected parties, minimize the risk of unexpected consequences; and to discover better implementation methods. Involving interested parties in policy development is also effective in increasing trust and engagement by promoting transparency and accountability, and improving awareness and understanding of the policy area. Consultation is also a tool to encourage public ownership of policies, thereby increasing public commitment. In an organization such as the CFIA, consultation is an important component of ensuring that regulated parties are committed to the basic intent of regulations, and thereby supporting compliance and enforcement.

The 2007 Cabinet Directive on Streamlining Regulation (CDSR) obliges all departments and agencies involved in the federal regulatory process to provide affected parties an opportunity to "[…] take part in open, meaningful, and balanced consultations at all stages of the regulatory process." The associated 2007 Guidelines for Effective Regulatory Consultation provides interpretation and guidance to departments in the implementation of CDSR, placing consulting, coordinating and cooperating at the centre of the regulatory life cycle as depicted in diagram 1 below.

Diagram 1: Regulatory Life Cycle^Footnote 7

The Guide to the Cabinet Directive on Law-Making notes that when a department is considering regulation, it should "…engage in consultation with those who have an interest in the matter, including other departments that may be affected by the proposed solution". The Strategic Environmental Assessment guide recommends analysis of the impact on the public. Specifically, "the analysis of potential environmental effects should indicate, where appropriate, concerns about these effects among those likely to be most affected, as well as among other stakeholders (that is, those with an interest in the policy) and the public."

The Communications Policy of the Government of Canada (2006) also requires consultation and citizen engagement including Canada Gazette and web postings of public consultations on the Canada Site (Requirement 9).

Internal to the CFIA, the Agency's Statement of Values specifically identifies consultation as a key value: "To develop effective policies and strategies, the CFIA values the perspectives of the stakeholders who are affected by its decisions".

The Report of the Independent Investigator into the 2008 Listeriosis Outbreak noted that "…as a regulatory agency, the Canadian Food Inspection Agency should create a formal and transparent consultation strategy which will define its required engagement with stakeholders" (recommendation 47). In response, the CFIA developed a Consultation Policy and Framework for the Canadian Food Inspection Agency which has been approved internally, posted for consultation and is expected to be finalized by the autumn of 2011. The policy states that The CFIA "…will undertake consultation with stakeholders, both regulated and non-regulated parties, on significant policy and program issues that impact them."

3.2 Alignment with Government Priorities

Are the topics the Agency is consulting on the most pressing and relevant to the CFIA and the federal government?

Major consultations conducted in recent years are aligned with commitments made by the Government of Canada and the CFIA, for example, imported foods, National Aquatic Animal Health Program (NAAHP) and Product of Canada. No gaps were identified.

Internal interviewees note that CFIA consults primarily on specific technical topics and there may be opportunities to broaden the topics to include more strategic issues. Concern with managing stakeholder expectations on the more long-term, strategic issues was raised.

3.3 Alignment with Federal Roles and Responsibilities

Is the CFIA's consultation approach congruent with applicable government policies and guidelines?

The Consultation Policy and Framework for the Canadian Food Inspection Agency (the Consultation Policy) includes a policy statement, objectives, and principles consistent with CDSR and the Guidelines for Effective Regulatory Consultation. It also outlines a common process to follow in the conduct of a consultation. These five stages (preparation, design, implementation, feedback and follow-up and evaluation) are also consistent with guidelines. The consultation policy is in the process of being finalized but has yet to be promoted throughout the Agency. Additional effort will be required to implement the policy.

While no CFIA policy existed internally in the past, review of past consultations indicate that approaches are generally consistent with federal policies and guidelines. Some areas of inconsistency are outlined here.

• The Communications Policy of the Government of Canada requires consultation and citizen engagement including Canada Gazette and web postings of public consultations on the Canada Site ("Consulting with Canadians"). This site has been operational since 2008 and CFIA has posted twice (CFIA Food Safety Consultations, 2009, Consultation Policy and Framework, 2011). Internally, many interviewees were unaware of the website or the requirement to post on it.
• CDSR and the CFIA's Consultation Policy both recommend providing feedback to stakeholders on the outcome of consultations; however, this is inconsistently done.
• The Consultation Policy also identifies an evaluation phase to assess the effectiveness of the consultation. This is occasionally completed.

4. Performance

4.1 Achievement of Expected Outcomes

4.1.1 Reach: Is the Agency consulting with the appropriate mix of stakeholders?

The Consultation Policy identifies appropriate stakeholder groups. CFIA's consultation activities have traditionally focused on regulated parties and industry, directly with companies and through industry associations. Some concerns were raised that reliance on industry associations as the primary point of consultation risks omitting some small and medium sized companies which may not be members of the association. The evaluation also noted that some industries are very well organized and engaged and have developed a number of forums which may result in some overlap.

CFIA also leads or participates in a range of well-established forums with key federal partners (Health Canada-HC, Public Health Agency of Canada-PHAC and Agriculture and Agri-Food Canada-AAFC). There are also a number of federal-provincial-territorial tables for ongoing dialogue. These forums occur at different levels (Directors, Directors General, Assistant Deputy Ministers) and have evolved over time with committees discontinued and established as needed.

Consumers have historically not been well engaged; however there have been a number of recent efforts. The Consumer Association Roundtable (CAR) was established in 2010. Many interviewees identified the challenges engaging consumers, such as the fact that they are a heterogeneous group, consumers are not well-organized in Canada compared to other countries and it can be difficult to engage on technical topics. Resource restrictions in consumer associations limit their ability to participate. This was identified as an area for continued improvement in terms of how best to engage and on what topics.

Internationally, the CFIA engages other governments through bilateral and multilateral forums. In addition, appropriate links are maintained with international organizations such as the World Trade Organisation (WTO), Codex Alimentarius, the Organisation for Economic Co-operation and Development (OECD) and the World Organisation for Animal Health.

Consultation with First Nations communities and organizations was identified as a possible gap in reach. Consultation with First Nations requires unique consultation processes, as well as guides and policies external to the CFIA.^Footnote 8 Some noted that the CFIA needs to monitor for new or emerging stakeholder groups.

In two consultations reviewed for this study, the stakeholder groups were not well-known to the CFIA and significant effort was expended to reach out and identify affected parties.

4.1.2 Success: How well do consultation activities meet the policy goals of informing stakeholders, gathering input and improving transparency and accountability? How does the CFIA use the stakeholder input received during consultations? To what extent do stakeholders view the Agency's consultations as open, meaningful and balanced?

Policy Goals

Interviewees noted the importance of making the purpose of the consultation clear. Key to managing stakeholder expectations is differentiating between situations in which CFIA is informing versus gathering input.

Most interviewees report the CFIA does a good job of informing stakeholders of regulatory and program changes. Stakeholders have mixed experience in receiving information from CFIA on program changes: some report it is excellent and some have experienced gaps in communication. Significant effort is expended to gather input on specific regulatory changes. Some reported that the CFIA is inconsistent in gathering input on less significant program changes.

Transparency and accountability is hampered by the lack of formal feedback on consultations and long elapsed times from consultation to implementation. However, there are pockets in the Agency where feedback is done very well.

Also impacting transparency, consultation documents are scattered throughout the CFIA external site and are difficult to locate. Most departments have a central, easy-to-find section of the website for consultations (for example, HC, TC, and EC). Content on the websites include open and closed consultations, summary reports, calendar of consultations, links, etc.

Use of Input

Consultation results are used to inform regulation, policy and program development. Results are summarised during the RIAS process for Gazette I when required. Overall, internal interviewees report stakeholder input is used, however the degree of impact will vary from file to file. Many interviewees report that the consultation results could be better shared within the Agency. A study in 2007^Footnote 9 found that the CFIA uses consultation to build public confidence and acceptance of agency decision-making, and also notes that the CFIA benefits from a range of expertise and views from outside of government.

A practice within one of the permanent stakeholder forums is to provide input in written format, based on consensus. This creates a more unified voice for industry and appears to reduce the influence individual or special interest voices, and improves the value of the input to the CFIA.

Several recent consultations have employed an iterative consultation process. It was reported that this is particularly effective on complex issues or early in the regulatory development process. The first round provides input into the design and the second round can be used to refine the regulation, program or implementation plan.

Industry associations report that generally they can see the impact of input provided on issues and had a good appreciation that not all comments can be accommodated. Consumer associations feel it is too early to see impacts; however, they are optimistic going forward. The lack of formal feedback is a concern, though some program areas within the CFIA do this well.

Some departments provide formal feedback to participants in consultations. For example, AAFC sends a roll-up report by email or by providing a link to the report. HC produces "what was said" reports or detailed summary reports with analysis and responses to the comments. Minutes or meeting summaries from permanent forums are also posted by some departments.

Open, Meaningful and Balanced

Associations broadly view the consultations as open, in that the purpose is transparent, there are no "hidden agendas," and staff is open to discussion. While there are exceptions, associations believe the consultation sessions are meaningful. Sometimes, the outcome appears to stakeholders to be pre-determined and there is less opportunity to influence. Overall consultations are perceived to have a good balance of stakeholders with the exception of consumer associations. A consumer association noted that they are segregated from industry and cited HC as an example of bringing consumer associations and industry together on some topics.

There is a need to manage expectations particularly regarding the time required to implement changes as industry finds the processes slow and frustrating at times and a common criticism was related to the length of time between consultation and implementation. Stakeholders report that status updates are useful when provided by the CFIA though this is not consistently done.

4.1.3 Program Design: Are the roles and responsibilities for conducting stakeholder consultations clear within the CFIA? Are the appropriate policies, frameworks and tools in place to support stakeholder consultations? Are consultation activities conducted throughout the regulatory life cycle?

Roles and responsibilities for conducting stakeholder consultations are not documented but are quite clear in practice, with a number of groups having a role in the function. Individual program areas within Programs and Policy Branch have primary responsibility for consultations related to regulatory change or significant program changes. Operations Branch consults with stakeholders from more of an implementation perspective. Public Affairs Branch is the identified lead for public opinion research and more recently consumers and the CAR secretariat. Staff seeking support or advice for consultations identify a gap within the CFIA. Communication between branches at the beginning of consultation processes has not always allowed for timely input.

Both HC and AAFC have focal points for consultation activities, though the mandate, functions and size vary considerably. HC has two groups: Consultation Services Directorate (CSD) and Office of Consumer and Public Involvement (OCAPI), with a combined total of 40 FTEs dedicated to consultation. CSD provides department-wide support and is responsible for common tools while OCAPI provides dedicated support to Health Products and Food Branch. In AAFC, the Communication and Consultation Branch (CCB) is the focal point for research, advice, and coordination of consultations and citizen engagement, with a consultation unit of about eight FTEs.

The CFIA Consultation Policy drafted in 2010^Footnote 10 is currently being finalized. While much of the policy is common practice, it has not yet been fully operationalized. Various tools were drafted by the Consultation Unit in existence in 2007 within the Corporate Secretariat. These included a guide, toolbox, roles and responsibilities etc. However the tools were not finalized or adopted before the unit dissolved. Currently there are no support materials on the CFIA intranet for consultations.

There was a strong and consistent expression of need for some common tools and support, a list of stakeholders, list of consultations, toolkit, single place to go for advice, stronger internal and external web presence etc. Many interviewees expressed a desire for better IT support such as faster posting on web, survey support, webcasting, and web meetings. It was noted that special skills and processes are required to consult with First Nations stakeholders.

Both HC and AAFC have a range of tools available to support consultations. HC tools include a centralized point on website, calendar, extensive toolkit, public involvement framework, policies, and a database of stakeholders and consultations. AAFC has an internal website with tools and policies and a well-maintained stakeholder and industry events database-AgriConnexions^Footnote 11.

Generally the bulk of consultation is completed when the regulatory response or program change is partially developed. Some interviewees report a move towards consultation earlier in the life cycle, during identification of the issue and design of the regulatory response. To date, consultations are limited in the post-implementation phase. Permanent forums and working groups provide some opportunities for dialogue post-implementation. Evaluations and reviews also engage stakeholders post-implementation.

4.2 Demonstration of Efficiency and Economy

4.2.1 Efficiency: Are the consultation efforts timely, efficient and coordinated across the CFIA?

Consultations are timely, in that consultation occurs well in advance of the proposed change. However, a common concern expressed by associations was that at times, the comment period is not long enough to consult their respective members. Generally, the time from consultation to implementation can be long.

Formal mechanisms for internal coordination are limited, however this does not appear to be creating significant issues to stakeholders. However, the lack of stakeholder lists, common tools and templates does create internal inefficiencies as development is duplicated and learning not well shared. Program areas appear to know the stakeholders implicated in each program, but this information often remains at the program level. No coordination mechanisms exist on the internal website dedicated to consultation. The 2007 CFIA Consultations Report also found that the CFIA needs improved cross-organizational cooperation and coordination in consultation planning, implementation and evaluation.

Very little overlap or duplication was specifically identified. Some industries have a significant level of engagement with the CFIA and are consulted from various perspectives. This results in industry and CFIA presence on numerous forums or tables.

Both departments reviewed have mechanisms for coordination and tracking of stakeholder contacts and consultation activities. For example, Health Canada:

• maintains a stakeholder database which lists organizations and industry associations to invite participants and track attendees;
• has an internal network of facilitators and consultation experts within Health Products and Food Branch (HPFB) that can be drawn on;
• CSD has a standing offer for facilitators/ report writing that is shared with PHAC; and
• provides standards for setting up and maintaining advisory committees e.g. TORs, nomination processes, reporting processes etc.

Similarly, AAFC:

• AgriConnexions is used for planning to ensure consultations do not conflict with major industry events and to select participants for a consultation; and
• had a central database of consultations in the past; however, it is no longer being maintained.

4.2.2 Economy: Is the approach to consultation the most appropriate? Are there alternative mechanisms? Are there opportunities to streamline the external consultation efforts?

Some interviewees noted it is important to match the method to the purpose of the consultation and that a mix of methods may be required. The concept of "what works best when" is an important theme in the literature related to consultation. It has been proposed that the mechanism and context of consultation must match or effectiveness decreases^Footnote 12. One paper describes the continuum of public engagement and how the mechanisms vary along the continuum, identifying more than 100 mechanisms that can be used^Footnote 13.

Some of the specific comments on the commonly used mechanisms within the CFIA follow:

• Face-to-face meetings are commonly used by CFIA though this may be decreasing due to cost. Internal and external interviewees generally perceive these to be time consuming and expensive but provide opportunity for better dialogue and are important in certain situations. Examples include at the kick off of a new issue or to build consensus through dialogue.
• Teleconferences are viewed as a cost-effective option but need to be well-managed.
• On-line consultations are being used with a mix of effectiveness.
• A number of permanent industry and government forums are in use for continuous dialogue, although a study in 2007^Footnote 14 found that the CFIA could further reinforce and expand these, especially those focused on strategic positioning and issues management.
• Some of the AAFC-managed Value Chain Roundtables (VCRTs) are reportedly being used as one component of a consultation. There may be an opportunity to engage more proactively.
• It is difficult to find consultations on the CFIA website.

Many interviewees reported the growth of web-based tools and the need for CFIA to develop capacity for methods such as on-line consultations, web casting and web meetings. The growth of e-engagement is also noted in the literature in that online communication is increasingly being applied to facilitate participation in the public sphere and such initiatives are variously termed e democracy, e-citizenship, digital democracy, teledemocracy and cyberdemocracy.^Footnote 15

5. Conclusions and Recommendations

Consultation with stakeholders is a good practice for policy development in general, and stakeholder consultation is a mandatory activity for federal regulatory agencies, with a strong foundation in policies and guidelines. The CFIA has a strong history and culture of consultation, with significant skills and expertise.

Reach to stakeholder groups is generally good with recent efforts to increase engagement of consumers. Stakeholders broadly view the CFIA's consultations as open, meaningful and balanced. Stakeholder consultations influence regulatory, policy and program development. In some cases, the lag between consultation and implementation, combined with a lack of feedback, affects goodwill.

Responsibility for the design and conduct of consultation is dispersed within the CFIA. This allows for flexibility in approach, responsiveness and good technical expertise. However, this also results in inconsistent processes, inefficiencies and unevenness in the conduct of consultation activities. The lack of a focal point in the Agency hampers sharing of best practices and the development and sharing of tools.

5.1 Conclusions

Relevance

Consultation with stakeholders is a good practice for policy development in general, and as a regulatory agency, the CFIA is obliged to consult with stakeholders who may be affected by regulations. There is a strong policy basis within the federal government, and the CFIA has recently developed a supporting consultation policy and framework.

The Agency consults on proposed regulatory or program changes, aligned with Government of Canada and CFIA priorities. There may be additional opportunities to consult on strategic issues of interest to the CFIA. Management of stakeholder expectations is also a factor.

Overall, the CFIA's consultation policy and conduct of consultations are consistent with government policies and guidelines. Some minor areas of weakness were identified in terms of use of the "Consulting with Canadians" website, providing feedback, and evaluating consultations.

Performance - Outcomes

Consultation activities have traditionally focused on regulated parties, industry and other government departments and levels of government. International organizations and other governments are regularly engaged. Consumers have not been well represented in the past though there is recent effort to address this. Other groups that may be less well represented include small and medium sized businesses, and Aboriginal peoples.

Overall, the CFIA is meeting the policy goals of informing stakeholders and gathering input, though there are inconsistencies. Improvements to transparency could be realised with improved feedback and better visibility of consultations. Managing expectations raised through the consultation process is important.

Input received from stakeholders is considered and influences regulatory and program changes to varying degrees. Industry associations are able to see their influence and appreciate that not all recommendations can be accommodated.

Stakeholders broadly view the Agency as open, meaningful and balanced. Elapsed time to implementation combined with a lack of feedback impacts goodwill in some instances.

Roles and responsibilities for consultation are dispersed across branches and are generally well understood. A gap exists in the provision of advice and the development and sharing of tools.

Generally consultations are completed when the regulatory response is partially developed, though there is a trend towards engaging stakeholders earlier in the regulatory life cycle. Less formal consultation is being conducted post-implementation.

Performance - Efficiency and Economy

Consultations generally occur in a timely fashion though comment periods are at times too short for associations to consult with their respective members adequately. Limited coordination mechanisms exist within the CFIA, resulting in inefficiencies and relearning; however, this does not appear to be affecting stakeholders significantly. No major overlap or duplication was identified, though some industries are engaged through multiple mechanisms.

The CFIA uses a range of mechanisms to engage stakeholders. Assessment and sharing of what mechanisms are most effective in which situations ("what works best when") could be improved. Additional capacity and support is required to better use web-based mechanisms. Opportunities exist to create a better coordinated approach to consultation, which may result in efficiencies based on sharing best practices and tools.

5.2 Recommendations

Given the strength and breadth of consultation skills and experience within the Agency, a focal point is required in order to operationalize the Consultation Policy and to improve the consistency and efficiency of stakeholder consultations. Specifically, the CFIA should:

1. Support Agency consultation activities by:
   1. Providing coordination, advice and tools within CFIA and a source of expert advice for Agency personnel;
   2. Providing key coordination mechanisms for stakeholder consultation such as, an Agency-wide list of stakeholder organizations (explore Agriconnexions) and a schedule of major consultation activities;
   3. Enhancing mechanisms for consultation, specifically, more consistent use of the Consulting with Canadians website, enhanced web-based tools and improved visibility of consultations on the CFIA external site, and;
   4. Promoting standardization and sharing of best practices through activities such as: development and promotion of a toolkit; creating a resource centre for consultation on the CFIA's intranet; and establishing a community of practice within the CFIA.
2. Develop and implement guidelines for providing feedback to participants of consultation activities.
3. Continue to enhance engagement of consumers and small and medium sized businesses and engage less traditional stakeholder groups, such as Aboriginal Peoples and vulnerable populations as required.

Annex A - Logic Model

The following diagram is a draft logic model for stakeholder consultation which illustrates the links from the activities the CFIA undertakes to the immediate and intermediate outcomes.

CFIA Stakeholder Consultations Logic Model

Annex B - Advisory Committee

The members of the Advisory Committee for the evaluation are listed below.

AERO Branch

• Peter Everson - Co-Chair, Executive Director and Head of Evaluation
• Brian Smith - Co-Chair, A/Executive Director and Head of Evaluation
• Joanne Roulston, Director, Evaluation Directorate
• Leah McKeown, Manager, Evaluation Directorate

Public Affairs Branch

• Janice Snyder - Co-chair, A/Executive Director, Communications Services and Outreach

Operations Branch

• Dr Harpreet Kochhar, A/Executive Director, Operations Strategy and Delivery

Policy and Programs Branch

• Barbara Doan, Director, Multilateral Relations

External (Ex-officio Member)

Julie Cote-Stordy, Senior Evaluation Analyst, Public Health Agency of Canada

Annex C - Evaluation Matrix

Relevance

Performance

Annex D - List of Key Documents and Web Sites

Websites:

Consulting with Canadians: www.consultingcanadians.gc.ca

Office of Consumer Affairs, Industry Canada: http://www.ic.gc.ca/eic/site/oca-bc.nsf/eng/home

Consumer Association Roundtable, CFIA: http://www.inspection.gc.ca/english/agen/constable.shtml

Liaison Magazine, CFIA: http://www.inspection.gc.ca/english/agen/liaison/liaisone.shtml

Regulatory Initiatives, CFIA: http://www.inspection.gc.ca/english/reg/consultation/reginite.shtml

Canada Gazette, Government of Canada: http://www.gazette.gc.ca/rp-pr/p1/index-eng.html

Community of Federal Regulators: http://www.cfr-crf.gc.ca/ne/index-eng.html

International Association of Public Participation: http://www.iap2.org/

Annex E - Interview Guides

Interview guides were prepared for:

• program managers and implementation staff; and
• key stakeholders.

These interview guides are contained in the pages which follow.

Interview Guide
Internal Interviews

The Audit, Evaluation and Risk Oversight (AERO) branch is conducting an evaluation of the CFIA's stakeholder consultation process. Broadly defined, stakeholders are "individuals, groups or organizations likely to be affected by proposed regulatory changes."

The objective of this evaluation is to assess the relevance and performance of the CFIA's activities in this area. It will cover the period from 2006/07 to 2010/11.

We would like to discuss stakeholder consultations with you, based on the areas outlined below. We anticipate requiring approximately 60 minutes of your time.

Introduction

1. Please briefly describe your role and involvement in stakeholder consultations within the Agency.

Performance - Success

2. In your experience, have consultation activities included the following groups:

• Regulated parties
• Consumers / Canadians
• Government Departments / Agencies
• Foreign governments and international organizations

What are the strengths/gaps in terms of consultation activities reaching each of these groups?

3. How well do consultation activities achieve the following goals:

• Informing stakeholders
• Gathering input
• Improving transparency and accountability

4. How does the CFIA use the stakeholder input received during consultations? Please provide examples.

5. Who is responsible for conducting stakeholder consultations? Are the roles and responsibilities clear?

6. Does the CFIA have the appropriate policies, tools and support to conduct stakeholder consultations? What is in place/missing and what could be improved?

7. Are stakeholder consultations conducted at various points in the regulatory life cycle, i.e.: identifying policy issues, selecting regulatory response, analyzing impacts, planning implementation, measuring performance, evaluating and reviewing regulations?

Performance - Efficiency and Economy

8. Are consultations conducted in a timely, coordinated fashion? Are there areas of duplication or overlap?

9. Are the mechanisms used (e.g. forums, advisory groups, web-based consultation, etc) efficient and economical? Are the resources broadly sufficient for each type of consultation? Are there any alternative mechanisms that should be considered?

Relevance

10. Is the CFIA consulting on the right topics in relation to CFIA priorities and Government of Canada priorities? Are there any significant gaps?

11. Is there anything else you would like to add?

Interview Guide
Key Stakeholders / Associations

The Canadian Food Inspection Agency (CFIA) is conducting an evaluation of its stakeholder consultation process. Broadly defined, stakeholders are "individuals, groups or organizations likely to be affected by proposed regulatory changes."

The objective of this evaluation is to assess the CFIA's activities in this area over the past five years and provide recommendations for areas of improvement.

We would like to discuss stakeholder consultations with you, based on the areas outlined below. We anticipate requiring approximately 30 minutes or less of your time.

Topics

1. Please briefly describe your role and involvement in stakeholder consultations with respect to the CFIA.

2. In your experience, has the CFIA engaged your organization on topics that might affect you? Do you feel there are any significant gaps in terms of topics?

3. Do you find that the CFIA's consultations are:

• Open (purpose is transparent)
• Meaningful (are able to influence the issue)
• Balanced (have the appropriate mix of stakeholders)

4. How would you describe the consultation materials provided to your organization in terms of:

• Relevance
• Quality
• Completeness
• Timeliness

5. Do you know how the CFIA used the stakeholder input received during consultations with your organization? Please provide examples.

6. Did you obtain the information that you may have been seeking e.g. on the CFIA's programs or regulatory changes?

7. In your experience, are stakeholder consultations conducted in a timely, coordinated fashion with relevant industry stakeholder or across your industry? Are there areas of duplication or overlap?

8. Are the mechanisms used (e.g. forums, advisory groups, web-based consultation, etc) efficient and economical? Are the available government staff and meetings broadly sufficient for each type of consultation? Are there any alternative mechanisms that should be considered?

9. Is there anything else you would like to add?