ARCHIVED - Evaluation of the Canadian Food Inspection Agency's (CFIA's) Plant Protection Program
Final Report

This page has been archived

Information identified as archived is provided for reference, research or record-keeping purposes. It is not subject to the Government of Canada Web Standards and has not been altered or updated since it was archived. Please contact us to request a format other than those available.

March 3, 2015

Table of Contents

List of Acronyms

AAFC
Agriculture and Agri-food Canada
AIRS
Automated Import Reference System
ALHB
Asian Long-Horned Beetle
BSLB
Brown Spruce Longhorn Beetle
CBSA
Canadian Border Services Agency
CFIA
Canadian Food Inspection Agency
CFPWG
Canadian Forestry Phytosanitary Working Group
CFS
Canadian Forest Service
CFTEP
Canadian Fruit Tree Export Program
CGC
Canadian Grain Commission
CGCP
Canadian Greenhouse Certification Program
CIPF
Canadian Invasive Plants Framework
CNCP
Canadian Nursery Certification Program
DFAIT
Department of Foreign Affairs and International Trade Canada
DFATD
Department of Foreign Affairs, Trade and Development
DFO
Fisheries and Oceans Canada
EAB
Emerald Ash Borer
EAC
Expert Advisory Committee
EC
Environment Canada
EOC
Emergency Operations Centre
ERP
Emergency Response Plan
ERT
Emergency Response Team
FTE
Full-time equivalent
HRCMC
Human Resources and Corporate Management Committee
IAS
Invasive alien species
IASDP
Invasive Alien Species Domestic Plant Health Program
ICS
Incident Command System
IGB
Investment Governance Board
IPPC
International Plant Protection Convention
ISPM
International Standard for Phytosanitary Measures
MAS
Market Access Secretariat
MOU
Memorandum of Understanding
NAPPO
North American Plant Protection Organization
NISC
National Import Service Centre
NPPO
National Plant Protection Organization
NRCan
Natural Resources Canada
OMAF
Ontario Ministry of Agriculture and Food
OSHA
Occupational Safety and Health Administration
P
Program
PAA
Program Alignment Architecture
PBL
Plant Business Line
PCA
Parks Canada Agency
PCN
Potato Cyst Nematode
PGIB
Project Governance and Investment Board
PHAC
Public Health Agency of Canada
PPA
Plant Protection Act
PPEP
Plum Pox Eradication Program
PPMC
Policy and Programs Management Committee
PPMMP
Plum Pox Monitoring and Management Program
PPP
Plant Protection Program
PPV
Plum Pox Virus
PRA
Pest risk analysis
PRP
Plant Resources Program
RMD
Risk Management Documents
RPP
Report on Plans and Priorities
SCN
Soybean Cyst Nematode
SMC
Senior Management Committee
SP
Subprogram
SPTQMP
Seed Potato Tuber Quality Management Program
TB
Treasury Board
TBS
Treasury Board of Canada Secretariat
WTO
World Trade Organization

Executive Summary

As per its 2012–2017 and its 2013–2018 Evaluation Plans, the Canadian Food Inspection Agency (CFIA or the Agency) has recently completed an evaluation of its Plant Protection Program (PPP). This report provides the findings, conclusions, and recommendations from the PPP evaluation.

Overview of the PPP

The PPP is one of four subprograms operating under the CFIA's Plant Resources Program (PRP). The PPP is meant to mitigate the risks associated with introduction and spread of plant pests, including Invasive Alien Species (IAS), into Canada. This involves limiting their entry into the country as well as detecting, tracking, and eradicating once they have been established in Canada. The legislative basis for the PPP is primarily the Plant Protection Act (PPA) and Regulations and, to a lesser extent, the Seeds Act and Regulations.

The program includes four commodity-specific areas: forestry; grain and field crops; horticulture; and potatoes. A fifth specialized area relates to Invasive Alien Species (IAS) and is referred to as the IAS Domestic Plant Health Program (IASDP). Each area is involved in a range of program activities specific to their own commodity(ies) or IAS in the case of the IASDP. Broadly speaking, these activities may be categorized into program support and program delivery activities.

Four program support activity groups include:

  • regulatory and policy analysis and development;
  • program design, advice, and training;
  • science advice and laboratory services; and
  • internal management.

Four program delivery activity groups include:

  • communication and stakeholder engagement;
  • inspection, surveillance, and certification;
  • contingency, preparedness, and mitigation; and
  • international engagement and standard setting.

The PPP falls within the overall governance structure for the CFIA as a whole. This is led by a Senior Management Committee (SMC), which is the centre for executive decision making within the Agency. The SMC is supported by three senior executive committees, including the Policy and Programs Management Committee (PPMC); the Human Resources and Corporate Management Committee (HRCMC); and the Investment Governance Board (IGB). Advice is provided to this senior executive level of the Agency through three business lines: Food Safety, Animal Health and Zoonotics, and Plant Resources – of which the PPP is a part.

While not the largest program at the Agency, the PPP does involve a significant resource commitment. For fiscal years 2009-10 to 2012-13, planned spending for the PPP was $414 million, while actual expenditures were $399 million. In 2012–2013, 672 full-time equivalents (FTEs) were used to deliver the PPP, which was about 2% more than initially planned for that fiscal year.

Evaluation scope and methodology

In accordance with the Treasury Board (TB) Policy on Evaluation and its supporting Directive and Standard, the evaluation examined the PPP with regards to the following issues:

  • the continued need for the program (rationale)
  • the program's alignment with government priorities (rationale)
  • the program's alignment with federal government roles and responsibilities (rationale)
  • the program's achievement of expected outcomes (performance)
  • the efficiency and economy of the program (performance)

When exploring these issues, particular focus was placed on the PPP's program delivery activities and their outcomes, with inferences about program support activities made through their relationship to program delivery.

The evaluation of the PPP included data collection in support of four main lines of evidence. It began with a literature and document review related to the PPP and plant protection generally. It then continued with a secondary data review and cost analysis based on available PPP activity, outcome, and financial information. Three case studies were undertaken to explore the following areas of interest within the PPP in greater detail: the Plum Pox Monitoring and Management Program (PPMMP) with consideration of the Plum Pox Eradication Program (PPEP), IAS, and compensation payments in accordance with regulations under the PPA. Finally, in-depth interviews with 57 key informants from inside and outside of the CFIA were used to supplement the other information gathered during the evaluation and explore many of the evaluation issues in more depth. The evaluation combined information from all four lines of evidence to speak to each of the evaluation issues.

An evaluation working group consisting of representatives from the PPP was established during the development of the PPP's evaluation framework. This group provided access to documentation, literature, and potential key informants; liaised with external stakeholders; coordinated evaluation support within their respective organizations; and validated the evaluation findings. This group included representation from the Plant Health and Biosecurity Directorate, the Operations Strategy and Delivery Directorate, the Plant Health Science Directorate, and Corporate Management Branch.

It is important to note that the evaluation confronted a number of challenges and limitations, four of which were particularly relevant to the interpretation of findings. First, given the significant range of activities undertaken as part of the PPP, it was not possible to conduct a detailed review of all aspects of the program. Second, although over 700 documents were reviewed during the evaluation, it was difficult to establish an overall picture of the PPP's operations without considerable key informant input. Third, limited information was available to contextualize much of the quantitative program data available to the evaluation. Fourth, the large number of external factors influencing the PPP complicated the attribution of changes in expected program outcomes to specific program activities.

Summary of Findings

In terms of the rationale for the PPP, evidence from the evaluation suggests that there are clear biosecurity risks associated with plant pests, including IAS, in Canada. These plant pests have the potential to result in a loss of biodiversity, restrict market access for Canadian producers, impose direct financial impacts on Canadians, reduce food security, and affect human health. Nationally, all of these can affect Canadian trade and economic interests. By attempting to limit the introduction and spread of plant pests in Canada, the PPP directly addresses this demonstrated need. At the same time, the program helps meet Canada's obligations under a number of international agreements and conventions. Importantly, the program does so in an environment where it would be unlikely that private sector stakeholders would consistently, and voluntarily, address biosecurity risks on their own.

The evaluation noted that three Government of Canada outcome areas are supported by the PPP. These include healthy Canadians, strong economic growth, and a clean and healthy environment. These outcome areas aligned quite closely with the CFIA's raison d'être, the PPP strategic objective, and the purpose of the PPA. While distinct, these three areas are related insofar as a healthy environment – and specifically a healthy plant base – provides the foundation for a variety of economic activities in Canada and contributes to the health of Canadians.

With that said, the strategic outcome for the CFIA, which is the endpoint in the PPP's logic model, does not explicitly acknowledge the program's contribution to the Canadian economy. Although this does not undermine the implied connection between a safe and accessible plant resource base and economic activity, arguably, it decreases its visibility as a program goal. This lack of visibility appears inconsistent with the explicit acknowledgment of economic activity in other PPP documentation and the many activities undertaken by the program to influence the health of the Canadian economy directly.

Setting aside the domestic importance of addressing biosecurity risks, Canada has an obligation to undertake risk reduction activities in the plant area as a signatory to a number of international agreements and conventions on plant health. The CFIA has been designated as the National Plant Protection Organization (NPPO) for Canada, and as a result, also has the responsibility for undertakings such activities to meet these commitments.

The PRP, of which the PPP is a part, includes four distinct areas of activity, which are meant to contribute to the CFIA's goals. These include regulating agricultural and forestry products; mitigating risks to the plant resource base (including crops and forests) from regulated pests and diseases; regulating the safety and integrity of seeds, fertilizers, and plant products; and managing plant health emergencies and incidents. The PPP's four program delivery groupings, as outlined in the program's logic model, all fall within one or more of these areas.

Establishing the PPP's performance was one of the most problematic elements of the evaluation. There was no evidence to suggest that the PPP is failing to undertake activities and produce outputs as generally planned and outlined in the available program documentation. PPP performance indicators included in the Agency's Departmental Performance Report also suggest that certain performance targets for the program are being met on an annual basis. In addition, anecdotal evidence can provide a number of examples of achievement resulting from program activity. Further, the plant health literature provides additional support for the notion that the PPP is likely contributing to its expected outcomes.

However, a number of factors complicate a clear understanding of activity, output, and outcome achievement, as well as the PPP's contribution to these.

  • First, contextual information necessary for the interpretation of existing data is limited. In the case of outputs for example, there is little readily available information to understand why budgeted and actual outputs differ either within or across regions.
  • Second, PPP outcomes are often causally linked to each other. This means that the achievement of one outcome can heavily influence the achievement of another. This makes it difficult to draw a direct link between marginal changes in an outcome and specific program activities.
  • Third, conceptual difficulties with the measurement of prevention and risk reduction mean that not all of the program's established indicators effectively speak to PPP outcome achievement. For example, a target of no new regulated foreign pests entering into Canada through regulated pathways and establishing themselves may be met by either increasing preventative activities or reducing pest surveillance – only one of these activities legitimately reduces risk.

Although the second complication above is an inherent part of the PPP's work, the CFIA or the program may address both the first and the third.

Assessing the PPP's contribution is further complicated by the role that other stakeholders – federal, international, provincial, and others – play in the achievement of the program's expected outcomes. In some instances, the PPP or the CFIA have an opportunity to influence stakeholders, to better achieve program goals. For example, key informants have pointed to opportunities for greater international collaboration, more certainty around compensation payments, and better communication with stakeholders. However, in other instances, formally established working relationships between the program and other stakeholders can affect program achievement and limit program delivery options. One key example cited on a number of occasions involves the working relationship between the CFIA and Canadian Border Services Agency (CBSA), as it relates to plant health border security. To some extent or another, nearly all suggested improvements to the PPP identified as part of the evaluation, including the expanded use of the program's current alternative service delivery approaches, rely on these formal and informal stakeholder relationships for their success.

Prior to discussing program efficiency and economy, it is important to understand that the CFIA organizes Agency funding, accountability, and decision making along its branch structure. This branch structure cuts across all Agency business lines. Branches draw on A-base funding for the Agency as a whole, and budgetary reallocations within a branch can redirect resources to, or away from, business lines. This can pose challenges for both planning and program implementation within a business line, or within a program like the PPP.

Over the past four fiscal years, annual expenditures for the PPP have been within 2% to 5% of planned spending. However, expenditures for IAS and PPEP/PPMMP have been well below planned amounts. The variance in spending for these two areas is most likely due to both the practice of pooling funds prior to distribution and challenges associated with activity coding in the Agency's financial tracking system. The latter being a concern for both the PPP and the Agency as a whole.

Despite many key informants suggesting that current PPP funding is insufficient and limits the program's ability to undertake proactive action, objectively assessing this claim was not possible during the evaluation. To do so would require the program to identify specific program goals on an annual basis, and then identify an anticipated level of funding required to achieve these goals. This anticipated level of required funding could then be compared to actual resource allocations and to the level of eventual goal achievement in order to determine if the program is adequately resourced to achieve its goals. While it was not the intent of the evaluation to assess the sufficiency of program resources, nor suggest that additional funding is needed, the fact that there appears to be no way of objectively assessing this sufficiency is, in and of itself, an important evaluation finding.

In terms of overall efficiency and economy, the financial tracking and measurement difficulties associated with the PPP and the CFIA generally, make a rigorous and quantitative assessment of the PPP's efficiency and economy very difficult. This task becomes nearly impossible when the PPP's difficulties with contextualizing its program outputs, and assessing its marginal impact on its outcomes, are also considered. This made a quantitative assessment of program efficiency and economy infeasible during the evaluation and limited insights in this area to suggestions for program improvements from key informants.

Recommendations

The following are recommendations stemming from the evaluation.

Recommendation Supporting rationale Implications of findings Intended outcome of recommendation
Recommendation 1:
Review the CFIA's plant protection-related arrangement with the CBSA to facilitate optimal programming options.
The delegation of responsibility for certain plant protection responsibilities to the CBSA has had a number of implications for the way in which the PPP undertakes its work. Perhaps more importantly, the current arrangement has the potential to affect the PPP's ability to implement program improvement, including an increased emphasis on prevention. Reviewing the arrangement between CFIA and the CBSA as it relates to plant protection appears to have merit, so as to ensure that the PPP continues to have the ability to undertake program improvements. This is particularly true given the lack of a regular CFIA presence at the border and the competing border security demands placed on the CBSA.

An increased focus on preventative activities or other program improvements have the potential to improve program outcomes or efficiency.

The delegation of plant-related border security activity to the CBSA has had implications for PPP programming.

Optimize programming options.
Recommendation 2:
Review and improve performance measurement associated with the PPP
Performance measurement is particularly difficult in the context of regulatory programming. However, there are opportunities for improvements in this area within the PPP. This includes not only the development of performance indicators that more accurately reflect outcome achievement, but also the development of a body of contextual data to support the interpretation of performance measures. A number of measures will be required to tell a clear story of the program's effects. Performance measurement within the program does not consistently provide the information necessary for PPP planning, to communicate the importance of the program, or to report on outcomes. Improved program planning, reporting, and ability to communicate the importance of the program within the CFIA and outside of the Agency.
Recommendation 3:
Assess the effectiveness of alternative service delivery options relative to traditional PPP delivery to better support the case for their continued or expanded use.
The evaluation found many examples of the PPP's use of alternative service delivery, and support for the continued or expanded use of these approaches appears to exist within the program and the CFIA. However, information on the relative efficiency and effectiveness of these alternatives is needed to support a strong case for their use. A number of factors identified in the evaluation bear particular consideration when undertaking this type of an assessment, examples of which can be found in Section 2 and 5. Examples of alternative service delivery within the PPP exist. However, information on the relative efficiency and effectiveness of these alternatives is needed to support their use. Improved case for the use of alternative service delivery models in specific circumstances in order to improve program efficiency and effectiveness – both for the Agency and Canadians.
Recommendation 4:
Based on prioritized goals, estimate PPP resource requirements during annual planning.
Having an objective means of assessing program resource sufficiency may benefit the PPP. One way to approach this assessment could involve the program identifying specific program goals on an annual basis and then identifying an anticipated level of funding required to achieve these goals. This anticipated level of required funding could then be compared to actual resource allocations and to the level of eventual goal achievement. The comparison would then help an assessment of both the adequacy of resources and whether estimated funding requirements by the program were accurate. Changes in goals based on Agency prioritization would need to be considered throughout this process. Under the current PPP budgeting approach, it is extremely difficult to assess resource sufficiency in an objective way. Doing so requires changing the way in which annual program goals are established and budgets are examined. Improved ability to understand necessary program resourcing for the achievements of planned program outcomes.

Evidence based decision making when assigning funding.

1.0 Introduction

The need for an internal evaluation of the Plant Protection Program (PPP) was identified in the Canadian Food Inspection Agency's (CFIA or Agency) Evaluation Plan (2013–2018) (CFIA, 2013d). The evaluation was meant to inform ongoing program design and decisions making. In addition, it was meant to support the CFIA's compliance with the requirements of the Treasury Board (TB) Policy on Evaluation (2009) for evaluation coverage of all direct program spending within a five-year period.

In accordance with the Policy on Evaluation, andits supporting directive and standard, the evaluation assessed the PPP in the context of the following evaluation issues:

  1. continued need for the program (Relevance)
  2. alignment with government priorities (Relevance)
  3. alignment with federal government roles and responsibilities (Relevance)
  4. achievement of expected outcomes (Performance)
  5. demonstration of efficiency and economy (Performance)

The evaluation was conducted over an eight-month period, from August 2013 to March 2014, and covers the period from April 1, 2009, to September 30, 2013. The evaluation team was composed of evaluators from PRA Inc. and CFIA's Audit and Evaluation Branch.

This report presents the findings, conclusions, and recommendations from the evaluation.

2.0 Program overview

The PPP is one of four subprograms operating under the CFIA's Plant Resources Program (PRP) (CFIA, 2013a). The PPP is meant to mitigate the risks associated with plant pestsFootnote 1 in Canada. This involves limiting the introduction of plant pests and diseases, as well as detecting, tracking, and eradicating them once they are in the country. The following subsections provide an overview of the PPP.

2.1 Program background and description

As identified in the Agency's Program Alignment Architecture (PAA), the PPP is one of four subprograms (SP) under the Agency's PRP. The PRP is one of the CFIA's five programs (P), which include: the PRP; the Food Safety Program; the Animal Health and Zoonotics Program; International Collaboration and Technical Agreements; and Internal Services (CFIA, 2013c).

The following figure presents the PAA for the CFIA.

Figure 1: CFIA Program Alignment Architecture (CFIA, 2013a)

Click on image for larger view
CFIA Program Alignment Architecture. Description follows.

Description for image - CFIA Program Alignment Architecture (CFIA, 2013a)

The CFIA Program Alignment Architecture is a series of boxes that display the CFIA's strategic outcome, programs and sub-programs. The figure is organized vertically with the CFIA strategic outcome at the top, programs in the middle and sub programs at the bottom.

Strategic Outcome

The strategic outcome box is a red rectangle with the following text inside of it: "Strategic Outcome A safe and accessible food supply and plant and animal resource base"

Program

The program section of the diagram consists of five green rectangles with the following text written in them from left to right:

  • P 1.1 – Food Safety Program
  • P 1.2 – Animal Health and Zoonotics Program
  • P 1.3 – Plant Resources Program
  • P 1.4 – International Collaboration and Technical Agreements
  • P 1.5 – Internal Services

Sub Program

Underneath the "P 1.1 – Food Safety Program" rectangle are seven blue rectangles, organized vertically, they have the following text written in them from top to bottom:

  • SP 1.1.1 – Meat & Poultry
  • SP 1.1.2 – Egg
  • SP 1.1.3 – Dairy
  • SP 1.1.4 – Fish & Seafood
  • SP 1.1.5 – Fresh Fruit & Vegetables
  • SP 1.1.6 – Processed Products
  • SP 1.1.7 – Imported and Manufactured Food Products

Underneath the "P 1.2 – Animal Health and Zoonotics Program" rectangle are three blue rectangles, organized vertically, they have the following text written in them from top to bottom:

  • SP 1.2.1 – Terrestrial Animal Health
  • SP 1.2.2 – National Aquatic Animal Health Program
  • SP 1.2.3 – Feed

Underneath the "P 1.3 – Plant Resources Program" rectangle are four blue rectangles, organized vertically, they have the following text written in them from top to bottom:

  • SP 1.3.1 – Plant Protection
  • SP 1.3.2 – Seed
  • SP 1.3.3 – Fertilizer
  • SP 1.3.4 – Intellectual Property Rights

The "P 1.4 – International Collaboration and Technical Agreements" rectangle does not have anything underneath it.

The "P 1.5 – Internal Services" rectangle has three multi-coloured rectangles underneath it organized vertically.

  • The top multi-coloured rectangle has four sections organized vertically, with the following text from top to bottom:
    • SP 1.5.1 – Governance and Management Support, this section is blue
    • SSP 1.5.1.1 – Management and Oversight, this section is white
    • SSP 1.5.1.2 – Communications, this section is white
    • SSP 1.5.1.3 – Legal, this section is white
  • The middle multi-coloured rectangle has five sections organized vertically, with the following text from top to bottom
    • SP 1.5.2 – Resource Management Services, this section is blue
    • SSP 1.5.2.1 – Human Resource Management, this section is white
    • SSP 1.5.2.2 – Financial Management, this section is white
    • SSP 1.5.2.3 – Information, this section is white
    • SSP 1.5.2.4 – Information Technology, this section is white
    • SSP 1.5.2.5 – Travel and Other Administrative Services, this section is white
  • The bottom multi-coloured rectangle has three sections organized vertically, with the following text from top to bottom
    • SP 1.5.3 – Asset Management Services, this section is blue
    • SSP 1.5.3.1 – Real Property, this section is white
    • SSP 1.5.3.2 – Materiel, this section is white
    • SSP 1.5.3.3 – Acquisitions, this section is white

2.1.1 Objectives

The objective of the PRP is to "mitigate risks to Canada's plant resource base, which is integral to a safe and accessible food supply, as well as to public health and environmental sustainability" (CFIA, 2013c).

The strategic objective of the PPP is "to prevent the introduction of plant pests within Canada, control or eradicate designated plant pests, and to support export of commodities from Canada to meet trading partners' phytosanitary requirements"Footnote 2 (CFIA, 2012m). The legislative base for the PPP is primarily the Plant Protection Act (PPA) and Regulations and, to a lesser extent, the Seeds Act and Regulations (GoC, 2005, 2012a, 2012b, 2013).Footnote 3 Invasive plantsFootnote 4 considered to meet the PPA definition of "pest" are regulated under the PPA or the Seeds Act.Footnote 5

  • The PPA and Regulations are meant "to protect plant life and the agricultural and forestry sectors of the Canadian economy by preventing the importation, exportation, and spread of pests and by controlling or eradicating pests in Canada" (GoC, 2005, 2013). Broadly speaking, the CFIA's responsibilities under the PPA and Regulations include:
    • the development and enforcement of import requirements to protect Canada from plant pests;
    • the development of standards and certification of materials to allow Canadian products to meet the requirements of foreign markets;
    • domestic quarantine/management of pests; and
    • international negotiations in bilateral and multilateral forums to promote and implement Canada's positions for both trade agreements and the development of international and regional standards.
  • The Seeds Act and Regulations pertain to the "testing, inspection, quality and sale of seeds" (GoC, 2012a, 2012b). Particularly relevant to the PPP is Part V of the Seeds Regulations, which provides specific conditions for the release of seeds with novel traits (e.g., genetically-modified plant material) to prevent the spread or minimize the interaction with the environment.

Additionally, as per the PPA and Regulations and the Seeds Act and Regulations, the CFIA is responsible for regulating the environmental release of plants with novel traits if "pests" are involved.Footnote 6

2.1.2 Risk

The CFIA's 2012 Corporate Risk Profile identifies key risks to Agency operations and effectiveness. The table below describes these risks and their applicability to the PPP.

Table 1: Corporate risks and their linkage to the PPP
Corporate risk Linkage to the PPP

Legislative, regulatory, and program frameworks – The threat to the ability of the current legislative, regulatory, and program frameworks to support the effective delivery of the Agency's mandate.

  • Plant health risks are affected by the program's ability to provide sufficient coverage of pests given the changing global supply chain and consumer import patterns for new commodities from origins that are infested with plant pests.

Inspection effectiveness – The threat to the ability to have the appropriate inspection effectiveness to expeditiously prevent, detect, and respond to food safety, animal, and plant threats and support consumer protection and market access.

  • PPP inspection effectiveness faces risks as a result of the availability of CFIA inspectors that are knowledgeable about plants and plant pests, the effectiveness of agreement with the Canadian Border Services Agency (CBSA), the use of systems approach to inspections, and the use of alternative service delivery arrangements.

Scientific capability – The threat to the ability to have the scientific capability to adapt and respond in a timely manner.

  • Increased demand for PPP expertise represents a threat to the ability of the program to meet its mandate. The rate and spread of plant diseases are on the rise globally, as are the range of diseases and their vectors. This is due, in part, to globalization of production systems and travel, as well as the effects of climate change. The result has been an increase in demand for complex risk assessments, multi-disciplinary research, surveillance, rapid diagnostic capabilities, and science communications products. There is also an increase in science demands aimed at managing high-visibility issues.

Emergency management – The threat to the ability to respond to multiple simultaneous or large-scale emergencies.

  • There are many factors that could lead to an increased number of emergencies that the Agency may have to manage. This increase could lead to a situation where multiple incidents are occurring simultaneously, requiring PPP program resources normally allocated to other ongoing program work.
    • The threat posed by diseases, pests, and invasive alien species (IAS) in North America has grown due to increasing population densities; human, animal, and plant interactions; and the high mobility of plants and plant products.
    • The rate and spread of plant diseases is also on the rise, as are the range of diseases and their vectors.
    • Climate change is impacting, and will continue to impact, how seeds and plant pests behave in the Canadian environment; this may have implications for the number of emergencies Canada faces in the future. For example, pests that could not survive the Canadian winter in the past will begin to survive and/or thrive in our environment, or spread geographically (e.g., Mountain Pine Beetle).

Management information and IM/IT infrastructure – The threat to the ability to make risk-based decisions due to the lack of timely, accurate, and useful data and information.

  • The combination of the need for a large volume of information to support effective program decision making and the limited IM/IT capacity within the Agency represents a considerable risk to the PPP.
    • Globalization and the overall increase in the speed and volume of trade require that the Agency have an integrated business model for its IM/IT-enabled services.
    • The increasing volume and variety of information both externally and internally requires increasingly sophisticated analytical capacity.
    • The Agency's diverse information requirements and national presence has resulted in an IM/IT infrastructure containing a complex mix of new and old equipment and software. The Agency's numerous laboratory, inspection, and program information systems sometimes operate independently and house information in separate databases. Some are significantly outdated, as are commercial software and operating systems.
    • Information and data are currently present in multiple paper and electronic formats across the Agency resulting in several separate information knowledge bases spread out geographically. The lifecycle of this information is also not always being respected or followed. This can impact its accessibility to analysts and decision makers, as information is needed 24/7/365.

Transparency and leveraging relationships – The opportunity for the Agency to increase transparency and accountability to stakeholders.

  • There are increasing demands for government institutions to be more open, transparent, collaborative, and accountable. Consumers and industry are seeking opportunities to participate and be more aware of regulatory decisions and issues that may affect their businesses and the plant resource base. Failing to meet these demands may increase the risk to the PPP effectiveness given the need for consumer and industry support during their work.

Managing change – The threat to the ability to effectively manage change on an ongoing basis.

  • There has been a fundamental shift from incremental change to wholesale examination of how the Agency addresses its raison d'être.
  • The proposed legislative and regulatory modernization and inspection modernization are examples of the fundamental changes that the Agency will be undertaking in the coming years. These changes are directly in response to the regulatory, scientific, and fiscal environment within which the Agency operates.
  • The new reality, as signalled by the Government, will require the Agency to ensure that it has the internal capacity to address fundamental change on an ongoing basis. A lack of such capacity within the PPP would represent a risk to the program's ability to manage this change.

Source: (CFIA, 2012a)

2.1.3 Program structure

The PPP includes five specialized plant protection areas. Four of these relate to specific commodities: forestry; grain and field crops; horticulture; and potatoes. Each of these commodity areas is involved in developing and maintaining policies and directives related to the import and export of their own specific commodity(ies). As with the PPP generally, this work is meant to help prevent the introduction and/or spread of plant pests in Canada and support commodity-specific export activities (CFIA, 2012i).

A fifth specialized plant protection area relates to IAS and includes the IAS Domestic Plant Health Program (IASDP). This area is meant to contribute to the "environmental and economic sustainability of the agriculture, horticulture and forestry sectors as well as the natural environment" (CFIA, 2011e, p. 10). Specific activities include regulating the import and domestic movement of plants, monitoring imports to prevent entry of IAS (including invasive plants), and engaging in surveillance to determine whether an IAS or invasive plant is present in Canada (CFIA, 2011d).

The work of IASDP spans the four commodities mentioned above, and although the PPP has been undertaking activities related to IAS for some time, the establishment of the IASDP as a specialized area itself occurred relatively recently in 2005. The establishment of the IAS area was the result of Canada's international plant protection obligations under a number of international agreements and the resulting IAS strategy developed by the federal government.

2.1.4 Governance

Governance at the Agency is led by the Senior Management Committee (SMC), which is the centre for executive decision making within the CFIA. The SMC is supported by three senior executive committees (CFIA, n.d):

  • the Policy and Programs Management Committee (PPMC)
  • the Human Resources and Corporate Management Committee (HRCMC)
  • the Investment Governance Board (IGB)Footnote 7

Guidance and advice is provided to the senior executive level committees of the Agency through three business lines, corresponding to three of the Programs identified in the PAA: Food Safety, Animal Health and Zoonotics, and Plant Resources. The PRP and the PPP are part of the CFIA's Plant Business Line (PBL). The PBL Committee is a forum for senior management within the plant business line to discuss policy and programs, science, and operational issues (CFIA, 2012g). The Committee's activities fall into the following four main areas:

  • "providing guidance on scientific, policy, program and operational items… to the CFIA senior executive level committees related to the plant business line
  • identifying [business line] risks and setting priorities upon which operational plans will be developed…
  • review[ing] in-year progress against plans and expenditure reports and develop[ing] mitigation strategies to deal with emerging pressures…
  • providing an information sharing function upward towards the senior executive level committees, horizontally towards the other business line committees, and downward towards the relevant subcommittees" (CFIA, 2012g)

The PBL Committee is supported by a subcommittee, with both the committee and subcommittee including representation from relevant branches across the Agency. Three branches are particularly important to the PPP:

  • Policy and Programs Branch provides advice related to legislation regulations and guidelines; provides advice regarding policies, program design, and functional directions; manages ministerial correspondence; provides advice and services related to Parliamentary affairs; and coordinates federal/provincial/territorial and international engagement (CFIA, 2014a).
  • Operations Branch sets operational policies related to the delivery of inspection; provides inspection oversight; delivers inspection programming; undertakes compliance and enforcement activity; and leads engagement activities related to program delivery (CFIA, 2014a).
  • Science Branch sets policies related to providing science advice; provides this advice; undertakes diagnostic and testing work; and undertakes research in support of the CFIA's mandate (CFIA, 2014a).

2.1.5 Program resources

The following table presents planned and actual expenditures for the PPP. As shown in the table below, the PPP's actual expenditures for each fiscal year were closely aligned with planned expenses, with a variance of only 2% to 5% less than planned.

Table 2: PPP financial resources
PPP Overall 2009–2010
($ million)
2010–2011
($ million)
2011–2012
($ million)
2012–2013
($ million)
Planned $118.6 $95.0 $100.7 $99.9
Actual $114.3 $93.5 $95.8 $95.0
Variance ($) $4.3 $1.5 $4.9 $4.9
Variance (%) 4% 2% 5% 5%

Source: Corporate Management Branch, CFIA

The figure below shows a breakdown of PPP expenditures by activity category for 2012–2013. As shown in the figure, about 54% of the PPP's expenditures were used for inspection, surveillance, and certification activities. Refer to Appendix F for a breakdown of PPP expenditures by activity category for other fiscal years.

Figure 2: PPP 2012–2013 expenditures by activity category

Click on image for larger view
PPP 2012–2013 expenditures by activity category. Description follows.

Description for image - PPP 2012–2013 expenditures by activity category

This figure is a pie chart that displays expenditure amounts in millions of dollars and percentage of total expenditures for the Plant Protection Program. The pie chart is broken up into multi-coloured sections as follows:

  • The Inspection surveillance & certification section of the pie chart is coloured pink and represents 51 million dollars of expenditures which is 54% of Plant Protection Program expenditures
  • The Contingency preparedness & mitigation section of the pie chart is coloured navy blue and represents 0.1 million dollars of expenditures which is 0% of Plant Protection Program expenditures
  • The Regulatory and policy analysis and development section of the pie chart is coloured red and represents 1 million dollars of expenditures which is 1% of Plant Protection Program expenditures
  • The International engagement and standard-setting section of the pie chart is coloured green and represents 3 million dollars of expenditures which is 3% of Plant Protection Program expenditures
  • The Communication and stakeholder engagement section of the pie chart is coloured purple and represents 3 million dollars of expenditures which is 3% of Plant Protection Program expenditures
  • The Internal management section of the pie chart is coloured deep blue and represents 7 million dollars of expenditures which is 7% of Plant Protection Program expenditures
  • The Science advice and lab services section of the pie chart is coloured orange and represents 18 million dollars of expenditures which is 19% of Plant Protection Program expenditures
  • The Program design, advice, and training section of the pie chart is coloured light blue and represents 12 million dollars of expenditures which is 13% of Plant Protection Program expenditures

The following table presents the 2012–2013 full-time equivalent (FTE)Footnote 8 staff resources for the PPP.Footnote 9 As shown in the table, the PPP used 2% more FTEs than planned.

Table 3: PPP FTE resources
PPP Overall 2012–2013
Planned 657
Actual 672
Difference 15
Variance (%) 2%

Source: (CFIA, 2013w)

2.2 Program activities and expected outcomes

Figure 3 presents the logic model for the PPP, representing a graphical depiction of the expected causal relationships between the activities, outputs and outcomes of the program. The model is based on the Agency-wide logic model developed for the Agency's Evaluation Plan.Footnote 10

As shown in the figure, the logic model includes eight groups of activities. Four of these are defined as program support activity groups, and four are defined as program delivery activity groups. They include the following:

Program support activities:

  • regulatory and policy analysis and development
  • program design, advice, and training
  • science advice and laboratory services
  • internal management

Program delivery activities:

  • communication and stakeholder engagement
  • inspection, surveillance, and certification
  • contingency, preparedness, and mitigation
  • international engagement and standard setting

The program delivery activities represent the main frontline activities of the PPP, while the program support activities are meant to ensure that the program delivery activities function in the best possible manner. The support activities thus provide the foundation for all of the PPP's program delivery work.

There are a number of expected immediate outcomes depicted in the logic model:

  • increased awareness of health-related risks and risks to the plant base among the Canadian public
  • increased awareness of policies, regulations, and legislation among stakeholders
  • increased level of compliance with program policies, requirements, and regulations among stakeholders
  • preparedness to prevent and address plant-related emergencies
  • contribution to international standards and agreements for plant-related risks

The achievement of these immediate outcomes is expected to translate into a series of intermediate outcomes relating to plant health, domestic compliance with established plant health standards, improved access to international markets for Canadian producers, and improved access to international plant products for Canadians. Ultimately, these intermediate outcomes are meant to support the Agency strategic outcome of "a safe and accessible food supply and plant and animal resource base" (CFIA, 2013c). In the PPP logic model presented on the next page, the strategic outcome is framed exclusively in terms of plants in order to more closely align with the PPP.

Figure 3: PPP Logic Model

Click on image for larger view
PPP Logic Model. Description follows.

Description for image - PPP Logic Model

This is a diagram that describes the activities and outcomes of the Plant Protection Program. On the left side of the diagram is a coloured coded legend. The legend has the following text written in it from top to bottom:

  • Strategic Outcome, written in green
  • Intermediate Outcomes, written in green
  • Immediate Outcomes, written in green
  • Program Delivery Activities, written in red
  • Program Support Activities, written in red

The logic model is organized the same as the legend with "Strategic Outcome" at the top and "Program Support Activities" at the bottom.

Program Support Activities

The Program Support Activities section is surrounded by a rectangle with a blue dashed edge. Inside of this rectangle are four blue rectangles organized horizontally. The four blue rectangles have the following titles from left to right:

  • Regulatory and policy analysis, as well as development
  • Program design, advice, and training
  • Science advice and laboratory services
  • Internal management

The "Regulatory and policy analysis, as well as development" rectangle has the following bullets:

  • Plant-related regulatory research
  • Plant-related regulatory renewal

The "Program design, advice, and training" rectangle has the following bullets:

  • Development of policies, manuals, work plans, and other policy documents.
  • Design, delivery, and receipt of CFIA training

The "Science advice and laboratory services" rectangle has the following bullets:

  • Provision of scientific advice
  • Plant-related risk assessment
  • Development of scientific partnerships

The "Internal management" rectangle has the following bullets

  • Performance management
  • Human resources policy
  • Resource allocation
  • Governance

There is an arrow going from the Program Support Activities section to the Program Delivery Activities section, there is also another arrow going in the opposite direction.

Program Delivery Activities

The Program Delivery Activities section is surrounded by a rectangle with a blue dashed edge. Inside of this rectangle are four blue rectangles organized horizontally. The four blue rectangles has the following titles from left to right:

  • Communication and stakeholder engagement
  • Inspection, surveillance, and certification
  • Contingency, preparedness, and mitigation
  • Internal engagement and standard setting

The "Communication and stakeholder engagement" rectangle has the following bullets:

  • Industry training
  • Response to industry questions
  • Consultation with stakeholder groups
  • Provisions of plant-related risk information, as well as control and prevention strategies

The "Inspection, surveillance, and certification" rectangle has the following bullets:

  • Certifications of establishments, plants, and plant products
  • Inspection of establishments, plants, and plant products
  • Pest surveys
  • Oversight of audit based programming
  • Response to non-compliance

The "Contingency, preparedness, and mitigation" rectangle has the following bullets:

  • Plant-related emergency response planning
  • Plant-related emergency response
  • Control and eradication of plant pests

The "Internal engagement and standard setting" rectangle has the following bullets

  • On-site verification of foreign certification systems
  • Developing national and international scientific partnerships
  • Negotiating for market access
  • Participation in international standard setting

There is an arrow going from the Program Delivery Activities section to the Immediate Outcomes section.

Immediate Outcomes

The Immediate Outcomes section is surrounded by a rectangle with a green dashed edge. Inside of this rectangle are five white rectangles organized horizontally. The five white rectangles has the following titles from left to right:

  • Increased awareness of health-related risks and risks to the plant base among the Canadian public
  • Increased awareness of policies, regulations, and legislation among stakeholders
  • Increased level of compliance with program policies, requirements, and regulations among stakeholders
  • Preparedness to prevent and address plant-related emergencies
  • Contribution to international standards and agreements for plant-related risks

The "Increased awareness of health-related risks and risks to the plant base among the Canadian public" rectangles has the following bullets:

  • Awareness of plant-related risk information
  • Access to plant-related risk information

The "Increased awareness of policies, regulations, and legislation among stakeholders" rectangle has the following bullets:

  • Awareness plant-related control and prevention strategies
  • Access to plant-related control and prevention strategy information

The "Increased level of compliance with program policies, requirements, and regulations among stakeholders" rectangle has the following bullets:

  • Compliance with domestic regulations
  • Compliance with import regulations
  • Compliance with export regulations

The "Preparedness to prevent and address plant-related emergencies" rectangle has the following bullets:

  • Prevention of potential threats to the plant resource base
  • Eradication and management related to plant health emergencies and incidents

The "Contribution to international standards and agreements for plant-related risks" rectangle has the following bullets:

  • Canadian contribution to international standards discussions
  • Development of international scientific partnerships
  • Development of agreements for market access

There is an arrow going from the Immediate Outcomes section to the Intermediate Outcomes section.

Intermediate Outcomes

The Intermediate Outcomes section is surrounded by a rectangle with a green dashed edge. Inside of this rectangle are eight green rectangles organized into a top and bottom row.

There are six rectangles on the bottom row, they have the following text written in them from left to right:

  • General public engages in behaviours to protect the plant base
  • Domestic products are compliant with Canadian requirements
  • Imported products are compliant with Canadian requirements
  • Canadian standards are recognized internationally
  • Risks to the Canadian plant base are mitigated, minimized, or managed
  • Canadian interests are reflected in science-based international rules, standards, and arrangements

There are two rectangles on the top row, they have the following text written in them from left to right:

  • International plants and their products are accessible to Canadians
  • International markets are accessible to Canadian plants and their products

There are arrows going from the "Imported products are compliant with Canadian requirements" and the "Canadian standards are recognized internationally" rectangles to the "International plants and their products are accessible to Canadians".

There are arrows going from the "Canadian standards are recognized internationally" and the "Risks to the Canadian plant base are mitigated, minimized or managed" rectangles to the "International markets are accessible to Canadian plants and their products".

There is an arrow going from the Intermediate Outcomes section to the Strategic Outcome section.

Strategic Outcome

The Strategic Outcome section is surrounded by a rectangle with a green dashed edge. Inside of this rectangle is a green rectangle with the following text inside of it "A safe and accessible food supply and plant resource base".

The following subsections discuss the PPP's program delivery activities in more detail.

2.2.1 Communication and stakeholder engagement

The PPP engages in a number of communication and stakeholder engagement activities. These activities are meant to provide relevant information to stakeholders, inform the development of PPP policy and actions, and ensure that stakeholders engage in activities that limit plant-related risks. There are many examples of communication and stakeholder engagement activities, and most focus on ensuring continued information sharing with the PPP's many stakeholders. Some examples include the following:

  • Engagement in industry training sessions to increase stakeholder awareness of plant pests and respond to questions (CFIA, 2013t).
  • Stakeholder consultations to inform the development of policies, including proposed directives and regulations. The PPP, like other CFIA programs, uses various ad hoc or ongoing mechanisms, including direct communication, taskforces, forums, steering committees, and industry working groups (CFIA, 2012c).
  • Communication of a range of plant-related risk information (including control and prevention strategies) through its website, fact sheets, news releases, campaigns, and attendance at various events (e.g., festivals). Additionally, the CFIA maintains an online repository of guidance documents (CFIA, 2013m).
  • Issuing various alerts and notices. Two examples of the alerts and notices prepared are border lookouts and notices to importers:
    • "Border Lookout – A system that [CBSA] uses to control products at the border and to inform the [CFIA] of their arrival in Canada. It provides the information and direction required to reduce or manage imports with identified risks" (OAG, 2008). These may also be used to "identify importers who are known for serious noncompliance in the past" (OAG, 2008).
    • Notices to Importer – Document issued for commodities which require 100 percent inspection by the [CFIA]" (OAG, 2008). "Notice to importers are similar in concept to border lookouts; that is, they generally identify high-risk commodities and high-risk exporting countries or regions" (OAG, 2008).

2.2.2 Inspection, surveillance, and certification

The CFIA is responsible for verifying and certifying that plants and plant product exports and imports are free of invasive plants, pests, and diseases (IPPC, 2007). In addition, the CFIA's "responsibilities include […] carrying out pest surveys and plant health risk assessments"Footnote 11 (OAG, 2008). Within the areas of export, import, and domestic activity, the CFIA undertakes a range of inspection, surveillance, and certification activities.

The CFIA inspects and certifies that the plants and plant products meant for export from Canada conform to the current phytosanitary regulations of the importing country (CFIA, 2009b). A phytosanitary certificateFootnote 12 is issued by the CFIA to an exporter when the Agency believes the plant, plant product, or other regulated article meets the import requirements of the importing country, and the importing country requires the certificate as a condition for import (CFIA, 2009b).

While the issuance of a phytosanitary certificate often requires direct inspection on the part of the CFIA, the PPP has implemented a number of programs whereby certificates are issued under alternative service delivery arrangementsFootnote 13 such that direct inspection is replaced with oversight of industry's preventative control plans. Among the alternative service delivery arrangements are programs that use an audit-based systems approach to inspections.Footnote 14 These programs require exporters to have CFIA-accepted controls and procedures in place to meet importing country requirements. Thus, rather than using a traditional approach to inspections where the CFIA inspects shipments directly, under the systems-based approach, the CFIA audits the controls and procedures that exporters have in place to ensure their adequacy. Examples of such audit-based export systems include the following:

  • The Canadian Greenhouse Certification Program (CGCP)
  • The Canadian Nursery Certification Program (CNCP)
  • Canadian Fruit Tree Export Program (CFTEP)
  • Seed Potato Tuber Quality Management Program (SPTQMP)
  • The Canadian Heat Treatment Wood Product Certification Program

Importers of plants and plant products into Canada also face regulatory requirements. Section 29 of the Plant Protection Regulations includes a requirement that importers obtain an import permit before a regulated commodity is permitted to enter Canada. However, import permits are subject to exceptions stipulated in sub sections (1.1) to (5). The Plant Protection Regulations identify the following two types of import permits:

  • Section 32(1): Where the Minister has reasonable grounds to believe, on the basis of a pest risk assessment, that the importation of a thing will result or would likely result in the introduction into Canada, or the spread within Canada, of a thing that is a pest, is or could be infested or constitutes or could constitute a biological obstacle to the control of a pest, the Minister shall issue a permit in respect of the thing if the Minister determines that every precaution necessary to prevent the introduction into Canada or the spread within Canada of any pest or biological obstacle to the control of a pest can and will be taken (GoC, 2013).

    Specific requirements such as treatment at origin, requirement for a Phytosanitary Certificate,Footnote 15 or another import document – may be required as part of Section 32 (CFIA, 2010b).

    The specific requirements and conditions under Section 32 import permits are currently managed by the CFIA through import-related plant health policy directives which apply across all of the commodity areas under PPP.

  • Section 43(1): Notwithstanding sections 38 and 42, the Minister shall issue a permit to a person in respect of any thing that is a pest, infested or a biological obstacle to the control of a pest or that does not meet the requirements of the Act or any regulation or order made thereunder where the Minister determines that (a) the thing is imported for the purpose of being used for scientific research, educational, processing, industrial,Footnote 16 or exhibition purposes; and (b) the person is able and willing to comply with the conditions to be set out in the permit and will take every precaution to prevent the spread of any pest or biological obstacle to the control of a pest (GoC, 2013).

The CFIA may also conduct on-site audits of foreign certification systems – as it does for audit-based systems operated in Canada – to ensure compliance with Canadian requirements (CFIA, 2011e).

The CFIA has overall responsibility for the regulation of imported plants and plant materials into Canada. However, in 2003, an Order-in-Council authorized the transfer of certain regulatory activities undertaken by the CFIA to the CBSA. These included traveller and initial inspections services for plants and plant products at the Canadian border. A Memorandum of Understanding (MOU) between the CBSA and the CFIA outlines roles and responsibilities following the transfer (CBSA, 2009).

As a result of the Order-in-Council and the associated MOU, the CFIA no longer has a presence at the Canadian border and relies on the CBSA to identify and prevent potential threats to the Canadian plant resource base from crossing the border into the country. The CFIA provides various supports for this work.

The following summarizes the at-border inspection process:

  • The CBSA is directed to use the CFIA's Automated Import Reference System (AIRS) to understand the import requirements for specific commodities. This may involve immediate approval for entry into Canada or a series of alternative steps, including, but not limited to, approval, refusal, or referral to the CFIA (CBSA, 2011).
  • When a referral is required, the CBSA may consult with CFIA staff at the National Import Service Centre (NISC) to receive further information regarding actions taken for specific commodities. This may involve release, refusal, a hold at the border until the product is inspected by a CFIA staff member, or shipment to its destination for eventual inspection by a CFIA staff member (CBSA, 2011).

As a result, the at-border process involves a considerable amount of commodity and context specific flexibility. In fact, the CBSA's memorandum 19-1-1, which identifies CBSA activities related to food, plants, animals, a specific set of actions in all circumstances for only two types of material. These include all packaging materials made of solid wood and any products found to be contaminated with soil (CBSA, 2011).

The CFIA, through the CBSA, may employ several enforcement measures in response to instances of noncompliance with regulatory requirements. The most common enforcement measures mentioned in the PPP directives are refusal of product entry into Canada, return of the product to origin, or disposal of the product if it does not meet phytosanitary requirements. For example:

  • D-11-05 outlines requirements for non-manufactured and non-propagative wood products to prevent Asian Long-Horned Beetle (ALHB). Specifically, it states that "imported shipments which do not meet requirements may be refused entry, returned to origin, or disposed of at the importer's expense" (CFIA, 2012e).
  • D-11-01 outlines import requirements for plants for planting and fresh branches to prevent entry of Asian and Citrus Long-Horned Beetles, specifying similar noncompliance enforcement measures such as shipment removal, return to origin, rerouting, treatment, and disposal. It notes that the importer is responsible for bearing these costs, and the detection of quarantine pests may result in suspension of importation of the commodity from that country (CFIA, 2011c).

An additional aspect of the PPP's activities is the conduct of national plant protection surveys – also referred to as pest surveys. The CFIA's national plant protection survey program provides the Agency with data to support sound decision making in import, export, and domestic regulatory activities. Surveys are used to detect new populations of quarantine pests, define the location of quarantine pests, and determine areas that are pest free. Survey detection is meant to inform the Agency's control and eradication programs (CFIA, 2013n).

"The CFIA's Plant Health Surveillance Unit is responsible for planning, coordinating, and administering the national pest survey program. The Unit also plays a lead role in designing new surveys, developing survey protocols, and refining ongoing techniques and tools as new methodologies develop. The actual survey work is done by the Agency's regional operations staff", sometimes with other partners (OAG, 2008).

Plant pest surveys target existing and potential pests. Most surveys are conducted for pests listed in Schedule II of the Plant Protection Regulations (PPR),Footnote 17 which includes quarantine pests that have not yet reached the limits of their potential range in Canada. Other surveys are designed to detect pest species not known to be present in Canada but that are considered to be a quarantine concern according to the results of the Agency's plant risk assessment process (CFIA, 2007).

To determine the appropriate locations to survey, the PPP uses a Plant Protection Survey Priority Identification and Rationale document for ranking the priority of various proposed surveys for consideration (CFIA, 2011f). Some of the key elements for identifying the priority of possible pest surveys include whether pests are present in Canada or if there are high-risk pathways of introduction of the pest into Canada (CFIA, 2011f). Key guiding questions include the following:

  • "Is this survey the best way to obtain the required information and have other alternatives been considered?
  • Is the pest or pathway expected to become an emerging issue requiring an elevated level of attention in the Agency within the next year?
  • Is the survey required to advance a critical file? For example, is a bilateral agreement, trade, policy development or a control program, dependent on survey data being generated within the next year?
  • Is the current survey data for this pest required to issue phytosanitary certificates or to defend an import policy?" (CFIA, 2011f, p. 1).

Other considerations include regional priority differences, survey effectiveness, cost–benefit analysis, delivery options, and lab infrastructure issues/practicality (e.g., rearing for Leafminers) (CFIA, 2011f). The prioritization tool then requires risk ranking of the prospective survey along five dimensions using a five-point scale indicating the risk of not doing the survey in the subsequent year. The five dimensions that are given a ranking include: timing or urgency, biological/environmental risk, risk to trade, reputation/stakeholder involvement risk, and risk to mandate and/or outcome (CFIA, 2011f).

2.2.3 Contingency, preparedness, and mitigation

Contingency, preparedness, and mitigation activities relate to the plant-related emergency response and planning, and control and eradication of plant pests. These activities, along with the inspection, surveillance, and certification activities noted above, form part of the CFIA's activities under Canada's IAS strategy, which is discussed in more detail below.

An emergency is defined as a "present or imminent event that requires prompt coordination of actions concerning persons or property to protect the health, safety, or welfare of people, or to limit damage to property or the environment" (CFIA, 2013l). The CFIA maintains an overarching Emergency Response Plan (ERP) for the Agency; Functional Plans for each business line (i.e., food, animal, and plant); and Hazard-Specific Plans (e.g., ALHB) (CFIA, 2013f, 2013i).

The CFIA has defined procedures for the activation of ERPs at the national level (CFIA, 2013h, 2013i). Once a potential issue has been identified, it is assessed against a predetermined set of criteria to determine the need to activate an ERP. Activation of an ERP involves the following actions:

  • mobilization of an Emergency Response Team (ERT)
  • use of the Incident Command System (ICS)Footnote 18
  • activation of an Emergency Operations Centre (EOC)Footnote 19

Through its support activities, and to provide a scientific basis for the management of risk associated with plant pests, the CFIA undertakes a three-stage PRA process to support its contingency, preparedness, and mitigation work involving:

  • initiation, which "involves identifying pests and pathways of concern and defining the PRA area;"
  • pest risk assessment, which "provides the scientific basis for the overall management of risk;" and
  • pest risk management, which is "the process of identifying and evaluating potential mitigation measures which may be applied to reduce the identified pest risk to acceptable levels and selecting appropriate measures" (CFIA, 2013s).

The CFIA publishes findings from its pest risk assessment and pest risk management processes in Risk Management Documents (RMD).

Once a pest is detected, the PPP conducts two main activities to eliminate the pest and/or slow its spread to other regions: eradication and issuing declarations.Footnote 20

  • Following international convention and scientific guidance, the PPP only uses eradication when "it makes sense to invest a lot of resources to reach an attainable goal of pest-freedom" (CFIA, 2012l, p. 2); in cases in which the Minister decides to proceed with eradication, the PPP develops eradication protocols to guide the process.
  • When a pest has established itself and the cost of eradication outweighs the negative impact of the pest and/or the potential likelihood of success is low, it no longer makes sense to eradicate, and the PPP may then undertake pest management, in an attempt to slow the spread of the pest (CFIA, 2012l).

The PPP's response to Plum Pox Virus (PPV) stands as an example of the PPP's use of both eradication and management. From the point of PPV's discovery in Canada in 2000, the response to the plant disease has involved three initiatives:

  • The PPV Suppression and Eradication Feasibility Program (2001–2002 to 2003–2004), the objective of which was to "assess the feasibility of eradication; eradicate the virus where possible; suppress the spread of PPV; maintain the viability of the affected industries (tender fruit, fruit processing, and nursery); and increase knowledge of the virus" (AAFC & CFIA, 2011).
  • The Plum Pox Eradication Program (PPEP) (2004–2005 to 2010–2011), the objective of which was to eradicate PPV in Canada to increase the tender fruit sector's viability and profitability.
  • The Plum Pox Monitoring and Management Program (PPMMP) (2011–2012 to 2015–2016), the objective of which is to contain PPV in the Niagara quarantine area, implement measures to manage and mitigate the spread of the virus, and maintain industry viability and profitability.

In addition to eradication and pest management, the PPP takes a number of additional regulatory activities. One such activity involves the issuance of movement certificates allowing facilities to transport goods across regulated areas within the country. For example, the PPP offers approved facilities compliance programs with respect to specific pests such as Emerald Ash Borer (EAB) and Hemlock Woolly Adelgid. These compliance programs enable approved facilities in good standing to ship regulated articles out of regulated areas without a pre-shipment CFIA inspection; they also allow facilities in non-regulated areas to receive regulated articles during certain low-risk times of the year (CFIA, 2009a, 2012d). Facilities must meet several requirements to become an approved facility and must pass regular audits; as an enforcement measure, they may lose their approval status if an inspection reveals they are no longer compliant with the relevant directive or program terms.

The CFIA also implements various risk-mitigation strategies. For example, the Brown Spruce Longhorn Beetle (BSLB) risk-mitigation strategy sets out requirements for treatment, containment, manufacturing, and disposal of regulated materials, such as spruce and firewood, which are at a high risk of spreading BSLB. The program allows qualified facilities to move regulated materials from regulated areas (as per the declaration) to non-regulated areas using CFIA Movement Certificates. Participants must "develop and document their quality system using a site-specific plan" and the PPP audits these facilities regularly to ensure compliance (CFIA, 2013b).

When requiring certain eradication or mitigation activities, the PPP has the ability to provide compensation payments to affected individuals. As noted in the PPA, the Minister may order compensation to be paid given any of the following circumstances:

  • "any treatment of a place or any treatment, storage or disposition of a thingFootnote 21 required under [the Plant Protection Act] or the regulations
  • any prohibition or restriction on the use of a place or on the movement of persons or things within, into or out of a place imposed under [the Plant Protection Act] or the regulations
  • any prohibition or restriction on the use of a thing or on the sale or other disposition of a thing imposed under [the Plant Protection Act] or the regulations" (GoC, 2005)

No compensation is payable if:

  • "a thing…is imported into Canada or exported from Canada in contravention of [the Plant Protection Act] or the regulations or a thing that is found to be a pest, to be infested with a pest or to constitute a biological obstacle to the control of a pest when it is inspected on importation or exportation;
  • [there is a] prohibition or restriction of the sale or movement of a thing where the sale or movement is prohibited or restricted as a result of an amendment, suspension or revocation of, or a refusal to issue or renew, a permit, certificate or other document that is required under [the Plant Protection Act] or the regulations; or
  • …a person…commits a violation, or an offence under [the Plant Protection Act], and claims compensation in respect of any place or thing by means of or in relation to which the violation or offence was committed" (GoC, 2005).

2.2.4 International engagement and standard setting

The PPP's international engagement and standard-setting activities include the following:

  • Maintaining scientific partnerships with various federal departments and agencies. These partnerships may inform, contribute to, and/or influence a wide range of activities, including, but not limited to, policy and standard setting, regulatory and program development, market access agreements, scientific risk assessments, surveillance, science foresight, inspection, and risk mitigation (CFIA, 2013o).
  • Facilitating market access. As noted in the logic model in Figure 3 above, one of the intermediate outcomes of the PPP is to maintain international market access for Canadian producers. This is "critical to the sustainability of the agriculture, fishery, forestry, and food industry and the economic prosperity of Canada" (CFIA, 2012i). The CFIA engages in the following market access activities for plant products:
    • "negotiating import and export conditions in a complex international environment;
    • demonstrating the integrity of Canada's food, plant and animal regulatory system to Canadians and our trading partners; and
    • representing Canada's interests in international forums through the provision of technical support and in negotiating technical agreements and standards" (CFIA, 2012i).

    The CFIA establishes its work priorities with respect to market access in collaboration with Agriculture and Agri-Food Canada's (AAFC) Market Access Secretariat (MAS). The following are some of the factors that the CFIA considers in establishing its work priorities:

    • value: economic importance; value to industry
    • cost: anticipated difficulty (resolution cost); system considerations (implementation cost)
    • constraints: tariffs; Ministerial commitments; scheduling urgency (CFIA, 2012b)

    The MAS' prioritization tool also helps inform work priorities. As shown in Table 4, this tool uses scoring criteria, with each area of evaluation assigned a respective weight, to prioritize market access issues. Using this methodology, the MAS ranks various market access issues in a Plant Issues Prioritization Report (AAFC, 2013b).

    Table 4: Scoring criteria for market access work priorities
    Area of evaluation Weight
    Industry priority rating 18
    Existing ministerial commitment 18
    Relative economic importance of the market 17
    Anticipated difficulty of resolution 15
    Systems considerations 14
    Preferential tariff treatment 10
    Scheduling urgency 8
    Total 100

    Source: (CFIA, 2012b)

  • Participating in international standard setting. Canada is one of 180 signatories to the International Plant Protection Convention (IPPC) agreement, which is an international agreement that "aims to protect cultivated and wild plants by preventing the introduction and spread of pests" (IPPC, n.d). As the NPPO for Canada, the CFIA participates in the development of International Standards for Phytosanitary Measures (ISPMs) for safeguarding plant resources (IPPC, n.d). ISPM 15,Footnote 22 for example, requires the standardized treatment of all non-manufactured wood packaging used in international trade.

    The CFIA's Engagement Plan in IPPC Activities identifies a number of planned undertakings for the coming fiscal year. These include, for example, meetings meant to:

    • develop new ISPMs;
    • discuss administrative issues related to ISPMs;
    • engage with bilateral delegates;
    • review standards for IPPC member consultations; and
    • develop treatment criteria under ISPMs (CFIA, 2013g).
  • Engaging in international forums. The CFIA, on behalf of Canada, is a member of the North American Plant Protection Organization (NAPPO). Additionally, the CFIA participates in multilateral forums with its regulatory counterparts in QUAD governments (Canada, the US, Australia, and New Zealand).

2.3 Program stakeholders

The work of the PPP is dependent on a number of external partners and organizations that have a vested interest in plant protection. Further, several federal departments, including some discussed above, work with the CFIA on a variety of activities related to plant protection. These include but are not limited to the following:

  • The CBSA enforces, at Canadian border entry points, the CFIA's policies and regulations for the importation of food, agricultural inputs, and agricultural products; this includes plants and plant products (including wood packaging material). Where required, the CBSA refers specific shipments to the CFIA for follow-up action (CFIA, 2012i). Roles and responsibilities between the CBSA and CFIA are governed through a MOU (CBSA, 2011).
  • AAFC provides the CFIA with guidance on policy, programs, and activities that support the agricultural sector, including research, maintenance of pest reference collections, and business risk management efforts, such as compensation payments for specific plant pests (CFIA, 2012i). Additionally, AAFC houses the MAS, which is "a single-window to the Federal Market Access Team, a portfolio which includes [AAFC, the CFIA], and Foreign Affairs and International Trade Canada (DFAIT)Footnote 23 and takes a collaborative approach to advance interests for Canadian agriculture abroad" (AAFC, 2013a).
  • Canadian Forest Service (CFS), operating under Natural Resources Canada (NRCan), is the principal provider of scientific and technical support on forest pest matters, including the identity, biology, and ecology of forest pests, as well as the ecological and economic impacts of forest pests, to all jurisdictions, including the CFIA. Further, the CFS develops toolsFootnote 24 and strategiesFootnote 25 for the CFIA to support evidence-based decision making and reduce the establishment, spread, and damage of insects and disease. For example, the CFS has worked with the CFIA to create and establish the ISPM 15 wood packaging standard and develop phytosanitary controls aimed at preventing the spread of alien pests (CFS, 2013a). The roles and responsibilities between the CFS and CFIA are governed through a MOU (CFIA, 2012i).
  • Health Canada is responsible for assessing the safety of novel foods and works with counterparts at the CFIA who are responsible for regulating plants with novel traits, novel supplements (micro-organisms), and novel feeds (CFIA, 2012i).
  • Canadian Grain Commission (CGC) works collaboratively with the CFIA in developing programs and policies that impact the grain industry. The CGC also performs grain elevator inspections on behalf of the CFIA (CFIA, 2012i). Roles and responsibilities between the CGC and CFIA are governed through a MOU (CFIA, 2012i).

The PPP, and the CFIA more generally, also maintain a number of partnerships in support of the IAS Strategy for Canada and its constituent action plans for plant pests. These involve, but are not limited to, the following federal partners:

  • AAFC is involved in a variety of research activities meant to support national responses to invasive plants and plant pests. It houses the AAFC Collection of Vascular Plants and the Canadian National Collection of Insects, Arachnids, and Nematodes; maintains a specialized library with related literature; develops diagnostic and surveillance techniques; and supports research conducted outside of the department (CFIA, 2006).
  • Environment Canada (EC) is the coordinating body for Canada's IAS strategy. The department contributes to a number of legislative efforts, programs, and policies related to invasive plants and plant pests, as well as engaging in communication activities with the general public. It "conducts and supports research, monitoring, and management activities for invasive alien plants that pose a risk to Canada's Species at Risk" (CFIA, 2006).
  • NRCan's CFS provides scientific expertise in support of the plant action plan, helping to reduce the impacts of IAS on Canadian forest and the forestry sector. A MOU exists between the CFS and the CFIA regarding the provision of various forms of scientific advice, interpretation, and support (CFIA, 2006).
  • Parks Canada Agency (PCA) is involved in the plant action plan through its management of federal land, where it is involved in efforts to maintain native ecosystems (CFIA, 2006).
  • The CBSA provides border security services related to invasive plants and plant pests under the direction of other federal departments (CFIA, 2006).

Under the plant action plan, a number of federal bodies indirectly contribute to the successful mitigation of plant-related IAS risks through their regular work. These include, for example, the National Research Council, Health Canada, the Pest Management Regulatory Agency, Transport Canada, and the Department of National Defence (CFIA, 2006).

Provinces, municipalities, First Nations, and nongovernmental organizations also contribute to the success of the plant action plan. As described in the action plan itself, the roles and jurisdiction with regards to invasive plants and plant pests for these groups are as follows:

  • Although provinces and territories do have some legislative authority allowing them to undertake action related to invasive plants and plant pests, there are gaps in various jurisdictions limiting the scope of provincial and territorial work. For example, in only some instances, weed control or agricultural legislation may be used to control IAS. In only two provinces, forestry legislation includes a prohibition on plant pest and disease imports (CFIA, 2006).
  • Where delegated authority from provinces exists, municipalities have the option to enact bylaws related to the control of invasive plants and plant pests. Further, municipal infrastructure may be used to undertake detection, surveillance, and direct response activities (CFIA, 2006).
  • First Nations may also enact bylaws under the Indian Act or self-government agreements (CFIA, 2006).
  • Nongovernmental organizations, with a mandate for environmental preservation, may be actively involved in public education related to invasive plants and plant pests. As noted in the action plan, the familiarity many of these organizations have with their local environments often allows members to identify changes that might be related to the presence of invasive plants and plant pests (CFIA, 2006).

An example of the coordinated efforts of some of these groups may be seen in the management of PPV. As part of the PPMMP, the Government of Canada allocated $300,000 to AAFC and the Ontario Ministry of Agriculture and Food (OMAF) for education and awareness activities, including publishing education materials; conducting industry presentations; and providing advice to, and training, growers.

Beyond these relationships, CFIA is also involved in various interdepartmental committees or working groups devoted to plant health-related issues. Examples include the following:

  • Canadian Invasive Plants Framework (CIPF) Steering Committee. The CFIA participates in the CIPF's Steering Committee along with the following federal departments: AAFC, EC, Fisheries and Oceans Canada (DFO), NRCan, and PCA. Additionally, the CFIA is the lead Agency responsible for the implementation of the CIPF, and its Plant Health Biosecurity Directorate serves as the Secretariat responsible for overall delivery of the Framework. The CIPF "provides the basis for a nationally coordinated, multi-jurisdictional response to invasive plants using both regulatory and non-regulatory tools" (CFIA, 2011b).
  • Early Detection and Rapid Response Forum. This working group comprises provincial and federal partners and other stakeholders with an interest in, and concerns about, invasive plants in Canada. Conference calls, usually chaired by the CFIA, are held monthly to discuss invasive plants topics raised by forum members and to share information among jurisdictions.
  • Canadian Forestry Phytosanitary Working Group (CFPWG). The CFIA, the CFS, and the Department of Foreign Affairs, Trade and Development (DFATD), along with representatives from industry, participate in the CFPWG (CFIA, 2008a). Created in 2008,the goal of the CFPWG is to establish an effective and efficient system of communication between stakeholders and partners on matters relating to forestry phytosanitary policies, procedures, and market access issues (CFIA, 2010a, 2012j).
  • Canadian Interdepartmental Working Group on the Implementation of ISPM 15. This working group, established by the CFIA, includes representatives from the CBSA, DFATD, EC, Industry Canada, the Privy Council Office, and Transport Canada. It is tasked with developing compliance and enforcement recommendations for the implementation of harmonized bilateral import requirements between the US and Canada under the ISPM 15 wood packaging standard (CFIA, 2013q).
  • BSLB Steering Committee. The CFIA and the CFS co-chair a BSLB Steering Committee, which includes industry stakeholders, community partners, and federal and provincial government representatives. The Committee works together to develop management strategies to effectively monitor and limit the spread of BSLB (CFIA, 2012l).
  • CFIA Expert Advisory Committee (EAC). Members of the CFIA's EAC include representatives of the CFIA, AAFC, Health Canada, the Public Health Agency of Canada (PHAC), universities, and the private sector (CFIA, 2012f). The mandate of the EAC is to provide the CFIA with objective professional and technical advice on key issues related to the Agency's three business lines, including plant health. This may include linking academic and scientific research, commenting on policy, program frameworks, priorities and risk management strategies, and studying industry and international trends and developments (CFIA, 2012k).

3.0 Evaluation objectives

The PPP evaluation had a number of objectives:

  • provide critical information in support of federal decision making and resource allocation
  • assess the relevance and performance of the program to inform program improvement and Agency Transformation
  • meet the requirements of TB Policy on Evaluation (TBS, 2012), as well as its supporting directive and standard and the related CFIA Policy on Evaluation (CFIA, 2013e)

As per the Treasury Board of Canada Secretariat's (TBS) Directive on the Evaluation Function, the evaluation explored the following five core evaluation issues:

  • continued need for the program
  • alignment with government priorities
  • alignment with federal roles and responsibilities
  • achievement of expected outcomes
  • demonstration of efficiency and economy (TBS, 2009)

Further, the CFIA's 2013–2018 Evaluation Plan noted that the PPP evaluation was to include an examination of three areas of interest:

  • the PPMMP (which also included consideration of the PPEP)
  • IAS
  • compensation payments in accordance with regulations under the PPA (CFIA, 2013d, p. 18)

Case studies on PPMMP and IAS were undertaken to provide a more in-depth look at these funded initiatives, supporting increased accountability and public reporting in these areas. The compensation payments case study was undertaken to explicitly satisfy evaluation coverage requirements for these payments and to explore any best practices or areas for improvement applicable to these or other compensation payments managed by the Agency.

Appendix B contains the evaluation matrix developed for the PPP evaluation. The evaluation matrix included a number of questions associated with each of the five core evaluation issues noted above. It further aligned each of these questions with the PPP as a whole, or one of the four commodity areas or IAS. This alignment was meant to identify the aspects of the program to which each question most directly related. Finally, the matrix included potential indicators and data sources for each evaluation question.

4.0 Evaluation design

The evaluation of the PPP was designed with four general steps in mind:

  • defining the evaluation scope
  • establishing the evaluation approach
  • identifying the evaluation methods
  • analyzing and integrating evaluation findings

Challenges and limitations associated with the evaluation design were subsequently identified and mitigating strategies were developed throughout the work.

4.1 Evaluation scope

The depth of inquiry for the evaluation of PPP was calibrated in accordance with the "category B" classification it was assigned during the development of the CFIA's 2013–2018 Evaluation Plan.Footnote 26 The PPP's logic model served as the lens through which to examine the program, focusing the work on the activities and outcomes included in the model. Particular focus was placed on program delivery activities and their outcomes, with inferences about program support activities made through their relationship to program delivery. The evaluation scope was further defined by the period over which the program was examined – April 1, 2009 to September 30, 2013 (CFIA, 2013v, p. 2).

4.2 Evaluation approach

Development of a framework to guide the evaluation was initiated in January of 2013. The evaluation, officially launched in August 2013, took place in three stages:

  • Stage I: Data collection – relevant data was collected from the program and a number of additional stakeholders. This stage took place from September 2013 to February 2014.
  • Stage II: Analysis and report writing – data collected during Stage I was analyzed and draft as well as final versions of the evaluation report were developed. This stage took place from February 2014 to March 2014.
  • Stage III: Report review – the final version of the evaluation report was reviewed by the various CFIA governance committees. This review process took place from March 2014 to August 2014.

Delays in data collection activity and the complexity of the program resulted in the unanticipated continuation of Stage I throughout the entirety of both Stage I and Stage II.

4.3 Methods overview

As per the evaluation matrix in Appendix B, the evaluation used the following four approaches to data collection:

  • a document and literature review
  • a secondary data review and cost analysis
  • case studies (n=3)
  • key informant interviews (n=57)

An evaluation working group was established during the development of the PPP's evaluation framework. This group included representation from the Plant Health and Biosecurity Directorate, the Operations Strategy and Delivery Directorate, the Plant Health Science Directorate, and Corporate Management Branch. Members of the working group provided access to documentation, literature, and potential key informants; liaised with external stakeholders; coordinated evaluation support within their respective organizations; and validated the evaluation findings.

4.3.1 Document and literature review

The document and literature review involved examining over 700 documents and served three main purposes:

  • First, by including internal CFIA documents, it helped establish the degree to which the PPP's main goals were defined, the degree to which its activities aligned with various government priorities, and whether the causal logic that underpins the PPP was articulated in CFIA documentation.
  • Second, by including external policy documentation – particularly documents related to broader federal priorities and responsibilities – it supported insights into the second and third core evaluation issues noted in the evaluation matrix in Appendix B.
  • Third, by including academic and professional literature, the review explored evidence of the PPP's regulatory success and alternative program delivery mechanisms offered in other jurisdictions. It also helped establish the rationale for the PPP.

4.3.2 Secondary data review and cost analysis

The secondary data review and cost analysis collected and analyzed available activity, outcome, and financial information related to the PPP. The intent of the secondary data review and cost analysis was twofold.

  • First, while focusing on the achievement of PPP outcomes, this line of evidence provided objective information about PPP success. This differed from the review of policy documentation or the collection of key informant interview opinion, which could have unintentionally introduced subjective bias into the evaluation.
  • Second, the combining of the outcome-related data discussed above and the PPP cost information enabled the evaluation to speak to issues of program efficiency and economy.

4.3.3 Case studies

Three case studies were undertaken to explore (in greater detail) the aforementioned three areas of interest:

  • the PPMMP (which also included consideration of the PPEP)
  • IAS
  • compensation payments in accordance with regulations under the PPA

Each of the case studies followed an established protocol, which generally involved the review of related documentation, including previous reports by the Office of the Auditor General and the CFIA, the review of available performance and cost data, and a series of key informant interviews. In this way, the case studies built on previous work completed in these three areas. A copy of this protocol is available in Appendix C.

There were initially five interviews planned for each case study, separate from the general evaluation key informant interviews discussed below. However, as the evaluation progressed, it became clear that each of the areas of interest overlapped heavily with other PPP-related work. This meant that some information from the general evaluation key informant interviews was also applicable to the case studies and was therefore integrated into the case study work. A copy of the case study interview guide is included in Appendix D.

4.3.4 Key informant interviews

Key informant interviews were used during the evaluation to supplement information gathered from the aforementioned methods. In addition, the key informant interviews allowed individuals inside and outside of the CFIA to provide explanatory and contextual comments related to the PPP's delivery and success.

Given the overlap between the case study interviews and those conducted as part of this data collection activity, it was difficult to establish independently the number of interviews associated with the case studies and the key informant interview activity. Nonetheless, as planned, nearly 60 interviews were completed as part of the evaluation. In addition, one group discussion was held with provincial representatives involved in plant health-related activities.

Interviewees included individuals within the CFIA and stakeholders from organizations outside of the Agency. The positions of interviewees within the CFIA, as well as the organizations outside of the Agency, who participated in the evaluation included, but were not limited to, the following:Footnote 27

Within the CFIA
  • executive directors
  • directors
  • program managers
  • national managers
  • program specialists
Outside of the CFIA
  • members of industry organizations
  • representatives from other federal departments and agencies
  • representatives from provincial agricultural ministries
  • a representative from the United States Department of Agriculture

The key informant interviews provided qualitative insight into the PPP, its activities, and its outcomes. The semi-structured nature of the interview process meant that even when more than one interviewee commented on a given theme or subject area, the nature of their comments often differed significantly and supported a detailed understanding of the theme or subject area.

Potential interviewees were identified based on consultations between the evaluation working group and the evaluation team, as well as suggestions from other key informants. Selection of the key informants to be interviewed took into consideration their area of expertise and the length of time they had been involved with the PPP. The final set of key informants interviewed included those who could provide broad perspectives on the PPP as a whole, as well as those who could speak to specific program activities.

To help interviewees prepare, a general interview guide was distributed in advance of each scheduled interview. However, interviewers were required to tailor each question to the individual respondents during the interview itself. Copies of the guide can be found in Appendix E. In some cases, less structured discussions were held with individuals to gather additional information about the PPP and program alternatives. When appropriate, key informants were asked for documentation to support their observations.

4.4 Analysis and integration of findings

As the evaluation matrix implies, no single data collection method was used to address an evaluation issue in its entirety. Rather, the evaluation combined information from all four methods to speak to each of the identified evaluation issues.Footnote 28

4.5 Challenges, limitations, and mitigating strategies

The following table presents a number of challenges and limitations facing the evaluation, along with the mitigation strategies undertaken to address each. The table also includes implications that these challenges and mitigation strategies may have had for the evaluation.

Table 5: Challenges, limitations, mitigation strategies, and implications
Challenge/limitation Mitigation strategy Implications
Program scope – the PPP involves a significant range of activities in a number of variations within each. It was not possible to review all aspects of the program's activities in detail during the evaluation. The evaluation defined its scope so as to directly assess four of the PPP's activity groups and associated outcomes while indirectly drawing inferences regarding the remaining four activity groups.

The evaluation team also consulted with the working group and advisory committee to identify critical evaluation issues to explore.

The evaluation scope was, in part, defined to address this challenge. The implication was that while some program issues were discussed in detail during the evaluation, other less pressing issues were not examined.
Disparate information – despite the significant number of documents available for the evaluation, many of these fell into one of two categories:
  1. High-level program documentation that spoke to the PPP in only the most general terms
  2. Technical documentation that provided a significant amount of detail on specific aspects of the PPP, rather than the program as a whole

This made it difficult to establish an overall picture of the PPP's operations during the evaluation, especially in areas where technical information discussed under point 2 above was limited.

The evaluation devoted a significant amount of time and effort to the document and literature review to establish a coherent picture of the PPP and its delivery.

The evaluation team engaged the working group to identify any missing information and correct any factual errors associated with the document and literature review.

The document and literature review required a considerable portion of the evaluation's resources. While this was necessary in order to develop a comprehensive profile of the program, it limited the evaluation's ability to pursue other lines of evidence as extensively as might have otherwise been possible. More extensive interviewing or case study work could have shed additional light on points raised during the document and literature review.
Difficulty contextualizing information – where quantitative activity data for the PPP were available for the evaluation, little information was available to contextualize these data. This made it difficult, if not impossible, to draw insight from these data in support of the evaluation. The evaluation relies more heavily on anecdotal evidence of success, along with logical arguments regarding the plausibility of the PPP's effectiveness. At the same time, the evaluation used this challenge as an opportunity to highlight the need for this type of data collection as part of the program. The evaluation relied less heavily on quantitative data to assess the impacts of the PPP while highlighting the effects that limited qualitative-based contextual information can have on the program.
Considerable external factor influence on the program – despite the fact that individuals were able to point to anecdotal evidence of program success, a significant number of external influences appear to heavily influence the PPP's outcomes. This made it difficult to assess the contribution of the program's activities, even notionally. Much like when addressing the difficulty of contextualizing information, the evaluation relied on anecdotal evidence along with a strong program theory to establish the contribution of the PPP to its outcomes. The evaluation relied less heavily on quantitative data to assess the impacts of the PPP. A detailed analysis of all external influences on the program was outside the scope of the evaluation. However, the evaluation was able to identify many of these influences and suggest how they might affect the program's performance.
A wide variety of stakeholders – the large number of stakeholders affected by or affecting the success of PPP makes it difficult to understand the way in which the program operates and is influenced by those outside of the Agency. The evaluation attempted to integrate input from a wide range of PPP stakeholders to better understand the program. The evaluation was able to speak to the complicated and sometimes conflicting interactions between the program and its stakeholders.

5.0 Findings

This section presents findings from the evaluation according to the issues identified in the evaluation matrix. These include:

  • program need;
  • government priorities;
  • federal roles and responsibilities;
  • achievement of expected outcomes; and
  • efficiencyFootnote 29 and economy.Footnote 30

5.1 Program need

Summary of findings: Evidence from the evaluation suggests that there are clear biosecurity risks associated with plant pests, including IAS, in Canada. These plant pests have the potential to result in a loss of biodiversity, restrict market access for Canadian producers, impose direct financial impacts on Canadians, reduce food security, and affect human health. Nationally, all of these can affect Canadian trade and economic interests. By attempting to limit the introduction and spread of plant pests in Canada, the PPP directly addresses this demonstrated need. At the same time, the program helps meet Canada's obligations under a number of international agreements and conventions. Importantly, the program does so in an environment where it would be unlikely that private sector stakeholders would consistently, and voluntarily, address biosecurity risks on their own.

5.1.1 Need to address risks and impacts associated with plant pests

The activities of the PPP attempt, in large part, to prevent and/or minimize biosecurity risks and impacts associated with plant pests, including IAS, in Canada. Biosecurity risk refers to the risk of plant pests entering and spreading within a region and damaging the Canadian plant resource base.

Globalization and trade represents, by far, the most commonly-cited source of biosecurity risk, as it enables plant pests and diseases to travel across borders quickly and through a variety of channels. These channels include travel, mail, and the transportation of goods by land, sea, and air (MacLeod, Pautasso, Jeger, & Haines-Young, 2010; Outhwaite, 2010; Roberts, 2009). These risks have increased over time due to increases in the volume of trade, the frequency and distance over which people and commodities move, greater variety of transport and packaging methods, and wider range of goods transported (Outhwaite, 2010). To the extent that nations are increasing their number of trading partners, this may also increase risk. All of this indicates a strong need for the numerous PPP activities (certification, inspection, etc.) that work to prevent biosecurity risks resulting from trade.

When these biosecurity risks are realized, and plant pests enter and spread throughout Canada, this can lead to numerous environmental and economic consequences. These include, but are not limited to, a loss of biodiversity, trade interruptions and/or restricted market access, financial implications, and reduced food security.

  • Loss of biodiversity.Footnote 31 The Secretariat of the Convention on Biodiversity identifies IASFootnote 32 as species that have moved beyond their natural habitats and, as a result, have an impact on their new environments (Secretariat of the Convention on Biological Diversity, 2014a). The introduction of IAS is recognized as "one of the most significant threats to biodiversity, after habitat loss" (Outhwaite, 2010).

    Studies estimate that 24% of the species at risk in Canada may be threatened with extinction by IAS (Environment Canada, n.d.). The case of American chestnut blight illustrates how plant biosecurity issues can reduce biodiversity. It was introduced to North America from Asia in the late 1890s, spread extensively, and eventually caused the virtual extinction of the American chestnut in its natural range (Outhwaite, 2010). Dutch elm disease provides another example, as it has virtually eliminated elms from their natural range (Outhwaite, 2010).

  • Restricted market access. Plant pests can have a substantial negative effect on a country's ability to export its own plant products. For example, McKirdy, Rodoni, Moran, & Sharma (2012) note that Australia's relative freedom from many harmful plant pathogens has provided the country with a competitive advantage in terms of market access. Their lack of plant pathogens means that Australia can easily export its agricultural products to trading partners who guard against the introduction of these pathogens. As Rodoni (2009) states, quality assurance measures for pathogens are necessary to promote market access to those jurisdictions with similar safeguards.
  • Financial implications. Although difficult to quantify, many experts agree that the economic impacts of plant pests can be significant (CFIA, 2008b):
    • The Secretariat of the Convention on Biodiversity estimates that annual global costs from IAS may exceed $1 trillion (Secretariat of the Convention on Biological Diversity, 2014d).
    • Environment Canada (n.d.) notes that the cost of damage caused by IAS affecting forestry and agriculture is approximately $7.5 billion annually.
    • A 10%–16% reduction in global harvest due to plant diseases has been estimated to translate into global financial losses of approximately $220 billion each year (Chakraborty & Newton, 2011). Assuming that plant diseases reduce Canadian harvest by 10%–16%, a rough estimate of the value of crop losses from plant diseases in Canada is $3–$6 billion per year.Footnote 33 This estimate does not include the forestry sector and the impact of plant pests. Therefore, the full economic loss of plant pests and diseases may be substantially higher.
    • The introduction of plant pests may lead to the use of pesticides to manage these pests, thereby imposing a direct cost on agricultural producers (CFIA, 2012a).
  • Reduced food security. The UN Food and Agricultural Organization's definition of food security states that "food security exists when all people, at all times, have physical and economic access to sufficient, safe and nutritious food that meets their dietary needs and food preferences for an active and healthy life" (FAO, 2006, p. 1). Biosecurity risks and the consequent loss of plant species create food security risks due to reduced crop yields. For example, Flood (2010) notes that plant pest problems are major contributing factors to crop losses, resulting in reductions of up to 30%–40% of crop production annually.

    Although food security is often considered to be a concern only for developing countries, it is increasingly relevant to developed countries such as Canada (Momagri, n.d.). This is because reductions in the food supply and increases in food prices may encourage individuals to shift to lower cost, less healthy food items.

  • Reduced human health: Invasive plants and plant pests may have immediate impacts on human health, including the introduction of disease (CFIA, 2012a). For example, fungi, including Ergot or Fusarium, found on some plants can be harmful to human health.

5.1.2 Need to meet international and national obligations

The PPP also helps Canada meet its obligations under a number of international agreements and conventions related to plant health:

  • IPPC. In 1951, Canada became a signatory to the IPPC. The convention was meant to promote the control of invasive plants and plant pests internationally. The "IPPC is the official standard setting body under the World Trade Organization Sanitary and Phytosanitary Agreement," and the agreement outlines basic plant health standards internationally (CFIA, 2006).
  • United Nations Convention on Biological Diversity. Following the signing of the IPPC, Canada ratified the United Nations Convention on Biological Diversity. Signed in 1992, the convention called for nations to address IAS actively through both limiting their introduction and undertaking measures to control and eradicate them once they are introduced. More specifically, the convention called for the development of national biodiversity strategies among signatory nations. Canada developed its first strategy in 1995 (Environment Canada, 2011).

In 2004, federal, provincial, and territorial ministers approved a blueprint for a national IAS strategy – entitled An Invasive Alien Species Strategy for Canada (Environment Canada, 2011). Three work plans were identified as part of the strategy that were related to the mitigation of risks associated with aquatic invasive species, terrestrial plants, and terrestrial animals (CFIA, 2006). Specifically, Canada's Action Plan for Invasive Alien Terrestrial Plants and Plant Pests outlined how Canada would coordinate its efforts to deal with invasive plants and plant pests.Footnote 34 Overall, the plan was to minimize the "negative effects on the environment, economy and social values" (CFIA, 2006). The CFIA, as the NPPO in Canada, is charged with contributing to the achievement of these goals and to those outlined under the international agreements and conventions above.

5.1.3 Government provision of plant biosecurity measures

In the field of economics, government intervention is often justified in situations where the private market fails to provide the optimal or "efficient" allocation of goods and services. There are a number of reasons why the private sector would fail to provide an optimal level of plant biosecurity, thereby justifying government intervention. These reasons include the existence of various negative externalities, the public goods nature of plant health, and the existence of information asymmetries in the area of plant health.

  • Negative externalities are costs imposed on third parties resulting from production or consumption of a good or service. To illustrate negative externalities in a plant protection context, consider the case of an importer and exporter of plant products. If an importer purchases a product from abroad that carries a plant pest, there is a possibility that the pest will infect plants in the importer's country. This will have wide-reaching effects far beyond the individual importer responsible for creating this biosecurity risk (MacLeod et al., 2010). Since most of these costs of dealing with these biosecurity risks are born by others – such as local producers and the general public – and not the importer and the associated exporter, they may not account for such costs (e.g., spending the time and effort to take precautions to mitigate this risk) in their decision to buy and sell the product. Therefore, government programs such as the PPP have a role to impose appropriate requirements on market participants through regulation, surveillance, inspection, certification, and enforcement so they have sufficient incentives and/or disincentives to avoid biosecurity risks associated with plant pests.
  • Public goods, have two defining features – non-rivalry (consumption of the good does not prevent others from consuming the good) and non-excludability (individuals who do not pay for the good cannot be excluded from consumption) (Lansink, 2011). For example, when a producer engages in pest management activities, such as removing infected plants to prevent the spread of pests, others benefit from their activities. Since the first producer bears the costs of these activities but the second gets a substantial portion of the benefits, it results in what is often called the "free-rider problem." The first producer is not fully compensated for their effort, and the second has no incentive to take similar actions.

    Extending this example to include consideration of compensation paymentsFootnote 35 demonstrates the rationale for government involvement in the protection of plant health. In this case, compensation payments offered under the PPA are meant to directly compensate individuals for taking mitigating activities to help prevent the further spread of pests within the country. In this scenario, the CFIA is using publicly-funded compensation payments paid to individuals to benefit a broader group of Canadians. Examples of other PPP activities intended to protect this public good include regulation, surveillance, inspection, certification, enforcement, and eradication.

    Some key informants argued that industry should bear a larger portion of the costs associated with protecting Canada's resource base. For example, they believe that since global trade of plants and plant products increase a country's exposure to plant pests, importers and exporters, who directly benefit from the trade activities, should pay a larger portion of inspection costs. Currently, the CFIA imposes user fees on importers and exporters. The fees are fixed under the authority of the Plant Protection Act and are for the purpose of recovering all or some of the costs that are incurred by the Agency in rendering the specified services to service recipients. Some key informants suggested that the user fees need to be updated, or other cost recovery measures need to be introduced.

  • Information asymmetryFootnote 36 theory predicts that industry will not consistently undertake costly risk-mitigation activities independently of government intervention. This is because of differences in information about plant health among buyers and sellers of plant products. Without some type of quality control system or certification process, the phytosanitary quality of a plant product will be known to sellers but unknown to buyers (Lansink, 2011). If a buyer knew about the poor phytosanitary quality of a plant product before they purchased it, they might require compensation in the form of a lower price to offset the costs of mitigating the phytosanitary risks. The seller might therefore take measures to mitigate this risk in advance of the sale in order to demand a full price. However, if the buyer is unaware of the quality, and will therefore not demand a lower price, there is no incentive for the seller to undertake costly mitigation activities (MacLeod et al., 2010). Programs such as the PPP help address this information asymmetry through plant inspection and certification processes.

5.2 Government priorities

Summary of findings: The evaluation noted that three Government of Canada outcome areas are supported by the PPP. These include healthy Canadians, strong economic growth, and a clean and healthy environment. These outcome areas aligned quite closely with the CFIA's raison d'être, the PPP strategic objective, and the purpose of the PPA. While distinct, these three areas are related insofar as a healthy environment – and specifically a healthy plant base – provides the foundation for a variety of economic activities in Canada and contributes to the health of Canadians.

With that said, the strategic outcome for the CFIA, which is the endpoint in the PPP's logic model, does not explicitly acknowledge the program's contribution to the Canadian economy. Although this does not undermine the implied connection between a safe and accessible plant resource base and economic activity, arguably, it decreases its visibility as a program goal. This lack of visibility appears inconsistent with the explicit acknowledgment of economic activity in other PPP documentation and the many activities undertaken by the program to influence the health of the Canadian economy directly.

5.2.1 Priorities at the government, Agency, and PPA levels

The table below summarizes the Government of Canada's and the CFIA's priorities and/or strategic outcomes. The CFIA, in support of its strategic outcome, which is a safe and accessible food supply and plant and animal resource base, delivers three core programs, including Food Safety, Animal Health and Zoonotics, and Plant Resources. The PPP forms part of the PRP.

Table 6: Government priorities and strategic outcomes
Government of Canada outcome areas supported by the PPP CFIA strategic outcome CFIA raison d'être PPP strategic objective PPA purpose
  • Healthy Canadians
  • Strong Economic Growth
  • A Clean and Healthy Environment

A safe and accessible food supply and plant and animal resource base

Safeguarding the food, animal, and plant health, which enhances the health and well-being of Canada's people, environment, and economy

To prevent the introduction of plant pests within Canada, control or eradicate designated plant pests, and support export of commodities from Canada to meet trading partners' phytosanitary requirements

To protect plant life and the agricultural and forestry sectors of the Canadian economy by preventing the importation, exportation, and spread of pests and by controlling or eradicating pests in Canada

Source: (CFIA, 2012a, 2012m, 2013c; GoC, 2005)

5.2.2 Addressing Government of Canada priorities

As noted in the Agency's 2012 Corporate Risk Profile, the PRP, of which the PPP is a sub-program, contributes to three Government of Canada outcome areas (CFIA, 2012a):

  • Healthy Canadians. As noted above, invasive plants and plant pests may have immediate impacts on human health, including the introduction of diseaseFootnote 37 and the increased need for the use of pesticides.Footnote 38 As described in Section 5.1.1, biosecurity risks associated with plant pests may lead to a reduction in the food supply, which, in turn, increases food prices and reduces availability. Increased food prices may encourage individuals to shift to lower cost, less healthy food items. The PPP attempts to reduce the introduction and spread of invasive species and plant pests, and therefore addresses this outcome area.
  • Strong Economic Growth. The PPP explicitly plays a role in achieving the economic and trade-related goals of the federal government. For example, the technical expertise housed within the Agency and associated with the PPP supports the operation of organizations such as the MAS. In addition, forestalling the entry of invasive plants and plant pests into Canada reduces the financial losses borne by, as well as the direct management costs incurred by, producers. It also helps ensure access to foreign export markets, thereby supporting Canadian trade.
  • A Clean and Healthy Environment. As discussed in Section 5.1.1., plant pests have the potential to impact biodiversity negatively in the environments in which they are introduced (Secretariat of the Convention on Biological Diversity, 2014c). This reduction in biodiversity has a clear negative impact on the plant base. By addressing the risk of plant pest introduction, the PPP is contributing to a clean and healthy environment.

5.2.3 CFIA and program priorities

According to the CFIA's Report on Plans and Priorities (RPP), the Agency's raison d'être is: "The CFIA is dedicated to safeguarding food, animal, and plant health, which enhances the health and well-being of Canada's people, environment, and economy. The CFIA develops and delivers inspection and other services in order to:

  • prevent and manage food safety risks;
  • protect plant resources from pests, diseases and invasive species;
  • prevent and manage animal and zoonotic diseases;
  • contribute to consumer protection; and
  • contribute to market access for Canada's food, plants, and animals" (CFIA, 2013c).

In addition, the purpose of the PPA, which is the foundation of the PPP, is"to protect plant life and the agricultural and forestry sectors of the Canadian economy by preventing the importation, exportation, and spread of pests and by controlling or eradicating pests in Canada" (GoC, 2005).

While both the Agency's raison d'être and the PPA explicitly acknowledge the importance of supporting economic activity in Canada, the CFIA's strategic outcome, which is included in the PPP's logic model, does not explicitly reference this economic goal. Some key informants noted that there is an implied connection between a safe and accessible plant base and economic goals, such as supporting domestic activity and international trade. However, many argued that presenting the connection to the economy within the strategic outcome would more explicitly highlight the importance of this aspect of the PPP's work and potentially improve its profile and prominence.

The issue of highlighting the connection between the PPP and Canadian economic activity is closely connected to comments about the prominence of the program both inside and outside of the Agency. On a number of occasions, program staff suggested that the PPP, and the PRP more generally, have less prominence within the CFIA than its food and animal programs. While not supported by all key informants, in particular more senior managers, the contention among these individuals is that this limited visibility affects resourcing and the ability of the program to undertake its work. In addition, among those arguing for greater PPP visibility were individuals who suggested that a variety of stakeholders do not necessarily understand the program's work to support economic activity and thus its full importance to Canadians.

Key informants suggested the lack of importance placed on plant health by the general public in Canada is highlighted by its more prominent profile in other jurisdictions. In the case of the United Kingdom, for example, one key informant pointed to the fact that plant pest infestations have galvanized the government and the public and helped both push for more robust plant health programming. In addition, other key informants suggested that in Australia and New Zealand, there is much greater appreciation of the effects of invasive pests.

5.3 Federal roles and responsibilities

Summary of findings: Setting aside the domestic importance of addressing biosecurity risks, Canada has an obligation to undertake risk reduction activities in the area of plant biosecurity as a signatory to a number of international agreements and conventions on plant health. The CFIA has been designated as the NPPO for Canada and, as a result, also has the responsibility for undertaking such activities to meet these commitments.

The PRP, of which the PPP is a part, includes four distinct areas of activity, which are meant to contribute to the CFIA's goals. These include regulating agricultural and forestry products; mitigating risks to the plant resource base (including crops and forests) from regulated pests and diseases; regulating the safety and integrity of seeds, fertilizers, and plant products; and managing plant health emergencies and incidents. The PPP's four program delivery groupings, as outlined in the program's logic model, all fall within one or more of these areas.

5.3.1 Roles and responsibilities derived from international and domestic commitments

As noted in the sections above, Canada has a responsibility for plant protection under a number of international agreements and conventions. As the NPPO for Canada, the CFIA is principally responsible at the federal level for undertaking activities to limit the introduction of new IAS into, and mitigate the risks associated with those that are established in, Canada. The CFIA primarily obtains its authorities for the PPP through the federal PPA, which is meant "to protect plant life and the agricultural and forestry sectors of the Canadian economy by preventing the importation, exportation, and spread of pests and by controlling or eradicating pests in Canada" (GoC, 2005, p. 1).

Aligning with the above considerations, the strategic objective of the PPP is "to prevent the introduction of plant pests within Canada, control or eradicate designated plant pests, and support export of commodities from Canada to meet trading partners' phytosanitary requirements" (CFIA, 2012m). Broadly speaking, the work that the CFIA undertakes as part of the PPP involves "pest risk analysis and policy-setting, surveillance and monitoring, pest detection and pest eradication or management" (CFIA, 2006).

5.3.2 Activities meant to meet roles and responsibilities

The following table shows how the PPP's program delivery activities support the PRP, which is meant to support achievement of the Agency's objectives.

Table 7: Activity alignment within the PPP
PRP activities meant to achieve Agency objectives PPP program delivery activities - Communication and stakeholder engagement PPP program delivery activities - Inspection, surveillance, and certification PPP program delivery activities - Contingency, preparedness, and mitigation PPP program delivery activities - International engagement and standard setting
Regulating agricultural and forestry products X X
Mitigating risks to the plant resource base (including crops and forests) from regulated pests and diseases X X X X
Regulating the safety and integrity of seeds, fertilizers, and plant products X X
Managing plant health emergencies and incidents X X

* (CFIA, 2013c, 2013u)

Additionally, the case studies conducted as part of the evaluation demonstrate how the PPP activities support the Agency's responsibilities for protecting the plant resource base. For example, federal responsibility for responding to PPV lies with the CFIA and AAFC (AAFC & CFIA, 2011). As PPV is a quarantineable pest under the PPA, the CFIA is responsible for implementing measures to prevent its spread and/or to eradicate it.Footnote 39

Since PPV was first discovered in Canada in 2000, the federal government has provided the CFIA and AAFC with $209.7 million in funding, which was used to implement a series of three programs to assess the feasibility of eradicating PPV; eradicate the virus where possible; and, eventually, to contain PPV in the Niagara quarantine area and implement measures to manage and mitigate the spread of the virus.

5.4 Achievement of expected outcomes

Summary of findings: Establishing the PPP's performance was one of the most problematic elements of the evaluation. There was no evidence to suggest that the PPP is failing to undertake its activities and produce the related outputs as generally planned and outlined in the available program documentation. PPP performance indicators included in the Agency's Departmental Performance Report also suggest that certain performance targets for the program are being met on an annual basis. In addition, anecdotal evidence can provide a number of examples of achievement resulting from program activity. Further, the plant health literature provides additional support for the notion that the PPP is likely contributing to its expected outcomes.

However, a number of factors complicate a clear understanding of activity, output, and outcome achievement, as well as the PPP's contribution to these.

  • First, contextual information necessary for the interpretation of existing data is limited. In the case of outputs for example, there is little readily available information to understand why budgeted and actual outputs differ either within or across regions.
  • Second, PPP outcomes are often causally linked to each other. This means that the achievement of one outcome can heavily influence the achievement of another. This makes it difficult to draw a direct link between marginal changes in an outcome and specific program activities.
  • Third, conceptual difficulties with the measurement of prevention and risk reduction mean that not all of the program's established indicators effectively speak to PPP outcome achievement. For example, a target of no new regulated foreign pest entering into Canada through regulated pathways and establishing themselves, may be met by either increasing preventative activities or reducing pest surveillance – only one of these activities legitimately reduces risk.

Although the second complication above is an inherent part of the PPP's work, the CFIA or the program may address both the first and the third.

Assessing the PPP's contribution is further complicated by the role that other stakeholders – federal, international, provincial, and others – play in the achievement of the program's expected outcomes. In some instances, the PPP or the CFIA have an opportunity to influence stakeholders, to better achieve program goals. For example, key informants have pointed to opportunities for greater international collaboration, more certainty around compensation payments, and better communication with stakeholders. However, in other instances, formally established working relationships between the program and other stakeholders can affect program achievement and limit program delivery options. One key example cited on a number of occasions involves the working relationship between the CFIA and CBSA, as it relates to plant health border security. To some extent or another, nearly all suggested improvements to the PPP identified as part of the evaluation, including the expanded use of the program's current alternative service delivery approaches, rely on these formal and informal stakeholder relationships for their success.

5.4.1 Measuring the effectiveness of the PPP

One of the most difficult activities for the PPP evaluation involved establishing the program's performance. A variety of anecdotal evidence was available to demonstrate that the program was achieving a number of its outcomes. Examples of the program's contribution to outcomes cited by key informants included the following:

  • the program's ability to help maintain market access to China despite the existence of Blackleg of Canola in Canada
  • the program's success at opening new markets, including recent access to China for cherry exporters in Canada
  • the program's contribution to establishing ISPM 15
  • instances of improved collaboration, such as the development of the working groups discussed in the description of the PPP

The Agency's 2012–2013 Departmental Performance Report provides some information to support these assertions of program effectiveness. The following table presents performance reporting related to the PRP. Many of the indicators included relate directly to the activities of the PPP.

Table 8: PRP performance indicators
Expected result Performance indicators Targets Performance
Risks to the Canadian plant resource base from imported plants and plant products are mitigated Number of regulated foreign plant pests that enter into Canada through regulated pathways and establish themselves 0 0
Domestic plants and plant products are compliant with Canadian regulations and international agreements Percentage of domestic seed, crop inputs, and plants with novel traits in compliance with Canadian regulations and international agreements 90% 93%
Confirmed introductions of quarantine pests in Canada are contained and risk-mitigated (e.g., through the issuance of Notices of Prohibition of Movement, Quarantine, up to and including the issuance of Ministerial Orders) Percentage of confirmed introductions of quarantine pests for which notices are issued 100% 100%
Percentage of notices issued in a timely manner 90% 100%
Canadian exports of plants and plant products meet the country of destination regulatory requirements and Canada's reputation is maintained Percentage of certified plants and plant products shipments (lots) that meet the country of destination phytosanitary import requirements 99% 99%

Source: (CFIA, 2013w)

The following table presents performance indicators from the Agency's 2012–2013 Departmental Performance Report related specifically to the PPP.

Table 9: PPP performance indicators
Expected result Performance indicators Targets Performance
Pre-Border plant pest risks are mitigated Percentage of inspected shipments from offshore system approaches or preclearance programs in compliance with federal regulations 85% 99%
At-border plant pest risks are mitigated Percentage of pre-arrival documentation in compliance with Canadian import requirements 90% 99%
Post-border plant pest risks are mitigated Percentage of new pest detections that have a science-based management plan initiated within one year 90% No new pests were detected

Source: (CFIA, 2013w)

The performance indicators included in the departmental performance reportFootnote 40 suggest that the PPP has been successful in achieving, or exceeding, its targets in a number of areas. However, a number of caveats condition the results included in the tables. For example, the target for the number of regulated foreign plant pests that enter into Canada through regulated pathways and establish themselves is based on historical trends. The target for percentage of domestic seed, crop inputs, and plants with novel traits in compliance with Canadian regulations and international agreements has been established as an interim target and is currently under review. The percentage of inspected shipments from offshore system approaches or preclearance programs in compliance with federal regulations has not been collected historically and, as such, there is no historical trend on which to base the target.

One important question regarding these targets is whether they are generally established based on historical trends or reflect a known relationship between risk mitigation and their achievement. Put more simply, does achievement of the targets mean that risk will be maintained at a known and acceptable level or simply that past performance will be met?

Further, it appears that the CFIA has achieved mixed results in the area of pest eradication and management. The following summarizes the current status of a number of regulated or previously-regulated plant pests in Canada, based on details of their eradication and management activities included in the document and literature review.

  • Emerald Ash Borer (EAB) attacks and kills all species of ash. It is believed to have been introduced to North America in the early 1990s on wood packaging material, but it was not detected in Canada until 2002, when it was found in Windsor, Ontario.

    The CFIA's initial response to the EAB was an aggressive containment and eradication effort, involving the removal of approximately 150,000 ash trees in an effort to create an ash-free zone in southwestern Ontario. However, EAB was detected beyond this zone in January 2005.

    As the pest has now established itself to the point where the costs of eradication outweigh the benefits, eradication is not possible. Nonetheless, a PRA determined that, due to the high value of potentially-affected stock and the large amount of stock that can be affected, the potential environmental and economic impacts associated with the pest can be severe. Based on the PRA findings, the CFIA recommended using phytosanitary measures to slow or restrict the spread of the pest.

    Following stakeholder consultations, the CFIA decided to focus its efforts to control EAB on slowing the artificial (i.e., human-assisted) spread of the pest using surveillance, regulation, and enforcement; investment in research; and communications and outreach activities. This involves expanding regulated areas to include high-risk corridors and represent more accurately the expected distribution of the pest.

  • Potato Cyst Nematode (PCN) is a type of parasite affecting the health of potato plants and reducing yields by up to 80%. Infestations are considered by the CFIA to be very difficult to eradicate because the pests can remain dormant in the soil for several decades. Two types of PCN have been detected in Canada: golden nematode, found in Newfoundland and Vancouver Island in the 1960s, and more recently in Quebec (2006) and Alberta (2007), and the pale cyst nematode found in Newfoundland.
    • In October 2006, in an effort to contain the golden nematode parasite following its detection in Quebec, the CFIA issued the Golden Nematode Infested Places Order. The Order required that a Movement Certificate be issued by the CFIA prior to moving regulated materials (including potatoes, tomatoes, farm implements, soil, and manure) out of and within the designated infested areas.
    • In November 2006, the Golden Nematode Compensation Regulations were implemented to provide compensation to persons affected by the disposal of potatoes in the affected areas described in the Golden Nematode Infested Places Order.
    • A total of approximately $13 million was identified by federal and provincial governmentsFootnote 41 to cover short-term and long-term transitional issues faced by potato farmers in Quebec. Further, a federal/provincial response was developed under the AgriRecovery Framework, making up to approximately $300,000 availableFootnote 42 to assist nurseries affected by restrictions imposed by CFIA on the movement of soil.
    • The detection of golden nematode in two separate seed potato fields in Alberta resulted in the US border being closed to Alberta seed potatoes in November of 2007 and the Mexican border following suit shortly after; it is estimated that the border closing cost Alberta seed potato growers $35 million in lost sales. The CFIA responded to this detection by implementing strict regulatory measures on the affected farms, placing the affected lands under a Notice of Prohibition or Restriction of Activity,Footnote 43 and restricted movement on regulated land. Key informants indicated that soil sampling and testing as per the Canada/US Guidelines has resulted in the removal of all PCN-related restrictions on all but one field.
    • The joint federal and provincial response under the AgriRecovery Framework provided approximately $16 million to seed potato growers in Alberta affected by the discovery of PCN and subsequent border closures. An additional $2.2 million via AgriInsurance was also provided to the producers that had their seed potatoes decertified by CFIA.

    The detection of PCN also resulted in the following two substantial national and international activities:

    • In October 2006, Canada and the US signed the PCN guidelines, which aimed to harmonize PCN survey strategies and ensure appropriate phytosanitary measures were implemented upon the detection of PCN in either country. The guidelines were later revised in 2008 and 2009 so as to support the continuation of seed potato trade between the two countries in the event of future positive tests for PCN in either jurisdiction.
    • In 2008, the CFIA, along with AAFC and Canadian Horticultural Council established the PCN Task Force to coordinate government and industry efforts to: 1) reopen the US border to Alberta seed potatoes; 2) establish sustainable market access; 3) develop PCN surveillance guidelines; 4) maintain transparent communication with producers; 5) identify research needs; and 6) develop an approach to transition/re-establishment.

    A 2011 Industry Bulletin indicates the Agency tested approximately 78% of the 2011 Canadian seed potato production area and found no cases of PCN.

  • Soybean Cyst Nematode (SCN), considered one of the most destructive pests of soybean, was first detected in Canada in Kent County, Ontario in 1987 and has since spread to infest numerous counties, accounting for approximately 75% of the soybeans produced in Ontario and half the total Canadian soybean production. Detection of SCN in Manitoba in 2009 illustrated the challenges of enforcing domestic movement regulations over a wider area.

    SCN was deregulated in the US in 1972 because domestic regulation and movement restrictions were deemed unsuccessful in preventing the spread of SCN. Following a pest risk assessment conducted in 2010, the CFIA announced its decision to deregulate SCN, which took effect in November of 2013.

    • The CFIA's argument for deregulation was based on the following considerations: 1) numerous states in the US could be infested and remain unregulated; 2) soil surveys had proven to be of limited sensitivity; 3) some SCN-positive counties in Ontario remain unregulated; and 4) Canadian regulations and practices had proven unable to prevent further spread of SCN and therefore Canada was unable to comply with its obligations under the World Trade Organization (WTO) and IPPC guidelines.
    • As part of the risk management process, the CFIA conducted stakeholder consultations on its plans to deregulate SCN. Although RMDs indicate that stakeholders from Manitoba and Quebec objected to deregulation, the CFIA argued that SCN's widespread presence in Ontario made it necessary to move SCN from a quarantine pest to a management pest. The CFIA also argued that Canada's strict import requirements and inconsistent enforcement of domestic regulations could be perceived as unjustified discrimination and restriction on international trade, which would make Canada vulnerable to WTO challenges.
  • Plum Pox Virus (PPV) is a viral disease that infects stone fruit trees, including peach, plum, apricot, and nectarine trees. First found in Canada in 2000, PPV threatened the country's $200 million tender fruit, fruit processing, and nursery industries.

    Since 2001, the federal government provided the CFIA and AAFC with $209.7 million in funding to implement a series of three programs to respond to the virus, including the PPV Suppression and Eradication Feasibility Program, the PPEP, and the PPMMP. Eradication of PPV was expected to return tender fruit production and exports to pre-PPV levels.

    Despite implementing 10 years of programming, including a myriad of activities such as conducting pest surveys, ordering the removal of trees, establishing quarantine areas, restricting movement of regulated materials, banning propagation, and establishing a clean stock program, Canada has not been able to eradicate PPV. In 2011, given the high cost and uncertainty associated with continuing to try to eradicate PPV, Canada shifted its efforts to a monitoring and management strategy. Even though eradication is no longer being pursued, there is "continued need for Canada to address PPV due to international responsibilities, potential trade implications, and the ongoing threat to […] tender fruit in Canada" (AAFC & CFIA, 2011, p. 16).

    Although full eradication was not achieved, Canada's response to the virus had some successes. For example, the three-year Suppression and Eradication Feasibility Program supressed PPV in the Niagara peninsula and made progress towards eradication in other areas of Ontario and Nova Scotia. During the seven-year PPEP, eradication of PPV from six of seven quarantine areas was achieved (AAFC & CFIA, 2011). Additionally, by 2010, the overall infection rate in Niagara had been reduced from 1.9% of trees sampled to less than 0.02%.

    Pestsurveys conducted in 2011 and 2012 did not result in any detections of PPV (CFIA, 2013p). However, some key informants indicated that, in 2013, there was one detection of PPV inside of the existing quarantine area. This detection resulted in a slight expansion of the western edge of the quarantine area boundary (CFIA, 2014b).

    In 1999, prior to the detection of PPV, 26,450 tons of peaches were marketed and produced in Ontario. By 2006, production and marketing of peaches was at about 83% of 1999 levels.

    The PPEP aimed to increase tender fruit production to 2006 levels, or 22,000 tonnes. By 2009, this target was exceeded, with production of 23,428 tonnes (AAFC & CFIA, 2011).

Additional measures of quantifiable program achievement were far more difficult to collect and extremely difficult to interpret, where available. An example of these difficulties may be demonstrated using the output data available for the evaluation. The dataset indicated the budgeted and actual number of outputs and FTEs for a number of PPP outputs. The following table presents product inspection and establishment verification/audit data for the PPP during the 2012–2013 fiscal year.

Table 10: Product Inspection - PPP budgeted and actual domestic outputs per PPP activity (2012–2013)
Task Atlantic area - Budgeted Atlantic area - Actual Ontario area - Budgeted Ontario area - Actual Quebec area - Budgeted Quebec area - Actual Western area - Budgeted Western area - Actual Total - Budgeted Total - Actual Total - Variance
Potato 10,429 10,320 396 550 2,240 2,547 4,937 4,991 18,002 18,408 -2.26%
Forestry 58 39 53 17 97 277 29 77 237 410 -73.00%
Grains and field crops 11 1 17 15 28 16 42.86%
Other (e.g., vehicles, tires) 220 222 2 167 119 20 1,195 242 1,703 -603.72%
Horticulture 128 130 63 37 13 3 94 86 298 256 14.09%
NFLD DMC 2 5 5 2 2 7 9 -28.57%
Sampling for lab analysis 421 612 323 330 83 70 79 1,204 906 2,216 -144.59%
Certificates issued 3,931 3,657 632 10,351 1,875 1,775 1,373 1,659 7,811 17,442 -123.30%

Source: CFIA, Management, Resources, and Results Structure System

Table 11 - Establishment verification/audits - PPP budgeted and actual domestic outputs per PPP activity (2012–2013)
Task Atlantic area - Budgeted Atlantic area - Actual Ontario area - Budgeted Ontario area - Actual Quebec area - Budgeted Quebec area - Actual Western area - Budgeted Western area - Actual Total - Budgeted Total - Actual Total - Variance
Forestry 66 66 46 21 6 2 5 5 123 94 23.58%
Grains and field crops 2 16 7 34 23 50 32 36.00%
Horticulture 15 12 81 107 42 45 18 40 156 204 -30.77%
Potato 32 46 8 46 9 33 76 88 125 213 -70.40%
Other 4 1 12 11 2 11 18 23 -27.78%
Weed seeds in grains/invasive plants 4 2 36 23 50 34 119 145 209 204 2.39%
Establishment audit/invasive plants 4 2 2 63 31 69 33 52.17%
Site monitoring/invasive plants 1 20 6 538 213 8 2 567 221 61.02%
IAS – Section 43 permit verifications 27 21 82 44 22 1 77 71 208 137 34.13%

Source: CFIA, Management, Resources, and Results Structure System

The information presented in the table above provides a clear indication of the number of outputs both planned and undertaken, regionally and for Canada as a whole. It also provides an indication of the variance between these two numbers at the national level.

Even in situations where these output data are perfectly accurate reflections of the outputs generated by the PPP, they cannot be interpreted without a variety of contextual information. For example, the output data do not provide a sufficient amount of information to understand why the variances are taking place or even how they might occur. In addition, regional differences and changes over time cannot be placed in an appropriate context since there is little information about the demand for such inspection and verification services.

It is important to understand that in some instances, contextual information is available from within and outside the Agency. However, as some key informants suggest, the state of the Agency's IM/IT systems makes compiling and using these data for planning and program implementation infeasible. While there are examples of attempts to measure and contextualize the PPP's work, such as the Plant Health Strategic Plan, this does not appear to be an activity undertaken on an ongoing basis within the program.

5.4.2 Expected contribution of the PPP to outcomes

What was particularly clear from numerous key informants was that the activities undertaken as part of the PPP are closely related. While to some extent, each operates independently, success in one area can affect success in another. For example, import controls reduce the risk of introducing an IAS to Canada. This reduced risk, in turn, reduces the need for costly eradication and mitigation activities – both on the part of the CFIA and on the part of other stakeholders. By extension, a pest-free Canadian environment provides a systematic advantage to Canadian plant and plant product exporters who are interested in accessing international markets and must meet their phytosanitary requirements.

This interrelationship is the basis for arguments supporting greater preventative emphasis within the PPP. As many key informants stressed, and as noted by the Secretariat of the Convention on Biological Diversity, preventative activities are regularly less expensive than eradication and mitigation work once a plant pest has established itself. This supports emphasizing stronger preventative measures regarding the Canadian plant base before plant pests become established.

The plant protection literature and documentation, as well as substantiating comments from key informants, suggest that the program's work should contribute to its expected outcomes. For example, acknowledging that preventative activities are generally more cost effective than eradication or management of IAS, The Secretariat of the Convention on Biological Diversity highlights certain common preventative activities generally seen as effective in mitigating risks. It lists, for example, "inspections of international shipments, customs checks and proper quarantine regulations" (Secretariat of the Convention on Biological Diversity, 2014b). However, it also stresses that effectively addressing the issue of IAS requires not just national, but international efforts. These activities represent precisely those undertaken as part of the PPP – such as inspection, certification, and emergency response, as well as international engagement and collaboration.

The academic and professional literature also provides support for several aspects of PPP programming, suggesting that these may be reasonably expected to lead to the PPP's expected outcomes. For example, some studies support the logic that inspections, surveillance, and certification can result in increased stakeholder compliance with biosecurity measures. Although these studies analyze other jurisdictions and do not refer specifically to plant pests, they remain relevant to the PPP.

Ko, Mendeloff, and Gray (2010) examine the Occupational Safety and Health Administration's (OSHA) inspections in the US to determine the effect of inspections, finding that the number of total violations cited fell by 28%-48% from the first to second inspection. Examining a dataset of polluting Norwegian industrial plants, Telle (2009) similarly found that the threat of inspection significantly reduced the probability of serious violation of environmental requirements. These studies support the program logic that inspections improve compliance.

With respect to surveillance and monitoring, Gray and Shimshack (2011) examine the effects of environmental monitoring and enforcement in the US. The authors found that monitoring and enforcement significantly reduced future violations for both targeted and non-targeted institutions and also improved longer-term outcomes such as reduced emissions. This supports the claim that PPP surveillance activities can improve compliance levels of producers.

There is also some evidence that eradication measures can be successful in certain settings. For example, Pluess et al. (2012) noted that management campaigns aimed at eradicating invasive alien invertebrates, plants, and plant pathogens were previously thought to be seldom achievable. However, many eradication campaigns have been successful, particularly when initiated early. The authors examined 173 eradication campaigns, of which 88 (51%) were classified as "successful." This supports PPP program logic that eradication efforts can mitigate risks to the plant base by eliminating the plant pests and diseases causing these risks.

This evidence, however, needs to be balanced with insights from key informants regarding the effectiveness of plant-related eradication activities specifically. Some key informants noted that many plant pest eradications are not successful due to the ability of pests to infest a large geographic area and a broad range of plant species. This makes eradication activities that may be successful in the animal sphere – such as depopulation and then repopulation – infeasible for the PPP. Similarly, site-specific or product-specific mitigation activities, such as those common with the Food Program, also appear less viable for the PPP.

The PPP logic further assumes that if the plant resource base is protected from phytosanitary risks, it will be accessible, both domestically and internationally. Several studies support this, as they have indicated that plant protection activities can increase accessibility of international markets for plant and plant products. In particular, McKirdy, Rodoni, Moran, and Sharma (2012) note that Australia's relative freedom from many harmful plant pathogens has provided the country with a competitive advantage in terms of market access. Furthermore, Rodoni (2009) also states that quality assurance measures – such as certification – to mitigate the risk of pathogens are necessary to promote market access. These studies suggest that PPP activities to protect the plant resource base plausibly increase the accessibility of foreign markets by ensuring the plant resource base is protected from phytosanitary risks.

Nearly all key informants interviewed as part of the evaluation supported these justifications for PPP activities. In addition, some stressed that maintaining the country's natural environment also contributed to Canadians' quality of life. All the while, certain key informants highlighted the fact that increasing international trade will only accentuate the phytosanitary risks discussed above, providing further justification for PPP activities.

5.4.3 Conceptual difficulties of measuring PPP performance

A number of conceptual difficulties complicate performance measurement within the PPP. Part of this has to do with the nature of the work conducted by the PPP and what this work attempts to achieve. In nearly all key informant interviews, participants pointed out that it is very difficult to "prove a negative" in order to establish the effectiveness of the PPP. For example, it is difficult to demonstrate the success of the program when success will result in no new plant pest being introduced into Canada.

Some key informants also pointed to the fact that there are few studies that accurately measure the potential impacts of pest infestations in Canada. This makes it difficult to illustrate the importance of the programming, even if its impacts could be established. This is compounded by the fact that not all plant pests will have the same effect on the environment.

Building on these points, other key informants still argued that the program does not always have effective measures of compliance that accurately capture whether or not compliance is taking place as required. Measures, such as the number of interceptions collected by the CBSA for example, can be problematic, as a reduction may be the result of better compliance on the part of importers, as well as less vigilance on the part of the PPP.

5.4.4 The effect of federal partners on outcomes

The PPP and the CFIA's relationship with other federal organizations can heavily influence outcomes. Arguably, effective federal partnerships are necessary to fully achieve the goals of the PPP. Key informants offered several examples of how changes to interactions with other federal organizations could improve the PPP's ability to achieve its intended outcomes.

  • Market access and trade. Technical barriers to trade, such as biosecurity requirements, are one of the factors that need to be considered when attempting to secure access to new markets for Canadian agricultural producers. However, key informants indicated that, while there is demand for access to many markets, the PPP can represent a bottleneck in the process as they provide much of the technical expertise to meet these requirements for plants and plant products. For example, one key informant indicated, while other departments that are attempting to open new markets may have funding to support this trade-related work, the PPP may not have sufficient resources to provide the technical expertise required to address plant-related trade issues. When this is the case, the technical trade barriers for lower priority markets get addressed later (i.e., in future years when more funding is available) influencing the speed at which certain new markets become accessible. With that said it was not possible during the evaluation to make an objective assessment of the appropriateness of the current speed at which these barriers are being addressed. Although resource figures related to the financial and in-kind contributions of the PPP to these market access activities may be available, in the absence of some measure of demand, it is impossible to assess objectively whether the program's resourcing is appropriate.
  • Border security. In 2003, an Order-in-Council transferred the CFIA's responsibility for traveller and initial border inspections services for plant and plant products to the CBSA. This transfer means that the CFIA no longer has a presence at the Canadian border. Given the considerations below, there may be a need to review the appropriateness and effectiveness of the MOU between the CFIA and the CBSA.
    • Destination inspections for certain plant products are required because of the CFIA's lack of presence at the Canadian border. Key informants noted that destination inspection increases the risk associated with import activity by introducing the possibility of pest establishment while goods are in transit between the Canadian border and their final destination.
    • Many key informants within the CFIA believe that the CBSA prioritizes plant protection below the regulation of drugs, firearms, and other such commodities.
    • Key informants expressed concern that challenges associated with identifying plant pests at the border are exacerbated by the fact that border security officers do not focus exclusively on plant-related health risks and therefore do not have the same level of plant-related expertise as CFIA inspectors.
  • Intelligence. The CBSA uses the CFIA's AIRS to determine an appropriate course of action when assessing plant imports. Despite not providing specific details as to its problems, some key informants suggest that this system requires improvement to more effectively support the PPP and its goals.
  • Awareness of regulations. Key informants reported that small exporters in other countries may not be as well-informed about Canadian plant health regulations and, as such, not adhere to these requirements. They indicated that this, combined with the increasing use of e-commerce, especially for small shipments from international points of origin, presents an emerging risk to plant health. Key informants suggested that changes to the interaction between the CFIA and the CBSA may help address the emerging issue of e-commerce and ensure that these types of shipments are identified for inspection on a consistent basis.
  • IAS. Recent reductions in funding for IAS also point to another possible opportunity for improved federal interaction. As noted in the IAS case study, more than one key informant noted that the elimination of funding for the Invasive Alien Species Partnership ProgramFootnote 44 has the potential to undermine the program's interactions with various stakeholders. Not only has the discontinuation of this program reduced the amount of interaction between EC and the CFIA, it also has the potential to limit the coordination taking place with other stakeholders. This is despite the fact that certain stakeholders have called for more such coordination.

5.4.5 The effect of international partners on outcomes

As demonstrated by the following two examples, the success of the PPP appears to be closely linked to the activities of the CFIA's international partners:Footnote 45

  • Policy or political decisions among trading partners can have a considerable influence on PPP outcomes. Despite the fact that the PPP has operated the SPTQMP, without threats to the flow of trade for a number of years, recent decisions in the United States have brought the viability of this program into question. Specifically, if authorities in the United States no longer accept this audit-based approach to certification for all producers, then the program will no longer support continued access to this market.
  • International standards can be an effective and efficient means of addressing plant-related risks because they can influence the behaviour of many stakeholders at once. For example, key informants reported that international standards, such as ISPM 15, have helped establish safe behaviours on the part of producers importing goods to Canada, thereby reducing the risk associated with this importation. This has the potential to reduce the level of inspection effort required by the PPP without affecting risk.

Some key informants believe that, given the importance of international relationships and the influence of policy and political decisions as well as international standards on PPP success, the CFIA does not devote enough resources to this activity. As noted in the secondary data review and cost analysis, in the 2012–13 fiscal year, the PPP spent about $2.7 million (or 2.9% of its annual expenditures) to international engagement and standard-setting activities. However, due to the lack of information on the outcomes of international engagement activities and their associated costs, the evaluation could not fully assess the adequacy of the level of funding for the PPP's international activities.

5.4.6 The effects of other stakeholders on outcomes

The interaction between the PPP and a variety of stakeholders beyond the program's international and federal partners can influence outcomes. For example, according to the findings of the compensation payment case study, when requiring eradication or mitigation actions on the part of stakeholders, the Minister has the authority to authorize compensation payments for these actions. Once the decision to award compensation has been made, a compensation regulation must be drafted, go through the proper approval channels (published in the Canada Gazette Part I for comment), and be finalized (published in the Canada Gazette Part II). Upon publication of the regulation, there is a clear application and disbursement processFootnote 46 in place.

Compensation payments under the PPA help relieve some of the immediate operational and financial impacts to producers and other stakeholders from the outbreak of plant pests. Producers' operations and finances are impacted by the effect of the pest on their crops, and they incur additional costs when CFIA requires the implementation of control and eradication measures. While no documentation that clearly outlines the objectives of compensation payments under the PPA was available, some commonly-understood objectives include:

  • encouraging early reporting of suspected plant pests by producers; and
  • encouraging producers' cooperation during survey, control, eradication, and/or suppression efforts (CFIA, 2011a).

The PPA states that the Minister may order compensation to be paid given any of the following circumstances:

  • "any treatment of a place or any treatment, storage or disposition of a thingFootnote 47 required under [the PPA] or the regulations [made under the PPA]
  • any prohibition or restriction on the use of a place or on the movement of persons or things within, into or out of a place imposed under [the PPA] or the regulations [made under the PPA]
  • any prohibition or restriction on the use of a thing or on the sale or other disposition of a thing imposed under [the PPA] or the regulations [made under the PPA]" (GoC, 2005)

The Minister's decision regarding whether to provide compensation is subject to his discretion and the principles of administrative law. Given that this decision is appropriately made on a case by case basis, the unintended impact is to provide uncertainty to producers despite the requirement to report under the PPA.

The following table presents total expenditure figures for the 2009–2010 to 2012–2013 periods, for compensation payments, by the regions in which these were provided. As the table demonstrates, only a small subset of regulated pests were associated with compensation payments (CFIA, 2013u).

Table 12: Compensations payments under the PPA between 2009-10 and 2012-13 by province and plant disease/pest ($)
Plant Diseases/ Pests 2009-10 - PEI 2009-10 - ON 2009-10 - BC 2010-11 - ON 2010-11 - BC 2011-12 - PEI 2011-12 - ON 2011-12 - BC 2012-13 - ON 2012-13 - BC Total
Emerald Ash Borer 216,411 116,491 424 6,075 339,401
Plum Pox Virus 774,525 388,331 79,866 1,242,722
Asian Longhorn Beetle 89,252 2,996 4,377 96,625
Potato Wart 300 52,940 53,240
Sudden Oak Death – Non-Taxable 53,721 101,046 45,364 786,680 986,811
Sudden Oak Death – Taxable 1,833,967 118,309 1,952,276
Total 300 1,080,188 1,887,688 507,818 219,355 52,940 84,667 45,364 6,075 786,680 4,671,075

Source: CFIA, Corporate Management Branch

As some key informants pointed out, mitigation and eradication actions on the part of stakeholders often come at direct cost. For agricultural producers, these additional costs affect their profitability. As the table above suggests, compensation was provided for only six plant pests between the 2009­–2010 and 2012–2013 fiscal years. This means that not all pests are associated with compensation payments and producers cannot be certain about payments for mitigation activities related to all pests. If there is uncertainty regarding whether a recommendation to provide compensation payments will receive approval, producers may be reticent to report a plant pest for fear of incurring additional, uncompensated costs. This is despite the requirement to report under the PPA.

Influencing stakeholder behaviour requires not only establishing appropriate incentives but also ensuring that stakeholders are well-informed about safe practices. For example, key informants suggested that a lack of communication regarding enforcement activities related to the transport of firewood on the part of the PPP may be contributing to risky stakeholder behaviour. In this case, they suggested that more publicity around the fines incurred for transporting firewood would help motivate others to not do the same.

5.4.7 Possible improvements to the PPP

Currently, the PPP offers a number of variations of its programming. During the evaluation, key informants suggested that expanding on some of these variations has the potential to more effectively and efficiently meet the program's and the Agency's responsibilities.

  • Use of country-of-origin inspections. One key informant indicated that undertaking inspections in the country-of-origin ensures that the plant health risks remained in that country. This key informant also noted that once a pest is identified, there is no need to return the shipment to its point of origin since it is already there. With that said, the same key informant highlighted the fact that country-of-origin inspections are a high-cost option that may not be feasible for all commodities.
  • Increased use of audit-based approach to inspections. Key informants regularly stressed the advantages that an audit-based approach to inspection has over the traditional inspection activities undertaken by the program.Footnote 48 Typically, these approaches involve NPPOs and industry undertaking a series of surveillance and mitigation activities throughout the production process. Once the process is established, products that go through it require less stringent inspection by trading partners (assuming they accept the audit-based system that has been put in place).
  • Mandatory use of audit-based inspections. Some key informants noted that the audit-based programs currently offered under the PPP are often voluntary and operate in tandem with the CFIA's traditional inspection activity (e.g., the SPTQMP). They suggested that moving existing audit-based programs from being voluntary to mandatory has the potential to reduce CFIA inspection costs and eliminate the inefficiency of operating a dual inspection approach. This dual inspection approach requires that CFIA staff not only know traditional inspection protocols but also be trained in audit-based approaches.

    Despite the current use of, and increased interest in, audit-based inspection approaches, it is important to note the need for caution around such approaches. In particular, the example of the SPTQMP points to the critical role that trading partners play in the viability of such systems. Although this program has operated successfully as a voluntary program for a number of years, some key informants noted that a plan to move to mandatory use of the program resulted in resistance from the US – Canada's main trading partner. In fact, correspondence from the CFIA suggests that, at the time that this change to the program was proposed, it was identified as carrying a significant risk because US officials had not yet provided a firm commitment to accept the change.

    Some key informants also cautioned that the profit incentive for producers made the use of industry in surveillance and risk-mitigation activities problematic, since these activities often come at a cost. In addition, developing audit-based approaches requires significant investments in training on the part of the CFIA so that stakeholders involved in the process understand their role and can undertake it effectively.

  • International standard setting appeared to be another area in which increased activity could benefit PPP operations. Key informants within the Agency who were actively involved in the program's international work argued that international standard setting provides an opportunity to improve risk mitigation, particularly in the face of budgetary constraints. They argue that commodity-specific standards, such as the ISPM 15 standard for wood packaging, help establish safe practices among a number of trading partners, thereby reducing the risk of the transfer of pests within all participating nations.

    One key informant also noted that international standards establish an even playing field for all trading partners by establishing benchmarks that partners must meet, regardless of their political or economic importance.

  • Bilateral and multilateral partnership agreements. Some key informants asserted that participation in these international collaborative efforts can improve the efficiency of program delivery by coordinating efforts with partners. Some key informants point to the North American Perimeter ApproachFootnote 49 undertaken by Canada and the United States as an example of such coordination. This approach involves, to the extent possible, aligning regulatory activity in Canada and the United States and streamlining plant and plant product movements between the two countries.

In addition to suggesting the expansion or more regular use of current programming approaches, the key informants identified a number of areas in which entirely new approaches to the PPP's work could help better meet the program's goals related to plant protection. These alternatives continue to fall within the program delivery areas identified in the PPP's logic model and, by extension, continue to align with the responsibilities of the Agency.

Many of the examples provided by key informants included approaches undertaken in other jurisdictions. One example, cited on a number of occasions, involved the use of inspection stations. In this case, specific points of entry to Canada would have facilities where shipments would be inspected and risk-mitigation activities could be undertaken at the first point of entry, prior to the products being released into the country.

Key informants argued that inspection stations would reduce the risks associated with the PPP's import activity since it would eliminate the need for destination inspection, which enables plant pests to establish themselves in Canada at any point between their point of entry and final destination within the country. Moreover, in instances where destination inspection takes place in areas of significant agricultural production, where large numbers of plants/operations are susceptible to infestation, the approach becomes particularly problematic.

The development and use of inspection stations, however, is not without challenges. One key informant stressed that the establishment of inspection stations would require a significant upfront capital investment on the part of the PPP, the CFIA, or the Government of Canada, which may not be feasible in the current fiscal environment. Additionally, other key informants cautioned that some pests and/or symptoms of diseases may not immediately express themselves and therefore may not be detected upon arrival in the country.

One key informant further suggested that industry could be required to help fund the development of these inspection stations. This key informant argued that this type of cost-sharing makes sense, given that importers and exporters bear significant responsibility for the risk of introducing plant pests to Canada and its trading partners. A similar argument was made with regards to the costs incurred by industry as a result of implementing systems approaches to inspection and the use of user fees. These approaches imply a trade-off between having risk-makers bear the costs and ensuring that Canadian producers remain competitive.

More importantly perhaps, the successful use of inspection stations in Canada could require significant changes to the previously discussed arrangement between the CFIA and the CBSA. In the United States, where inspection stations are used to regulate the import of propagative plant materials, United States Department of Agriculture maintains a border presence.

Certain informants suggested that third-party inspection, such as that undertaken in jurisdictions like New Zealand, could provide efficiencies for the program. In these cases, third-party organizations would be designated to undertake inspection activities currently conducted by the PPP. This, however, may require a regulatory amendment.

Key informants offered further suggestions for changes to the PPP to improve its effectiveness. One such suggestion involved the need for improved communication regarding the PPP. This related to not only better communication within the CFIA regarding the PPP but also outside of the CFIA. Inside of the CFIA, improved communication was seen by some as a way of addressing the perceived low profile of the PPP within the Agency. Outside of the Agency, key informants argued for greater communication with producers and, in particular, the public. They argued that both the Animal Health and Food Safety Programs are more publicly visible since they more easily get media attention, and thus the need for improved external communication on the part of the PPP was even more pressing.

More effective use of IM/IT, as well as improvements to the CFIA's IM/IT systems, were seen as ways to improve PPP program delivery for the better. One key informant suggested that remote data entry for field staff would considerably improve the efficiency of some aspects of program delivery. Some key informants also stated that an electronic phytosanitary certification process would not only meet the demands of stakeholders outside of the Agency, it would also improve the efficiency of the program's operations. In addition, and as discussed above, under the measurement of program outcomes, improved IM/IT systems could support better contextualization of PPP data.

Finally, one key informant from outside of the CFIA suggested that it would be helpful if the PPP worked with stakeholders to better manage native species in one region of the country that could potentially move to other non-native regions. This individual argued that without federal participation, provinces will not take management actions in order to safeguard the environment in another province.

5.5 Efficiency and economy

Summary of findings: Prior to discussing program efficiency and economy, it is important to understand that the CFIA organizes Agency funding, accountability, and decision making along its branch structure. This branch structure cuts across all Agency business lines. Branches draw on A-base funding for the Agency as a whole, and budgetary reallocations within a branch can redirect resources to, or away from, business lines. This can pose challenges for both planning and program implementation within a business line, or within a program like the PPP.

Over the past four fiscal years, annual expenditures for the PPP have been within 2% to 5% of planned spending. However, expenditures for IAS and PPEP/PPMMP have been well below planned amounts. The variance in spending for these two areas is most likely due to both the practice of pooling funds prior to distribution and challenges associated with activity coding in the Agency's financial tracking system. The latter being a concern for both the PPP and the Agency as a whole.

Despite many key informants suggesting that current PPP funding is insufficient and limits the program's ability to undertake proactive action, objectively assessing this claim was not possible during the evaluation. To do so would require the program to identify specific program goals on an annual basis, and then identify an anticipated level of funding required to achieve these goals. This anticipated level of required funding could then be compared to actual resource allocations and to the level of eventual goal achievement in order to determine if the program is adequately resourced to achieve its goals. While it was not the intent of the evaluation to assess the sufficiency of program resources, nor suggest that additional funding is needed, the fact that there appears to be no way of objectively assessing this sufficiency is, in and of itself, an important evaluation finding.

In terms of overall efficiency and economy, the financial tracking and measurement difficulties associated with the PPP and the CFIA generally, make a rigorous and quantitative assessment of the PPP's efficiency and economy very difficult. This task becomes nearly impossible when the PPP's difficulties with contextualizing its program outputs, and assessing its marginal impact on its outcomes, are also considered. This made a quantitative assessment of program efficiency and economy infeasible during the evaluation and limited insights in this area to suggestions for program improvements from key informants.

5.5.1 PPP expenditures and funding allocation

The following table presents planned and actual expenditures for the PPP.

Table 13: PPP resources
PPP Overall 2009–10
($ million)
2010–11
($ million)
2011–12
($ million)
2012–13
($ million)
Planned $118.6 $95.0 $100.7 $99.9
Actual $114.3 $93.5 $95.8 $95.0
Variance ($) $4.3 $1.5 $4.9 $4.9
Variance (%) 4% 2% 5% 5%

Source: CFIA, Corporate Management Branch

The CFIA's business lines (i.e., Plant, Animal, and Food) are governance bodies and feed into senior levels of management. However, as key informants associated with the PPP suggest, funding, accountability, and decision making are often also organized according to the branch structure (Programs & Policy, Operations, Science, etc.). These branches cut across all business lines and draw on A-base fundingFootnote 50 for the Agency as a whole; therefore, budgetary reallocations within a branch can result in the reallocation of resources to, or away from, various PPP activities, specialized plant protection areas, and/or initiatives. For the PPP, and other programs, this can pose a challenge for both planning and program implementation. Nonetheless, as shown in Table 13, over the past four fiscal years, annual expenditures for the PPP have been within 2% to 5% of planned spending.

The evaluation examined the differences in availability of resources for IAS and PPEP/PPMMP. The CFIA's share of TB submission funding represents the funding allocation initially received for the respective initiatives; however, because they were not TB fenced funds, they were pooled with the Agency's A-base funding before final distribution to branches. As shown below in Table 14 and 5, spending associated with IAS and PPEP/PPMMP is well below planned amounts. Although there are reported underspending in IAS and PPEP/PPMMP, the overall PPP actual expenditure is very close to the planned spending, which includes IAS and PPEP/PPMMP funding.

The variance within each fiscal year of the period examined may be attributed to the fact that the CFIA has integrated many of its initiatives into its core program activities. The CFIA faces an inherent challenge to track and report specific initiative expenditures in their entirety, as they are often integrated with and overlap other program activities and tasks. For example, an IAS task can be a sub-task of a regular import control task.

Further, the financial system used by CFIA is not sufficiently sophisticated to track the expenditures based on tasks or a portion of tasks delivered for IAS and PPEP/PPMMP purposes. Therefore, it is possible that a portion of IAS and PPEP/PPMMP expenditures may be reported as overall PPP expenditures. Key informants reported that the Agency is in the process of developing an Electronic Service Delivery Platform (ESDP) that will improve the data tracking and reporting capacity at the service delivery level. They indicated that the improved operational data can be integrated with the financial information in order to provide more reliable reporting on expenditures for specific initiatives. However, the implementation of this platform is set to begin in mid-2016 with plant health inspection activities to be integrated during the subsequent year.

Table 14: Areas of interest resources – IAS
Expenditures 2009–10
($ million)
2010–11
($ million)
2011–12
($ million)
2012–13
($ million)
Total
($ million)
CFIA share of TB submission $11.5 $11.3 $11.3 $11.3 $45.4
RPP planned spending $11.5 $11.3 $11.3 $8.7 $42.8
Actual spending $8.8 $7.0 $6.1 $6.3 $28.2
Variance from planned spending ($) $2.7 $4.3 $5.2 $2.4 $14.6
Variance from planned spending (%) 23% 38% 46% 28% 34%
Variance from CFIA share of TB submission spending (%) 23% 38% 46% 21% 32%

Source: CFIA, Corporate Management Branch

Notes:

  • In the 2012–2013 RPP, Planned Spending for IAS was adjusted to account for funding reallocated to support Agency priorities.
  • Public Works and Government Services Canada costs are excluded from all figures.
  • Numbers may not add up due to rounding.
Table 15: Areas of interest resources – PPEP/PPMMP
Expenditures 2009–10
($ million)
2010–11
($ million)
2011–12
($ million)
2012–13
($ million)
Total
($ million)
CFIA Share of TB submission $5.8 $5.8 $2.2 $2.4 $16.2
RPP planned spending $5.8 $5.8 $2.2 $2.4 $16.2
Actual spending $3.5 $2.6 $1.7 $2.0 $9.8
Variance from planned spending ($) $2.3 $3.2 $0.5 $0.4 $6.4
Variance from planned spending (%) 40% 55% 23% 17% 40%
Variance from CFIA share of TB submission spending (%) 40% 55% 23% 17% 40%

Source: CFIA, Corporate Management Branch

Notes:

  • Public Works and Government Services Canada costs are excluded from all figures.
  • Numbers may not add up due to rounding.

The financial tracking and measurement difficulties noted above make a rigorous and quantitative assessment of the PPP's efficiency and economy very difficult. This task becomes nearly impossible when the difficulties with contextualizing the program outputs, and assessing the marginal impact of the program on its outcomes, are also considered.

5.5.2 Program funding approach

Many key informants suggested that current levels of funding are insufficient to offer the best possible programming under the PPP. Among these key informants, individuals also suggested that insufficient funding could only allow the PPP to offer programming reactively. As one key informant suggested, this limits the ability of the program to undertake proactive action to support the effective operation of the program.

Using a relatively simple example, the key informant pointed to the need to update work instructions in the plant biosafety office. They noted that the work instructions were out of date and no longer fit the needs of industry. As a result, the PPP has attempted to take appropriate action on a case-by-case basis rather than strictly adhere to the outdated work instructions. However, at times, this has led to complaints about consistency in the program's approach. This need for consistency was later reiterated during the interviews with external stakeholders. While a seemingly small issue, such an issue can have a significant impact on producer behaviour, and this behaviour directly influences the ability of the PPP to undertake its work successfully.

Despite these calls on the part of many key informants for additional program resources, and some suggestions that the work of the PPP is heavily demand-driven, the evaluation was unable to identify an overall budgetary planning process that explicitly aligned the anticipated demand for plant protection services with resources. Discussions with members of the working group established for the evaluation suggested that, currently, initial budgets are regularly based on the prior year's spending. Other selected key informants noted that the inspection work plan and laboratory sample plan are reviewed and approved annually by the PBL Committee to ensure that inspection and testing resources are allocated to key priorities. They also mentioned that any urgent initiatives without a funding source are referred to the Policy and Program Management Committee and SMC to be considered for funding.

While it was not the intent of the evaluation to assess the sufficiency of program resources, nor suggest that additional funding is needed, the fact that there appears to be no way of objectively assessing this sufficiency is, in and of itself, an important evaluation finding. The current allocation of available funding based on priorities within the Agency ensures that overall expenditures remain within budgeted levels. However, it makes planning for specific goals within the PPP difficult.

5.5.3 Governance and the impacts on efficiency

The PPP falls under the CFIA's PBL and, as such, fits within the broader Agency governance structure. Governance at the Agency is led by the SMC, which is supported by three senior executive committees, which in turn receive guidance and advice related to the PBL from a PBL committee. A subcommittee that includes membership from a variety of Agency branches supports the PBL committee.

While detailed examination of the Agency's governance structure was outside the scope of the evaluation, some key informants believe it is a potential area of inefficiency for the PPP. For example, a number of key informants, particularly at the program delivery level, expressed concern regarding the timeliness of decision making. They also suggested that many technical issues, which could be resolved at lower levels, require decisions by senior management.

Aside from noting the above-mentioned potential areas of concern, the evaluation is not in a position to make conclusions regarding the efficiency of the Agency's governance structure. To support a comprehensive assessment of the efficiency of the governance structure, three pieces of information are required. First, it requires a clear indication of the types of decisions that should be made at various levels within the Agency. Second, it requires an objective assessment of whether or not actual decisions are being made at the appropriate levels. Third, assuming that decisions are being made at the appropriate levels, it requires an assessment of the degree to which decision making delays action on the part of the program.

6.0 Conclusion

In terms of the rationale for the PPP, evidence from the evaluation suggests that there are clear biosecurity risks associated with plant pests, including IAS, in Canada. These plant pests have the potential to result in a loss of biodiversity, restrict market access for Canadian producers, impose direct financial impacts on Canadians, reduce food security, and affect human health. Nationally, all of these can affect Canadian trade and economic interests. By attempting to limit the introduction and spread of plant pests in Canada, the PPP directly addresses this demonstrated need. At the same time, the program helps meet Canada's obligations under a number of international agreements and conventions. Importantly, the program does so in an environment where it would be unlikely that private sector stakeholders would consistently, and voluntarily, address biosecurity risks on their own.

The evaluation noted that three Government of Canada outcome areas are supported by the PPP. These include healthy Canadians, strong economic growth, and a clean and healthy environment. These outcome areas aligned quite closely with the CFIA's raison d'être, the PPP strategic objective, and the purpose of the PPA. While distinct, these three areas are related insofar as a healthy environment – and specifically a healthy plant base – provides the foundation for a variety of economic activities in Canada and contributes to the health of Canadians.

With that said, the strategic outcome for the CFIA, which is the endpoint in the PPP's logic model, does not explicitly acknowledge the program's contribution to the Canadian economy. Although this does not undermine the implied connection between a safe and accessible plant resource base and economic activity, arguably, it decreases its visibility as a program goal. This lack of visibility appears inconsistent with the explicit acknowledgment of economic activity in other PPP documentation and the many activities undertaken by the program to influence the health of the Canadian economy directly.

Setting aside the domestic importance of addressing biosecurity risks, Canada has an obligation to undertake risk reduction activities in the plant area as a signatory to a number of international agreements and conventions on plant health. The CFIA has been designated as the NPPO for Canada, and as a result, also has the responsibility for undertakings such activities to meet these commitments.

The PRP, of which the PPP is a part, includes four distinct areas of activity, which are meant to contribute to the CFIA's goals. These include regulating agricultural and forestry products; mitigating risks to the plant resource base (including crops and forests) from regulated pests and diseases; regulating the safety and integrity of seeds, fertilizers, and plant products; and managing plant health emergencies and incidents. The PPP's four program delivery groupings, as outlined in the program's logic model, all fall within one or more of these areas.

Establishing the PPP's performance was one of the most problematic elements of the evaluation. There was no evidence to suggest that the PPP is failing to undertake activities and produce outputs as generally planned and outlined in the available program documentation. PPP performance indicators included in the Agency's DPR also suggest that certain performance targets for the program are being met on an annual basis. In addition, anecdotal evidence can provide a number of examples of achievement resulting from program activity. Further, the plant health literature provides additional support for the notion that the PPP is likely contributing to its expected outcomes.

However, a number of factors complicate a clear understanding of activity, output, and outcome achievement, as well as the PPP's contribution to these.

  • First, contextual information necessary for the interpretation of existing data is limited. In the case of outputs for example, there is little readily available information to understand why budgeted and actual outputs differ either within or across regions.
  • Second, PPP outcomes are often causally linked to each other. This means that the achievement of one outcome can heavily influence the achievement of another. This makes it difficult to draw a direct link between marginal changes in an outcome and specific program activities.
  • Third, conceptual difficulties with the measurement of prevention and risk reduction mean that not all of the program's established indicators effectively speak to PPP outcome achievement. For example, a target of no new regulated foreign pest entering into Canada through regulated pathways and establishing themselves, may be met by either increasing preventative activities or reducing pest surveillance – only one of these activities legitimately reduces risk.

Although the second complication above is an inherent part of the PPP's work, the CFIA or the program may address both the first and the third.

Assessing the PPP's contribution is further complicated by the role that other stakeholders – federal, international, provincial, and others – play in the achievement of the program's expected outcomes. In some instances, the PPP or the CFIA have an opportunity to influence stakeholders, to better achieve program goals. For example, key informants have pointed to opportunities for greater international collaboration, more certainty around compensation payments, and better communication with stakeholders. However, in other instances, formally established working relationships between the program and other stakeholders can affect program achievement and limit program delivery options. One key example cited on a number of occasions involves the working relationship between the CFIA and CBSA, as it relates to plant health border security. To some extent or another, nearly all suggested improvements to the PPP identified as part of the evaluation, including the expanded use of the program's current alternative service delivery approaches, rely on these formal and informal stakeholder relationships for their success.

Prior to discussing program efficiency and economy, it is important to understand that the CFIA organizes Agency funding, accountability, and decision making along its branch structure. This branch structure cuts across all Agency business lines. Branches draw on A-base funding for the Agency as a whole, and budgetary reallocations within a branch can redirect resources to, or away from, business lines. This can pose challenges for both planning and program implementation within a business line, or within a program like the PPP.

Over the past four fiscal years, annual expenditures for the PPP have been within 2% to 5% of planned spending. However, expenditures for IAS and PPEP/PPMMP have been well below planned amounts. The variance in spending for these two areas is most likely due to both the practice of pooling funds prior to distribution and challenges associated with activity coding in the Agency's financial tracking system. The latter being a concern for both the PPP and the Agency as a whole.

Despite many key informants suggesting that current PPP funding is insufficient and limits the program's ability to undertake proactive action, objectively assessing this claim was not possible during the evaluation. To do so would require the program to identify specific program goals on an annual basis, and then identify an anticipated level of funding required to achieve these goals. This anticipated level of required funding could then be compared to actual resource allocations and to the level of eventual goal achievement in order to determine if the program is adequately resourced to achieve its goals. While it was not the intent of the evaluation to assess the sufficiency of program resources, nor suggest that additional funding is needed, the fact that there appears to be no way of objectively assessing this sufficiency is, in and of itself, an important evaluation finding.

In terms of overall efficiency and economy, the financial tracking and measurement difficulties associated with the PPP and the CFIA generally, make a rigorous and quantitative assessment of the PPP's efficiency and economy very difficult. This task becomes nearly impossible when the PPP's difficulties with contextualizing its program outputs, and assessing its marginal impact on its outcomes, are also considered. This made a quantitative assessment of program efficiency and economy infeasible during the evaluation and limited insights in this area to suggestions for program improvements from key informants.

6.1 Recommendations

The recommendations listed below are based on the detailed findings discussed in Section 5.0, the summary of which is included immediately above.

Recommendation Supporting rationale Implications of findings Intended outcome of recommendation
Recommendation 1:
Review the CFIA's plant protection-related arrangement with the CBSA to facilitate optimal programming options.
The delegation of responsibility for certain plant protection responsibilities to the CBSA has had a number of implications for the way in which the PPP undertakes its work. Perhaps more importantly, the current arrangement has the potential to affect the PPP's ability to implement program improvement, including an increased emphasis on prevention. Reviewing the arrangement between CFIA and the CBSA as it relates to plant protection appears to have merit, so as to ensure that the PPP continues to have the ability to undertake program improvements. This is particularly true given the lack of a regular CFIA presence at the border and the competing border security demands placed on the CBSA.

An increased focus on preventative activities or other program improvements have the potential to improve program outcomes or efficiency.

The delegation of plant-related border security activity to the CBSA has had implications for PPP programming.

Optimize programming options.
Recommendation 2:
Review and improve performance measurement associated with the PPP.
Performance measurement is particularly difficult in the context of regulatory programming. However, there are opportunities for improvements in this area within the PPP. This includes not only the development of performance indicators that more accurately reflect outcome achievement, but also the development of a body of contextual data to support the interpretation of performance measures. A number of measures will be required to tell a clear story of the program's effects. Performance measurement within the program does not consistently provide the information necessary for PPP planning, to communicate the importance of the program, or to report on outcomes. Improved program planning, reporting, and ability to communicate the importance of the program within the CFIA and outside of the Agency.
Recommendation 3:
Assess the effectiveness of alternative service delivery options relative to traditional PPP delivery to better support the case for their continued or expanded use.
The evaluation found many examples of the PPP's use of alternative service delivery, and support for the continued or expanded use of these approaches appears to exist within the program and the CFIA. However, information on the relative efficiency and effectiveness of these alternatives is needed to support a strong case for their use. A number of factors identified in the evaluation bear particular consideration when undertaking this type of an assessment, examples of which can be found in Section 2 and 5. Examples of alternative service delivery within the PPP exist. However, information on the relative efficiency and effectiveness of these alternatives is needed to support their use. Improved case for the use of alternative service delivery models in specific circumstances in order to improve program efficiency and effectiveness – both for the Agency and Canadians.
Recommendation 4:
Based on prioritized goals, estimate PPP resource requirements during annual planning.
Having an objective means of assessing program resource sufficiency may benefit the PPP. One way to approach this assessment could involve the program identifying specific program goals on an annual basis and then identifying an anticipated level of funding required to achieve these goals. This anticipated level of required funding could then be compared to actual resource allocations and to the level of eventual goal achievement. The comparison would then help an assessment of both the adequacy of resources and whether estimated funding requirements by the program were accurate. Changes in goals based on Agency prioritization would need to be considered throughout this process. Under the current PPP budgeting approach, it is extremely difficult to assess resource sufficiency in an objective way. Doing so requires changing the way in which annual program goals are established and budgets are examined. Improved ability to understand necessary program resourcing for the achievements of planned program outcomes.

Evidence based decision making when assigning funding.

Appendix A – References

References

  • AAFC. (2013a). Market Access Secretariat. Retrieved February 19, 2014, from http://www.agr.gc.ca/eng/industry-markets-and-trade/market-access/market-access-secretariat/?id=1257798989228
  • AAFC. (2013b). Plant Issues June 14, 2013. RDIMS #4115623.
  • AAFC, & CFIA. (2011). Lessons Learned: The Plum Pox Eradication Program.
  • California Biodiversity Council. (2008). What is Biodiversity? Retrieved November 12, 2013, from http://biodiversity.ca.gov/Biodiversity/biodiv_definition.html
  • CBSA. (2009). ARCHIVED - Audit of the Memorandum of Understanding Between the Canada Border Services Agency and the Canadian Food Inspection Agency. Retrieved August 30, 2013, from http://www.cbsa-asfc.gc.ca/agency-agence/reports-rapports/ae-ve/2009/cfia-aicia-eng.html#a1
  • CBSA. (2011). Memorandum D19-1-1 - Food, Plants, Animals and Related Products. Retrieved August 30, 2013, from http://www.cbsa-asfc.gc.ca/publications/dm-md/d19/d19-1-1-eng.pdf
  • CFIA. (n.d). Senior Management Committee (SMC) Terms of Reference. RDIMS #3139538.
  • CFIA. (2006). Action Plan for invasive alien terrestrial plants and plant pests - Phase 1 – Key Initiatives. Retrieved September 17, 2013, from http://www.collectionscanada.gc.ca/webarchives/20071124173415/http://www.inspection.gc.ca/english/plaveg/invenv/action/phase1e.shtml
  • CFIA. (2007). Plant Pest Surveillance - Overview. Retrieved September 18, 2013, from http://www.collectionscanada.gc.ca/webarchives/20071120232331/http://www.inspection.gc.ca/english/sci/surv/obje.shtml#ann
  • CFIA. (2008a). Canadian Forest Phytosanitary Working Group (CFPWG) Terms of Reference November 2008 (Final Version). RDIMS #4114007.
  • CFIA. (2008b). Invasive Alien Plants in Canada - Technical Report. Retrieved September 17, 2013, from http://epe.lac-bac.gc.ca/100/206/301/cfia-acia/2011-09-21/www.inspection.gc.ca/english/plaveg/invenv/techrpt/techrese.shtml
  • CFIA. (2009a). D-07-05: Phytosanitary requirements to prevent the introduction and spread of the hemlock woolly adelgid (Adelges tsugae Annand) from the United States and within Canada. Retrieved October 9, 2013, from http://www.inspection.gc.ca/plants/plant-protection/directives/forestry/d-07-05/eng/1323754212918/1323754664992
  • CFIA. (2009b). D-99-06: Policy on the Issuance of Phytosanitary Certificates. Retrieved September 23, 2013, from http://www.inspection.gc.ca/plants/plant-protection/directives/exports/d-99-06/eng/1323852257037/1323852328308
  • CFIA. (2010a). Canadian Forest Phytosanitary Working Goup Draft Agenda - November 24, 2010. RDIMS #4109605.
  • CFIA. (2010b). D-97-04: Application, procedures, issuance and use of a Permit to Import under the Plant Protection Act. Retrieved September 23, 2013, from http://www.inspection.gc.ca/plants/plant-protection/directives/imports/d-97-04/eng/1323791055523/1323803716515
  • CFIA. (2011a). Analysis of Compensation Provided under the Authority of the Health of Animals Act and the Plant Protection Act.
  • CFIA. (2011b). Canadian Invasive Plant Framework: A Collaborative Approach to Addressing Invasive Plants in Canada. RDIMS #4109373.
  • CFIA. (2011c). D-11-01: Phytosanitary Requirements for Plants for Planting and Fresh Branches to Prevent the Entry and Spread of Anoplophora spp. Retrieved October 9, 2013, from http://www.inspection.gc.ca/plants/plant-protection/directives/forestry/d-11-01/eng/1322607792840/1322607917651
  • CFIA. (2011d). Invasive Plants In Canada. Retrieved June 21, 2013, from http://www.inspection.gc.ca/plants/plant-protection/invasive-plants/eng/1306601411551/1306601522570
  • CFIA. (2011e). Plant Program Framework (Baseline of Understanding).
  • CFIA. (2011f). Plant Protection Survey Priority Identification and Rationale. RDIMS #2864711.
  • CFIA. (2012a). 2012 Corporate Risk Profile. RDIMS #3606503.
  • CFIA. (2012b). CFIA-AAFC Portfolio _Market Access: Secretariat Update. RDIMS #3548468.
  • CFIA. (2012c). Consultation Policy and Framework. Retrieved November 19, 2013, from http://www.inspection.gc.ca/about-the-cfia/accountability/consultations/consultation-policy-and-framework/eng/1330978800191/1330978883936
  • CFIA. (2012d). D-03-08: Phytosanitary Requirements to Prevent the Introduction Into and Spread Within Canada of the Emerald Ash Borer, Agrilus planipennis (Fairmaire). Retrieved October 9, 2013, from http://www.inspection.gc.ca/plants/plant-protection/directives/forestry/d-03-08/eng/1323821135864/1323821347324
  • CFIA. (2012e). D-11-05: Phytosanitary Requirements for Non-Manufactured and Non-Propagative Wood Products to Prevent the Introduction from the Continental United States and Spread Within Canada of the Asian Long-horned Beetle, Anoplophora glabripennis (Motschulsky). Retrieved October 9, 2013, from http://www.inspection.gc.ca/plants/plant-protection/directives/forestry/d-11-05/eng/1326390329570/1326390421438
  • CFIA. (2012f). Expert Advisory Committee - Biographies - About the Canadian Food Inspection Agency. Retrieved October 29, 2013, from http://www.inspection.gc.ca/about-the-cfia/accountability/consultations/consultative-groups/expert-advisory-committee/biographies/eng/1331068204268/1331068538061#buntain
  • CFIA. (2012g). Plant Business Line Committee (PBLC) Terms of Reference. RDIMS #3485603.
  • CFIA. (2012h). Plant Health Glossary of Terms. Retrieved September 17, 2013, from http://www.inspection.gc.ca/plants/plant-protection/directives/glossary/eng/1304730588212/1304730789969
  • CFIA. (2012i). Plant Program Framework Final – May 2012. RDIMS #2955374.
  • CFIA. (2012j). Report of Meeting - Canadian Forest Phytosanitary Working Group (CFPWG) - September 6, 2012. RDIMS #4113996.
  • CFIA. (2012k). Terms Of Reference - Expert Advisory Committee. Retrieved October 29, 2013, from http://www.inspection.gc.ca/about-the-cfia/accountability/consultations/consultative-groups/expert-advisory-committee/terms/eng/1331069942826/1331070029555
  • CFIA. (2012l). The Canadian Food Inspection Agency's (CFIA) role in the overall management of Brown Spruce Longhorn Beetle (BSLB) (options discussion document).
  • CFIA. (2012m). The CFIA at a Glance.
  • CFIA. (2013a). 2013-14 CFIA Program Alignment Architecture.
  • CFIA. (2013b). BSLB Risk Mitigation Program. Retrieved November 26, 2013, from http://www.inspection.gc.ca/plants/plant-protection/insects/brown-spruce-longhorn-beetle/mitigation-program/eng/1332994769175/1332994867264
  • CFIA. (2013c). Canadian Food Inspection Agency 2013–14 Estimates Part III–Report on Plans and Priorities.
  • CFIA. (2013d). Canadian Food Inspection Agency 2013 Evaluation Plan.
  • CFIA. (2013e). Canadian Food Inspection Agency - Policy on Evaluation.
  • CFIA. (2013f). CFIA Emergency Management. RDIMS #4027552.
  • CFIA. (2013g). CFIA Engagement Plan in IPPC Activities.
  • CFIA. (2013h). CFIA Procedure for the Activation of Emergency Response Plans at the National Level (deck). RDIMS #3693938.
  • CFIA. (2013i). CFIA Procedure for the Activation of Emergency Response Plans at the National Level (report). RDIMS #3568497.
  • CFIA. (2013j). D -95-03: Plant protection policy for marine vessels arriving in Canada from areas regulated for Asian Gypsy Moth (Lymantria dispar, Lymantria albescens, Lymantria postalba, Lymantria umbrosa). Retrieved October 9, 2013, from http://www.inspection.gc.ca/plants/plant-protection/directives/forestry/d-95-03/eng/1321945111492/1321945247982
  • CFIA. (2013k). Draft - Framework for Canada-United States Perimeter Approach to Plant Protection. Retrieved August 30, 2013, from http://www.inspection.gc.ca/plants/plant-protection/imports/perimeter-approach/eng/1375978355009/1375978422848
  • CFIA. (2013l). Food Safety Emergency Response Functional Plan.
  • CFIA. (2013m). Guidance Document Repository. Retrieved November 19, 2013, from http://www.inspection.gc.ca/guidance-document-repository/eng/1374161650885/1374161737236
  • CFIA. (2013n). Plant Pest Surveillance. Retrieved September 20, 2013, from http://www.inspection.gc.ca/plants/plant-protection/surveillance/eng/1344466499681/1344466638872
  • CFIA. (2013o). Plant Protection Organizations. Retrieved September 4, 2013, from http://www.inspection.gc.ca/plants/plant-protection/plant-protection-organizations/eng/1312732291086/1312732417251
  • CFIA. (2013p). Plum Pox Monitoring and Management Program Update meetings. RDIMS #4135585.
  • CFIA. (2013q). Report of the Canadian Interdepartmental Working Group on the Implementation of ISPM 15 at the Canada-U.S. Border 4 September 2013. RDIMS #4115920.
  • CFIA. (2013r). Request for Comments: The Canada-United States Perimeter Approach to Plant Protection. Retrieved August 30, 2013, from http://www.inspection.gc.ca/plants/plant-protection/imports/request-for-comments/eng/1375970281790/1375970433984
  • CFIA. (2013s). RMD-11-02: Pest Risk Management Document for Deregulation of Heterodera glycines Ichinohe (Soybean Cyst Nematode) - Plants - Canadian Food Inspection Agency. Retrieved September 20, 2013, from http://www.inspection.gc.ca/plants/plant-protection/directives/risk-management/rmd-11-02/eng/1377523533087/1377523534384
  • CFIA. (2013t). RMD-13-01: Regulated Areas for Emerald Ash Borer (EAB) (Agrilus planipennis Fairmaire). Retrieved October 9, 2013, from http://www.inspection.gc.ca/plants/plant-protection/directives/risk-management/emerald-ash-borer/eng/1368741925939/1368741926892
  • CFIA. (2013u, March 14). Pests Regulated by Canada. Retrieved April 4, 2014, from http://www.inspection.gc.ca/plants/plant-protection/pests/regulated-pests/eng/1363317115207/1363317187811
  • CFIA. (2013v, August). Terms of Reference - Evaluation of the Plant Protection Program.
  • CFIA. (2013w, August 22). 2012-2013 Departmental Performance Report. Retrieved April 4, 2014, from http://www.inspection.gc.ca/about-the-cfia/accountability/reports-to-parliament/2012-2013-dpr/eng/1377176926809/1377177134114?chap=3#s9c3
  • CFIA. (2014a). Mandate, Roles, and Responsibilities. RDIMS #3331371.
  • CFIA. (2014b). Regulated area expanded for plum pox virus in the Niagara region. Retrieved May 9, 2014, from http://news.gc.ca/web/article-en.do?nid=836699
  • CFS. (2013a). Canadian Forest Service: Responding to invasive and alien forest pests. Retrieved October 25, 2013, from http://cfs.nrcan.gc.ca/pages/346
  • CFS. (2013b). Canadian Forest Service: Risk-based decision making. Retrieved October 25, 2013, from http://cfs.nrcan.gc.ca/pages/343
  • Chakraborty, S., & Newton, A. C. (2011). Climate change, plant diseases and food security: an overview: Climate change and food security. Plant Pathology, 60(1), 2–14. doi:10.1111/j.1365-3059.2010.02411.x
  • Environment Canada. (2011). An Invasive Alien Species Strategy for Canada. Retrieved February 9, 2014, from http://www.ec.gc.ca/eee-ias/default.asp?lang=En&n=1A81B051-1
  • Environment Canada. (2013). What Are Alien and Invasive Alien Species? Retrieved May 8, 2014, from http://www.ec.gc.ca/eee-ias/default.asp?lang=En&n=986EC44D-1
  • Environment Canada. (n.d.). Proposed Action Plan for Invasive Alien Terrestrial Plants and Plant Pests: Phase 1. Retrieved from http://ec.gc.ca/eee-ias/4C8E46C8-E648-4528-B12C-33F102C130B2/plants_e.pdf
  • FAO. (2006). The State of Food and Agriculture, 2006: Food Aid for Food Security?. Food & Agriculture Org. Retrieved from http://books.google.com/books?hl=en&lr=&id=76BLbxGS4k4C&oi=fnd&pg=PR6&
    dq=%22the+past+two+decades,+the+number+of+food+emergencies+has+risen+from+an+average+
    of+15+per+year+in+the+1980s+to+more+than+30+per+year%22+&ots=r2OAhO0Cm7&sig=E4wX6PoE2Hseb4RrOM8mDzQnLl4
  • Flood, J. (2010). The importance of plant health to food security. Food Security, 2(3), 215–231. doi:10.1007/s12571-010-0072-5
  • GoC. (2005). Plant Protection Act S.C.. 1990 c.22. Retrieved August 30, 2013, from http://laws-lois.justice.gc.ca/PDF/P-14.8.pdf
  • GoC. (2012a). Seed Regulations (C.R.C. c. 1400).
  • GoC. (2012b). Seeds Act R.S.C., 1985, c.S-8. Retrieved May 8, 2014, from http://laws-lois.justice.gc.ca/eng/acts/S-8/FullText.html
  • GoC. (2013). Plant Protection Regulations (SOR/95-212). Retrieved September 17, 2013, from http://laws-lois.justice.gc.ca/PDF/SOR-95-212.pdf
  • Government of Canada, S.C.. (2013, March 26). Farms, average operating revenues and expenses, by farm type (Crop production). Retrieved November 12, 2013, from http://www.statcan.gc.ca/tables-tableaux/sum-som/l01/cst01/agri113a-eng.htm
  • Gray, W. B., & Shimshack, J. P. (2011). The Effectiveness of Environmental Monitoring and Enforcement: A Review of the Empirical Evidence. Review of Environmental Economics and Policy, 5(1), 3–24. doi:10.1093/reep/req017
  • IPPC. (n.d). Who we are | International Plant Protection Convention. Retrieved October 28, 2013, from https://www.ippc.int/about
  • IPPC. (2007). Description of the National Plant Protection Organization for Canada. Retrieved September 17, 2013, from https://www.ippc.int/sites/default/files/documents/20130423/1177612896233_Canadian_NPPO___CFIA___April_2007_2013042321%3A18En.pdf
  • Ko, K., Mendeloff, J., & Gray, W. (2010). The role of inspection sequence in compliance with the US Occupational Safety and Health Administration's (OSHA) standards: Interpretations and implications. Regulation & Governance, 4(1), 48–70. doi:10.1111/j.1748-5991.2010.01070.x
  • Lansink, A. O. (2011). Public and private roles in plant health management. Food Policy, 36(2), 166–170. doi:10.1016/j.foodpol.2010.10.006
  • MacLeod, A., Pautasso, M., Jeger, M. J., & Haines-Young, R. (2010). Evolution of the international regulation of plant pests and challenges for future plant health. Food Security, 2(1), 49–70. doi:10.1007/s12571-010-0054-7
  • McKirdy, S., Rodoni, B., Moran, J., & Sharma, S. (2012). Microbial threat - a growing challenge for plant biosecurity. Microbiology Australia.
  • Momagri. (n.d.). Food Security: Emerging Risk for the 21st Century and Main Theme of the World Economic Forum. Retrieved November 12, 2013, from http://www.momagri.org/UK/personal-accounts/Food-Security-Emerging-Risk-for-the-21st-Century-and-Main-Theme-of-the-World-Economic-Forum-_261.html
  • OAG. (2008). Report of the Auditor General of Canada to the House of Commons: Chapter 4 Managing Risks to Canada's Plant Resources – Canadian Food Inspection Agency. Retrieved August 28, 2013, from http://www.oag-bvg.gc.ca/internet/docs/parl_oag_200812_04_e.pdf
  • Outhwaite, O. (2010). The international legal framework for biosecurity and the challenges ahead. Review of European Community & International Environmental Law, 19(2), 207–226.
  • Parks Canada. (2009, July 16). Parks Canada - Cape Breton Highlands National Park - Why is biodiversity important? Retrieved November 12, 2013, from http://www.pc.gc.ca/pn-np/ns/cbreton/natcul/natcul1/d/iii.aspx
  • Pluess, T., Jarošík, V., Pyšek, P., Cannon, R., Pergl, J., Breukers, A., & Bacher, S. (2012). Which Factors Affect the Success or Failure of Eradication Campaigns against Alien Species? PLoS ONE, 7(10), e48157. doi:10.1371/journal.pone.0048157
  • Roberts, W. (2009). The Revised International Plant Protection Convention – a New Context for Plant Quarantine. In R. N. Strange & M. L. Gullino (Eds.), The Role of Plant Pathology in Food Safety and Food Security (pp. 133–136). Dordrecht: Springer Netherlands. Retrieved from http://www.springerlink.com/index/10.1007/978-1-4020-8932-9_11
  • Rodoni, B. (2009). The role of plant biosecurity in preventing and controlling emerging plant virus disease epidemics. Virus Research, 141(2), 150–157. doi:10.1016/j.virusres.2008.11.019
  • Secretariat of the Convention on Biological Diversity. (2014a). What are Invasive Alien Species? Retrieved February 9, 2014, from http://www.cbd.int/invasive/WhatareIAS.shtml
  • Secretariat of the Convention on Biological Diversity. (2014b). What Needs to be Done? Retrieved February 9, 2014, from http://www.cbd.int/invasive/done.shtml
  • Secretariat of the Convention on Biological Diversity. (2014c). What's the Problem? Retrieved February 9, 2014, from http://www.cbd.int/invasive/problem.shtml
  • Secretariat of the Convention on Biological Diversity. (2014d). Why Does it Matter? Retrieved February 9, 2014, from http://www.cbd.int/invasive/matter.shtml
  • TBS. (2009, March 23). Directive on the Evaluation Function [Directive]. Retrieved April 4, 2014, from http://www.tbs-sct.gc.ca/pol/doc-eng.aspx?id=15681&section=text#appA
  • TBS. (2012). Policy on Evaluation. Retrieved May 8, 2014, from http://www.tbs-sct.gc.ca/pol/doc-eng.aspx?id=15024&section=text
  • Telle, K. (2009). The threat of regulatory environmental inspection: impact on plant performance. Journal of Regulatory Economics, 35(2), 154–178. doi:10.1007/s11149-008-9074-z

Appendix B – Evaluation Matrix

Evaluation Questions

Program Need

Table 16: What demonstrated needs does the PPP address? (i.e., What is the rationale for the program?)
Potential Indicators PPP Overall Horticulture Potato Forestry Grains and Field Crops Invasive Alien Species/Invasive Plants Data Sources

Documented need for plant protection in Canada

X X X X X X
  • Document and literature review
  • Secondary data and cost analysis
  • Case studies
  • Key informant interviews

Identified instances of plant-related emergencies

X X
  • Document and literature review
  • Secondary data and cost analysis
  • Case studies
  • Key informant interviews
Table 17: What existing evidence suggests that the activities undertaken as part of the PPP will address demonstrated needs? (i.e., What evidence supports the program logic?)
Potential Indicators PPP Overall Horticulture Potato Forestry Grains and Field Crops Invasive Alien Species/Invasive Plants Data Sources

Documented evidence of the effectiveness of programming similar to that offered under the PPP

X
  • Document and literature review
  • Secondary data and cost analysis
  • Case studies
  • Key informant interviews

Perception of program design and delivery staff

X X X X X X
  • Document and literature review
  • Secondary data and cost analysis
  • Case studies
  • Key informant interviews
Table 18: Are there ways of addressing these needs, other than through the current activities of the PPP? (i.e., Are there alternatives to the current program delivery structure)
Potential Indicators PPP Overall Horticulture Potato Forestry Grains and Field Crops Invasive Alien Species/Invasive Plants Data Sources

Evidence of effective alternatives to the PPP in the literature

X
  • Document and literature review
  • Case studies
  • Key informant interviews

Successful alternatives identified by program design and delivery staff

X X X X X X
  • Document and literature review
  • Case studies
  • Key informant interviews

Government Priorities

Table 19: Is the PPP designed to support the CFIA strategic outcome? (i.e., Does the program logic lead to the PPP contributing to the Agency's strategic outcome?)
Potential Indicators PPP Overall Horticulture Potato Forestry Grains and Field Crops Invasive Alien Species/Invasive Plants Data Sources

Documented evidence of the Agency's strategic outcome being considered in the PPP's design

X
  • Document and literature review
  • Key informant interviews

Perception of PPP managers regarding the PPP's support of the Agency's strategic goals

X
  • Document and literature review
  • Key informant interviews
Table 20: Other than the CFIA's strategic outcome, to which federal priorities is the PPP expected to contribute? (i.e., Does the program logic suggest that the PPP will contribute to additional federal priorities? Which priorities are these?)
Potential Indicators PPP Overall Horticulture Potato Forestry Grains and Field Crops Invasive Alien Species/Invasive Plants Data Sources

Documented evidence of other federal priorities being considered in the PPP's design

X X X
  • Document and literature review
  • Key informant interviews

Perception of PPP managers regarding PPP support of other federal goals

X
  • Document and literature review
  • Key informant interviews

Federal Roles and Responsibilities

Table 21: Are the responsibilities of the CFIA, as they relate to plant protection, clear and well-documented? (i.e., Is there clarity around the PPP's scope and areas of jurisdiction?)
Potential Indicators PPP Overall Horticulture Potato Forestry Grains and Field Crops Invasive Alien Species/Invasive Plants Data Sources

Documentation regarding the Agency's responsibilities under the program

X
  • Document and literature review
  • Secondary data and cost analysis
  • Key informant interviews

Level of understanding of Agency's responsibilities under the program

X X X X X X
  • Document and literature review
  • Secondary data and cost analysis
  • Key informant interviews
Table 22: Are the activities of the CFIA, under the PPP, within the scope of the Agency's responsibilities for plant protection? (i.e., Is the Agency acting within its mandate under the program?)
Potential Indicators PPP Overall Horticulture Potato Forestry Grains and Field Crops Invasive Alien Species/Invasive Plants Data Sources

Program profile of the PPP, including itemization of activities compared to documented Agency responsibilities for plant protection

X
  • Document and literature review
  • Secondary data and cost analysis
  • Key informant interviews

Perception of program design and delivery staff and managers regarding program delivery activities

X X X X X X
  • Document and literature review
  • Secondary data and cost analysis
  • Key informant interviews
Table 23: Are other federal departments who support the work of the PPP meeting their program delivery requirements? (i.e., Are all federal partners in the PPP actively participating as planned?)
Potential Indicators PPP Overall Horticulture Potato Forestry Grains and Field Crops Invasive Alien Species/Invasive Plants Data Sources

Documentation of the roles and responsibilities of other federal departments in the PPP, if any

X X X X
  • Document and literature review
  • Key informant interviews

Measure of other federal departments' contribution (financial, in-kind, etc.) to the PPP

X X X X X
  • Document and literature review
  • Key informant interviews

Management and program delivery perception of the contribution of other federal departments to the PPP

X X X X X X
  • Document and literature review
  • Key informant interviews

Achievement of Expected Outcomes

Table 24: Has the CFIA implemented the PPP as planned? (i.e., Does program delivery align with the Agency's planning documentation, manuals, etc.?)
Potential Indicators PPP Overall Horticulture Potato Forestry Grains and Field Crops Invasive Alien Species/Invasive Plants Data Sources

Financial data on resources allocated to various program areas

X
  • Secondary data and cost analysis
  • Case studies
  • Key informant interviews

Feedback from program design and delivery staff on implementation of programming relative to activities identified in the program profile

X X X X X
  • Secondary data and cost analysis
  • Case studies
  • Key informant interviews
Table 25: What opportunities are there to improve the way in which PPP activities are undertaken, in order to produce the PPP's outputs more effectively? (i.e., How can the program be improved?)
Potential Indicators PPP Overall Horticulture Potato Forestry Grains and Field Crops Invasive Alien Species/Invasive Plants Data Sources

Perception of program design and delivery staff and managers regarding improvements to the PPP

X X X X X X
  • Key informant interviews
Table 26: Has the PPP contributed to its expected immediate outcomes? (i.e., Has the program met its immediate goals?)
Potential Indicators PPP Overall Horticulture Potato Forestry Grains and Field Crops Invasive Alien Species/Invasive Plants Data Sources

Number of regulated foreign plant pests that enter into Canada through regulated pathways and establish themselves

X
  • Secondary data and cost analysis
  • Case studies
  • Key informant interviews

Percentage of confirmed introductions of quarantine pests for which notices are issued

X X
  • Secondary data and cost analysis
  • Case studies
  • Key informant interviews

Percentage of notices issued in a timely manner

X X
  • Secondary data and cost analysis
  • Case studies
  • Key informant interviews

Percentage of certified plants and plant products shipments (lots) that meet the country of destination phytosanitary import requirements

X X X X
  • Secondary data and cost analysis
  • Case studies
  • Key informant interviews

Percentage of inspected shipments from offshore system approaches or preclearance programs in compliance with federal regulations

X X X X
  • Secondary data and cost analysis
  • Case studies
  • Key informant interviews

Percentage of pre-arrival documentation in compliance with Canadian import requirements

X X X X
  • Secondary data and cost analysis
  • Case studies
  • Key informant interviews

Percentage of new pest detections that have a science-based management plan

X X
  • Secondary data and cost analysis
  • Case studies
  • Key informant interviews

Percentage of new pest detections that have a science-based management plan initiated within one year

X X
  • Secondary data and cost analysis
  • Case studies
  • Key informant interviews

Degree of participation in international organizations related to plant protection

X
  • Secondary data and cost analysis
  • Case studies
  • Key informant interviews

Percentage of the Canadian public aware of plant-related risks

X
  • Secondary data and cost analysis
  • Case studies
  • Key informant interviews
Table 27: To what degree has the PPP contributed to its immediate outcomes? (i.e., What has been the PPP's marginal contribution to its immediate outcomes?)
Potential Indicators PPP Overall Horticulture Potato Forestry Grains and Field Crops Invasive Alien Species/Invasive Plants Data Sources

Marginal impact assessment of indicators in Question 11

X X X X X X
  • Secondary data and cost analysis
  • Case studies
  • Key informant interviews
Table 28: Do the PPP's immediate outcomes translate into the achievement of its expected intermediate outcomes? (i.e., Are expected downstream outcomes linked to immediate outcomes?)
Potential Indicators PPP Overall Horticulture Potato Forestry Grains and Field Crops Invasive Alien Species/Invasive Plants Data Sources

Documented evidence in the literature of a link between PPP immediate and intermediate outcomes

X
  • Document and literature review

Efficiency and Economy

Table 29: What are the financial costs of delivering the PPP? (i.e., What is the economy and efficiency of the PPP?)
Potential Indicators PPP Overall Horticulture Potato Forestry Grains and Field Crops Invasive Alien Species/Invasive Plants Data Sources

Current costs of delivering the PPP

X
  • Secondary data and cost analysis
  • Case studies

Changes in the PPP costs over time

X
  • Secondary data and cost analysis
  • Case studies

Changes in PPP outputs over time relative to costs

X
  • Secondary data and cost analysis
  • Case studies
Table 30: What are the costs of delivering the PPP relative to its contribution to its expected outcomes? (i.e., What is the cost-effectiveness of the PPP?)
Potential Indicators PPP Overall Horticulture Potato Forestry Grains and Field Crops Invasive Alien Species/Invasive Plants Data Sources

Comparison of PPP costs to marginal contribution to outcomes

X
  • Secondary data and cost analysis
  • Case studies
Table 31: How does the cost of delivering the PPP compare to prior PPP expenditure levels and/or the cost of alternatives? (i.e. What is the relative cost-effectiveness of the PPP?)
Potential Indicators PPP Overall Horticulture Potato Forestry Grains and Field Crops Invasive Alien Species/Invasive Plants Data Sources

Comparison of PPP cost-effectiveness relative to alternative programming

X
  • Secondary data and cost analysis
  • Case studies
Table 32: Is there evidence that alternatives to the PPP could contribute to the same PPP outcomes more efficiently? (i.e., Do alternatives to the PPP offer a more cost-effective way of achieving outcomes?)
Potential Indicators PPP Overall Horticulture Potato Forestry Grains and Field Crops Invasive Alien Species/Invasive Plants Data Sources

Documented evidence in the literature of equally effective but more efficient plant protection programming

X
  • Document and literature review

Appendix C – Case study protocol

Case Study Protocol for the Evaluation of the Plant Protection Program (PPP)

Three case studies are to take place as part of the evaluation of the PPP. The framework for the evaluation identifies the following case study topics:

  • the Plum Pox Monitoring and Management Program (PPMMP)
  • Invasive Alien Species activities (IAS)
  • compensation payments (CPs) in accordance with the regulations under the Plant Protection Act

Overall, the case studies are meant to provide deeper insight into the activities of the PPP by providing detailed examples – or cases – of the program's work in specific areas. Moreover, the case studies are meant to leverage information on the implementation of these three topics to assess their relevance, effectiveness, and determine how the CFIA's experience with each can suggest improvements to delivery.

Generally, all three case studies will follow a standardized approach. This approach involves four main activities, which, in some instances, may occur concurrently. For each topic, there will be:

  • a review of related program documentation
  • a review of available budget and expenditure data
  • key informant interviews/focus groups with stakeholders
  • drafting of case study technical reports

At this point in the evaluation, a significant number of plant protection-related documents have been provided by the CFIA. In addition, the Agency has provided financial information related to its work and, in particular, the three topics noted above. A portion of these documents and the financial information has already been identified as specifically related to each of the three topics. The list of references included in Appendix A to this protocol provides a preliminary list for illustrative purposes. With that said, additional documents and information will be added over time.

A brief review of the documentation included in Appendix A suggests that both the volume and type of information related to each of the three topics differs. For example, a more limited amount of documentation specifically related to CPs is available. Further, the documents related to PPMMP and IAS often explore activities related to both topics and provide a fair amount of information on planning. While these documents, along with the available financial information, provide excellent data to support the case studies, additional information and interpretation is required to provide an integrated picture of each topic.

As a result, each case study will include up to five key informant interviews and/or focus groups with stakeholders familiar with one or more of the case study topics. In all cases, key informants will be asked to reflect on the following:

  • the rationale or need for the activities involved in the topic
  • how the activities related to the topic evolved over time
  • whether it is effective at achieving its stated goals
  • how other federal departments, orders of government, and nongovernmental organizations have been involved with the case study topic
  • how the CFIA's experiences with the topic can point to ways of improving the Plant Protection Program's work

For each of the case study topics, specific areas of inquiry will be particularly important. These include the following:

PPMMP

  • When and how Plum Pox was first identified.
  • How the risk associated with Plum Pox was assessed and the degree to which the assessment changed over time.
  • What the initial response to the virus was and how this developed over time.
  • How compensation payments associated with PPMMP were established and distributed.
  • Whether the CFIA's response was effective, and whether or not this effectiveness was conditioned by the actions of other stakeholders.
  • If there are lessons to be learned from the risk mitigation approaches taken to Plum Pox that may be more generally applied to the PPP's work.
  • Examples of success in managing Plum Pox.
  • Challenges experienced during the PPMMP.
  • Whether the PPMMP represented an efficient use of CFIA resources.

IAS

  • What initially motivated an action plan for IAS affecting plants.
  • Whether the development of the action plan fundamentally altered plant protection activities already taking place at the Agency.
  • How the CFIA's role in this action plan was identified and whether it has changed over time.
  • How the roles of federal, provincial, and other stakeholders were initially determined.
  • Why the CFIA's funding for IAS initially resulted from a Treasury Board submission but will eventually be rolled into A-base funding.
  • How funding for IAS will be determined, monitored, and used once this change occurs.
  • Why budgeted allocations for IAS have been consistently higher than actual expenditures.
  • Whether the PPP's activities related to IAS have evolved over time.
  • Whether IAS activities have been an effective and efficient use of Agency resources.
  • Examples of success related to IAS.
  • Challenges experienced during the delivery of IAS activities.

CPs

  • Whether there is a clear understanding about the intended effect of compensation payments.
  • How determinations are made regarding whether compensation payments are made.
  • How determinations are made regarding level of compensation payments.
  • Whether there are demonstrable outcomes of compensation payments beyond the mitigation of direct financial loss.
  • How other stakeholders such as federal departments, other orders of government, and nongovernmental organizations also support CP related outcomes.

The key informant interviews will also provide an opportunity to request additional documentation in support of assertions made during the discussions.

In order to ensure consistency of reporting, key findings from the case studies will be presented in a standardized format. Each case study write-up will include the following sections:

  • Introduction – This section will provide a brief introduction and explanation of the case study topic.
  • Background – This section will explain the initial motivation, operation, and evolution of the topic over time.
  • Goals and impacts – This section will outline the intended goals of the case study topic and present evidence of their achievement.
  • Lessons learned – This section will provide suggestions regarding lessons learned from the implementation of each case study topic, including those related to the efficiency of their delivery.

The three case study write-ups will be combined into the final case study technical report. This case study report will present each write-up independently and then provide a concluding section where information from the write-ups will be used to address specific PPP evaluation questions.

Case Study Protocol Appendix A

PPMMP

  • AAFC, & CFIA. (2011). Lessons Learned: The Plum Pox Eradication Program - Draft Report.
  • CBSA. (2011). 2011 Border Threat and Risk Assessment (BTRA). RDIMS #2971526.
  • CFIA. (n.da). Communication products for the PPMMP. RDIMS #4135568.
  • CFIA. (n.db). Industry Public Notice: Prohibition on the Propagation of Plants Susceptible to Plum Pox Virus. RDIMS #4135572.
  • CFIA. (n.dc). National Plum Pox Monitoring and Management Program Sampling Plan for 2011-2015 and On-going. RDIMS #4135576.
  • CFIA. (n.dd). Number of samples tested by CFIA Plant laboratories from 2009-10 to 2012-13. RDIMS #4104262.
  • CFIA. (n.de). Plum Pox Monitoring and Management Sampling Plan **CFIA sampling only**. RDIMS #4135574.
  • CFIA. (n.df). Plum Pox Virus: Brainstorming items. RDIMS #4135562.
  • CFIA. (n.dg). Plum Pox Virus: Detection Scenario & Action Document. RDIMS #4135580.
  • CFIA. (n.dh). Plum Pox Virus: Regulatory (Monitoring) Program. RDIMS #4135559.
  • CFIA. (n.di). PPV: Past eradication and new direction. RDIMS #4135564.
  • CFIA. (n.dj). Proposal to implement elements of the comprehensive three year plan to address the presence of the Plum Pox Virus (PPV) in Canada.
  • CFIA. (n.dk). Public Notice Plum Pox Virus (PPV). RDIMS #4135573.
  • CFIA. (2005). Canadian Food Inspection Agency 2004-2005 Performance Report.
  • CFIA. (2006). Canadian Food Inspection Agency 2005-2006 Performance Report.
  • CFIA. (2007a). Canadian Food Inspection Agency 2006-2007 Performance Report.
  • CFIA. (2007b). Plum Pox Virus (PPV) - Context. Retrieved September 17, 2013, from http://www.collectionscanada.gc.ca/webarchives/20071124141616/http://www.inspection.gc.ca/english/plaveg/pestrava/ppv/contexte.shtml#surv
  • CFIA. (2008). Plum Pox Virus Infested Places Order, 2008. Retrieved September 17, 2013, from http://www.inspection.gc.ca/plants/plant-protection/diseases/plum-pox-virus/monitoring-and-management-program/order/eng/1329402508382/1329402732284
  • CFIA. (2009). Canadian Food Inspection Agency 2008-2009 Performance Report.
  • CFIA. (2010). Canadian Food Inspection Agency 2009-2010 Performance Report.
  • CFIA. (2011a). 2011 Plum Pox Virus Survey Data. RDIMS #4135577.
  • CFIA. (2011b). Approval of the Plum Pox Monitoring and Management Program. RDIMS #4135570.
  • CFIA. (2011c). Canadian Food Inspection Agency 2010-2011 Performance Report.
  • CFIA. (2011d). Compensation summary for Plum Pox Virus (2004-10+). RDIMS #4135588.
  • CFIA. (2011e). D-08-04: Plant Protection Import Requirements for Plants and Plant Parts for Planting: Preventing the Entry and Spread of Regulated Plant Pests Associated with the Plants for Planting Pathway - Plants - Canadian Food Inspection Agency. Retrieved September 17, 2013, from http://www.inspection.gc.ca/plants/plant-protection/directives/imports/d-08-04/eng/1323752901318/1323753560467
  • CFIA. (2011f). Notice Plum Pox Virus Quarantine Area Propagation Ban (door Hanger). RDIMS #4135571.
  • CFIA. (2011g). Plant Business Line Plan for 2011-12: Overview of the Strategic and Beyond. RDIMS #2864818.
  • CFIA. (2011h). Plum Pox Virus - Plants - Canadian Food Inspection Agency. Retrieved September 17, 2013, from http://www.inspection.gc.ca/plants/plant-protection/diseases/plum-pox-virus/eng/1323888514908/1323889333540
  • CFIA. (2011i). Plum Pox Virus Monitoring and Management Program - Plants - Canadian Food Inspection Agency. Retrieved September 17, 2013, from http://www.inspection.gc.ca/plants/plant-protection/diseases/plum-pox-virus/monitoring-and-management-program/eng/1323887724804/1323889930176
  • CFIA. (2011j). Plum Pox Virus Plum Pox Monitoring and Management Program Consultation Plan. RDIMS #4135563.
  • CFIA. (2011k). Plum Pox Virus: Information Release. RDIMS #4135566.
  • CFIA. (2011l). Plum Pox Virus: Monitoring and Management Program - Agenda - OTFPMB / CNLA / OMAFRA / AAFC / CFIA. RDIMS #4135561.
  • CFIA. (2011m). Plum Pox Virus: Regulatory (Monitoring) Program 2011 ~ 2015 and beyond. RDIMS #4135558.
  • CFIA. (2011n). PPMMP Industry Bulletin Following Approval. RDIMS #4135560.
  • CFIA. (2011o). PPV Products Communications Strategy. RDIMS #4135567.
  • CFIA. (2012a). Canadian Food Inspection Agency 2011-2012 Performance Report.
  • CFIA. (2012b). Canadian Food Inspection Agency 2012–13 Estimates Part III–Report on Plans and Priorities.
  • CFIA. (2012c). Emergency Management Research In Action: Plant Health.
  • CFIA. (2012d). Mission to Europe: Trip report (PPV). RDIMS #4135582.
  • CFIA. (2012e). Plum Pox Virus Regulatory (monitoring) Program. RDIMS #4135578.
  • CFIA. (2013a). 2012-13 DPR PPMMP Horizontal Initiative Table. RDIMS #4135584.
  • CFIA. (2013b). D- 99-07: Policy for Importation from the United States and Domestic Movement of Plum Pox Virus (PPV) susceptible Prunus Propagative Plant Material - Plants - Canadian Food Inspection Agency. Retrieved September 17, 2013, from http://www.inspection.gc.ca/plants/plant-protection/directives/date/d-99-07/eng/1312320268313/1312324196014
  • CFIA. (2013c). IAS and PPV/PPMMP from fiscal years 2009-10 to 2012-13 Planned and Actual Expenditures ($ millions). RDIMS #4135083.
  • CFIA. (2013d). Merlin and Infomedia Results: invasive alien species, plum pox virus, emerald ash borer. RDIMS #4254020.
  • CFIA. (2013e). Plum Pox Monitoring and Management Program Update meetings. RDIMS #4135585.
  • CFIA. (2013f). Regulations Amending the Plum Pox Virus Compensation and Plant Protection Regulations. Retrieved September 17, 2013, from http://www.inspection.gc.ca/about-the-cfia/acts-and-regulations/forward-regulatory-plan/2012-2014/plant-protection-regulations/eng/1366731067188/1366731244606
  • CFIA, & AAFC. (2011). Proposed Research Initiatives for the PPV Management & Monitoring Program. RDIMS #4135565.
  • CFIA, & OMAFRA. (n.d). Mission to Europe: Spain, France, and United Kingdom's Perspective on Plum Pox Virus. RDIMS #4135581.
  • GoC. (2011). Lessons Learned: The Plum Pox Eradication Program: Presentation to the CFIA Evaluation Committee.
  • GoC. (2013). Plum Pox Virus Compensation Regulations, 2004 (SOR/2005-131). Retrieved September 17, 2013, from http://laws-lois.justice.gc.ca/PDF/SOR-2005-131.pdf
  • TBS. (n.d). Treasury Board Submission: Plum Pox Eradication Program - Additional Funding.
  • TBS. (2004). Treasury Board Submission: Plum Pox Eradication Program.
  • TBS. (2011). Treasury Board Submission: Funding for CFIA and AAFC to Monitor and Implement Measures to Mitigate the Spread of Plum Pox Virus (PPV).

IAS

  • AGRI. (n.d). Standing Committee on Agriculture and Agri-Food Potential IAS Questions and Canadian Food Inspection Agency (CFIA) Answers. RDIMS #4116096.
  • CFIA. (n.da). Canadian Food Inspection Agency Invasive Plants Policy - IASDP - Information for PBLC. RDIMS #4006434.
  • CFIA. (n.db). Canadian Food Inspection Agency Invasive Plants Policy - IP - Public Invasive Plants Policy Document. RDIMS #2800218.
  • CFIA. (n.dc). Canadian Food Inspection Agency Response to MINO Questions on Invasive Plants Policy. RDIMS #4114379.
  • CFIA. (n.dd). Invasive Alien Species (IAS) (Table D). RDIMS #4116092.
  • CFIA. (n.de). Invasive Plants Policy. RDIMS #4116095.
  • CFIA. (n.df). Minister of State Briefing on WCG: Speaking Notes. RDIMS #4116099.
  • CFIA. (n.dg). Number of samples tested by CFIA Plant laboratories from 2009-10 to 2012-13. RDIMS #4104262.
  • CFIA. (2005). Action Plan for invasive alien terrestrial plants and plant pests - Phase 2 – Proposed Implementation Plan. Retrieved September 17, 2013, from http://www.collectionscanada.gc.ca/webarchives/20071124185600/http://www.inspection.gc.ca/english/plaveg/invenv/action/phase2e.shtml
  • CFIA. (2006). Action Plan for invasive alien terrestrial plants and plant pests - Phase 1 – Key Initiatives. Retrieved September 17, 2013, from http://www.collectionscanada.gc.ca/webarchives/20071124173415/http://www.inspection.gc.ca/english/plaveg/invenv/action/phase1e.shtml
  • CFIA. (2007). CBIN - Invasive Alien Species. Retrieved September 17, 2013, from http://www.collectionscanada.gc.ca/webarchives/20071116034707/http://www.cbin.ec.gc.ca/issues/ias.cfm?lang=e
  • CFIA. (2008a). Invasive Alien Plants in Canada - Summary Report. Retrieved September 17, 2013, from http://epe.lac-bac.gc.ca/100/206/301/cfia-acia/2011-09-21/www.inspection.gc.ca/english/plaveg/invenv/techrpt/summrese.shtml
  • CFIA. (2008b). Invasive Alien Plants in Canada - Technical Report. Retrieved September 17, 2013, from http://epe.lac-bac.gc.ca/100/206/301/cfia-acia/2011-09-21/www.inspection.gc.ca/english/plaveg/invenv/techrpt/techrese.shtml
  • CFIA. (2010). Consultations on the Least Wanted Invasive Plants Project and Invasive Plants Policy. Retrieved October 9, 2013, from http://epe.lac-bac.gc.ca/100/206/301/cfia-acia/2011-09-21/www.inspection.gc.ca/english/plaveg/invenv/consult/consulte.shtml
  • CFIA. (2011a). Bridge to the Future II: Building Opportunities Meeting of Board of Directors for Invasive Alien. RDIMS #4114376.
  • CFIA. (2011b). Canadian Food Inspection Agency Invasive Alien Species Program - Draft for IAS Sub-program discussion April 11, 2011. RDIMS #4116117.
  • CFIA. (2011c). Canadian Invasive Plant Framework: A Collaborative Approach to Addressing Invasive Plants in Canada. RDIMS #4109373.
  • CFIA. (2011d). CFIA's Invasive Alien Species Initiative – Moving Forward Collaboratively. RDIMS #4135745.
  • CFIA. (2011e). D-11-03: Phytosanitary requirements to prevent the spread of Eriochloa villosa (woolly cup grass) and control infestations in Canada. RDIMS #4116093.
  • CFIA. (2011f). Invasive Alien Species Minister's Speaking Points. RDIMS #4116116.
  • CFIA. (2011g). Invasive Plants In Canada. Retrieved June 21, 2013, from http://www.inspection.gc.ca/plants/plant-protection/invasive-plants/eng/1306601411551/1306601522570
  • CFIA. (2011h). Nematodes / Other - Plants. Retrieved September 17, 2013, from http://www.inspection.gc.ca/plants/plant-protection/nematodes-other/eng/1321563900456/1321564231938
  • CFIA. (2011i). Plant Diseases. Retrieved September 17, 2013, from http://www.inspection.gc.ca/plants/plant-protection/diseases/eng/1322807235798/1322807340310
  • CFIA. (2012a). IAS Program Highlights: National Invasive Species Forum. RDIMS #4114381.
  • CFIA. (2012b). Invasive Alien Species - Anticipated Questions. RDIMS #4116098.
  • CFIA. (2012c). Invasive Alien Species: External Audit Committee. RDIMS #4135748.
  • CFIA. (2012d). Invasive Plants - Fact Sheets. Retrieved September 17, 2013, from http://www.inspection.gc.ca/plants/plant-protection/invasive-plants/fact-sheets/eng/1331614724083/1331614823132
  • CFIA. (2012e). Invasive Plants Policy. Retrieved June 21, 2013, from http://www.inspection.gc.ca/plants/plant-protection/invasive-plants/policy/eng/1328298038970/1328298211382
  • CFIA. (2012f). Invasive Species. Retrieved September 4, 2013, from http://www.inspection.gc.ca/plants/plant-protection/invasive-species/eng/1328325263410/1328325333845
  • CFIA. (2012g). Memorandum to the President CFIA's Regulatory Activities for Woolly Cup Grass (for Decision). RDIMS #4116097.
  • CFIA. (2012h). Update on the CFIA's Invasive Alien Species (IAS) Program. RDIMS #4116103.
  • CFIA. (2013a). IAS and PPV/PPMMP from fiscal years 2009-10 to 2012-13 Planned and Actual Expenditures ($ millions). RDIMS #4135083.
  • CFIA. (2013b). Implementation Plan: Invasive Plants Program. RDIMS #4135822.
  • CFIA. (2013c). Import Procedures - Plants. Retrieved September 17, 2013, from http://www.inspection.gc.ca/plants/plant-protection/imports/eng/1324569244509/1324569331710
  • CFIA. (2013d). Insects - Plants. Retrieved September 17, 2013, from http://www.inspection.gc.ca/plants/plant-protection/insects/eng/1307077188885/1307078272806
  • CFIA. (2013e). Invasive Alien Species Anticipated Question (January 9, 2013). RDIMS #4116104.
  • CFIA. (2013f). Invasive Alien Species Program. RDIMS #4135836.
  • CFIA. (2013g). Memorandum of Understanding for Critical Plant Pests in Manitoba. RDIMS #4109375.
  • CFIA. (2013h). Memorandum regarding Asian Gypsy Moth Certification for Marine Vessels Entering Canadian Ports (Notice to Industry). RDIMS #4116105.
  • CFIA. (2013i). Memorandum to the Minister Posting of the Canadian Food Inspection Agency's Invasive Plants Invasive Plants Directive (D-12-01). RDIMS #4135834.
  • CFIA. (2013j). Memorandum to the President the CFIA's Proposed Voluntary Commitment to Environment Canada's Federal Sustainable Development Strategy (for Approval) (IAS). RDIMS #4116106.
  • CFIA. (2013k). Merlin and Infomedia Results: invasive alien species, plum pox virus, emerald ash borer. RDIMS #4254020.
  • CFIA. (2013l). Regular Briefings for MinO - Updates on Invasive Plants Program and Grain Import Framework - June 20, 2013. RDIMS #4114384.
  • CFIA. (2013m). Response Plan for Paterson's Curse in Saskatchewan (Echium Plantagineum). RDIMS #4116108.
  • CFIA. (2013n). Summary Table of Invasive Plant Species Considered As Regulated Pests under the Plant Protection Act. RDIMS #3874323.
  • CFIA. (2013o). Summary Table of Invasive Plant Species Considered As Regulated Pests under the Plant Protection Act. RDIMS #4135833.
  • CFIA. (2013p). White Paper: Virtual Invaders: The Increasing Risk e-Commerce Plays in the Imports of Regulated Pests and Invasive Species in Canada. RDIMS #4116107.
  • EC. (2012a). Invasive Alien Species Partnership Program (IASPP). Retrieved October 9, 2013, from http://www.ec.gc.ca/eee-ias/default.asp?lang=En&n=A49893BC-1
  • EC. (2012b). Invasive Alien Species Partnership Program: 2005–2010 Report. Retrieved October 9, 2013, from http://www.ec.gc.ca/nature/default.asp?lang=En&n=B008265C-1#_Toc315269275
  • EKOS. (2010). Canadians' Views on Specific Risks to Canada's Ecosystems – Invasive Species.
  • GAO. (2002). Invasive Species: Clearer Focus and Greater Commitment Needed to Effectively Manage the Problem.
  • GAO. (2006). Invasive Forest Pests: Lessons Learned from Three Recent Infestations May Aid in Managing Future Efforts.
  • GoC. (2004). An Invasive Alien Species Strategy for Canada. Retrieved September 17, 2013, from http://www.collectionscanada.gc.ca/webarchives/20071116043522/http://www.cbin.ec.gc.ca/issues/ias/documents/final_ias_strategic_plan_smaller_e.pdf
  • TDV Global Inc. (2008). Invasive Alien Species Program Evaluation. RIDMS #2360940.

CPs

  • CFIA. (n.d). Emerald Ash Borer Compensation & Audit Log. RDIMS #4108132.
  • CFIA. (2010). Discussion Paper: Plant Health Regulatory Proposal - Amending regulations made under the Plant Protection Act. RDIMS #2953377.
  • CFIA. (2011a). Compensation summary for Plum Pox Virus (2004-10+). RDIMS #4135588.
  • CFIA. (2011b). Plant Health Regulatory Proposal - Amending regulations made under the Plant Protection Act. RDIMS #2953373.
  • CFIA. (2013a). 2012-13 Compensation Payments by Province Actual Expenditures (in dollars) As at March 31, 2013 - Final. RDIMS #4120335.
  • CFIA. (2013b). Asian Long-horned Beetle Compensation & Audit Log. RDIMS #4108130.
  • CFIA. (2013c). Canadian Food Inspection Agency 2012-13 Compensation Payments by Province Actual Expenditures (in dollars) As at March 31, 2013 - Final. RDIMS #4135084.
  • GoC. (2005). Plant Protection Act S.C.. 1990 c.22. Retrieved August 30, 2013, from http://laws-lois.justice.gc.ca/PDF/P-14.8.pdf
  • Potato Committee. (2011). Report to the Potato Committee to the 89th Annual General Meeting of the Canadian Horticultural Council. Retrieved October 2, 2013, from http://www.hortcouncil.ca/uploads/file/English/Committee%20Reports/2011/2011_Potato_Committee_report_ENGLISH.pdf

Appendix D – Case study interview guide

Key Informant Interview Guide – Invasive Alien Species

Evaluation of the Plant Protection Program (PPP)

In accordance with the CFIA 2013 Evaluation Plan, the Agency is currently evaluating its Plant Protection Program (PPP). As one of four subprograms operating under the CFIA's Plant Resources Program, the PPP attempts to mitigate the risks associated with plant pests and diseases in Canada.

As per the Treasury Board Secretariat's (TBS) directive on the evaluation function (April 2009), the evaluation addresses the issues of relevance/rationale and performance (effectiveness, efficiency, and economy). More specifically, the evaluation will look at the:

  • continued need for the program;
  • alignment of the program with government priorities;
  • alignment of the program with federal roles and responsibilities; and
  • achievement of outcomes.

The CFIA, Evaluation Directorate (ED) has hired PRA Inc., an independent research firm, to conduct this evaluation.

As part of the evaluation, PRA is conducting three case studies related to the following topics:

  • the Plum Pox Monitoring and Management Program (PPMMP)
  • the Invasive Alien Species strategy (IAS)
  • compensation payments (CPs) in accordance with the regulations under the Plant Protection Act

These case studies involve a review of related documentation, a review of financial records, and a series of key informant interviews. As someone familiar with IAS, we hope that you will consider participating in an interview as part of that particular case study.

It is important to understand that this interview guide is meant to facilitate an open-ended discussion of the IAS. While the questions below touch on issues of particular importance to the PPP evaluation, please feel free to raise additional issues that you consider important to the IAS.

During the discussion, please speak from your own experience with the IAS. If there are any questions that you cannot address, simply say so, and the interviewer will move on to the next question. Since some of the questions ask how the IAS relates to other activities of the PPP, a logic model for the program is included as an appendix to this interview guide.

Participation in the evaluation interviews is voluntary. Responses provided will be administered in accordance with the Privacy Act and other applicable privacy laws. Further, all reporting will be written to provide aggregate results only, and no comments will be linked back to you, individually.

If at any time you have questions regarding the evaluation or this interview process, please do not hesitate to contact either the ED or PRA.

Introduction
  • 1. To help us better understand your answers, could you please introduce yourself and describe your work related to IAS?
Rationale for IAS
  • 2. Could you please help us understand what initially motivated an action plan for IAS affecting plants?
  • 3. Did the development of the action plan fundamentally change the way that the PPP operated? Did it identify new activities for the program, consolidate certain activities, etc.?
  • 4. How was the CFIA's role and the role of the PPP first identified in the action plan, and have they changed over time?
  • 5. How were the roles of other federal, provincial, and non-governmental stakeholders first determined? Do they still play the same IAS-related role?
  • 6. Initially, IAS funding appears to have been the result of a Treasury Board Submission. However, this funding will soon be rolled into A-base funding. How will this affect the way in which funding for IAS will be determined, monitored, and used?
  • 7. Budgeted amounts for IAS appear to have exceeded expenditures consistently over recent years. Is there a reason for this difference? Does this affect IAS delivery or success?
Achievement of expected outcomes
  • 8. In your opinion, how successful has IAS been in mitigating the risks associated with invasive species? What would you point to as an example of this success?
  • 9. Was this success conditional on the actions of other stakeholders, such as other federal departments, other orders of government, and other non-governmental stakeholders?
  • 10. What challenges have IAS activities faced, and do these persist?
  • 11. Are there lessons to be learned that may help inform future IAS work?
Demonstration of efficiency and economy
  • 12. In your opinion, are IAS activities an efficient use of plant health resources? If not, how could these resources be better used?
Conclusion
  • 13. Do you have any other comments about IAS?

Thank you

Key Informant Interview Guide – Compensation Payments

Evaluation of the Plant Protection Program (PPP)

In accordance with the CFIA 2013 Evaluation Plan, the Agency is currently evaluating its Plant Protection Program (PPP). As one of four subprograms operating under the CFIA's Plant Resources Program, the PPP attempts to mitigate the risks associated with plant pests and diseases in Canada.

As per the Treasury Board Secretariat's (TBS) directive on the evaluation function (April 2009), the evaluation addresses the issues of relevance/rationale and performance (effectiveness, efficiency, and economy). More specifically, the evaluation will look at the:

  • continued need for the program;
  • alignment of the program with government priorities;
  • alignment of the program with federal roles and responsibilities; and
  • achievement of outcomes.

The CFIA, Evaluation Directorate (ED) has hired PRA Inc., an independent research firm, to conduct this evaluation.

As part of the evaluation, PRA is conducting three case studies related to the following topics:

  • the Plum Pox Monitoring and Management Program (PPMMP)
  • the Invasive Alien Species strategy (IAS)
  • compensation payments (CPs) in accordance with the regulations under the Plant Protection Act

These case studies involve a review of related documentation, a review of financial records, and a series of key informant interviews. As someone familiar with CPs, we hope that you will consider participating in an interview as part of that particular case study.

It is important to understand that this interview guide is meant to facilitate an open-ended discussion of CPs. While the questions below touch on issues of particular importance to the PPP evaluation, please feel free to raise additional issues that you consider important to CPs.

During the discussion, please speak from your own experience with CPs. If there are any questions that you cannot address, simply say so, and the interviewer will move on to the next question. Since some of the questions ask how CPs relate to other activities of the PPP, a logic model for the program is included as an appendix to this interview guide.

Participation in the evaluation interviews is voluntary. Responses provided will be administered in accordance with the Privacy Act and other applicable privacy laws. Further, all reporting will be written to provide aggregate results only, and no comments will be linked back to you, individually.

If at any time you have questions regarding the evaluation or this interview process, please do not hesitate to contact either the ED or PRA.

Introduction
  • 1. To help us better understand your answers, could you please introduce yourself and describe your work related to CPs?
Rationale for CPs
  • 2. Clearly, there is a statutory obligation for CPs under certain circumstances. However, from your perspective, what is the rationale for offering these payments?
  • 3. Could you please describe how decisions are made regarding whether compensation payments occur?
  • 4. Is there a systematic way in which the level of compensation is determined?
  • 5. Under what circumstances would levels of compensation change?
Achievement of expected outcomes
  • 6. Are there other outcomes expected because of compensation payments other than the mitigation of direct financial loss? If so, what are these outcomes?
  • 7. If there are expected outcomes other than the mitigation of direct financial loss, what would you point to as a demonstration of their achievement?
  • 8. What roles do other stakeholders, such as other federal departments, other orders of government, and other non-governmental organizations, play in the provision of CPs?
  • 9. Are there challenges to the provision of plant-related CPs?
  • 10. Are there lessons to be learned regarding CPs that may be applied to the PPP generally?
Demonstration of efficiency and economy
  • 11. In your opinion, are CPs an efficient use of CFIA resources? If not, how could these resources be better used?
Conclusion
  • 12. Do you have any other comments about CPs?

Thank you

Key Informant Interview Guide – Plum Pox Monitoring and Management Program

Evaluation of the Plant Protection Program (PPP)

In accordance with the CFIA 2013 Evaluation Plan, the Agency is currently evaluating its Plant Protection Program (PPP). As one of four subprograms operating under the CFIA's Plant Resources Program, the PPP attempts to mitigate the risks associated with plant pests and diseases in Canada.

As per the Treasury Board Secretariat's (TBS) directive on the evaluation function (April 2009), the evaluation addresses the issues of relevance/rationale and performance (effectiveness, efficiency, and economy). More specifically, the evaluation will look at the:

  • continued need for the program;
  • alignment of the program with government priorities;
  • alignment of the program with federal roles and responsibilities; and
  • achievement of outcomes.

The CFIA, Evaluation Directorate (ED) has hired PRA Inc., an independent research firm, to conduct this evaluation.

As part of the evaluation, PRA is conducting three case studies related to the following topics:

  • the Plum Pox Monitoring and Management Program (PPMMP)
  • the Invasive Alien Species strategy (IAS)
  • compensation payments (CPs) in accordance with the regulations under the Plant Protection Act

These case studies involve a review of related documentation, a review of financial records, and a series of key informant interviews. As someone familiar with the PPMMP, we hope that you will consider participating in an interview as part of that particular case study.

It is important to understand that this interview guide is meant to facilitate an open-ended discussion of the PPMMP. While the questions below touch on issues of particular importance to the PPP evaluation, please feel free to raise additional issues that you consider important to the PPMMP.

During the discussion, please speak from your own experience with the PPMMP. If there are any questions that you cannot address, simply say so, and the interviewer will move on to the next question. Since some of the questions ask how the PPMMP relates to other activities of the PPP, a logic model for the program is included as an appendix to this interview guide.

Participation in the evaluation interviews is voluntary. Responses provided will be administered in accordance with the Privacy Act and other applicable privacy laws. Further, all reporting will be written to provide aggregate results only, and no comments will be linked back to you, individually.

If at any time you have questions regarding the evaluation or this interview process, please do not hesitate to contact either the ED or PRA.

Introduction
  • 1. To help us better understand your answers, could you please introduce yourself and describe your work related to the PPMMP?
Rationale for PPMMP
  • 2. Could you please describe how Plum Pox was first identified?
  • 3. Once identified, how was the risk associated with Plum Pox assessed? Who was responsible for this assessment, and has it changed over time?
  • 4. What was the initial response to Plum Pox, and how has this response developed over time? Is this development typical of plant health risks identified in Canada?
  • 5. How were the compensation payments associated with Plum Pox first determined? Were these payments meant to simply compensate for financial losses, or was there another intent?
Achievement of expected outcomes
  • 6. In your opinion, how effective was the CFIA's response to Plum Pox? What would you point to as an example of this success?
  • 7. Was this success affected by the actions of other stakeholder's such as other federal departments, other orders of government, and other non-governmental stakeholders?
  • 8. What challenges were faced when responding to Plum Pox? Were these addressed effectively?
  • 9. Are there lessons to be learned from the risk mitigation approaches undertaken as part of the PPMMP?
Demonstration of efficiency and economy
  • 10. In your opinion, was the PPMMP an efficient use of plant health resources? If not, how could these resources have been better used?
Conclusion
  • 11. Do you have any other comments about the PPMMP?

Thank you

Appendix E – Interview guides

Key Informant Interview Guide

Evaluation of the Plant Protection Program (PPP)

In accordance with the CFIA 2013 Evaluation Plan, the Agency is currently evaluating its Plant Protection Program (PPP). As one of four subprograms operating under the CFIA's Plant Resources Program, the PPP attempts to mitigate the risks associated with plant pests and diseases in Canada.

As per the Treasury Board Secretariat's (TBS) directive on the evaluation function (April 2009), the evaluation addresses the issues of relevance/rationale and performance (effectiveness, efficiency, and economy). More specifically, the evaluation will look at the:

  • continued need for the program;
  • alignment of the program with government priorities;
  • alignment of the program with federal roles and responsibilities; and
  • achievement of outcomes.

The CFIA, Evaluation Directorate (ED) has hired PRA Inc., an independent research firm, to conduct this evaluation.

As part of the evaluation, PRA is conducting a series of key informant interviews with representatives of the CFIA. These interviews are meant to gather information directly from senior CFIA managers and PPP representatives. As one such individual, we appreciate you taking the time to participate in an interview.

When reading the interview questions below, you will note that most refer to the PPP generally. However, please feel free to answer based on your own experience with one of the program's four areas. For your information, these include:

  • Forestry;
  • Grains and field crops;
  • Horticulture; and
  • Potatoes.

Thus, if you are most familiar with the forestry area, rather than the PPP as a whole, please feel free to speak directly to forestry.

In addition, there may be questions that you cannot address. If this is the case, please simply say so when the question is asked. The interviewer will continue on to the next question. For your information, and to help you prepare to the interview, the PPP logic model is included with this interview guide.

As you already know, participation in the evaluation interviews is voluntary. Responses provided will be administered in accordance with the Privacy Act and other applicable privacy laws. In addition, all reporting will be written to provide aggregate results only, and no comments will be linked back to you, individually.

If at any time you have questions regarding the evaluation or this interview process, please do not hesitate to contact either the ED or PRA.

Introduction
  • 1. To help us better understand your answers, could you please introduce yourself and describe your work related to the PPP?
Continued need for the program
  • 2. A key element in most evaluations involves understanding whether there is a continued need for programming. From your perspective, are the activities undertaken as part of the PPP still important to Canadians? Please feel free to refer to the attached logic model for a listing of these activities.
  • 3. What needs do you see the activities of the PPP addressing? Can you point to specific evidence of these needs? Can you provide examples of how specific activities undertaken as part of the PPP have attempted to address these needs?
Alignment with government priorities
  • 4. Are there ways in which the PPP could better support the CFIA's strategic outcome?
  • 5. Other than the Agency's strategic outcome, are there other federal priorities to which the PPP and its activities contribute? How do the activities of the CFIA's partner departments, agencies, and organizations affect these contributions?
  • 6. From your perspective, is the CFIA's governance approach effectively supporting the PPP and its established goals? Does it allow the program to identify opportunities for improvement and act on these opportunities in a timely manner?
  • 7. Are the internal planning and reporting processes undertaken as part of the PPP sufficient for the CFIA management to oversee the program adequately and meet accountability requirements? Are there aspects that can be improved?
Alignment with federal roles and responsibilities
  • 8. Are there activities undertaken as part of the PPP that do not align with federal or Agency responsibilities?
  • 9. From your perspective, are all activities taking place under the PPP best situated within the CFIA? That is, are there other federal departments, orders of government, or non-governmental organizations that are better placed to undertake some of the activities under the PPP?
  • 10. Is the division of plant protection responsibilities between federal departments appropriate? Does this division support effective plant protection work?
Achievement of expected outcomes
  • 11. What evidence can you point to that demonstrates the level of achievement of PPP outcomes? Is this evidence readily available to the public or within the Agency?
  • 12. Are there challenges with the availability of specific inputs to PPP program delivery that affect the achievement of these outcomes? Is the distribution of PPP resources across program activities the most appropriate to ensure plant safety?
  • 13. Are there specific challenges to program delivery that affect the successful achievement of these outcomes?
  • 14. Is it possible for you to assess the degree to which factors outside the control of the CFIA affect the program's outcomes? Please be specific about the factors and their influence.
  • 15. Are there either positive or negative outcomes of the PPP that you have observed that have been unexpected?
Demonstration of efficiency and economy
  • 16. In your experience, has the PPP used its available resources effectively? How did you come to this conclusion?
  • 17. Is there an alternative approach to delivering some or all PPP activities that would make more efficient use of the program's resources? Are there alternatives to these activities that would achieve the same results?
Conclusion
  • 18. Do you have any other comments about the PPP? This can relate to any of the items discussed above.

Thank you

Appendix F – PPP expenditures by activity category

The following table provides a breakdown of PPP expenditures by activity category. The categories for which expenditures are tracked was revised in 2011–12 to reflect changes made to the CFIA's PAA.

Table 33: PPP expenditures by activity category
Activity category 2009–10
($ million)
2010–11
($ million)
2011–12
($ million)
2012–13
($ million)
7711 Plant protection $81.6 $65.1
7714 Plant protection education, awareness, and outreach programs $0.5 $0.6
7715 plant protection emergency response $24.4 $18.8
7643 Export certification activities – plant $7.9 $8.9
6211 Plant protection – regulatory and policy analysis and development $1.3 $1.1
6212 Plant protection – science advice $6.8 $6.9
6213 Plant protection – communication and stakeholder engagement $2.4 $2.9
6214 Plant protection – program design, advice and training $11.4 $12.0
6215 Plant protection – inspection/surveillance $50.7 $43.7
6216 Plant protection – laboratory diagnostics/services $10.0 $10.8
6217 Plant protection – contingency and preparedness $0.2 $0.1
6217 Plant protection – internal management $3.5 $7.0
6418 ICTA – export certification – plant protection $7.2 $7.7
6441 Plant protection – international engagement and standard-setting $2.2 $2.7
Total $114.3 $93.5 $95.8 $95.0

Note: Totals may not sum due to rounding.

Source: CFIA, Corporate Management Branch

Date modified: