Language selection

Search

Archived - Evaluation of the Bovine Spongiform Encephalopathy (BSE) Management Program

This page has been archived

Information identified as archived is provided for reference, research or record-keeping purposes. It is not subject to the Government of Canada Web Standards and has not been altered or updated since it was archived. Please contact us to request a format other than those available.

Phase 1: Canadian Food Inspection Agency, Evaluation Directorate
(Sections 3-5 plus appendices; Sections 1-2 are combined)

Phase 2: Health Canada/Public Health Agency, Evaluation Directorate
(Sections 6-9; Sections 1-2 combined)

September 2014

Tabled at Evaluation Committee June 10, 2014
Approved by the President September 25, 2014

Contents

Tables and Figures

Table 1: Funding for Continuation of Work on Core BSE Activities (2009-10 to 2013-14)
Table 2: Annual CFIA Planned and Actual Spending (2009-10 to 2013-14), per Activity
Table 3: Distribution of interviewees across stakeholder groups
Table 4: Scale to quantify interview response
Table 5: Overview of evaluation challenges, limitations and mitigation strategies
Table 6: BSE Surveillance numbers by province from August 2011 to July 31, 2012
Table 7: BSE Planned and Actual Spending 2009-10 to 2012-13, $ million, per Activity
Table 8: Planned FTEs, Actual FTEs per Activity as of April 2012, and Projected FTEs required to deliver the BSE Program based on 2012-13 Data
Table 9: Annual Health Canada Planned and Actual Spending (2009-10 to 2013-14), per Activity
Table 10: Annual Public Health Agency of Canada Planned and Actual Spending (2009-10 to 2013-14), per Activity
Table 11: Key Informant Interviews
Table 12: Challenges, limitations and mitigation strategies
Table 13: Health Canada File/Product Assessment Activities 2009-10 to 2012-13
Table 14: BSE/TSE Research and related outputs supported by Health Canada
Table 15: Summary of reported scientific and technical outputs from Health Canada/Health Canada funded studies
Table 16: Definite and probable cases of CJD in Canada, by subtype (partial 2013 data)
Table 17: Incidence of CJD deaths in Canada, by year of death (partial 2013 data)
Table 18: Autopsies on Suspected Cases of CJD by the Public Health Agency of Canada
Table 19: Summary of scientific and technical outputs from the Public Health Agency of Canada
Table 20: Reference services outputs from Public Health Agency of Canada
Table 21: Data Collection Matrix (DCM)
Table 22: Health Canada and Public Health Agency of Canada Data Collection Matrix

Figure 1: BSE CFIA Governance Structure
Figure 2: BSE Management Program Logic Model – CFIA-led Activities
Figure 3: Perceived adequacy of information (training, guidance, support, etc.) offered to field staff by the CFIA in the last five years
Figure 4: BSE Management Program Logic Model – HC/PHAC-led activities
Figure 5: Notifications, Referrals and Confirmed CJD cases in Canada over time
Figure 6: Graph on usage of CAHSN BSE module from January 2011 to October 2012
Figure 7: Planned vs Actual Spending on BSE in Health Canada

Abbreviations

AAFC
Agriculture and Agri-Food Canada
AHBLC
Animal Health Business Line Committee
ATQ
Agri-Traçabilité Québec
BC
British Columbia
BSE
Bovine spongiform encephalopathy
CAHSN
Canadian Animal Health Surveillance Network
CBSA
Canada Border Services Agency
CCIA
Canadian Cattle Identification Agency
CFIA
Canadian Food Inspection Agency
DFATD
Department of Foreign Affairs, Trade and Development (formerly DFAIT)
DPR
Departmental Performance Report
EAC
Evaluation Advisory Committee
EFB
Enhanced Feed Ban
EU
European Union
FTE
Full-time equivalent
FY
Fiscal year
GoC
Government of Canada
HC
Health Canada
MAS
Market Access Secretariat
MoP
Manual of Procedures
MoU
Memorandum of Understanding
OGDs
Other government departments (federal)
OIE
World Organization for Animal Health
PHAC
Public Health Agency of Canada
RCMP
Royal Canadian Mounted Police
SRM
Specified risk material
TB
Treasury Board
TSE
Transmissible spongiform encephalopathy
UK
United Kingdom
US
United States
vCJD
Variant Creutzfeldt-Jakob disease

1.0 Executive Summary

Introduction

Bovine spongiform encephalopathy (BSE) is a progressive, fatal disease of the nervous system of cattle. BSE has been a reportable disease in Canada since 1990 and BSE surveillance was implemented in 1992.

The current comprehensive BSE Management Program is led by the Canadian Food Inspection Agency (CFIA). The CFIA is responsible for six activities that fall within three thematic areas:

  1. Managing BSE-Related Risks
    • Removal of specified risk material (SRM) from the human food supply
    • Import controls
  2. Monitoring BSE-Related Risks
    • BSE surveillance
    • Cattle identification
  3. Communicating Canada's Program
    • Export certification
    • Technical market access support regarding BSE controls

Other federal partners in the comprehensive BSE Management Program include Health Canada (HC) for research and risk assessment activities and the Public Health Agency of Canada (PHAC) for surveillance and targeted supporting research. Agriculture and Agri-Food Canada (AAFC) has also been involved in supporting, stabilizing and repositioning Canada's beef and cattle industry.

Between 2009-10 and 2013-14, the Government of Canada has allocated $228.5 million in BSE programming to the CFIA, HC and PHAC. The CFIA received a total of $193.5 million to fund the six CFIA activities over this period, and HC and PHAC received $31 and $4 million, respectively, for their activities. HC and PHAC activities were included in the scope of a separate evaluation, as outlined below.

Evaluation

The evaluation of the BSE Management Program was identified as a priority in the CFIA's Evaluation Plan (2012, 2013). The evaluation covers the period from fiscal year 2009-10 to 2012-13 and focuses on CFIA's BSE Program activities. The evaluation does not include the Enhanced Feed Ban, as this was evaluated in 2011.

An interdepartmental summative evaluation of the horizontal BSE Program with HC and PHAC led by CFIA was planned to be conducted in 2012-13; however, discussions with HC and PHAC resulted in a plan for a two phased approach, included in this report in separate sections. Phase I sections address the six CFIA led activities. Phase II was led by HC and PHAC covering the remaining two activities over fiscal years 2009-10 to 2012-13.

Evaluation Findings

(Findings related to activities of the CFIA result from evidence presented in Section 4 and findings related to the activities of HC/PHAC result from evidence presented in Section 8)

Relevance

There is a clear ongoing need for the BSE Management Program. The program is consistent with government-wide priorities and the CFIA, HC and PHAC mandates. The unique nature of BSE—in particular its long incubation period and the sporadic emergence of atypical strains of the disease—poses potential long-term risks. Accordingly, the range of activities within this Program are necessary to protect human and animal health from potential BSE-related risks, as well as to support the Canadian beef and cattle industry in light of the ongoing and significant economic and trade impacts of this disease.

Performance

All evidence to date indicates that BSE-related risks to the health of the Canadian population and the Canadian cattle herd have been monitored and controlled. The BSE Management Program has clearly contributed to restoring, maintaining and expanding access to international markets.

Limitations in the available performance measurement data and financial tracking have restricted the CFIA's and HC's ability to monitor the Program's implementation and outcomes, and to make informed adjustments to the Program. These issues limit the ability of CFIA, HC and PHAC to demonstrate the Program's effectiveness and efficiency.

Collaboration and complementary efforts of partners and stakeholders continue to be vital to the effectiveness and efficiency of the Program across all of its activities. However, as fewer new BSE cases are detected and the Program becomes more routine and integrated into other CFIA activities, there is a risk of an increased lack of awareness and concern of the hazards of BSE both within and outside the CFIA. The risk is not tied to a particular BSE activity, but reflects early evidence of lack of concern and understanding of BSE risks. If left unchecked, this indication of complacency could represent a threat to the future success of the Program, as the long incubation period of the disease and ongoing potential risks means BSE management is a long-term commitment. As noted above, this evaluation does not include the Enhanced Feed Ban Program, already evaluated in 2011. The Enhanced Feed Ban, which removes SRM from animal feed, is a key component in protecting public and animal health from BSE. The 2012 evaluation found no evidence of complacency related to that Program.

There were some limitations found in the sharing of project results among Program partners. Program partners would benefit from having regular and formalized opportunities to share project results and identify future priorities to optimize Program implementation. The CFIA has already begun addressing this issue, such as through the creation of the CanSurvBSE structure to support the national collaborative surveillance model and the consultative BSE Roadmap process.

The CFIA's in-house BSE expertise was found to have greatly contributed to the Program's effectiveness. In particular, the CFIA's strong position in BSE/TSE (transmissible spongiform encephalopathy)-related research (especially surveillance) has contributed to enhancing the credibility of the BSE Program on the international stage. Emerging scientific knowledge on BSE and other TSEs further justifies the need for ongoing efforts by all partners. Unfortunately, maintenance of this expertise across the CFIA is threatened by ongoing retirements/turnover. In addition, BSE succession planning presents unique difficulties considering the need to sustain a specialized skillset, commitment and vision over a long period, including at senior management levels.

In Health Canada, BSE/TSE risk assessment activities have been integrated with routine product assessment, tracking and tracing functions, and, generally, represent a small proportion of these functions. Health Canada continues to maintain a health risk assessment capacity to support CFIA's risk mitigation measures with regard to potential implications of BSE/TSE on human health. Nonetheless, Health Canada has consistently and increasingly underspent (from 5% to 39%) its funding allocation as this integration has occurred (i.e., the funds allocated exceeded what was required to manage the BSE Program) as a result of change management and limited ability to staff key positions. The Public Health Agency of Canada has consistently spent all resources allocated to the BSE Program. Research and related scientific activities in both Health Canada and the Public Health Agency of Canada have contributed to increased expertise and knowledge of BSE/TSE and associated risks. The 2011 changes to the Health Canada blood donation deferral policy provided an example of how research and surveillance activities have increased knowledge-based decision making, led to the development of scientifically informed policies, and provided an ongoing picture of disease risks faced by Canadians. Ongoing research efforts related to atypical BSE, Chronic Wasting Disease (CWD) and diagnostic methods for human and animal TSEs (e.g., secondary human to human transmission of vCJD) were seen as important to provide the information required to confirm or adjust the policy framework for BSE/TSE moving forward. While performance frameworks are in place, they have never been fully implemented in Health Canada and are not seen by Health Canada staff as fully relevant to current scientific and Program needs. Nonetheless, research activities undertaken by Health Canada have resulted in a significant addition to the scientific knowledge associated with BSE/TSE to support improved risk assessment and diagnosis capacity. It is important that the scientific output gathered to date be further analyzed, upon completion of the ongoing research projects, to support updates of guidance and risk management measures associated with BSE/TSE in food.

Evaluation Recommendations

As noted in this report, the integration of BSE activities with other Agency activities prevents financial data from fully reflecting allocations of BSE funds. However, this report does not recommend improved financial tracking because the Agency has recently developed a comprehensive costing model, positively reviewed by an independent auditing firm, which will be used for future costing and reporting.

Recommendation 1. The CFIA should improve its performance monitoring practices relating to the BSE Program.

Recommendation 2. The CFIA should develop and implement an internal and external engagement strategy sufficient to build awareness of the continued need for a BSE program, especially in the absence of recent BSE cases.

Recommendation 3. The CFIA should implement a succession strategy to ensure it maintains sufficient BSE expertise within the Agency for the management and delivery of BSE-related activities.

Recommendation 4. The CFIA should also ensure that employees responsible for implementing BSE activities have adequate training to fulfill their roles and responsibilities.

Recommendation 5 (HC/PHAC). In addition to the ongoing role in BSE/TSE risk assessment and research to inform policy and regulatory development, Health Canada and the Public Health Agency of Canada should work with CFIA to institute regular opportunities to exchange information on Program activities and results in order to inform future work and facilitate collaborative activities.

2.0 Introduction

Note: The Program components related to Health Canada and the Public Health Agency of Canada are located starting in Section 6 of this report, followed by their methodologies, findings and conclusions.

Refer to Appendix F (10.6) for a copy of the logic model covering BSE funded activities delivered by CFIA, HC and PHAC.

2.1 Program Overview

2.1.1 BSE Management Program Profile

Bovine spongiform encephalopathy (BSE) is a progressive, fatal disease of the nervous system of cattle, which is associated with the presence of an abnormal protein called a prion. There is no test to diagnose BSE in live animals, although a tentative diagnosis may be made based on clinical signs. Diagnosis can only be confirmed after the animal's death by microscopic examination of its brain. In humans, BSE is linked to a rare, fatal disease called variant Creutzfeldt-Jakob Disease (vCJD).

BSE has been a reportable disease in Canada since 1990 and active BSE surveillance was implemented in 1992. In 1993, BSE was found in a cow that had been imported from the United Kingdom (UK) in 1987. Canada's first case of BSE in a domestic animal was found in May 2003. Between May 2003 and 2011, a total of 18 cases of BSE were confirmed in the Canadian cattle herd. There have been no cases of vCJD linked to eating Canadian beef.

Through the horizontal BSE Program established in 2003-04, the Government of Canada augmented past BSE-related efforts and developed a suite of internationally recognized, science-based measures to:

The current comprehensive BSE Management Program is led by the Canadian Food Inspection Agency (CFIA). The CFIA is responsible for six activities in the three following thematic areas These are also described in more detail in Appendix A, as well as in Logic Model (Figure 2).

A. Managing BSE-Related Risks

B. Monitoring BSE-Related Risks

C. Communicating Canada's Program

Note that BSE activities related to the Enhanced Feed Ban (EFB), covering 2004-05 to June 2011, was recently evaluated and is therefore not being evaluated as part of this project. The purpose of the EFB is to accelerate Canada's progress in BSE management by preventing more than 99% of potential infectivity from entering the feed system and enhancing risk management of transmission of BSE in the cattle herd.

Other federal government partners in the comprehensive BSE Management Program include Health Canada (HC) and the Public Health Agency of Canada (PHAC). HC conducts research and risk assessments regarding human exposure to BSE and other transmissible spongiform encephalopathies (TSEs), whereas PHAC carries out surveillance and targeted supporting research. Agriculture and Agri-Food Canada (AAFC) has also been involved in supporting, stabilizing and repositioning Canada's beef and cattle industry, including by hosting the joint AAFC-CFIA Market Access Secretariat (MAS).

There are a number of stakeholders and beneficiaries of the BSE Program, including: Canadian beef and cattle farmers and related industries, federal government departments and agencies (CFIA, HC, PHAC, AAFC and Canada Border Services Agency) provincial governments, World Organization for Animal Health, and international governments and the Canadian public.

2.1.2 BSE Management Program Delivery and Governance

Multiple branches and programs within the CFIA are involved in the design, implementation and delivery of the BSE Management Program, including the Operations Branch, Policy and Programs Branch, and Science Branch, while some branches and programs were involved intermittently or as necessary, such as the Public Affairs Branch. Branches play specific roles and responsibilities in delivering the six CFIA-led activities included in the Program, and also work in collaboration in several cases (see Appendix A for more details).

Under the current structure, the BSE Program Advisory Committee reports to the BSE Steering Committee. The BSE Steering Committee reports to the Animal Health Business Line (AHBL). The AHBL makes recommendations (particularly in relation to activity prioritization) to the Policy and Program Management Committee.

For reference, please note the changes in the various bodies that have been involved in the governance of the BSE Management Program within the CFIA during the period under evaluation:

Figure 1: BSE CFIA Governance Structure. Description follows.
Description for Figure 1: BSE CFIA Governance Structure

This figure shows the Canadian Food Inspection Agency's governance structure for the BSE Program. There are five rectangles with rounded edges organized in a vertical row. Between the rectangles are arrows pointing up to the next rectangle in the row. The following text is written in the rectangles:

  • The words "BSE Program Advisory Committee" are written in the bottom rectangle
  • The rectangle second from the bottom has "BSE Steering Committee" written in it
  • The middle rectangle has the words "Animal Health Business Line" written in it
  • Second from the top is a rectangle that has "Policy and Program Management Committee" written in it
  • The top rectangle has "Senior Management Committee" written in it.

2.1.3 BSE Program Funding

In total, the Government of Canada has invested about $3.6 billion in BSE programming between 2003-04 and 2013-14. Of this amount, AAFC received approximately $2.9 billion (AAFC funding ended in 2008-09), the CFIA received approximately $634 million, HC received approximately $72 million, and PHAC received approximately $8 million.

The BSE funding allocated to the CFIA, HC and PHAC from 2009-10 to 2013-14 was for continuation of work on core BSE activities. The breakdown of the total funding amount ($228.5 million) across the three recipients is provided in Table 1.

Table 2 provides details on planned and actual spending for CFIA-led activities during the evaluation period (2009-10 to 2013-14). The CFIA received $38.7 million per year during this time. While the DPR states that: "The variance between planned spending and actual spending is related to the reallocation of resources to other agency priorities"Footnote 2, the Agency has since developed a comprehensive costing model which identified all relevant activities and the time spent on each. Using actual delivery results, the model suggests that expenditures were in fact in line with what was plannedFootnote 3.

Table 1: Funding for Continuation of Work on Core BSE Activities(2009-10 to 2013-14)
Department/Agency 2009-10 2010-11 2011-12 2012-13 Sub-total 2013-14 Total
CFIA $38.7M $38.7M $38.7M $38.7M $154.8M $38.7M $193.5M
HC $6.2M $6.2M $6.2M $6.2M $24.8M $6.2M $31.0M
PHAC $0.8M $0.8M $0.8M $0.8M $3.2M $0.8M $4.0M
Total $45.7M $45.7M $45.7M $45.7M $182.8M $45.7M $228.5M

Source: CFIA Departmental Performance Reports: 2009-10 to 2012-13.

Table 2: Annual CFIA Planned and Actual Spending (2009-10 to 2013-14), per Activity
(in $M) 2009-10 to 2013-14

Planned (annual)Table Note 1

2009-10

Actual

2010-11

Actual

2011-12

Actual

2012-13

Actual

2013-14

Actual

SRM Removal from human food supply 9.5 8.4 7 6.7 6.1 NA
Import Controls 0.3 0.6 0.4 0.2 0.5 NA
BSE Surveillance 15.4 18.2 18.8 15.4 13.8 NA
Cattle Identification 2.8 2.1 2.2 2.3 2.4 NA
Export Certification 5.7 3.7 3.4 3.3 2.7 NA
Technical Market Access Support 5 1.2 1.8 2 1.7 NA
Total 38.7 34.2 33.6 29.9 27.2 NA
Variance with planned 4.5 5.1 7.4 11.5 NA
Variance % 12% 13% 20% 30% NA

Table Notes

Table Note 1

Planned spending for Technical Market Access Support was reduced by $1.4 million in 2011-12 (only) to take into account amounts transferred to the DFATD to provide support to CFIA staff located at missions abroad.

Return to table note 1 referrer

Source: Financial data provided by Corporate Management Branch, CFIA; consistent with DPR reporting.

2.1.4 Logic Model

The following logic model (Figure 2) for the BSE Management Program was adapted from the updated BSE Program Logic Model, dated 2012. It excludes both HC and PHAC activities in order to present only the CFIA-led activities included in this section of the evaluation. HC and PHAC activities are included in the logic model in Section 6.

Figure 2: BSE Management Program Logic Model – CFIA-led Activities. Description follows.
Description for Figure 2: BSE Management Program Logic Model – CFIA-led Activities

This is a diagram that describes activities, outputs and outcomes of the BSE Management Program. On the left side of the diagram is a vertical colour coded legend with the following written in it from top to bottom:

  • "Thematic Areas" which are shaded red
  • "1. Activities/Strategies" which are shaded gray
  • "2. Activities" which are shaded light blue
  • "3. Outputs" which are shaded dark blue
  • "4. Immediate Outcomes" which are shaded light green
  • "5. Intermediate Outcomes" which are shaded dark green
  • "6. Long-Term Outcomes" which are shaded purple

The items in the legend are organized vertically in the logic model in the same order as the legend with "Thematic Areas" at the top and "6. Long-Term Outcomes" at the bottom.

Thematic Areas

The contents of the Thematic Areas section of the logic model are contained inside of a rectangle with a red edge. Inside of that rectangle are three rectangles that have been shaded red and are organized horizontally.

  • The left rectangle has "A. Managing BSE-Related Risks" written it
  • The middle rectangle has "B. Monitoring BSE-Related Risks" written it
  • The right rectangle has "C. Communicating Canada's Program" written it

1. Activities/Strategies

The Activities/Strategies section of the logic model is contained inside of a rectangle with a gray edge. Inside of that rectangle are six rectangles that have been shaded gray and are organized horizontally.

  • The left most rectangle says "Removal of specified risk material (SRM) from the human food supply" and has A1 written in the bottom left corner
  • The rectangle second from the left says "Import controls" and has A2 written in the bottom left corner
  • The rectangle third from the left says "BSE surveillance" and has B4 written in the bottom left corner
  • The rectangle third from the right says "Cattle Identification" and has B5 written in the bottom left corner
  • The rectangle second from the right says "Export Certification" and has C7 written in the bottom left corner
  • The right most rectangle says "Technical market access support regarding BSE controls" and has C8 written in the bottom left corner

2. Activities

The Activities section of the logic model is contained inside of a rectangle with a blue edge the Outputs section of the logic model is also inside of this rectangle. The Activities section has twenty-four rectangles shaded light blue organized horizontally under the rectangles A1, A2, B4, B5, C7, C8 from the Activities/Strategies section.

There are four rectangles under the A1 rectangle organized vertically; they have the following text from top to bottom:

  • Enforcement and verification of SRM regulations by industry
  • Verification of SRM removal in products (federally registered establishments)
  • Provide technical support to provinces for implementation of SRM requirements & verify SRM removal (non-federally reg. establishments)
  • Maintenance of existing inspection at federally registered establishments

There are three rectangles under the A2 rectangle organized vertically; they have the following text from top to bottom:

  • Maintaining & updating import policies
  • Consultation with Canadian industries & international trading partners
  • Conduct risk assessments to determine appropriate mitigation measures

There are eight rectangles under the B4 rectangle organized vertically; they have the following text from top to bottom:

  • BSE surveillance program training
  • Sample collection
  • Verify the eligibility for the sample reimbursement program
  • Lab testing
  • Compilation/analysis of data & data reporting
  • Collaboration with partners & stakeholders
  • Re-design of surveillance strategy
  • Mgmt of sample reimbursement program

There are two rectangles under the B5 rectangle organized vertically; they have the following text from top to bottom:

  • Training inspectors and veterinarians
  • Outreach and communications

There are three rectangles under the C7 rectangle organized vertically; they have the following text from top to bottom:

  • Verify export requirements are met & provide export certificates
  • Development & certification of quality assurance programs
  • Bilateral technical negotiations

There are four rectangles under the C8 rectangle organized vertically; they have the following text from top to bottom:

  • Conduct risk assessments
  • Review & revise domestic policies
  • Science advice to inform policy & decision-making
  • Technical veterinarians staffed for key posts abroad

3. Outputs

The Outputs section of the logic model is contained inside of the same rectangle with a blue edge as the Activities section. The Output section has eighteen rectangles shaded dark blue also organized horizontally under the rectangles A1, A2, B4, B5, C7, C8 from the Activities/Strategies section.

There are three rectangles under the A1 rectangle organized vertically; they have the following text from top to bottom:

  • Federally registered establishment compliant with SRM removal procedures
  • Technical support to provinces
  • Performance of four SRM removal-related CVS tasks

There are three rectangles under the A2 rectangle organized vertically; they have the following text from top to bottom:

  • Revised & updated import policies
  • Communication & consultation products
  • Risk assessments conducted & mitigation measures in place

There are two rectangles under the B4 rectangle organized vertically; they have the following text from top to bottom:

  • Communication & consultation products
  • Re-designed surveillance strategy based on consultation

There are three rectangles under the B5 rectangle organized vertically; they have the following text from top to bottom:

  • Inspection tools
  • Inspections, enforcement actions
  • Communications products & activities

There are three rectangles under the C7 rectangle organized vertically; they have the following text from top to bottom:

  • Export certificates
  • Quality assurance programs
  • Decreased number of technical BSE-related conditions related to export requirements that are above and beyond OIE recommendations for a controlled BSE risk country to be met by industry & certified by inspectors

There are four rectangles under the C8 rectangle organized vertically; they have the following text from top to bottom:

  • Risk assessments
  • Revised & updated domestic policies
  • Increased science-based advice
  • Six technical positions abroad (include veterinarians and others)

4. Immediate Outcomes

The Immediate Outcomes section of the logic model is contained inside of a rectangle with a green edge the Intermediate Outcomes section of the logic model is also inside of this rectangle. The Immediate Outcomes section has seven rectangles shaded light green organized horizontally under the rectangles A1, A2, B4, B5, C7, C8 from the Activities/Strategies section.

  • The rectangle under A1 has the words "Compliance of the hygienic removal of SRM at slaughter & processing facilities" written in it
  • The rectangle under A2 has the words "Canada's BSE import requirement are risk-based & meet international standards" written in it
  • The rectangle under B4 has the words "Efficient & accurate monitoring of the prevalence of BSE in Canada is tracked through surveillance" written in it
  • There are two Immediate Outcomes rectangles under B5. The top one says "Regulated parties & inspectors are aware of the requirements". The Bottom one says "International markets are aware of the program"
  • The rectangle under C7 has the words "Awareness by foreign countries of Canada's BSE status & risk mitigation measures" written in it
  • The rectangle under C8 has the words "Trading partners are confident that BSE Risk mitigation measures implemented in Canada assure the sanitary safety of cattle & cattle products" written in it

5. Intermediate Outcomes

The Intermediate Outcomes section of the logic model is contained inside of the same rectangle with a green edge as the Immediate Outcomes section. The Intermediate Outcomes section has three rectangles shaded dark green also organized horizontally under the rectangles A1, A2, B4, B5, C7, C8 from the Activities/Strategies section.

  • Under the A1, A2, B4 and partially under the B5 rectangles there is a rectangle with the words "BSE-related risks to the Canadian population is controlled" written in it
  • Partially under the B5 and C7 rectangles there is a rectangle with words "Traceability data is complete, accurate and up-to-date" written in it
  • Partially under the B7 rectangle and fully under the C8 rectangle there is a rectangle with words "Restore, maintain and expand access to international market" written in it

6. Long-term Outcomes

The Long-term Outcomes section of the logic model is contained inside of a rectangle with a purple edge. Inside of that rectangle are four rectangles that have been shaded purple and are organized vertically.

  • The top rectangle has this text inside of it "Protection from preventable health risks related to food safety and product safety or the transmission of animal diseases to humans"
  • The rectangle second from the top has this text inside of it "Safer food and health products"
  • The rectangle second from the bottom has this text inside of it "A sustainable animal resource base"
  • The bottom rectangle has this text inside of it "Maintain public/consumer confidence in the animal, production and food system"

2.2 Evaluation Objectives and Scope

2.2.1 Evaluation Objectives

In accordance with the Treasury Board Secretariat Policy on Evaluation (2009), and its supporting directive and standardFootnote 4, the evaluation assessed the BSE Management Program as per the following evaluation issues and provides recommendations for improvement:

These five evaluation issues were addressed through specific evaluation questions, which are presented in the Data Collection Matrix, Appendix B. The evaluation approach used multiple indicators and lines of evidence to address each of the evaluation questions.

2.2.2 Evaluation Scope

The evaluation of the BSE Management Program was identified as a priority in the CFIA's Evaluation Plan (2012, 2013). This is the second evaluation of the CFIA's BSE programming and covers activities conducted since the last evaluation (completed in 2008-09Footnote 5). More specifically, this evaluation covers the period from fiscal year 2009-10 to 2012-13. Note that this evaluation also served to follow-up on the implementation of the recommendations from 2009 evaluation (see Appendix F). This is also the second evaluation of BSE activities for HC and PHAC. The last evaluation covering activities under BSE I & II was completed in 2013. The current evaluation covers activities carried out between fiscal years 2009-10 and 2012-13.

An interdepartmental summative evaluation of the horizontal BSE Program with HC and PHAC led by CFIA was planned to be conducted in 2012-13; however, discussions with HC and PHAC have resulted in the current evaluation being conducted in two phases with separate reports, both of which are included in the present report.

To support Phase I and Phase II of the evaluation and horizontal integration of the BSE function, CFIA, HC and PHAC maintained ongoing communication throughout the evaluation. A representative of PHAC/HC was a member of the CFIA's evaluation Advisory Committee, which provided oversight for the evaluation. In addition, working documents were shared between the CFIA, HC and PHAC.

Phase I: CFIA Thematic Areas: This first stage of the evaluation assessed the six activities of the BSE Management Program under the purview of the CFIA (see Section 2.1.1).

Phase II: HC and PHAC Thematic Areas: The second stage of the evaluation assessed the two areas of the BSE Program under the purview of HC and PHAC. HC and PHAC components of the report start at Section 6.

3.0 CFIA Methodology

The evaluation was carried out in three stages from February to January 2013.

  1. Stage I (March 2013 to June 2013): Development of a detailed evaluation framework, including specific evaluation questions, indicators, and corresponding data sources and data collection methods.
  2. Stage II (July 2013 to November 2013): Fieldwork to gather the necessary information and data.
  3. Stage III (December 2013 to January 2014): Analysis and integration of the collected information and data to synthesize and report evaluation findings.

An Evaluation Advisory Committee (EAC) provided input and feedback on the evaluation framework, preliminary findings, and on the draft evaluation report. Any clarifications and revisions necessary in light of the comments received were made accordingly.

3.1 Methods Overview

3.1.1 Document and File Review

This method involved a comprehensive review of approximately 250 internal and external documents and files to gain a solid understanding of the BSE Management Program's implementation and delivery. This included background information on the history and objectives of the Program, as well as those that could provide insight into any important shifts/changes that may have occurred during the life of the program.

Reviewed documents included (but were not limited to) the following:

3.1.2 Stakeholder Interviews

A total of 67 telephone interviews were conducted with internal (41) and external (26) stakeholders to obtain views and input on various aspects of the BSE Management Program to inform the evaluation of both relevance and performance issues. The interview guides are provided in Appendix C. A breakdown of the interviews by stakeholder group is shown below.

Table 3: Distribution of interviewees across stakeholder groups
Stakeholder group # Interviews Planned # of Interviews
Internal
Senior Management 2 3
Policy and Programs Branch 18 25-30
Science Branch 5 25-30
Operations Branch 7 25-30
Inspection Managers 9 25-30
Internal – Total 41 28-33
External
Other Government Departments (OGDs: Health Canada, PHAC, AAFC) 4 5-6
Provincial government representatives 8 8-9
Industry associations and company representatives 13 8-10
International Government/Agency representatives 0 3
Academic and scientific BSE experts (national, international) 1 3-4
External – Total 26 27-32
Grand Total 67 55-65

Internal Stakeholders: An original list of potential interviewees within relevant branches, programs and regions was compiled; this list was complemented by suggestions for additional and back-up interviewees from CFIA staff. The sampling maximized the inclusion of key CFIA staff involved in the design, implementation and delivery of the BSE Program. The interviewees included senior CFIA management representatives involved in the design, implementation and delivery of the EFB; managers from Operations, Policy and Programs, and Science Branches; and inspection managers from across the country.

External Stakeholders: The draft list of potential external stakeholder interviewees was based on the main groups of external stakeholders identified in the design. This list was complemented by suggestions for additional and back-up interviewees from CFIA staff and external representatives. The interviewees included representatives from other government departments (OGDs), provincial government representatives from across the country, industry association and company representatives, and academic and scientific BSE experts. As explained in the Challenges, Limitations and Mitigation Strategies section below (Section 3.3), fewer external stakeholders were interviewed than planned, particularly among company representatives and BSE experts.

3.1.3 Survey

An online survey was conducted to consult CFIA regional and field staff involved in relevant BSE activities. The questions in the survey related to the delivery of planned activities, training, data collection and communication relating to their BSE work. The survey questionnaire is provided in Appendix D.

A list of CFIA regional and field staff (inspectors and veterinarians) involved in BSE activities was compiled using the compliance verification system. Of the 208 respondents identified and invited, 122 completed the survey (response rate of 58.7%, margin of error of ±5.7%).

3.1.4 Case Study

A case study on BSE performance measurement mapped out how and by whom performance data is collected and complied, how the data is used and identified, and provides insight on how these processes can be better aligned with operational and strategic needs.

The case study involved three main collection methods:

  1. An in-depth review of relevant documents;
  2. 10-12 targeted interviews with CFIA staff (separate from stakeholder interviews); and
  3. Questions specific to performance measurement data were added to the survey questionnaire.

3.2 Analysis and Integration of Data

Evaluation findings were analyzed and integrated across multiple lines of evidence. Information and data was validated within each method (i.e., corroborating the responses of multiple interviewees and multiple stakeholder groups), as well as between methods (i.e., corroborating interviewee responses with evidence from the document and file review as well as from the survey results). The preliminary evaluation findings were presented to the EAC and other key CFIA staff to identify factual errors and further confirm the validity of results and conclusions. This feedback was addressed and taken into account in drafting this report.

Although the interviews were not designed or used to collect quantitative data, the interpretation of findings should take into account the extent to which certain perceptions or views were expressed or shared by interviewees. Table 4 defines terms (e.g., "almost all,", "most," etc.) that are used in this report to quantify the proportion of interviewees or the proportion of groups of interviewees (e.g., internal stakeholders, industry or association representatives) who expressed similar experiences, views and opinions.

Table 4: Scale to quantify interview response
Term Proportion of interviews
"Almost all" or "consensus" Findings reflect the experiences, views and opinions of more than 85% of the interviewees.
"Most" Findings reflect the experiences, views and opinions of more than 50% of the interviewees.
"Many" Findings reflect the experiences, views and opinions of more than 25% but no more than 50% of the interviewees.
"Some" Findings reflect the experiences, views and opinions of more than 10% but no more than 25% of the interviewees.
"A few" Findings reflect the experiences, views and opinions of less than 10% of the interviewees.

3.3 CFIA Challenges, Limitations and Mitigation Strategies

The evaluation challenges and limitations and the corresponding mitigation strategies are described in Table 5.

Table 5: Overview of evaluation challenges, limitations and mitigation strategies
Challenge and limitation Mitigation strategy Implications/Impact on Evaluation
Due to a lack of clear separation of structure and activities between the BSE Management Program and the Enhanced Feed Ban (EFB), it was often challenging to attribute outcomes or issues to the BSE Management Program (particularly with regard to SRM Removal from the Human Food Supply) vs. the EFB. Similarly, interviewees at times did not make a distinction between the EFB and other related programming (e.g., SRM removal) when discussing the relevance and performance of the broader BSE Management Program. This was particularly problematic for external interviewees, who were less familiar with the distinction between individual components of the broader program. In cases where this affected the interpretation of evidence in documents, files and data, multiple analysts have reviewed and sought to eliminate possible overlap/confusion with the EFB. Clarifications were made/sought during interviews to obtain interviewee opinions, feedback and input only on the BSE Management Program—excluding the EFB—to the extent possible. When interpreting data, EFB-specific issues were excluded from the analysis whenever judged irrelevant to the scope of the present evaluation; CFIA staff and Evaluation Advisory Committee members also provided useful feedback regarding the relevance of EFB activities. In some instances, the impacts of this limitation on the evaluation findings were somewhat significant despite mitigation efforts. In particular, it affected the ability of the evaluation to assess the extent to which some BSE Program activities, outputs and outcomes had increased as a result of the Program (i.e., delivery and performance data, see below). These instances were noted and highlighted in the evaluation report, wherever applicable.
For the purpose of this evaluation, available quantitative (performance and financial) data on BSE Management Program-specific activities and outcomes often had significant limitations:
  • Many of the quantitative indicators in the Data Collection Matrix for this evaluation were based on the indicators presented in the Enhanced BSE Programming's Performance Measurement Framework (December 2012). However, during the fieldwork for this evaluation, data were not collected or readily available for several of these indicators;
  • Data in CFIA systems or sources were not always BSE-specific and there are known issues with tracking BSE financial data. In some cases, BSE-specific data could not be readily compiled in a useful format without manual validation by expert analysts, which fell outside the scope of this evaluation; and
  • Certain CFIA information systems had specific limitations (e.g., available data was fragmented, stored in multiple locations, not cross-linked across systems), which affected data consistency across regions and/or over time.
From project initiation, repeated efforts were made to identify and validate reliable quantitative data with CFIA representatives. Some data were ultimately obtained in the later stages of the fieldwork that addressed some of these limitations. The impacts of this limitation on the evaluation findings are fairly significant in terms of conclusively determining the effectiveness and efficiency of several BSE activities. Despite mitigation efforts, the impacts of this limitation on the evaluation findings were significant in several instances. Overall, the lack of valid, reliable and consistent quantitative data limited the ability of the evaluation to conclusively determine the effectiveness and efficiency of most of the BSE Management Program activities. For example, it was not possible to quantitatively assess and confirm that BSE activities and requirements are being implemented effectively (e.g., inspections, compliance and enforcement), or to track BSE Program funded resources and FTEs over time.

The impact of this limitation on the evaluation findings relating to effectiveness and efficiency are discussed in more detail in sections 4.5.1, 4.5.2 and 4.5.3.

Since CFIA staff and in some cases interviewees provided the names of the internal stakeholders interviewed for this evaluation, this might have resulted in a possible selection/sampling bias (i.e., a systematic error due to a non-random sample of a population, causing some members of the population to be less likely to be included than others and resulting in a sample in which all population members are not equally balanced or objectively represented). The impacts of a potential selection/sampling bias were minimized by framing interview questions and prompts in a manner that encouraged interviewees to provide verifiable examples/supporting documents in relation to their answers, wherever applicable. This is also mitigated by triangulating multiple lines of evidence (e.g., review of internal and external documents to identify and assess any concerns or opinions that might not have been reported/shared by internal stakeholders). Overall, the mitigation measures were effective in minimizing the impacts of a potential selection/sampling bias. Nevertheless, some caution should be used in generalizing the findings attributed to internal stakeholders in this report across all relevant CFIA branches/programs/regions.
Scheduling of external stakeholders became a challenge during the fieldwork phase, as many international and company external stakeholders who were originally suggested as primary or back-up interviewees declined to participate or did not—or, in the case of US representatives, could not, as they were affected by the government shutdown—respond to invitations.

Overall, the original target number for specific stakeholder groups was not reached as fewer international stakeholders and industry representatives were interviewed than originally planned. In particular, interviews with beef and cattle companies (which were intended to replace a survey of industry stakeholders) did not attain intended numbers (2-3 interviews short).

Additional external stakeholder interviews were conducted to attempt to reach the targets. Further, some of the external stakeholders who declined to participate also suggested others who could be interviewed within their respective organizations.

Despite this, certain groups are under-represented in the external interviewees, notably beef and cattle company stakeholders. While fewer company representatives were interviewed than originally planned, additional interviews with CFIA field managers and industry association representatives took place and the survey of CFIA field staff was expanded to include further open ended commentary on their interface with industry.

Meanwhile, the reduced international perspective was mitigated by lengthy and supplementary interviews with CFIA veterinarians abroad, as well as with a representative from a US industry association.

Overall, these measures ensured that the combined lines of evidence for this evaluation provide sufficient evidence from the external perspective.

Ultimately, although the number of external stakeholders interviewed did not reach the expected total, especially for international stakeholders and for beef and cattle company representatives, the addition of interviewees including field staff, industry association representatives as well as longer than planned interviews with CFIA vets abroad mitigated some of the evaluation challenges.

Nonetheless, caution was and should be used in interpreting and generalizing the findings attributed to external stakeholders in this report.

4.0 CFIA Key Findings

4.1 Relevance: Continued need for program

4.1.1 Question 1: Is there continued need for the BSE Management Program, both overall and for each program component?

Given the long-term nature of potential BSE-related risks, the risks to human and animal health, as well as economic and trade impacts, there is a continued need for the BSE Management Program to protect animal as well as public health, and to support the Canadian cattle industry. This need applies to the BSE Management overall, as well as to its individual components.

Overall, all lines of evidence confirmed the ongoing need for the BSE Management Program, in whole and for each of its individual components. The unique nature of BSE, in particular its long incubation period, has wide ranging implications in terms of the ongoing potential risks posed by the disease to:

Moreover, new insights on atypical strains of BSE, which are believed to arise spontaneouslyFootnote 8, also have implications in terms of the ongoing need to manage and monitor BSE risks over the long term.

The full suite of BSE Management Program measures—along with the Enhanced Feed Ban (EFB)—has been designed to address and mitigate these risks in concert with each individual component of the programming. This Program design serves one or more specific and complementary purposes relating to public and animal health, and to supporting domestic and international market benefits (see below). Importantly, corroborating the documentary evidence, the majority of internal and external interviewees considered that the Program should continue to contribute to protecting Canadians from preventable health risks. The Program does this by eliminating the primary route of potential public health exposure by requiring the removal of SRM from beef for food purposes (as of 2003Footnote 9). The interviewees also felt the Program should continue to help sustain a healthy and sustainable animal resource base, and thus contribute to the livelihood of Canadian farmers (i.e., when they are able to sell and trade livestock and cattle products).

One indicator of the Program's success is the reduction in the number of reported cases of BSE in the Canadian cattle herd, i.e., 3 cases detected post-2009 vs. 14 in the 2004-08 period. However, it is the birth date of BSE positive animals that is the best indicator of infectivity. Based on this indicator, no BSE case has been detected in Canadian animals born after 2004, which suggests the situation has improved since that timeFootnote 10. However, given the long incubation period, surveillance data continues to be collected to assess the incidence of BSE in cattle born after 2004, especially in cohorts born after the introduction of the EFB in 2007 (i.e., to assess the effectiveness of the EFB at preventing the transmission of BSE).

The World Organization for Animal Health (OIE) officially recognizes disease-free areas of countries for trade purposes, including for BSEFootnote 11. While OIE standards are not legally mandatory, many countries, including Canada, recognize and apply them as minimum standards when determining import policies and controls. Consequently, adherence to OIE standards helps secure and maintain access to most export marketsFootnote 12. The OIE identifies a country's status as "controlled", "negligible" or "undertermined" for BSE. Based on an overall assessment of risk and annual reports—including information relating to feed ban measures, analysis of surveillance data, and BSE case response—since 2007, Canada has maintained a controlled BSE risk statusFootnote 13.

A country may apply for "negligible risk status" if there has been no indigenous case of BSE born within the last 11 years. The most recently affected birth cohort currently known was 2004, making Canada eligible to apply for "negligible BSE risk" status as of 2015. The OIE would render a decision in 2016, as long as no new BSE case is detected in an animal born after August 2004. Several factors could potentially affect this outcome, including possible identification of a BSE case in cattle born after 2004Footnote 14. Failure to comply provides grounds for the OIE to revoke the given statusFootnote 15. As such, activities to manage and monitor BSE risks are necessary to maintain Canada's current controlled risk status from the OIE and to eventually qualify for and maintain negligible risk status, with associated benefits in terms of continued and expanded access to export markets.

The confirmation of indigenous BSE cases in Canada in 2003 led to a closure of borders for Canadian beef in many international markets. Many markets have since re-opened in part or in full (including as a result of the BSE Management Program, as elaborated in Question 7). Given ongoing risks relating to BSE for market access, the BSE Management Program therefore continues to be an essential part of this country's economic and trade interests.

Economic impacts of closed markets for beef products are very significant for a sector as large as the beef sector. The discovery of BSE in Alberta in 2003 shut the border to these sales to the U.S. and approximately 40 other countries. The U.S. accounted for 80% of Canadian beef exports and almost 100% of cattle exportsFootnote 16. An industry-led analysis concludes the beef industry generated just below 15% of farm cash income during the period under evaluationFootnote 17, and is responsible for $33 billion worth of sales of goods and services, contributing $13 billion to the country's gross domestic product (GDP)Footnote 18. Canada is also one of the largest exporters of red meat and livestock in the world, with exports representing up to 50% of its production in 2012. Exports to the US market alone generate $1.8 billion in total sales for Canada's beef sectorFootnote 19.

Evidence supporting the continued need for each individual component of the BSE Management Program is discussed below.

Many interviewees stressed that demonstrably science-based decisions need to be clearly at the centre of the Program. The CFIA uses risk assessments and other science-based evidence (including that produced by others) to inform decisions as to BSE-related regulations, policies and procedures. In addition, the CFIA itself conducts research on TSEs and uses the internationally recognized tests to confirm positive BSE results (OIE Reference Laboratory). Available evidence also indicates that the Program is aligned with several internationally recognized practices/standards for BSE/TSE.

Interviews and documents indicate that the CFIA continues to participate in BSE science at the international level, by contributing knowledge and expertise, and by providing science-based advice to international organizations (e.g., OIE, the World Health Organization), and other countriesFootnote 28. For example, the CFIA contributes to international science-based disease control standards, which are critical components of the foundation for safe trade and the maintenance of international market confidence.

4.1.2 Question 2: To what extent does the BSE Management Program respond to the needs, priorities, and mandates of its major stakeholders?

The Program responds to a great extent to the needs, priorities, and mandates of its major stakeholders, namely Canadian consumers, the Canadian beef and cattle industry, as well as international trading partners and organizations.

Within Canada, the beef sector involves a wide array of diverse stakeholder groups including those outside industry (e.g., governments, academics, NGOs). Consumers are considered to be the ultimate stakeholders in this systemFootnote 29. The BSE Management Program is designed to safeguard the beef-consuming Canadian public from BSE-related risks, while inspiring confidence in the human food supply, as widely confirmed by interviews and documents.

Both industry documents and interviews with industry representatives confirm there is an overall appreciation of the BSE Management Program by the Canadian beef and cattle industry. The CFIA is seen to directly support consumer confidence and market access, which are the main needs expressed by industry. Note that OGDs are directly involved in efforts to support market access needs, most notably AAFC and DFATD. For example, one of the main beef industry stakeholders, the Canadian Cattlemen's Association, recognizes that the Government of Canada as a whole has engaged in a number of activities to support market access for Canadian beef (e.g., via the Agricultural Market Access Secretariat housed within the AAFC, in which CFIA is involved), but states that there were insufficient human resources to carry out all the work necessary across the federal governmentFootnote 30.

Moreover, the level of effort and costs associated with the Program (e.g., for SRM removal – also relates to the EFB; surveillance sampling; cattle identification) is an ongoing concern for some industry stakeholders. As such, there is some tension between, on one hand, the high expectations of industry placed on the effectiveness of the CFIA's BSE-related programing to help maintain market access and to further open markets and, on the other hand, their desire to minimize their efforts and costs relating to monitoring and managing BSE. Despite this tension, the BSE Management Program clearly works closely with and is generally appreciated by a range of industry partners.

Relating to the needs of international organizations and trading partners, the CFIA communicates its BSE programming and controls through its technical support for market access activities. In some cases, these efforts respond directly to conditions of trade agreements or reporting commitments with international bodies. The CFIA hosts visits from trading partners to confirm first-hand the integrity and implementation of Canada's inspection controls, as well as participating in technical missions abroad. For example, a 2010 EU audit carried out in Canada concluded that measures to protect public health and animal health (including with regard to BSE) under a Canada-EU Agreement supporting the trade in live animals and animal products were implemented as per the Agreement, including the roles and responsibilities specific to the CFIAFootnote 31. The CFIA also provides analysis and reporting to maintain Canada's official designation as a BSE controlled risk country through the evaluation process of the OIE.

4.2 Relevance: Alignment with government priorities

4.2.1 Question 3: Does the BSE Management Program continue to be consistent with government-wide priorities and the CFIA mandate?

The Program continues to be consistent with both government-wide priorities and the CFIA mandate, particularly relating to protecting human and animal health, and by enabling an economically vibrant industry.

Both the document review and interviews confirm that the BSE Management Program is consistent with both federal government priorities and the CFIA mandate. A review of the Government of Canada's Budgets and Speeches from the Throne indicate that a competitive agricultural sector and protecting the health and safety of Canadians (including through safe food) have remained among the main priorities of the federal government throughout the period under evaluationFootnote 32, Footnote 33. In 2011 and 2012, these sources also emphasized the Government's support to the access of Canadian agricultural products to foreign marketsFootnote 34, Footnote 35.

There was also broad consensus among internal and external interviewees that the Program continues to be consistent with the CFIA mandate relating to protecting human health, animal health, and supporting a strong agricultural industry. In particular, the Health of Animals ActFootnote 36 provides the CFIA authority to carry out activities in support of the prevention of the introduction of animal diseases into Canada, and to control and eliminate diseases that either affect human health or could have a significant economic effect on the Canadian livestock industry. Overall, the fact that BSE deals with both food safety and animal health (as well as having international import and export implications) underlies why the Program aligns so closely with the CFIA's mandate.

Finally, the BSE Program fits within the CFIA's current Program Alignment Architecture mainly under Program Activity 1.2 - Animal Health and Zoonotics Program, specifically under Sub-Program 1.2.1 - Terrestrial Animal HealthFootnote 37. It also relates to Program Activity 1.1 – Food Safety Program (Sub-Program 1.1.1 – Meat & Poultry) and Program Activity 1.4 – International Collaboration and Technical Agreements. Overall, the Program contributes to the CFIA's strategic outcome: "A safe and accessible food supply and plant and animal resource base".

4.3 Relevance: Alignment with federal government roles and responsibilities

4.3.1 Question 4: Is there a legitimate and necessary role for the federal government in the BSE Management Program?

As confirmed in the legislation, the federal government's role in the Program is legitimate and necessary for animal health (e.g., BSE is a federally reportable disease) and public health (e.g., responsibilities for food safety in federally regulated establishments). The federal government also has unique roles and responsibilities for international relations, borders and trade.

The legitimacy of the federal government and the CFIA as lead in the Program is confirmed by multiple lines of evidence. First, the legislative framework for the BSE Management Program stems from several federal acts such as the Health of Animals ActFootnote 38, Food and Drugs ActFootnote 39, and the Meat Inspection ActFootnote 40, and their associated regulations. These outline the role of the federal government relating to BSE monitoring and control, including inspection of SRM removal from meat intended as food (i.e. public health), as well as responsibilities relating to animal health. Importantly, approximately 95% of animals slaughtered in Canada are slaughtered in federally regulated establishments (including for meat transported between provinces or exported out of the country), which are inspected by the CFIAFootnote 41, Footnote 42.

Second, CFIA was established as the lead for the federal government response to BSEFootnote 43. Other key partners (including response to TSEs more broadly) are HC, the PHAC and AAFC; the Canada Border Services Agency (CBSA) also assists the CFIA with border clearance responsibilities. Notably, as per the Reportable Disease Regulations, BSE is a federally reportable disease for terrestrial animals in Canada; as such, the CFIA is responsible for BSE surveillance on behalf of the federal government. As noted above, the legislative framework indicates that CFIA is directly responsible for ensuring inspection in federally registered establishments and develops appropriate import controls (including policies specific to BSEFootnote 44), as well as for ensuring export requirements are met and issuing export certification in response to international trade agreementsFootnote 45. Relating to this last point, Canada has an agreement with the EU on "sanitary measures to protect public health and animal health in respect of trade in live animals and animal products", which outlines roles and responsibilities for the CFIA relating to the supervision and enforcement of BSE measures to support trade with EU countriesFootnote 46.

Both internal and external stakeholders further confirmed the federal government's role outlined in the legislative framework. In particular, almost all internal interviewees associated the legitimacy of the federal role with the CFIA's mandate to enhance animal health through the control and eradication of BSE—a federally reportable disease. Importantly, they also recognize the need for ongoing provincial efforts around this disease, given the provincial roles and responsibilities relating to food safety and animal health. Although only a few external interviewees provided views on this question, they indicated that the federal government needs to play a prominent role when dealing with emerging human health issues such as BSE, as well as with international trade issues.

4.3.2 Question 5: Are the roles and responsibilities of the CFIA, other federal and provincial governments departments involved in the BSE Management Program, and the regulated industries clear and well-understood by internal and external stakeholders?

The roles and responsibilities were generally clear and well-understood by internal and external stakeholders during the evaluation period, although changes in responsibilities (e.g., transfer of inspection tasks to some provinces) are sometimes associated with a lack of clarity. These roles and activities are highly complementary to those of provinces, industry and OGDs.

Almost all of the interviewees within the CFIA and many outside the CFIA indicated that the overall understanding of CFIA's roles and responsibilities relating to the BSE Management Program by internal and external stakeholders was relatively high, especially given that this is now a long-standing program (i.e., established in 2003-04). However, some respondents did identify some weak points relating to the understanding of BSE-related roles and responsibilities.

In particular, while OGDs and other key external delivery partners clearly recognize and understand the CFIA's role as the federal lead for the horizontal BSE Program at the organizational level, there remains some confusion about specific points of contact or responsible groups/individuals for BSE-related issues or activities. This was related to the fact that, following the urgent need to implement BSE response measures brought on by the initial BSE cases in Canada, the CFIA originally created ad hoc structures dedicated to the management of this disease. Subsequently, BSE measures have been increasingly integrated within other animal health and meat inspection activities and this has made it more difficult for external parties to identify groups or individuals responsible or accountable for BSE-related issues.

The CFIA's coordination of BSE-related roles and activities with provincial counterparts varies given the very different context and capacities of individual provinces, especially with regard to provincial inspection for SRM removal and BSE surveillance sample practices. Even in this context, the roles and responsibilities of the CFIA were found to be quite well-understood by provincial representatives. In particular, both the document review and interviews indicated that there continues to be close coordination and collaboration across the federal/provincial TSE laboratory network. Meanwhile, interview and survey data indicate that industry stakeholders are generally savvy about the CFIA's roles and responsibilities but their understanding of, reactions to, and cooperation or involvement with the BSE-related activities are not homogenous.

One emerging issue identified during the fieldwork relates to the current transfer of inspection responsibilities for provincial establishments (to be completed by the end of 2013) from the CFIA to some provinces – namely British Columbia, Saskatchewan and ManitobaFootnote 47. Up until now, CFIA inspectors provided meat inspection services to BC, SK and MB. All other provinces remain responsible for their own inspections. The ongoing transfer of this responsibility to the three western provinces is currently causing some confusion among those involved. More specifically, negotiations for this transfer were long and complex, and there were some difficulties noted among interviewees within and outside the CFIA in understanding and/or adjusting to the new roles and responsibilities of various parties for the inspection and enforcement of SRM removal. In contrast, there is less confusion in provinces where SRM removal responsibilities are well-established and not experiencing the same level of change, such as Quebec and Alberta.

Overall, a high level of complementarity of the Agency's roles and activities—rather than duplication—with those of provinces, industry and OGDs was observed by CFIA interviewees. External interviewees generally understood the roles and responsibilities of the CFIA and of other federal and provincial instances involved in the BSE Management Program, although many had limited detailed knowledge of specific activities. Almost all external stakeholders mentioned the existence of programs run by federal or provincial departments that complement (not duplicate) the work done by CFIA, especially in the activities related to surveillance. Another notable area of complementarity is for cattle identification, for which key services and databases are administered and coordinated by industry/provincial organizations (e.g., livestock traceability information is managed by Canadian Cattle Identification Agency and Agri-Traçabilité Québec, which are recognized as an administrator and an organization that manages the animal identification system respectively as per the Health of Animals RegulationsFootnote 48).

4.4 Performance: Achievement of expected outcomes

4.4.1 Question 6: To what extent have planned activities for each component of the BSE Management Program actually been implemented by the CFIA and produced the intended outputs?

CFIA has continued to implement planned activities across all six components of the BSE Management Program. Based on available (and often limited) data, the level of delivery for specific activities appears to have met targets but more often under-delivered; planned activities have therefore only produced the intended outputs to a moderate extent. Planned activities that were under-delivered include:

Whenever possible, findings on program implementation (i.e., planned vs. delivered activities and outputs) are based on multiple sources of data, such as database extractions, files and reports, as well as interviews and the survey of field staff. The two main databases used are:

These databases or other documentary sources containing data on the implementation of the BSE Management Program are often limited in terms of providing complete or fully reliable measures of outputs and in terms of the ability to use the information to assess whether activities were delivered as planned. In particular:

CFIA's CMB Branch led a recent exercise to identify resource requirements for the BSE Program, assessing all BSE-related inspection tasks, from those that are only 1% related to BSE to those that are 100% BSE. The results of this exercise were used for this Evaluation (as seen in Question 13). Data on the number, distribution and results of surveillance samples come from a number of internal sources (e.g., CFIA website, materials prepared for BSE committees or CanSurvBSE working group), while the annual Departmental Performance Report (DPR) also contains useful delivery and output indicatorsFootnote 49. Survey respondents were also asked to indicate whether activities relating to each of the main components of the BSE Management Program were delivered as planned (including frequency, where, how, etc.).

Evidence on the implementation and outputs for each individual component of the BSE Management Program is discussed below.

Removal of SRM from the Human Food Supply: Documentary data (from DPR, CVS, and MRRS; some caveats apply) indicate that that the frequency of the delivery of some tasks for this activity have decreased over the evaluation periodFootnote 50. The same documentation, however, cannot confirm whether the level of planned delivery of verification and enforcement tasks for this activity also decreased in parallel over the evaluation period, as historical projection of planned activity data is overwritten each year to reflect the actual end of year data. Nonetheless, most internal interviewees and field staff (80%) reported that SRM removal activities remain aligned with planned implementation, especially in federally regulated slaughter and processing establishments from where 95% of the beef slaughtered in Canada originates.

Given the high degree of variability in how inspection and enforcement of SRM removal from human food is conducted across provinces in provincially regulated facilities, there was more limited data on the delivery in these establishments, including on audits conducted by the CFIA (or collected by CFIA) of provincial inspections for SRM removal which prevents a proper monitoring of compliance and possibly issues with enforcement. A few CFIA respondents expressed a concern around the low level of inspection in provincially regulated facilities. Although these facilities only slaughter about 5% of Canadian animals (for local consumption only), animals brought to these facilities may represent a higher-risk population for BSEFootnote 51. As noted previously, there is also some confusion around the CFIA's specific role and responsibilities relating to SRM removal in provincially regulated facilities, especially given the current transition in three provinces (BC, SK and MB), where inspections are currently being transferred to provincial staff.

A 2010 EU audit of Canadian meat sanitary controls (mentioned previously, not exclusive to BSE) concluded that, in general, the control system relating to the production of fresh meat, farmed game meat, minced meat, meat preparations and casings is functioning in the manner specified in the Agreement between the European Community and CanadaFootnote 52. More specifically, training and resources for these activities were reviewed and, in general, the organization of control systems and verifications was considered to have been applied adequately. Some specific deviations from the agreed standards were observed, and recommendations seek to address shortcomings for non-cattle products (i.e., pig, horse, or bison). Of the 7 recommendations, 3 were for other products than beef and the other 4 were more general for meat; 2 were related to export controls and 2 were related specifically to SRM removal inspection and enforcement (CVS/CARs), which specified that SRM removal should be strengthened for enforcement in the case of non-complianceFootnote 53.

Similarly, a recent CFIA internal audit of the Corrective Action Requests Management Process concluded that improvements were needed in several areas (e.g., documentation/record-keeping to provide information for decision-making, risk-based approach to monitoring and oversight) to strengthen the CFIA's enforcement processFootnote 54. While not specific to BSE, these improvements would also strengthen enforcement in cases of non-compliance with SRM removal.

Import Controls: As stated in the annual Departmental Performance Reports (DPRs), import control policies have been reviewed and updated as planned by the Policy and Programs Branch. However, interviews with relevant CFIA staff and survey data suggest that the implementation of import policies and associated inspection control tasks in the field (i.e., by Operations staff) are lacking clarity. To note, these activities are not specific to BSE and many are conducted on-demand, such that assessing planned delivery rates for import control tasks is not generally meaningful.

The survey of CFIA inspectors and veterinarians revealed that about 30% of field staff considered that import control activities related to BSE were not clearly defined or planned, whereas 43% indicated that they were. In addition, only 40% of survey respondents considered that their delivery of import control tasks (i.e., enforcement and verification) related to BSE was conducted as expected, whereas 37% indicated that these tasks were not delivered at all or to a limited extent. It is also important to note that no BSE funding is allocated to Operations Branch for these activities (funds allocated only to PPB).

BSE Surveillance: Overall, Canada's BSE surveillance infrastructure is well developed. Document and interview data converged to show that these activities (sampling and testing) have generally been conducted as planned by the Science Branch, particularly in the testing of samples (which is directly related to the number of samples submitted), as well as analysis and reporting of surveillance data. Evaluation evidence also indicates that the Policy and Programs Branch planning of surveillance activities has been conducted as expected for both sample collection (e.g., determining the minimal sample size, establishing annual contracts with the dead stock collection industry as per the reimbursement program) and for training and quality control activities.

However, the survey suggests possible under-delivery of BSE surveillance activities in the field (i.e., by Operations staff), with less than two thirds (63%) of respondents indicating that they delivered related tasks as planned. MRRS data (some caveats) suggests about 80% of planned tasks relating to BSE surveillance were conducted by field staff. Similar to issues noted under inspection of SRM removal from the human food supply, surveillance sampling processes vary across provinces (e.g., number of samples; locations sampled; collection sometimes but not always conducted by CFIA staffFootnote 55; coverage of provincially regulated establishments). Some interviewees therefore suggested that surveillance practices could be revisited and adjusted according to a collaborative surveillance model (i.e., with industry and provincial actors).

As will be discussed in more detail under Question 7, the number of submitted and tested samples annually fell below the CFIA's established minimum of 30,000 samples (a target set in 2004) in 2012. Dropping below this target puts the credibility and effectiveness of Canada's BSE surveillance system at risk. This trend has been in large part attributed to the decrease in industry participation (particularly in the Prairie provinces), presumably due to termination of some provinces' reimbursement program and an associated lack of understanding of the need to maintain the level of effort for surveillance. To reverse this trend (and to address a variety of surveillance issues such as those noted in the previous point), CFIA has been implementing a national collaborative surveillance initiative, called CanSurvBSE, to involve and raise awareness among industry and provincial stakeholders. To date, the number of samples submitted in 2013 suggests that the target will be achieved for this year.

The National BSE Surveillance Reimbursement Program (SRP) remains activeFootnote 56. Whereas provincial reimbursement programs vary across Canada, the federal SRP assists producers and veterinarians when services are provided to the CFIA to cover a portion of the veterinary examination fees and carcass disposal costs. According to the DPRs, each year, in compliance with the national reimbursement program eligibility criteria, payments approximately equivalent to the number of samples are provided to eligible partners of the program including cattle producers, private veterinarians and dead stock collection operators.

In the case of a positive BSE case (of which there were three during the evaluation period, one each in 2009, 2010, and 2011), the CFIA conducts an investigation, as per its BSE Manual of Procedures, requiring an "epidemiological investigation based on the OIE Terrestrial Animal Health Code for a country with controlled risk status, comprised of the trace out of all cattle reared with the BSE case during its first year of life (feed cohort) as well as all cattle born in the same herd as, and within 12 months of the birth of the BSE case (birth cohort)."Footnote 57, Footnote 58

Note that the Science Branch also conducts research, risk assessments, and other science-based activities to support both BSE surveillance (e.g., coordination in the Canadian Animal Health Surveillance Network (CAHSN), which is partially funded via HC) and technical market access; the outputs of these activities are not captured systematically and were not measured although they were mentioned by interviewees.

Cattle Identification: Cattle identification (i.e., presence of tags) inspections relating to the BSE Program have been largely—but not completely—conducted as planned in the last five years according to both MRRS (75%) and survey data (75%). Interview and document evidence shows that there was a delay in the implementation of inspection tasks related to cattle identification in CVS (to be fully rolled out, including associated training, in 2014), including the broader traceability component (i.e., beyond compliance with tagging). Some internal interviews also noted some issues with uniformity across the country. In fact, this component is being developed into a comprehensive livestock identification and traceability program (including a regulatory framework and data system, see Question 7) that includes other species and that can be applied to investigations of other animal diseases beyond BSE. Recent reports show significant progress for the livestock and traceability program in the last yearFootnote 59.

Export Certification: While technical negotiations with trading partners and participation in foreign audits supported successful opening of markets, CFIA interviewees identified difficulties in measuring the outputs of these activities. Almost 60% of veterinarians in the field survey report that export certification activities have been delivered as planned (in terms of frequency, where how, etc.). Considering that these activities are conducted on-demand, it is unclear how to interpret this finding without further enquiry. To note, MRRS activities for export certification are not BSE-specific (1% to 75% linked to BSE) and could not be used to assess the level of delivery.

Technical Market Access Support: The consensus among internal stakeholders is that BSE has gone from initial crisis mode to being an integral part of Canada's communications strategy with trading partners. Almost all reported that the six veterinarians posted abroad provided a physical presence that acts as a regular reminder to foreign government officials and facilitates real-time communication with trading partners on Canada's BSE programming and so contributes to the participation of the Meat Program and Science Branch experts in trade missions and general provision of technical expertise.

4.4.2 Question 7: To what extent have implemented activities for each component of the BSE Management Program produced the expected outcomes (immediate, intermediate, long-term, and the CFIA's strategic outcomes)?

Overall, the BSE Management Program has been successful at producing the expected outcomes, as a whole and for each of the program components. Some weaknesses were identified that result from delivery/output issues such as the adequacy of the implementation of import policies, decline in samples submitted for BSE testing, and minor animal identification tagging issues.

Evidence on the outcomes for each individual component of the BSE Management Program is discussed below. Because this suite of measures works in concert (along with the EFB and activities led by OGDs), they are all currently contributing to the ultimate outcomes of the BSE Program:

Indeed, both the detailed evidence below and the general views of interviewees indicate that the CFIA's BSE Program has been effectively contributing to all three of these outcomes. Frequently cited examples include the lack of BSE-related human health incidents due to consumption of Canadian beef, the demonstration of a relatively low level of infectivity in the Canadian cattle herd (and—also related to the EFB, the absence of BSE cases detected in animals born after 2007), and the continued confidence of consumers and trading partners in Canadian beef and beef products (i.e., domestic sales and expanded international market access). Both internal and external interviewees also often cited the key role of the CFIA's BSE Program in maintaining Canada's controlled BSE risk status from the OIE, as well as in contributing to Canada's eventual eligibility to apply for negligible BSE risk status. It should be stressed here that, given the long incubation period for BSE and ongoing potential risk posed by BSE to each of these outcomes, these achievements do not diminish the need for the continuation of the BSE Program (see Question 1).

Immediate outcome – Managing Risks (by outcome): Verifying the hygienic removal of SRM at slaughter and processing facilities

Overall, this outcome was largely achieved, particularly in federally registered establishments. Indeed, verification of slaughter and processing facilities indicates a high level of compliance for the removal of SRM from human food. More specifically, since 2008, a compliance rate of at least 99% has been achieved each year at federally regulated establishments across SRM removal tasksFootnote 60. Evidence from 41 Corrective Action Requests and follow-up by CFIA staff relating to relevant SRM removal tasks during the evaluation period (up to May 2013) indicates that these have since all been closedFootnote 61.

As noted under Question 6, concerns were observed in both internal documents (including records from BSE management committees) and internal interviews regarding the level of inspection and compliance at provincially regulated facilities given limited data available on these verifications, on the variable processes and capacity between provinces, as well as in the context of the current transition to provincial inspection in some provinces.

Immediate outcome – Managing Risks (by outcome): Canada's BSE import requirements are risk-based and meet international standards

Overall, this outcome was largely achieved through the development of import requirements relating to BSE, but some questions remain around the adequacy of implementation of these policies.

Documents and interviews confirm that BSE import policies are updated based on risk assessments and follow the OIE recommendations. These revisions also seek to harmonize BSE requirements across North AmericaFootnote 62. Each year (2009-10 to 2011-12), the CFIA has reviewed and updated for precision and clarity, as required, at least 25% of the procedural imports policies, by commodity type. The "umbrella" BSE import policy was implemented in 2010Footnote 63.

Other tasks to implement import controls that are conducted by the CFIA include the review of import requests by industry or foreign governments (including risk assessments, when appropriate)Footnote 64. On-site foreign audits may also be conducted to assess the veterinary infrastructure, as well as certification and control measures in place for new or major established trading partners. However, these on-site audits are not frequent due to the associated costs and none were reported in recent years. This limits Canada's ability to verify that import control policies are followed by critical trading partners.

The issuance and assessment of import permits is delivered by CFIA and CBSA field staffFootnote 65. Although some interview data and the survey (see Question 6) both suggest that implementation of some import policies has been delayed and/or that inspection and issuance of permits for imported animals and goods as part of BSE control has been unclear and possibly insufficient, the evidence collected for this evaluation does not allow for an adequate assessment of whether this increases the risk of BSE entering Canada from foreign sourcesFootnote 66. Import-related field tasks cover a wide range of animal health issues besides BSE and no BSE-specific funding is earmarked for them. While there is no evidence to suggest that import controls have not been effective (i.e., there is no evidence of BSE or other reportable animal diseases that have entered into Canada via specified regulated pathways from imported animals, products or by-products between 2009-10 and 2011-12), they are important to ensure that import policies are effective at preventing BSE infectivity from entering Canada from foreign sources.

Immediate outcome – Monitoring Risks: Efficient and accurate monitoring of the prevalence of BSE in Canada through surveillance

Overall, the CFIA's BSE surveillance efforts have met this outcome, despite the gradual drop in the number of samples submitted for BSE testing. Enhanced communication and coordination efforts led by the CFIA, including the CanSurvBSE structure, appear to be reversing this trend.

The associated infrastructure is well developed (including the Canadian Animal Health Surveillance NetworkFootnote 67), leverages the work of partners (including provincial laboratories), and has contributed to ensuring the continuation of Canada's controlled BSE risk status from the OIE. The BSE surveillance program continued to test cattle samples with an emphasis placed on cattle falling under the 4D category (dead, down, dying and diseased); this aligns with OIE guidelines on priority subpopulations identified for surveillance purposesFootnote 68.

However, there has been a gradual drop in sample numbers submitted and tested over the evaluation period, from about 35,000 in 2009, 2010 and 2011 to 27,000 samples in 2012; note that these numbers peaked between 2005 and 2007 at over 55,000 samples per yearFootnote 69. In particular, Prairie provinces dropped below targets in 2012 (Table 6); this is associated with the end of some provincial reimbursement programs, although the national SRP remains activeFootnote 70. Evidence from both the internal interviews and document review indicated that, should this trend not be reversed, there could be a potential negative impact on the continued effectiveness and credibility of the surveillance. Some issues were also noted regarding the quality of data on these samples (e.g., year of birth), which is necessary to conduct analyses (e.g., epidemiological studies for animal and public health) and to monitor accurately BSE prevalence and infectivity.

Increased communication and the new CanSurvBSE structure have been implemented to raise awareness and increase collaboration in order to increase the number of samples submitted and more generally to help strengthen the collaborative surveillance programFootnote 71. Interviewees indicated that the CanSurvBSE structure is expected to facilitate some BSE activities and communication with other stakeholders; as such, most interviewees clearly supported its continuation. A total of about 24,000 samples were collected between January and September 2013; at this rate, the 30,000 sample target will be reached in 2013Footnote 72.

Table 6: BSE Surveillance numbers by province from August 2011 to July 31, 2012
Province Adult Cattle
(January 2012)
% of national adult herd Surveillance target # tested % of target
British Columbia 266,500 5.1% 1,533 2,728 178%
Alberta 1,750,000 33.6% 10,070 5,446 54%
Saskatchewan 1,323,000 25.4% 7,613 2,068 27%
Manitoba 542,800 10.4% 3,123 2,605 83%
Ontario 638,900 12.3% 3,676 8,059 219%
Quebec 581,000 11.1% 3,343 4,958 148%
Atlantic provinces 111,500 2.1% 642 1,124 175%
Total 5,213,700 100% 30,000 26,988 90%

Source: CFIA. (2012, November). BSE Surveillance in Canada and other BSE affected countries.

Intermediate outcomes – Managing Risks: BSE-related risks to the Canadian population are controlled; BSE-related risks to the Canadian cattle population are controlled

All evidence to date indicates that these outcomes have been met, although ongoing surveillance evidence is necessary to confirm this finding given the long-term nature of BSE.

With regard to public health, most interviewees considered that SRM removal from the human food supply and import controls have effectively contributed to the control of BSE risk and supported continued consumer confidence by helping demonstrate that BSE poses a minimal risk for human health. Most notably, there have been no cases of vCJD in Canada associated with the consumption of Canadian beef (the only Canadian case, in 2002, was related to the person's multiple stays in the UK during the BSE outbreak)Footnote 73, Footnote 74. However, due to the long onset of the disease, a future occurrence in Canada cannot be ruled out at this time.

With regard to animal health, the BSE surveillance program has continued to provide reliable evidence that BSE-related risks to the Canadian cattle population are controlled (i.e., that BSE measures, including the EFB, are effective). For example, BSE surveillance activities between 2004 and 2010 have provided insight into the evolution and distribution of BSE in Canada: the collection of samples has permitted the estimation of prevalence by year of birth (a more meaningful measure of the effectiveness of controls than crude prevalence estimates), which confirms that the prevalence of BSE is very low within any particular birth cohortFootnote 75. As noted above, this outcome is contingent upon maintaining an adequate surveillance program for BSE.

Immediate outcomes – Monitoring Risks: Regulated parties and inspectors are aware of the requirements (cattle ID/traceability); International markets are aware of the program

Overall, considerable progress was noted towards the achievement of these outcomes, especially in the second half of the evaluation because of advancements in the development of the broader livestock identification and traceability program. Ongoing support for this program has been observed from industry and government, who recognize its value in helping control and eliminate serious animal diseases—including but not limited to BSE. In fact, both documents and interviewees identified the importance of continued cooperation and support (including financial) as being critical to the future development of the full traceability program (see below, intermediate outcome).

Awareness of tagging requirements is relatively high: compliance for cattle identification as verified by CFIA inspection staff between 2009-10 and 2011-12 has ranged between 97% and 99% for federally-inspected abattoirs, as well as at auctions, dead stock collectors, farms and livestock operations, feedlots, provincial abattoirs, renderers and tag distributorsFootnote 76. Moreover, training is being gradually rolled out for inspectors as livestock identification and traceability tasks have been integrated into CVS. In 2012-13, 193 inspectors completed either the online or classroom courses for this program, and more are to be trainedFootnote 77.

However, some ongoing minor tagging issues were noted by interviewees and survey respondents, and CFIA is well-positioned to further improve awareness of the broader livestock identification and traceability requirements. The field survey reported that 65% considered that regulated parties were aware of cattle identification requirements and 65% also felt that CFIA's communication efforts were effective at promoting compliance with these requirements. Overall, the cattle identification and traceability activity as a whole is expected to improve as the broader program is further developed.

In addition, between 2011-12 and 2012-13, the CFIA carried out several communication and outreach activities under the traceability program to increase awareness of foreign trade partners about these measures (e.g., presentations to foreign delegations from Taiwan, Russia and Chile)Footnote 78. Little information was available on the extent to which foreign markets are actually aware of these requirements, but a 2010 EU audit of Canadian sanitary controls for meat noted progress on the traceability front. Most notably, progress was noted with regard to the registration of cattle farms, which is now compulsory in most of CanadaFootnote 79.

Intermediate outcome – Monitoring Risks: Traceability data is complete, accurate and up-to-date

Cattle identification is a pre-requisite for improving traceability data, which interviewees noted has posed a challenge in some prior BSE investigations. As noted above, traceability data will be critical to support efforts in managing multiple diseases, not only BSE. In fact, several other countries are implementing similar initiativesFootnote 80. Once fully implemented, Canada's traceability policy would likely be more advanced than in some other countries, including the US, Japan, EU, Uruguay and ArgentinaFootnote 81. Overall, recent progress has improved the available traceability data but further advancements are expected.

Since 2011, the CFIA and its partners have proceeded with the development of a comprehensive legislative and regulatory framework for implementing life-cycle traceability (i.e., from birth or import to death, slaughter or export), for a number of species (i.e., beginning with cattle, sheep, poultry and hogs, and extending to other livestock sectors)Footnote 82. This framework involves requirements across the three pillars of traceability: animal identification, premises identification, and movement reporting. The CFIA also has a national information management system (i.e., the Traceability National Information Portal), which provides authorized users with a single point of access to information about animal identification, location identification and movements of animals, as maintained in a number of provincial and industry-led databases. The necessary data sharing agreements to support this system have already been signed with four provinces (Alberta, Saskatchewan, Manitoba and Prince Edward Island) and discussions are ongoing with several othersFootnote 83. The CFIA has also completed various assessments of these provincial and industry-led databases to ensure the integrated system contains accurate and complete data.

Overall, these outcomes have been largely achieved through the verification of requirements for export and issuing of export certification, as well as through the provision of technical support in the context of trade negotiations and missions with foreign countries. Indeed, the latter are seen as instrumental to ensure awareness and confidence of trading partners in Canada's sanitary controls for beef and beef products relative to BSE. Note that awareness by foreign countries was not directly measured, so these findings are based on documents and the perceptions of Canadian interviewees.

A 2010 EU audit of Canadian meat sanitary controls (mentioned previously, not exclusive to BSE) concluded that controls for export certification for meat were adequate; however, some shortcomings were raised, notably that supervision or certification was not adequate to address the deficiencies notedFootnote 84. As such, the audit recommended the regular review of approval conditions for export to the EU, which the CFIA has committed to doing. Specific export requirements relating to BSE are also verified by field staff via tasks in CVS (Section 3), and the CFIA also facilitates foreign country audits and delegations coming to assess Canada's BSE control measures.

Almost all interviewees generally praised the work of the CFIA (and the federal government more broadly) in supporting the negotiation of and participation in bilateral agreements with international trading partners in order to maintain, regain and open new markets. The technical market access support (including the veterinarians abroad) was perceived by several internal stakeholders as highly effective and essential to communicate science-based decisions on BSE and other food safety issues. In addition, almost all internal and external stakeholders saw the CFIA as having been effective in representing Canada before the OIE. Moreover, documentary evidenceFootnote 85 and stakeholders confirmed that CFIA is also contributing to and influencing international science-based disease control standards for BSE (e.g., within the OIE), which are contributing factors to the development of measures for safe trade and to maintain the confidence of international markets.

That said, some criticisms from the international community of Canada's BSE measures were notedFootnote 86, mainly for the EFB, but also for "minor shortcomings" regarding the realignment of BSE surveillance, traceability, SRM removalFootnote 87. Achieving and communicating a sufficient level of ongoing inspection oversight (especially for the EFB), BSE surveillance and other BSE measures will help continue to offset these criticisms and demonstrate the effectiveness of Canada's BSE risk mitigation measures.

Intermediate outcome – Communicating Canada's Program: Restore, maintain and expand access to international markets

Given the number of markets that have been restored, in full or in part, or new markets expanded for Canadian beef and beef products, the BSE Management Program has clearly contributed to achieving this outcome. More specifically, a list provided by the CFIAFootnote 88 that stems from the annual Agriculture and Agri-Food Market Access ReportsFootnote 89 and internal input and documents (e.g., Monthly Market Access Successes reports from the Market Access Secretariat) indicates that between 2011 and 2013:

This list includes priority markets identified by the federal government as key trading partners such as the EU, US, Japan, and South Korea, as well as emerging markets such as China, Indonesia and Russia. These achievements are in addition to markets restored, maintained or expanded prior to 2011. As noted above, interviewees widely perceived this achievement to be a result of the CFIA's efforts (along with OGDs) since 2003 and, within the current evaluation period, of the communication of the BSE risk mitigation measures abroad.

However, not all markets are fully open and Canadian beef continues to face challenges in the international trade environment. Some interviewees also suggested that, despite Canada's efforts to promote an international regulatory framework to protect Canadian exporters from discriminatory and unnecessary barriers, BSE is still occasionally used as a trade barrier by foreign countries. Canada also continues to experience strong competition from the US, which has received negligible BSE risk status from the OIE in early 2013Footnote 90. The US also announced in November 2013 a final rule that aligns its import regulations for BSE with OIE standards (Canada's are already aligned with OIE standards), which is expected to increase their competitiveness in terms of accessing international marketsFootnote 91.

4.4.3 Question 8: Have there been any unintended (or unexpected) outcomes (positive and negative) of the BSE Management Program?

A positive unintended outcome of these activities is that Canada has become a more important international player with regard to prion diseases. The experience of responding to BSE has also helped drive a new model of emergency preparedness and improved awareness of risks in Canada, leading to better vigilance.

Negative unintended outcomes included the higher associated costs of stricter requirements. Canadian importers now face much more stringent requirements. In addition, industry stakeholders are generally dissatisfied with the high costs associated with sampling and disposal of SRM.

CFIA stakeholders indicated that the BSE Management Program has advanced the scientific understanding of BSE and other animal disease models, and transferred this knowledge to a range of partners within and outside Canada. A positive unintended outcome of these activities is that Canada has become a more important international player with regard to prion diseases. The experience of responding to BSE has also helped drive a new model of emergency preparedness and improved awareness of risks in Canada, leading to better vigilance (e.g., for vCJD).

The Program was also widely perceived to be instrumental in generating awareness and reinforcing collaboration between CFIA and industry stakeholders. For example, animal identification activities have created an enhanced sense of discipline among industry partners who have realized they can no longer produce anonymously. Similarly, in managing BSE, the Program deepened relationships with territories and provinces, as well as improved bilateral relations and more direct interactions with trading partners. Not only did the latter result in positive benefits for BSE-related outcomes (e.g., contributing to market access) but also helped interpret other food safety policies from international governments.

Negative unintended outcomes identified by interview participants included the higher associated costs of stricter requirements. Indeed, Canadian importers now face much more stringent requirements. In addition, industry stakeholders are generally dissatisfied with the high costs associated with sampling, disposal or SRM (also associated with the EFB). Moreover, animal identification and traceability has not produced the premium prices that were initially projected. A few interviewees also reported that, combined with these costs, the fall of prices caused by the initial case contributed to forcing smaller operations out of business.

4.4.4 Question 9: What are the major internal and external factors contributing to or constraining the performance of the BSE Management Program?

The most important contributing factors included expert staff and cross-organizational engagement and collaboration. Conversely, constraining factors include growing challenges at maintaining the necessary expertise to manage this disease, diminished vigilance (associated with a lack of awareness)—both internally and externally, as well as financial and performance tracking difficulties.

Internal contributing factors: The unique nature of BSE requires a critical mass of disease-specific expertise. As such, CFIA stakeholders identified the Agency's highly expert staff, who are world leaders in BSEFootnote 92, emergency management and disease control, as the principal internal contributing factor. In the future, however, BSE succession planning presents unique challenges considering the slow incubation period of this disease and the need to sustain critical expertise over longer periods. Indeed, ongoing and anticipated retirements are seen to be creating and widening internal gaps in expertise relevant to BSE.

Two other internal contributing factors identified were 1) the CFIA's ability to engage a large number of players across governments, industry and expert boundaries, both within and outside of Canada, as well as 2) its enhanced and proactive international relationship and communication channels for CFIA issues, that now go beyond BSE (see also Questions 8 and 11).

Internal constraining factors: While deficiencies in the understanding and awareness of the unique nature and implications for managing this disease were more commonly seen among external stakeholders (see below), these were also found within the CFIA. For example, there is a perception among some internal stakeholders that less vigilance can be applied to the BSE program as no new cases have been detected. This is not the case because of the long incubation period and ongoing potential risks means the BSE program is a long-term commitmentFootnote 93. Interviewees also indicated that federal budget reduction efforts that began in 2008-09 raised the possibility of cuts to the federal Sample Reimbursement Program (SRP), which industry and the provinces may have mistakenly taken as an indication that BSE surveillance was less critical than previously.

Almost all interviewees identified elements of complacency and the risk of it increasing with respect to BSE management – both within government and externally. As Canada moves further away from the date of our last BSE incident, it was noted that the risk of BSE recurring in Canada seems less real, and the need for vigilance less critical. However, it should be noted that some degree of complacency is not unexpected given that the BSE program is now routine and that many BSE activities are integrated into other CFIA activities. This passage of time, coupled with factors such as mounting budget constraints, varied appreciation of BSE risks, and mixed understanding of the importance of separate and combined BSE Program elements have led to an increased lack of awareness and concern. One CFIA stakeholder succinctly summarized this as follows: "...12 years ago before our first case of BSE, we were horrified by the term. Now we are very familiar with it and we do our measures and we tend to be almost nonchalant. It's very critical to think about how devastating this disease can be!"

It is important to reiterate here that this evaluation does not include the Enhanced Feed Ban Program, already evaluated in 2012. The Enhanced Feed Ban, which removes SRM from animal feed, is a key component in protecting public and animal health from BSE, and the 2012 evaluation found no evidence of complacency related to that Program.

Further, almost all CFIA stakeholders raised the issue that new inspection requirements are integrated into staff duties without funding or time being provided for these additional requirements. Meanwhile, budget pressures have generated questions about the need for the same level of attention and resources for the BSE Program delivery. These questions are more commonly raised in regions less historically impacted by BSE. As noted under Questions 12 and 13, this has indirectly resulted in financial tracking and performance measurement difficulties (e.g., to track program delivery) given the integration of BSE into A-base funding and broader inspection tasks; this has constrained the funding accountability and other management aspects of the Program.

Also related to BSE expertise and HR issues noted above, a few internal interviewees observed that turnover in upper management since 2003 has worked against the long-term commitment and vision needed to manage a disease like BSE. In addition, inspection and veterinarian staff turnover were seen to have created a growing competency gap (see also training data discussed in Question 13). Many CFIA stakeholders were also of the opinion that multi-disciplinary and cross-trained responsibilities are not properly trained for and/or resourced (e.g., animal health at border crossings).

Finally, particularly for the Science Branch, reductions in external funding available for CFIA-led BSE/TSE research and general budget constraints for internally funded research projects were found to be jeopardizing current levels of research and risk assessment capacity for BSE within the CFIA (i.e., lack of renewal or dilution of scientific expertise).

External contributing factors: External positive factors that facilitated the performance of the Program as identified by both internal and external stakeholders were generally associated with closer collaboration and coordination with stakeholders across OGDs, industry and government. Notable examples include enhanced data-sharing practices among provincial and partner laboratories (e.g., surveillance) and for traceability, as well as collaborative research practices (e.g., research networks). On a related point, participation in the Program of many government departments (e.g., DFAIT, CBSA, HC, AAFC, PHAC) was cited by many CFIA stakeholders as evidence of a more effective whole-of-government approach to address the challenge that is BSE. Finally, the opening of markets, largely dependent on trading partner governments but supported by the BSE Program, has contributed to the overall credibility of and goodwill towards the CFIA's BSE programming.

External constraining factors: Almost all CFIA stakeholders, as well as some industry stakeholders, noted an increased BSE "fatigue" among external stakeholders (e.g., producers, governments) associated with a lack of awareness about the nature of BSE. For example, as noted previously, reduced vigilance and cuts in provincial sample reimbursement programs were seen to result in a drop-off in surveillance sample numbers submitted by producers. This in turn threatens to prevent Canada's OIE status upgrade, which jeopardizes further market access.

In addition, many external interviewees considered that a major factor constraining the performance of the BSE Management Program is the budget cuts that have occurred at both federal and provincial levels. Though BSE funding has been fixed, successive rounds of funding reductions in overall programing indirectly affects the BSE Program given the cross-utilisation of resources. Also as noted above, this situation has affected the external funding available for research projects on BSE and related human health issues, as well as the ability of producers to remain competitive in an open market. Finally, some external stakeholders indicated that some countries are not compliant with OIE regulations relating to import controls, often implementing additional requirements which contribute to unpredictability for the Canadian beef and cattle sector. From a competitiveness perspective, the US's 2013 upgrade to negligible BSE risk status from the OIE and change in rules for import regulations (see Question 7) are likely to increase the competitiveness of the US in the international market, which poses further challenges for Canadian beef and cattle sector.

4.4.5 Question 10: Are there appropriate management and decision-making structures in place to meet the objectives of the BSE Management Program? Is the structure working as intended?

Internal management and decision-making structures have evolved and improved over the evaluation period to better support the Program, but further improvements are possible.

Various management and communication channels have been established both within the CFIA and with external stakeholders, for both general management of the BSE Management Program, as well as for specific needs, such as during the preparation of the BSE Roadmap, 2012 EFB review, and the CanSurvBSE structure (see also Question 11).

The following internal BSE-related governance and decision-making bodies were in place during the evaluation period and actively contributed to management and decision-making for the Program:

Indeed, as demonstrated by meeting minutes and/or records of decision, the two main BSE-specific committees have met approximately once a month between 2010 and 2013 to provide updates on activities, and to discuss tasks, issues and progress achieved. Since 2011, updates and discussion of BSE items have frequently been on the agenda of the Animal Health Business Line Committee. Some BSE issues, such as a strategy for future programming and renewal, were also discussed at the Senior Management Committee in May 2013. Both internal interviews and documentsFootnote 96 confirm that these structures, including changes made in response to previous evaluations, have helped clarify BSE issues, enhanced horizontal integration and promoted information exchange within the Agency.

However, these various committees appear to lack the authority to effectively address ongoing issues (e.g., financial tracking, performance measurement, surveillance awareness). Although these issues are frequently discussed and appear to be well-understood, only a few issues appear to have been brought to the Senior Management Committee and concrete measures and/or resolution of issues is more rarely observed (see, for example, the results of an in-depth review of Committee records for the follow-up on recommendations, Appendix E). Accountability is also a concern since BSE and other animal health activities have become so integrated, that it is difficult for any one person or committee to be pinpointed as the responsible expert or management representative for a given issue. Given this situation, there was a general consensus among internal interviewees that further modifications to these BSE-related structures could further improve decision-making and accountability (e.g., BSE oversight could be extended to include other groups besides the Animal Business Line). Moreover, a few interviewees and internal correspondence suggest that the CFIA is considering merging the Steering and Program Advisory Committees into one committee going forward.

4.4.6 Question 11: To what extent is collaboration and coordination with partner organizations and stakeholders adequately integrated in program design and delivery?

Overall, the CFIA's collaboration and coordination efforts with OGDs, provinces and industry have helped avoid duplication and ensure that activities are complementary. However, some challenges remain, suggesting an ongoing need for continued and enhanced coordination efforts.

The CFIA has engaged in intensive communication regarding its main BSE activities with relevant groups of stakeholders across the evaluation period. Examples of effective collaboration and coordination by the CFIA with partner organizations across each of the six BSE activities were cited especially for science and surveillance, which involves OGDs, provinces (e.g., lab network), or multiple stakeholders (e.g., CanSurvBSE, a multi-stakeholder coordination structure developed to support collaborative national surveillanceFootnote 97).

Specific to OGDs, several examples of partnerships involving the CFIA were found to have improved outcomes through enhanced collaboration and coordination:

The CFIA was also seen to collaborate fairly effectively with both OGD partners in the Comprehensive BSE Program, namely PHAC and HC. In particular, given that HC sets human health standards and conducts risk assessments relating to BSE, it has financed research projects as well as data collection to improve data collection and access via the Canadian Animal Health Surveillance Network (CAHSN)Footnote 100. However, rules and regulations around data sharing restrict some of the coordination with HC (as well as other federal and provincial government departments regarding traceability efforts). The CFIA has recently drafted an Information Sharing Agreement for CAHSN that allows sharing of test results, including BSE test results, between CFIA and its CAHSN partners, and this is currently in the process of being signed by the partners. Indeed, efforts are ongoing to develop an information-sharing agreement between HC and the CFIA, which requires and action plan with clear milestones and senior management supportFootnote 101. The CFIA has also collaborated with PHAC's Prion Diseases Program. For example, PHAC has been invited to participate in several international discussions of beef trade with Mexico, by presenting Canadian surveillance data on human prion diseases (CJD). PHAC and CFIA laboratory scientists have also collaborated directly in biochemical and genetic studies to better understand the epidemiology of BSE in Canada, and in research targeted to discovery and characterization of novel diagnostic markers for BSE and other prion diseases.

With provincial counterparts, CFIA also effectively collaborates on many BSE components. For instance, communications with provincial Chief Veterinarian Officers are constant and open. The Science Branch has a close and cooperative relationship, including communication and ongoing research, with provincial and university laboratories. For animal identification, CFIA participates in the Federal-Provincial-Territorial Traceability Task Team. However, shifting responsibilities between the provinces and the CFIA, including the transition for SRM removal to provincial inspectors, presents some current coordination challenges. As such, a few external interviewees indicated that an even closer collaborative approach with provincial partners would help improve the effectiveness of the BSE Management Program.

CFIA collaboration with industry was viewed positively by both sides, including by field staff. For animal identification, CFIA collaborates with the industry-led Canadian Cattle Identification Agency and Agri-Traçabilité Québec (co-led by industry and the Province of Quebec). Industry associations and federally registered establishments also work in close conjunction with the CFIA to achieve BSE objectives, although non-federally registered establishments have had some difficulty meeting all SRM-related requirements (also linked to EFB).

Finally, regarding overall collaboration and coordination, it is important to note that the Agency proposed a two-step approach in fall 2012, which included the development of the EFB 2012 Review in the short term and Canada's BSE Roadmap in the long termFootnote 102. The EFB 2012 Review is the first step in the overall review of Canada's BSE program. The purpose of the BSE Roadmap will be to communicate to the public and domestic and international stakeholders the Canadian long term approach to BSE disease control. The roadmap will require a full review of all BSE programs in Canada (from both government and industry). All policy and regulatory options will be considered in this exercise. Decisions on future BSE programming will need to be supported by strong scientific evidence, international standards, and the consideration of various stakeholder views.

4.4.7 Question 12: Is the performance measurement and monitoring strategy for the BSE Management Program and each of its components adequate and appropriate?

Performance measurement and monitoring processes are adequately fulfilling the basic functions for the CFIA's operational and strategic needs, but further improvements to financial tracking and data collection systems are needed.

Consistent evidence across the survey, as well as documents and interviews show that some appropriate performance measurement structures have been (or are being) developed, including various data collection systems (e.g., for inspection and surveillance), a performance measurement framework at the agency level for the DPR, as well as performance targets for the horizontal BSE program, reported separately to the Treasury Board Secretariat (TBS)Footnote 103. In fact, a detailed case study on BSE—related data collection, compilation and use within the CFIA found that there is a wealth of data collected in various systems, including by delivery partners (e.g., surveillance, traceability). However, although required performance measurement data is available, it is often difficult to collect and requires validation by subject matter experts.

Documents, interviews and the survey also indicate that basic operational needs (e.g., managing and planning program delivery) and strategic needs (e.g., accountability and reporting obligations, external communication) are generally adequately met by the CFIA's performance measurement and monitoring systems and processes. Moreover, there have been improvements made to performance measurement systems in the last five years, as well as efforts to resolve issues were noted in previous evaluations (BSE/EFB; see follow-up on recommendations in Appendix E).

However, there remain known and ongoing issues with performance measurement, and limited internal awareness about the value or use of such data outside of key users or those directly responsible for systemsFootnote 104. This poses an issue given the variety of data collection/compilation systems in place, and the number of individuals feeding data into these systems, such as field staff, laboratory staff and external partners. For strategic needs, meeting reporting requirements has been a challenge in terms of providing timely, targeted, comprehensive reports to various stakeholders, including the TBS, OIE and trading partners. In particular, the lack of adequate delivery data has posed a challenge for this evaluation, including with regard to determining BSE-specific expenditures (see Question 6, and 13 to 15). For example, as noted under Question 6, operations databases (e.g., MRRS, CVS) and the multiple surveillance databases are not standardized, are faced with various data quality and consistency issues, and therefore often require manual validation to ensure accurate interpretation. Similarly, data for indicators identified in the BSE performance measurement strategy (dated December 2012) were not always readily available or adequate to support this evaluation (e.g., training data, export certification/technical market access outputs).

Issues and potential improvements to better meet strategic and operational needs for specific individual program components include:

4.5 Performance: Demonstration of efficiency and economy

4.5.1 Question 13: Have the allocated financial resources and human resources (HR; FTEs) been adequate for the activities of each component of the BSE Management Program?

The evaluation cannot fully and conclusively assess the adequacy of allocations for the Program because of known issues with financial tracking (e.g., BSE funds are pooled with A-base funding and cross-utilization of resources make financial coding challenging and inconsistent). Various lines of evidence, including reported financial data, suggest that a proportion of BSE funding has been allocated to other priorities. However, recent model projections indicate this proportion is likely smaller than originally reported since financial data did not fully reflect allocations to BSE-related activities. As such, FTEs required to deliver the BSE program are estimated to be nearly equivalent to the amount allocated. Additional HR-related issues were also noted around succession planning (i.e., maintenance of necessary BSE expertise within the CFIA), as well as around training, support and guidance provided to field staff delivering BSE activities.

All lines of evidence point to persistent and well-known issues with financial tracking relating to the BSE Management Program. Cross-utilization of resources across both BSE and non-BSE activities affects all branches, such that, as explained by interviewees, there are few, if any individuals dedicated solely to BSE within the Agency. As such, one cannot simply count FTEs funded via the BSE allocation. Moreover, given that many BSE activities are integrated with other tasks, financial data likely does not fully reflect allocations of BSE funds; e.g., inspections and export certification may include both BSE and non-BSE tasks but time spent on BSE vs. non-BSE tasks are not always accounted for separately. On the other hand, tracking of allocations for BSE surveillance is more straightforward, so these can be more easily tied to BSE funding. A recent activity-based projection of FTEs required to deliver the BSE program (led by the CFIA's Corporate Management Branch) confirms the existence of a gap between reported and required resources (see below), with more required than reported.

Moreover, when the new business line structure was implemented across the CFIA in 2011, BSE funds were pooled with A-base funding. As such, each branch is now responsible for tracking its BSE allocations. However, this tracking is not consistent because of coding issues (e.g., funding for vets abroad are not coded as BSE allocations; also, guidance was provided to regional staff to limit the number of programs they may code their activities to in order to reduce the administrative burden and complexity of coding), challenges associated with the CFIA's existing financial systems, and because of challenges in delineating BSE activities for the reasons noted above.

Issues with financial tracking for BSE (and the EFB) have been repeatedly discussed by several BSE governance committees across the evaluation period, including at senior management levelsFootnote 109. These committees discussed and identified a number of operational and strategic needs for improved tracking and coding (e.g., to determine internal priorities, to assist in planning and delivering operations, to strengthen accountability and reporting, to make the case for funding renewal, etc.). Consequently, various actions have been initiated to better capture all expenditures of BSE funds, from internal communication to internal reviews/assessments dedicated to this purpose. In 2012, the BSE Advisory Committee stated the need for "a complete financial crosswalk, by late October [2012]" for BSE funds; however the current status is unclear and associated challenges were still on the agenda in May 2013. The activity-based exercise conducted in 2013 represents the most recent effort to address this issue. The analysis below discusses all available evidence, but concludes it was not possible to fully and conclusively assess the adequacy of allocations for the BSE Management Program overall, or for individual program components.

The CFIA received $65.3 million per year for BSE-related activities from 2009-10 to 2013-14, $38.7 million of which is for the Comprehensive BSE Programming annually (i.e., the components within the scope of this evaluation; the rest is for the EFB)Footnote 110. However, the Agency has reportedly spent about 70-90% of these planned resources on BSE-related activities (Table 7): the annual actual spending for the BSE Program activities was between $34.2M and $27.2M between 2009-10 and 2012-13. The variance has been steadily increasing since 2009-10, from 12 to 30%.

Looking at individual activities, the differences between actual and planned funds vary greatly. Actual allocations were lower than planned for technical market access support, SRM removal, export certification and cattle identification. In contrast, actual spending was generally higher than planned for BSE surveillance and import controls. Note that, given the explicit nature of the tasks involved, these last two activities are more easily tracked as being BSE-specific compared to the other four. In terms of trends across the evaluation period, actual/reported spending decreased over the period for all BSE activities except cattle identification (for which there were some initial implementation delays). To note, BSE surveillance funding depends to some extent on the number of samples submitted; thus, actual expenditures for surveillance exceeded planned amounts in the first two years, when the number of samples submitted was higher (about 35,000, above the 30,000 target; see Question 7) than in the two most recent years.

Table 7: BSE Planned and Actual Spending 2009-10 to 2012-13, $ million, per Activity
2009-10
Planned
2009-10
Actual
2010-11
Planned
2010-11
Actual
2011-12
Planned
2011-12
Actual
2012-13
Planned
2012-13
Actual
SRM Removal from human food supply 9.5 8.4 9.5 7.0 9.5 6.7 9.5 6.1
Import Controls 0.3 0.6 0.3 0.4 0.3 0.2 0.3 0.5
BSE Surveillance 15.4 18.2 15.4 18.8 15.4 15.4 15.4 13.8
Cattle Identification 2.8 2.1 2.8 2.2 2.8 2.3 2.8 2.4
Export Certification 5.7 3.7 5.7 3.4 5.7 3.3 5.7 2.7
Technical Market Access Support 5.0 1.2 5.0 1.8 3.6Table Note 2 2.0 5.0 1.7
Total 38.7 34.2 38.7 33.6 37.3 29.9 38.7 27.2
Variance 4.5 5.1 7.4 11.5
Variance % 12% 13% 20% 30%

Table Notes

Table Note 2

Planned Spending was reduced by $1.4 million to take into account amounts transferred DFATD to provide support to CFIA staff located at missions abroad.

Return to table note 2 referrer

Sources: Data provided by Corporate Management Branch, CFIA (BSE request for Eval, #4020908 – v4)

These financial data were reported in the DPR each year, in which the CFIA indicated that the variance between planned spending and actual spending is related to the reallocation of resources to other agency prioritiesFootnote 111. This was echoed by several internal interviewees, who pointed to the pooling of BSE funding with A-base funds as a factor that facilitated the redistribution of funds (e.g., to support baseline animal health activities or other CFIA priorities). However, interviewees could provide little further evidence on how these funds were used, and no files were obtained that documented the basis on which such reallocations were made.

As noted above, a recent activity-based CMB projection of the resources required to deliver the Program provides some insight on this issue. This analysis strongly suggests that all activities have underreported the use of BSE resources to some extent—some greater than others. Table 8 shows the planned FTEs as per initial program documents, those reported as actual FTEs for 2011-12 in a draft activities table prepared in April 2012, and the results of the CMB project that was based on detailed 2012-13 data (note: FTE data was not available for other years). This CMB projection is based on a detailed analysis of BSE resource requirements. This analysis involved consultations with program experts, focus groups and a survey to identify the proportion of FTEs or tasks that are related to BSE for each activity. It further calculated BSE resource requirements, by activity and overall, based on this information and on data recorded in existing systems (i.e., MRRS and CVS) for actual delivery of tasks in 2012-13Footnote 112. This model was assessed by KPMG in November, 2013, who concluded:

"Our independent review did not identify any significant quantifiable differences in the resource requirement estimate resulting from CFIA's application of the resource requirement model"Footnote 113.

Table 8: Planned FTEs, Actual FTEs per Activity as of April 2012, and Projected FTEs required to deliver the BSE Program based on 2012-13 Data
Activities Planned FTE Actual FTETable Note 3 (2011-12) Projection (based on 2012-13 data)
SRM Removal from Food 89 53 87.3
Import Controls 2 4 6.0
BSE Surveillance 97 96 96.5
Cattle ID 28 18 26.8
Export Certification 33.4 26 48.2
Technical Market Access 14 11 20.7
Enhanced BSE (excluding EFB) sub-total 263 208 285.5
Total (including EFB) 503 352 NA

Table Notes

Table Note 3

Actual refers to what was recorded in the financial system, which, as noted, is incomplete.

Return to table note 3 referrer

Source: CFIA, Corporate Management Branch (2012, April). Overall BSE Activities Template; CFIA. (2013). Estimated number of FTEs required for BSE related activities. Prepared by CFIA's Financial Services, October 2013. Sources provided in RDIMS #4079213.

The April 2012 analysis suggested that, based on available coding, the BSE Program operated with 208 FTEs in 2011-12, which was 21% (55 FTEs) less than originally allocated to the Program (i.e., planned)Footnote 114. However, the projection based on the 2012-13 model shows that the actual number is likely much closer to the original planned FTEs—at about 285Footnote 115. The reported data and projection each covered only an individual year, so it is not possible to confirm whether either represents the situation in other years during the period under evaluation. However, the model results suggest that an approximately equivalent (or slightly elevated) amount of the allocated funding is adequate to deliver the BSE Management Program as a whole. The Agency is currently determining how the model will be implemented to support planning and reporting in future.

Many interviewees provided corroborating views on the general adequacy of financial resources for the BSE Management Program: other than for research-related activities conducted by the Science Branch, the level of available resources was perceived to have been adequate to deliver planned activities during the period under evaluation. For example, almost all inspection managers interviewed reported that staffing was adequate for BSE-related tasks conducted in the field.

However, many interviewees (both internal and external) referred to ongoing government budget restrictions when raising concerns about whether the level of staffing and funding is (and will continue to be) adequate to fully deliver the BSE Program. Concerns were especially noted if there is indeed reallocation to baseline animal health activities that have been experiencing reduced budgets since 2008. It is important to note that BSE funding has remained stable over the evaluation period. As mentioned above, activities believed to have already been affected by restricted budgets include funding for research staff and projects on BSE and human health. Representatives from the Science Branch pointed out that research activities are currently facing an additional challenge given the shrinking pool of external R&D funding (see Question 9).

Indeed, more general resource-related concerns also apply to the BSE Program given the importance of cooperation with external partners and collaborative delivery of several activities (e.g., SRM removal in provincially regulated establishments, BSE surveillance, cattle identification, research). Indeed, all internal interviewees reported that support (financial and otherwise) for BSE measures has suffered in recent years given the diminishing interest/vigilance demonstrated by both internal and external stakeholders. This is believed to be putting additional pressure on the CFIA if the BSE Program is to continue to achieve its outcomes. For example, considering cuts to provincial sample reimbursement programs in recent years and the associated drop in sample numbers, the federal Sample Reimbursement Program and CFIA-led collaborative initiatives such as CanSurvBSE have taken on greater importance to maintain surveillance targets.

Moving from financial resources to HR capacity, most CFIA interviewees perceived there to be issues related to succession planning and general maintenance of necessary BSE expertise within the CFIA. Internal interviewees found that general budget reductions within the Agency, ongoing staff turnover, and reduced awareness and/or attention to BSE not only threaten the level of staffing and funding for BSE, but also have already begun to erode the institutional memory relating to BSE management. Given the long-term commitment and vision needed to manage a disease like BSE, this maintenance of expertise across all branches is required, especially for senior staff and management responsible for the response to BSE. This issue was specifically highlighted as being critical for the Science Branch (especially CFIA-based research activities) and technical market access support. Reduced scientific capacity and technical expertise within the Agency poses a risk for the future effectiveness of the program—including for market access—given its science-based nature and emerging scientific knowledge on BSE/TSEs. As such, external interviewees strongly supported further research and identified a need to ensure BSE measures are firmly grounded in science.

Finally, issues around training, support and guidance were noted by Operations Branch and field staff. Veterinarians and inspection staff consulted via the survey most often reported being offered formal training for cattle identification (71% of inspectors and 64% of veterinarians), and SRM removal (61% of inspectors and 45% of veterinarians). In contrast, BSE surveillance training was offered to 52% of the inspectors and 37% of the veterinarians, while the lowest rates were observed relating to training offered for BSE-related import controls: 43% of the inspectors and 20% of the veterinarians. Generally, veterinarians were more likely to declare not having received formal training (36-55% depending on the activity) compared to inspectors (21-47%). It is worth mentioning that inspectors who were hired in the last two years, or more than five years ago, reported having been offered training related to SRM removal more frequently than those with 3-5 years of experience, suggesting a recent improvement. However, interview data suggests that inspection and veterinarian staff turnover are creating a competency gap—including for SRM removal—despite new online, video based and on-the-job training.

Perhaps even more telling is the field staff's perceived inadequacy of the information, including training but also support and guidance, offered by the CFIA in the last five years. In fact, at most 54% of respondents believed the information provided for specific BSE-related activities was adequate. These views often depend on the type of activity. Some of the most notable findings from the survey data include the following (see also Figure 3).

Overall, this suggests a potential need for additional training, support and guidance offered to field staff by the CFIA to support the delivery of BSE-related activities by inspectors and veterinarians, and some activities appear to merit more immediate attention than others (i.e., import controls and BSE surveillance). Note that limited data was obtained to assess the impact of this issue (i.e., whether this situation affects the effectiveness of program delivery or its efficiency, and if so, to what degree), or whether other channels or sources are available and sufficient to mitigate this information gap. Further review or analysis of additional training, support and guidance by the CFIA would therefore help to assess the severity of this situation and/or identify mitigating measures.

Figure 3: Perceived adequacy of information (training, guidance, support, etc.) offered to field staff by the CFIA in the last five years. Description follows.
Description for Figure 3: Perceived adequacy of information (training, guidance, support, etc.) offered to field staff by the CFIA in the last five years

Figure 3 is a bar chart that displays the results of a survey of inspectors and veterinarians. The survey pertained to perceived level of adequacy of information. On the vertical axis of the bar graph there are the following categories from top to bottom: Export certification, Import controls for BSE, BSE surveillance, Cattle/animal identification and SRM Removal from human food supply. The categories on the vertical axis are divided into Veterinarian and Inspectors to show the survey results for each group. Percentages are displayed on the horizontal axis starting at 0% and going up to 100% in intervals of 10%.

There is a legend below the horizontal axis, yellow represents a survey result of very inadequate or inadequate, green represents moderately adequate, blue represents very adequate or adequate, gray represents Don't know/Not Applicable/No reply.

The data ranges described below are approximate values.

Export certification

  • The veterinarian bar is yellow from 0 to 22%, green from 22 to 50%, blue from 50 to 78% and gray from 78 to 100%
  • There is no inspector bar

Import Controls for BSE

  • The veterinarian bar is yellow from 0 to 20%, blue from 20 to 60% and gray from 60 to 100%
  • The inspector bar is yellow from 0 to 33%, green from 33 to 53%, blue from 53 to 70% and gray from 70 to 100%

BSE Surveillance

  • The veterinarian bar is yellow from 0 to 32%, green from 32 to 57%, blue from 57 to 78% and gray from 78 to 100%
  • The inspector bar is yellow from 0 to 21%, green from 21 to 47%, blue from 47 to 86% and gray from 86 to 100%

Cattle/animal identification

  • The veterinarian bar is yellow from 0 to 18%, green from 18 to 50%, blue from 50 to 95% and gray from 95 to 100%
  • The inspector bar is yellow from 0 to 19%, green from 19 to 39%, blue from 39 to 89% and gray from 89 to 100%

SRM Removal from human food supply

  • The veterinarian bar is yellow from 0 to 23%, green from 23 to 41%, blue from 41 to 82% and gray from 82 to 100%
  • The inspector bar is yellow from 0 to 17%, green from 17 to 38%, blue from 38 to 93% and gray from 93 to 100%

Note: 99 inspectors and 22 veterinarians responded to the survey. Respondents were only asked to provide a rating for activities in which they were involved (e.g., no inspectors conduct export certification so they are shown as NA for this activity).

Source: Survey of CFIA field staff conducted by Science-Metrix.

4.5.2 Question 14: Is the BSE Management Program cost-effective as currently delivered?

4.5.3 Question 15: How efficiently and effectively have the BSE Management Program funds been used, both overall and for each program component?

Data limitations prevent the evaluation from supporting a strong conclusion for these two questions. Given that the Program has achieved most of its intended outcomes to date within the allocated budget, it is likely that the Program is fairly cost-effective. Internal interviewees identified very few opportunities to make changes to the Program in order to implement more efficient practices. Achieving short-term efficiencies without transparent documentation of the changes or cutting back on specific BSE activities may also pose a disproportionately high risk to the ultimate outcomes of the Program (e.g., negative perceptions affecting Canada's OIE BSE risk status or the confidence of trading partners and consumers, especially if there continues to be new BSE cases).

The findings for these two questions draw on evidence discussed in previous questions, especially 6, 7 and 13. The same caveats regarding data limitations noted in those questions therefore apply here as well. As such, the conclusions for these questions are not strong. Overall, it appears that the BSE Management Program has been delivered in a cost-effective manner given that the BSE Management Program has generally achieved its intended outcomes within its allocated budget and despite actual outputs being at slightly below the planned level (see Question 6). In addition, as BSE activities are incorporated in other programs, theoretically there are less separate program management costs attributable to BSE compared to other programs with separate management oversight. A broad consensus also emerged from the internal interviews that the Program, overall and for each component, was cost-effective as delivered, especially if a portion of the allocated BSE budget was freed up to support other priorities or baseline activities.

On the other hand, if the entire BSE allocation is indeed required to deliver the BSE Program (as shown in the projection based on 2012-13 data), the fact that some activities were slightly under-implemented suggests that efficiency could be enhanced in order to increase delivery to the planned level for these activities. In addition, this means the Program may actually not be as efficient as perceived by interviewees. However, when asked about any possible efficiencies to be gained, CFIA interviewees explained that anything that has been deemed non-essential has already been eliminated and thus, across Branches and BSE activities, BSE resources are already fully stretched.

Documentary evidence also showed ongoing efforts to identify or implement changes within the Agency, which affect its core BSE activities, in order to achieve higher cost-effectiveness. These include the recent integrated Animal Health Business Line Planning Process (which is an approach to activity planning using a risk-based and results driven list of priorities based on available capacity)Footnote 116, ongoing inspection modernization efforts (a 2011, $100 million initiative to improve inspection delivery, training and tools for inspection staff, scientific capacity in food laboratories, and information management and technology)Footnote 117, as well as the analysis of three different legislative options to introduce mandatory life-cycle traceability (which includes a resource and financial analysis)Footnote 118.

Relating to the implementation of the business line model, cross-usage of resources within the animal health business line (i.e., same resources conduct a variety of tasks, not only BSE-related, and very few are actually dedicated to BSE only) and pooling of BSE funds with A-base were said by some CFIA representatives to have resulted in increased economy (i.e., reducing costs/inputs) and efficiency (i.e., delivering the same outcomes with less outputs) of the Program and of the CFIA more generally. However, challenges in tying BSE funds to actual outputs (and in determining what proportion of BSE funds were used to support other activities), and in precisely measuring the outputs of the Program make it difficult to confirm that efficiency gains were truly achieved.

Interviewees and documents also pointed to some important consequences to consider when reallocating funds, particularly when it cannot be assessed precisely to what extent Program delivery, outputs and objectives are actually being met. Unless adjustments to the BSE program are transparently documented and evidence-based, they are not likely to be viewed favourably by the OIE, trading partners or consumers. As such, decisions taken on resource allocation or to reduce the level of any of the BSE activities for efficiency reasons need to take full account of potential impacts on domestic and international consumer confidence and Canada's progress to negligible BSE-risk status. These decisions may also send a signal to internal and external stakeholders about the level of importance or value of these activities, such that budget reductions or reallocation of resources for specific activities may reduce the commitment of CFIA staff, OGDs, provinces and industry (i.e., key delivery collaborators) to supporting the BSE program.

Thus, the long-term risk of the current financial management practices is difficult to predict and implies a need to improve financial tracking, as well as a greater need to demonstrate that the Program is achieving its outcomes in order to maintain Canada's reputation and to support the maintenance and upgrading of Canada's OIE BSE-risk status. Some interviewees pointed out that these considerations would be especially important if there continue to be new cases of BSE detected (especially in any animals born after the implementation of the EFB in 2007) in order to avoid negative perceptions and potential repercussions by OIE, trading partners, or consumers.

5.0 CFIA Conclusions, Best Practices and Recommendations

Relevance

The unique nature of BSE—in particular it's long incubation period and the sporadic emergence of atypical strains of the disease—poses potential long-term risks. Accordingly, the range of activities within this Program are necessary to protect human and animal health from potential BSE-related risks, as well as to support the Canadian beef and cattle industry in light of the ongoing and significant economic and trade impacts of this disease. There is, therefore, a clear ongoing need for the BSE Management Program.

The BSE Management Program deals with food safety and animal health, as well as supporting the access of Canadian agricultural products to foreign markets. These activities remain clearly aligned with the Agency's food safety and market access strategic outcomes as well as the federal government's priority of a competitive agriculture sector and protecting the health and safety of Canadians.

The Agency's legislative framework and its international commitments (e.g., agreement with the EU on sanitary measures to protect public health and animal health) make the BSE Management Program consistent with the CFIA mandate and federal roles and responsibilities.

Performance

The BSE management Program has been implemented as planned, with few exceptions, and appears to be achieving its expected outcomes. First and foremost, there has been no case of BSE in Canada since 2011 and many markets that closed to Canada for beef export have since been reopened, along with new ones. All the required preventative components are in place to help ensure that BSE is as rare an occurrence as possible and that Canada will be in a position to apply in 2015 for negligible risk status.

The Agency has gained valuable experience in designing, implementing and managing this program. The performance measurement framework took many years to develop and provided lessons learned regarding the importance, challenges and necessity of performance data collection. This framework also provided the Agency with an understanding of the various differences, deficiencies and alignments between the various data collection instruments and bodies that continue to pose reporting challenges. The Case Study on BSE performance measurement undertaken as a component of this evaluation outlines gaps and opportunities for improvement in this area, and provides an overview of related performance measurement activities and issues.

Since BSE is not something that is foreseen to be eradicated in any current reader's lifetime, most of the program elements have become imbedded in the day to day activities of the Agency. This is both natural and problematic. The natural inclusion of these activities helps to ensure that a lack of concern about the reoccurrence of BSE does not diminish the Agency's vigilance in the defence against it, though complacent behaviour is a rising challenge for the Program. The problems stemming from the integration lie in the difficulty in tracking data separately from other activities, including for expenditures, which represents an area where the Agency continues to struggle and limits its ability to demonstrate the Program's effectiveness and efficiency.

Best Practices

The BSE Management Program has also supported scientific research that has advanced the Agency's international reputation. The evaluation found great value in the Agency's scientific BSE/TSE activities, including research and risk assessments. This is considered a best practice and its continuation is seen as key to maintain the benefits that the Agency has already wrought from these efforts.

The research capacity of OGDs in the horizontal BSE program is seen as an opportunity to avoid overlap and ensure that adequate federal government expertise is maintained. Further developing science-based collaboration with partners, including by attracting external R&D funding, could also be beneficial. Key partners relative to human health research include HC and PHAC, whereas provincial laboratories and academia—both within Canada and internationally—hold a wide range of complementary BSE research expertise.

Within CFIA, with planned future retirements, maintaining a resource base of employees experienced with BSE could prove challenging and present risks to the program. In addition, the evaluation identified a range of gaps in training related to BSE. These signal the need for a strategic BSE HR Plan to address BSE succession planning, coupled with targeted BSE training opportunities in the field.

Recommendations

As noted in this report, the integration of BSE activities with other Agency activities prevents financial data from fully reflecting allocations of BSE funds. However, this report does not recommend improved financial tracking because the Agency has recently developed a comprehensive costing model, positively reviewed by an independent auditing firm, which will be used for future costing and reporting.

Recommendation 1

The CFIA should improve its performance monitoring practices relating to the BSE Program.

Many systems are cross-utilized by other programs/activities and so improvements with multiple benefits (i.e., beyond BSE) should be targeted. Improving tracking and reporting would also support risk assessments, epidemiological studies and other scientific endeavours and by providing information for sound decision-making going forward (e.g., risk-based inspection planning). The Case Study on BSE performance measurement provides an overview of performance measurement and outlines gaps and opportunities for improvement in this area.

Recommendation 2

Coordination and communication continue to be vital to the effectiveness and efficiency of the Program, but that there was a growing level of complacency noted both within and outside the CFIA with regard to BSE:

The CFIA should develop and implement an internal and external engagement strategy sufficient to build awareness of the continued need for a BSE program, especially in the absence of recent BSE cases.

The BSE Roadmap process is a potentially useful tool to engage and communicate to the public and domestic and international stakeholders the Canadian long term approach to BSE disease control. Key partners that should be involved include provincial governments, industry, and OGDs (e.g., AAFC for market access; CBSA for import controls; HC and PHAC for public health). There is also a strong rationale for the continuation of CanSurvBSE to support the continued effectiveness of the surveillance program. Comments within CFIA also suggest that there is not a consistent understanding of the importance of maintaining surveillance efforts.

The CFIA could leverage the development of the traceability program as an opportunity to demonstrate the benefits of a collaborative model for BSE and beyond. Improved traceability is important not only for management of BSE cases and eradication of BSE, but also to respond more effectively to other animal diseases. This program has made recent progress and is aligned with both industry and government priorities.

Recommendation 3 and 4

Given the growing HR-related issues noted with regard to BSE expertise, training and succession planning within the Agency:

The CFIA should implement a succession strategy to ensure it maintains sufficient BSE expertise within the Agency for the management and delivery of BSE-related activities.

The CFIA should also ensure that employees responsible for implementing BSE activities have adequate training to fulfill their roles and responsibilities.

A concerted HR/succession strategy would help ensure that BSE expertise is maintained across multiple levels of the Agency and across branches. Particular components that are most likely at risk are those that require technical or scientific expertise, including technical market access, surveillance and other components that are informed by risk assessments.

There is a need for the CFIA to continue updating and reviewing training, support and guidance offered to field staff and key delivery partners, particularly in areas where verification and enforcement of BSE regulations and policies were found to be lacking or inconsistent across the country. In particular, this review could improve support offered for provincial inspection of provincially regulated facilities. In addition, although the CFIA's import controls are compliant with international standards, improving the training and support offered to field staff engaged in the verification and enforcement of these controls can further ensure that BSE is prevented from entering the country from foreign sources. Considering the long-term nature of BSE, additional training and support are also necessary to help counter the observed complacency with regard to BSE programming.

6.0 HC and PHAC Program Description

6.1 Thematic Area A – Managing BSE-Related Risks

6.1.1 Activity A3: Risk Assessment and Target Research Regulated Programs - Health Canada ($6.2M/year planned)

This activity aims to address evolving science, uncertainty, and emerging findings linking BSE and other animal TSEs to human health outcomes, and will permit engagement of domestic and international discussions on BSE and TSE science, and regulatory topics. Risk assessment and research activities will support domestic an international collaboration on regulatory and policy decisions.

6.2 Thematic Area B –

6.2.1 Activity B6: Surveillance and Research for Human TSEs – Public Health Agency of Canada ($0.8M/year planned)

This activity aims to ensure national surveillance and research to human TSEs through the CJDSS. High quality surveillance data on the incidence and character of human TSEs, along with surveillance data on the incidence of BSE I n cattle, provide unique scientific evidence that measure to mitigate risk of BSE transmission (such as SRM removal from the human food chain) remain effective.

6.3 HC and PHAC BSE Program Logic Model

Figure 4: BSE Management Program Logic Model – HC/PHAC-led activities. Description follows.
Description for Figure 4: BSE Management Program Logic Model – HC/PHAC-led activities

This is a diagram that describes activities, outputs and outcomes of the BSE Management Program for Health Canada and Public Health Agency of Canada. On the left side of the diagram is a vertical colour coded legend with the following written in it from top to bottom:

  • "Thematic Areas" which are shaded red
  • "1. Activities/Strategies" which are shaded gray
  • "2. Activities" which are shaded light blue
  • "3. Outputs" which are shaded dark blue
  • "4. Immediate Outcomes" which are shaded light green
  • "5. Intermediate Outcomes" which are shaded dark green
  • "6. Long-Term Outcomes" which are shaded purple

The items in the legend are organized vertically in the logic model in the same order as the legend with "Thematic Areas" at the top and "6. Long-Term Outcomes" at the bottom.

Thematic Areas

The contents of the Thematic Areas section of the logic model are contained inside of a rectangle with a red edge. Inside of that rectangle are two rectangles that have been shaded red and are organized horizontally.

  • The left rectangle has "A. Managing BSE-Related Risks" written it
  • The right rectangle has "B. Monitoring BSE-Related Risks" written it

1. Activities/Strategies

The Activities/Strategies section of the logic model is contained inside of a rectangle with a gray edge. Inside of that rectangle are two rectangles that have been shaded gray and are organized horizontally.

  • The left rectangle says "Risk Assessment and targeted research regulated programs (Health Canada)"
  • The right rectangle says "Surveillance and Research for Human TSEs (Public Health Agency of Canada)"

2. Activities

The Activities section of the logic model is contained inside of a rectangle with a blue edge the Outputs section of the logic model is also inside of this rectangle. The Activities section has five rectangles shaded light blue organized into two columns.

There are two rectangles in the left column:

  • The top one says "Conduct research for the safety of animal or human sourced ingredients in food and products regulated by Health Canada and product surveillance"
  • The bottom one says "Conduct risk assessments of animal or human sourced ingredients in food and products regulated by Health Canada"

There are three rectangles in the right column:

  • The top one says " Comprehensive detection and characterization of all human TSEs in Canada"
  • The middle one says "Research and development to improve diagnostic and surveillance capabilities for human TSEs"
  • The bottom one says "Targeted research to improve scientific understanding of biology of humans TSEs"

3. Outputs

The Outputs section of the logic model is contained inside of the same rectangle with a blue edge as the Activities section. The Output section has thirteen rectangles shaded dark blue also organized into two columns.

There are five rectangles in the left column; they have the following text from top to bottom:

  • Data analysis
  • Research papers
  • Laboratory studies
  • Research findings
  • Risk assessments (including recommendations)

There are eight rectangles in the left column; they have the following text from top to bottom:

  • Human TSE case investigations
  • Diagnostic laboratory services
  • Statistics on human TSEs in Canada
  • Responses to human TSE incidents
  • New and/or improved diagnostic tests
  • New scientific knowledge of TSE biology
  • Human biobank to support research
  • Expert advice for regulatory issues

4. Immediate Outcomes

The Immediate Outcomes section of the logic model is contained inside of a rectangle with a green edge the Intermediate Outcomes section of the logic model is also inside of this rectangle. The Immediate Outcomes section has two rectangles shaded light green.

  • The left rectangle says "Increased expertise and knowledge of BSE/TSE science, risks and product surveillance"
  • The right rectangle says "Increased information, expertise and knowledge through TSE science and surveillance, leading to improved foundation for evidence-based decision making."

5. Intermediate Outcomes

The Intermediate Outcomes section of the logic model is contained inside of the same rectangle with a green edge as the Immediate Outcomes section. The Intermediate Outcomes section has two rectangles shaded dark green.

  • The left rectangle says "Increased knowledge based decision-making"
  • The right rectangle says "Scientifically informed policies and decision to deal with the public health risks posed by human TSEs in Canada"

6. Long-term Outcomes

The Long-term Outcomes section of the logic model is contained inside of a rectangle with a purple edge. Inside of that rectangle are four rectangles that have been shaded purple and are organized vertically.

  • The top rectangle has this text inside of it "Protection from preventable health risks related to food safety and product safety or the transmission of animal diseases to humans"
  • The rectangle second from the top has this text inside of it "Safer food and health products"
  • The rectangle second from the bottom has this text inside of it "A sustainable animal resource base"
  • The bottom rectangle has this text inside of it "Maintain public/consumer confidence in the animal, production and food system"

6.4 BSE Program Funding

The following financial expenditure information was derived from the CFIA's Departmental Performance Report's Supplementary Tables:

Table 9: Annual Health Canada Planned and Actual Spending (2009-10 to 2013-14), per Activity
(in $M) 2009-10 to 2013-14
Planned (annual)
2009-10
Actual
2010-11
Actual
2011-12
Actual
2012-13
Actual
2013-14
Actual
Health Products Risk Assessment & Research 4.4 5.9 3.5 4.2 2.0 NA
Food Safety and Nutrition Risk Assessment & Research 1.8 5.9 1.3 4.2 1.8 NA
Total 6.2 5.9 4.8 4.2 3.8
Variance with planned 0.3 1.4 2.0 2.4 NA
Variance % 5% 23% 32% 39% NA
Table 10: Annual Public Health Agency of Canada Planned and Actual Spending (2009-10 to 2013-14), per Activity
(in $M) 2009-10 to 2013-14
Planned (annual)
2009-10
Actual
2010-11
Actual
2011-12
Actual
2012-13
Actual
2013-14
Actual
Surveillance & Population Health Assessment (Surveillance, Research & Reference Services) 0.8 0.8 0.8 0.8 0.8 NA
Total 0.8 0.8 0.8 0.8 0.8 NA
Variance with planned 0 0 0 0 NA
Variance % 0% 0% 0% 0% NA

7.0 Health Canada and Public Health Agency of Canada Methodology

The Health Canada-Public Health Agency of Canada evaluation was carried out from June 2013 to January 2014 in the following stages:

  1. Stage I: Request for documentation and performance information as per the evaluation framework.
  2. Stage II: Data collection to gather evidence for the evaluation.
  3. Stage III: Analysis of the evidence and drafting of the evaluation findings.

The methodology chosen was based on a calibrated approach where a reduced level of effort for Health Canada and the Public Health Agency of Canada was deemed appropriate due to two factors: 1) these organizations represent only 15% of total BSE III resources, and 2) an evaluation of BSE I and II initiatives was recently completed (July 2013) and activities have been substantially unchanged since this last evaluation.

More specific details on the data collection and analysis methods are provided in Appendix B.

7.1 Methods Overview

7.1.1 Document and File Review

This method involved a review of over 30 documents and files to gain an understanding of the BSE Management Program in Health Canada and the Public Health Agency of Canada. These documents were primarily related to Program planning and performance information and informed both relevance and performance issues in the evaluation. Key documents included:

7.1.2 Key Informant Interviews

A total of 17 individuals were interviewed either through a telephone conversation or through in-person interviews (some held in groups)to obtain views and input on various aspects of the BSE Management Program in Health Canada and the Public Health Agency of Canada. The interview guides are provided in Appendix C. These views informed primarily the performance issues in the evaluation. A breakdown of the interviews by department is provided in Table 11:

Table 11: Key Informant Interviews
Department/Agency # Interviews
Health Canada 11
Public Health Agency of Canada 2
CFIA 4

These internal stakeholders were identified as those most involved in, knowledgeable about and/or responsible for the BSE Management Program in Health Canada and the Public Health Agency of Canada. They included senior managers, Program managers and research managers.

7.2 Analysis and Integration of Data

Evaluation findings were analyzed and integrated across the lines of inquiry. Facts and data were validated by Program management, through their review of a draft report and through a gap filling exercise. Management reviewed key conclusions and recommendations, and appropriate changes were made where relevant and supported by evidence.

7.3 Health Canada and Public Health Agency of Canada Challenges, Limitations and Mitigation Strategies

The challenges and limitations for Health Canada and the Public Health Agency of Canada, and the corresponding mitigation strategies, are described below in Table 12.

Table 12: Health Canada and Public Health Agency of Canada Challenges, limitations and mitigation strategies
Challenge & limitation Mitigation strategy Implications/Impact on Evaluation
The collection of evaluation evidence focused on key performance documents and select key informant interviews (i.e., external interviews and a survey were not conducted). This had the potential to limit the evidence available for the evaluation. The methodology was based on a risk-based approach where a reduced level of effort for Health Canada and the Public Health Agency of Canada was deemed appropriate as they represented only 15% of total BSE III resources; as well, there was a recent evaluation of BSE I and II Initiatives conducted for Health Canada and the Public Health Agency of Canada (July 2013), and activities have been substantially the same since this last evaluation. The mitigation strategy and evaluation approach met the needs of the evaluation and served to update the evaluation issues and findings from the previous evaluation and provide current evidence in line with the evaluation approach defined by CFIA.
There were some limitations in the performance and financial data available for the evaluation. Specific data was sought against the pre-defined performance measurement framework and allocated budgets. Gaps were identified and filled by Program staff and additional context was obtained through interviews. Quantitative and qualitative data was provided against all performance indicators; however, they may not be fully comprehensive and specific financial details were lacking. As such, the findings and conclusions, particularly regarding efficiency, limited the evaluation's ability to fully demonstrate effectiveness and efficiency.
The relatively small number of interviews, and the fact that they were all internal interviews, may have provided a bias in interview findings. To support the views provided in the interviews, evidence was requested to minimize potential bias. As well, views from CFIA were sought to corroborate Health Canada and Public Health Agency of Canada views. The mitigation measures minimized the potential impacts. However, caution should be used in generalizing the findings beyond what was specifically found.

8.0 Key Findings in Health Canada and the Public Health Agency of Canada

8.1 Relevance: Continued Need for the Program

Given the potential risks that BSE continues to pose to the health of Canadians, the long-term nature of these risks, and the fact that gaps continue to exist in terms of scientific understanding of the disease and its potential transmission pathways, there is a continued need for the Health Canada and Public Health Agency of Canada components of the BSE Management Program.

Both the document review and interviews conducted as part of this evaluation confirmed the ongoing need for both Health Canada and the Public Health Agency of Canada's components of the BSE Management Program.

The BSE Management Program has been in place since 2003, when a case of BSE was first discovered in Canada. One of the key purposes of the first BSE strategy was to safeguard the health of Canadians by minimizing their risk of exposure to BSE. While significant progress has been made in this regard (e.g., by removing SRM agents from the human food supply), other potential exposure pathways exist that continue to require research, risk assessment, tracking and surveillance. The 2012 Health Canada and Public Health Agency of Canada Summative Evaluation of BSE I and II highlighted a number of features of TSEs, including BSE, that remain not well understood with many areas of uncertainty regarding public health. These areas underscore a strong, ongoing need for a human health focus with respect to ongoing surveillance and supporting research activities, for exampleFootnote 119:

In addition, there continue to be significant scientific uncertainties related to the biology of TSEs and their diverse risks for human health. These are exacerbated by the diseases' long incubation periods, which in turn necessitate research projects that last longer than many other such initiatives. The evidence suggests that vCJD is evolving and may manifest itself differently than it did in earlier cases. This too emphasizes the need for an active, ongoing and engaged presence on the part of Health Canada and the Public Health Agency of Canada.

8.2 Relevance: Alignment with Government Priorities

The Health Canada and Public Health Agency of Canada components of the BSE Management Program continue to be consistent with the priorities and mandates of both organizations as they pertain to the protection of human health in Canada. These components contribute to the ability of other organizations to deliver on their responsibilities related to BSE.

The Public Health Agency of Canada was created to help deliver on the Government of Canada's commitment to public health in order to protect and improve the health and safety of all Canadians. In particular, it has a mandate to mobilize pan-Canadian action to prevent disease and injury and promote and protect national and international public health, through surveillance, use of tools, information sharing and building and sustaining public health networks with stakeholders. The Public Health Agency of Canada components of the BSE Management Program are consistent with this mandate and the Agency's focus on the prevention of infectious diseases (including food-borne illness) through surveillance, control measures, advice to regulators and policy-makers, and world-class science and research.

The Public Health Agency of Canada components of the BSE Management Program contribute specifically to the Agency's overall Strategic Outcome, which is "Protecting Canadians and empowering them to improve their health", as well as to Program 1.2, 'Health Promotion and Disease Prevention'. In particular, BSE activities contribute to Sub Program 1.2.1 (Infectious Disease Prevention and Control), and Sub-sub Program 1.2.1.3: Food-borne, Environmental and Zoonotic Infectious Diseases.

As expressed in its 2013-14 RPP, the Public Health Agency of Canada aims to undertake "common primary public health functions of health promotion, surveillance, science and research on diseases and associated risk and protective factors to inform evidence-based frameworks, strategies, and interventions". The Public Health Agency of Canada components of the BSE Management Program, which include national surveillance and research for human TSEs, including vCJD, are consistent with these priorities. The surveillance data these activities generate also provides critical evidence to confirm the effectiveness of risk control measures (such as removal of SRM from the human food supply), which in turn provides a crucial underpinning of the entire BSE Management Program and other departmental priorities, such as CFIA's mandate to ensure a healthy agricultural sectorFootnote 121.

Health Canada is responsible for the establishment of food quality and safety standards in Canada, as well as for ensuring the safety, quality and efficacy of human and animal pharmaceuticals, natural health products, vaccines, human blood, blood products, and cell-based therapies as well as cosmetics.

Health Canada fulfills these mandates through both the Food and Drugs Act and associated regulations, and the Department of Health Act. The Health Canada components of the BSE Management Program – which includes research and risk assessment activities regarding human exposure to BSE and other TSEs - are fully consistent with these statutory and regulatory responsibilities.

The Health Canada components of the BSE Management Program are also consistent with departmental priorities, as expressed by the department's Program Activity Architecture (PAA) and the 2013-14 Report on Plans and Priorities (RPP). In particular, these activities contribute to PAA Strategic Outcome 2: "Health risks and benefits associated with food, products, substances and environmental factors are appropriately managed and communicated to Canadians" and PAA Program 2.2, Food Safety and Nutrition.

8.3 Relevance: Alignment with Federal Roles and Responsibilities

There is a legitimate and necessary role for the federal government in the BSE Management Program from a public health perspective. The Health Canada and Public Health Agency of Canada components of the Program help the federal government fulfill its responsibilities related to human health while also contributing to broader efforts to manage the risks posed by BSE. The federal government has a unique role and responsibility to participate in international public health efforts related to TSEs.

Both the document review and interviews confirmed that the Health Canada and Public Health Agency of Canada components of the BSE Management Program remain consistent with overall federal roles and priorities. Protecting the health and safety of Canadians remains a core government priority, as expressed in successive government Budgets and Speeches from the Throne. For example, while BSE is not referenced specifically, the 2013 Speech from the Throne emphasized the ongoing importance of strengthening Canada's 'world-class food safety system'. Similar high-level priorities related to food safety and health are expressed in the 2010 Speech from the Throne and Canada's Economic Action Plans (2011 and 2012).

Furthermore, the Health Canada and Public Health Agency of Canada components of the Program are consistent with federal legislative and regulatory frameworks related to human health, infectious disease and food safety. Because these activities help ensure that the overall BSE Management Program is effectively managing the risk posed by BSE, they also contribute to federal responsibilities related to the health of the agricultural sector and its access to export markets. The information generated through these activities is expected to play a key role in the success of any future application to the World Organisation for Animal Health (OIE) to transition Canadian beef from "controlled BSE risk" to "negligible BSE risk" status.

The federal government has a unique role with respect to international efforts to promote and protect public health. The contribution made by the Health Canada and Public Health Agency of Canada components of the BSE Program to international scientific and risk management efforts related to BSE and other TSEs is a reflection of this legitimate and necessary role.

8.4 Performance: Achievement of Expected Outcomes

8.4.1 To what extent have the expected outputs and immediate outcomes been achieved?

Immediate outcome #1: Increased expertise and knowledge of BSE/TSE science, risks and product surveillance (Health Canada)

There has been ongoing risk assessment consisting of tracking and tracing, product assessment and the development of policies and guidelines related to BSE/TSE, which has been integrated into routine Directorate activities. There has been increased expertise and knowledge of BSE/TSE science and risks over the last 4 years due to Health Canada supported research.

Evidence on progress made towards achieving the outputs and outcome is based on documentation provided by Health Canada, supplemented by information from interviewees. Evidence relates to the two core areas of service provided for BSE in Health Canada, each of which are detailed below:

  1. Risk Assessment; and
  2. Targeted Research.
I. Risk Assessment:

Ongoing work to minimize Canadians' health risks from BSE focuses on the following activities: tracking and tracing (surveillance) of various health products; product assessment; risk assessment studies and the development of policies and guidelines. This work is conducted in a variety of Directorates across Health Canada that conduct risk assessment as a routine aspect of product review and approval. This process screens for animal or human sourced materials in, for example, pharmaceuticals, natural health products, human vaccines, human blood and blood products and cosmetics.

Tracking and Tracing:

This activity consists of the identification and ongoing surveillance of bovine-sourced materials that may be used as ingredients in food, biological/therapeutic products, drugs for human or animal use, natural health products, medical devices or cosmetics.

This activity has been ongoing without fundamental changes over the evaluation period; however changes are currently being implemented aimed at increasing the efficiency of the surveillance process through the use of more advanced software systems. Since this project has yet to be completed, there is currently no data to assess any resulting changes in efficiency.

Product Assessment:

This activity consists of the ongoing, systematic review of priority food and health products with the potential to present a risk to the health of Canadians, due to bovine-sourced material or other ingredients that may pose a BSE infection risk. Sponsors are required to identify animal-sourced ingredients in new products as part of the product application process. Health Canada has submission requirements and forms requiring manufacturers to identify and provide information on animal-sourced ingredients in their products. The information captured is the main way in which Health Canada tracks and traces information.

This activity has been ongoing without fundamental changes over the evaluation period.
Table 13 below provides an indication of the level of activity in Health Canada, including the review of master files and product licence applications which contain ingredients that are sourced from animals which are susceptible to TSE.

Table 13: Health Canada File/Product Assessment Activities 2009-10 to 2012-13
Fiscal Year Drug Identification Number (DIN) to biologic and genetic products which contained bovine-sourced ingredients Drug Identification Number (DIN) to therapeutic products which contained bovine-sourced ingredients Number of master files or product licence applications reviewed which contain ingredients where the ingredients may be at risk of TSE/BSE contamination

Biological, Genetic & Therapeutic Products

Number of master files or product licence applications reviewed which contain ingredients where the ingredients may be at risk of TSE/BSE contamination

Natural Health Products

2009-10 8 42 9 Master Files 4,966
2010-11 7 181 15 Master Files 2,448
2011-12 6 219 17 Master Files 1,968
2012-13 6 310 22 Master Files 2,346
Risk Assessment Studies, Guidelines, Partnerships and Consultations:

This activity involves risk assessments to determine potential prion disease risks to humans posed by identified food and health products and ingredients. This activity includes the development of policy, guidelines and protocols to improve the methods of selecting and screening products. Consultations and partnerships also help to inform risk assessment.

Periodic risk assessments and risk communications may be conducted in relation to new or emerging situations or scientific information. The following were reported over the evaluation period:

In terms of policy, in Health Canada the Food Directorate's Policy on Specified Risk Material (SRM) in the Food Supply (2003) has not changed since its implementation. As well, the Biologics and Genetic Therapies Directorate updated the blood donation deferral policy over the evaluation period to respond to the health risk assessment noted above.

In terms of guidelines, the Health Canada BSE I and II evaluation noted that some directorates had not yet published policies and guidance documents for industry pertaining to the reduction of specific BSE/TSE-related risks. Over the current evaluation period, two guidance documents were updated:

However, in other areas (e.g., Therapeutic Products) there is no perceived need for a separate policy or guidance document as notices and instructions have previously been sent out to industry asking them to supply information such as specific animal sources, tissues used and country of origin. Policies are not developed for every potential contaminant, and the consideration of BSE as a potential contaminant is one among many managed as part of the overall risk assessment process.

Current submission application forms for therapeutic drugs, biologics and veterinary drugs require sponsors to provide information on the source, the tissue or fluid of origin, the age of the animal and the country of origin. However, complete information on animal-derived ingredients in drug products currently on the market is not yet available. For example, 58% of all marketed DIN have supplied information, as have 33% of all Drug Master Files (as of 2012-13). Of note, the data collected by Health Canada is that reported by the manufacturer and thus the department has to rely on these self-reports as providing accurate and complete information.

To continually inform and contribute to risk assessment, Health Canada has a number of ongoing partnerships, including ongoing work through a MOA (Memorandum of Agreement) / LOU (Letter of Understanding) / Contract with the United States Department of Agriculture, and continued scientific and consultative partnerships as a member of Canada's national and international delegations for BSE and TSEs. For example, in 2010-11, there were international collaborative efforts to harmonize strategies for managing BSE and vCJD risk in biologics.

Specific consultations also help to inform risk assessment in Health Canada. In 2011-12, Health Canada held nine external consultations related to BSE including atypical BSEs and their pathogenesis, detection, risk assessment, SRM as defined for food and feed, and the safety of Canadian beef and beef products. Additionally, CWD and scrapie were also consulted on to a lesser degree but followed the same approach as for BSE. All consultations focused on scientific findings, and their interpretation directed to improve and maintain human and animal health and to seek harmonisation with respect to BSE and other TSE policies based on science.

II. Research:

Targeted research is conducted in Health Canada in order to: improve the understanding of BSE pathogenesis; increase the capacity to identify exposure risks through food and health products; improve standardized diagnostics; develop a better understanding of BSE and other TSE zoonotic potential; and mitigate potential public health risks from BSE and other animal TSE diseases.

Over the evaluation period, Health Canada undertook a number of collaborative research projects. These are listed below along with the scientific and technical outputs that resulted from each project:

Table 14: BSE/TSE Research and related outputs supported by Health Canada
(2009-10 to 2012-13)
Targeted Research Projects Health Canada Funds Reported Scientific and Technical Outputs
Food and Nutrition Projects
  • Data collection to populate the Canadian Animal Health Surveillance Network (CAHSN)Table Note 4 data system with BSE surveillance information, under the LOU between Health Canada, Public Health Agency of Canada and CFIA (2009-2013).

$1.8M

  • Compilation of BSE test results from cattle from all laboratories (including provincial) conducting BSE testing across the country (100% complete by 2011-12)
  • Addition of other TSE results (scrapie and CWD) to database following stakeholder consultation and needs assessment (40% complete for scrapie and CWD in 2012-13)
  • Completion of change management process, data verification processes, training, data reporting and communications to meet BSE surveillance personnel needs
  • Oral Transmission and Tissue Infectivity Study for All Types of BSE Diagnosed in Canadian Cattle, under LOU between Health Canada and the CFIA (2009-2013).Table Note 5

$1.0M

  • 62 presentations/ discussions of project/ project data
  • 4 external collaborations
  • 6 student training sessions
  • 7 posters and abstracts
  • 12 oral presentations and abstracts
  • Experimental CWD and BSE in domestic cats: comparative oral transmission studies, under the LOU between Health Canada and CFIA (2009-2013).

$0.60M

  • 1 poster and abstract
  • Determination of tissue distribution of and ability to detect scrapie in young lambs, under LOU between Health Canada and CFIA (2012-2014).

$0.16M

  • No outputs yet as project started in 2012
Canadian and Foreign Projects
  • TSE studies in non-human primates and the implications for human health, under the signed MOA between Health Canada and the Commissariat à l'énergie atomique et aux énergies alternatives, France (2009-2013).

$2.3M

  • 3 peer-reviewed publications
  • 8 posters
  • 6 oral presentations
  • 6 invitations to conferences/workshops
  • Detailed analysis of abnormal prion protein (PrPRes) and infectivity detection in materials from cattle incubating classical or atypical BSE in order to achieve a more substantiated specified risk material (SRM) definition, under signed MOA between Health Canada and Friedrich-Loeffler-Institute, Germany (2009-2013).

$1.4M

  • 13 peer-reviewed publications
  • 10 oral presentations
  • 5 poster presentations
  • 7 European Food Safety Authority (EFSA) Journal publications
  • EFSA BSE TSE Network of National Focus Points: Annual meetings in 2009, 2010, 2011
  • EFSA Technical Report
Biologics and Genetics Projects
  • 7. Analysis of the structure of the prion protein by NMR: Determination of the mechanism of prion insertion into membranes (2012-2013).Table Note 6

$0.34M

  • 3 oral presentations
  • 3 poster presentations
  • 1 peer-reviewed publication

Table Notes

Table Note 4

The Canadian Animal Health Surveillance Network (CAHSN) has established a partnership of federal, provincial, territorial and academic animal health agencies and laboratories, with the objective of strengthening laboratory linkages to facilitate an expeditious response to animal disease emergencies and emerging issues. A primary component of this initiative is to develop an IT infrastructure to link laboratories and create a central data repository. This data repository is housed within a secure, web-based application in the Canadian Network for Public Health Intelligence (CNPHI). Data from the repository are available to stakeholders for further analysis and review provided the relevant legislation and agreements around data sharing and privacy are adhered to.

Return to table note 4 referrer

Table Note 5

Request for extension to this research has been made to account for long BSE incubation times in cattle.

Return to table note 5 referrer

Table Note 6

This research supports assessment of cellular and tissue fractions in biological materials which may transmit the TSE/BSE agent.

Return to table note 6 referrer

In addition, results of these projects have been presented by Health Canada at various formal and informal venues. Formal and informal presentations have included, upon request, presentations of results to CFIA, AAFC, APRI, PrioNet Canada, Beef Value Chain Roundtable, and international governments and NGOs. Table 15 compiles and summarizes the reported formal scientific and technical outputs by year from the above Health Canada funded research projects (may not be inclusive of all activities).

Table 15: Summary of reported scientific and technical outputs
from Health Canada/Health Canada funded studies
Types of outputs 2009-10 2010-11 2011-12 2012-13 Total
Peer-reviewed publications 2 6 7 15
Invited/oral presentations 7 18 14 27 66
Poster presentations at international scientific meetings 4 2 10 5 21
European Food Safety Authority publications 1 2 3 1 7

In addition, Health Canada partnered with PrioNet Canada and the Alberta Prion Research Institute to host the world's largest international prion research conference, PRION 2011, in Montreal, Quebec from May 16-19, 2011. This was the first time this conference was held outside of Europe. It welcomed over 600 attendees from around the world, including representatives from CFIA, DFAIT, AAFC, provincial and international government agencies. Health Canada both sponsored and participated in this meeting, with one Prion training session held together with the meeting.

Immediate outcome #2: Increased information, expertise and knowledge through TSE science and surveillance, leading to an improved foundation for evidence-based decision-making (Public Health Agency of Canada)

The Public Health Agency of Canada's knowledge of TSE science and surveillance has increased over the last 4 years due to research and surveillance, leading to an improved foundation for evidence-based decision-making.

Evidence on progress made towards achieving this outcome is based on interviews and documentation provided by the Public Health Agency of Canada. It relates to the three core areas of service provided for BSE:

  1. Surveillance;
  2. Research (Scientific and Technical); and
  3. Laboratory reference services.
I. Surveillance:

The Public Health Agency of Canada Creutzfeldt-Jakob Disease Surveillance System (CJDSS) continues to conduct comprehensive national surveillance of human TSEs (see detailed statistics). All forms of human TSE, sporadic, genetic, and those acquired by infectious transmission, continue to be detected in Canada, as a direct result of the laboratory services and non-laboratory support provided to Canadian health professionals by the CJDSS.

Tables 16 and 17 summarize key features of surveillance data on human TSEs in Canada collected by the Public Health Agency of Canada during and after the period of evaluation to provide the most up-to-date information (up to November 30, 2013). One crucial finding is that the observed annual incidence of human prion diseases in Canada fell within the range expected according to international CJD surveillance (1–2 cases per million of population per year) each year during the period of review. With regard to zoonotic human TSE, in February 2011, the Agency confirmed a second case of VCJD in a Canadian resident through its surveillance activities, and conducted a full epidemiologic characterization (see Public Health Agency of Canada). The individual was concluded to have acquired the disease outside of Canada; as a result of this finding Health Canada updated its geographically based blood donation deferral policy to include individuals who had resided for at least six months in Saudi Arabia between 1980 and 1996.

Table 16: Definite and probable cases of CJD in Canada, by subtype
(partial 2013 data)
Year SporadicTable Note 7 IatrogenicTable Note 7 GeneticTable Note 7 vCJD Total
2009 48 0 5 0 53
2010 35 0 3 0 38
2011 46 0 3 1 50
2012 60 0 0 0 60
2013 (to Nov 30) 23 0 1 0 24
Total 212 0 12 1 225

Table Notes

Table Note 7

Variant Creutzfeldt-Jakob disease (vCJD) is the only known zoonotic TSE, and results from human exposure to BSE. Classical forms of CJD, which are not known to be zoonotic in origin, are classified into three main types. Sporadic CJD, which occurs without a specific traceable genetic or infectious cause, is thought to arise spontaneously through abnormal processing of the prion protein (PrP) in tissues. Genetic CJD is caused by heritable mutations in the gene that codes for PrP. Iatrogenic CJD has occurred through accidental exposure of patients to TSE infectivity during medical procedures.

Return to table note 7  referrer

Table 17: Incidence of CJD deaths in Canada, by year of death
(partial 2013 data)
Year of Death Total cases Population of Canada Incidence (per million)
2009 53 33,739,814 1.57
2010 38 33,841,500 1.12
2011 50 34,019,075 1.47
2012 60 34,482,779 1.74
2013 (to Nov 30) 24 34,880,491 0.75

As part of the surveillance efforts, the Public Health Agency of Canada responds to referrals of suspected cases of CJD. The figure below illustrates, on a yearly basis from 1994 to 2012, the notifications of suspected CJD, referrals (i.e., full case investigations in which an autopsy is requested) and confirmed CJD cases.

Figure 5: Notifications, Referrals and Confirmed CJD cases in Canada over time. Description follows.
Description for Figure 5: Notifications, Referrals and Confirmed CJD cases in Canada over time

This figure is a line graph that shows the number of notifications, full investigations and confirmed cases of CJD over time. At the top of the graph is a legend with the following information: a circle with a white interior represents notifications, a square with a white interior represents full investigations and black circles represent confirmed cases. Also, on the legend is a dashed line that represents a three year average. On the vertical axis are number starting at zero and going to two hundred and fifty going in intervals of fifty. On the horizontal axis are years starting at 1994 and ending with 2012. The values below are approximates.

Confirmed Cases

Below are the data points for the confirmed cases (represented by black circles)

  • 1994: 0
  • 1995: 0
  • 1996: 10
  • 1997: 15
  • 1998: 20
  • 1999: 30
  • 2000: 32
  • 2001: 30
  • 2002: 33
  • 2003: 28
  • 2004: 40
  • 2005: 38
  • 2006: 40
  • 2007: 35
  • 2008: 42
  • 2009: 45
  • 2010: 33
  • 2011: 45
  • 2012: 50

The black circles are connected using a solid black line and there is a dashed line that runs close to the data points

Full Investigations

Below are the data points for the full investigations (represented by squares with white interiors)

  • 1994: 0
  • 1995: 0
  • 1996: 0
  • 1997: 1
  • 1998: 40
  • 1999: 60
  • 2000: 80
  • 2001: 101
  • 2002: 105
  • 2003: 75
  • 2004: 82
  • 2005: 100
  • 2006: 75
  • 2007: 105
  • 2008: 102
  • 2009: 110
  • 2010: 75
  • 2011: 107
  • 2012: 110

The squares with white interiors are connected using a solid black line and there is a dashed line that runs close to the data points

Notifications

Below are the data points for the notifications (represented by circles with white interiors). Data starts at 2004.

  • 2004: 162
  • 2005: 150
  • 2006: 140
  • 2007: 179
  • 2008: 212
  • 2009: 231
  • 2010: 198
  • 2011: 235
  • 2012: 233

The circles with white interiors are connect using a solid black line and there is a dashed line that runs close to the data points.

Table 18 summarizes the numbers of autopsies conducted on suspected cases of CJD since 2008 (note that data from 2013 is not yet complete):

Table 18: Autopsies on Suspected Cases of CJD by the Public Health Agency of Canada
(partial 2013 data)
Year # of Autopsies
2008 57
2009 75
2010 46
2011 54
2012 69
2013 50

In each case, the process occurs in several phases:

  1. A contracted funeral transport service provider transports the deceased person's remains from the place of death to a hospital pathology laboratory.
  2. A local pathologist working in the hospital laboratory performs the primary autopsy, in which the brain tissue is recovered.
  3. The contracted funeral transport service provider transports the deceased person's remains from the pathology laboratory to a funeral facility (usually for cremation).
  4. The brain tissue is shipped to the University of Ottawa for expert neuropathological examination, from which the Public Health Agency of Canada obtains the final case diagnosis.

The total cost of this process varies considerably from case to case, depending on factors such as driving distance, degree of prior preparation of the tissue before it gets to the University of Ottawa, etc. However, a rough average cost per case is about $5,000.

II. Research (Scientific and Technical):

Research continues with the goal of developing improved methodologies for detection and diagnosis of human and animal prion disease. For example, the Public Health Agency of Canada Prion Diseases Program has discovered potential urine-based biomarkers for BSEFootnote 122; and the CJDSS has completed a six-year prospective study of the accuracy of biomarkers in cerebrospinal fluid for diagnosis of human prion disease.

Table 19 summarizes the scientific and technical outputs from the Prion Diseases Program in the Public Health Agency of Canada:

Table 19: Summary of scientific and technical outputs from the
Public Health Agency of Canada
Types of outputs 2009-10 2010-11 2011-12 2012-13 Total
Peer-reviewed publications 2
(54 citations)
3
(37 citations)
5
(20 citations)
5
(2 citations)
15
Invited presentations 7 13 12 2 34
Poster presentations at international scientific meetings 6 7 2 8 23
Peer recognition Chair of Session at TSE Signatures
TSE laboratory test methodologies presented/ published 5
(9 citations)
2 7
III. Laboratory Reference Services:

Table 20 provides a summary of laboratory reference-services reports delivered by the National Microbiology Laboratory and as reported by the Public Health Agency of Canada. The Cerebral Spinal Fluid (CSF) protein-marker test panel and the genetic analysis are accredited under ISO 17025 (accreditation was achieved within the last four years).

Table 20: Reference services outputs from
the Public Health Agency of Canada
Types of outputs 2009-10 2010-11 2011-12 2012-13
CSF 234 201 241 237
Genetics (blood) 18 17 32 46
Genetics (tissue) 5 22 21 32
Glycoforms 4 57 56 78
Autopsies 75 46 54 69
Total 336 343 404 462

To support reference services, over the last few years, the Public Health Agency of Canada has entered into an international consortium (UK, Italy, Germany, Japan, US) to develop standardized methods for diagnostic tests. The focus is on the QuIC (quaking-induced conversion) test for CJD in Cerebral Spinal Fluid (anti-mortem). Once completed, this test can provide further validation of the disease and be used internationally for surveillance purposes.

8.4.2 To what extent have the intermediate outcomes been achieved?

Intermediate outcome #1: Increased knowledge-based decision making (Health Canada)

Intermediate outcome #2: Scientifically informed policies and decisions to deal with the public health risks posed by human TSEs in Canada (Public Health Agency of Canada)

Knowledge acquired through surveillance and research activities has influenced decision-making and has had an impact on the development of scientifically-informed policies (e.g., changes to the blood donation deferral policy).

A key example emerged over the evaluation period to illustrate how surveillance and research results have influenced health-based decision-making and policy. In 2011, Health Canada followed-up on reports of an individual with vCJD infection from surveillance data collected by the Public Health Agency of Canada in order to assess potential risks to the blood supply and to determine whether or not changes should be made to the blood donor screening criteria. The Public Health Agency of Canada identified a patient in Canada with vCJD as having contracted the disease in Saudi Arabia. Health Canada then expanded the policy to defer blood donations from all individuals who had spent a cumulative six months or more between 1980 and 1996 in Saudi Arabia.

In addition, progress has been made in providing access to BSE test results from laboratories across Canada, which should provide improved information for decision-making in Health Canada. By 2012-13, the Canadian Animal Health Surveillance Network had stated that data reporting and communications that met the needs of the BSE surveillance personnel and decision makers were virtually complete, with regular reports being provided to Health Canada, CFIA and others. By incorporating TSE/BSE surveillance data into CAHSN, data users are able to access provincial and national surveillance information to assist in their reporting, disease control and risk assessment activities. A fully integrated surveillance system will streamline TSE reporting and help provide data to facilitate risk and decision analyses. The following figure illustrates usage of CAHSN data related to BSE.

Figure 6: Usage of CAHSN BSE module from January 2011 to October 2012. Description follows.
Description for Figure 6: Usage of CAHSN BSE module from January 2011 to October 2012

This figure is a line graph that shows the number of unique users and total accesses of the CAHSN BSE application by month. At the top of the figure there is a legend, red triangles represent unique users and blue squares represent total accesses. There are numbers on the vertical axis starting with 0 and going to 200 in intervals of 20, there is a horizontal line across the graph marking each interval. The horizontal axis shows the month and year starting with January 2011 and ending with November 2012. The values below are approximates.

Unique Users

Below are the data points for the unique users (represented by red triangles).

  • 01-2011: 4
  • 02-2011: 3
  • 03-2011: 5
  • 04-2011: 5
  • 05-2011: 4
  • 06-2011: 3
  • 07-2011: 3
  • 08-2011: 4
  • 09-2011: 17
  • 10-2011: 18
  • 11-2011: 18
  • 12-2011: 14
  • 01-2012: 14
  • 02-2012: 15
  • 03-2012: 21
  • 04-2012: 19
  • 05-2012: 15
  • 06-2012: 13
  • 07-2012: 15
  • 08-2012: 18
  • 09-2012: 16
  • 10-2012: 12

There is a red line connecting these data points

Total Accesses

Below are the data points for total accesses (represented by blue squares)

  • 01-2011: 34
  • 02-2011: 28
  • 03-2011: 41
  • 04-2011: 26
  • 05-2011: 25
  • 06-2011: 16
  • 07-2011: 4
  • 08-2011: 35
  • 09-2011: 177
  • 10-2011: 177
  • 11-2011: 146
  • 12-2011: 91
  • 01-2012: 133
  • 02-2012: 136
  • 03-2012: 163
  • 04-2012: 143
  • 05-2012: 122
  • 06-2012: 102
  • 07-2012: 153
  • 08-2012: 166
  • 09-2012: 112
  • 10-2012: 172

There is a blue line connecting these data points.

However, while progress has been made with CAHSN, interviewees indicated that data is not yet fully able to provide uniform, accurate and complete BSE/TSE surveillance information for decision-making. Work remains to validate and provide more confidence in the data before it can be fully used for risk assessment purposes.

In addition, as identified in the previous evaluation, efforts are ongoing to develop a mechanism for information-sharing between Health Canada and the CFIA. Such agreement mechanism would enable Health Canada to access CFIA compliance data, which it requires to fully understand industry compliance with the SRM removal policy and update its risk assessments of food products accordingly.

Finally, the variety of investments made over the last four years in Health Canada and the Public Health Agency of Canada to enhance BSE research knowledge has the potential to influence decision-making when complete (as noted previously, research requires a longer term investment time due to the incubation period for BSE). Research findings have the potential to both confirm that the current BSE management regime is adequately protecting human health and/or also provide the rationale to adapt the management regime and develop new risk management options. Key areas of future research noted by interviewees included:

8.4.3 To what extent has the longer term outcome been achieved?

Longer term outcome: Protection from preventable health risks related to food safety and product safety or the transmission of animal diseases to humans (Health Canada and Public Health Agency of Canada)

Health Canada's BSE/TSE risk management measures appear to provide a comprehensive framework to reduce risks and ensure safer food and health products. The Public Health Agency of Canada's Creutzfeldt-Jakob Disease Surveillance System (CJDSS) provides ongoing evidence to demonstrate the low risk of Canadians acquiring this disease from possible past exposure to domestic sources.

The BSE initiatives are intended to reduce exposure to the risks associated with the use of animal-sourced materials in food and health products regulated by Health Canada, and reduce the risk of acquiring human TSEs associated with these ingredients, ultimately leading to safer food and health products. The previous Health Canada evaluation of BSE I and II noted that there was general agreement among both internal and external key informants that Health Canada's risk management and control measures have contributed to achieving these outcomes.

Although, there is insufficient data to definitively prove the long term outcome has been achieved, the following findings emerged from the previous Health Canada evaluation or from this current evaluation:

8.4.4 To what extent is collaboration and coordination with partner organizations and stakeholders adequately integrated in Program design and delivery?

While there were several examples of collaboration and coordination between partner organizations, particularly for research studies, this process is not formally integrated into Program design.

The Health Canada Evaluation on BSE I and II noted that there was limited collaboration and coordination among partners and the absence of a coherent profile for the TSE Secretariat the role of which was to coordinate Health Canada's overall approach to BSE/TSE. Since that time, Health Canada decided to review the role of the TSE Secretariat by March 2014 and to examine alternative measures to support enhancements to information sharing related to BSE amongst all partners. In fact, BSE management was integrated as part of core risk management activities related to various product lines regulated by HC such as health products.

Specifically, the CAHSN project is an example of collaborative efforts to share data for various reasons including informing risk assessments. This project has been underway for a number of years and was designed in part for this reason. The key goal for Health Canada is to have full access to up-to-date and national surveillance data (incidence of BSE in cattle) as well as access to the data on industry compliance with the SRM removal policy. This information helps Health Canada update its risk assessment with respect to food products. As noted in Section 8.4.2, work remains to make CAHSN's BSE data collection fully functional for risk assessment, and data sharing in terms of compliance with the SRM removal policy (feed ban) remains to be realised. As noted in CFIA's findings (Section 4.4.6): "rules and regulations around data sharing restrict some of the coordination with Health Canada (as well as other federaland provincial government departments regarding traceability efforts). Indeed, efforts are ongoing to develop an information-sharing agreement between Health Canada and the CFIA, which requires an action plan with clear milestones and senior management support".

Recently the CFIA began to report to the same minister as Health Canada and the Public Health Agency of Canada for matters related to food safety. This change may also provide enhanced opportunities for collaboration among Health Portfolio organizations.

Health Canada and the Public Health Agency of Canada interviews confirmed that informal and individual professional relationships are active, but that formal collaboration and cooperation mechanisms have not yet been put in place. For example, several of the research projects are done in collaboration with CFIA. As well, the CAHSN project involves the CFIA, Health Canada and the Public Health Agency of Canada. There is active participation in international scientific fora (e.g., CFIA led missions, Beef Value Chain Roundtable Sessions). However, there are no regular and formal opportunities for the partners to exchange information about activities, results, and future needs. Such opportunities would be welcomed by those interviewed and would support integrated Program design and delivery.

8.5 Performance: Demonstration of Economy and Efficiency

Health Canada consistently and increasingly underspent its funding allocation for the BSE Management Program. While performance frameworks are in place, they have never been fully implemented in Health Canada to facilitate economy and efficiency in decision-making.

The Treasury Board of Canada's Policy on Evaluation (2009) and guidance document, Assessing Program Resource Utilization When Evaluating Federal Programs (2013), defines the demonstration of economy and efficiency as an assessment of resource utilization in relation to the production of outputs and progress toward expected outcomes. This assessment is based on the assumption that departments have standardized performance measurement systems and that financial systems link information about program costs to specific inputs, activities, outputs and expected results.

The data structure of the detailed financial information provided for the Program in Health Canada did not facilitate the assessment of whether Program outputs were produced efficiently or whether expected outcomes were produced economically. Considering these issues, the evaluation provided observations on economy and efficiency based on findings from the key informant interviews and available relevant financial data. In addition, the findings below provide observations on the adequacy and use of performance measurement information to support economical and efficient Program delivery and evaluation.

8.5.1 Observations on Economy

Annually, Health Canada has received $6.2M for BSE since 2009-10 and the Public Health Agency of Canada has received $0.8M (similar to BSE II resource levels). In total over five years, Health Canada was allocated $31M for risk assessment and targeted research and the Public Health Agency of Canada was allocated $4M for surveillance and research for TSEs.

The Public Health Agency of Canada spent its full allocation as planned. However, Health Canada consistently underspent its allocation with the amount underspent increasing from 2009-10 to 2012-13 (from 5% to 39% underspending) – see Figure 7.

Figure 7: Planned vs. Actual Spending on BSE in Health Canada. Description follows.
Description for Figure 7: Planned vs. Actual Spending on BSE in Health Canada

This figure is a bar chart that displays planned v.s. actual spending on BSE by Health Canada. The vertical axis displays spending dollars in millions, the vertical axis starts at zero and goes to seven million in intervals of one million. On the horizontal axis is fiscal years starting with 2009-10 and going to 2012-13. To right of the chart is a legend it shows that a blue bar represents planned spending and a red bar represents actual spending. For each fiscal year the blue bar shows 6.2 million dollars of planned spending. Below are the actual spending amounts shown by the red bar:

  • 2009-10: 5.9 million dollars
  • 2010-11: 4.8 million dollars
  • 2011-12: 4.2 million dollars
  • 2012-13: 3.8 million dollar

This figure was generated from information in Table 9 which compared planned to actual spending over the last five years based on DPR Supplementary Tables.

The rationale provided for underspending in Health Canada, as noted in DPRs, included:

The previous evaluation on BSE I and II also noted significant underspending by Health Canada, suggesting that the department may not need this level of resources to conduct the currently prescribed activitiesFootnote 123.

8.5.2 Observations on Efficiency

In terms of efficiency, document review and interviewees provided some observations.

In Health Canada, the risk assessment, product assessment and tracking and tracing activities have been integrated into routine Directorate activities, and represent a small percentage of these activities. Overall, the BSE/TSE component represents less than 20% of product assessment, tracking and tracing activities). As such, the full allocation of resources does not appear to be required on a regular basis. In this, way, activities in Health Canada can be seen as efficiently implemented and likely no longer require the same level of funding.

In terms of research in Health Canada, this has been implemented in a collaborative manner with partners in Canada and internationally that have the expertise required in this specialized area. This approach was seen as efficient as it builds some capacity in Canada for novel research but also works with broader internal networks to lever their infrastructure, capacities and expertise.

In the Public Health Agency of Canada, it was noted that the number of samples being handled in the laboratory have steadily increased, and yet the resources to conduct these samples has remained the same. This would suggest that the efficiency in conducting the analyses has increased.

As well, it was noted that there are efficiencies in having reference and post mortem tests being done in central, specialized facilities by highly trained individuals, rather than having these tests distributed across a variety of laboratories that may not be able to ensure consistency, quality control and economies of scale. These specialized and rare tests require dedicated 'Level 3 containment' and specialized analytical skills that are most efficiently provided in a central location since the number of samples examined each year is relatively small.

8.5.3 Observations on the Adequacy and Use of Performance Measurement Data

As noted in the 2012 Health Canada Summative Evaluation of BSE I and II, there remains some ambiguity regarding responsibilities for performance monitoring and reporting. As a result, although performance frameworks are in place, they have never been fully implemented in Health Canada (i.e., performance data is not systematically collected, reported and used for program decision-making). Some interviewees also suggested that existing performance measurement tools have not adapted over time and as a result, do not fully reflect current scientific and Program needs in Health Canada.

Nevertheless, Health Canada did provide some performance information in the DPR Supplementary Tables annually. However, there were some inaccuracies found in this reporting for and this information was not presented against the outcomes and indicators defined in the "Performance Measurement Strategy (PMS) for the Enhanced BSE Program" (May 2011-Draft). Health Canada programs did compile some performance data to respond to the outcomes and indicators for this evaluation, however, only an incomplete performance record emerged.

In addition, as part of this evaluation, the Public Health Agency of Canada provided detailed information against the outcomes and indicators stated in the PMS.

9.0 Conclusions and Recommendations

9.1 Health Canada and the Public Health Agency of Canada Conclusions

While Health Canada's BSE/TSE risk management measures and the Public Health Agency of Canada's Creutzfeldt-Jakob Disease Surveillance System (CJDSS) provided ongoing evidence to support the low risk of Canadians acquiring BSE/TSE, there is a continued need for the Health Canada and Public Health Agency of Canada components of the BSE Management Program. This is due to the potential risks that BSE continues to pose to the human health of Canadians, the long-term nature of these risks, the fact that gaps continue to exist in terms of scientific understanding of the disease and its potential transmission pathways, and the relevance of the work to the priorities and mandates of both agencies.

There has been ongoing risk assessment, product assessment and tracking and tracing related to BSE/TSE which has been integrated into routine Health Canada Directorate activities. Research and surveillance in both Health Canada and the Public Health Agency of Canada have contributed to increased expertise and knowledge of BSE/TSE and the associated risks. The 2011 changes to the blood donation deferral policy are an example of both how such activities have increased knowledge-based decision making and scientifically informed policies, and of effective inter-agency coordination. Some Health Canada directorates have also published guidance for industry on how to reduce BSE/TSE risks. In addition, ongoing research efforts related to atypical BSE, CWD and human-to-human transmission are seen to be important to provide the information required to confirm or adjust the policy framework for BSE/TSE.

While there were several examples of collaboration and coordination between partner organizations, particularly for research studies, this process is not formally integrated into Program design. There are no regular and formal opportunities for the partners to exchange information about activities, results, and future needs. Such opportunities would support more fully integrated Program design and delivery.

There were a number of examples to demonstrate efficient Program delivery in both Health Canada and the Public Health Agency of Canada. While the Public Health Agency of Canada spent its full allocation as planned, Health Canada consistently and increasingly underspent its funding allocation as risk management was integrated into regular assessment and surveillance activities. As a result, the funds allocated exceeded what was required to manage the BSE Program. While performance frameworks are in place, at Health Canada they have never been fully implemented and are not seen by Program staff as fully relevant to current scientific and Program needs. The Public Health Agency of Canada did provide detailed information against the outcomes and indicators stated in the PMS. The BSE I and II evaluation made similar observations about performance measurement and financial tracking at Health Canada and included recommendations to address these issues.

9.2 Health Canada and Public Health Agency of Canada Recommendations

Recommendation 5

In addition to the ongoing role in BSE/TSE risk assessment and research to inform policy and regulatory development, Health Canada and the Public Health Agency of Canada should work with CFIA to institute regular opportunities to exchange information on Program activities and results in order to inform future work and facilitate collaborative activities.

The evaluation found that risk assessment has been integrated into the routine product assessment and surveillance activities, and that collaborative and agency-specific research is key to informing future Program directions. Continuation of these efforts will ensure the risk management framework for BSE/TSE is appropriate and relevant. Research activities undertaken by Health Canada have resulted in a significant addition to the scientific knowledge associated with BSE/TSE to support improved risk assessment and diagnosis capacity. It is important that the scientific output gathered to date be further analyzed, upon completion of the ongoing research projects, to support updates of guidance and risk management measures associated with BSE/TSE in food.

There remains an ongoing need to maintain regular and clear communication channels between the departments to share not only research results, but also surveillance and compliance data that can influence risk assessment measures. Finally, as performance and financial information has not been systematically reported and used to inform Program delivery, greater communication and coordination is required to review what has been done by all parties and what resources continue to be required to meet Program objectives.

10.0 Appendices

10.1 Appendix A – Overview of CFIA BSE Activities/Strategies

The following is an overview of the six CFIA-led activities that were evaluated under three thematic areas. This includes the two Health Canada and Public Health Agency of Canada-led activities that were evaluated under two thematic areas.

10.1.1 Thematic Area A – Managing BSE-Related Risks

Measures to manage BSE risks aim to effectively minimize the likelihood of exposure, amplification, and spread of BSE within the cattle population and to protect consumers from the associated human health risks. These measures prevent BSE-infected material from entering the human food chain and ultimately contribute to objectives to protect the public from preventable health risks. Additional measures to prevent BSE-infected material from entering the animal food chain – the enhanced feed ban (EFB) – are not covered in this evaluation, but these remain critical to managing BSE-related risks in Canada.

Activity A1: Removal of SRM from the human food supply ($9.5M/year planned)

This activity aims to eliminate the possibility of exposure of Canadians to BSE through the removal of SRM (the tissues where the vast majority of the BSE infective agent is concentrated in infected animals) from all cattle slaughtered in Canada to ensure that this material does not enter the human food supplyFootnote 124. CFIA inspection staff (Operations Branch) conduct verifications and enforcement activities to ensure compliance of industry with these regulations in federally registered establishments, where 95% of Canada's cattle are slaughteredFootnote 125. Non-federally registered establishments are under the responsibility of provincial governments. CFIA provides technical support for the implementation and compliance verification of SRM requirements (e.g., provides verification checklist, directives, conducts joint inspections/audits with provincial staff)Footnote 126. These activities include input from the Policy and Programs Branch (e.g., risk-based determination of the frequency of inspections).

Activity A2: Import controls ($0.3M/year planned)

This activity aims to prevent the entry into Canada of animals or products that could carry BSE. Import controls are achieved by verifying that countries from which Canada imports products have appropriate risk mitigation measures that are consistent with Canada's standards for the protection of human and animal health. This involves maintain and updating current import policies and conditions for BSE to reflect changes in international standards and science (e.g., those outlined by the OIE), as well as consultations with affected Canadian industries and international trading partners. CFIA (Policy and Programs Branch; Operations provide inspection activities for verification and enforcement) is responsible for import compliance with respect to food products. In this regard, the CFIA also works with Health Canada (responsible for import compliance with respect to health products) and other partners (e.g., Canadian Border Services Agency [CBSA], with which the CFIA operates the National Import Service Centre).

Activity A3: Risk Assessment and Target Research Regulated Programs - Health Canada ($6.2M/year planned)

This activity aims to address evolving science, uncertainty, and emerging findings linking BSE and other animal TSEs to human health outcomes, and will permit engagement of domestic and international discussions on BSE and TSE science, and regulatory topics. Risk assessment and research activities will support domestic an international collaboration on regulatory and policy decisions.

10.1.2 Thematic Area B – Monitoring BSE-Related Risks

Risk monitoring activities assess the effectiveness of the risk management measures and assist in responding when BSE cases occur, by helping to trace and mitigate the domestic spread of BSE.

Activity B4: BSE surveillance (15.4M/year planned)

This activity monitors the level of BSE in cattle in order to measure the effectiveness of Canada's BSE control measures in preventing the spread of BSE in the Canadian cattle population. A BSE surveillance program is required by the OIE as part of a suite of measures to prevent and control BSE in accordance with the country's BSE risk statusFootnote 127: the CFIA submits surveillance information to the OIE each year, including an analysis to demonstrate that an appropriate level of surveillance is in place. Canada's surveillance program focuses on higher-risk animals such as those displaying clinical signs consistent with BSE and specific sub-populations (4D: dead, dying, diseased and downer) and based on a target set by the CFIA of 30,000 samples/year.

This program is delivered through a collaborative effort involving federal and provincial governments (e.g., Chief Veterinary Officers) and facilities (i.e., laboratories), veterinary practitioners and industry stakeholders. A federal sample reimbursement program is in place to increase the number of samples received from veterinary practitioners, producers and dead stock operators to offset their costs associated with the sampling proceduresFootnote 128. Within the CFIA, surveillance activities primarily involve the Science Branch (Animal Health Science Directorate). Surveillance sampling activities are also conducted in some provinces by CFIA inspectors and veterinarians (Operations Branch).

Activity B5: Cattle identification ($2.8M/year planned)

Cattle identification provides the means to assist in tracing animals, such as in the event of an investigation of suspected or confirmed cases of BSE (or other disease outbreaks). The cattle identification program is administered by the Canadian Cattle Identification Agency and Agri-Traçabilité Québec, while the CFIA's involvement includes managing the policy and regulatory framework, enforcing the regulations and conducting the BSE outbreak investigationsFootnote 129. With regard to enforcement, the CFIA monitors the premises associated with the movement of cattle (e.g., auctions, federally and provincially inspected abattoirs and dead stock collection sites) to verify that cattle are identified with an approved ear tag, and take enforcement actions when appropriate. The CFIA's Traceability Unit is within the Policy and Programs Branch, while the inspections (compliance verification and enforcement) are delivered by the Operations Branch.

Activity B6: Surveillance and Research for Human TSEs – Public Health Agency of Canada ($0.8M/year planned)

This activity aims to ensure national surveillance and research to human TSEs through the CJDSS. High quality surveillance data on the incidence and character of human TSEs, along with surveillance data on the incidence of BSE I n cattle, provide unique scientific evidence that measure to mitigate risk of TSE transmission (such as SRM removal from the human food chain) remain effective.

10.1.3 Thematic Area C – Communicating Canada's Program

The CFIA promotes and explains Canada's comprehensive BSE measures to stakeholders in order to support the maintenance and expansion of market access for Canadian cattle and cattle products. These measures require the services of experts in BSE and risk assessment, and involve highly technical dialogue with training partners. The CFIA collaborates with the Market Access Secretariat (MAS, housed at AAFC) and DFATD for these activities.

Activity C7: Export certification ($5.7M/year planned)

The CFIA provides export certification services to a wide range of industry sectors, such as producers and other establishments that export to foreign countries, by verifying compliance with BSE-related import conditions imposed by trading partners. Note that these conditions apply to a broad range of products and vary between countries. CFIA veterinarians (Operations Branch) also issue certificates based on the outcome of these inspections.

In addition, the CFIA provides support for bilateral technical negotiations around regulatory import requirements with the goal of further expanding market access and minimizing the number of technical BSE-related conditions that have to be met by industry and certified by CFIA inspectors. The also CFIA develops and certifies quality assurance programs that facilitate opportunities to introduce a broader range of products to the domestic and international marketplace. These activities are mainly delivered by the Policy and Programs Branch.

Activity C8: Technical market access support ($5.0M planned)

To provide technical market access support regarding BSE controls, the CFIA conducts risk assessments, reviews and amends domestic policies to reflect evolving science and lessons learned, and contributes to science in support of regulatory decision-making. Canadian animal health experts from the CFIA also help maintain strong relationships with trading partners and provide global leadership regarding international policies and standards developed by organizations such as the OIE. As part of these efforts, veterinarians are placed in key posts abroad to help build these relationships and facilitate foreign audits of Canada's BSE controls. Technical experts from the CFIA also meet with foreign delegations when the conduct these audits. These various activities are delivered by the Policy and Programs Branch, with support from the Science Branch.

10.2 Appendix B – Data Collection Matrix

The table below presents the draft Data Collection Matrix (DCM) for the CFIA's evaluation, which cross-references the evaluation issues and questions against indicators and proposed methods.

Table 21: Data Collection Matrix (DCM)
Evaluation Issues and Questions Indicators Method
Document Review
Method
Interviews
Method
Survey(s)
Method
PM Case Study
Relevance: Continued need for the program

1. Is there a continued need for the BSE Management Program, both overall and for each program component?

  • Is the program reflective of current science on TSEs?
  • Stakeholder views and documentary evidence of types of needs addressed by the program, by main stakeholder group (when feasible)
  • Stakeholder views and documentary evidence of the extent to which these needs are ongoing, overall and by program component
  • Expert views and documentary evidence that the program is aligned with current science on TSEs
  • Expert views and documentary evidence that the program is aligned with international science-based practices relating to TSEs

Check

Check

2. To what extent does the BSE Management Program respond to the needs, priorities, and mandates of its major stakeholders?

  • Can the design and delivery of the program be changed to better address the needs of the CFIA, other federal government departments, provincial government departments, industries, and international stakeholders?
  • Stakeholder views and documentary evidence of the extent to which the specific needs, priorities and mandates of the program's major stakeholders are addressed by the program
  • Stakeholder views on potential changes to the design and delivery of the program that would better address the needs of major stakeholders, by main stakeholder group (when feasible)

Check

Check

Relevance: Alignment with government priorities

3. Does the BSE Management Program continue to be consistent with government-wide priorities and the CFIA mandate?

  • Stakeholder views and documentary evidence of areas of alignment between the program, government priorities and the CFIA mandate

Check

Check

Relevance: Alignment with federal government roles and responsibilities

4. Is there a legitimate and necessary role for the federal government in the BSE Management Program?

  • What is the role of the CFIA according to legislation and stakeholders?
  • Documentary evidence of the CFIA's legislative role of relating to the program
  • Stakeholder views on the reasons underlying the legitimacy and necessity (or lack thereof) of the federal government role in the program (including with regard to performance measurement and reporting)

Check

Check

Check

5. Are the roles and responsibilities of the CFIA, other federal and provincial government departments involved in the BSE Management Program, and the regulated industries clear and well-understood by internal and external stakeholders?

  • Are they well aligned, or are there gaps or overlaps?
  • Stakeholder views on the specific roles and responsibilities of the CFIA relative those of other federal and provincial governments and of industry
  • Stakeholder views on the level of alignment between the roles and responsibilities of these actors, including evidence of gaps or overlaps (including with regard to performance measurement and reporting)

Check

Check

Check

Performance: Achievement of expected outcomes

6. To what extent have planned activities for each component of the BSE Management Program actually been implemented by the CFIA and produced the intended outputs?

  • Documentary evidence of activities and outputs:
    • SRM inspections/year (planned/delivered)
    • # revised/updated import policies/year
    • # consultation and communication products (import requirements)
    • # risk assessments conducted and mitigation measures in place (import)
    • #/frequency of surveillance data reports
    • $/% of surveillance samples reimbursed
    • # inspection tools (cattle ID)
    • # communication products and activities (cattle ID)
    • #/frequency export certificates issued
    • # quality assurance programs developed (export)
    • # risk assessments conducted (market access)
    • # revised/updated domestic policies/year
  • Stakeholder views on the extent to which activities were implemented as planned, by component
  • Assessment of performance measurement and management of these activities and outputs (collection/use of data)

Check

Check

Check

Check

7. To what extent have implemented activities for each component of the BSE Management Program produced the expected outcomes (immediate, intermediate, long-term, and the CFIA's strategic outcomes)?

  • To what extent are the measurable outputs and outcomes attributable to the BSE Management Program activities?
  • Given the outputs and outcomes of the BSE Management Program to date, how should the CFIA and partners/stakeholders position and envision the orientation of the Program going forward?

See indicator breakdown below, by individual outcome

Check

Check

Check

Immediate outcome for A1 – Managing Risks: Verifying the hygienic removal of SRM at slaughter and processing facilities

  • Compliance of industry for removal of SRM from human food at federally-registered establishments
  • Audits of provincial inspection staff to verify their oversight of SRM removal at non-federally registered establishments
  • % of the CFIA inspectors working in a beef slaughter or boning establishments are properly trained with regards to SRM removal
  • # of technical support training initiatives to the provinces in non-federally regulated sector to ensure the implementation of SRM removal (i.e., provincial training initiatives planned/delivered)
  • Stakeholder views on the quality and adequacy of the support (guidance, training, etc.) offered to ensure implementation of SRM removal in federally and non-federally regulated establishments
  • Assessment of performance measurement for this outcome (collection/use of data)

Check

Check

Check

Immediate outcome for A2 – Managing Risks: Canada's BSE import requirement are risk-based and meet international standards

  • Canada's import policies are aligned with domestic regulatory requirements and OIE
  • # of consultations with industry and trading partners on proposed changes to BSE import requirement
  • % import certificates are valid and consistent with Canada's import policy requirements
  • Stakeholder views on the extent to which import controls are risk-based and meet international standards
  • Assessment of performance measurement for this outcome (collection/use of data)

Check

Check

Check

Intermediate outcome for A1-2 – Managing Risks: BSE-related risks to the Canadian population is controlled

  • # of cases of domestic vCJD acquired after 2003 (% attributed to non-compliance of SRM removal from human food)
  • Stakeholder views on the contribution of the SRM removal and import control measures to the control of BSE risks
  • Assessment of performance measurement for this outcome (collection/use of data)

Check

Check

Check

Immediate outcome for B4 – Monitoring Risks: Efficient and accurate monitoring of the prevalence of BSE in Canada through surveillance

  • % of CFIA inspectors who have received training specific for the BSE surveillance program
  • Maintain BSE surveillance at sufficient level: # samples collected and tested
  • Science-based analysis of the surveillance program
  • Stakeholder views and documentary evidence on the efficiency and accuracy of BSE surveillance in Canada (quality of sampling, including unfit sample, geographic coverage, target sub-populations)
  • Stakeholder consultation and communication (collaborative surveillance network) to ensure stakeholder understanding and support of BSE Surveillance
  • Stakeholder views on the quality and adequacy of the support (guidance, training, etc.) offered specific to BSE surveillance
  • Assessment of performance measurement for this outcome (collection/use of data)

Check

Check

Check

Check

Intermediate outcome for B4 – Monitoring Risks: BSE-related risks to the Canadian cattle population is controlled

  • Enhanced commitment by stakeholders down to grassroots levels through the collective efforts of the Collaborative Surveillance Network, as demonstrated by ongoing participation in surveillance program (including level of effort required by CFIA to submit samples) and network activities
  • Stakeholder views on the contribution of the BSE surveillance program to the control of BSE risks, including awareness of the link between surveillance program and consumer confidence and on value of participating in the surveillance program (survey)
  • Stakeholder views and documentary evidence that the CFIA has maintained adequate surveillance numbers (% samples planned vs. tested/year) to be able to comment on any trends in BSE cases

Check

Check

Check

Immediate outcome for B5 – Monitoring Risks: Regulated parties and inspectors are aware of the requirements

  • % of inspectors who have received training specific for the traceability program
  • % of inspections planned versus conducted (one CVS task is the equivalent of one inspection)
  • # of quality management system opportunities for improvement received
  • % of tasks rated acceptable (TRACE PMS: # of violations to regulatory requirements)
  • Stakeholder awareness of requirements (survey - inspectors, regulated parties)
  • Stakeholder views (survey, staff only) on the quality and adequacy of the support (guidance, training, etc.) offered to specific to the traceability program
  • Assessment of performance measurement for this outcome (collection/use of data)

Check

Check

Check

Immediate outcome for B5 – Monitoring Risks: International markets are aware of the program

  • # of outreach activities/year
  • International stakeholder awareness of traceability program

Check

Check

Intermediate outcome for B5 – Monitoring Risks: Traceability data is complete, accurate and up-to-date

  • # of program enhancement recommendations made by veterinarians, scientists and decision-makers
  • Stakeholder views (interviews, survey) on the quality (completeness, accuracy, up-to-date) and adequacy of the traceability data (e.g., for investigations), including performance measurement (case study)

Check

Check

Check

Immediate outcome for C7 – Communicating: Awareness by foreign countries of Canada's BSE status and risk mitigation measures

  • # of ongoing technical negotiations that took place to raise awareness of the effectiveness of Canada's BSE risk mitigation measures thereby facilitating market access
  • International stakeholder awareness of Canada's status and risk mitigation measures

Check

Check

Immediate outcome for C8 – Communication: Trading partners are confident that BSE Risk mitigation measures implemented in Canada assure the sanitary safety of live cattle, cattle products and by-products

  • Stakeholder views and documentary evidence that the CFIA communicated changes of policies and responded to BSE related questionnaires/requests in a timely fashion
  • # of staff of the Vet abroad program (technical expertise and capacity maintained)
  • Maintain "controlled-risk" status from OIE annually
  • Stakeholder views and documentary evidence of the level of confidence of trading partners regarding the effectiveness of Canada's BSE measures

Check

Check

Intermediate outcome for C7-C8 – Communicating: Restore, maintain and expand access to international market (INT)

  • # of new export certifications with at least one BSE related condition that have been negotiated
  • # of countries that moved from an Embargo or Partial Access to Canadian beef meat, cattle, bovine genetics or products/by-products to an Expanded Access
  • Stakeholder views on the effectiveness of technical negotiations (BSE-related export conditions) and science advice to support the restoration, maintenance and expansion of market access
  • Assessment of performance measurement for this outcome (collection/use of data)

Check

Check

Check

Ultimate outcomes (CFIA-specific)

  • # of food safety/human health incidents
  • # of classical BSE incidents born after 2007
  • Stakeholder views and documentary evidence of public/consumer confidence maintained or increased by the measures in place (including public opinion reports and/or changes in beef consumption and prices relative to BSE cases)
  • Stakeholder views and documentary evidence continued improvement in the OIE risk status and BSE related restrictions (full market restoration)

Check

Check

8. Have there been any unintended (or unexpected) outcomes (positive and negative) of the BSE Management Program?

  • How have they been addressed by the CFIA?
  • Stakeholder views and documentary evidence of unintended or unexpected outcomes – positive
  • Stakeholder views and documentary evidence of unintended or unexpected outcomes – negative
  • Stakeholder views and documentary evidence of effectiveness of CFIA response to these outcomes

Check

Check

9. What are the major internal and external factors contributing to or constraining the performance of the BSE Management Program?

  • How have these factors been addressed in program design and delivery?
  • Have any best practices/lessons learned been identified/documented with respect to program design and delivery?
  • Stakeholder views and documentary evidence of internal factors contributing or constraining the performance of the program
  • Stakeholder views and documentary evidence of external factors contributing or constraining the performance of the program
  • Stakeholder views and documentary evidence of effectiveness of CFIA response to these factors
  • Stakeholder views and documentary evidence of best practices/lessons learned that were identified/ documented with respect to program design and delivery

Check

Check

10. Are there appropriate management and decision-making structures in place to meet the objectives of the BSE Management Program? Is the structure working as intended?

  • Have internal communication, outreach, engagement, and training activities been adequate and effective?
  • Are there opportunities to improve coordination and integration among the program components?
  • Stakeholder views and documentary evidence of appropriateness of management and decision-making structures
  • Stakeholder views and documentary evidence of adequacy and effectiveness of internal communication, outreach, engagement, and training activities
  • Stakeholder views on opportunities for improvement of coordination and integration

Check

Check

Check

Check

11. To what extent is collaboration and coordination with partner organizations and stakeholders adequately integrated in program design and delivery?

  • Have partner and stakeholder communication, outreach, engagement, and training activities been adequate and effective?
  • Are there overlaps or gaps?
  • Are there particular conditions and environments where collaboration and coordination efforts work best (e.g., small vs. large companies)?
  • Stakeholder views and documentary evidence of adequacy and effectiveness of partner and stakeholder communication, outreach, engagement, and training activities, overall and for particular conditions and environments
  • Stakeholder views and documentary evidence of overlaps or gaps in collaboration and coordination relating to the program with partner organizations and stakeholders

Check

Check

Check

Check

12. Is the performance measurement and monitoring strategy for the BSE Management Program and each of its components adequate and appropriate?

  • Has the performance measurement and monitoring strategy for the BSE Management Program and each of its components been implemented and delivered as intended?
  • Are appropriate data collection procedures and systems in place?
  • Do existing data collection procedures and systems provide a complete picture of performance in a timely and cost-effective manner?
  • Stakeholder views and documentary evidence of extent to which performance measurement and monitoring for the program, overall and for individual components has been implemented as planned
  • Stakeholder views and documentary evidence of appropriateness, timeliness and cost-effectiveness of data collection procedures and systems for use to support operational needs
  • Stakeholder views and documentary evidence of appropriateness, timeliness and cost-effectiveness of data collection procedures and systems for use to support strategic needs
  • Stakeholder views on potential improvements to data collection procedures and systems, and to the performance measurement and monitoring strategy more broadly, to better meet these needs

Check

Check

Check

Check

Performance: Demonstration of efficiency and economy

13. Have the allocated financial resources and human resources (FTEs) been adequate for the activities of each component of the BSE Management Program?

  • $ for the program, overall and by component
  • FTEs for the program, overall and by component
  • Comparison planned vs. actual $ and # FTEs
  • Stakeholder views and documentary evidence of adequacy (quantity, quality, timeliness) of financial resources and FTEs to deliver the program, overall and by component

Check

Check

14. Is the BSE Management Program cost-effective as currently delivered?

  • Are there alternative approaches to the current design and delivery of the program that would allow it to meet its objectives and mandate in a more effective and efficient manner?
  • What would be the cost implications of these alternative approaches?
  • Are there opportunities to improve leveraging with partner organizations and stakeholders?
  • Stakeholder views on alternative approaches to deliver the program in a more effective and efficient manner
  • Stakeholder views on opportunities to improve leveraging with partner organizations and stakeholders
  • Comparison with national approaches or benchmarks (with cost details, if possible) to deliver similar activities and outputs (e.g., inspections), taking into account differences in objectives and mandates
  • Comparison with international BSE risk management and monitoring approaches (with cost details, if possible), taking into account differences in BSE risk status, objectives and mandates
  • Cost implications of alternative approaches identified above

Check

15. How efficiently and effectively have BSE Management Program funds been used, both overall and for each program component?

  • Are current financial reporting and accounting procedures adequate to track the use of funds for each program component?
  • Stakeholder views and documentary evidence of management practices to optimize inputs
  • Stakeholder views and documentary evidence of management practices to optimize outputs
  • Stakeholder views and documentary evidence of adequacy of accounting procedures to track the use of funds, overall and for each component

Check

Check

Check

Follow-up on recommendations

State of implementation of recommendations from the previous BSE evaluation

  • N/A

Check

Check

Check

The table below presents the Data Collection Matrix for the Health Canada and Public Health Agency of Canada aspects of the evaluation.

Table 22: Health Canada and Public Health Agency of Canada Data Collection Matrix
Evaluation Issues and Questions Method
Document Review
Method
Interviews
Relevance: Continued need for the program
1. Is there a continued need for Health Canada and Public Health Agency of Canada's components of the BSE Management Program? Check Check
Relevance: Alignment with government priorities
2. Does the BSE Management Program continue to be consistent with Health Canada and Public Health Agency of Canada priorities and mandate? Check
Relevance: Alignment with federal government roles and responsibilities
3. Are the activities that Health Canada and Public Health Agency of Canada engage in aligned with their federal roles and responsibilities? Check
Performance: Achievement of expected outcomes
4. To what extent have planned activities for the Health Canada and Public Health Agency of Canada components of the BSE Management Program been implemented and produced the intended outputs and outcomes? Check Check
5. What additional activities are required moving forward to produce the expected outcomes? Check
6. Have there been any unintended (or unexpected) outcomes (positive and negative) from the Health Canada and Public Health Agency of Canada components of the BSE Management Program? Check
7. What are the major internal and external factors contributing to or constraining the performance of Health Canada's and Public Health Agency of Canada's components of the BSE Management Program? Check
8. To what extent is collaboration and coordination with CFIA integrated in Health Canada's and Public Health Agency of Canada's program design and delivery? Check
Performance: Demonstration of efficiency and economy
9. Have the allocated resources been adequate and used economically for the Health Canada and Public Health Agency of Canada components of the BSE Management Program? Check Check
10. How efficiently and effectively have the Health Canada and Public Health Agency of Canada BSE Management Program funds been used? Check
11. Is the performance measurement and monitoring strategy for the Health Canada and Public Health Agency of Canada components of the BSE Management Program used, adequate and appropriate? Check Check

10.3 Appendix C – Interview Guides

10.3.1 Internal Interview Guide: CFIA Senior Management

A. Introduction

Q1. Before we start, do you have any questions about this evaluation or about this interview?

B. Context

Theme 1: Context
Objectives: Establish the interviewee's context vis-à-vis the BSE Management Program
Duration: 5 minutes

Q2. Please describe your role within the BSE Management Program (current and former, if any) for the period from 2008-09 to 2012-13.

Q3. To what extent were you involved in each of the following six CFIA-led BSE-related activity areas? You may refer to the Program Logic Model at the end of this guide.

Managing Risks:

Monitoring Risks:

Communicating Canada's Program:

C. Continued need for the Program

Theme 2: Relevance
Objectives: Identify the program rationale and responsiveness to needs
Duration: 10 minutes

Q4. In your opinion, is there an ongoing need for each of the six CFIA BSE-related activities? Why?

Q5. To what extent does the BSE Management Program respond to the needs of its major stakeholders?

Q6. To what extent is the BSE Management Program aligned with current international science-based practices relating to TSEs (i.e., "grounded in science")? Please explain.

Theme 3: Alignment with Government Priorities and with Federal Roles and Responsibilities
Objectives: Determine alignment with federal government priorities and departmental mandate
Duration: 5 minutes

Q7. In your opinion, is the work carried out by the CFIA for its BSE-related activities aligned with:

  1. The CFIA's mandate? Please elaborate.
  2. Federal government-wide priorities? Please elaborate.
  3. Federal roles and responsibilities? Please elaborate

Q8. Is there similar programming regionally or across the country (carried out by other federal government departments, other levels of government or industry) that duplicates or complements the work of the CFIA?

D. Performance

Theme 4: Achievement of Expected Outcomes
Objectives: Assess program progress toward expected results
Duration: 20 minutes

Q9. The long-term objectives of the BSE Management Program are the following:

  1. Protection from preventable health risks related to food safety and product safety or the transmission of animal diseases to humans;
  2. Safer food and health products;
  3. Sustainable animal resource base; and
  4. Maintain public/consumer confidence in the animal, production and food system.

To what extent do you think the work carried out under the CFIA BSE Management Program contributed to the achievement of these objectives to date? Please justify your answer by explaining how the six BSE-related activities contributed to these objectives.

Q10. Are you aware of any other outcomes that occurred as a result of the BSE Management Program activities?

Q11. Have there been any internal or external factors over the last five years affecting the performance of the BSE Management Program?

Factors on performance
Internal Factors
(within CFIA)
External Factors
(Outside CFIA)
Positive Factors (+)
Negative Factors (-)

Q12. To what extent has the CFIA integrated collaboration and coordination with its partner organizations and its stakeholders into the BSE Management Program design and delivery:

  1. Other federal organizations
    • Public Health Agency of Canada (PHAC)
    • Health Canada
    • Department of Foreign Affairs, Trade and Development (DFATD, formerly DFAIT)
    • Others
  2. Provinces
  3. Industry
  4. Others

Theme 5: Demonstration of efficiency and economy
Objectives: Assess the level of resource use for the production of outputs and progress toward expected outcomes
Duration: 25 minutes

Q13. In your opinion, are the current levels of financial and human resources, including training, adequate to address the overall objectives of the six BSE-related activities of the CFIA? Please elaborate.

Q14. a) Have funds been used in an efficient and effective manner for the BSE Management Program and for each Program component?

b) Considering the management, programming and operations of the CFIA's BSE-related activities, could the intended objectives be achieved more efficiently, or at a lower cost? Please elaborate.

Q15. Are the management and decision-making structures of the CFIA adequately supporting the objectives of the BSE Management Program? How, if at all, could these structures be improved?

Q16. In your opinion, is the existing performance measurement strategy for the BSE Management Program adequate and effective? Please explain.

Q17. To what extent has the current financial tracking (accounting and reporting) been efficient and effective for the program?

Q18. In addition to opportunities noted under Question 14, please highlight any practices or opportunities that could improve the performance of the BSE Management Program.

Q19. Do you wish to add comments or observations for this evaluation of the CFIA's BSE Management Program?

10.3.2 Internal Interview Guide: CFIA Science Branch Management

A. Introduction

Q1. Before we start, do you have any questions about this evaluation or about this interview?

B. Context

Theme 1: Context
Objectives: Establish the interviewee's context vis-à-vis the BSE Management Program
Duration: 5 minutes

Q2. Please describe your role within the BSE Management Program (current and former, if any) for the period 2008-09 to 2012-13.

You may refer to the activities listed below, which are described in more detail in the BSE Management Program Logic Model at the end of this guide.

Managing Risks:

Monitoring Risks:

Communicating Canada's Program:

C. Continued need for the Program

Theme 2: Relevance
Objectives: Identify the program rationale and responsiveness to needs
Duration: 15 minutes

Q3. In your opinion, is there an ongoing need for CFIA's BSE-related activities, overall and for certain specific activities (e.g., surveillance, risk assessment)? Why?

Q4. To what extent is the BSE Management Program aligned with current science-based practices relating to TSEs (i.e., "grounded in science")? Please explain.

Q5. To what extent does the BSE Management Program respond to the needs of its major stakeholders?

Q6. To what extent is the BSE surveillance activity aligned with international OIE measures to prevent and control BSE in Canada?

Theme 3: Alignment with Government Priorities and with Federal Roles and Responsibilities
Objectives: Determine alignment with federal government priorities and departmental mandate
Duration: 5 minutes

Q7. In your opinion, is the work carried out by the CFIA for its BSE-related activities aligned with:

  1. The CFIA's mandate? Please elaborate.
  2. Federal government-wide priorities? Please elaborate.
  3. Federal roles and responsibilities? Please elaborate

Q8. Is there similar programming regionally or across the country (carried out by other federal government departments, other levels of government or industry) that duplicates or complements the work of the CFIA?

D. Performance

Theme 4: Achievement of Expected Outcomes
Objectives: Assess program progress toward expected results
Duration: 25 minutes

Q9. To what extent have the CFIA's surveillance activities been adequate? Please refer to the each of the following, when relevant:

  1. BSE-related research conducted by the Science Branch
  2. BSE-related training (i.e., by reference lab)
  3. Testing - Quantity of samples
  4. Testing - Quality of samples (i.e., quality assurance process)
  5. Testing - Geographic coverage
  6. Testing - Target sub-populations (e.g., 4Ds: Diseased, Down, Dying, Dead; clinical signs)

Q10. To what extent have the main activities of the CFIA's BSE Management Program in which you were involved delivered as planned in order to produce the expected or intended outcomes (immediate & intermediate)?

Note: Some of these may not be relevant to your experience. Please refer to the Program Logic Model provided as the end of this guide if necessary.

a. Monitoring Risks: BSE Surveillance
Not at all Slightly Partially Mostly Completely Not Applicable
1 2 3 4 5 NA

Rating rationale:

b. Monitoring Risks: Cattle Identification (if relevant)
Not at all Slightly Partially Mostly Completely Not Applicable
1 2 3 4 5 NA

Rating rationale:

c. Managing Risks: Removal of specified risk material from the human food supply (if relevant)
Not at all Slightly Partially Mostly Completely Not Applicable
1 2 3 4 5 NA

Rating rationale:

d. Managing Risks: Import controls (if relevant)
Not at all Slightly Partially Mostly Completely Not Applicable
1 2 3 4 5 NA

Rating rationale:

e. Communicating Canada's Program: Export Certification if relevant)
Not at all Slightly Partially Mostly Completely Not Applicable
1 2 3 4 5 NA

Rating rationale:

f. Communicating Canada's Program: Technical market access support regarding BSE controls (if relevant)
Not at all Slightly Partially Mostly Completely Not Applicable
1 2 3 4 5 NA

Rating rationale:

Q11. The long-term objectives of the BSE Management Program are the following:

  1. Protection from preventable health risks related to food safety and product safety or the transmission of animal diseases to humans;
  2. Safer food and health products;
  3. Sustainable animal resource base; and
  4. Maintain public/consumer confidence in the animal, production and food system.

To what extent do you think the work carried out under the CFIA BSE Management Program contributed to the achievement of these objectives to date? Please justify your answer by explaining how the Science Branch's BSE-related activities contributed to these objectives.

Q12. Are you aware of any other outcomes that occurred as a result of the BSE Management Program activities?

Q13. Have there been any internal or external factors over the last five years affecting the performance of the BSE Management Program?

Factors on performance
Internal Factors
(within CFIA)
External Factors
(Outside CFIA)
Positive Factors (+)
Negative Factors (-)

Q14. To what extent has the CFIA integrated collaboration and coordination with its partner organizations and its stakeholders into the BSE Management Program design and delivery:

  1. Other federal organizations
    • Public Health Agency of Canada (PHAC)
    • Health Canada
    • Department of Foreign Affairs, Trade and Development (DFATD, formerly DFAIT)
    • Others
  2. Provinces
  3. Industry
  4. Others

Theme 5: Demonstration of efficiency and economy
Objectives: Assess the level of resource use for the production of outputs and progress toward expected outcomes
Duration: 25 minutes

Q15. In your opinion, are the current levels of financial and human resources, including training, adequate to address the objectives of the BSE-related activities of the CFIA? Please elaborate.

Q16. a) Have funds been used in an efficient and effective manner for the BSE Management Program overall and for the Science Branch activities specifically?

b) Considering the management, programming and operations of the CFIA's BSE-related activities, could the intended objectives be achieved more efficiently, or at a lower cost? Please elaborate.

Q17. Are the management and decision-making structures of the CFIA adequately supporting the objectives of the BSE Management Program? How, if at all, could these structures be improved?

Q18. In your opinion, is the existing performance measurement strategy for the BSE Management Program adequate and effective? Please explain, with reference to specific elements of the performance measurement and monitoring that have been implemented.

Q19. To what extent has the current financial tracking (accounting and reporting) been efficient and effective for the Program and for the Science Branch activities specifically? (DCM 15.2.3)

Q20. In addition to opportunities noted under Question 16, please highlight any practices or opportunities that could improve the performance of the BSE Management Program.

Q21. Do you wish to add comments or observations for this evaluation of the CFIA's BSE Management Program?

10.3.3 Internal Interview Guide: CFIA Policy and Programs Branch Management

A. Introduction

Q1. Before we start, do you have any questions about this evaluation or about this interview?

B. Context

Theme 1: Context
Objectives: Establish the interviewee's context vis-à-vis the BSE Management Program
Duration: 5 minutes

Q2. Please describe your role within the BSE Management Program (current and former, if any) for the period from 2008-09 to 2012-13.

Q3. To what extent were you involved in each of the following six CFIA-led BSE-related activity areas?

Managing Risks:

Monitoring Risks:

Communicating Canada's Program:

C. Continued need for the Program

Theme 2: Relevance
Objectives: Identify the program rationale and responsiveness to needs
Duration: 15 minutes

Q4. In your opinion, is there an ongoing need for each of the six CFIA BSE-related activities? Why?

Q5. To what extent does the BSE Management Program respond to the needs of its major stakeholders?

Q6. To what extent is the BSE Management Program aligned with current international science-based practices relating to TSEs (i.e., "grounded in science")? Please explain.

Theme 3: Alignment with Government Priorities and with Federal Roles and Responsibilities
Objectives: Determine alignment with federal government priorities and departmental mandate
Duration: 5 minutes

Q7. In your opinion, is the work carried out by the CFIA for its BSE-related activities aligned with:

  1. The CFIA's mandate? Please elaborate.
  2. Federal government-wide priorities? Please elaborate.
  3. Federal roles and responsibilities? Please elaborate.

Q8. Is there similar programming regionally or across the country (carried out by other federal government departments, other levels of government or industry) that duplicates or complements the work of the CFIA?

D. Performance

Theme 4: Achievement of Expected Outcomes
Objectives: Assess program progress toward expected results
Duration: 25 minutes

Q9. To what extent have the main activities of the CFIA's BSE Management Program in which you were involved delivered as planned in order to produce the expected or intended outcomes (immediate & intermediate)?

Note: Some of these may not be relevant to your experience. Please refer to the Program Logic Model provided as the end of this guide if necessary.

a. Managing Risks: Removal of specified risk material from the human food supply
Not at all Slightly Partially Mostly Completely Not Applicable
1 2 3 4 5 NA

Rating rationale:

b. Managing Risks: Import controls
Not at all Slightly Partially Mostly Completely Not Applicable
1 2 3 4 5 NA

Rating rationale:

c. Monitoring Risks: BSE Surveillance
Not at all Slightly Partially Mostly Completely Not Applicable
1 2 3 4 5 NA

Rating rationale:

d. Monitoring Risks: Cattle Identification
Not at all Slightly Partially Mostly Completely Not Applicable
1 2 3 4 5 NA

Rating rationale:

e. Communicating Canada's Program: Export Certification
Not at all Slightly Partially Mostly Completely Not Applicable
1 2 3 4 5 NA

Rating rationale:

f. Communicating Canada's Program: Technical market access support regarding BSE controls
Not at all Slightly Partially Mostly Completely Not Applicable
1 2 3 4 5 NA

Rating rationale:

Q10. The long-term objectives of the BSE Management Program are the following:

  1. Protection from preventable health risks related to food safety and product safety or the transmission of animal diseases to humans;
  2. Safer food and health products;
  3. Sustainable animal resource base; and
  4. Maintain public/consumer confidence in the animal, production and food system.

To what extent do you think the work carried out under the CFIA BSE Management Program contributed to the achievement of these objectives to date? Please justify your answer by explaining how the six BSE-related activities contributed to these objectives.

Q11. Are you aware of any other outcomes that occurred as a result of the BSE Management Program activities?

Q12. Have there been any internal or external factors over the last five years affecting the performance of the BSE Management Program?

Factors on performance
Internal Factors
(within CFIA)
External Factors
(Outside CFIA)
Positive Factors (+)
Negative Factors (-)

Q13. To what extent has the CFIA integrated collaboration and coordination with its partner organizations and its stakeholders into the BSE Management Program design and delivery:

  1. Other federal organizations
    • Public Health Agency of Canada (PHAC)
    • Health Canada
    • Department of Foreign Affairs, Trade and Development (DFATD, formerly DFAIT)
    • Others
  2. Provinces
  3. Industry
  4. Others

Theme 5: Demonstration of efficiency and economy
Objectives: Assess the level of resource use for the production of outputs and progress toward expected outcomes
Duration: 25 minutes

Q14. In your opinion, are the current levels of financial and human resources, including training, adequate to address the objectives of each of the six BSE-related activities of the CFIA? Please elaborate for the activities with which you are familiar.

Q15. a) Have funds been used in an efficient and effective manner for the BSE Management Program and for each Program component?

b) Considering the management, programming and operations of the CFIA's BSE-related activities, could the intended objectives be achieved more efficiently, or at a lower cost? Please elaborate.

Q16. Are the management and decision-making structures of the CFIA adequately supporting the objectives of the BSE Management Program? How, if at all, could these structures be improved?

Q17. In your opinion, is the existing performance measurement strategy for the BSE Management Program adequate and effective? Please explain, with reference to specific elements of the performance measurement and monitoring that have been implemented.

Q18. To what extent has the current financial tracking (accounting and reporting) been efficient and effective for the Program and for the Policy & Programs Branch activities specifically?

Q19. In addition to opportunities noted under Question 15, please highlight any practices or opportunities that could improve the performance of the BSE Management Program.

Q20. Do you wish to add comments or observations for this evaluation of the CFIA's BSE Management Program?

10.3.4 Internal Interview Guide: CFIA Operations Branch Management

A. Introduction

Q1. Before we start, do you have any questions about this evaluation or about this interview?

B. Context

Theme 1: Context
Objectives: Establish the interviewee's context vis-à-vis the BSE Management Program
Duration: 5 minutes

Q2. Please describe your role within the BSE Management Program (current and former, if any) for the period from 2008-09 to 2012-13.

You may refer to the activities listed below, which are described in more detail in the BSE Management Program Logic Model at the end of this guide.

Managing Risks:

Monitoring Risks:

Communicating Canada's Program:

C. Continued need for the Program

Theme 2: Relevance
Objectives: Identify the program rationale and responsiveness to needs
Duration: 10 minutes

Q3. In your opinion, is there an ongoing need for each of the six CFIA BSE-related activities? Why?

Q4. To what extent does the BSE Management Program respond to the needs of its major stakeholders?

Theme 3: Alignment with Government Priorities and with Federal Roles and Responsibilities
Objectives: Determine alignment with federal government priorities and departmental mandate
Duration: 5 minutes

Q5. In your opinion, is the work carried out by the CFIA for its BSE-related activities aligned with:

  1. The CFIA's mandate? Please elaborate.
  2. Federal government-wide priorities? Please elaborate.
  3. Federal roles and responsibilities? Please elaborate.

Q6. Is there similar programming regionally or across the country (carried out by other federal government departments, other levels of government or industry) that duplicates or complements the work of the CFIA?

D. Performance

Theme 4: Achievement of Expected Outcomes
Objectives: Assess program progress toward expected results
Duration: 20 minutes

Q7. To what extent have the CFIA's verification, enforcement and other BSE-related inspection activities been delivered as planned in order to produce the expected or intended outcomes (immediate & intermediate)?

Note: Some of these may not be relevant to your experience. Please refer to the Program Logic Model provided as the end of this guide if necessary.

a. Managing Risks: Removal of specified risk material from the human food supply
Not at all Slightly Partially Mostly Completely Not Applicable
1 2 3 4 5 NA

Rating rationale:

b. Managing Risks: Import controls (if relevant)
Not at all Slightly Partially Mostly Completely Not Applicable
1 2 3 4 5 NA

Rating rationale:

c. Monitoring Risks: BSE Surveillance (if relevant)
Not at all Slightly Partially Mostly Completely Not Applicable
1 2 3 4 5 NA

Rating rationale:

d. Monitoring Risks: Cattle Identification (if relevant)
Not at all Slightly Partially Mostly Completely Not Applicable
1 2 3 4 5 NA

Rating rationale:

e. Communicating Canada's Program: Export Certification (if relevant)
Not at all Slightly Partially Mostly Completely Not Applicable
1 2 3 4 5 NA

Rating rationale:

Q8. The long-term objectives of the BSE Management Program are the following:

  1. Protection from preventable health risks related to food safety and product safety or the transmission of animal diseases to humans;
  2. Safer food and health products;
  3. Sustainable animal resource base; and
  4. Maintain public/consumer confidence in the animal, production and food system.

To what extent do you think the work carried out under the CFIA BSE Management Program contributed to the achievement of these objectives to date? Please justify your answer by explaining how the six BSE-related activities contributed to these objectives.

Q9. Are you aware of any other outcomes that occurred as a result of the BSE Management Program activities?

Q10. Have there been any internal or external factors over the last five years affecting the performance of the BSE Management Program?

Factors on performance
Internal Factors
(within CFIA)
External Factors
(Outside CFIA)
Positive Factors (+)
Negative Factors (-)

Q11. To what extent has the CFIA integrated collaboration and coordination with its partner organizations and its stakeholders into the BSE Management Program design and delivery:

  1. Other federal organizations
    • Public Health Agency of Canada (PHAC)
    • Health Canada
    • Department of Foreign Affairs, Trade and Development (DFATD, formerly DFAIT)
    • Others
  2. Provinces
  3. Industry
  4. Others

Theme 5: Demonstration of efficiency and economy
Objectives: Assess the level of resource use for the production of outputs and progress toward expected outcomes
Duration: 10 minutes

Q12. In your opinion, are the current levels of financial and human resources adequate to address the BSE-related objectives of the Operations Branch? Please elaborate.

Q13. To your knowledge, has the training and level of guidance provided to inspectors and field staff been adequate to ensure the effective and efficient implementation of:

  1. SRM removal from food in federally and non-federally regulated establishments?
  2. BSE surveillance testing?
  3. The cattle traceability program?

Q14. a) Have funds been used in an efficient and effective manner for the BSE Management Program overall and for the Operations Branch activities specifically?

b) Considering the management, programming and operations of the CFIA's BSE-related activities, could the intended objectives be achieved more efficiently, or at a lower cost? Please elaborate.

Q15. Are the management and decision-making structures of the CFIA adequately supporting the objectives of the BSE Management Program? How, if at all, could these structures be improved?

Q16. In your opinion, is the existing performance measurement strategy for the BSE Management Program adequate and effective? Please explain, with reference to specific elements of the performance measurement and monitoring that have been implemented.

Q17. To what extent has the current financial tracking (accounting and reporting) been efficient and effective for the Program and for the Operations Branch activities specifically?

Q18. In addition to opportunities noted under Question 14, please highlight any practices or opportunities that could improve the performance of the BSE Management Program.

Q19. Do you wish to add comments or observations for this evaluation of the CFIA's BSE Management Program?

10.3.5 Internal Interview Guide: CFIA Inspection Managers

A. Introduction

Q1. Before we start, do you have any questions about this evaluation or about this interview?

B. Context

Theme 1: Context
Objectives: Establish the interviewee's context vis-à-vis the BSE Management Program
Duration: 3 minutes

Q2. Please describe your role within the BSE Management Program (current and former, if any) for the period from 2008-09 to 2012-13.

C. Performance

Theme 2: Achievement of Expected Outcomes
Objectives: Assess program progress toward expected results
Duration: 15 minutes

Q3. Please identify the different types of CFIA field staff involved in verification, enforcement and inspection-related activities in the context of the BSE Management Program and briefly describe their role. Based on your knowledge, you may refer to one or more of the following:

  1. Removal of SRM from the human food supply
  2. Cattle Identification
  3. BSE Surveillance
  4. Import Controls

Q4. Please indicate the extent to which the following field inspection activities are conducted as planned, using on a scale from 1 to 5 (see below). Please explain your answer.

a. Enforcement & verification of SRM regulations by industry
Not at all Slightly Partially Mostly Completely Not Applicable
1 2 3 4 5 NA

Rating rationale:

b. Verification of SRM removal from the human food supply in federally registered establishments
Not at all Slightly Partially Mostly Completely Not Applicable
1 2 3 4 5 NA

Rating rationale:

c. Provision of technical support to provinces for the implementation of SRM requirements & verifying SRM removal from the human food supply in non-federally registered establishments
Not at all Slightly Partially Mostly Completely Not Applicable
1 2 3 4 5 NA

Rating rationale:

d. Inspection and enforcement actions for the cattle identification program (if relevant)
Not at all Slightly Partially Mostly Completely Not Applicable
1 2 3 4 5 NA

Rating rationale:

e. BSE surveillance sample collection (if relevant)
Not at all Slightly Partially Mostly Completely Not Applicable
1 2 3 4 5 NA

Rating rationale:

Q5. From your perspective, to what extent have the following outcomes been achieved as a result of the CFIA's BSE-related activities?

a. Compliance of the hygienic removal of SRM from the human food supply at federally registered slaughter & processing facilities
Not at all Slightly Partially Mostly Completely Not Applicable
1 2 3 4 5 NA

Rating rationale:

b. Compliance of the hygienic removal of SRM from the human food supply at non-federally registered facilities
Not at all Slightly Partially Mostly Completely Not Applicable
1 2 3 4 5 NA

Rating rationale:

c. Regulated parties and inspectors are aware of the requirements for cattle identification
Not at all Slightly Partially Mostly Completely Not Applicable
1 2 3 4 5 NA

Rating rationale:

Q6. To your knowledge, has the training and level of guidance provided to inspectors and field staff been adequate to ensure the effective and efficient implementation of:

  1. SRM removal from food in federally and non-federally regulated establishments?
  2. BSE surveillance testing?
  3. The cattle traceability program?

Q7. Is the current level of consultation and communication with industry stakeholders adequate to ensure industry compliance (SRM removal) and understanding of the CFIA's BSE-related requirements? Please elaborate.

Theme 3: Demonstration of efficiency and economy
Objectives: Assess the level of resource use for the production of outputs and progress toward expected outcomes
Duration: 10 minutes

Q8. In your opinion, are the current levels of financial and human resources adequate to address the BSE-related objectives of the Operations Branch? Please elaborate.

Q9. How would you describe the quality and quantity of internal communication, outreach, and engagement activities undertaken within the CFIA in the past five years?

Q10. Please describe the way in which BSE-related inspection data is collected and transmitted to from the field inspection level through to the Operations, Science and Policy Branches of the CFIA?

Q11. Please highlight any practices or opportunities that could improve the performance of the BSE Management Program.

D. Continued need for the Program

Theme 4: Relevance & Alignment with Federal Roles and Responsibilities
Objectives: Identify the program rationale and responsiveness to needs
Duration: 5 minutes

Q12. From your perspective, to what extent does the BSE Management Program respond to the needs of its major stakeholders?

Q13. Is there similar programming regionally or across the country (carried out by other federal government departments, other levels of government or industry) that duplicates or complements the work of the CFIA relative to BSE?

Q14. Do you wish to add comments or observations for this evaluation of the CFIA's BSE Management Program?

10.3.6 External Interview Guide: External Key Informants

A. Introduction

Q1. Before we start, do you have any questions about this evaluation or about this interview?

B. Context

Theme 1: Context
Objectives: Establish the interviewee's context vis-à-vis the BSE Management Program
Duration: 5 minutes

Q2. Please describe your role within your organization for the period from 2008-13 and explain how this role pertains to BSE.

Q3. a) How familiar are you with the CFIA's BSE-related activities? The six main activities are listed below.

Managing Risks:

Monitoring Risks:

Communicating Canada's Program:

b) Which activities have you and/or your organization been involved in (if any)?

C. Performance

Theme 2: Achievement of Expected Outcomes
Objectives: Assess program progress toward expected results
Duration: 15 minutes

Q4. To what extent do you think the BSE-related activities carried out by the CFIA in the last five years have contributed to controlling the following BSE-related risks?

Please explain your rating, referring to specific CFIA activities when possible.

For the Canadian cattle population (animal health)?
Not at all Slightly Partially Mostly Completely Not Applicable
1 2 3 4 5 NA

Rating rationale:

For the Canadian public (consumers, public health)?
Not at all Slightly Partially Mostly Completely Not Applicable
1 2 3 4 5 NA

Rating rationale:

For trading partners and related stakeholders (market access, OIE risk status)?
Not at all Slightly Partially Mostly Completely Not Applicable
1 2 3 4 5 NA

Rating rationale:

Q5. To what extent do you believe that the CFIA has been effective at communication, outreach and engagement with industry in the last five years in order to ensure that regulated parties are aware of:

  1. Requirements and regulations relating to the removal of SRM from human food in federally and non-federally regulated establishments?
  2. Requirements and regulations relating to cattle identification?
  3. The link between the BSE surveillance program and consumer confidence (and therefore the value of participation in the surveillance program)?

Q6. To what extent do you believe that the CFIA has been effective at communication, outreach and engagement with international stakeholders in the last five years in order to ensure that foreign countries (markets) are aware of and have confidence in Canada's risk mitigation measures relating to BSE (SRM removal, import controls, surveillance, traceability, export certification, etc.)?

Q7. To what extent has the CFIA integrated collaboration and coordination with your organization (or with other key partners/stakeholders) into the design and delivery of its BSE-related activities? Please explain.

Q8. Are you aware of any other outcomes that occurred as a result of the CFIA's BSE-related activities?

Q9. Have there been any major factors over the last five years that have affected the capacity of your organization (or industry) to help manage BSE-related risks in Canada?

D. Continued need for the Program

Theme 3: Relevance
Objectives: Identify the program rationale and responsiveness to needs
Duration: 5 minutes

Q10. In your opinion, is there an ongoing need for the CFIA's BSE-related activities, overall or for specific activities? Please explain.

Q11. a) To what extent have the CFIA's BSE-related activities responded to the needs of your organization (current or past needs, if these have changed over the last 5 years)? Please explain.

b) Could these activities be changed to better address your current needs? Please explain.

Theme 4: Alignment with Government Priorities and with Federal Roles and Responsibilities
Objectives: Determine alignment with federal government priorities and departmental mandate
Duration: 5 minutes

Q12. In your opinion, is the work carried out by the CFIA for its BSE-related activities aligned with:

  1. The CFIA's mandate?
  2. Federal roles and responsibilities?

Q13. Is there similar programming regionally or across the country (carried out by other federal government departments, provincial governments or industry) that duplicates or complements the work of the CFIA?

Q14. Do you wish to add any comments or observations for this evaluation?

10.3.7 HC-PHAC Interview Guide: Interview guide for Health Canada

Relevance
  1. What are the current issues for the BSE Management Program in HC?
Performance:
  1. Can you please discuss and describe your food or health products health risk assessment activities for the period time 2009-10 to 2012-13?
  2. What data to advance the human health risk assessments, standards and policies to protect consumers was developed during this time period?
  3. How has HC used CFIA compliance and enforcement information in its health risk assessment?
  4. Please describe your BSE/TSE research activities for the period of time 2009-10 to 2012-13?
  5. How has the BSE/TSE research been used to manage and evaluate the risk of BSE and other TSE risks associated with food and health products that use animal-sourced ingredients?
  6. Please describe how HC used the research that conducted in Europe to address the evolving science linking BSE and other TSEs to human health outcomes?
  7. What standards and policies have been developed or updated during this time period?
  8. Funding for BSE III also was used to support issues of import control, compliance and enforcement, product tracking, and co-ordination. If possible, can you please describe what activities or progress was made these areas?
  9. What BSE prevention measures were developed during this time?
  10. How has HC contributed to Canada maintaining or improving its Organization for Animal Health "controlled BSE-risk" status?
  11. Have there been any unintended (or unexpected) outcomes (positive and negative) from the HC components of the BSE Management Program?
  12. What are the major internal and external factors contributing to or constraining the performance of HC's components of the BSE Management Program? What else is needed?
  13. How efficiently have the HC BSE Management Program funds been used?
  14. Are communications between HC, PHAC and CFIA adequate to meet program needs?
  15. Is there adequate internal coordination in HC to meet program needs?

10.3.8 HC-PHAC Interview Guide: Interview guide for the Public Health Agency of Canada

Relevance
  1. What are the current issues for the BSE Management Program in HC?
Performance:
  1. Can you please discuss and describe your food or health products health risk assessment activities for the period time 2009-10 to 2012-13?
  2. What data to advance the human health risk assessments, standards and policies to protect consumers was developed during this time period?
  3. How has HC used CFIA compliance and enforcement information in its health risk assessment?
  4. Please describe your BSE/TSE research activities for the period of time 2009-10 to 2012-13?
  5. How has the BSE/TSE research been used to manage and evaluate the risk of BSE and other TSE risks associated with food and health products that use animal-sourced ingredients?
  6. Please describe how HC used the research that conducted in Europe to address the evolving science linking BSE and other TSEs to human health outcomes?
  7. What standards and policies have been developed or updated during this time period?
  8. Funding for BSE III also was used to support issues of import control, compliance and enforcement, product tracking, and co-ordination. If possible, can you please describe what activities or progress was made these areas?
  9. What BSE prevention measures were developed during this time?
  10. How has HC contributed to Canada maintaining or improving its Organization for Animal Health "controlled BSE-risk" status?
  11. Have there been any unintended (or unexpected) outcomes (positive and negative) from the HC components of the BSE Management Program?
  12. What are the major internal and external factors contributing to or constraining the performance of HC's components of the BSE Management Program? What else is needed?
  13. How efficiently have the HC BSE Management Program funds been used?
  14. Are communications between HC, PHAC and CFIA adequate to meet program needs?
  15. Is there adequate internal coordination in HC to meet program needs?

10.3.9 HC-PHAC Interview Guide: Interview guide for Canadian Food Inspection Agency

Relevance
  1. What are the current issues for the BSE Management Program for the Federal BSE Initiative?
Performance
  1. To what extent is collaboration and coordination with CFIA integrated in HC's and PHAC's program design and delivery?
    1. Are communications between PHAC/HC and CFIA adequate to meet program needs?
    2. Is data/information sharing occurring as planned and to meet BSE Management Program needs?
    3. Is BSE research/data/information from CFIA used in PHAC and/or HC to meet program needs?
    4. Is BSE research/data/information from PHAC and/or HC used in CFIA to meet program needs?
    5. Is research in HC and/or PHAC informed by CFIA program needs?
  2. How has CFIA compliance and enforcement information contributed to HC's health risk assessments and BSE standards and policy?

10.4 Appendix D – Survey Questionnaire

Available upon request

10.5 Appendix E – Follow-up on 2009 Recommendations

Evaluation Recommendation December 2011 Update Document and File Review 2013
Recommendation 1:
Coordination and communication

To strengthen coordination and communication with regard to BSE-related activities, both within the CFIA and with external partner organizations and stakeholders, a dedicated, expert-led coordination structure should be established in which roles and responsibilities are clearly established and communicated.

Complete

A BSE Program Management Committee was created to provide oversight for the CFIA's BSE-related activities. Roles and responsibilities are described in the terms of reference. Terms of reference for the committee were completed in 2010.

The committee ensures that the CFIA's BSE-related activities for all program elements, including the feed ban, are discussed and developed in a coordinated, complimentary and consistent manner.

At least once a year, the committee will review and communicate its performance and terms of reference to ensure it is serving its intended objectives and to recommend any changes it considers necessary.

Successful, yet sustained action required

The creation and recent changes to the BSE committee structure and the CFIA's participation in the single-window Market Access Secretariat demonstrate ongoing efforts to address coordination and collaboration issues. The CanSurvBSE has also had a positive effect on coordination and communications with external stakeholders.

It is, however, important to note that communications and information flow between CFIA branches and with other stakeholders are evolving and these efforts need to continue, or even be enhanced in the case of emerging issues. As such, this issue was identified an ongoing concern for the BSE program in the present evaluation.

Recommendation 2:
Performance measurement and reporting

To improve current performance data collection and reporting practices, the CFIA should consider the following measures: dedicate targeted funding for BSE performance measurement and reporting and subsequent evaluations; develop and implement a results-based management plan that integrates both program management and senior management needs and external reporting needs. To this effect, the CFIA should dedicate at least one position to the coordination of BSE performance measurement activities across the organization and promote a performance management culture at all levels of the agency.

Partially complete

All Program indicators have been identified in 2009, tested in 2010 and are currently in production.

This recommendation has been completed as far as AHD role in the development of the Program KPI is concerned However; horizontal BSE related indicators have not yet been utilised.

The electronic (EOR) mechanism is in place; however not extensively used and the collection and manual entry of this data remains a concern.

There is no position dedicated to the coordination of BSE performance measurement activities.

Some progress, but ongoing difficulties

Despite actions to address the identified issues, ongoing challenges have been reported and were experienced during this evaluation (see limitations) regarding performance measurement and reporting.

As a complex/multi-activity horizontal program that has been integrated with other human and animal health activities as well as trade programming, BSE data collection and reporting remains diffuse, vast and not uniform.

Various BSE and AHBL committees have had lengthy and ongoing discussions around performance measurement and reporting. However, establish clear consensus and implementing proposed plans have met with limited success. As such, this issue was identified an ongoing concern for the BSE program in the present evaluation.

Recommendation 3:
HR and BSE expertise

To develop and sustain a critical mass of BSE expertise within the CFIA, succession planning should center on a commitment to hiring highly qualified staff and allow for an extended transition period so that new staff can be mentored and trained by existing BSE specialists within the agency.

Complete

The Agency's Human Resources Branch (HRB) and the Operations Branch have been working together to pilot succession planning.

Operations Branch has also been provided with key demographic information to inform the Agency's succession plans.

Comprehensive training programs with respect to BSE have been developed and implemented in the Animal Feed, Animal Health and Meat Inspection programs.

Limited tangible progress

A few discussions around training, mentoring and succession plans were held by the BSE Program Management Committee, AHBL and BSE Program Advisory Committee; none in the last year.

There appears to have been limited actions taken to address issues relating to HR and BSE expertise in the longer term.

This issue was identified an ongoing concern for the BSE program in the present evaluation.

Recommendation 4:
Financial tracking

To better manage and assess outputs of BSE tied funds, the CFIA should track funds at the level of specific activities funded under the BSE initiative. This would also allow for cost-effectiveness analysis, both at the level of sub-activities and program components.

Complete

In April 2009, procedures were put in place to enhance tracking and reporting of financial information.

In April 2011, an Oversight Committee for BSE Funding was created and is chaired by the Executive Director, Animal Health Directorate. The mandate of this committee is to ensure that Agency activities pertaining to the distribution and use of BSE funds are in accordance with the commitments made in the TB Submissions and the development of a long term strategic action plan to address reallocated funds.

Ongoing difficulties

Financial tracking continues to be ineffective. Despite the various BSE committees' concentrated efforts (in particular the BSE Fund Oversight Committee), financial tracking to tie BSE funds to BSE activities continued to be a critical challenge for the Agency.

When the business line structure was installed, BSE funding (and other types of "special funds") was pooled with A-base funds to be allocated within each Branch. However, this has not adequately addressed the financial tracking issues (and likely aggravated them). As a result, there is contradictory evidence about the use of BSE funding for BSE activities (e.g., some have reportedly been allocated to other priorities).

This issue was identified a priority concern for the BSE program in the present evaluation.

Recommendation 5:
BSE surveillance testing and the Sample Reimbursement Program

The CFIA's future decisions as to its BSE surveillance testing targets and the Sample Reimbursement Program (SRP) should continue to be supported by science-based guidelines addressing both the effectiveness of the risk management measures for BSE in animals as well as the effectiveness of the feed ban. The CFIA should also develop a communication strategy (including a consultation strategy) to better inform stakeholders with regard to ongoing discussions and developments on the future of the SRP. To this end, better coordination and communication is needed between those at the front lines of BSE testing.

Complete

A review of the BSE surveillance program from 2004 to 2010 was completed in December 2010. Terrestrial Animal Health Division will lead consultation on adjustments to the BSE surveillance program consultations with appropriate representation from the Animal Health and Food Safety Business Lines.

A briefing note outlining this approach and the major recommendations of the review will be put through CFIA Agency governance.

In order to more fully engage industry in the on-going design and implementation of the BSE surveillance program, Terrestrial Animal Health Division (TAHD) will use an on-going formal collaborative arrangement.

Successful – The Federal SRP has been maintained and additional actions taken to achieve sample submission targets

Although cuts to the federal Sample Reimbursement Program were anticipated, this program remains active at this time.

In addition to the maintenance of the federal SRP, the CanSurvBSE was recently created to address the falling surveillance numbers attributed in part to cuts to provincial sample reimbursement programs.

Recommendation 6:
Tracking, tracing and enforcement

The CFIA should continue to work with industry and other governments to ensure that the national cattle identification (ID) program and database is standardised and includes date of birth, movement capture, identification of deadstock, and better identification of premises. The CFIA should also enforce activities beyond tagging compliance (e.g., enforce the submission of data to the national database).

Complete

A review of the Strategic Management Plan is scheduled in winter 2012. Acceptable regulatory amendments were identified in 2011. Consultations are scheduled to be held in early 2012, with a regulatory amendment in 2014.

An IM-IT guidance document was completed in 2008. An audit of traceability data collected by the Canadian Cattle Identification Agency (CCIA) was completed in spring 2011. An auditing framework should be completed in 2012.

A manual of procedures for the program was completed in 2011. A compliance verification and enforcement training course, the implementation of CVS to the program was implemented in 2011.

The current Federal legislative framework does not provide the authority for parties to report cattle birth dates. This information is collected under provincial acts and regulations.

The Traceability National Information Portal approved in October 2011, is a project to develop a national information portal using information from industry and governments to provide authorized users with up to date traceability information.

CFIA has advanced the proposed legislative changes for traceability, and has completed a series of consultations with key stakeholders to facilitate the development of a traceability framework for Canada.

Moving forward with slight delays

The recent coordination, policy and legislative efforts around the livestock identification and traceability program will continue to help address the tracking, tracing and enforcement issues relative to the cattle identification program.

However, the cattle identification program is experiencing some delays due at least in part to its expansion into a wider animal identification program for other diseases beyond BSE, as well as some enforcement and verification issues in the field.

Recommendation 7:
Expanding export markets

To help manage industry expectations, the CFIA should clarify its role and responsibilities for this component, particularly in relation to other government agencies involved (e.g., DFAIT, AAFC). The CFIA should also continue to integrate BSE trade-related activities into broader country- or commodity-based strategies and confirm the added value of its regionally focused technical experts

Complete

The creation of the Market Access Secretariat (MAS), housed at AAFC, has provided a focal point for interdepartmental collaboration on the identification and prioritization of market access opportunities and the development of strategies and action plans to advance market access issues. An MOU outlines roles and responsibilities of CFIA and AAFC, and establishes a governance process to support our joint efforts. CFIA has co-located technical specialists with AAFC MAS employees to contribute to strategies and action plans. This is augmented by the extended team of program specialists that continue to negotiate technical conditions and certificates as appropriate. DFAIT is also an active participant in the MAS-led activities.

IPD conducted a review in 2009, with industry and government stakeholders, of the pilot program which posted CFIA veterinarians in key embassies abroad. The review confirmed the value of CFIA specialists in establishing stronger relationships with key regulatory counterparts to resolve market access issues and gain a better understanding of foreign systems in managing risks to food safety and animal and plant health. The review also identified opportunities to improve the program, including training and support to specialists before posting, as well as ongoing communication with and integration of specialists abroad into the broader market access team. The program has now been formalized as an ongoing. Two positions have been added with specialists now for Beijing, Tokyo, Mexico City, Brussels, Moscow and New Delhi.

Complete

The CFIA's participation in the single-window Market Access Secretariat, housed at the AAFC, demonstrate ongoing efforts to address coordination and collaboration around export markets.

Also, the work of the recently restructured BSE committees is clarifying the requirement to open markets by hosting talks with foreign delegations. This information serves as an important input into determining future levels of effort required on behalf of CFIA and the beef industry as a whole.

Recommendation 8:
Continuity and scope

As part of its continued efforts to develop an integrated approach to food safety, public health and risk management for animals and humans, the CFIA should expand the scope of specific activities within the Enhanced BSE Initiative beyond BSE, particularly to other issues related to transmissible spongiform encephalopathies (TSEs) and other emerging diseases. Finally, the CFIA must ensure that it continues to meet stakeholder expectations and maintain confidence regarding the legitimacy of its regulatory/enforcement role for BSE.

Complete

The CFIA organized a forum on the BSE controls in 2010, a broad range of stakeholders and international speakers were brought together to develop an improved understanding of perspectives on risks, allowing for the development of a road map for the future of BSE control.

CFIA envisions the roadmap as a communication tool and part of its approach to long term BSE management planning. The roadmap will be a living document and the first version is expected to be completed by December 2012.

Progress, but key activities ongoing

Since 2011, the newly created BSE committee structure has been significantly mobilized to expanding the scope of programming to include of other TSEs. Intensive discussions in committees, including the CanSurvBSE, are yielding inclusion of other animal groups in the continuity and scope of the program (e.g., goat, sheep, etc.).

Continuing efforts around program renewal and the BSE Roadmap (not yet completed, put on hold until after the 2012 EFB Review) should also help address ongoing issues around the continuity and scope of the Program.

10.6 Appendix F - Comprehensive Strategy for Managing BSE in Canada (Enhanced BSE Programming) Logic Model

Available upon request

10.7 Appendix G – Summary of Case Study

Available upon request

Date modified: