Archived - Evaluation of the Fresh Fruit and Vegetables Program

Table of Contents

• Executive Summary
• 1.0 Introduction and Context
  • 1.1 Description
  • 1.2 Evaluation Context (Scope and Objectives)
• 2.0 Methodology
  • 2.1 Description
  • 2.2 Limitations
• 3.0 Key Findings
  • 3.1 Evaluation Issues/Questions
    • 3.1.1 Relevance: Continued Need for Program
    • 3.1.2 Relevance: Alignment with Government Priorities
    • 3.1.3 Relevance: Alignment with Federal Roles and Responsibilities
    • 3.1.4 Performance: Achievement of Expected Outcomes
    • 3.1.5 Performance: Demonstration of Efficiency and Economy
  • 4.0 Conclusions and Recommendations
• Annex A: A Profile and description of Fresh Fruit and Vegetable Safety and Quality Program Context, Activities and Issues
• Annex B: The FFVP Results Logic and Systems Performance Framework
• Annex C: Evaluation Approach
• Annex D: Risk Management and Outcomes-Based Standards: A Review of International Experience Related to Food Safety
• Annex E: Interview Guides
• Annex F: Evaluation Matrix
• Annex G: Bibliography

Executive Summary

The need for an internal evaluation of the Fresh Fruit and Vegetable Program (FFVP) was identified in the CFIA's Evaluation Plan (2012-2017), approved by the Evaluation Committee in June 2012. The FFVP falls under Program Activity 1, Food Safety Program in the CFIA's Program Alignment Architecture (PAA) which links to the overall strategic outcome of having a safe and accessible food supply and plant and animal resource base.

This evaluation examines the FFVP's relevance and performance during the period from 2008 to 2013, with respect to the Program's two foci:

1. food safety goals; and,
2. food quality, related to marketing and market access

At the request of senior management, this evaluation also examines the implications of the past and present with regard to future directions, including current Safe Food for Canadians Action Plan and Inspection Modernization (IM) initiatives.

For the purposes of this evaluation, the FFVP consists of all areas in the Agency that address fresh fruit and vegetable issues, whether housed in Policy and Programs, Operations or Science Branch.

Relevance

Document reviews, a literature review and interviews and consultations with key informants suggest that the FFVP is relevant to the food safety needs of the FFV sector. However, its inspection activities focus heavily on serving market access and quality needs.

Systematic risk assessments worldwide and key informant interviews suggest that there is a strong need for a regulatory program devoted to food safety in the FFV sector. International agreements related to specific commodities continue to require grading and other quality-related support. This suggests a need for some form of market access support.

A review of documents shows that the FFVP's food safety objectives clearly align with federal government priorities, as set out in the Safe Food for Canadians Act (SFCA), Inspection Modernization (IM) initiative and with Agency strategic outcomes. The support for market access is explicit in current regulations. Once in force, the Safe Food for Canadians Act will provide authorities for regulations to support trade. Support for market access is indirectly aligned with the Agency's current strategic outcomes.

Up to now, the CFIA's main FFV safety role, responsibilities and activities have been oriented to selective monitoring, reaction and remediation, as there are no comprehensive federal food safety regulations in this sector that would facilitate robust establishment-based risk analysis and oversight. Furthermore, the CFIA's main active FFV role to date, as shown by delivery activities and resource use, has been to support market access. The FFVP has a long standing history oriented to market access, beginning with grade standards in the 1930s. Implementation of the SFCA is expected to significantly strengthen the food safety aspects of the FFVP, through the implementation of preventive food safety regulations.

All evidence clearly suggests that the FFVP will need more resources in order to deliver on its quality aspects. Similarly, considerable new resources will be required to achieve the identified food safety objectives for this sector. Considerable effort will need to be spent on repositioning the Agency with FFV stakeholders, who have historically seen the CFIA primarily as an enabler in securing market access, rather than providing regulatory oversight to meet food safety objectives. The evidence further suggests that a number of fundamental Program foundations, which are necessary for implementation of the SFCA and IM food safety agendas, have not yet been developed. These include knowledge of industry players and associated risks, as well as a shared understanding with all stakeholders of outcome-based FFV safety requirements. Factors relating to broad socio-economic and historical conditions, regulatory instrument requirements, governance and management and the engagement of regulated parties, present particular challenges that are expected to create a difficult context for CFIA's planned transition. A FFVP pilot project conducted from 2010 to 2012 saw inspectors across the country inspecting vegetable establishments for the first time. Many of the inspectors were not prepared to undertake this new work in the area of safety when they had only been involved in supporting market access up to that point. A great deal of communication, engagement and training is deemed necessary in order to undertake such work in future.

Performance

Evidence suggests that industry requirements and private sector programs play a stronger role in influencing FFV safety than existing CFIA program activities. Industry leadership for food safety is appropriate and expected, as the safety of its products are its responsibility. In the case of the FFV sector, the food safety role of the CFIA has been particularly weak because of the absence of a comprehensive CFIA FFV food safety program, including preventive food safety regulations and extensive sampling and testing.

CFIA activities for produce safety are reactive and focus on limited national sampling and testing programs for hazards. As a result, there is insufficient evidence to conclusively state to what degree FFV products in Canada are safe. However, it should be noted that there is a large body of evidence, both from the literature and from domestic surveillance, monitoring and studies, indicating that fresh fruits and vegetables are increasingly becoming a food safety concern.

The FFVP quality and market access activities have successfully supported the sector's needs, though some concerns have been raised regarding the Agency's resource limitations and, given these limitations, its allocation of resources between safety and market access.

There is no consistent accounting of resource investments in key FFVP activities and, as a result, the Program's economy and efficiency cannot be calculated or demonstrated.

The findings from the evaluation suggest that the CFIA will face many important challenges to integrate FFV components and stakeholders into a single food safety program by 2015-16, when the SFCA is planned to come into force. In order to support the FFV food safety and quality objectives, the CFIA should consider the following key recommendations.

Recommendations

List of Abbreviations

AAFC

Agriculture and Agri-Food Canada

CAPA

Canada Agricultural Products Act

CFIA

Canadian Food Inspection Agency

CPLR

Consumer Packaging and Labelling Regulations

CPLA

Consumer Packaging and Labelling Act

FDA

Food and Drugs Act

FDR

Food and Drug Regulations

FFVs

fresh fruits and vegetables

FFVP

Fresh Fruit and Vegetables Program

FFVR

Fresh Fruit and Vegetable Regulations (or FFV Regulations)

FSAP

Food Safety Action Plan

FSIP

Food Safety Investigation Program

HC

Health Canada

IM

Inspection Modernization

LAR

Licensing and Arbitration Regulations

MCAP

Multi Commodity Activities Program

PAA

Program Alignment Architecture

PCP

Prevention Control Plan

PHAC

Public Health Agency of Canada

RCC

Regulatory Cooperation Council

RPWs

registered produce warehouses

SFCA

Safe Food for Canadians Act

TBS

Treasury Board of Canada Secretariat

1.0 Introduction and Context

1.1 Description

Program Structure

The Canadian Food Inspection Agency (CFIA) regulates fresh fruits and vegetables (FFVs) to verify that fresh produce imported, exported and marketed inter-provincially under Federal legislation is safe, wholesome and graded according to standards, packaged and labelled to avoid fraud or misrepresentation and marketed in an orderly fashion. The average Canadian consumes about 55 kg of vegetables each year, and 39kg of fruit. Canada imported an average of $5.9 billion worth of fruits and vegetables annually from 2008-11, and exported an average of $1.5 billion.^Footnote 1

The FFVs industry is, for the most part, an unregistered sector. The sector is not subject to any mandatory registration requirements, although there are approximately 90 registered produce warehouses (RPWs) across Canada, which participate in a voluntary program for permission to grade potatoes, apples and blueberries marketed in interprovincial trade. By contrast, the typical registration function of other CFIA food programs is for the purposes of granting "permission to operate" and for implementation of food safety systems in food establishments.

The Fresh Fruit and Vegetables Program (FFVP) is housed in the Agrifood Division of the Policy and Programs Branch and is internally divided into two sections: safety and quality. In 2011-12, the Program's expenditures represented slightly more than 3.5% of the Agency's overall budget, that is, $27.9 million (and $33.6 million in 2012-13). Historically, oversight of the two sections was provided via two chiefs that reported to a single national manager responsible for all program policy, performance, reporting and implementation. During a recent restructuring, responsibility for the quality (non-food safety) section of the FFV Program was transferred to the National Manager of Processed Products, Maple and Honey Program, involving the transfer of two full-time equivalents (FTEs) at Headquarters. The Operations Branch supports the FFV Program as a whole, coordinating delivery between safety and quality sections and prioritizing tasks required for Program delivery. The Science Branch manages the sampling programs, including all planning, laboratory analysis and reporting, plus targeted surveys.

Table 1 shows FFVP expenditures and budgets for fiscal years 2008-13.

*See Table 2 for breakdown

The way in which the Agency reported FFVP finances changed dramatically in 2011-12 because the Agency realigned its Program Alignment Architecture (PAA), downsizing from eight Programs to five. The breakdown of Program activities is shown in Table 2. The Agency has produced no reports to indicate which activities (see sub-sub activities in Table 2) were added or newly attributed to the FFVP to account for the near tripling of expenditures from 2010-11 to 2011-12. There are also no reports showing the breakdown of budget figures by activity or by any other criteria for the Program as a whole. To determine Program budget figures per activity would require a review of the FFVP financial planning per branch. Each branch manages its own budget and does not participate in any summary exercise to create a single comprehensive budget for all Agency expenditures associated with fresh produce. Expenditures are similarly documented within each branch separately. This gap in accounting and planning is not reflected in any of the recommendations because the FFVP will become part of the Agency's larger food program during the Agency's modernization initiatives over the next few years. However, for planning purposes, it is important for the Agency to ensure that such expenditures are more accurately, consistently and comprehensively tracked. Tables 1 and 2 were created from information developed for the evaluation by the Corporate Management Branch.

Notes on Tables 1 and 2: A) Actual expenditures by sub-sub activity come from the year-end actual expenditures by sub-sub activity report; B) The allocation of internal services is included pro rata in the actual expenditures by sub-sub activity (ranging from 17-21%), which accounts for the difference in the figures in the Departmental Performance Report; C) The budget for the FFVP specific to the International Collaboration and Technical Agreements (ICTAs) sub-sub activities was allocated on a pro-rata basis according to the fiscal year end sub-sub-activity expenditures in ICTAs; D) Main Estimates of 2011-12 by PAA were restated for this exercise. In 2011-12, the Agency realigned its PAA, downsizing from 8 Programs to 5. During the realignment, the CFIA worked to accurately apportion its Planned Spending to the New Programs. However, while preparing the 2011-12 Departmental Performance Report, it was noticed that authorities did not properly align with actuals. This issue was corrected in the 2013-14 Report on Plans and Priorities. Therefore, Main Estimates of 2011-12 by PAA were restated in this exercise to provide a more accurate basis for evaluation; E) Some of the increase in Science Advice expenditures in 2012-13 is reportedly due to the transfer of approximately 50 CFIA staff members from the Policy and Programs Branch to the Science Branch.

The number of staff working directly in the FFVP is outlined by Branch in Table 3.

Table 3: FFVP Full Time Equivalent Staff 2012-13^Footnote 2

Authorities

The following acts and regulations are enforced by the FFVP:

• Canadian Food Inspection Agency Act (CFIAA)
• Canada Agricultural Products Act (CAPA), to be repealed on the coming into force of new regulations under the Safe Food for Canadians Act
• Consumer Packaging and Labelling Act (CPLA)
• Consumer Packaging and Labelling Regulations (CPLR), food related provisions are also intended to be repealed with the making of new regulations under Safe Food for Canadians Act
• Food and Drugs Act (FDA)
• Food and Drug Regulations (FDR)
• Fresh Fruit and Vegetable Regulations (FFVR or FFV Regulations)
• Licensing and Arbitration Regulations (LAR), *possible removal of the LAR is proposed under a Regulatory Cooperation Council Initiative.

The Safe Food for Canadians Act (SFCA) was enacted in November 2012. It will come into force along with the making of new regulations, whose target date is 2015-16. Under the new Act and proposed regulations, commodity-based programs such as the FFVP will cease to exist as distinct programs, as the Agency moves to a risk-based multi-commodity approach to food safety oversight, making prevention a priority across all food commodities and implementing a common inspection approach across all foods. The proposed regulations will have a number of horizontal requirements applicable to all federally regulated food commodities traded inter-provincially and internationally, such as involving licensing, preventive controls, traceability, record-keeping and a review mechanism. Certain commodity specific food safety requirements will be maintained (e.g. regarding slaughter), as will other commodity-specific and consumer-protection provisions (e.g. standards of identity, grades, inspection marks, labelling and packaging). While many of the FFVP functions will continue, this transition will have broad implications for the FFVP sector.

Regulatory Framework

The FFVP has two distinct mission areas, with different histories, cultures, relationships and levels of CFIA experience. These two areas are:

1. food quality, related to marketing and market access; and
2. food safety

Quality-Marketing-Market Access

The marketing of fresh fruit and vegetables is regulated under the Canada Agricultural Products Act (CAPA) by way of the Fresh Fruit and Vegetable Regulations (FFV Regulations) and the Licensing and Arbitration Regulations (LAR).

The CAPA regulates the marketing of agricultural products in import, export and interprovincial trade and provides for national standards and grades of agricultural products for their inspection and grading, for the registration of establishments and for standards governing establishments.

The FFV Regulations set out packaging, labelling, non-food safety (grade standards) and food safety requirements for fresh fruits and vegetables marketed in import or interprovincial trade, whether they are supplied fresh to the consumer or for food processing.

The FFV Regulations provide that for the following commodities traded inter-provincially, each load must undergo a CFIA shipping point inspection and be certified as meeting an established grade:

• Apples from British Columbia, Ontario, Quebec, New Brunswick or Nova Scotia;
• Potatoes from Ontario, Quebec, New Brunswick, Nova Scotia or Prince Edward Island; and
• Blueberries packaged in containers of 6 litres or less from Nova Scotia, New Brunswick or Prince Edward Island.

Inspection certificates are valid for not more than three days (including Saturdays and holidays). Exceptions for mandatory shipping point inspections are made for establishments that are registered with the CFIA in the Registered Produce Warehouse Program. Establishments who are part of this program are subject to regular inspections to demonstrate the continued ability to pack product to quality standards, as an exception to the general requirement for inspection for each load.

The Licensing and Arbitration Regulations were designed to ensure fair and ethical trading practices in the international and inter-provincial trade of fresh fruits and vegetables. The Regulations require Canadian importers (with more than $230,000 in annual sales) to be licensed with the CFIA and/or be a member of the Dispute Resolution Corporation (DRC). A CFIA license or membership in the DRC provides a mechanism for dispute resolution of any quality or payment issues in produce transactions. There are approximately 115 license holders and 800 DRC members. Licenses are issued by the CFIA's Destination Inspection Service (DIS). The DIS is an impartial service provided by the CFIA to support the resolution of disputes related to FFV quality. Inspections offered by the CFIA deal with the following issues: defects, witnessing of product disposal and weight, size, or actual count. There are fees associated with DIS services and the CFIA is moving towards full cost-recovery. The DIS was created in 2007 from functions that were performed up to then by the FFVP.

There are programs within the FFVP that were established to ease the grade inspection burden. As referenced above, the Registered Produce Warehouse Program (RPWP) allows products to move between provinces without the need for mandatory quality inspections on each outbound load. The regulatory basis for this Program is contained in Part X of the FFV Regulations, which allows any establishment that prepares commodities for which the Regulations have established grades^Footnote 3 to become registered. However, in practice, only those establishments preparing apples and potatoes have registered as they are the commodities for which an inspection is mandatory. The purpose of the Program is two-fold: to provide the industry with flexibility to ship produce without waiting for an inspector (by demonstrating capacity and compliance with grading and grade declarations); and to reduce the CFIA's resource requirements by reducing the number of shipping point inspections. There are 90 potato and apple establishments in this program.

The FFVP's Canadian Partners in Quality (C-PIQ) Program allows potato producers to enter into an agreement with the CFIA requiring them to maintain performance standards which allows them to complete their own export certificates for loads of potatoes being shipped to the U.S. The Agency conducts audits and reviews company quality assurance manuals and records to verify compliance. There are approximately 15 establishments in the C-PIQ Program. This program is not supported through regulations but instead uses legally binding participation agreements between the CFIA and potato establishments.

Safety

A general requirement for safe food is found in the FFV Regulations under the CAPA. Subsection 3 (1) of the Regulations states that "no person shall market produce in import, export or interprovincial trade as food unless it . . ." is safe to consume. In addition, the Regulations set out in detail various banned contaminants. The delivery of the safety aspects of the CFIA's FFVP is broken into two basic components: (1) sampling conducted under the National Microbiological Monitoring Program (NMMP), the National Chemical Residue Monitoring Program (NCRMP) or the Food Safety Action Plan (FSAP); and, (2) reactive investigations, including recalls, trace backs and directed sampling.

The CFIA uses two broad strategies to monitor for the presence of various hazards in foods including FFVs. These two strategies include regular monitoring programs such as the NMMP and the NCRMP as well as targeted surveys and studies. The NMMP and NCRMP are monitoring programs that apply to a wide variety of domestic and imported food products (not exclusively FFVs). The NMMP monitors the level of microbiological contamination in the food supply, testing for a variety of high-risk pathogens including E. coli, Listeria monocytogenes, Salmonella and Shigella. The NCRMP tests for the occurrence of chemical residues, food additives and toxic metals. Both monitoring programs are intended to identify trends, risks and areas for follow-up actions to help minimize potential health risks for consumers. Based on multiple factors such as consumption by Canadians, compliance history and considerations regarding vulnerable sub-populations, the level of monitoring is identified and adjusted as needed. In addition to regular monitoring activities, additional testing is conducted under the FSAP in areas of highest or emerging concerns. Under the FSAP enhanced surveillance program, information is collected through targeted surveys on chemical and microbial hazards as well as allergens. Currently these surveys include the testing of identified priorities (FFV examples include cantaloupes, tomatoes, leafy greens and fresh herbs) for various microbial hazards. Surveys investigating the levels of allergens (sulfites in various FFV commodities such as lychees and grapes) have also been completed). Studies on pesticide levels in FFV traded intra-provincially have also been completed.

The investigative component of the FFVP does not include prescriptive requirements such as licensing (outside LAR requirements), on-site inspections, or the development and implementation of preventive control plans, as no preventive food safety regulations currently exist for this sector. As a result, current oversight is primarily responsive to food recalls and emergency issues, such as responses to outbreaks of food-borne illnesses, complaints from the public and evidence of microbiological contamination from the NMMP which may require directed sampling as part of an investigation into the origin of the contamination.

It is interesting to note that despite the absence of formal food safety requirements and protocols through legislation or CFIA policy, a voluntary industry-led program has emerged called CanadaGAP. CanadaGAP is a food safety program for companies that produce, pack and store fruits and vegetables, designed to help implement effective food safety procedures within fresh produce operations^Footnote 4. This industry-run safety program was developed with the help of funding from Agriculture and Agri-Food Canada (AAFC). The industry estimates that possibly as many as 70% of FFV producers in Canada are members of this program. This includes those selling to large retailers who are said to be increasingly insisting on the implementation of effective risk mitigating processes and strategies in this sector.

A full profile and description of these two functions, primarily compiled by FFVP management for the purposes of this study, is provided in Annex A. This information will be drawn upon in addressing the issues in Section 3.

Summary Program Context

The FFVP has a long standing history, oriented to market access, quality and, sometimes, to manage the availability and supply of certain commodities. This began as far back as the 1890s for grade standards for apples, and more broadly across commodities in the 1930s, all to assist trade through the use of a common language. See the "Service" History with Sector discussion in Section 3.1.4 below. The regional presence to the sector in these areas remains strong. In this context, the CFIA is seen by industry as a key partner in helping them market their products and secure access to other markets.

Inspection Modernization (IM) and the new Safe Food for Canadians Act (SFCA) represent transformative initiatives for CFIA which place the emphasis on food safety. With respect to the oversight of FFV products, stronger food safety rules under the new Act will mark an expansion of the regulatory scope of the Program which had previously been focused on quality and market access.

1.2 Evaluation Context (Scope and Objectives)

The need for an internal evaluation of the FFVP was identified in the CFIA's Evaluation Plan (2012-2017), which was reviewed by the Evaluation Committee and approved by the CFIA President in June 2012 (confirmed in the Evaluation Plan 2013). The timing of this evaluation allows the CFIA to conform to requirements of the Policy on Evaluation for evaluation coverage of all direct program spending, and to provide observations about the FFVP as changes in program design and regulations are being made.

The FFVP falls under the CFIA's PAA Program Activity 1, the Food Safety Program, which is linked to the overall strategic outcome of having a safe and accessible food supply and plant and animal resource base.

This evaluation examines the FFVP's relevance and performance during the period from 2008 to 2013. However, it also examines the implications of the past and present with regard to future directions, including current modernization initiatives.

For the purposes of this evaluation, the "Program" under evaluation includes all related areas across the Agency, including those in Policy and Programs, Operations and Science Branches. Relevant FFVP activities undertaken by external programs that interface with the FFVP such as the Plant Protection Program, Destination Inspection Services and the Office of Food Safety and Recall were also taken into consideration. This evaluation was initiated in March, 2013. Its data collection was completed in November, and its reporting in December.

Evaluation Issues

As required by the TBS Policy on Evaluation (2009), the main aim of the CFIA evaluation of the FFVP was to address the five core evaluation issues outlined below. In addition, the evaluation team was asked by its Advisory Committee to look at examples of other program areas that transitioned from a quality to a safety focus and how other governments regulate FFV, all to share lessons learned. See Methodology Section for details.

Relevance

Issue #1 Continued Need for Program

Assessment of the extent to which FFVP policies, processes and relationships continue to address a demonstrable need and are responsive to the needs of Canadians.

Issue #2 Alignment with Government Priorities

Assessment of the linkages between FFVP objectives and (i) federal government priorities and (ii) Agency strategic outcomes.

Issue #3 Alignment with Federal Roles and Responsibilities

Assessment of the roles and responsibilities for the federal government in delivering the FFVP.

Performance (effectiveness, efficiency, and economy)

Issue #4 Achievement of Expected Outcomes

Assessment of progress toward expected outcomes (including immediate, intermediate, and ultimate outcomes) with reference to performance expectations and program reach, as well as program design, including the linkage and contribution of outputs to outcomes.

Issue #5 Demonstration of Efficiency and Economy

Assessment of resource utilization in relation to the production of outputs and progress toward expected outcomes.

2.0 Methodology

2.1 Description

A realistic contribution analysis approach^Footnote 5 was used for this evaluation to respond to the five core issues of the evaluation. This approach provided a rigorous process for validating and refining the understanding of the results logic and the factors affecting performance. (See Annex C for further details on the approach.) The Evaluation Directorate worked with the FFVP to further identify areas of risk to the Program and their impacts. Multiple lines of evidence were used to provide a better understanding of the Program. These multiple lines of evidence included a needs review, instruments review, literature review related to instruments and initiatives, international comparative case analyses, file review evidence analysis and interviews to validate, extend and address issues. Triangulation of data was done by issue.^Footnote 6 Documents reviewed are included as Annex G. An Advisory Committee was established for the Evaluation with senior management representation from the Science Branch, Operations Branch, Policy and Programs Branch and Public Affairs Branch. In addition, the Committee included the Director of Evaluation from Transport Canada to provide an outside perspective.^Footnote 7 The Committee reviewed and provided advice on the Evaluation Framework, the preliminary findings and draft report.

footnotes: *Interviews followed the protocol set out in the interview guides, in Annex E. Note that in several cases multiple call-backs were conducted with interviewees.
**Consultations were conducted by e-correspondence, meetings and phone calls and aimed at developing and testing the evaluators' understanding of key issues related to the FFVP and the FFV sector. Examples include consultations on past, present and proposed processes as applied, compared and contrasted for FFVs vs other groups, and it included attempts to clarify performance and resource usage information. The numbers in this category exclude the call-backs to respondents interviewed in the previous column. Note that while the perspectives of major stakeholders were considered key, non-stakeholders were included where appropriate.

A performance measurement strategy did not exist for the FFVP^Footnote 8. As a result, a logic model for the Program was produced by the evaluation team as part of the evaluation planning, and was subsequently validated through interviews and consultations, including with program staff. A systems performance framework was developed, identifying key factors deemed to be critical to the successful delivery of the FFVP. An FFV Evaluation Framework was developed at the evaluation's outset and included a list of the key sectors involved in the achievement of the FFVP missions. These components represent a theory of the Program and the relations between them were tested as part of data collection and analysis. This information should also serve to inform the development of the integrated food program, as it identifies key FFVP-related outcomes and activities. Specific FFVP elements can be considered in conjunction with the generic model to determine which are specific to this sector. See Section 3.1.4 and Annex B for the logic model.

2.2 Limitations

The circumstances in which this evaluation was conducted created special limitations:

Some of the limitations noted above presented significant challenges to being able to conduct a conventional evaluation. The basic strategy to address the data quality problem included a major review of key mechanisms and the experiences of other groups with such mechanisms. The results of the review were used to complement the limited and sometimes non-existent experience of the FFVP in areas like establishment-based inspections, risk-based prevention activities, and outcome-based standards. This approach was designed to leverage as many data sources as possible. Interviews were conducted with a variety of stakeholders: CFIA staff, industry players, other government departments and international groups.

3.0 Key Findings

3.1 Evaluation Issues/Questions

3.1.1 Relevance: Continued Need for Program

Assessment of the extent to which FFVP policies, processes and relationships continue to address a demonstrable need and are responsive to the needs of Canadians.

There is a need for the FFVP to continue in some form, whether as a distinct program approach or as part of an integrated approach envisaged under the Safe Food for Canadians Act.

The Issues Analysis Summary, issued in April 2013 by the CFIA's Food Safety Division, suggests that there is a 'large body of evidence', both domestically and internationally, relating to increased incidents of FFV-related food-borne illnesses. The report concludes that fresh fruits and vegetables "are increasingly becoming a food safety concern."

From a food safety perspective, the health risks inherent to this sector are very significant. According to recent U.S. studies, food-borne illness outbreaks for produce are reportedly comparable in number to those for meat products. Although a number of differences characterize these two sectors, including regulatory and export requirements, it is worth noting that the Agency spends significantly more on regulatory oversight for the meat sector than it does for the FFV sector. In 2011-12, the Agency allocated 35.2% of its budget to Meat and Poultry, and 3.8% to Fresh Fruits and Vegetables.

A study conducted in 2013 by the U.S. Centre for Disease Control (CDC) calculated that for 4,589 foodborne illness outbreaks with single etiology between 1998 and 2008, causing 120,321 illnesses, it was produce commodities that accounted for 46% of these illnesses while meat and poultry accounted for 22%^Footnote 10.

An alternate view was provided in a 2013 paper by the Center for Science in the Public Interest (Outbreak Alert! 2001-2010 A review of Foodborne Illness in America). This paper agreed with the basic numbers of the CDC report "which put the spotlight on produce as causing the most cases of foodborne illness", but it arrived at different conclusions. The paper analysed the outbreaks by volume of product consumed, which identified seafood as the greatest cause of disease, followed by poultry^Footnote 11. Both reports take great pains to list many caveats and assumptions, but both recognized produce as a major cause of food-borne illness outbreaks.

Recent assessments suggest that FFV risks are heightened, or greater than previously thought, due to:

• Consumer habits (fresh, but conveniently packaged food);
• General lack of a "kill step" as products are often eaten raw;
• Large batches of fresh fruits and vegetables transported over long distances;
• Globalization of produce value chains and increases in both the variety and quality of products produced has dramatically increased the number of potential contamination points;
• Increases in imports from developing countries with various safety systems;
• Increased scientific knowledge and experience regarding microbial risks in fresh produce have illustrated the risks; and,;
• Increased interest by key trading partners (U.S., EU, Australia, New Zealand) and recognition that a reactive system is not acceptable.^Footnote 12

Given the findings from both internal and external reviews of the sector, it is clear that there is an ongoing need for some form of programming for this sector in order to satisfy food safety objectives.

Looking ahead, the Agency's modernization initiatives demonstrate a continued focus on food safety and on building a more consistent and comprehensive framework across commodities. These efforts are intended to strengthen food safety, including for the FFV sector. From a stakeholder perspective, major associations support, both in words and in actions, the CFIA's shift in orientation more towards safety – further supporting the perceived need for a program covering this sector. This is evidenced by industry led quality and safety assurance programs that have emerged in Canada, the most prominent one being CanadaGAP, specifically for the FFV sector.

From a quality-marketing-market access perspective, given U.S. certification and interprovincial inspection requirements, there appears to be a continued need for this aspect of programming. The CFIA's role in certifying that products meet quality standards (grades) for the FFVP sector has dominated the relationship between the CFIA and industry. While the CFIA's role is traditionally one of regulator for most other commodities, it is generally considered to be providing a necessary service by FFV producers who rely on it. The current need for the service stems from regulatory requirements driven by trade and market access objectives. Its activities largely involve certifying grades to demonstrate adherence to target product specifications relating to product characteristics, including the purpose of avoiding fraud, with rare or no links to food safety.

Pursuant to subsection 2.2(2) of the Fresh Fruit and Vegetable Regulations, the Minister may issue a Ministerial Exemption (ME) to exempt a product from regulatory labelling and/or packaging requirements in order to prevent or alleviate a shortage of product in Canada. The intent is to reduce shortages while maintaining overall supply levels at a controlled level. This is intended to maintain relative price stability which in turn serves the commercial interests of Canadian producers and consumers. While only a handful of interviewees commented on MEs, they consistently perceived them positively as providing critical support to manage the availability and supply of certain commodities. The number of MEs varies from 10,000 to 20,000 loads of FFVs per year, essentially allowing bulk product shipments to address shortages for the purpose of managing prices.

A regulatory requirement exists for apples, potatoes and blueberries, destined for interprovincial trade, to be inspected to ensure they meet the established grade. Subsection 29 (2) allows for a release permit (referred to as a "Y" release) to be issued by inspectors if they do not have time to inspect the load within the prescribed time (24 hours if an inspector is in the area or 48 if not). Over a three year period from 2010 to 2013, the CFIA issued 2,648 "Y" releases and conducted 646 inspections. These results illustrate that the Agency had capacity to inspect these products only 25% of the time. As a result, the core quality-based inspection requirement was not met 75% of the time. In other words, these products are shipped and traded without an inspection most of the time. The level of 'exemptions' from inspection through "Y" releases raises questions about the need for such inspections.

The need for the Registered Product Warehouses Program and for the Canadian Partners in Quality (C-PIQ) Program is similarly linked exclusively to supporting industry's trade needs. The former allows products to move between provinces without the need for mandatory quality inspections on each outbound load, which provides the industry with the flexibility to ship produce without waiting for an inspector, and reduces the CFIA's resource requirements by reducing the number of shipping point inspections. The C-PIQ Program allows potato producers to enter into an agreement with the CFIA requiring them to maintain performance standards, which allows them to complete their own export certificates for loads of potatoes being shipped to the U.S.

These programs represent examples of how quality requirements are satisfied through alternative means that have received support from industry. These illustrate that even where regulatory requirements exist relating to issues of quality, the role of the CFIA may vary. Industry has expressed the need for such services, but no evidence was found to indicate that government has to be the one that provides them.

Quality related services have been successfully disengaged from the Agency and taken up by the private sector for such commodities as seeds, beef and fertilizer. The experiences from these changes in responsibility provide rich lessons from which the Agency can draw during any consideration of transitioning FFV grading services to the FFV sector. These experiences also point to challenge areas. The following are some of the key contextual factors that appear to have been important to the transitions:

• Relative stability in terms of grading practices and low controversy in terms of grade levels were important conditions;
• A low number of key players kept the negotiations straight forward;
• Comparatively high price increases (cost recovery by the CFIA) just prior to transition helped to incentivize the industry to take over grading;
• Financial assistance was provided by AAFC to help industry make the transition;
• Trained government graders were recruited and used in the private take-over;
• Resistance was strongest from small growers who preferred the simpler (and at one time free) Government of Canada grading services, and;
• Respected industry players were key to leading the transition.

The extent to which these success factors are present has not been fully explored in this evaluation.

3.1.2 Relevance: Alignment with Government Priorities

Assessment of the linkages between FFVP objectives and (i) federal government priorities and (ii) Agency strategic outcomes.

FFVP Objectives vs. Federal Government Priorities

Food safety and market access, the two key objectives of the FFVP, have been long-standing government priorities. They were once again recognized in the 2013 Speech from the Throne (delivered on October 16, 2013) as priorities for the Canadian Government going forward. The recently passed Safe Food for Canadians Act (SFCA) was also recognized as "a significant milestone in strengthening Canada's world-class food safety system". In addition, trade and market access priorities are also recognised in the 2013 Speech from the Throne, further re-enforcing the linkage between the objectives of the FFVP and the current government priorities.

The Safe Food for Canadians Act and the proposed regulations that are under development are expected to provide a clear foundation for the food safety objectives relating to FFV. With regard to the Government and especially the Agency's role with respect to quality, marketing and market access for FFV products, there is an overall commitment found in the CFIA Act's preamble, which states that "the Government of Canada wishes to promote trade and commerce". Under the authority of the Canadian Agriculture Products Act, the FFV Regulations outline requirements for product grades and other marketing criteria. The regulations are referred to as Regulations Respecting the Grading, Packing and Marking of Fresh Fruit and Vegetables. They include "Schedules" that outline grade and container size requirements. The Agency is expected to import the existing FFV Regulations under the regulatory framework supporting the SFCA, so they will continue to be in force after the implementation of the SFCA. The plan to continue these regulatory requirements supports the view that this objective and these provisions remain necessary and relevant at this time, though, as noted above, there is some uncertainty regarding the fundamental need that they serve. A number of interviewees questioned the continued need for the federal government to have a role in trade support for the FFV sector.

FFVP Objectives vs. Agency Strategic Outcomes

Analysis of the FFVP's objectives vs. the Agency's strategic outcomes shows that the Program is generally consistent with the CFIA's strategic outcome: A safe and accessible food supply and plant and animal resource base^Footnote 13. Clearly, the food safety aspects of the Program align directly with this strategic outcome, but the FFVP's quality and market access activities are more indirectly linked to it.

The Program's quality and grading support work addresses this strategic outcome if 'accessible' is interpreted as including access to foreign markets. The results statement for the CFIA's International Collaboration and Technical Agreements Program states: "international markets are accessible to Canadian food, animals, plants and their products."^Footnote 14 The extent to which the CFIA should go to encourage accessibility is not elaborated on. The prescribed indicator "number of unjustified non-tariff barriers resolved" does not necessarily suggest an active role in export certifications. However, it should be noted that in the Agency's 2014-15 Report on Plans and Priorities, under the "Raison d'être" heading, it identifies "contribute to market access for Canada's food, plants, and animals" as the fifth of five reasons for the delivery of inspections and other services. While fulfilling trade objectives for this sector is clearly relevant to the Agency's mandate and strategic outcomes, program documents, interviews and performance data raised uncertainty regarding whether existing regulations and programming best align with current objectives and priorities.

In short, the Agency's own outcome statements, particularly those in the SFCA and related consultation documents, focus on food safety. However, there are currently more inspectors at the CFIA working on quality and market access for FFV than on FFV safety.

The existing FFVP is out of alignment with a number of key elements or principles being implemented under the CFIA's modernization agenda. Notably:

• A risk-based preventive approach: According to CFIA internal documents^Footnote 15 and interviews, there is a lack of preventive food safety programming specific to fresh produce in Canada - except for those run by industry;
• Outcome-based standards: A review of current practice suggests that there are currently no safety related outcome-based standards for fresh produce in place. According to a recent CFIA consultation document, outcome-based approaches to standards emphasize what must be achieved, not necessarily how it must be achieved^Footnote 16. While the Agency is currently developing such standards, there are a number of challenges, such as the requirement for substantial guidance documentation (also under development). See Annex D for an analysis of international experience in the development of outcome-based standards. This is also discussed in Section 3.1.4.
• Comprehensive licensing: There is currently only 'voluntary' registrations for market access purposes in place in this sector,^Footnote 17; and,
• Standardized establishment-based inspections. Such inspections have never been done by CFIA for the FFV sector, outside of a small pilot project in 2010-12.

The extent to which key elements of the new initiatives can and will be effectively implemented by the Agency with respect to the FFV sector is daunting given its past orientation, limited capacity and its service relationship with the sector. (See performance observations in Sections 3.1.4 and 3.1.5) Actions underway as part of the broader Agency transformation process are intended to address many of these implementation challenges. This includes industry and stakeholder consultation on the development of stronger food safety rules (including outcome–based standards and licensing requirements), the delivery of improved training to inspectors to align with the new inspection model, the implementation of a Statement of Rights and Service to better articulate roles and responsibilities for industry and consumers and guidance and compliance promotion materials (including model systems) for regulated parties.

3.1.3 Relevance: Alignment with Federal Roles and Responsibilities

Assessment of the roles and responsibilities for the federal government in delivering the FFVP.

Recent FSAP test results, CFIA risk assessments and key interviews point to the importance of emerging CFIA roles, such as those related to microbial (pathogens) threats (see the Science Branch's 2013 risk assessment^Footnote 18).

As noted above in Section 3.1.2, the increasing linking of FFVP with food safety roles and responsibilities is demonstrated by recent legislative initiatives and the transformational agenda in the Safe Food for Canadians Action Plan consultation document^Footnote 19. At the same time, the Agency has signalled its intention to integrate the provisions relating to quality and market access into the proposed regulations supporting the SFCA. This suggests an intention to maintain a dual focus on quality and safety for the FFV sector.

Reviews are mixed with respect to the CFIA's role in quality and market access. The role has been seen by some as posing an unnecessary burden on industry. Some industry representatives questioned whether it was necessary to subject them to CFIA verifications of their grade in order for them to sell their product in certain markets.

The CFIA's increasing emphasis on food safety programming for the FFV sector is nascent and unclear in some areas at the time of this evaluation (e.g. the full extent of the CFIA's authorities to inspect and regulate on the farm have been inconsistently interpreted by CFIA staff). Given the intention to maintain support for the quality objectives while apparently significantly expanding the safety objectives for this sector, there will be a strong need for additional resources to effectively satisfy both types of objectives. The current level of coverage for food safety is recognized as insufficient in CFIA planning documents and by all the CFIA managers who were consulted^Footnote 20.

3.1.4 Performance: Achievement of Expected Outcomes

Assessment of progress toward expected outcomes (including immediate, intermediate, and ultimate outcomes) with reference to performance expectations and program reach, as well as program design, including the linkage and contribution of outputs to outcomes.

The provision of grading and export services is discussed below.

The CFIA's Performance Measurement Framework presents the FFVP's expected results in terms of compliance to regulations (as is standard for most of the Agency's outcomes):

• Federally registered fresh fruits and vegetable establishments meet federal regulations, and
• Fresh fruit and vegetable products for domestic consumption meet federal regulations.

Regarding the first expected result, the only registered FFV establishments are in the Registered Produce Warehouse Program (RPWP), focussed on quality and market access. Quality and market access performance issues are discussed in the second part of this Section. The second expected result primarily relates to food safety, and is discussed in the first part of this section below.

Program documents provide a bit more detail on expected outcomes. The FFVP's Program Basics document states that intended outcomes are to ensure that fresh fruits, vegetables exported, imported or marketed inter-provincially comply with federal regulations, are graded for economically significant factors, and are packaged, labelled and traded to avoid fraud. An additional objective is to facilitate access to international markets by providing export certification to meet the import requirements of other countries^Footnote 21. As discussed in Section 1, the regulatory requirements are primarily found in the FFV Regulations. Regarding food safety, it essentially states that no person shall sell food that is not safe to consume.

As noted in section 3.1.1, three quarters of the produce subject to interprovincial certification is currently being shipped without a physical inspection, under a "Y" release. This suggests that CFIA physical inspection has not been a significant factor in inter-provincial trade. While the scope of this evaluation did not focus on the detailed requirements of quality inspections, interviews and the review of documents suggest that current C-PIQ and RPW processes have imposed some undue burden in the eyes of some regulated parties.^Footnote 22

Safety

A study published in 2013 in the Journal of Food Protection, highlighted that foodborne disease outbreaks associated with fresh fruits and vegetables have been increasing in occurrence worldwide. Some recent high profile examples include over 11,000 cases of Norovirus related to strawberries in Germany and China in 2012 and 147 illnesses and 33 deaths related to Listeria contamination in cantaloupes in the United States (2011).^Footnote 23 The global context is relevant to domestic safety, given that Canada is a net importer of fresh fruits and vegetables, with an estimated value of $5.9 billion annually. Domestic consumption of imported products is particularly prominent in winter months, contributing to the import of 86% of the fruit and 41% of the vegetables consumed in 2000.^Footnote 24

In Canada, 27 produce-related outbreaks, resulting in over 1,500 cases of illness, have occurred from 2001 to 2009. At the federal level, between 2002 and 2011, the CFIA conducted 57 recalls of fresh fruits and vegetables. Of these, 12 were class I recalls indicating a high risk of serious health problems; 32 were class II recalls indicating that consumption can lead to short-term non-life-threatening health problems.

While available data suggests that Canadians experience relatively low levels of food-borne illness outbreaks, analysts have pointed out that Canada's food safety system is reactive, lags behind other countries, and investment is needed to ensure it can adequately protect Canadians, as stated in a 2013 article in the Canadian Medical Association Journal by Richard Holley.^Footnote 25 According to this study, foodborne illness surveillance is needed to ensure safety from gastrointestinal infections caused by bacteria such as toxigenic E. coli, Salmonella, Campylobacter and Listeria (all prevalent in the FFV sector). Without a more rigorous surveillance system such as those used in the United States, Australia / New Zealand and Europe,^Footnote 26 there is a possibility that Canadian estimates are not as accurate as other developed countries and that Canadians are not as relatively 'safe' as they might think.

While reported product test results are consistently over the Agency's 95% target for compliance with the regulations, there are significant limitations to the coverage of this generalized reporting^Footnote 27 and key interviews and profile documents suggest that the sampling is not generally representative of the sector or the risks.

Under the NMMP, CFIA inspectors sample produce based on an annual plan. The sampling is random but follows the planned selection of product type, based on those at higher risk of contamination. However, the sampling plans are not generally based on statistical analysis, but rather on the available operational capacity to obtain and ship samples to the labs and the labs' capacity and capability. Over the past five years, the collection of samples has ranged from 76-89% of what was planned, as shown in Table 6 below. The Agency's Performance Measurement Framework has a target of 100% for "percentage of fresh fruits and vegetable samples inspected versus the number planned".

Table 6: NMMP Samples 2008-13^Footnote 28

Samples taken include monitoring and baseline studies. Compliance percentages are not available for the above table.

Compliance rates for the NMMP were not available for the years prior to 2011. For 2011-12, they were 99.2% for vegetables and 99.5% for fruit, which were described as fairly typical rates. The figures for 2012-13 were not available.

Under the NCRMP fresh produce samples are assessed for pesticide residues as well as heavy metals, based on maximum residue limits (MRLs) set by the Pest Management Regulatory Agency (PMRA). Exceeding an MRL is a violation of Canadian regulatory requirements that triggers Program and Operations follow up; however, it is not generally regarded as an immediate health risk as the danger is due to exposure over time. Chemical residue testing of FFVs is more extensive than sampling for microbial hazards. Shipments are selected randomly, samples are collected throughout the year, and the number of samples planned is proportional to the importance of the commodity in the diet, taking into consideration the origin of the food and the volume of imports. As such, the sample numbers are statistically representative of the commodities, but not of the countries of origin.^Footnote 29

The NCRMP under-delivered on its sampling against annual plans and to a greater extent than the NMMP, with delivery rates ranging between 59.6% and 81.5% as shown in Table 7 below. It is also worth noting that while the highest performance was in 2012-13, at 81.5%, the level of planned sampling for that year was significantly lower (by more than 30%) compared to previous years.

Note that the source for the compliance percentage is CFIA's DPR. Compliance was not calculated the same way in each fiscal year.
For 2011-12 and 2012-13 imported and domestic FFV compliance was reported separately, their compliance percentages were averaged for this table

As discussed in previous sections, there are concerns regarding the level of sampling. These concerns were confirmed by interviews with key CFIA Science and Program staff and relate primarily to the fact that the low number of samples taken in the NMMP is not representative enough to provide any early warning of a food safety issue. Inspection Modernization plans for FFVs focus on establishment-based safety inspection, rather than only sampling at warehouses, packing plants and retail outlets. In a 2010 article in the Canadian Medical Association Journal, Richard Holley suggested that because pathogens occur with low frequency, the statistical power of product sampling is inadequate to provide confidence in the overall safety of foods subject to testing.^Footnote 30 Arguably, this supports the contention that industry run proactive systems reviewed by a regulatory body may be superior to a product sampling regime – especially one which is already struggling in terms of representativeness.

The problem with moving to a proactive systems based approach is that there is currently limited knowledge about some of the higher risk areas – such as establishments that grow, and in many cases package, produce. There is no history of inspection of produce establishments, let alone sampling. This represents an unknown set of food safety risks. The FFVP has noted that if implementing food safety measures to prevent the introduction of known or reasonably foreseeable biological hazards is the objective, then consideration must be given to the most common sources of contamination of fresh produce - the on-farm production or growing practices for fruits and vegetables, and extending into on-farm and post-farm practices of harvesting, packing, holding, handling, preparing, processing and distributing of fresh produce commodities of high risk – in order to bring improved food safety controls further upstream from the retail marketplace. (See Annex A.)

A pilot project was conducted from 2010 to 2012 with the aim of inspecting 30 establishments that packaged and re-packaged fresh leafy vegetables, green onions and/or herbs in the four geographic areas of the country (Atlantic, Quebec, Ontario and Western). Only 17 of the establishments were inspected, for reasons outlined below. While the pilot could not be fully implemented, it was successful in identifying institutional barriers to produce establishment inspection. A key challenge identified was inspector capacity and support. While guidance material was developed to support the pilot, this was a completely new activity for many inspectors. Some of the inspectors who participated in the pilot considered the type of work to be the responsibility of inspectors at a higher pay level. In addition, some inspectors and their managers were not comfortable taking on a food safety inspection role because their experience and prior training had been limited to assessing quality. Inspectors also noted challenges associated with their lack of familiarity with the establishments and their business and operational scope. The draft Establishment Inspection Pilot Project report explained that the move to establishment-based inspection was a major change for inspectors that would require significant additional communication, engagement and training. The study further noted a lack of performance information on compliance at both an output and outcome level. It is not evident that any improvement has been made in this situation since the draft report was written in 2012.

The SFCA and attendant modernization agenda commit the Agency to require such establishments to obtain licenses, starting in 2015. The licenses would require a food safety system, referred to as a "preventive control plan" by the CFIA, such as a Hazard Analysis Critical Control Point (HACCP) system. Until now, the Agency's move in this direction has been limited to regulatory planning (drafting outcome-based regulations and guidelines) and the pilot noted above. In the meantime, the FFV sector has developed its own establishment safety system program, known as CanadaGAP. The development of CanadaGAP illustrates a recognized need by industry for FFV safety programs and measures. CanadaGAP is recognized by the Global Food Safety Initiative and the CFIA. The existence and high level of voluntary adherence to the CanadaGAP represent an opportunity for the CFIA to co-ordinate its preventive control plans and overall risk-management approach with stakeholders, as the Agency seeks to significantly expand its food safety oversight for FFVs.

Outside of the FFVP, the CFIA provides support for the development of preventive control plans, like HACCP, through the On Farm Food Safety Recognition Program. This Program is funded by AAFC and delivered by the CFIA. It formally recognizes on-farm food safety systems that industry has developed and implemented and that meet the regulatory and program requirements. The Program has completed its Part One Technical Review of CanadaGAP at the time of this evaluation and anticipates an application for full recognition shortly.^Footnote 31 In this way, the CFIA has influenced industry food safety by supporting an industry led self-credentialing system. CanadaGAP is also one of the few associations that have begun their assessment application process for the more nascent Post Farm Food Safety Recognition Program.

As noted above, the FFVP's safety monitoring has been minimal so information on compliance is insufficient to assess achievement of food safety outcomes. However, while the targeted studies under the Food Safety Action Plan have indicated strong overall compliance, they have also identified commodities through imports with worrisome results. There is an indication that risks may exist in less tested commodities involving rarer, though highly consequential sensitivity reactions. As yet unreleased FSAP results reveal contraventions of requirements under the Canadian Food and Drug Act. While this is only one finding, it suggests that some importers are ignoring Canadian residue regulations in areas not frequently tested.

Quality and Market Access

As previously noted, the quality and market access components of the FFVP took up the bulk of the operational resources during the review period and represent the key historical basis of the Program. There are 30 produce commodities for which grade standards have been prescribed under Section 3 (1) of the FFV Regulations (See Annex A). Most of these were established in the 1930s. However, only four products now require inspections (apples, potatoes, blueberries and onions). Despite this significant decrease in products subject to regulatory grading coverage, the FFVP is not able to meet the requirements in many instances.

As noted in Section 3.1.1, the Agency's use of "Y" releases as the rule instead of the clearly intended exception strongly suggests that the marketing support function is under-resourced to meet regulatory and Program requirements.

There is also a grade inspection requirement for potatoes, onions and field tomatoes traded with the U.S. A reciprocal agreement allows Canada and the U.S. to accept certificates from each other that verify grade requirements. There are close to 20,000 inspections required annually of the CFIA for the export of these products to the U.S., making these inspections the single largest resource cost for the FFVP.^Footnote 32

There are programs within the FFVP that have been established to ease the grade inspection burden. The Registered Produce Warehouse Program (RPWP) allows products to move between provinces without the need for mandatory quality inspections on each outbound load. The purpose of the RPWP is two-fold: to provide industry with the flexibility to ship produce without waiting for an inspector; and to reduce the CFIA's resource requirements by reducing the number of shipping point inspections. There are 90 potato and apple establishments in this program.

Under the RPWP, registered establishments must demonstrate the continued ability to pack product to quality standards. The frequency of the inspector's verification of this is based in part on the compliance history of the establishments. This amounts to one Program planned inspection per week for some establishments, and as little as one per month for those with good systems and history. However, limited FFVP resources mean that the average number of inspections per establishment is usually less than once every two months, as shown in Table 8 below.

The table illustrates that in 2010-11, the average number of inspections per establishment per year was less than 4. This number has since risen to more than 6.5, demonstrating a greater level of delivery on this quality inspection requirement. Note, however, that the Agency's Performance Measurement Framework has a target of 100% for "percentage of inspections of registered fresh fruit and vegetable establishments versus planned". (The RPW system is one example of the FFVP's limited experience with establishment inspections, though these are not related to food safety but rather to a service function to confirm compliance with quality standards.) The Agency's only safety-related enforcement experience comes from the 2010-12 pilot referenced earlier, aside from recalls, intensified sampling and post-incident inspections and verification of corrective actions.

The table below shows the average number of RPW inspections per establishment per year. Following the standard of one to four inspections per month outlined above, the minimum number of inspections per establishment per year would be 12 (if all establishments were inspected only once a month, i.e. if all establishments had the highest rate of compliance).

Table 8: Average Number of RPW Inspections per Establishment per Year^Footnote 33

Another aspect of the FFVP delivery relating to market access relates to supporting the issuance of Ministerial Exemptions. As noted in the discussion on Relevance (Section 3.1.1), Ministerial Exemptions were seen positively by industry interviewees as a critical support to supply management.

Table 9: Loads of Produce Moving Under Ministerial Exemptions by Year^Footnote 34

While food quality, grading, export certification and supply management functions appear to have contributed to market access and market stability in the past, concerns were raised by some stakeholders about the limited CFIA resource commitments and the perceived unevenness in delivery. The level of 'exemptions' from inspection through "Y" releases and the Agency's inability to meet inspection frequency requirements for RPWs again suggest serious under resourcing to meet Program and regulatory requirements.

Judging from an overall lack of complaints and the maintenance of access with major trading partners such as the U.S., it appears that the FFVP has succeeded in contributing to the marketing and market access of key Canadian FFV products. External stakeholders were all generally supportive of the effectiveness of the CFIA's quality program.

The traditional quality and market access focus of the FFVP has established a relationship with the industry such that the CFIA is seen as a service provider, providing support related to quality reviews (e.g. grade monitoring) and facilitating trade. This role will fundamentally shift as new safety requirements under the SFCA and the IM come into effect, such as mandatory licensing and periodic inspection. Mandatory establishment-based safety inspections have not been part of the CFIA's traditional role in the FFV sector. The planned change in its role represents an about-face, moving from providing a service, to inspecting establishments and enforcing food safety requirements, using the attendant sanctions.

FFVP Conditions of Success and Failure

In order to understand the outcomes of the FFVP, it is important to understand the conditions under which it operates. These conditions were identified and verified in evaluation interviews and consultations and are summarized below in the outcome-based performance analysis presented in a logic model and validated through discussions. The evaluation developed the logic model for the FFVP to identify more meaningful outcomes than compliance rates. By themselves, compliance rates are only as useful as the representativeness of the sample to the general population (statistical significance) and the degree to which they represent the key risks. Neither of these attributes applies to the NMMP, and only partially to the NCRMP, as outlined above. Furthermore, there are a number of outcomes that the FFVP needs to achieve to support such compliance. The outcomes in the logic model were verified with interviewees.

The logic model below introduces a brief discussion of the challenges in achieving its outputs and outcomes.

CFIA Fresh Fruit and Vegetable Program Logic Model
(Aligned with the CFIA corporate logic model and PMF)

Description for photo - CFIA's Fresh Fruit and Vegetable Program Logic Model

This is an overall logic model that describes the CFIA's Fresh Fruit and Vegetable Program. The logic model is organized vertically into the following sections: Inputs, Activities / Outputs, Reach / Engagement, Immediate Outcomes, Intermediate Outcomes and Strategic Outcome(s), with Inputs being at the bottom and Strategic outcome(s) being at the top.

Inputs

• At the bottom of logic model is the inputs section
• There is a rectangle with the words "Authorities, Acts and regulations, Human, Financial and Technological Investments" written in it. There is an asterisk beside the writing in the inputs box, it corresponds to the following note below the logic model: "The FFVP has been challenged by authorities oriented to goals supporting both trade and safety, coupled with significant gaps in resourcing levels for both safety and for quality grading activities. The CFIA Act is not explicit about the CFIA's role in providing market access support, and history has conditioned industry to expect the CFIA to continue to meet this need. The Act does require the Minister to support CAPA, which supports the FFV Regulations, though CAPA is to be repealed with the making of new regulations under the SFCA. The existing legislative framework does not contain all of the components of a strong safety-based regulatory system (though this is expected to be addressed with the coming into force of the SFCA). There is limited knowledge of the sector, its players and the food safety challenges, considerations and risks."
• There are four arrows going from the Inputs rectangle to a large rectangle in the Activities/ Outputs section

Activities / Outputs

• In the large rectangle of the Activities / Outputs section the words "outreach, consultation, surveillance, registration, inspection, investigation, recalls, enforcement redress" are written. There are two asterisks beside the writing in the activities/outputs box, it corresponds to the following note below the logic model: "The FFVP has gaps in performance and sector information necessary to support risk based decision making, and there are some internal operational challenges related to encouraging inspectors to take on new roles, such that an establishment-based inspection pilot could not be fully executed in 2010-12."
• The large rectangle has three lines coming from it, that link to three smaller rectangles in the Immediate Outcomes section

Reach / Engagement

• In the Reach / Engagement section there is a rectangle with dashed sides. This rectangle overlaps the rectangles in the Activities / Outputs and Immediate Outcomes sections
• Inside the Reach / Engagement rectangle the words "Constructive engagement" are written

Immediate Outcomes

• The three Immediate Outcomes rectangles have the following text inside them from left to right
  1. "Increased competencies and awareness of FFVP policies, regulations and legislation among regulatory partners and stakeholders". There are three asterisks beside the writing which corresponds to the following note below the logic model: "The engagement of regulatory partners has been challenging both internationally (e.g., in terms of ensuring compatibility with US requirements) and domestically (given the Agency's variable relationships with provinces, some federal agencies and association intermediaries)"
  2. "Increased level of compliance with FFVP policies, requirements, and regulations among stakeholders". There are four asterisks beside the writing which corresponds to following note below the logic model: "The regulated parties have historically been engaged through support of market access needs, which is used at the discretion of a select group of industry actors and is not necessarily representative of the highest risk groups."
  3. "Contribution to domestic, international standards and agreements for FFV production, marketing, distribution and use"
• There are arrows and lines connecting the three Immediate Outcomes rectangles to the three rectangles in the Intermediate Outcomes section

Intermediate Outcomes

• There are five boxes in the intermediate outcomes section; three of which are in a line, the two other are in a line above them
• The three lower boxes have the following text in them from left to right
  1. "FFV Products are compliant with marketplace requirements" There are five asterisks beside the writing which corresponds to the following note: "Food safety practices have arguably been mostly influenced by marketplace conditions and by the industry-led adoption of voluntary CanadaGAP certifications over the past five years. The CFIA is incorporating elements of these standards for its emerging model guidelines into the model preventive control plan guidelines that it is developing. Note that this finding of the key role of the marketplace and industry is supported in the literature (see Munro et al., 2012), which states that the measurable contribution of government inspection activities in minimizing food safety risks is unclear and that the failure to collect, analyze and or publish data makes it difficult to determine the magnitude of any contribution."
  2. "Marketplace practices are safe, efficient, innovative and environmentally sustainable" There are five asterisks beside the writing which corresponds to the same note shown for Intermediate Outcome 1.
  3. "Canadian standards are recognized internationally" There are five asterisks beside the writing which corresponds to the same note shown for Intermediate Outcome 1.
• The two boxes above have the following text in them from left to right
  • "Risks to the Canadian food supply from FFV are minimized or mitigated"
  • "Domestic and International markets are accessible to FFV industry and trade barriers eliminated"
• All the Intermediate Outcomes boxes are interconnected with lines and arrows
• There is an arrow coming from the top two Intermediate Outcome boxes that goes to a big rectangle in the Strategic Outcomes section

Strategic Outcome(s)

• The strategic outcomes rectangle has the following text written in it "A safe and accessible food supply, and plant and animal base" followed by these bullets
  • Minimized occurrence of preventable health incidents related to FFV in Canada
  • Production and distribution risks are mitigated, agricultural inputs to FFV are used in a sustainable manner, environmental risks are minimized
  • Middle sector of industry implements foods safety controls
  • Level playing field for imports and domestic manufacturers / Facilitated imports and exports
  • Safety + quality of domestic, imported + exported produce is maintained
  • There are no gaps between FFVP regulations and other relevant legislation including provincial, federal and international regulations
  • Public confidence in the regulatory framework and the integrity of the Canadian FFV industry is maintained (domestically / internationally)
  • Orderly marketing of FFV to provide fairness in the market place

*Inputs: "Authorities, Acts and Regulations, Human, Financial and Technological Investments" The FFVP has been challenged by authorities oriented to goals supporting both trade and safety, coupled with significant gaps in resourcing levels for both safety and for quality grading activities. The CFIA Act is not explicit about the CFIA's role in providing market access support, and history has conditioned industry to expect the CFIA to continue to meet this need. The Act does require the Minister to support CAPA, which supports the FFV Regulations, though CAPA is to be repealed with the making of new regulations under the SFCA. The existing legislative framework does not contain all of the components of a strong safety-based regulatory system (though this is expected to be addressed with the coming into force of the SFCA). There is limited knowledge of the sector, its players and the food safety challenges, considerations and risks.

**Activities, outputs: "outreach, consultation, surveillance, registration, inspection, investigation, recalls, enforcement, redress"
The FFVP has gaps in performance and sector information necessary to support risk based decision making, and there are some internal operational challenges related to encouraging inspectors to take on new roles, such that an establishment-based inspection pilot could not be fully executed in 2010-12.

***Immediate outcome 1: "Increased competencies and awareness of FFVP policies, regulations and legislation among key regulatory partners and stakeholders"
The engagement of regulatory partners has been challenging both internationally (e.g., in terms of ensuring compatibility with US requirements) and domestically (given the Agency's variable relationships with provinces, some federal agencies and association intermediaries)

****Immediate outcome 2: "Increased level of compliance with FFVP policies, requirements and regulations among stakeholders" The regulated parties have historically been engaged through support of market access needs, which is used at the discretion of a select group of industry actors and is not necessarily representative of the highest risk groups.

*****Intermediate outcomes:

1. "FFV products are compliant with marketplace requirements"
2. "Marketplace practices are safe, efficient, innovative and environmentally sustainable"
3. "Canadian standards are recognized internationally"

Food safety practices have arguably been mostly influenced by marketplace conditions and by the industry-led adoption of voluntary CanadaGAP certifications over the past five years. The CFIA is incorporating elements of these standards for its emerging model guidelines into the model preventive control plan guidelines that it is developing. Note that this finding of the key role of the marketplace and industry is supported in the literature (see Munro et al., 2012), which states that the measurable contribution of government inspection activities in minimizing food safety risks is unclear and that the failure to collect, analyze and or publish data makes it difficult to determine the magnitude of any contribution.

The following discussion of the FFV sector's conditions provides more detailed findings affecting the achievement of outcomes presented above.

"Service" History with Sector

As has been noted, the history of the FFVP has been based primarily on grading (quality) and discretionary assistance. There is no history of the CFIA conducting compulsory food safety inspections. No sanctions are currently used, meaning that there is no apparent deterrence mechanism for non-compliance.^Footnote 35

The relationship between the CFIA and industry associations continues to encounter some challenges. The intended change for the CFIA from a service role to an enforcement role has been explained to the sector's associations. However, this change is not fully supported by the associations, some of which openly question the need for any inspector to enter their properties. The associations have been very vocal about their main concern that nobody should be exempt from the licensing requirement. They are somewhat distrustful of the CFIA further to the federal government's 2012 Budget announcement (March) that container size regulations would be repealed - a decision that was made without industry consultation. There was significant and sustained criticism of the decision from the produce sector and beyond (container size regulations also affect other commodities, including processed products and meat). In November 2012 the Agency announced an extended period of consultation on the intended repeal. The current plan is to publish the regulatory amendments in the spring of 2014.^Footnote 36

Stakeholders are strongly advocating for the creation of a level playing field with no exceptions for smaller firms or for other reasons. The inclusion of any exceptions could continue to negatively affect the CFIA's relationships with stakeholders. The importance of stakeholder engagement and support for any given regulation has been very clearly linked as a key success factor to compliance.^Footnote 37

Challenges Unique to FFV Food Safety

The FFV sector's food safety risks are significant because there is often no "kill step" in food preparation. Cooking is the kill step for most food products, but the nature of fruit and to a lesser extent vegetables, means they are often eaten raw.

The FFV sector is also very heterogeneous and different areas have different risks (one size does not fit all). Particularly for fruit, of which Canada imports approximately 80%, there is an increasing variety of products and countries from which they are imported. The challenge of inspecting such a large and increasing variety of products is particularly acute for the CFIA given the limited FFVP resources.

A more recent change to the sector is the increasing food safety risks. There are strong consumption trends recently for pre-cut and packaged FFVs, which heightens the risk of contamination.

Surveillance Information and Knowledge of Sector 'On-site' Practices

The CFIA does not register or inspect FFV establishments for produce safety. This lack of a registration framework limits the CFIA's knowledge of the market players and its ability to effectively engage them. This has caused a challenge to proactively implement food safety priorities, such as the recent Listeria policy.

FFVP's safety monitoring has been minimal so information on compliance is insufficient to assess achievement and identify risks, outside of a few focused FSAP projects, as previously discussed. Without a history of establishment inspections, there are no performance data or baseline numbers on the types of establishments and their food safety practices. This lack of knowledge of establishments and their food safety risks suggests that the move to establishment inspections will require a long-term, multi-year implementation strategy. This is also supported by the results of the 2010-12 pilot study, discussed above. It will therefore take a number of years before sufficient risk data is collected to develop a truly risk-based inspection system for the FFV sector. In other words, in the absence of knowledge about most of the existing produce establishments, including where and how they operate and how compliant their products currently are, appropriate risk analysis, according to World Health Organization standards, cannot be conducted.^Footnote 38

The planned integration of the FFVP into a new food program under the SFCA is expected to significantly strengthen the Agency's food-safety oversight of the FFV sector. The new programming for this sector should provide the Agency with greater insight into the sector and a deeper understanding of the food and facility-level risks. This should in turn contribute to more effective and strategic surveillance to help strengthen the Agency's risk-based delivery model for this sector.

Authorities and Jurisdiction

Food-borne illness outbreaks related to FFVs locally sourced or otherwise limited to intra-provincial movement are considered by some in the CFIA to be outside of the Agency's purview, though the Food and Drugs Act does provide authority. However, all provinces except Quebec rely on the CFIA's national recall infrastructure and authorities even when provincial legislation supports similar reactive mechanisms. Note that most recalls are in the non-federally registered sector, which includes FFVs. Many key risk areas are covered by what is considered by some CFIA staff to be provincial jurisdiction (namely on farm production) and federal–provincial relationships and governance has, according to Program management, been uneven with regard to FFV oversight and responsibility.

The FFVP has noted that there is potential, within the shared jurisdiction, to provide seamless oversight along the fresh produce continuum, including the potential setting of standards by Health Canada for elements such as the agricultural water, biological soil amendments (composted manure), the health and hygiene of handlers, the proximity of animals and farm equipment, buildings and tools, which would require enforcement by provincial and territorial governments (Note that standards set by Health Canada may be enforced only if they are reflected in legislation or regulations). Of benefit would be the building of synergies with federal, provincial and territorial partners (authorities at farm level for production control oversight). (See Annex A.)

Economic Trade (Exports) Considerations

As noted, most international trade for the FFV sector is with the U.S. The U.S. views fresh produce as a priority in light of numerous outbreaks. The anticipated Produce Rule under the U.S. Food Safety Modernization Act will introduce preventive controls far exceeding current Canadian requirements, though new SFCA regulations are expected to require a significant and possibly comparative number of new controls (this potential for higher U.S. requirements creates heightened market access concerns).

When major contextual factors such as those noted above are considered, it is difficult to attribute past and present food safety outcomes to the FFVP. (See Annex B for a summary of the contribution analysis performed for FFVs.) The CFIA and the FFVP have been ill-equipped in terms of authorities and regulatory tools such as compulsory safety-based licensing, establishment based inspection, investigation and enforcement and historical relationships. The CFIA and the FFVP also have gaps in information regarding the sector, its practices and its situational risks. The sector itself has shown a strong will and initiative to develop its own safety-oriented certification procedures which are globally recognized and cover an estimated 70% of the FFVs sold in Canada. The privately run CanadaGAP initiative has evolved over the last decade to apparently become the major program influencing FFV food safety in Canada. This has been recognized by the CFIA in the fact that the CanadaGAP program has been identified as a model preventive control plan in the guidelines it is currently developing for the sector. Up to now, food safety practices in the Canadian FFV marketplace have apparently been led by private-sector initiatives, assisted by Canadian government financial support (AAFC). The opportunity at present for the CFIA to influence food safety appears limited to emergency activities exercised such as recalls and food-borne illness outbreak warnings.

CFIA Capacity

Interviews and document reviews all suggested that there are both quantitative and qualitative capacity gaps in the FFVP's ability to implement a stronger risk-based orientation, including comprehensive licensing, establishment-based inspection, outcome-based standards and a prevention focus. The shortfalls in both safety sampling and quality certifications have been described in previous sections. A gap in the qualitative aspects of the new type of inspection process required by the IM initiatives as applied to FFVs was shown by the results of the 2012 pilot study, as noted in previous sections. (See Annex D for a more complete analysis of the requirements for risk assessment, management and communications, as well as for outcome-based standards.) Further, the FFVP faces significant challenges to risk assessment, management and engagement.

Some of these challenges may be addressed through broader activities already underway at the Agency. The CFIA is implementing a broad range of initiatives and changes to support the modernization of its regulatory framework and food inspection model. These modernization efforts are intended to integrate most programming across all food commodities to create a single food program framework. Efforts to support this modernization include compliance promotion and communication products and stakeholder consultations and engagement. As these efforts continue to unfold, it will be important to ensure that unique FFV considerations are adequately incorporated to address particular FFV sector challenges.

However, it is too early to pronounce on the success of major new elements supporting the Agency's modernization initiative, and these initiatives appear to be dependent on some factors that will be difficult to achieve for the FFV sector. In particular, the FFVP faces significant challenges in terms of risk assessment, management and engagement.

The above noted areas of concern represent important risks to the FFVP component's state of readiness to integrate effectively into the Agency's modernization efforts. The lack of knowledge of the sector and level of compliance and the lack of understanding about the associated risks and challenges distinguishes this sector from other regulated sectors (e.g. meat). At the same time, the lack of awareness about and relationship with many of the players in the sector is similar to challenges faced by the non-federally regulated sector. However, the long-established history with the largest players in the FFVP, which is focused almost exclusively on service delivery to support their trade and marketing requirements, will represent a particular challenge unique to this sector. Given the integration of the FFVP into the broader food program, challenges and delays for this sector could have broader impacts on modernization efforts.

3.1.5 Performance: Demonstration of Efficiency and Economy

Assessment of resource utilization in relation to the production of outputs and progress toward expected outcomes.

As noted in section 1.1, accounting information for the specific resource investments in key FFVP activities is limited. Estimates on the time and money allocated to specific functions have varied extensively between groups,^Footnote 39 such as the number of inspectors dedicated to safety activities (from 7 to 18).

According to interviews with FFVP staff, and a review of files, there is an apparent potential for testing overlap and duplication with the Public Health Agency of Canada and the Pest Management Regulatory Agency with regard to separately funded FSAP testing. According to consultations with the Programs and Science Branches, information on sampling and testing is not consistently shared, or co-ordinated across groups. This suggests an opportunity for both better economy and efficiency and better risk management if sharing and collaborating were improved, starting with clarified roles and relationships.

Relationships with and the roles of provinces vary greatly in the FFV sector, presenting an opportunity for greater governance, co-ordination, consistency and integration with the CFIA for future food safety functions. The confusion recently observed in the sector during a Sector-Government meeting about provincial roles in regulating traceability, illustrates the problems caused by the lack of role clarity (e.g. one provincial ministry representative reportedly expressed the view that traceability is only a federal responsibility).

The level of inspection and required traceability has been seen by some industry representatives as unduly burdensome. Industry interviews have suggested this in general; however it is notable that specific burden levels have not been tracked.

Effectiveness, efficiency and economy apparently vary by FFV commodity area and respondent area. In some cases both quality and safety inspections and tests, as they currently exist, are not seen as adding value according to some sector respondents interviewed.

According to interviews with FFVP staff and a review of files, the FFVP is challenged by some of the major elements of the Agency's modernization's planned implementation. For example, as discussed, there are some limitations to the level and quality of Program engagement with stakeholders; some of the roles and responsibilities are not clearly defined, and there is a lack of resourcing and performance data.

4.0 Conclusions and Recommendations

Evidence suggests that the FFVP is relevant to the food safety needs of the sector, though its efforts are heavily weighted to serving market access and quality needs. While the food safety aspects of the FFVP are closely aligned with the Government of Canada's priorities and the Agency's strategic outcomes, roles and responsibilities, this alignment is less clear with respect to the quality aspects of the current program. While the intended trade objectives are aligned with government priorities, it is unclear whether the existing regulatory requirements supporting the quality role are necessary to satisfy the underlying trade policy objectives (outside the scope of this evaluation). It is also unclear if the CFIA's current delivery model is fully aligned with its roles and responsibilities or is optimal from an efficiency and outcome perspective.

A number of contextual factors severely hinder the FFVP's ability to influence food safety at the present time (e.g. broad socio-economic and historical conditions and the engagement of intermediaries and regulated parties). These factors also create a difficult environment for FFVP's intended transition to Inspection Modernization and the single food program. The FFVP faces acute challenges in risk assessment, management and stakeholder engagement, outcomes based standards, the preventive control plan (PCP) process itself, and its relationships with industry. (See Annex D for a review of the risk management and outcome based standards challenges, as they have been experienced internationally.) The FFVP also faces inspection and testing challenges owing to factors noted above and the reduced resources available to conduct inspections and product safety testing. The Program already suffers from a gap in resources necessary to provide full support to market access, as demonstrated by the high proportion of "Y" releases used to avoid direct inspection.

The implication is that it will be extremely challenging for the FFVP to achieve a "fully operational" component of the food safety program in the near term. Given the integration the FFVP into the broader food program, challenges and delays for this sector could have broader impacts on modernization efforts.

Systematic risk assessments worldwide and key informant interviews suggest that there is a strong need for a regulatory program devoted to food safety in the FFV sector. International agreements related to specific commodities continue to require grading and other quality-related support. This suggests a continued need for some form of market access support.

Recommendation 1

Up to now, CFIA's main FFV safety role, responsibilities and activities have been oriented to selective monitoring, reaction and remediation rather than being driven by establishment-based risk analysis. Furthermore, the CFIA's overall main active role to date, as shown by delivery activities and resource use, has been to support market access. The passage of the SFCA demonstrates the government's commitment to enhanced food safety for all programs and once implemented is expected to improve the situation.

CFIA activities for produce safety are reactive and focus on limited national sampling and testing programs for hazards. As a result, there is insufficient evidence to conclusively state to what degree FFV products in Canada are safe. However, it should be noted that there is a large body of evidence, both from the literature and from domestic surveillance, monitoring and studies, indicating that fresh fruits and vegetables are increasingly becoming a food safety concern.

Evidence suggests that industry requirements and private sector programs play a stronger role in influencing FFV safety then current CFIA program activities. Contributing to this is the absence of a comprehensive CFIA FFV food safety program, including preventive food safety regulations and extensive sampling and testing.

The FFVP quality and market access activities have successfully supported the sector's needs, though some concerns have been raised regarding the Agency's resource limitations and the relative resource allocation between safety and market access concerns given these limitations.

The evaluation identifies a range of challenges and complexities relating to the FFVP, including the noted gaps in regulatory oversight of safety objectives and delivery on quality and marketing objectives. While the transition of the FFVP into the larger new Food Program is expected to address core gaps in food safety programming for the sector, this transition will add a further degree of complexity and added challenge to ensuring that objectives particular to the sector are realized. As the Agency's new Food Program is being designed and operationalized to incorporate all commodities including FFVs, it will be important to ensure that key FFVP-related outcomes are achieved through effective activities, and evidenced through appropriate performance measures.

The Treasury Board Secretariat (TBS) requires all program managers to develop and implement performance measurement strategies for their programs.^Footnote 40 TBS further notes that "for new programs, the PM Strategy should be developed at the program design stage when key decisions are being made about the programming model, delivery approaches, reporting requirements, including those of third parties, and evaluations".^Footnote 41

Given this context, it is recommended that:

Recommendation 2

The CFIA should develop and implement a performance measurement strategy for the FFV components of the new Food Program as a foundational element to program design and ongoing monitoring of effectiveness. This FFV focused PM Strategy should be integrated into the overall Food Program PM strategy.

Recommendation 3

To support the implementation of the SFCA, the CFIA should develop a targeted engagement strategy for the FFV sector that recognizes the distinct features and relationship challenges to this sector. This should particularly recognize the new risk-based food safety oversight for this sector which will represent a fundamental change from the current service orientation to facilitate trade.

Annex A - A Profile and description of Fresh Fruit and Vegetable Safety and Quality Program Context, Activities and Issues

The following profile information is based on information developed and provided by FFVP staff in support of this evaluation study.

Fresh Fruit and Vegetable – Safety

The regulatory system, program policies and operational activities for fresh produce have primarily been focused on grades rather than safety. The approach to food safety has been characterized by sector management itself as reactionary - driven by complaints and cases of foodborne illness, rather than proactive and preventive - driven instead by establishment inspection and risk analysis.

The CFIA currently conducts monitoring for microbial and chemical hazards in fresh fruits and vegetables as part of the National Microbiological Monitoring Program and National Chemical Residue Monitoring Program. These regulatory programs monitor products under the CFIA jurisdiction to ensure compliance with Canadian food safety standards. Generally speaking, monitoring is used as a tool by regulatory agencies when there is no evidence that there is a specific hazard present in the food. Monitoring is used when there is a high expectation that the food is in compliance.

Under the National Microbiological Monitoring Program, products that are deemed to be at higher risk of contamination are sampled randomly by the CFIA inspection staff at fresh fruit and vegetable facilities (e.g. packers, re-packers, wholesalers, or importers). A full lot is selected and 5 sampling units are taken for analysis in accordance with Health Canada's Compendium of Methods. Regulatory action is taken in the case of reported violation(s). The total number of samples tested annually ranges from 523 to 1038 and targets all types of high-risk products such as lettuce, spinach and green onions, as well as all types of peppers, tomatoes, melons and berries. All samples are analyzed at the CFIA laboratories. According to program managers, the planning of sampling plans for microbiological hazards testing is not based on statistical analysis, rather it is largely based on the available Operational capacity to obtain and ship samples to the labs and the lab capacity and capability (available methods) to accept and analyze FFV samples among other commodity groups. During 2011-12 692 fresh and 100 ready-to-eat vegetables were subject to 3327 tests, overall 99.2% of the vegetables sampled were assessed as satisfactory. Also during 2011-12 a total 202 fresh and ready to eat fruits were sampled and were subject to 828 tests, overall 99.5% of the samples were assessed to be compliant.^Footnote 42 Note concerns were expressed regarding the number of samples taken.

Under the National Chemical Residue Monitoring Program, the sampling and testing of fresh fruits and vegetables for chemical residues is to ensure compliance with Canadian standards or maximum residue limits (MRL). Fresh produce samples are assessed for pesticide residues as well as heavy metals. In Canada, these pesticides are subject to a pre-market assessment by the Pesticide Management Regulatory Agency (PMRA) that establishes the safety of the food that can contain pesticide residues. As the health effect of these chemicals via the oral route is predominantly chronic in nature, the exposure scenario is generally long term. Exceeding a MRL is still a violation of Canadian regulatory requirements that triggers Program and Operations follow up; however, it is not generally regarded as an immediate health risk therefore the priority to follow up may be relatively lower compared to other more acute issues. Continual exposure at levels above the legal limit could be a concern that may lead to further regulatory actions against the responsible party. The exception is highly toxic chemicals prompting acute reactions. As Canada is a net importer of FFV, it is important to also monitor imported fresh fruits and vegetables for residues for which there is no Canadian registration or use (i.e. there may not be a pre-market assessment carried out by Health Canada). During the October 1, 2010 to March 31, 2012 time period 34,058 tests were performed on 9,090 samples for FFV as part of the National Chemical Residue Monitoring Program. Overall compliance on a per test basis was 99.17% for domestic and 98.05 for imported FFV^Footnote 43.

Plans for chemical residue testing of FFV are more extensive than sampling for microbial hazards. Shipments are selected randomly, collected throughout the year and samples allocated are proportional to the importance of the commodity in the diet, taking into consideration the origin of the food and volume of imports. As such, the sample numbers are a statistically representative sample of a commodity, but not of the country of origin. Samples are picked up by CFIA staff at major distribution and importing points and analyzed at CFIA and/or CFIA contracted laboratories. All directed (follow up) samples are analyzed by the CFIA laboratories.

Sampling plans for FFV to verify their compliance with the regulatory requirements for irradiation were discontinued in 2011 as this monitoring sampling activity had been identified as a low risk and low priority.

Food Safety Action Plan: Targeted studies

The Food Safety Action Plan (FSAP) aims to modernize and strengthen Canada's food safety system in order to better protect Canadians from unsafe food and ultimately reduce the burden of foodborne illness. The funds allocated under the FSAP allowed for the implementation of an enhanced surveillance program with the objective of collecting information on the priority hazards in various food products by conducting targeted surveys over a five-year period from 2008/09 to 2012/13. These food products include a large number of samples of fresh produce under the targeted surveys.

In the area of microbiological hazards in food, the targeted surveys focus on particular foodborne pathogens of concern in specific fresh produce items (i.e. pathogen/commodity combination). The sampling is conducted at retail or farmers' markets/gates where the sampler purchases a targeted commodity available for sale to the consumer. The types of produce targeted are those that have been linked to foodborne illness outbreaks. A variety of commodities and pathogens were targeted over the first 3 years of the studies.

For chemical hazards, testing under the FSAP initiative was focused on testing domestic products that are traded intra-provincially and interprovincially for over 400 pesticide residues, in order to establish a baseline to compare to imported product which is interprovincially traded. Additional surveys for FFV included testing for allergens and chemical hazards other than pesticides.

In 2007, the FFV Program began its evolution (FFV Modernization) under the FSAP and Listeria policy implementation initiatives to achieve active prevention and targeted oversight objectives and to improve rapid response capabilities:

• Food safety expertise/capacity built in FFV Program
• Pilot project for FFV establishment inspections
• Operational resources for new inspection activity (11 FTE)
  • Introduction of establishment inspections for 2013-2014
• Training modules developed and delivered
  • Introduction to Fresh Produce Safety
• Listeria Policy Implementation (Phases I,II, III)
  • Enhanced offshore efforts - trends in globalization of the fresh produce supply in developed, developing and underdeveloped countries
• Multi component strategy to be developed
  • Reactive: Bilateral engagement on specific issues (e.g. US, Mexico, Guatemala) in response to a particular issue, as needed and appropriate
  • Proactive: Capacity building and technical information sharing to proactively improve food safety (i.e. Global Food Safety Partnership – World Bank, Codex, etc.) and influence more widely on global fresh produce safety activities (upstream control measures)
• Support collaboration and relationship building with other federal and provincial/territorial governments (FPT)

Program Management has noted in its own profile documents that a greater emphasis on food safety activities has been deemed paramount in current initiatives to achieve preventive controls for fresh produce safety. Further, there is a recognized need to develop oversight programs to address the control of food safety risks along the fresh fruit and vegetable continuum (packers, repackers, minimal processors, etc.).

The following strategies / approaches to proactively prevent food safety risks are integral to the design of the FFV Food Safety Program:

• preventive measures and monitoring and oversight activities are essential
• safety verification at the appropriate stage in a product's life cycle
• new inspection activities (establishments, processes, handling) should be risk-based to ensure appropriate resources are targeted to the high-risk commodities establishments, and/or practices
• program re-engineering should complement changes to fresh produce safety activity globally - Canadian producers will need to meet the requirements of other countries, in particular the USA as the largest export market

In support of this effort, the Science Branch (CFIA) prepared a risk analysis document for the purpose of reviewing and synthesizing the evidence on hazards from fresh produce and how they occur, along with an examination of the current regulatory system, monitoring and oversight by CFIA to determine if further risk prevention and reduction measures are necessary.

Report summary:

• many sources of contamination (microbial pathogens) in the pre-harvest production, harvest and post-harvest/processing stages;
• prevention of contamination from primary production to the point of consumption is required to prevent illnesses/outbreaks;
• allergens pose a risk when not labelled;
• other high risk-ranked commodity/pathogen combinations have kill step, produce doesn't;
• complex/complicated distribution – challenge to pin point source of contamination;
• reports on causes of outbreaks underscores the need to prevent contamination at the farm, processing (packing) and retail level to best protect consumers (various points along the continuum); and,
• produce is generally safe in its own right, but potential for cross-contamination based on agricultural and handling practices can introduce hazards such as fecal contaminants, becoming a source of human pathogens – applies to domestic and imported produce.

More recently, the FFV Modernization initiative has been refined to maintain alignment with the CFIA Change Agenda, including Regulatory Modernization and Inspection Modernization

FFV Modernization Initiative: The following points highlight key elements of the Fresh Produce Safety component of the FFV Modernization Initiative:

• There is a large body of evidence, both from the literature and from domestic surveillance, monitoring and studies, indicating that fresh fruits and vegetables are increasingly becoming a food safety concern.
• Considerable research and studies have been and will continue to be conducted for the identification of potential sources and entry points of contamination of fresh produce and specific factors associated with the handling of fresh produce which allow contaminants to persist and proliferate. In collaboration with our food safety partners, the CFIA will need to develop mechanisms to assess, oversee and monitor the practices of the fresh produce industry using this information.
• Public health risks associated with fresh fruit and vegetables (FFV) are best minimized and managed through targeted oversight and active prevention along the entire supply chain.
• The fresh produce continuum is a complex/complicated distribution chain, making it a challenge to pin point sources of contamination. Many sources of contamination (microbial pathogens) in the pre-harvest production, harvest and post-harvest/processing stages are possible.
• The FSAP project focused on the non-federally registered sector (NFRS), and in particular, on the import of these defined commodities into Canada. While fresh produce was identified as one of the high priority areas due to lack of federal registration of facilities, the licensing regime and requirement for preventive food safety systems have not up to now been applicable to the FFV Program which operates under additional legislative and regulatory requirements than the NFRS.
• The FFV Program began to evolve under the FSAP umbrella with limited funding to build food safety expertise and capacity in the Program, develop training modules for inspection staff and information sessions for industry. Program re-engineering is being planned to address food safety requirements for the domestic production and distribution of produce in Canada and for export with the introduction of facility inspection and oversight activities where gaps were identified. Currently the system in place appears to be fragmented and reactive rather than preventive.
• A lesson learned under the auspices of the FSAP project is that a demonstrable and effective domestic food safety system (appropriate government oversight) should secure access to foreign markets, respond appropriately to foreign audit findings and appropriately set the stage for import controls or requirements (as per Sanitary Phytosanitary agreement).
• The trend in globalization of the fresh produce supply from developed, developing and underdeveloped countries demands that the FFVP efforts also include the influencing of food safety strategies to foreign suppliers as upstream control measures.
• Investments in fresh produce safety will also serve to address the shared jurisdiction to provide seamless oversight along the supply chain – build synergies with federal, provincial and territorial partners (authorities at farm level for production control oversight); thereby demonstrating to our foreign partners that the Canadian food safety system for fresh produce is effective.
• Linkages amongst Health Canada (HC), Pesticide Management Regulatory Agency (PMRA), Agriculture and Agrifood Canada (AAFC), CFIA and provincial and territorial (P/T) government food safety authorities is intended to support a Canada-wide framework that is appropriately represented as a collective competent authority (CA) in the global food forum.
• The Food Investigation and Outbreak Response Protocol (FIORP) describe the framework for the investigation of and response to multi-jurisdictional food-borne illness outbreaks in Canada.
• An emphasis of the overall regulatory modernization initiative is safety. "By doing a structured and comprehensive review, the CFIA can put into place a modernized regulatory framework that will ensure that safety is the overarching consideration"^Footnote 44

If food safety measures to prevent the introduction of known or reasonably foreseeable biological hazards is the objective, then consideration must be given to the most common sources of contamination of fresh produce: on-farm production or growing practices for fruits and vegetables, including tree nuts and extending into on-farm and post-farm activities of harvesting, packing, holding, handling, preparing, processing and distribution of fresh produce commodities of high risk to bring improved food safety controls further upstream from the retail marketplace.

The industry's CanadaGAP program supports this need. CanadaGAP, the food safety program for produce, introduces systems of recording, documenting and verifying that the production practices are being followed and are effective in controlling potential contamination.^Footnote 45

In 2000, the Canadian Horticultural Council (CHC), the national industry association for fruit and vegetable producers, in response to market demands for food safety assurances, began developing the On-Farm Food Safety (OFFS) Guidelines. Between 2002 and 2008, the CHC developed eight commodity specific HACCP-based food safety modules. By 2008, these eight modules were consolidated into six. The CHC's OFFS Guidelines became known as CanadaGAP, a HACCP-based food safety system for suppliers of Canadian fresh fruit and vegetables. The CanadaGAP certification program was launched in 2008. The program involves having a third-party auditor from a certification body visit the farm, review the food safety manual(s) and related records, interview the operator and staff, and assess the company's conformance to the CanadaGAP Audit Checklist. ^Footnote 46

There is potential, within the shared federal-provincial jurisdiction, to provide seamless oversight along the fresh produce continuum, including potential setting of standards by Health Canada for elements such as the agricultural water, biological soil amendments (composted manure), health and hygiene of handlers, proximity of animals and farm equipment, buildings and tools, which would require enforcement by provincial and territorial governments. New federal regulations under SFCA will propose the requirement for licensing and preventive controls for farms packaging/selling produce interprovincially or internationally that are intended for human consumption. In addition, preventive control plans for packing facilities will need to address "inputs" as part of these plans, which will further cover produce in its post-harvest raw state.

The Inspection and Regulatory Modernization initiatives will serve to close gaps in CFIA oversight of the fresh produce industry. Of benefit would be the building of synergies with federal, provincial and territorial partners (authorities at farm level for production control oversight); to demonstrate that the Canadian food safety system for fresh produce is effective.

Proposed regulatory modernization design components to address deficiencies in the current FFV program design

• Mandatory domestic establishment identification and registration or licensing
  • Proposed regulatory requirement for all food (single Food Regulation), serving to identify all parties in fresh produce continuum
• Requirement for risk-based preventive food safety control systems
  • Strengthening Canada's food safety system by providing CFIA with a legislative mandate to require comprehensive, science-based preventive controls across the food supply rather than relying primarily on reacting to problems after they occur
  • Food safety system implementation as condition of license or permission to operate
  • Serves to clearly communicate food safety responsibilities and accountabilities to the fresh produce sector here and abroad
• Inspection and oversight of FFV establishments (Risk-based)
  • Permits building on private and/or public internationally recognized food safety system criteria (Codex, ISO, etc.) for incorporation into the re-engineered model to complement facility inspection of FFV establishments for the purpose of monitoring safety
  • Regulatory/policy controls would ensure impartiality, competency, conformance to audit standards, correction of deficiencies and reporting of food safety issues
• Collaboration with FPT for oversight on the fresh produce continuum
  • Farm level - key to addressing fresh produce safety; requires a uniquely "Canadian" approach to addressing jurisdictional issues – Canada's National Dairy Code is a potential model

Status in other countries with respect to fresh produce safety

A review of international trends suggests that the situation in Canada mirrors the trend seen in other industrialized countries with an increased number of food-borne illness outbreaks linked to fresh produce in the last 15 years. According to the FFVP, fresh fruits and vegetables represent the fastest growing microbiological safety risk among food commodities. Over the last several years, the detection of outbreaks of food-borne illness associated with fresh fruits and vegetables has increased with over 60% being associated with fresh produce and resulting in numerous illnesses. These attribution estimates are important because they emphasize the need for regulatory agencies and industry (including farms) to target prevention efforts that will improve the safety of fresh produce by preventing the introduction of known or reasonably foreseeable hazards.

International partners are increasingly requiring demonstration of comprehensive food safety systems that focus on the supply chain from farm to fork. For example, the United States are now imposing requirements on harvesters and growers. This is extended to foreign farms, thereby requiring that imported food commodities comply with the same requirement as domestically grown products. Therefore, it is expected that persons exporting to the United States will request the CFIA to issue a confirmation that their products meet the requirements of the United States, including a preventive food safety plan.

Change Management

Equally important to the modernization of the FFV Program is the communication with and growing awareness of the purpose and intent of the initiative with inspection staff, intermediaries and the regulated parties (industry). An early review of the situation suggests that the shift from quality-based inspection parameters to that of food safety is a culture change for all and will require knowledge, support, engagement and reinforcement to realize the desired outcomes and to sustain them.

Fresh Fruit and Vegetable Quality

Grades for Fresh Fruits and Vegetables

Grade standards for fresh fruit and vegetables were developed in consultation with industry for fruits and vegetables that were of economic significance to Canadian producers. Standards were first introduced around the 1890's for apples but the majority of grades that are currently in regulations were established around the 1930's.

There are 30 produce commodities for which grade standards have been prescribed under Section 3 (1) of the FFVR, as listed in the table below. Grades are used between producers, packers, wholesalers and retailers when trading fresh fruits and vegetables in order to: establish prices, facilitate trade internationally, and to protect consumers from economic fraud. In Canada, regulations require that all fresh fruit and vegetables with a grade standard be graded and labelled prior to being sold interprovincially or imported into Canada. All other produce is marketed without having established grades.

Grade Standards

Fruit

• Apples
• Apricots
• Blueberries (exception – if for processing)
• Cantaloupes
• Cherries
• Crabapples
• Cranberries
• Grapes
• Peaches
• Pears
• Plums and Prunes
• Field Rhubarb
• Strawberries (voluntary grade declaration)

Vegetables (miniature vegetables are excluded; however, there is a grade for miniature cucumbers)

• Asparagus (exception – white asparagus)
• Beets (exception – beets with tops)
• Brussel sprouts
• Cabbages
• Carrots (exception – carrots with tops)
• Cauliflower
• Celery
• Sweet Corn
• Field Cucumbers (exception – pickling cucumbers)
• Greenhouse Cucumbers
• Head Lettuce – Iceberg type
• Onions (exception – onions with tops)
• Parsnips
• Potatoes (excluded if certified for seed)
• Rutabagas
• Field Tomatoes (exception – cherry tomatoes)
• Greenhouse Tomatoes

Quality Inspection Activities

The FFV Regulations prescribe that for the following commodities traded interprovincially, each load must be inspected and certified to meet an established grade:

Apples from British Columbia, Ontario, Quebec, New Brunswick and Nova Scotia;

Potatoes from Ontario, Quebec, New Brunswick, Nova Scotia and Prince Edward Island; and

Blueberries packaged in containers of 6 litres or less from Nova Scotia, New Brunswick or Prince Edward Island.

Each load of the above commodities must be shipped with a valid inspection certificate once inspected by the CFIA during what is termed a shipping point inspection. Inspection certificates are valid for not more than three days (including Saturdays and holidays). Exceptions for mandatory shipping point inspections are made for establishments that are registered with the CFIA in the Registered Produce Warehouse Program. This program will be described in detail below.

There is a provision in the FFVR that allows an inspector to sign a release permit in the event that a hands-on quality inspection cannot be performed for shipments of the above commodities moving interprovincially. These are commonly referred to as Y-releases. Y-releases were intended to allow trade to continue for highly perishable commodities on those occasions when inspection resources were not available to perform mandated inspections. Each year, approximately 200 shipping point inspections and 1,000 release permits are issued by CFIA Operations staff. Below is a table that displays the number of shipping point inspections vs. the number of Y releases for apples and potatoes by area, resourcing information is also included.

Number of Shipping Point Inspections for Apples and Potatoes^Footnote 47

The FFV Regulations also require that each load of apples, onions and potatoes imported into Canada be inspected and certified to meet an established grade. These shipments are inspected by CFIA inspectors at point of entry or at another point (destination) as designated by the inspector and are referred to as customs clearance inspections. These products are not able to enter into the Canadian market without a signed inspection certificate. There are, on average, approximately 800 requests made to the CFIA each year for customs clearance inspections.

To facilitate trade between Canada and the United States, an agreement is in place whereby the CFIA accepts quality certificates for apples, onions and potatoes that have been inspected to Canadian grade requirements by the United States Department of Agriculture (USDA). A reciprocal agreement is in place whereby the CFIA may inspect and certify shipments of Canadian potatoes, onions and field tomatoes that are to be exported to the US in order to meet their quality standards. On average, 19 000 export certificates are issued each year by the CFIA for shipments of potatoes, onions and field tomatoes being exported to the US.

Registered Produce Warehouses – Voluntary Registration

The Registered Produce Warehouse (RPW) Program was introduced to allow products to move between provinces without the need for mandatory quality inspections on each outbound load. The desired outcomes of the Program were to allow more flexibility to the Industry for shipping (no longer need to request an inspection and wait for an available inspector prior to shipping their load) and to reduce the resource requirements for the CFIA to perform high numbers of shipping point inspections. Registration in this quality-based program is voluntary and there are approximately 22 and 70 registered establishments for apples and potato establishments respectively.

The regulatory basis for Registered Establishments is contained in Part X of the Fresh Fruit and Vegetable Regulations (Regulations). Establishments which prepare commodities for which the Regulations establish grades may become Registered Establishments, however, in practice, only those establishments trading potatoes or apples have joined the Program due to the mandatory inspection requirements laid out in the Regulations. Under the RPW Program, registered establishments are exempt from the mandatory shipping point inspections as long as they are able to demonstrate the continued ability to pack product to quality standards. With this program, the packers are responsible for the quality of their production. The inspector's role is to evaluate the consistency of a RPW to pack a quality product and to assist packers to interpret the regulatory grade standards.

In order to verify industry compliance, the CFIA routinely inspects produce packed within each RPW, and the frequency of inspection is varied according the level of compliance demonstrated. Product Inspection is based on a statistically valid method of sampling, where the results of several such inspections over a period of time identify whether or not a particular RPW continuously packs its produce according to grade standards. This is referred to as "Frequency of Inspection Level (F.O.I.L.)".

The F.O.I.L. concept is based on the premise that an establishment's compliance rate reflects its ability to meet Regulatory requirements. As such, the frequency of monitoring visits should, consequently, be based on an establishment's compliance rate. If the inspector does not find any non-compliant product over a certain number of visits, the frequency of inspection level can be reduced. Conversely, if the inspector finds one or more lots of produce repeatedly failing the grade, the frequency of inspection level can be increased. At present, the reduced inspection frequency is set at 1 visit every 4 weeks (RPW is in compliance with the grade standards), although operationally this target has not been maintained, thereby making the tightened frequency of one visit per week for non-compliance non-feasible. In essence, the FOIL concept is no longer applicable given that Operations Branch does not have enough resources to consistently meet the even the most lenient of FOILs, let alone the tightened frequency required by the Program should a company demonstrate the inability to grade their product. Below is a table displaying the average number inspections for RPWs per year.

Average Number of Inspections per Establishment per Year^Footnote 48

Canadian Partners in Quality Program (C-PIQ)

The Canadian Partners in Quality Program (C-PIQ) is a voluntary integrated inspection system, which provides an alternative to the traditional hands-on quality inspection by CFIA for tablestock potatoes exported to the United States. As a component of the Program, Establishments are able to complete their own export certificates for loads of potatoes being shipped to the United States. Companies wishing to operate under C-PIQ must meet a number of requirements in order to be registered by CFIA and must maintain a specific performance standard in order to remain in the C-PIQ Program. There are currently 15 establishments participating in the C-PIQ Program.

C-PIQ transfers the accountability for quality control of their products to Industry, while CFIA plays an oversight/auditing role of the HACCP-based system. Participants are responsible for the development of their own quality assurance system and production controls in accordance with C-PIQ requirements, which are implemented to ensure that quality (grade, size, maturity), and applicable packaging and labelling requirements are met.

CFIA inspection staff performs activities prior to the acceptance of an establishment into the program, and following the signing of the participation agreement. Activities include the review of the company QA manual, up to three (3) pre-acceptance validation audits, up to 12 validation audits as well as routine audits. Routine audits by CFIA Operations staff are scheduled according to either Phase I (one audit every 28 production days) or Phase II (one audit in every 50 production days). Whether an establishment is audited according to Phase I or II depends on numerous factors, although ultimately reflects the level of confidence that the CFIA has in their ability to consistently package and ship quality potatoes to the United States.

The program is not supported through regulations and as such, the fees are not established within the CFIA fees notice. Instead, the C-PIQ program is operated through legally binding participation agreements between the potato establishment and the CFIA which outline the responsibilities for each party as well as fee structures.

Ministerial Exemptions

Fruits and vegetables for which there are grade names established in the Regulations are subject to packaging and labelling rules when marketed in interprovincial and import trade. Pursuant to subsection 5(2) of the Regulations, produce for which a grade is established may not be marketed in import or interprovincial trade in containers exceeding 50 kg, or 200 kg for apples, apricots, peaches and pears (200 kg in accordance with Test Markets). As such, produce moving in bulk containers (bins, sacks, etc.) in excess of the prescribed weight or without proper markings, whether graded to meet the minimum quality standards or not, do not comply with the regulatory requirements set out in the Regulations for packaging and/or labelling.

Under certain circumstances, regulatory requirements may be exempted and movement of non-compliant product from another province or country may be permitted. Pursuant to subsection 2.2(2) of the Regulations, the Minister or a delegate of the Minister may exempt a product from the minimum grade (quality), labelling and/or packaging requirements prescribed by the Regulations, where the Minister considers it necessary in order to prevent or alleviate a shortage of product in Canada. The number of loads of produce moving under a ME varies from year to year, the table below shows loads of produce moving under ministerial exemption by year.

Loads of Produce Moving Under Ministerial Exemptions by Year^Footnote 49

The labelling, packaging and quality requirements may be waived when a Ministerial Exemption (ME) is requested for processing purposes. Only the labelling and packaging requirements may be waived when a ME is requested for repacking purposes. The quality requirement will be dictated by import or interprovincial regulatory requirements and in some cases, the applicant's specific requirements. It should be noted that in all cases, health and safety requirements will not be waived.

Such discretionary authority provides the Minister with the ability to assist packers and/or processors to obtain the supplies necessary to continue operating and to meet demand for products by sourcing available supplies from another province or another country. In late 2012, the delegation for signing MEs for imported product was transferred from Programs to Operations. As MEs are issued on a per-load basis and original documents must accompany each load, there is a requirement for control visits by inspectors to ensure that imported product is compliant and matches the product specified on the ME.

Control of MEs for apples, onions and potatoes coming from the United States is supported by the United States Department of Agriculture – Agricultural Marketing Service (USDA-AMS). Once MEs are signed by the appropriate Operations delegate for the correct number of loads coming to Canada from the US, the USDA-AMS issues the MEs on a load-by-load basis and monitors the number of loads and product compliance with that listed on the ME. While witnessing the loading of the product for transport, US inspectors also ensure that all plant health requirements such as vehicle cleanliness and declaration of Country of Origin are met.

Other commodities imported under an ME for imports have the ME with original signature as a documentation requirement to clear the border.

Test Markets

The Fresh Fruit and Vegetable Regulations contain provisions to allow for the granting of exemptions in order to test market new product standards for the grading, packaging and labelling of these products prior to a regulatory amendment. The Regulations allow the Minister (or delegate) to authorize a dealer to test market a product where any produce, container or label does not comply with the Regulations for a period of up to 24 months. Applicants are required to provide their request in writing and following industry consultation, if supported, the test market can be approved for a period of up to 24 months, and applies to all industry in the requested geographical region, not just the applicant.

The intent of the test marketing provisions is to provide the opportunity to evaluate new marketing ideas on the marketplace prior to making permanent regulatory amendments. Test marketing provisions facilitate the introduction of, among other things, modifications to grade requirements, package sizes, etc. into the marketplace and provide an opportunity to assess consumer acceptance and market impact. If the test market is successful, the Regulations should be amended. The reality of the test market situation for fresh fruits and vegetables is that the majority of test markets have been in place for a much longer timeframe than the regulated 24 months, due to low priority in the current regulatory amendment process.

Canadian Import Requirements for Fresh Fruits and Vegetable

Under the Licensing and Arbitration Regulations (LAR), Canadian importers are required to be licensed with the Canadian Food Inspection Agency (CFIA) and/or be a member of the Dispute Resolution Corporation (DRC). Importers that are retailers selling directly to consumers with sales under $230,000 per year are exempt from this requirement. A CFIA license and/or membership with the DRC provide a mechanism for dispute resolution of any quality or payment issues in produce transactions. There are approximately 115 licence holders and 800 DRC members, and under the Regulatory Cooperation Council (RCC) initiative, the feasibility of the future repeal of the LAR and an addition to the FFVR mandating the requirement for all dealers to be members of the DRC is being explored.

The produce license applies equally to interprovincial dealers.

Canadian Export Requirements for Fresh Fruits and Vegetables

There are no export requirements for produce being shipped from Canada within the Fresh Fruit and Vegetable Regulations other than for health and safety.

Under Part VII, Section 40 of the FFV Regulations, applicants may request inspections for the purpose of meeting U.S. requirements for potatoes, onions and field tomatoes going to the U.S. or Puerto Rico. Sections 17 (a), (b) and (c) of the CAP Act state that no person shall market, possess for the purpose of marketing, or possess an agricultural product that has been marketed in contravention of the CAP Act and Fresh Fruit and Vegetable Regulations. In light of this, any requests for export certification to the US are handled as a priority and are considered Export Compulsory inspections.

There are no requirements for packaging and labelling for export in the Fresh Fruit and Vegetable Regulations, therefore, product cannot be refused certification for export if the packaging and labelling do not meet Canadian requirements. The importing country may have legislation that would require proper certification, packaging, labelling and grading to the importing country's standards. It is the shipper's responsibility to ensure that they are meeting the packaging and labelling requirements of the importing country. In the US, labelling requirements may vary by state.

Destination Inspection Service

The CFIA provides destination inspections for buyers of shipped produce to provide an impartial inspection report for the resolution of buyer/seller disputes regarding the quality of fresh fruit and vegetables. These services are cost recovered by DIS, and services offered include inspections for: grade, permanent and condition defects, destruction or dump certificates (witnessing of product disposal), gross weight, size or count as well as inspections for fresh-cut and/or bagged salads.

Although (DIS) is not part of the Fresh Fruit and Vegetable Program, the activities provided by DIS are strongly linked to the Program. Many field inspectors perform both FFV and DIS inspection activities and a large proportion of the reference materials used by inspection staff are shared.

During 2012-13 inspection demand for DIS inspections decreased by approximately 37% relative to 2011-12 (8,737 inspections vs. 13,790 in 2011-12). The factors possibly affecting demand for DIS services are^Footnote 50 :

• DIS pricing; user-fees were adjusted in April 2012 to recover 75% of program costs,
• Maturing relationships between buyers and sellers of fresh produce,
• Industry investment in in-house quality control staff and protocols,
• Private service providers delivering similar services,
• Larger lots and lower commodity prices,
• Less opportunity for sales of lower quality produce and rising transportation costs.

Status in Other Provinces/Countries with Respect to Fresh Fruit and Vegetable Quality Standards

In Ontario, with the exception of potatoes, there are no longer any grade standards for produce. Producers may still use the Federal standards, but if used on a product marketed interprovincially, the producer must ensure that the product meets the standard as set out in the Fresh Fruit and Vegetable Regulations. Potato grade standards and package size requirements will be revoked in Ontario as of July 1, 2016.

In the United States, the United States Department of Agriculture (USDA), Agricultural Marketing Service (AMS) is responsible for maintaining fresh fruit and vegetable quality grade standards (over 300 grade standards). The use of a grade standard in the US is voluntary, but where a grade is stated, the product must meet the standard. USDA-AMS also provides on a cost-recovery basis various inspection activities that industry can use to help promote and communicate quality such as: grading, certification, auditing, inspection, and laboratory analysis.

The United Nations Economic Commission for Europe (UNECE) develops global agricultural quality standards in order to facilitate international trade. The standards are published in three languages (English, French and Russian) and are used internationally by Governments, producers, traders, importers, exporters and international organizations. Any interested parties are welcome to participate in the development of the standards.

Challenges Faced by Fresh Fruit and Vegetable Quality Program

There have been many discussions, both internally and with stakeholders, as to whether the grade standards still meet the objectives for which they were originally developed. Some of the comments the FFV Program have received from industry have been captured in point form below.

• It has been recognized that the regulatory changes cannot keep pace with market innovation and industry needs.
• The CFIA has demonstrated there are challenges associated with resource allocations for quality inspection activities where there are no direct impacts on human health.
• Buyer standards are higher than those established within the Regulations.
• Most interprovincial certification of apples, potatoes and blueberries is being replaced with release permits (Y-releases), which begs the question as to whether there is a need for interprovincial inspection of these commodities or to what commercial or marketing purpose do these compulsory inspections serve?
• The majority of customs clearance inspections indicate compliance with requirements and importers do not see a need for these inspections which are mandated within the FFVR.
• Processors have argued that mandatory inspection requirements impede trade since they must have produce that will be transformed meet grade and be inspected.
• Due to resource limitations, there is no inspection for compliance with grade standards for commodities other than apples, onions and potatoes.
• There is a challenge within CFIA to maintain inspection/commodity expertise for those commodities other than apples, onions and potatoes.
• Retention of inspection staff in the FFV quality program poses additional challenges due to the lower level of classification of these inspectors, based on the lack of food-safety activities performed by FFV field staff. This results in a higher level of staff turnover and these positions are often seen as entry level positions, amplifying the loss of commodity expertise.

Annex B - The FFVP Results Logic and Systems Performance Framework

Available upon request

Annex C - Evaluation Approach

The Evaluation approach description in this section is a summary of the approach developed and approved in the evaluation plan for this study.

Realistic Contribution Analysis

A theory based approach (realistic contribution analysis)^Footnote 51 was used for this evaluation to support the answering of the five core Government of Canada issues while accommodating the pending transformation of the Program and the information needs of management going forward.

The strength of the realistic contribution analysis approach was the systematic, structured and disciplined approach that led to a greater understanding of the results logic related to the FFVP and different contextual factors that impact its delivery (These contextual factors were summarized in Table 3 of the main report).

The evaluation approach accounted for the following challenges:

• The complex nature of the FFVP regulatory and other activities (timelines, attribution issues, multiple actors, evolving production, processing, manufacturing and distribution and product changes, etc.);
• The links between policy and regulations and operations in the regulation of this area;
• The somewhat complicated delivery mechanisms (e.g., multiple agencies, multiple groups, multiple jurisdictions, intermediaries and regulated parties); and
• Limitations in legislative and regulatory authority.

Needs Review
The needs review involved a review of domestic and / or international literature as well as CFIA documentation (secondary data sources) and from interviews (primary data sources) to determine why the FFVP activities have been tried in terms of the needs they were intended to address.

Instruments Review
The review examined both the overall 'instrument' or 'mechanism' theories of the FFVP activities (Regulations, policies, codes of practice, control systems such as HACCP, Quality Management Programs, Food Safety Action Plan, Inspection Modernization, Canadian Partners in Quality (CPIQ) and memoranda and agreements etc.). It looked at the sub-component theories of establishment identification, registration and licensing, detection, response, enforcement, assessment and monitoring (or other process elements) along with theories of deterrence and compliance promotion. The goal of this work was to characterize the strategies and tactics used by the FFVP and to establish some basic parameters and criteria for assessment. Note that this work drew on TBS guidance for realist evaluation and contribution analysis as well as work conducted for CFIA Evaluation. (Note the regulatory instruments considerations in Table D-1 in Annex D.)

Literature, Comparative Analysis and Synthesis
The evidence synthesis involved the searching, appraising, extracting and synthesizing of evidence addressing the FFVP evaluation issues and questions. The main focus of this synthesis of literature, special studies, published and non-published materials was to establish what works for whom, how and under what circumstances for various mechanisms and instruments such as those currently used and proposed / underway (under modernization) of the FFVP (e.g. What works for whom in what conditions and why, when moving to outcome-based regulatory approaches?) In this case, particular attention was be paid to international practices in related areas.

International Comparative Analysis
In order to help to understand how current initiatives like Inspection Modernization (IM) and the Food Safety Action Plan (FSAP) affected the FFVP, a review of international practices was conducted. The review examined the US Food and Drug Administration (FDA) work and other international initiatives from the European Union and Australia / New Zealand focussing on mechanisms such as comprehensive licensing, proactive inspection, outcome based codes and standards and similar risk-oriented initiatives in terms of what has worked for whom in what conditions and why. Limited direct consultations complemented third party information as methods and sources for the analysis. This analysis and synthesis provided strong support to future decision-making around the implementation of IM and FSAP for FFV and potentially across other CFIA areas. (See Annex D for two major summaries of this work. Further details are contained in separate reports prepared for this study.)

File Reviews Evidence Analysis
Data on the use of FFVP activities across areas were assessed for consistency and accuracy. The review of documents regarding FFVP activities by area served as enhanced profile information and looked at the extent and magnitude of operations and reach, as well as possibly to pick up patterns.

Interviews to Validate, Extend and Address Issues
Over 75 interviews and consultations with key informants (including all key levels of related CFIA stakeholders as well as other stakeholders such as related government departments (e.g. AAFC, HC, PS-CBSA, P/Ts and others), including regulated party representatives were conducted to address the needs, results theories and data synthesis assembled from the previous methods. Note that interview 'burden' was controlled by ensuring that appropriate 'homework' and background reviews was conducted before soliciting persons for interview. Select interviews were conducted early in the process to assist with identifying hypothesis, theories and key literature. Several respondents were consulted multiple times, as new information emerged from various sources. In this way interviews enabled the team to best distill lessons on best practices and principles, as well as to address key relevance and performance issues. Interviewees included a representation of most informed key stakeholders.

Case Studies
Case studies of particular single events or themed 'stream' areas were conducted so as to illustrate and describe certain phenomena and / or to test instrument or initiative elements. The cases included a review of the pilot trial of new inspection practice for leafy green vegetables (Evaluation of the FFV Establishment Inspection Pilot Project for Packers and Re-packers of Fresh Leafy Vegetables, Green Onions and/or Herbs, March 2012), a review of specified FSAP chemical residue testing of leafy greens and the comparative analysis of outcome based standards and risk management as applied internationally. The last two of these cases are contained in Annex D.

Annex D - Risk Management and Outcomes-Based Standards: A Review of International Experience Related to Food Safety

Available upon request

Annex E – Interview Guides

Interview Questions for Fresh Produce Safety Section

CFIA Regulatory System

Q 1 Please describe your role and responsibilities as a program specialist with the Fresh Fruit and Vegetable Program, Fresh Produce Safety section. (i.e. How is this linked to the Multiple Commodity Activities Program? Cognos?)

Q 2 How would you describe the links between the Fresh Produce Safety section, regional office in Mississauga with Head office of CFIA Governing Bodies, Key Associations and Canadians?

Q 3 When was the Fresh Produce Safety section established and what are your observations about the creation of the Fresh Produce Safety section pre-post modernization activities?

Q 4 How would you describe CFIA's role in Food Safety vs. Food Quality (i.e. Should the CFIA be more active in areas of FFV safety? If so, what activities should it take on, and what would be the effect on its FFV quality work and traditional emphasis on trade?)

Q 5 How would you describe CFIA's regulatory system, pre-post modernization plans?

Program Delivery Activities

• Risk-oriented and proactive program inspection system
• Efficient, consistent and responsive operational, scientific & internal services

Familiar Pre Modernization

Q 6 According to the other informants and documents CFIA food safety activities have been primarily reactive in nature. In your opinion, why is this and what worked /didn't work with this approach?

Q 7 What are your thoughts about the processes (i.e. random sampling) and systems (National Microbiological Monitoring Program and National Chemical Residue Monitoring Program) that are currently in place for monitoring for microbial and chemical hazards?

Q 8 What is working /not working with respect to the activities, such as sampling and testing being applied consistently and cumulatively (e.g. taking into account past results)?

Q 9 What worked /did not work when conducting environmental scans/risk analysis (i.e. data collection, analytical frameworks, usability etc.)?

Q 10 Does the Fresh Fruits and Vegetables Program do what it says it will do, such as with its sampling and testing or is the work on the ground different from what is outlined in manuals and other documents?

Q 11 Do the activities of the FFV Program change under certain conditions or with certain parties? Are the differences in Program activities appropriate for the changed conditions?

Post Modernization

Q 12 Following the 2009 Weatherill report on the 2008 Listeriosis Outbreak the Agency moved forward with recommendations to take a risk oriented and preventive approach to food safety.

In your opinion, what factors need to be taken into consideration to develop an effective risk orientated and preventive food safety environment?

Q 13 What are the implications of the transition towards risk based and preventive food safety systems on the Agency, Stakeholders and Consumers?

Q 14 What are your thoughts about the processes (i.e. random sampling) and systems (National Microbiological Monitoring Program and National Chemical Residue Monitoring Program) that may need to be in place to support regulatory modernization and inspection plans?

Q 15 From a food safety perspective, are there any aspects of the proposed new licensing requirements that are cause for concern? (e.g. licensing of all regulated parties, on farm inspections, oversight and monitoring preventive control plans; communication and engagement)

Q 16 Do you think increased accountability for producers and importers for the safety of imported foods will increase food safety? Why or why not?

Q 17 Do you have any suggestions about the role that industry can and should play in future transitions away from food quality vs. safety, such as certification?

Q 18 In your opinion, what roles /challenges do equivalencies, system recognition and standards play with respect to food safety in the Agency's modernization plans?

Q 19 In your opinion, what factors and conditions are important for the organization to allow the regulatory and inspection modernization transition to achieve its goals?

Q 20 Can you provide a brief overview of the Multi-Commodity Activities Program (MCAP)? What are some of its strengths and weaknesses? Are any updates planned for MCAP moving forward?

Interview Questions FFVP Quality Section

Q 1. Please describe your role and responsibilities as a program specialist with the Fresh Fruit and Vegetable Program, Fresh Produce Quality section.

Q 2. (a) What are the main activities of the Quality Section of the FFVP?

Q 3. When was the safety and quality sections of the FFVP divided between two chief and quality moved to the Processed Products, Maple and Honey Program (PPMH)?

Q 4. (a) In your opinion what were the key reasons the Quality section of the FFVP was transferred to PPMH?
(b) What processes were used to determine the decision to transfer the Quality program to PPMH?
(c) How did these processes work and what might have been done differently?

Q 5. How is the transition to PPMH working in comparison to when the two programs operated under the same program with respect to the following:
(a) reporting
(b) linkage with Quality and Safety
(c) prioritization
(d) appropriate capacity and resourcing for quality section
(e) relevance of Quality with mandate of the PPMH
(f) reactions to the transition of Quality to PPMH (staff/inspectors)

Q 6. (a) How would you describe CFIA's role in Food Quality vs. Food Safety?
(b)Should the CFIA be more active in areas of FFV safety? If so, what activities should it take on, and what would be the effect on its FFV quality work?

Q 7. How would you describe CFIA's current regulatory system?
Regulatory Instruments (FFV Legislation and Act pertaining to Quality)

Q 8. (a) Can you please clarify which authorities are currently most relevant to the Quality and Grading of FFV?
(b) From a Quality perspective, what are your thoughts about the regulatory modernization plans to consolidate the Canada Agricultural Products Act (CAPA) and the Consumer Packaging and Labelling Regulations (CPLR) under the Safe Food for Canadians Act (SFCA) and the possible repeal of the Licensing and Arbitration Regulations (LAR)?
(c) What are your thoughts about proposed changes to packaging requirements (i.e. getting rid of container sizes)?

Q 9.(a) In your opinion, should regulatory requirements for Quality be commodity specific or apply to all foods? Why or why not?
It is our understanding that Quality includes Grades.
(b)Why is it necessary for the FFV regulations to distinguish between Grade and Quality standards?
(c)How and who decides what commodities require grading?

Q 10. In your opinion are the grades for fresh fruit and vegetables in keeping with Fresh Fruit and Vegetables that are of current economic significance to Canadian Producers? (i.e. is there need to revise grades?)

Q 11. From a Quality perspective, do you have any key concerns about the regulatory and inspection modernization plans that you would like to share with us? (i.e. the intent of Preventive Control Plans will include grade standards – will this provide the same level of protection as provided under the quality-grade requirements in regulations?

Program Delivery

• Risk-oriented and proactive program inspection system
• Efficient, consistent and responsive operational, scientific & internal services

Q 12. In your opinion what influence does market innovation and industry needs have on establishing Quality standards/Grading and the work of the Quality division?

Q 13. What are industry's current concerns regarding the FFV Quality Program and Quality and Grading of FFV?

Q 14. (a) What are your thoughts about how Quality inspections are working under the Registered Produce Warehouse Program (RPWP)?
(b)In your opinion is it feasible or not to encourage more establishments to use this service? why or why not?
(c)In your opinion should the Frequency of Inspection Level (F.O.I.L.) be continued? Why or why not?

Q 15. What are your thoughts on the use of Y-Releases in comparison with shipping point inspection certificates?

Q 16. What are your thoughts on the Canadian Partners in Quality Program (C-PIQ) which provides an alternative to the traditional hands-on quality inspection by CFIA for table stock potatoes exported to the United States? (i.e. value in expanding it to other FFV commodities?)

Q 17. What are your thoughts about Ministerial exemptions for Quality and Grading Requirements?

Q 18. (a) In your opinion, how is test marketing for Quality and Grading working under current processes?
(b) What elements of test marketing exemptions are favorable?
(c) What elements of the test marketing exemptions are unsatisfactory?

Q 19. What are your thoughts about regulating the quality and grading of exports?

Q 20. What are the challenges / benefits of the Custom Clearance Exemptions to meet grade requirements?

FFV External Stakeholder Interview Guide

1. What has been your involvement and relationship with the FFVP?
2. How do you interact with the FFVP? What activities do you engage in with the FFVP?
3. What needs does the FFVP serve?
4. As external stakeholders, what do you provide to the FFVP?
5. I would like to review a basic results framework with you. Can you comment on:
   1. The (desired) results statements. Do they represent your understanding? If not how would you change them?
   2. The extent to which the results are achieved, with whom and why?
   3. The key assumptions and factors which influence results (see the numbered 'buttons' and statements on the following page). To what extent do these assumptions hold true? How do they affect results? Are there other factors?
6. How familiar are you with CFIA regulatory modernization? (e.g. The principles of Inspection Modernization, the new Food Safety Act.) [Not at all, a little, somewhat, very]
   1. Have you noticed a difference in your interactions with FFVP Pre and Post Modernization initiatives? How would you classify these differences?
   2. What dimensions of regulatory modernization are working well / not well for your organization? Why or why not? What has been so far / will be the implications on your organization?
7. What is your understanding of the role CFIA plays regarding FFV safety? How does your organization contribute to FFV safety?
8. What assumptions and constraints does your organization face in aligning with federal government priorities and the strategic outcomes of the department?
9. How does the traditional orientation to quality and the subsequent transition to safety affect your organization's relationships and interactions with FFVP?
10. What current risks, emerging issues and trends associated with FFV safety has your organization noted?
11. Do you observe differences in understanding of how FFVP activities are supposed to work? What are those differences? Among whom?
12. Under what different conditions and contexts has your organization experienced FFVP initiatives working?
13. How has food safety risk management factored into your relationship, interactions and achieving results with FFVP?
14. Do you work with other stakeholders to achieve your organizational results? Do these align with those of FFVP? With other stakeholders such as CBSA, HC, AAFC? Other levels of government and other stakeholders in the food value chains? How does this improve effectiveness, cost-effectiveness, economy and/or efficiency?
15. In your view, are there more cost-effective ways to achieve the results we have discussed? What are these? (e.g. Devolution of quality assurance to an industry-led organization, monetary penalties, outcome based codes and standards, various inspection modernization initiatives.)
16. What would happen without FFVP activities?
17. How would you describe the costs to various stakeholders?

Annex F - Evaluation Matrix

Relevance

Performance

Annex G - Bibliography

Available upon request