Internal Audit Report
Audit of the National Chemical Residue Monitoring Program

Table of contents

• 1.0 Executive Summary
• 2.0 Introduction
  • 2.1 Background
  • 2.2 Objective
  • 2.3 Scope
  • 2.4 Approach
  • 2.5 Statement of Conformance
• 3.0 Findings and Recommendations
  • 3.1 Management Control Framework
  • 3.2 Sample Plan Development
  • 3.3 Sample Collection
  • 3.4 Private Laboratory Monitoring
  • 3.5 Information Management and Information Technology
  • 3.6 Violation Identification and Follow-up
• Appendix A: Audit Criteria
• Appendix B: Management Response and Action Plan

1.0 Executive Summary

The Canadian Food Inspection Agency (CFIA) is responsible for monitoring the food supply for chemical residues and contaminants and enforcing compliance with maximum residue limits (MRLs), tolerances and maximum levels established by Health Canada.

The National Chemical Residue Monitoring Program (NCRMP) provides information about the residue levels and compliance status of the Canadian food supply. The NCRMP allows the CFIA to identify trends over time, gauge the effectiveness of policies and programs, and develop strategic plans to minimize potential health risks to Canadians. It also enables the CFIA to identify where follow-up and enforcement action is necessary.

In addition, the NCRMP supports the export of Canadian products to trading partners such as the United States and the European Union by demonstrating the equivalence of Canada's residue control system.

The objective of the audit was to provide reasonable assurance that there are adequate processes and controls in place for the management and delivery of the NCRMP. The audit covered the past three fiscal years (2014–15 to 2016–17).

The audit concluded that process and control improvements are required, most notably with respect to establishing clear accountability, roles and responsibilities, and program objectives; and strengthening the management of follow-up on identified violations to ensure that actions and decisions are taken and documented.

The audit provides seven recommendations, aiming to:

• Establish, formalize and communicate accountability, responsibilities, program objectives and plans, and performance monitoring and reporting requirements for the NCRMP.
• Improve the NCRMP sample plans by ensuring that the risk and assumptions associated with each commodity are periodically reviewed.
• Establish clear roles, responsibilities and processes for the dissemination of NCRMP sample plans;
• Put in place a formal process to track and communicate NCRMP sample collection gaps and errors;
• Enhance the monitoring of private laboratory activities;
• Strengthen Information Management and Information Technology procedures and controls associated with the NCRMP database;
• Confirm and communicate accountabilities and roles and responsibilities, and implement a formal process for following up on identified NCRMP violations.

2.0 Introduction

2.1 Background

The Government of Canada operates an extensive food safety program designed to prevent and manage food safety risks. In addition to food safety programs implemented by industry and regular Canadian Food Inspection Agency (CFIA) inspection programs, an essential aspect of food safety is the control of chemical residues and contaminants.

Health Canada establishes maximum residue limits (MRLs) for pesticide and veterinary drug residues for foods sold in Canada, whether produced domestically or imported. Health Canada also establishes maximum levels for food-borne contaminants, including natural toxins, metals, and environmental and food packaging contaminants.

The CFIA is responsible for monitoring the food supply for chemical residues and contaminants and enforcing compliance with MRLs, tolerances and maximum levels established by Health Canada. CFIA's monitoring responsibilities are derived from the Canadian Food Inspection Agency Act, which stipulates that the CFIA is to enforce and administer the provisions of the Food and Drug Act (FDA) that prohibit the sale of food that contains poisonous or harmful substances and/or is adulterated.

The audit of the National Chemical Residue Monitoring Program (NCRMP) was authorized as part of the CFIA's 2016–19 risk-based internal audit plan, which was approved by the CFIA President in June 2016.

This audit is important because the NCRMP provides information about the residue levels and compliance status of the Canadian food supply. The NCRMP allows the CFIA to identify trends over time, gauge the effectiveness of policies and programs, and develop strategic plans to minimize potential health risks to Canadians. It also enables the CFIA to identify where follow-up and enforcement action is necessary.

In addition, the NCRMP supports the export of Canadian products to trading partners such as the United States and the European Union by demonstrating the equivalence of Canada's residue control system.

As a monitoring program, the NCRMP consists of unbiased random sampling and analysis. Sampled lots are not held and are usually available to consumers before test results are known. The NCRMP is designed to select foods for testing based on potential health risk. As such, food items consumed in greater quantities by Canadians and those that may be more contaminated and/or pose a health concern are sampled and tested in greater numbers. NCRMP samples are collected for domestic and imported meat products, dairy products, eggs, honey, fresh fruits and vegetables, processed products, and maple products.

The selection of substances (residues and contaminants) for inclusion in the NCRMP depends on many factors, including historical prevalence and compliance data, availability of a suitable analytical method, laboratory capacity, and access to foreign markets.

The NCRMP sample collection plans specify the type, time and location that a sample is to be collected, and to which accredited laboratory the sample is to be sent for analysis. In accordance with the sample collection plan, CFIA inspectors collect samples of individual lots of domestically-produced and imported foods. Domestic samples are collected as close as possible to the point of production in the distribution system (e.g., slaughter establishments, distribution centers). Imported samples are collected at the point of entry into Canadian commerce.

The majority of the samples collected under the NCRMP are analyzed by private laboratories under contract with the Government of Canada. These laboratories must be accredited to international standards and the analytical methods used must be listed on their scope of accreditation and meet the performance criteria specified by the CFIA.

The results of sample testing are sent to the CFIA and assessed by Science Branch against applicable Canadian MRLs or maximum residue levels. Residues detected at or below established levels are in compliance and do not require enforcement or follow-up action. When a violation is identified, or if a residue is detected where no MRL or maximum level has been established, the result is assessed to determine the appropriate follow-up action.

Follow-up actions vary according to the magnitude of the health risk, with the objective of preventing any repeat occurrence or further distribution of items still in the marketplace. These actions can include further assessment to determine action limits where there is no MRL, notification to the producer or importer, follow-up inspections, further directed sampling, or recall of products if it is determined that the product could pose a safety concern to consumers.

The CFIA publishes the data generated by the NCRMP on its website. Data sets are also produced specifically for trading partners and select NCRMP results are reported in the CFIA Departmental Performance Report.

The table below presents the number of samples delivered (collected and received in good condition) and expenses attributed to private laboratories for the past three fiscal years.

2.2 Objective

The objective of the audit was to provide reasonable assurance that there are adequate processes and controls in place for the management and delivery of the NCRMP.

2.3 Scope

The audit focussed on the processes and controls to support the CFIA's ability to provide information about residue levels and compliance status of the Canadian food supply, as defined by the audit criteria. The audit covered the past three fiscal years (2014–15 to 2016–17).

The audit did not include the following:

• Sample testing performed by CFIA laboratories, since the majority (approx. 90%) of testing is done by private laboratories on contract with the Government of Canada.
• Chemical residue monitoring of fish products, since these activities are not part of the NCRMP.

2.4 Approach

The audit was conducted in accordance with the requirements of the Treasury Board Policy on Internal Audit and the Directive on Internal Audit, which provide mandatory procedures for Internal Auditing in the Government of Canada.

The audit was planned and performed so as to obtain reasonable assurance that the audit objective was achieved. A risk assessment was conducted during the planning phase of the audit to establish the audit criteria, which were accepted by management. The audit findings are based on a comparison of the conditions in place at the time of the audit with the audit criteria.

Audit procedures included:

• Interviews with stakeholders, including Science, Operations, Policy and Programs and Corporate Management branch management and staff, representatives from Health Canada and laboratories under contract with the Government of Canada;
• Document review;
• Process walkthroughs;
• Analysis of data provided by Science Branch related to sample collection, sample testing, analysis of test results, and reporting; and,
• Testing of private laboratory test service standards, private laboratory invoices, violation follow-up files, and reported NCRMP data.

2.5 Statement of Conformance

The audit conforms to the Institute of Internal Auditors' International Professional Practices Framework, as supported by the results of the CFIA's internal audit quality assurance and improvement program. Sufficient and appropriate auditing procedures were performed and evidence gathered in accordance with the Institute of Internal Auditors' International Standards for the Professional Practice of Internal Auditing to provide a reasonable level of assurance over the findings and conclusion in this report. The findings and conclusions expressed in this report are based on conditions as they existed at the time of the audit, and apply only to the entity examined.

3.0 Findings and Recommendations

3.1 Management Control Framework

We expected that a management control framework be in place over the NCRMP and communicated to those involved in program delivery. The expected framework would comprise the following elements:

• Accountability and responsibilities over the NCRMP are clearly established and communicated, to ensure that NCRMP activities are carried out as intended.
• Program objectives and plans are in place so that each branch understands the desired output of the program and allocates the appropriate level of resources required to achieve it.
• Management monitors actual performance against planned results and adjusts course as needed.
• Reliable and timely information is reported to the proper level to demonstrate what has been achieved and what has been learned.

With program delivery occurring across three branches, these elements are especially important to ensure that objectives are met, continuous improvement can take place, and those accountable take responsibility for program performance.

We reviewed program documentation and conducted interviews across the branches involved in NCRMP delivery. We reviewed operational plans (financial and human resources) related to the NCRMP, and we examined reports generated for internal and external stakeholders. The following are the audit findings.

Accountabilities and Responsibilities

We found that the NCRMP is largely managed by Science Branch as part of a suite of food surveillance activities. Program activities are generally carried out as follows: Science Branch prepares the sampling plans; manages the private laboratories; assesses test results, and reports on sampling and testing results. Policy and Programs Branch inputs NCRMP sampling plans into the CFIA's inspection work plans and advises on follow-up on identified violations. Operations Branch collects NCRMP samples and performs follow-up on identified violations.

We did not find any governance document or charter, either NCRMP-specific or as part of a suite of other surveillance activities, that describes accountabilities and roles and responsibilities.

We reviewed CFIA's Sampling and Testing Policy (October 2012), which outlines roles and responsibilities of the Vice Presidents of Science, Operations, and Policy and Programs Branches with respect to testing and sampling activities. We found that the Science Branch was performing activities, such as those related to program design, that according to the Sampling and Testing Policy are the responsibility of the Policy and Programs Branch. We also found that accountability and responsibilities across the branches are not clearly established for the communicating, tracking, and monitoring of follow-up activities taken to address non-compliant NCRMP test results (violations).

In January 2017, the CFIA released a primer document (Mandate, Roles and Responsibilities) for employees that describe high level responsibilities for each branch. The document reaffirmed Policy and Programs Branch's lead role in designing programs for the CFIA.

Without clearly established accountabilities and responsibilities there is a risk that NCRMP activities will not be carried out as intended.

Program Objectives and Plans

We found that the NCRMP's objectives and desired outputs are not documented and NCRMP-specific operational plans are not in place. Science Branch's human resource planning for surveillance activities does not specifically identify NCRMP resource requirements; and, financial planning specific to the NCRMP is limited to budgeting and accounting for private laboratory expenses.

The audit also found that the sample and testing plans are not aligned with allocated resources. For example, the value of the test plans in 2016–17 ($ 10.1 million) distributed to the laboratories exceeded Science Branch's testing budget ($8.5 million). Although the actual number of samples delivered over the three years examined did not result in the testing budget being exceeded, should Operations Branch achieve full delivery against the sample plan there would not be sufficient funds available to cover all planned testing.

It is important that program objectives and the desired level of output be clearly established and communicated so that those involved in program delivery can ensure the required level of resources are allocated and plans are met.

Monitoring and Reporting

We found that management's monitoring of NCRMP activities is mainly focussed on sample collection by the inspectorate. Quarterly sample collection delivery reports are generated by the Science Branch and dashboards providing high-level narrative analysis are presented to Food Business Line Committee. As discussed later in this report, the audit found opportunities to improve monitoring of the private laboratories and the tracking of violation follow-up activities.

NCRMP data and analysis is reported through CFIA's Departmental Performance Report (DPR) and it is shared with other stakeholders, such as trading partners (e.g. United States, European Union) and Health Canada. We analysed the information in the most recent DPR and trade partner reports and we were able to reproduce compliance rates and validate data points presented in the reports.

The audit identified an opportunity for management to define how and when information generated by the NCRMP is publically reported. The CFIA's Sampling and Testing Policy requires that information obtained though sampling and testing be communicated to the public. NCRMP data is available online in the form of an annual report; however, the most recent report is for the 2013–14 fiscal year. Timely public release of NCRMP sample testing results is important to ensure that the information is relevant and useful.

Recommendation 1

The Vice President, Policy and Programs Branch, in collaboration with CFIA stakeholders, should put in place a formal framework for the NCRMP that communicates the following:

• The mandate and objective of the NCRMP.
• Accountability for NCRMP activities, including the roles and responsibilities of each branch.
• The desired level of output of the NCRMP.
• A strategy to monitor and report to CFIA management on the actual performance of the NCRMP against planned results.
• Public reporting guidelines for the NCRMP.

3.2 Sample Plan Development

Science Branch prepares NCRMP sample plans on an annual basis. The plans identify the type of sample to be collected, the amount required (e.g. size of the sample), the approximate date and time for collection, and the area where the sample is to be collected.

The audit expected that a process for developing NCRMP sample plans is established and followed. Further, we expected that plan preparation includes consultation with stakeholders and is informed by relevant internal and external information.

We found that standard operating procedures for NCRMP sample planning exist and records of decision detailing the rationale for the sample numbers are documented, indicating that staff is following the procedures. The records of decision also describe Science Branch's consultations with the Policy and Programs and Operations Branches. Although we did not find documented evidence of consultations with Health Canada specifically for plan preparation, we were able to confirm that communication occurs between CFIA and Health Canada with respect to NCRMP issues.

The audit identified an opportunity to improve the sample test plans by ensuring that the planning risk model is updated regularly. Science Branch officers model the risks associated with chemical residues based on internationally recognized principles. The risk model determines the volume of sample testing required and in turn the number of samples necessary to collect. We noted that the risk-model is not regularly run on all commodities, and sample numbers are often carried over from the previous year's plans. We found that over the audit scope period, only two commodities were updated (meat and egg). When sample and test numbers are not periodically examined and updated, there is a risk that the test and sample plans may not adequately reflect the level of risk associated with each commodity and analyte.

Recommendation 2

The Vice President, Science Branch should establish procedures to ensure that risks and assumptions associated with each commodity are periodically reviewed, and the sample test plans updated accordingly.

3.3 Sample Collection

Sample Plan Dissemination and Collection

The audit expected that roles, responsibilities and processes for the dissemination of NCRMP sample plans and for the gathering of samples are established and communicated.

Upon receipt of the annual sample plans, the Operations Branch disseminates the plans to regions across Canada, and ultimately to the inspectors assigned to collect the samples. This requires effort and relies on local knowledge of Operations Branch staff since some products are only available in certain regions and/or at certain times of the year. It is important that dissemination activities are well managed, since sample collection can only occur when the products are available.

The audit found that roles and responsibilities for sample plan dissemination were not clear within Operations Branch, and the methods used to disseminate the sample plans to the inspectorate vary by area. Methods ranged from a collaborative and coordinated approach involving Regional Program Officers, to unilateral dissemination by an Area Coordinator using the previous year as a base. Without clear roles and responsibilities and processes, there is a risk that dissemination activities will not be effective and result in under delivery against the sample plans.

With respect to gathering the samples, the audit found that roles, responsibilities, processes and guidance are established within Operations Branch. Manuals of procedures for each commodity are in place and communicated. The manuals explain the sample collection process, including detailed guidance on how to gather and ship the samples to the private laboratories.

Recommendation 3

The Vice President, Operations Branch should formally establish roles, responsibilities and processes for the dissemination of NCRMP sample plans to the inspectorate.

Sample Collection Tracking and Monitoring

We expected that sample collection is tracked and monitored with delivery gaps acted upon.

We found that NCRMP sample collection delivery is being tracked. Each month, the private laboratories submit reports to Science Branch indicating the number of samples they received for testing. This information supports Science Branch in determining sample collection rates and identifying where extra effort is needed to meet sample collection delivery targets. Although narrative analyses of NCRMP sample delivery at the commodity level (e.g. dairy, honey) were presented to the Food Business Line Committee, we did not find evidence of detailed sample collection delivery information being communicated to the inspectorate. Without a feedback loop to the inspectorate it is not clear how specific delivery gaps can be acted upon and improvements be made to the sample collection process.

In addition to providing monthly reports on samples received, the laboratories advise Science Branch when they encounter sample collection errors (deviations). Examples of deviations include: wrong sample; spoiled sample; incorrect information provided with sample; duplicate sample. These errors are communicated on an ongoing basis via email. Science Branch then works to correct the deviation. Actions may include contacting the inspector who collected the sample, reassigning the sample to another test plan, or disposing the sample. While some ad hoc tracking of deviations has taken place, we found that Science Branch does not systematically track the number or types of deviations that are occurring. This tracking information would support the identification and correction of systemic collection errors.

Recommendation 4

The Vice President, Science Branch, in consultation with the VP Operations, should:

1. Establish a formal process for communicating NCRMP sample collection gaps to the inspectorate.
2. Ensure that NCRMP sample collection errors are tracked and communicated to the inspectorate.

3.4 Private Laboratory Monitoring

The majority of NCRMP sample testing is performed by private laboratories. The laboratories receive call ups against their standing offer resulting in a contract to perform the required testing. The terms of the contract include test turnaround times that allow the laboratories to accumulate samples and test them in batches.

The audit expected that the CFIA monitors private laboratories against the terms outlined in their standing offers. We focussed on the current standing offers, which had been in place since April 2015 and expire March 2018. We examined how the Science Branch: performs onsite facility evaluations; performs technical verification of amounts invoiced by laboratories; and, monitors the laboratories' adherence to service standards.

Onsite Facility Evaluations

The standing offers outline provisions for the CFIA to conduct onsite facility evaluations to verify that the laboratory's technical capabilities, security status, and human and material resources are carried out as required by the standing offer and any resulting contracts. The audit found that the Science Branch has established and follows processes for facility verifications, including the use of standardized checklists. The audit reviewed documentation related to five facility evaluations and found that they were generally conducted in line with standing offer requirements.

Technical Invoice Verifications

The Science Branch has established processes for technical verification of invoices received from the laboratories for test activities in support of authorization under Section 34 of the Financial Administration Act. We reviewed documentation related to five invoices and determined that the established process was followed.

Service Standards Monitoring

The audit identified areas where the monitoring of private laboratory activities can be strengthened. We found that CFIA had paid for test results that were performed outside of the identified service standards. The Standing Offers state that the Government of Canada is not responsible for costs incurred for tests completed after 120 days of sample reception. Analysis of data provided by the Science Branch revealed that CFIA had paid for test results that were performed outside of these standards. For example, in 2016–17, 4.6% of test results (value of $1,562,461) received from private laboratories were completed outside of the 120 day service standard (1.0% in 2015-16; value of $212,744).

We also found that some private laboratories reached the maximum they could invoice under a given contract, resulting in planned tests being cancelled as samples aged beyond the 120 day service standard. In 2016–17, 6140 tests were cancelled on samples that were received in good condition (1,889 in 2015–16). We found that the laboratories did not always consult with the Science Branch prior to cancelling the planned tests. Without CFIA involvement in these decisions, there is a risk that the private laboratories will make decisions that are not in line with CFIA priorities.

Recommendation 5

The Vice President, Science Branch should enhance the monitoring of private laboratory activities to ensure that samples are tested within required timeframes and that decisions related to cancelling tests on samples received in good condition are made in line with CFIA priorities.

3.5 Information Management and Information Technology

The audit expected that IM/IT systems and processes support the delivery of the NCRMP.

NCRMP data is managed outside of the CFIA's internal laboratory sample tracking system because the private laboratories do not have access to the system. NCRMP sample collection information is captured on paper at the time of sample collection and included with the sample as it is shipped to the private laboratory. Upon receipt of the sample, the laboratories record the information into their own information systems. The laboratories provide CFIA with monthly spreadsheet reports listing the samples they received, as well as test result data.

Science Branch has developed a Microsoft Access database to store, track, analyse and generate reports on NCRMP data. Data received from the laboratories is uploaded into the database. The data undergoes automated quality control before Science Branch officers assess the test results against MRLs or maximum levels established by Health Canada. Reports on the results of the analysis can be customized and generated.

As a user-developed system, the NCRMP database has not been subject to an IT system accreditation process, including a formal threat and risk assessment. During the planning phase of the audit we noted efforts within CFIA to integrate NCRMP activities into CFIA and inter-departmental information systems. However, these initiatives were in early business case development and were not examined by the audit.

The audit focussed on the controls over the existing NCRMP database. Our assessment was guided by the Institute of Internal Auditors' Practice Guide Auditing User-developed Applications, which provides direction on how to scope audits of user-developed databases. We assessed the controls in place to mitigate risks related to data integrity, loss of data, incomplete process and control documentation, and limited segregation of duties. These controls are important to ensure that the IT environment supports accurate information being inputted, processed and reported, and to minimize the risk of loss of data. The following is a summary of related audit findings.

System Access Control

The audit found that the majority of users with access privileges to the database had full access rights, including the ability to add new users. During testing, the auditors were able to circumvent the access controls in place and open the database. The audit also noted that specific database system access rights, justification for providing access rights and approval of access rights were not documented. Access controls are critical to support data integrity and mitigate the risk of unauthorized changes to data.

System Change Management

There is an opportunity to establish and document a change management process. Currently there is limited separation between database development and the end user. This can allow design, programming, and/or logic errors to exist without detection. A formal change management process, including thorough testing of new programing and queries before going live, would mitigate this risk of error.

Documented Procedures and Controls

There is an opportunity to review and ensure that process and control documentation is complete. The audit found that there are work instructions describing how to access the database and analyse data for purposes such as quality assurance and private laboratories invoice verification. However, the instructions are not complete as some front-end procedures, such as how to retrieve the test results to be assessed, are not included in the documents currently available to staff. In addition, the audit did not find formal and documented descriptions of key IM/IT controls. For example, although database backup occurs regularly, the requirements and procedures for data backup are not documented. Documented procedures and controls help ensure consistent work processes and are important to support business continuity should key personnel leave.

Data Integrity

Assessing test results against the MRLs or maximum level is a manual process and performed on live data. There are limited preventative controls in place to guard against human error, such as inadvertent change or deletion of data. While the audit noted Science Branch's efforts to automate the assessments to improve efficiency and reduce the chance of error, there is an opportunity to implement controls such as moving the assessments offline and implementing an automated audit trail to track changes.

Data Quality Assurance

The database has built-in data integrity checks, which flag possible non-conforming data for review. These checks are performed when the test result data is uploaded in the database. A secondary check on the reasonableness of select data is also periodically performed by Science Branch staff. However, issues found during these two processes, along with the changes made to correct them, are not documented and tracked. Tracking would support the identification and correction of systemic data errors.

Recommendation 6

The Vice President, Science Branch, should ensure that Information Management and Information Technology controls associated with the NCRMP database are strengthened. Specifically:

• Strengthen and document access controls, including justification and approval for providing access rights.
• Establish and document a process to manage changes made to the database, including testing of new programing and queries before going live.
• Review and ensure procedures and controls, such as work instructions and backup procedures, are sufficiently documented.
• Enhance data integrity controls, such as moving the assessment of test results offline and implementing an automated audit trail.
• Document and track issues found and corrections made during data quality assurance.

3.6 Violation Identification and Follow-up

We expected that processes are in place and being followed for analysing the NCRMP sample test data received from private laboratories to identify non-compliant results (violations). We also expected protocols to be in place for the communication and risk assessment of violations, as well as for performing and documenting the resulting follow-up action. The NCRMP identifies approximately 500 violations a year and it is important that CFIA has protocols in place to ensure that appropriate follow-up action is taken when required.

Analysing sample test data and identifying violations

We reviewed documentation and performed a detailed walkthrough of the process to identify violations. The assessments of NCRMP sample test results and identification of violations are performed by Science Branch staff. It is mainly a manual process requiring scientific expertise and in some cases consultation with Health Canada. There are general procedures and guidance for conducting assessments. Our walkthrough revealed that Science Branch staff understand how to conduct assessments.

Follow-up activities on identified violations

Science, Policy and Program and Operations Branches are all involved in follow-up activities on identified NCRMP violations. The process for managing follow-up activities generally involves three steps 1) communicating the violation; 2) risk assessing and determining what follow-up actions are required; and 3) performing and documenting the follow-up actions.

We interviewed CFIA officers involved in the follow-up process across Science, Policy and Program and Operations Branches and requested evidence of violation reports being communicated, assessed and actioned. We also reviewed a selection of follow-up files in domestic meat, fresh fruit and vegetables, and imported food.

We found that the process for communicating, assessing, performing and documenting violation follow-up activities varies by commodity and it is not always clear to those involved who is responsible for what step, and who is ultimately accountable for following up. We were not able to obtain evidence that all identified violations had been communicated. We found evidence that CFIA is assessing risks associated with identified violations in consultation with Health Canada; however, there is no systematic risk ranking or prioritization of violations. With the exception of domestic meat, we were not able to obtain evidence that all identified violations over the audit period had been actioned.

We identified good practices in the follow-up activities for domestic NCRMP meat violations. When a domestic meat violation is identified, a Science Branch officer creates a file directly into a dedicated information system for action by Operations Branch. We were provided a list of the domestic meat violations, including each file's status. The files we selected for review provided evidence that the violations were allocated to inspection staff, and that follow-up activities were performed and approved by the Operations Branch Area Coordinator.

In 2016, the European Commission's Directorate-General for Health and Food Safety examined CFIA's follow-up activities and recommended that CFIA ensure that the follow-up of violations is carried out for milk, eggs and honey. In response, the CFIA committed to reconfirm the roles and responsibilities within CFIA and the lines of internal and external communication between all stakeholders in order to ensure that food business operators are notified when violations are identified in milk, eggs and honey. The commitment included ensuring that the inspectors who collected the samples are notified when non-compliant results are identified, and investigate/complete the appropriate follow-up action, as well as document the outcome.

Our findings indicate that a formal violation follow-up process should be established for all food commodities and it should be made clear who is responsible and who is ultimately accountable for these activities. Further, there is an opportunity to provide guidance on how to assess risk and determine what associated actions and decisions are to be taken and documented. These actions would support a standardized approach to addressing violations identified by the NCRMP.

Recommendation 7

The Vice President, Policy and Programs Branch, in consultation with the Vice Presidents of Science and Operations, should:

1. Confirm and communicate accountabilities and roles and responsibilities for the follow-up on identified NCRMP violations, addressing all commodities, both domestic and imports.
2. The accountable official should implement a formal process for following up on identified NCRMP violations for all food commodities, including guidance on how to assess risk, and determine what associated actions and decisions are to be taken and documented.

Appendix A: Audit Criteria

Line of Enquiry 1: NCRMP Management Control Framework

1.1 Accountability and responsibilities over the NCRMP are clearly established and communicated.

1.2 Program has established clear objectives and operational plans are in place.

1.3 Management monitors actual performance against planned results and adjusts course as needed.

1.4 Reliable and timely information is reported to the proper level to demonstrate what has been achieved and what has been learned.

Line of Enquiry 2: Sample Plan Development Processes and Controls

2.1 A process for developing NCRMP sample plans is established and followed.

2.2 Plan preparation includes consultation with stakeholders and is informed by relevant internal and external information.

Line of Enquiry 3: Sample Collection

3.1 Roles, responsibilities and processes for the collection of samples are established and communicated.

3.2 Sample collection is tracked and monitored with delivery gaps acted upon.

Line of Enquiry 4: Laboratory and Data Management

4.1 CFIA monitors private laboratory compliance to NCRMP contracts.

4.2 IM/IT systems and processes support the delivery of the NCRMP.

Line of Enquiry 5: Data Analysis and Follow Up of Violations

5.1 Processes are in place and being followed for analysing the data, including comparing NCRMP lab test results to Health Canada residue / contaminants regulations (e.g. Maximum Residue Limits) to identify violations.

5.2 Protocols are in place to manage violations.

Appendix B: Management Response and Action Plan

Recommendation 1

The Vice President, Policy and Programs Branch, in collaboration with CFIA stakeholders, should put in place a formal framework for the NCRMP that communicates the following:

• The mandate and objective of the NCRMP.
• Accountability for NCRMP activities, including the roles and responsibilities of each branch.
• The desired level of output of the NCRMP.
• A strategy to monitor and report to CFIA management on the actual performance of the NCRMP against planned results.
• Public reporting guidelines for the NCRMP.

Recommendation 2

The Vice President, Science Branch should establish procedures to ensure that risks and assumptions associated with each commodity are periodically reviewed, and the sample test plans updated accordingly.

Recommendation 3

The Vice President, Operations Branch should formally establish roles, responsibilities and processes for the dissemination of NCRMP sample plans to the inspectorate.

Recommendation 4

The Vice President, Science Branch, in consultation with the VP Operations, should:

1. Establish a formal process for communicating NCRMP sample collection gaps to the inspectorate.
2. Ensure that NCRMP sample collection errors are tracked and communicated to the inspectorate.

Recommendation 5

The Vice President, Science Branch should enhance the monitoring of private laboratory activities to ensure that samples are tested within required timeframes and that decisions related to cancelling tests on samples received in good condition are made in line with CFIA priorities.

Recommendation 6

The Vice President, Science Branch, should ensure that Information Management and Information Technology controls associated with the NCRMP database are strengthened. Specifically:

• Strengthen and document access controls, including justification and approval for providing access rights.
• Establish and document a process to manage changes made to the database, including testing of new programing and queries before going live.
• Review and ensure procedures and controls, such as work instructions and backup procedures, are sufficiently documented.
• Enhance data integrity controls, such as moving the assessment of test results offline and implementing an automated audit trail.
• Document and track issues found and corrections made during data quality assurance.

Recommendation 7

The Vice President, Policy and Programs Branch, in consultation with the Vice Presidents of Science and Operations, should:

• a. Confirm and communicate accountabilities and roles and responsibilities for the follow-up on identified NCRMP violations, addressing all commodities, both domestic and imports.

Recommendation 7 (cont.)

• b. The accountable official should implement a formal process for following up on identified NCRMP violations for all food commodities, including guidance on how to assess risk, and determine what associated actions and decisions are to be taken and documented.