Audit of the Northern Ontario Development Program

PDF version

December 2013

Recommended for Approval to the Deputy Minister by the Departmental Audit Committee on January 30, 2014.

Approved by the Deputy Minister on February 6, 2014.

This publication is also available online in HTML at www.ic.gc.ca/eic/site/ae-ve.nsf/eng/h_03683.html.

To obtain a copy of this publication or an alternate format (Braille, large print, etc.), please contact:
The Audit and Evaluation Branch – Office of the Chief Audit Executive
613-954-5073

Permission to Reproduce

Except as otherwise specifically noted, the information in this publication may be reproduced, in part or in whole and by any means, without charge or further permission from Industry Canada, provided that due diligence is exercised in ensuring the accuracy of the information reproduced; that Industry Canada is identified as the source institution; and that the reproduction is not represented as an official version of the information reproduced, nor as having been made in affiliation with, or with the endorsement of, Industry Canada.

For permission to reproduce the information in this publication for commercial purposes, please contact the:

Web Services Centre
Industry Canada
C.D. Howe Building
235 Queen Street
Ottawa, Ontario Canada
K1A 0H5

Telephone (toll-free in Canada): 1-800-328-6189
Telephone (Ottawa): 613-954-5031
Fax: 613-954-2340
TTY (for hearing-impaired): 1-866-694-8389
Business hours: 8:30 a.m. to 5:00 p.m. (Eastern Time)

Email: ic.info-info.ic@canada.ca

© Her Majesty the Queen in Right of Canada,
represented by the Minister of Industry, 2014
Catalogue Number Iu4-153/2014E
ISBN 978-1-100-23285-0

Aussi offert en français sous le titre Vérification du Programme de développement du Nord de l'Ontario.

Table of contents


List of Initialisms and Acronyms Used in Report

List of Initialisms and Acronyms Used in Report
Acronym Meaning
AEB Audit and Evaluation Branch
DG Director General
FAA Financial Administration Act
FedNor Federal Economic Development Initiative for Northern Ontario
FMC FedNor Management Committee
G&C Grants and Contributions
IC Industry Canada
IPR Integrated Planning and Reporting
MPO Monitoring and Payment Officer
NODP Northern Ontario Development Program
PAF Project Analysis Form
PDM Program Delivery Manager
PDO Program Delivery Officer
PSF Project Summary Form
SIS Science and Innovation Sector
T&C's Terms and Conditions
TB Treasury Board of Canada

1.0 Executive Summary

1.1 Background

The objective of the Northern Ontario Development Program (NODP) is to promote economic development, economic diversification, job creation and sustainable, self-reliant communities in Northern Ontario. This is achieved by providing financial support, through transfer payments, to small and medium-sized enterprises and not-for-profit organizations, including municipalities, municipal organizations, community development organizations and research institutions, in three priority areas: community economic development; business growth and competitiveness; and innovation.

The Program is delivered within the Science and Innovation Sector. Within the sector, the Federal Economic Development Initiative for Northern Ontario (FedNor) is responsible for designing and administering two core programs that support regional economic development: the Community Futures Program and NODP.

Since April 2006, FedNor has approved more than $263.5 million in support of 1,322 projects through the NODP to benefit Northern Ontario's economy.

Over the last few years, a number of program-related and organizational changes have had an impact on Program processes. The program-related changes include:

  • Putting in place a Performance Measurement Strategy with enhanced measurement of medium- and longer-term outcomes
  • Implementation of a two-phase application process and an automated application tool
  • Introduction of formal service standards for the application process
  • Renewal of the NODP Terms and Conditions in 2011, with minor amendments to add clarity and address requirements of the Treasury Board Policy on Transfer Payments and Directive on Transfer Payments
  • Implementation of a new risk model for proponent risk assessments, currently used for projects over $500K. The new model, largely based on the departmental model, offers a common approach for project risk assessment and systematic mitigation measures proportionate to the project's level of risk.

In addition to the above program-related changes, FedNor has experienced some organizational changes as a result of the implementation of the FedNor Organizational Restructure Plan. FedNor reviewed its structure and alignment of responsibilities with the objective of restructuring the organization to become more effective, to focus on results-based programming, and to clarify accountabilities. The organizational restructure included clarifying roles and responsibilities to eliminate duplication.

FedNor is also currently being affected by fiscal restraints set out in the 2012 federal Budget and other Industry Canada departmental reductions, which were addressed using work force adjustments and employee attrition.

The changes will continue to have an impact on FedNor in the coming years. In 2012-13, FedNor held a $10.3M salary and operating budget. FedNor's overall contribution to the reductions was a decrease of $4.6M in grants and contributions expenditures and $2M in salary and operating expenditures. This resulted in a decrease in FedNor's Full-Time Equivalent (FTE) position count to a total of 94 positions by 2014-15.

In 2012-13, NODP's planned and actual spending on transfer payments was $36.7M and $32.6M respectively; for 2013-14, planned contribution funding is $37.1M, decreasing to $31.8M in 2014-15 and 2015-16.

1.2 Audit Objective

In accordance with the approved Industry Canada 2013-14 to 2015-16 Multi-Year Risk-Based Audit Plan, the Audit and Evaluation Branch undertook an audit of the Northern Ontario Development Program. The objective of the audit was to provide assurance that the Program's governance, risk management and internal control processes support the delivery of the Program's mandate and priorities in the areas of:

  • Risk management and change / people management at the Program level, and
  • Proponent assessment / approval, monitoring and claims review.

1.3 Main Findings and Recommendations

Change Management

FedNor has processes in place to manage organizational change that supports evolving Program priorities. An opportunity exists to further strengthen current processes.

Recommendation 1
The Director General, FedNor, should ensure that a consistent approach is applied to managing Program priorities. That approach should always consider the potential impact on people, processes and technology; list activities and timelines required for implementation; and assign individuals responsible for executing the activities.

Risk Management

FedNor has processes in place to manage Program risks. There is an opportunity to strengthen risk management practices to regularly identify, assess and review risks.

Recommendation 2
The Director General, FedNor, should implement a formal risk management approach to assist the Program in the identification, assessment and mitigation of key Program risks on a periodic basis. The approach should ensure that mitigation strategies are assigned to an appropriate risk owner with articulated timelines for implementation, and that implementation efforts are periodically monitored.

Proponent assessment / approval, monitoring and claims review

Control Design

Overall, the controls over the proponent assessment / approval, monitoring and claims review processes are adequately designed to mitigate Program risks. Some areas for improvement are noted as follows:

There is no formal approach to documenting the Phase I initial application assessment.

Recommendation 3
The Director, Corporate Services and Policy (FedNor), should implement a formal approach to documenting the results of the Phase I initial application assessment.

There is currently no defined approach to formally track and document monitoring activities within a project file.

Recommendation 4
The Director, Corporate Services and Policy (FedNor), should establish an approach to formally track and document monitoring activities in a project file, and ensure that it is applied consistently.
Control Operating Effectiveness

Based on sample testing performed, certain key controls over the proponent assessment / approval, monitoring and claims review processes are operating effectively. However, exceptions were noted in certain controls, resulting in a conclusion of ineffective operation of controls.

Recommendation 5
The Director General, FedNor, should ensure consistent application of established Program controls over the proponent assessment / approval, monitoring and claims review processes.

1.4 Audit Opinion and Conclusion

In my opinion, overall, the Northern Ontario Development Program's governance, risk management and internal control processes support the delivery of the Program's mandate and priorities with some exceptions noted. Improvements are needed to address low to moderate risk exposures.

1.5 Conformance with Professional Standards

This audit was conducted in accordance with the Internal Auditing Standards for the Government of Canada, as supported by the results of the Audit and Evaluation Branch's quality assurance and improvement program.

space to insert signature

Susan Hart
Chief Audit Executive, Industry Canada


2.0 About the Audit

2.1 Background

In accordance with the approved Industry Canada (IC) 2013-14 to 2015-16 Multi-Year Risk-Based Audit Plan, the Audit and Evaluation Branch (AEB) undertook an audit of the Northern Ontario Development Program.

The objective of the Northern Ontario Development Program (NODP) is to promote economic development, economic diversification, job creation and sustainable, self-reliant communities in Northern Ontario. This is achieved by providing financial support, through transfer payments, to small and medium-sized enterprises and not-for-profit organizations, including municipalities, municipal organizations, community development organizations and research institutions, in three priority areas: community economic development; business growth and competitiveness; and innovation.

The Program is delivered within the Science and Innovation Sector (SIS). Its activities are directly linked to IC's Strategic Outcome that Canadian businesses and communities are competitive. Within the sector, the Federal Economic Development Initiative for Northern Ontario (FedNor) is responsible for designing and administering two core Programs that support regional economic development: the Community Futures Program and NODP.

FedNor is headed by a Director General, who reports to the Associate Assistant Deputy Minister of SIS. FedNor has three directorates that work together to deliver the Program:

  • Program Delivery and Special Initiatives
  • Corporate Services and Policy
  • Communications.

The majority of FedNor employees are involved in Program Delivery and Special Initiatives.

The Program Delivery and Special Initiatives Directorate is split into two regions (Northeastern and Northwestern) and operates out of two main offices: Sudbury and Thunder Bay. It also has four satellite offices in strategic locations  Sault Ste. Marie, Kenora, North Bay, and Timmins  to serve proponents. Officers are located in areas where they can stay in close contact with the communities. The directorate consists mainly of field staff and monitoring and payment officers whose NODP-related responsibilities include:

  • Assessing proponent applications for funding
  • Performing risk assessments on proponents to determine the risk profile
  • Developing Contribution Agreements and monitoring plans for proponents
  • Establishing and maintaining relationships with Program stakeholders
  • Monitoring proponent activity and reporting
  • Evaluating and analyzing expenditures, deliverables and cost forecasts to meet budget requirements in the delivery of the Program.

The Corporate Services and Policy Directorate operates mainly out of the Sudbury office; its functions include:

  • Interpreting and providing guidance on various items including the Treasury Board (TB) Policy on Transfer Payments and Program Terms and Conditions (T&C's)
  • Performance measurement and service standards
  • Developing and maintaining Program Guidelines, procedures documents, tools and training materials, including implementation of the new Corporate Risk Management Framework for Grants and Contributions (G&C) Programs
  • Payment verification and internal controls
  • Corporate Planning.

NODP's program portfolio varies from year to year based on proposals received. In 2012-13 the Program received 169 proposals of which 100 were approved for funding. Of the projects approved, 50% of the funding assistance was allocated to Community Economic Development projects, 43% to Business Growth & Competitiveness projects, and 7% to Innovation projects.

The Program does not have any formal targets for allocation of funds; approval is subject to the proponent/proposal's eligibility, and alignment to the Program's three priority areas, as well as the FedNor priorities defined in its annual Business Plan.

During 2012-13, the Program disbursed $32.6M (planned: $36.7M) in transfer payments, and created and maintained 1,628 jobs in Northern Ontario. The Program's contribution funding for 2013-14 is estimated at $37.1M; funding will be decreasing to $31.8M in 2014-15, which is mainly attributed to fiscal restraints set out in the 2012 federal Budget and other departmental reductions.

Other strategies to deal with the fiscal restraints included a combination of work force adjustments and attrition. In 2012-13, FedNor held a $10.3M salary and operating budget. The overall contribution to the 2012 federal Budget and other Industry Canada departmental reductions was a decrease of $2M in salary and operating expenditures. This will result in a decrease in FedNor's Full-Time Equivalent (FTE) position count to a total of 94 positions by 2014-15.

FedNor also recently finalized the implementation of an organizational restructure. The objective of the reorganization was to become more effective, to focus on results-based programming and to clarify accountabilities. The organizational restructure included clarifying roles and responsibilities to eliminate duplication.

Over the last few years, there have also been a number of program-related changes that have had an impact on Program processes. The changes include:

  • Putting in place a Performance Measurement Strategy with enhanced measurement of medium and longer term outcomes
  • Implementation of a two-phase application process and an automated application tool
  • Introduction of formal service standards and related tracking and monitoring. Standards published on the Program's website are:
    • Acknowledge receipt of an application within 3 working days, and
    • Provide a decision on applications within 80 working days of receipt of a fully completed application
  • Renewal of the NODP T&C's in 2011, with minor amendments to add clarity and address requirements of the TB Policy on Transfer Payments and Directive on Transfer Payments
  • Implementation of a new risk model for proponent risk assessments, currently used for projects over $500K. The new model, largely based on the departmental model, offers a common approach for project risk assessment and systematic mitigation measures proportionate to the project's level of risk.

2.2 Objective and Scope

The objective of the audit was to provide assurance that the Program's governance, risk management and internal control processes support the delivery of the Program's mandate and priorities in the areas of:

  • Risk management and change / people management at the Program level, and
  • Proponent assessment / approval, monitoring and claims review.

The audit covered Program activities and processes from April 1, 2011 to July 31, 2013. For sampling purposes, the audit team performed testing of controls over the proponent assessment / approval, monitoring and claims review processes.

2.3 Audit Approach

This audit was conducted in accordance with the Internal Auditing Standards for the Government of Canada and the TB Policy on Internal Audit. The audit procedures followed and the data collected are sufficient and appropriate to attest to the accuracy of the conclusion and the opinion expressed in this report. This opinion is based on a review of the situations identified in time and place, based on pre-established audit criteria agreed upon with management. This opinion applies solely to the entity reviewed and the framework described in this report.

During the planning phase of this audit, AEB completed a risk assessment of the Program to confirm the audit objective and identify areas requiring a more in-depth review during the conduct phase. Risks were identified by understanding business processes through reviewing key documents and conducting preliminary interviews.

Based on the identified risks, AEB developed audit criteria that linked back to the overall audit objective. Appendix A lists these audit criteria.

The methodology for this audit included document review, interviews with Program management and officers, walkthroughs, file reviews and controls testing. The nature and extent (sample size) of controls testing performed over each process varied and was dependent on factors such as overall population size and the significance and implementation date of key controls. A combined judgemental and random sampling approach was used for file reviews and control testing. The selection of key controls included consideration of controls that most effectively mitigate risk, the Program's process documentation, TB Policy and Directive requirements and departmental manuals for financial assistance programs.

For each process where controls were tested, the following number of proponent files was reviewed:

  • Proponent assessment / approval (20 files)
  • Monitoring (19 files)
  • Claims review (40 files).

A debrief meeting was held with management on November 29, 2013 to validate the accuracy of the findings contained in this report and to discuss potential recommendations.


3.0 Findings and Recommendations

3.1 Introduction

This section presents detailed findings from the audit of the Northern Ontario Development Program. The findings are based on evidence and analysis from the initial risk assessment and the detailed audit work.

In addition to the findings below, AEB has communicated to management, either verbally or by management letter, findings for consideration that were either non-systemic, of low risk or not directly related to the audit objective and criteria.

3.2 Change Management

FedNor has processes in place to manage organizational change that supports evolving Program priorities. An opportunity exists to further strengthen current processes.

Organizational change management is defined as a concerted, planned effort to increase organizational effectiveness. Elements that support effective management of change include the identification of change priorities, and the establishment of operational plans that list the activities required for implementation and assign activity owners and timelines. The potential impacts on people, processes and technology are also important considerations when developing plans to address organizational change priorities.

The effective management of change is essential in addressing the organization's readiness for and acceptance of change and ensuring the continued delivery of the Program in a consistent and timely manner.

Over the past few years, FedNor has been experiencing a number of program-related and organizational changes that have had and will continue to have an impact on Program processes. In the 2012-13 FedNor Business Plan, people and change management have been identified as key priorities. Examples of change priorities in the Plan include:

  • Finalize the implementation of the reorganization by September 30, 2012 to clarify roles, unit responsibilities and improve span of control, and implement 2012 Workforce Adjustment measures
  • Facilitate the transition through talent management, training, learning and career development to help staff perform their work responsibilities effectively and, when possible and appropriate, to advance their career development goals.

Discussions with FedNor management indicated that priorities from the Business Plan are assigned to various units in FedNor for roll out and implementation. Individuals are held accountable for progress against priorities through their Performance Management Agreements, and management is kept informed on progress through discussions at the FedNor Management Committee (FMC).

To assess the effectiveness of current processes, the audit team selected a sample of priorities from the 2012-13 FedNor Business Plan to determine:

  • if documentation exists to support the management of each priority
  • whether consideration was given to the potential impacts on people, processes and technology.

The audit team found a variety of documentation, in various formats, listing the activities and actions planned to address established priorities. Some included a detailed plan, whereas in other cases a task list or analysis working document had been prepared. For some priorities, activity owners, timelines and consideration of the potential impacts on people, processes and technology were documented. The extent of documentation varied depending on the significance and impact of each priority to the organization.

Based on our review, the approach for managing Program priorities does not consistently:

  • consider the potential impacts on people, processes and technology
  • list the activities and timelines for implementation
  • assign individuals responsible for the execution of activities.

An informal or ad hoc plan for implementing priorities may overlook potential impacts on the organization that could prevent the successful implementation of change initiatives. Furthermore, without a detailed plan that lists activities, owners and timelines, there is a risk that priorities may not be implemented in a timely manner.

Recommendation 1
The Director General, FedNor, should ensure that a consistent approach is applied to managing Program priorities. That approach should always consider the potential impact on people, processes and technology; list activities and timelines required for implementation; and assign individuals responsible for executing the activities.

3.3 Risk Management

FedNor has processes in place to manage Program risks. There is an opportunity to strengthen risk management practices to regularly identify, assess and review risks.

Effective risk management adds value as a key component of decision-making, business planning, resource allocation and operational management. The TB Framework for the Management of Risk defines risk management as a systematic approach to setting the best course of action under uncertainty by identifying, assessing, understanding, making decisions on and communicating risk issues.

To complement the TB Framework, IC's Integrated Risk Management Framework requires sectors to identify, prioritize and clearly understand program and organizational risks, and put in place appropriate mitigation strategies and action plans to respond to identified risks.

Based on interviews and a review of Program documentation, FedNor has both formal and informal processes to periodically assess the environment to identify and mitigate risks.

Formal processes include:

  • As part of the IC Integrated Planning and Reporting (IPR) exercise, FedNor submits a risk identification and analysis template for risks considered to be significant enough for consideration in the Corporate Plan. No risks were identified as part of FedNor's contribution to Phase I of the 2013-14 IPR exercise
  • During the Program renewal process, which occurs every five years, FedNor is required to prepare a Risk-Based Framework for the Program. The framework includes identification of Program level risks and corresponding mitigation strategies. The most recent program renewal process took place in 2011.

Informal processes undertaken by the Program include:

  • Risks are discussed at regular meetings between Program managers, directors, the Director General (DG) and the Associate Assistant Deputy Minister. They are also discussed at FMC as they arise
  • In the course of day to day activities in delivering the Program, risks identified by staff are brought to the manager's attention during staff meetings or individual meetings
  • Mitigation strategies for NODP risks are informally monitored through a variety of mechanisms including performance agreements, discussions at FMC and responses to audits and evaluations.

In assessing FedNor's current processes, the audit team found that Program risks are not reviewed or updated periodically. Risks identified by the Program are not assessed for impact and likelihood, and the corresponding mitigation strategies are not formally assigned risk owners and timelines for implementation.

Furthermore, the Program does not formally monitor progress in implementing mitigation activities. While the risk identification and analysis submitted as part of the IPR process would address a number of these elements, the exercise focuses on corporate level risks and would not encompass Program level risks.

Without a consolidated and documented list of all key Program risks, the risks/issues identified informally throughout the year may be overlooked and subsequently not be considered as part of the planning and priority setting process.

Furthermore, if risk owners and timelines are not articulated as part of the risk management process, the Program may not implement mitigation strategies in a timely manner. In the absence of a formal process to monitor implementation, mitigation activities may be set aside for more urgent needs. The consideration of likelihood and impact during the risk assessment process would help the Program prioritize activities when time or resource constraints arise.

Recommendation 2
The Director General, FedNor, should implement a formal risk management approach to assist the Program in the identification, assessment and mitigation of key Program risks on a periodic basis. The approach should ensure that mitigation strategies are assigned to an appropriate risk owner with articulated timelines for implementation, and that implementation efforts are periodically monitored.

3.4 Proponent assessment / approval, monitoring and claims review

Control Design

Overall, the controls over the proponent assessment / approval, monitoring and claims review processes are adequately designed to mitigate Program risks.

The proponent assessment / approval, monitoring and claims review processes are important elements in the Program Delivery phase of a G&C's lifecycle. These processes provide the basis for:

  • determining proponent eligibility
  • making funding decisions
  • monitoring recipient compliance with the T&C's of funding agreements
  • ensuring costs claimed are eligible, accurate and in line with Program requirements.

It is therefore important to have controls that are adequately designed to support the effective management of a transfer payment program and to ensure achievement of overall Program objectives.

To assess control design, the audit team first performed walkthroughs with process owners to gain an understanding of each of the above processes. A brief description of each process is provided below.

Assessment / Approval
  • Proponents submit an initial application (Phase I) which is assessed for basic eligibility. If deemed eligible, proponents provide a detailed submission (Phase II) which is assessed for completeness. An analysis of the applicant's managerial, financial and technical resources is also performed
  • Results of the due diligence performed are documented within the Project Analysis Form (PAF) and Risk Assessment Checklist. For projects being recommended for funding, a Project Summary Form (PSF) is prepared and approved
  • Once the approval has been obtained based on the IC Delegation of Authorities, a contribution agreement is drafted and signed by the Program and the recipient.
Monitoring
  • The Program prepares a monitoring plan that identifies key monitoring activities (e.g. site visits, activity reporting) and the frequency with which each activity is to be performed
  • Proponents are monitored throughout the course of the project (e.g. review of activity reports) and appropriate action is taken if necessary (e.g. project amendments, cash flow updates).
Claims review
  • Proponents submit a completed claim package which is verified for completeness by the Program
  • The Program performs an initial validation of the claim summary, which includes checking for prior conditions and verifying that the claim does not exceed the project's maximum assistance or the Program's available budget
  • The Program performs a detailed review of the claim, including the verification of invoices and proofs of payment, to validate the accuracy, eligibility and reasonableness of costs claimed. The results of the assessment are documented in a G&C Account Verification Checklist that is signed to certify the expenditures, pursuant to the authority under Section 34 of the Financial Administration Act (FAA).

Once the walkthroughs were completed, the audit team identified risks in each process that could preclude the achievement of Program objectives. Examples include: the risk that funding is provided to ineligible recipients and/or for ineligible activities; the risk that changes to a proponent's environment are not considered or assessed; and the risk that payments are made for ineligible costs. The audit team then identified controls in place and assessed whether they were adequately designed to mitigate key risks.

Based on the assessment performed, overall, controls are in place and adequately designed to mitigate Program risks. Some areas for improvement are noted below.

There is no formal approach to documenting the Phase I initial application assessment.

The Phase I assessment process involves a review of the proponent's initial application for basic eligibility (e.g. eligible recipient, activities align to NODP priorities). The Program Delivery Officer (PDO) is tasked with performing this assessment and sharing the results with the Program Delivery Manager (PDM), who provides approval.

Based on the walkthroughs performed, there is currently no formal output or documentation (e.g. initial eligibility checklist) that would capture the results of the Phase I assessment and would support the final decision.

There is a risk that the Program does not maintain sufficient documentary evidence to support the Phase I decision regarding proponent eligibility.

Recommendation 3
The Director, Corporate Services and Policy (FedNor), should implement a formal approach to documenting the results of the Phase I initial application assessment.

There is currently no defined approach to formally track and document monitoring activities within a project file.

For each proponent that is approved for funding, the Program prepares a monitoring plan that identifies key monitoring activities (e.g. site visits, activity reporting) and the frequency with which each activity is to be performed.

The current monitoring plan template includes a field for the PDO / Monitoring & Payment Officer (MPO) to enter the target date and actual date for each monitoring activity. However, these fields are currently not being populated by the program.

During the walkthroughs, the audit team noted that the task management feature in Microsoft Outlook is the primary method used by officers to track and document monitoring activities. However the approach pertaining to which activities are logged and the level of detail provided is not applied consistently. As a result, there is currently no formal mechanism in place to ensure that monitoring activities identified as part of the monitoring plan are in fact being performed and at the desired frequency.

Program management indicated that FedNor is currently in the process of acquiring a client case management software system that will assist in coordinating, accessing and sharing proponent information, including tracking and documenting monitoring activities.

In the absence of a formally defined and consistently applied approach to documenting monitoring activities, there is a risk that these activities may not be performed or executed in a timely manner (as per the monitoring plan). This could result in ineffective monitoring of the proponent and ultimately lead to Program objectives not being achieved.

Recommendation 4
The Director, Corporate Services and Policy (FedNor), should establish an approach to formally track and document monitoring activities in a project file, and ensure that it is applied consistently.

Control Operating Effectiveness

Based on sample testing performed, certain key controls over the proponent assessment / approval, monitoring and claims review processes are operating effectively. However, exceptions were noted in certain controls, resulting in a conclusion of ineffective operation of controls.

In the previous section, the audit concluded that the Program has established controls that are adequately designed to mitigate risk in the areas of proponent assessment / approval, monitoring and claims review. In order to conclude whether an effectively designed control is also operating effectively, it must be evidenced to have operated consistently without exception.

To conclude on the operating effectiveness of controls, the audit team selected key controls from each process and tested for consistent application in proponent files. In selecting key controls, consideration was given to testing significant controls (i.e. those that most effectively mitigate risk), the Program's process documentation, the TB Policy on Transfer Payments, the TB Directive on Transfer Payments, and both the departmental Program Procedures Manual and the Program Policy Manual for financial assistance programs.

A summary of controls testing results is shown below:

Assessment / Approval

Controls found to be operating effectively include:

  • A risk assessment is performed for each proponent, included on file and signed by both the PDO and MPO
  • A PSF is completed, on file and approved in accordance with the IC Delegation of Authorities
  • Section 32 FAA sign-off is appropriately exercised
  • Contribution Agreements are signed by the DG
  • Rejection letters are signed by the DG.

Controls where exception(s) were noted, resulting in a conclusion of ineffective operation of controls, include:

  • Required documentation is not always kept in the project master file (e.g. PAF)
  • Certain sign-offs are missing on key project documentation (e.g. PAF, PSF, Intended Results Form)
  • Evidence does not always exist on file to support a rejection (e.g. Record of Decision).
Monitoring

Controls found to be operating effectively are:

  • A monitoring plan is completed for each project, on file and signed by both the PDO and MPO
  • Project Amendment Forms are prepared, reviewed, approved and signed by the PDO and PDM, and subsequently approved by the authorized officer as per the IC Delegation of Authorities
  • The G&C Closeout Checklist is on file and is reviewed and signed off by the MPO
  • Cash Flow update forms are on file and prepared, reviewed and signed off by the MPO and PDM respectively.

Controls where exception(s) were noted, resulting in a conclusion of ineffective operation of controls, are:

  • Certain sign-offs are missing on key project documentation (e.g. Final Results Report, PDM sign-off on monitoring plan)
  • Risk re-assessments are not being performed for projects over 12 months in duration.
Claims review

Controls found to be operating effectively are:

  • For each proponent, a Claims Package is on file and reviewed for completeness by the MPO
  • The sampling approach applied to a claim, is aligned to the Program's Payment Verification Matrix and there is evidence of review performed by the MPO
  • The G&C Account Verification Checklist and Applicant Claim Summary are reviewed and approved by the MPO and PDM, and evidenced by Section 34 sign-off.

Controls where exception(s) were noted, resulting in a conclusion of ineffective operation of controls, are:

  • The G&C Accounts Verification Checklist is not always fully completed
  • Evidence of PDO review and concurrence of activity reports is not always on file
  • Certain sign-offs are missing on key project documentation (i.e. Section 33 FAA sign-off).

In light of the control exceptions found pertaining to the assessment / approval, monitoring and claims review processes, the Program could be exposed to various risks, including:

  • Incorrect decisions being made regarding proponent eligibility due to inconsistent application of controls
  • A lack of uniformity in file documentation, which could lead to confusion regarding how final funding decisions were made
  • An increased chance that changes to a proponent's environment are not reflected in the risk assessment particularly for longer duration projects (i.e. more than 12 months). This could result in an inappropriate level of project monitoring that is not aligned to actual risk
  • Claims paid out by the Program are not appropriately approved, eligible and in line with the Program's T&C's.
Recommendation 5
The Director General, FedNor, should ensure consistent application of established Program controls over the proponent assessment / approval, monitoring and claims review processes.

3.5 Management Response and Action Plan

The findings and recommendations of this audit were presented to Program management. Management agreed with the findings included in this report and will take actions to address the recommendations by June 30, 2014.

The Program will develop and implement a consistent approach to managing Program priorities. Key Program risk factors, mitigation activities and timelines will be captured in a report that will be reassessed and reviewed on a regular basis. In addition, FedNor will also establish a process for conducting monitoring activities to ensure that there is consistent application of all Program controls.


4.0 Overall Conclusion

The results of the audit revealed that, with exceptions, the Program's governance, risk management and internal control processes support the delivery of the Program's mandate and priorities in the areas of:

  • Risk management and change / people management at the Program level, and
  • Proponent assessment / approval, monitoring and claims review.

Improvements are needed to address low to moderate risk exposures.

Appendix A: Audit Criteria

Appendix A: Audit Criteria
Audit Criteria Criteria Met, Met with Exception or Not Met
Governance
1. The Program effectively manages organizational change that supports evolving Program priorities. Met with Exception
Risk Management
2. The Program has a formal approach to managing Program level risks to support the achievement of Program objectives. Met with Exception
Internal Control
3. Controls over the following processes are adequately designed to mitigate Program risks:
  • Proponent assessment / approval
  • Development and execution of the monitoring plan
  • Accuracy and eligibility of proponent claims
Met with Exceptions
4. Controls over the following processes are operating effectively:
  • Proponent assessment / approval
  • Development and execution of the monitoring plan
  • Accuracy and eligibility of proponent claims
Not Met