Audit of the Broadband Rural and Northern Development Pilot Program—Table of Contents

Final Audit Report

Industry Canada
Audit and Evaluation Branch

July 30, 2007

Recommended for Approval to the Deputy Minister by the DAC on December 05, 2007

Approved by the Deputy Minister on December 12, 2007

Table of Contents

• 1.0 Executive Summary
  • 1.1 Background
  • 1.2 Audit Objectives
  • 1.3 Audit Scope
  • 1.4 Key Findings
  • 1.5 Recommendations
  • 1.6 Lessons Learned
  • 1.7 Audit Conclusion
• 2.0 Statement of Assurance
• 3.0 Background
  • 3.1 Program
  • 3.2 Audit Objectives
  • 3.3 Audit Scope
  • 3.4 Audit Methodology
• 4.0 Detailed Findings and Recommendations
  • 4.1 Effectiveness of the management control framework
  • 4.2 Compliance with the Policy on Transfer Payments
  • 4.3 Findings and Recommendations
• 5.0 Lessons Learned
• Appendix A – Audit Criteria
• Appendix B – Management Response and Action Plan

This publication is available upon request in accessible formats.

Contact:
Multimedia Services Section
Communications and Marketing Branch
Industry Canada
Room 264D, West Tower
235 Queen Street
Ottawa ON K1A 0H5

Tel.: 613-948-1554
Fax: 613-947-7155
Email: ic.cmb-creative.ic@canada.ca

Permission to Reproduce

Except as otherwise specifically noted, the information in this publication may be reproduced, in part or in whole and by any means, without charge or further permission from Industry Canada, provided that due diligence is exercised in ensuring the accuracy of the information reproduced; that Industry Canada is identified as the source institution; and that the reproduction is not represented as an official version of the information reproduced, nor as having been made in affiliation with, or with the endorsement of, Industry Canada.

For permission to reproduce the information in this publication for commercial redistribution, please email: copyright.droitdauteur@pwgsc.gc.ca

Cat. No. Iu4-119/2007E-PDF
ISBN 978-0-662-47482-1

60397

Aussi offert en français sous le titre Vérification du Programme pilote de services à large bande pour le développement rural et du Nord

1.0 Executive Summary

1.1 Background

The Broadband Rural and Northern Development Pilot Program ("the Broadband Program" or "the Program") has a mission to "support the deployment of broadband to un-served communities in order to demonstrate the economic, social and cultural benefits of broadband and to improve their participation in the national and global economy". Funding under the program was provided in two phases. Phase I provided seed funding to a maximum of $30,000 for the development of business plans. Phase II provided funding for the implementation of broadband services in remote communities. A total 154 projects were funded under Phase I ($4 million) and 63 projects were funded under Phase II ($80 million). The Broadband Program is administered through Industry Canada's National Broadband Office in Ottawa and is supported by regional delivery out of five offices across Canada.

1.2 Audit Objectives

The audit of the Broadband Program was selected in accordance with Industry Canada's multi year internal audit plan. The objective of the audit was to provide an independent and objective assessment of the design and operating effectiveness of the Program's management control framework (MCF) and the extent to which the transfer payments under the Program are managed in accordance with the Policy on Transfer Payments.

1.3 Audit Scope

The scope of the audit covered the Program's operations from its inception in 2002 through to May 2007 and all five regions from which the Program is delivered. The audit focused on the areas of highest risk facing the achievement of the Program's objectives. Projects under the National Satellite Initiative (NSI) and broadband projects approved under the Canadian Strategic Infrastructure Fund (CSIF) were outside the scope of this audit.

1.4 Key Findings

The audit noted a number of management controls and operational practices in place within the Program to help ensure sound governance, internal controls, and risk management. The management of the program is consistent with the expectations of the Treasury Board Secretariat Policy on Transfer Payments. A summary of the key audit observations is provided below.

Internal Controls

Program management has established and implemented a number of practices to help ensure compliance with the Policy on Transfer Payments. These include: clearly defined and documented applicant eligibility criteria and assessment processes; standard claims packages and procedures; risk-based approaches for the selection of recipient audit activities; and, standard terms and conditions for each contribution agreement. The contribution agreements include accountability-based clauses related to the project's objectives, roles and responsibilities, project monitoring and reporting requirements, and the conditions required to be met prior to payment.

The Program has adopted an objective and transparent approach to the selection and approval of applications for funding under both funding Phases. The approach and criteria for selection were publicly disclosed. The use of a National Selection Committee comprised of experts in the broadband services area from across the country further contributed to strengthening the objectivity of the project selection and approval process based on merit principles.

Risk Management

The Program has formally defined and documented the key risks facing the achievement of its objectives. Although each project is risk rated against nine pre-defined risk factors, no differences were considered in the extent of ongoing monitoring and claims verification procedures performed. This increased the time and costs associated with meeting ongoing project reporting requirements on the part of recipients. A risk-based approach for recipient monitoring and claims verification activities may have allowed for a more efficient use of scarce Program resources for ongoing risk assessment and monitoring of the overall Program.

In addition, inconsistencies were identified in the level and extent of monitoring activities performed between regions. Additional training and one on one follow up with the regions would help strengthen the consistency of monitoring and other activities across the country.

Governance

The Program is subject to the management controls as defined within the Information Highway Application Branch (IHAB) Management Control Framework as well as the IHAB Regional Management Accountability Framework. Critical success factors that were identified in the Program's management and delivery included: the existence of a formally documented management control framework and supporting policies and procedures; monitoring and reporting on program results against pre-defined performance indicators; dedicated regional resources; and, effective communication between headquarters and the regional offices on a regular basis.

1.5 Recommendations

The Broadband Program is currently being wound down, and is expected to be fully completed by the end of March 2008. In recognition of this, recommendations for improvement are provided for consideration in future pilot programs or other similar programs within the Department. It is recommended that the Director General, IHAB:

• in consultation with the Contributions Quality Assurance Unit, should ensure a riskbased approach to the review and processing of claims is adopted, and supported through formal guidelines and training sessions;
• should ensure that to enable greater consistency in monitoring activities, more frequent formal training sessions are provided to regional staff; and
• should consider assigning a centralized Quality Assurance position with responsibility for following up on regional understanding and application of Program monitoring policies and for providing assistance in conducting recipient monitoring activities.

1.6 Lessons Learned

A number of key lessons learned were identified through the audit for consideration in future Program service delivery:

• Program policies and guidelines should include clear statements of roles and responsibilities for program officers and applicants;
• Due to changes in the composition of assigned resources, continuous training should be provided to program officers;
• Sound knowledge and information management, including regular teleconferences to discuss project issues and practices, should be made available to Program Managers; and
• Flexible approaches to program management, delivery and oversight for pilot programs may help to improve the level of support for the program's clients while still enabling management to focus on results and accountability.

1.7 Audit Conclusion

Overall, we found that the Broadband Program's management control framework and related practices and internal controls are in place. They are operating as intended and are in compliance with Treasury Board's Policy on Transfer Payments. The audit noted one key issue related to the need to adopt a more risk-based approach to recipient monitoring and claims verification activities. This is reported in detail in Section 3 of this report.

2.0 Statement of Assurance

We have completed the internal audit of the Broadband Program. The objectives of this internal audit were to provide an independent and objective assessment of: the design and operating effectiveness of the management control framework (MCF) in place within the Broadband Program; and, the extent to which the transfer payments under the Broadband Program are managed in accordance with the Policy on Transfer Payments. The design and operating effectiveness of management controls within the Broadband Program are the responsibility of Broadband Program management.

The audit examined the management controls in place within the Broadband Program, including management and operational practices, integrated risk management practices, and information management and reporting for decision making in support of the achievement of overall objectives of the Program, and the Program's compliance with the Policy on Transfer Payments including the processes in place to track and monitor recipient compliance with the terms and conditions of their contribution agreements and the practices in place to ensure funds are used for intended purposes.

The scope of the audit included the Broadband Program operations in all five regions from which the Program is delivered across Canada from September 2002 through May 2007. The National Satellite Initiative (NSI) and broadband projects approved under the Canadian Strategic Infrastructure Fund (CSIF) were excluded from the scope of this audit.

Our internal audit conclusions were based on the assessment of findings against preestablished audit criteria and agreed to by management and reflect the audit work conducted between March and May of 2007.

In my professional judgement as Chief Audit Executive, sufficient and appropriate audit procedures have been conducted and evidence gathered to support the accuracy of the conclusions reached and contained in this report. The conclusions are based on a comparison of the conditions, as they existed at the time, against pre-established audit criteria that were agreed with management. The conclusions are applicable only to the entity examined.

3.0 Background

3.1 Program

The Broadband Program operates under Industry Canada's Spectrum, Information Technologies and Telecommunications (SITT) Sector within the Information Highway Applications Branch (IHAB). The Program's objectives directly support Industry Canada's SITT sector objectives of accelerating Canada's transition to the network economy through the development and use of information and communications technologies (ICTs) for the economic, social, cultural and civic benefit of all Canadians.

Launched in September 2002 as a three-year, $105 million pilot program, the Broadband Program's mission is to "support the deployment of broadband to un-served communities in order to demonstrate the economic, social and cultural benefits of broadband and to improve their participation in the national and global economy". The Program had an aim of servicing approximately 400 communities or 10% of un-served Canadian communities. In 2004, Treasury Board Secretariat extended the Program's mandate to 2007 to allow for the completion of funded projects. The Broadband Program was delivered through two funding phases:

• Phase I funding provided the lesser of $30,000 or 50 percent of eligible costs to eligible recipients for the development of business plans. A total of $4 million in funding was provided for 154 projects under this Phase, all of which have been completed.
• Phase II funding provided up to 50% of eligible costs to recipients for the implementation of networks to build broadband infrastructure in un-served communities. Over $80 million in funding was provided for 63 projects under this Phase, of which 55 have been completed as of June 2007. Of the 63 approved projects in this Phase, three did not proceed with full implementation as planned due to the following: one project was merged with another project; and two projects were terminated prior to full implementation as the Community Champions were unable to complete the projects. The average contribution agreement for Phase II projects was $1,267,125, ranging from the highest contribution agreement of $6,194,923 to the lowest contribution of $40,272.

The table below summarizes the total number of projects and total contributions funded by region for Phase II projects:

Eligible recipients under the Broadband program are defined as Community Champions. Community Champions are mainly not-for-profit organizations acting as sponsors to organize and develop a business plan or manage the project's implementation on behalf of un-served eligible communities. All applications for funding required matching funds commitments from community businesses, residents, public institutions, and other community stakeholders. Community Champions were responsible for managing stakeholder relationships and funding.

The program is currently being wound down and is expected to be fully completed by the end of March 2008.

3.2 Audit Objectives

The objectives of the audit were to provide assurance that:

1. a management control framework ("MCF") and related management, operational, and risk management practices were in place and operating as intended; and
2. the transfer payments under the Program were managed in accordance with the Policy on Transfer Payments.

In support of the requirements under Treasury Board's Policy on Internal Audit, the audit criteria were linked to each audit objective and were categorized by internal controls, governance and risk management (please see Appendix A for the specific criteria).

3.3 Audit Scope

The scope of the audit covered the Broadband Program's operations from its inception in 2002 to May 2007 and all five regions from which the Program is delivered across Canada. The specific criteria to be examined and the approach and procedures to be performed under the audit were focused on the areas of highest risk facing the achievement of the Program's objectives.

Projects under the National Satellite Initiative (NSI) and broadband projects approved under the Canadian Strategic Infrastructure Fund (CSIF) were outside the scope of this audit.

3.4 Audit Methodology

The audit was conducted in accordance with Treasury Board Secretariat's Guide to the Planning, Conducting, and Reporting of the Internal Audit Assurance Engagements in the Federal Government of Canada and the Institute of Internal Auditors' International Standards of the Professional Practice of Internal Auditing.

The approach to the audit was detailed in the Final Audit Plan dated March 9, 2007 and consisted of the following:

• Relevant documentation was examined to obtain an understanding of the current risks, risk management processes, management control frameworks, information used for decision making, and internal controls with respect to the Program;
• A sample of Phase I and Phase II files was selected and examined for compliance with Program policies and procedures. Projects from each phase were selected on a judgmental basis, resulting in a sample with the highest dollar value contribution agreements, coverage across each geographical region, and a mix of both closed and open projects. Five project files were selected from Phase I for a total of $148,322 or 4% of total Phase I funding. Ten project files were selected from Phase II representing $29.8 million or 37.3% of the total Phase II funding. A greater proportion of Phase II projects were selected in our sample based on the risk assessment process conducted through the audit planning phase through which Phase I funding was perceived to be of lower risk in comparison to Phase II funding, primarily due to the nature of the projects funded.
• Phase I funding was provided at a maximum of $30,000 for the development of business plans and were short term in nature; in contrast, Phase II funding was provided to fund the actual implementation of broadband access in remote communities and involved multi-year projects of much greater complexity, many of which were individually of higher dollar value. While the audit involved an examination of the management practices and program delivery associated with both Phases, a greater proportion of our file examination process was focused on the Phase II funding period given the higher inherent risk associated with this Phase. The table below summarizes by region: the total number of projects; the total contribution funded; and coverage of total contributions from Phase II files reviewed by the audit team.

• Fifteen interviews with representatives from across the Broadband Program and selected Community Champions were conducted; and
• The information gathered through the above procedures was analyzed against each of the audit criteria.

4.0 Detailed Findings and Recommendations

In accordance with the March 9, 2007 Broadband Program Preliminary Survey Report, management practices and procedures within the Program were examined against each of criteria listed in Appendix A. We found that the Broadband Program's management control framework and related practices and internal controls are in place. They are operating as intended and are in compliance with Treasury Board's Policy on Transfer Payments. It is recognized that the Program is a pilot program currently in the close out phase. Given this, our recommendations for improvement are intended to be considered for implementation within other current or future programs within the Department.

4.1 Effectiveness of the management control framework

A number of strong management practices were identified through the audit, providing evidence that the Program's management control framework is well designed and operating effectively. The audit did not observe any significant issues or deficiencies against the established audit criteria within this audit objective.

Critical success factors that were identified as being in place within the Program included:

• the existence of a formally documented management control framework;
• documented policies and procedures covering all aspects of the program's life cycle;
• regular monitoring and reporting on program results;
• strong financial controls over the review and payment of claims;
• the Program's approach in working with Community Champions; and,
• the frequency of communication between headquarters and the regional offices.

Observations under the categories of internal control, risk management, and governance are provided below.

Internal Control

Our interviews with Broadband program management and staff indicate that the required skills, knowledge and capacity to deliver and manage the Program were in place. Management also continuously worked on securing required resources to match the variations in workload and project volume that occurred in each major phase of the Program's lifecycle.

Training sessions were conducted at key phases of the program, with the last session occurring in September 2004. Due to the high level of turnover and changes in the Program's life cycle since 2004, additional training sessions may have promoted greater consistency in the approaches adopted by the Regions. However, since this time, Program management has held weekly national teleconferences to discuss project issues and share "best practices" with both management and non-management in the regional offices and headquarters. This practice was perceived to be a key success factor in strengthening program skill sets and building consistency in the activities throughout the Broadband Program. These teleconferences would have been beneficial as regular practice from the inception of the Program.

The Program adopted a Community Champion/Aggregator model in dealing with its recipients. This enabled a flexible approach to building local community capacity by combining local level skills with support from Broadband regional officers and engineers. Under this model, Community Champions were provided with dedicated regional Program Officers that resulted in a single point of contact with the Program. Recipients could discuss issues and concerns, project progress, and other matters of relevance to the overall project's success with their dedicated Officer. This model is considered a key success factor in building relationships with the Community Champions and contributing to the building of capacity at the local level.

Risk Management

The Program has formally defined and documented the key risks facing the achievement of its objectives. Changes in risks affecting the achievement of Program objectives are monitored through the monthly reporting package described in the previous section of this report, and management action is taken to respond to changes in risks. At the project level, risks are formally assessed against pre-defined risk assessment criteria at the approval stage. Changes in project level risks are monitored and acted upon through formal project monitoring activities.

Some inconsistencies were identified in the nature of communication provided by the regional offices to Community Champions. Our interviews with recipients identified instances in which the timelines involved in finalizing initial contribution agreements and the effort required to meet ongoing reporting requirements were not clear. In some instances, the announcement of project funding occurred in advance of contribution agreement finalization due to Canadian Environmental Assessment Act (CEAA) requirements which led to construction delays.

The CEAA requirements are clearly described in the Broadband Guidelines for Applicants. However, this section of the Guidelines was not clearly understood by some applicants. It is recognized that risks related to inconsistencies in program delivery are inherent in regional delivery models. The use of formal training sessions and regular information sharing mechanisms, such as weekly teleconferences and applicant guidelines and tools, are key controls in mitigating these risks.

Governance

The Program is subject to the management controls as defined within the IHAB Management Control Framework as well as the IHAB Regional Management Accountability Framework. These frameworks formally define: management principles and values to guide accountabilities; roles and responsibilities; governance structure; regional activities and monitoring responsibilities; and reporting and communication protocols.

Formal reporting to the Director General of IHAB on Program performance against key performance indicators and expected results occurred on a monthly basis. Monthly reports were prepared based on data collected through the Program's in-house database. The database collects information on project progress and status against key performance indicators. The monthly reporting package provides statistical figures and analysis on the number of projects approved, communities served, funding granted, service providers, and other key sources of information required to assess and monitor the Program's overall performance against objectives.

4.2 Compliance with the Policy on Transfer Payments

The audit found that the Program's management practices are in compliance with the Policy on Transfer Payments. Key practices in place to help ensure compliance with the Policy include:

• clearly defined and documented applicant eligibility criteria and assessment processes;
• standard claims packages and procedures to review and verify the validity of costs funded;
• the development of a Risk-based Audit Framework (RBAF) and a Results-based Management and Accountability Framework (RMAF) for the Program; and
• standard terms and conditions for each contribution agreement that include accountability-based clauses related to the project's objectives, roles and responsibilities, project monitoring and reporting requirements, and the conditions that are required to be met prior to payment.

Observations under the categories of internal control, risk management, and governance are provided below.

Internal Controls

The Program adopted an objective and transparent approach to the selection and approval of applications for funding under both funding Phases. The approach and criteria for selection were publicly disclosed on the Program website and in application guides. Projects were selected by a National Selection Committee comprised of experts in the broadband services area from across the country.

This selection model further strengthened the objectivity in the selection and approval process based on merit principles.

Our examination of a sample of Phase I and Phase II project files indicated that strong financial controls are in place and operating as intended. In general, all required documentation in support of claims was on file or was in the process of being followed up by the Program Officer. All approvals were obtained in accordance with delegated signing authorities for the Program. Program procedures also require Program Engineers to certify that all deliverables have been provided prior to approving the completion of a project milestone. These procedures were adhered to in the sample of files examined.

In addition to financial and monitoring controls performed at the regional office level, all claims and supporting documentation were received by management at headquarters for review and approval. This secondary review process allowed for a centralized monitoring role through which project status and overall compliance with Program terms and conditions could be monitored.

Where issues with claims were identified, there was evidence on file of communication between the Claims Officer, Regional Officer and the Community Champion representative to address and resolve the issue in a timely manner.

Risk Management

The Program's RBAF requires a risk assessment of each approved project based on nine defined risk factors. Each factor is assessed on a scale from 1 to 5, resulting in an overall risk assessment of low, medium, or high. The resulting overall risk ratings are used to determine the nature and timing of recipient audits. At the time of this report, eight recipient audits had been completed.

In general, recipient audit reports indicated that the expenditures claimed were in accordance with the terms and conditions of the contribution agreements. While some issues were noted through recipient audit reports, such as instances in which recipients had not deducted interest earned on advances from future claims, there was documented evidence of management's timely follow up and resolution of issues identified. Management's follow up procedures also included consultations with legal counsel when necessary to appropriately respond to and resolve issues identified. Based on these findings, it appears that the Program has developed controls to ensure that high risk projects are appropriately identified, monitored and selected for audit. Management has also followed up on issues identified through its monitoring and audit activities. These controls are operating effectively.

Governance

As described previously, the Program is subject to the management controls as defined within the IHAB Management Control Framework as well as the IHAB Regional Management Accountability Framework. As required under these Frameworks, the Program has designed and implemented policies and procedures to ensure compliance with the Policy on Transfer Payments through all phases of the contribution life cycle. In addition to the Broadband Program's financial and monitoring procedures, IHAB's Quality Control Assurance Unit (CQAU) also conducts periodic reviews of recipient claims for compliance with IHAB policies and procedures across all programs within the Branch.

4.3 Findings and Recommendations

Finding #1

The recipient monitoring and claims verification process is not efficient as standardized monitoring and verification procedures are performed regardless of the project's risk level.

Although a risk rating scale has been defined in the Program RBAF and is used in the selection of recipient audits, no differences were considered in the extent of ongoing monitoring and claims verification procedures based on project risk ratings. This finding is consistent with the December 2006 Blue Ribbon Panel Report on Grant and Contributions Programs which observed that a stronger risk management approach for grant and contribution programs should be developed government-wide.

Our interviews with Broadband management and staff identified that the multiple layers of control in place to review and approve claims and progress reports increased the amount of time and resources required for more "administrative" activities and may have decreased the amount of time available to perform more value added activities, such as overall program risk and performance monitoring. Although the use of standard procedures and multiple layers of review provide a high level of control, the extent of controls to which the Broadband Program was required to adhere to may have been extensive given this was a pilot program with a predetermined lifecycle. In addition, this also increases the time and costs associated with meeting ongoing project reporting requirements on the part of recipients.

The adoption of a more risk-based approach to recipient monitoring and claims verification procedures may improve the efficiency. It may also allow for a more focused level of monitoring of recipient progress against business plans and objectives and monitoring of overall program performance. A risk-based approach would require a determination by program management of an appropriate materiality threshold for claims verification and monitoring requirements.

There are a number of factors that may be considered in development of a more riskbased approach. For example, guidelines for supporting documentation may be based on the number of invoices to review expressed as a percentage of the dollar value of the claim, e.g. for claims in excess of $100,000, 75% of the invoices need to be submitted. Verification may also be based on proof of payment of invoices expressed as a percentage of the dollar value of each claim, e.g. for claims in excess of $100,000, proofs of payment may be obtained for the highest invoices totalling 50% of the dollar value of the claim.

Similarly, consideration could be given to providing payments only upon achievement of specific project milestones, allowing for different delegations of authority based on risk and dollar value, and requiring site visit and engineering reviews at pre-defined stages based on project risk. To supplement a reduced verification process based on risk, additional requests for supporting documentation should be made on a periodic or random basis to provide additional support that funding is being used for intended purposes.

The adoption of a more risk-based approach would need to be supported by the development of guidelines that define the specific documentation required, extent of review and approval required, and approach and timing of random additional verification activities. Once revisions to the guidelines and process have been made, a formal session should be held with program staff and recipients to clarify reporting expectations.

Recommendation #1:

The Director General, IHAB, in consultation with the Contributions Quality Assurance Unit, should ensure a risk-based approach to the review and processing of claims is adopted, and supported through formal guidelines and training sessions.

Finding #2:

Inconsistencies in the level and nature of monitoring activities were identified between the Regions.

Our review of a sample of Phase II recipient files noted isolated instances of late, incomplete and/or missing milestone reports, site visit reports, engineer review forms, and other related documents in 9 out of the 10 files examined. Although follow-up correspondence between the Claims Officer, the Regional Officer, and a representative from the Community Champion were on file in these instances, they are indicative of differences in the level and extent of monitoring activities performed across the country. Regional Officers follow a checklist when conducting their monitoring activities. However, the extent of documentation on file may vary from region to region in practice.

Recommendation #2A:

The Director General, IHAB, should ensure that to enable greater consistency in monitoring activities, more frequent formal training sessions are provided to regional staff.

Recommendation #2B:

The Director General, IHAB, should consider assigning a centralized Quality Assurance position with responsibility for following up on regional understanding and application of Program monitoring policies and for providing assistance in conducting recipient monitoring activities.

5.0 Lessons Learned

A number of key lessons learned and critical success factors for pilot programs were identified during the audit which merit consideration for implementation on a wider basis to promote improved transparency and accountability. These include the following:

• Clear statements of responsibilities in Program policies and procedures for both program officers and applicants will help enable a clearer understanding of respective roles and responsibilities. The Broadband Program had a very transparent approach to publishing the selection and approval process.
• Transparency and objectivity in the selection and approval processes is a key attribute of public accountability. The Broadband Program utilized a National Selection Committee comprised of national representatives with direct industry experience to assist in the project selection process. The assessment criteria, selection and approval process, and composition of the Committee was publicly disclosed, providing for a high level of transparency and increased recipient confidence in the equity of the funding process.
• The use of dedicated regional personnel for each recipient was a key success factor in building local community capacity and providing recipients with a "one-stop shop" for advice, guidance, and issue resolution. In return, the Program benefited by having greater knowledge and continuity with individual recipients which enhanced the quality of project monitoring.
• Communication and information management mechanisms, including regular teleconferences and national training sessions, are key to improving overall awareness, understanding, and consistency between regional offices and headquarters.
• Formal and regular reporting on overall Program risks and performance against predefined indicators is critical to demonstrating and monitoring Program results. The Broadband Program has established an in-house database to systematically monitor and report against key performance indicators. This strengthened the quality and efficiency of monthly management monitoring and reporting of results against objectives.
• Flexible approaches to program management, delivery and oversight are desirable in pilot programs. While sound internal controls are required in any program, a flexible approach is also required in pilot programs to enable management to provide an appropriate level of support to recipients and to allow for sufficient focus on the activities required to achieve the program's overall objectives. This includes the adoption of riskbased approaches to project monitoring and claims verification activities.
• Turnover during the wind-down phase is a common issue in pilot programs. Sufficient planning and resource allocations are required to ensure that a complete and sufficient closure of the Program can be attained. In particular, management has a responsibility to assess and report on the overall achievements of the Program against original objectives and needs to have the necessary resources to support these types of Program wind up activities.

Appendix A—Audit Criteria

The following table outlines the audit criteria that were used to assess Industry Canada's management of the Broadband Program for each audit objective.

Audit Objective #1: A management control framework is in place to effectively and efficiently manage the Broadband Program.

Audit Objective #2 a): Internal controls related to the selection, approval, payment and review of Broadband Projects are in place and working as intended to ensure funds are used for intended purposes.

Audit Objective #2 b): The Broadband Program is managed in compliance with Treasury Board Secretariat's Policy on Transfer Payments, including the existence of effective management practices and controls to monitor recipient compliance with the terms and conditions of their contribution agreements.

Appendix B—Management Response and Action Plan

In recognition of the fact that the Broadband Program is currently being wound down, recommendations for improvement are provided for consideration in future pilot programs or other similar programs within the Department.