Archived — Audit of Early Warning System for Contract Proposals

Archived Information

Archived information is provided for reference, research or recordkeeping purposes. It is not subject to the Government of Canada Web Standards and has not been altered or updated since it was archived. Please contact us to request a format other than those available.

Audit and Evaluation Branch
Industry Canada

September 2008

Recommended for Approval to the Deputy Minister
by the DAC on September 25, 2008
Approved by the Deputy Minister on October 27, 2008


This publication is available upon request in accessible formats. Contact:
Multimedia Services Section
Communications and Marketing Branch
Industry Canada
Room 264D, West Tower
235 Queen Street
Ottawa ON K1A 0H5

Tel.: 613-995-8552

Fax: 613-995-8552

Email: multimedia.production@ic.gc.ca

Permission to Reproduce
Except as otherwise specifically noted, the information in this publication may be reproduced, in part or in whole and by any means, without charge or further permission from Industry Canada, provided that due diligence is exercised in ensuring the accuracy of the information reproduced; that Industry Canada is identified as the source institution; and that the reproduction is not represented as an official version of the information reproduced, nor as having been made in affiliation with, or with the endorsement of, Industry Canada.

For permission to reproduce the information in this publication for commercial redistribution, please email: droitdauteur.copyright@tpsgc-pwgsc.gc.ca

Cat. No. Iu4-141/4-2008E-PDF
ISBN 978-1-100-11184-1
60533

Aussi offert en français sous le titre Vérification du Système de déclaration d'avertissement rapide pour les propositions de marché.

Table of Contents


1.0 Executive Summary

1.1 Introduction

Contracts Review, Program and Services Board (CRPSB) was created with the mandate to provide recommendations and support of corporate direction in the implementation of governmental and departmental policies, operational procedures and performance monitoring related to Industry Canada (IC) contracting activities. CRPSB is responsible for providing recommendations on whether to proceed on all Advance Contract Award Notices; all confirming orders over $25,000; sole source service contracts over $25,000; as well as any amendments to existing sole source service contracts that would bring the total value over this threshold.

In February 2007, the Early Warning System for Contract Proposals (EWSCP) was introduced as an additional control to oversee IC contracting activities and it represents an excellent practice in managing the procurement process. All competitive contracts exceeding the North American Free Trade Agreement limit (at the time $84,000) are to be presented to CRPSB for their review and recommendation. A description of the EWSCP mandate (roles and responsibilities) is provided in Appendix A.

The audit of EWSCP is in keeping with the approved 2008–2009 Audit Plan and is a mandatory audit stemming from a Treasury Board decision letter. Audit results must be provided to Treasury Board Secretariat by October 1, 2008. The objective of the audit is to assure that the EWSCP performance is as per its mandate (see Appendix A). The scope of the audit included an examination of selected procurement documents and contracts issued between April 1, 2007 and March 31, 2008 (FY 2007-08).

1.2 Main Findings

Governance

  • 1. No formal policy or documentation on submission requirements for the EWSCP to provide up-to-date guidance to users is published.
  • 2. The mandate of EWSCP does not include the review of all procurement projects in excess of $84,000. Just under 25% of procurement expenditures over $84,000 for FY 2007-08, all of the Task Authorizations, were not subject to review by EWSCP.

Controls

  • 3. No effective system exists to identify contract proposals required for submission, or to track and monitor the disposition of contract proposals submitted to EWSCP.

Risk Management

  • 4. No formal risk management activities (e.g.: identification, assessment, and mediation strategies) were undertaken to establish risks, risk exposures and levels related to the mandate, roles and responsibilities of EWSCP.

1.3 Recommendations

Governance

  • 1. The Director General (DG), Procurement, Stewardship and Security Branch (PSSB) should ensure that the IC Contracting Policy and related procedures regarding EWSCP are brought up-to-date, published and effectively communicated.
  • 2. The DG, PSSB, in consultation with the Secretary to the PSB, should review and revise the current mandate and activities of EWSCP to ensure that all proposed procurement projects in excess of $84,000 are reviewed.

Controls

  • 3. The DG, PSSB, in conjunction with the Secretary of the PSB, should implement a process and tracking system to ensure that all contracts that require EWSCP review are submitted and that all submissions are recorded and monitored to assess compliance with EWSCP Records of Decision.

Risk Management

  • 4. The DG, PSSB, in conjunction with the Secretary of the PSB, should ensure that the mandate and practices of EWSCP are supported by the results of a formal risk management process.

1.4 Statement of Assurance

In my professional judgment as Chief Audit Executive, sufficient and appropriate audit procedures have been conducted and evidence gathered to support the accuracy of the opinion provided and contained in this report. The opinion is based on a comparison of the conditions, as they existed at the time, against pre-established audit criteria that were agreed with management. The opinion is applicable only to the entity examined.

1.5 Audit Opinion

In my opinion, the Early Warning System for Contract Proposals has weaknesses, with moderate risk exposures, related to its governance processes, controls and risk management that require management attention.

space to insert signature
Richard Willan
A/Chief Audit Executive, Industry Canada
space to insert date
Date
top of page

2.0 About the Audit

2.1 Background

Contracts Review, Program and Services Board (CRPSB) was created with the mandate to provide recommendations and support in terms of corporate direction in the implementation of governmental and departmental policies, operational procedures and performance monitoring related to IC contracting activities. CRPSB responsibilities are to provide recommendations on whether to proceed on all Advance Contract Award Notices, all confirming orders over $25,000, sole source service contracts over $25,000 as well as any amendments to existing sole source service contracts that would bring the total value over this threshold.

In February 2007, the Early Warning System for Contract Proposals (EWSCP) was introduced as an additional control to oversee IC contracting activities. EWSCP was introduced to increase effective management over investment decisions and contract management with concrete action to strengthen is governance. All competitive contracts exceeding the North American Free Trade Agreement limit (at that time $84,000) were to be presented to CRPSB for their review and recommendation. A full description of expected EWSCP roles is provided in Appendix A.

For the fiscal year 2007-08, a total of 124 procurement projects valued over $84,000 were undertaken totaling just over $35.4 million. Of those, 82 projects were contracts totaling just under $26.6 million and included those administered by PWGSC as well as by Industry Canada. In addition to these 82 contracts; 42 Task Authorizations (TAS) valued at just over $8.8 million — not regarded as competitive contract proposals due to the fact that they are issued under a previously-awarded contract — were also valued over $84,000 during FY 2007-08.

2.2 Objective

In keeping with the approved 2008–2009 Audit Plan, and at the request of Treasury Board, the Audit and Evaluation Branch (AEB) has conducted an audit of the EWSCP for contract proposals.

The objective of the audit is to assure that EWSCP performance is as per its mandate (included in part in Appendix A). This was accomplished through three audit sub-objectives:

  • To assess whether and to what extent the governance process defines and preserves values, sets goals, monitors activities and performance and defines accountability methods;
  • To assess the adequacy of the internal control system; and
  • To assess the effectiveness of the risk management system.

2.3 Scope

The audit was carried out during the period June to September 2008 and included an examination of contract activities related to EWSCP between April 1, 2007 and March 31, 2008.

The audit scope included all contracts expenditures over the value of $84,000 including those administered through Public Works and Government Services Canada (PWGSC). The following are exclusions from the EWSCP and from the audit scope: contracts for commissionaire's services, shipping and receiving services, courier services, contracts related to facilities (building, moving and designer services), Public Opinion Research and Advertising, translation services and IM/IT licensing (Outlook, Microsoft Suite etc.).

2.4 Methodology

In support of the requirements under TB Policy on Internal Audit, audit criteria were developed and linked to each audit objective under the categories of governance, internal controls, and risk management.

The audit examined the Management Control Framework (MCF) in place to identify and track contract proposals submitted to EWSCP. The audit work was substantially completed on August 20, 2007 and consisted of:

  • A review of management practices, system of internal controls, related policies, procedures, guidelines and processes;
  • Interviews with departmental officials, including the Manager of Contract and Materiel Management (CMM), Branch Managers, Procurement and Contracting staff and the recording Secretary of the PSB; and
  • Examination and review of procurement and contracting files, EWSCP submissions, EWSCP Records of Decision and the contracting and financial databases.
top of page

3.0 Findings and Recommendations

3.1 Introduction

This section presents detailed findings from the audit of EWSCP. Findings are based on the evidence and analysis from the detailed audit conduct.

3.2 Governance

Finding 1.0: No formally published documents related to EWSCP processes and requirements.

No formal policy or documentation on submission requirements for the EWSCP to provide up-to-date guidance to users is published.

IC policies, guides and communication related to EWSCP contracting activities should be sufficient, available and provided where required in a timely manner.

EWSCP was introduced in February 2007 and a brief communication (email) describing EWSCP requirements was sent to senior departmental managers who were requested to advise respective staff. No subsequent information on EWSCP requirements was included in the IC Contracting Policy or other communication means such as the Bulletin or Staying in Touch. Consequently, there is a risk that all contract proposals are not being submitted to CRPSB as required.

The full draft mandate for EWSCP (roles and responsibilities are included in Appendix A) details the information required from the responsible sector when submitting a contract proposal to EWSCP. The mandate remains in draft form and reflects an evolving process and continued risk that requirements for submission through EWSCP will not be understood until formalized and fully communicated.

Recommendation 1.0

The DG, PSSB should ensure that the IC Contracting Policy and related procedures regarding EWSCP are brought up-to-date, published and effectively communicated.

Finding 2.0: EWSCP mandate does not require the review of all proposed procurement projects in excess of $84,000.

The mandate of EWSCP does not include the review of all procurement projects in excess of $84,000. Just under 25% of procurement expenditures over $84,000 for FY 2007-08, all of the Task Authorizations, were not subject to review by EWSCP.

Given the concerns of Treasury Board, we would expect that the mandate of EWSCP would include a provision to ensure that all procurement projects valued in excess of $84,000 would be subject to EWSCP review.

The EWSCP process however, is limited to competitive contract proposals greater than $84,000 (subject to the specific exclusions noted earlier) and does not include procurement expenditures such as Task Authorizations (TAS) that are not competitive contracts by virtue of being part of and existing contract.

An analysis of CMM data for 2007–2008 indicates that there were 124 procurement projects with a value greater than $84,000 totalling just over $35.4 million. Of this amount, 42 procurement projects representing just less than 25% (just over $8.8 million) of the total were not reviewed by EWSCP. These 42 projects are all Task Authorizations that represent a broad range of contract activities such as; the provision of IM/IT services, developing Project Charters, Project Estimates, Managing Project Scope, Managing Project Plan, and Budgeting.

The exclusion of specific types of procurement projects (such a Task Authorizations) represents a risk that certain expenditures are not brought to the attention CRPSB and EWSCP for their review and recommendation to help ensure consistency with departmental priorities, compliance with policy, reduce project redundancy, provide value for money and provide a heads-up on procurement projects proposed within the department.

Recommendation 2.0

The DG, PSSB, in consultation with the Secretary to the PSB, should review and revise the current mandate and activities of EWSCP to ensure that all proposed procurement projects in excess of $84,000 are reviewed.

3.3 Controls

Finding 3: Ineffective system to identify and monitor contract proposals.

No effective system exists to identify contract proposals required for submission, or to track and monitor the disposition of contract proposals submitted to EWSCP.

We would expect that a system should exist to: a) identify all proposed contracts that should be reviewed by EWSCP; and b) to monitor reviewed and awarded contracts to assess if they are in keeping with EWSCP records of decision.

Responsibility for the management of information pertaining to EWSCP submissions is not explicitly assigned and no EWSCP tracking system exists to allow EWSCP records of decisions to be readily associated with a contract once it is awarded.

Information from the Integrated Financial and Materiel System (IFMS) indicates that based on the established criteria (over $84,000), 82 contracts should have been submitted for review by EWSCP during FY 2007-08. Records of decision from EWSCP indicate that 62 contract proposals received for review during the same period; representing a disparity of 20 contracts. No system exists to ensure that all contract proposals meeting the EWSCP criteria are submitted for review. And, while CMM has been tasked with following up to ensure the appropriate actions take place, there is also no mechanism is in place that allows CMM to reconcile IFMS information with EWSCP records of decision.

Having no effective system to identify all contract proposals required for review and no method to assess the disposition of EWSCP recommendations further increases the risk that contract proposals over the $84,000 threshold that should be reviewed, are not considered by EWSCP; and even if reviewed are not monitored to assess compliance with EWSCP records of decisions.

Recommendation 3.0

The DG, PSSB, in conjunction with the Secretary of the PSB, should implement a process and tracking system to ensure that all contracts that require EWSCP review are submitted and that all submissions are recorded and monitored to assess compliance with EWSCP Records of Decision.

3.4 Risk Management

Finding 4: No formal risk management to support EWSCP.

No formal risk management activities (e.g.: identification, assessment, and mediation strategies) were undertaken to establish risks, risk exposures and levels related to the mandate, roles and responsibilities of EWSCP.

We would expect that management would identify, assess and formally respond to risks that may preclude the achievement of EWSCP objectives.

No risk management activities were undertaken either at inception or during the life of the EWSCP initiative. The stated intent of EWSCP is to review all competitive contract proposals greater than $84,000; yet this intent is subject to certain exclusions. In addition, while the intent is also to "…ensure that procurement projects submitted to the Board…" meet certain criteria (see Appendix A); EWSCP has excluded procurement projects (such as the Task Authorizations noted above). No formal risk management activities were completed to identify, assess and mediate the possible risks that may be associated with these limitations on scope of EWSCP activities.

Continuing current EWSCP practice, without the support of basic risk management, leaves EWSCP open to the risks that potentially higher-risk procurement projects are not being reviewed, while lower-risk procurement projects may unnecessarily consume resources.

Recommendation 4.0

The DG, PSSB, in consultation with the Secretary to the PSB, should ensure that the mandate and practices of EWSCP are supported by the results of a formal risk management process.

top of page

Appendix A—Detailed Audit Criteria

Proposed Mandate for PSB Members

Review and make recommendations on competitive contract proposals with a value greater than $84,000 to ensure that procurement projects submitted to the Board meet the following criteria:

  • Aligns with the departmental priorities
  • Compliance to Policies
  • Enhance synergy between Sectors and reduce redundancy of procurement projects
  • Provide value for money
  • Heads up on procurement projects being developed within the department

Roles and Responsibilities

Sponsoring Sector/Branches

  • Using the template for procurement projects greater than $84K; provide as much information as possible on the procurement project. The template should be completed as per guidelines in the "Information required" section.
  • As soon as the template is completed, sponsoring sector/branch should provide a copy of their submission to the CMM contracting officer for review and comments.
  • Upon review of the submission, CMM shall inform the sector of any revisions/changes made and shall request the Sector's / Branch's approval to go forward with the revised/updated submission.
  • The sponsoring Sector / Branch must ensure that a representative (usually the Project Authority) will be available to attend the PSB meeting (if requested by the PSB secretariat) to present the procurement project and respond to questions from PSB members.

Contracting and Materiel Management

  • Assist sponsoring sector/branch in preparing the template for procurement project with a value greater than $84K and provide advice on the procurement vehicle.
  • Review the template and make changes if required and provide the updated version to the Sector's /Branch's Point-of-Contact person for the Sector's final approval prior to CMM forwarding the submission to the PSB secretariat for tabling at the next available PSB meeting.
  • The CMM Manager shall attend the pre-PSB and provide background of the procurement project to the PSB Chair. Following pre-PSB, the PSB Secretariat may contact the sponsoring Sector / Branch to invite a representative (usually the Project Authority) to present the procurement project to PSB members.
  • After the PSB meeting, CMM manager will contact the sponsoring sector/branch to inform him/her of the PSB recommendation or any follow up to PSB questions.

Appendix B: Detailed Audit Criteria

For the risk management and governance process, the extent to which Industry Canada's EWSCP related Management Control Framework (MCF) is adequate and is functioning as intended.

  • Risk Assessment / ManagementIC identifies, evaluates and manages risks pertaining to contracting (contract proposals). Risk assessment is documented.
  • Policy FrameworkEWSCP policies and procedures are sufficient, available and provided where required in a timely manner.
  • Roles & ResponsibilitiesEWSCP related roles and responsibilities are clearly defined, understood and documented.

IC's internal controls for EWSCP activities ensure compliance with the stated mandate for the review of contract proposals.

  • Contract Planning — Contract planning activities clearly define requirements, taking applicable policies and the rules of fair competition into account.
  • Solicitation Activities — Solicitation activities comply with the procurement process in accordance with the approval set forth in the applicable EWSCP Record of Decision.
  • Contract Award — Contracts are awarded in accordance with the approval set forth in the applicable EWSCP Record of Decision.
top of page

Appendix C: Management Response and Action Plan

Project Name / Number:
Audit of Early Warning System for Contract Proposals / 08-02
Updated As At:

Management Response

In response to your request dated September 18, 2008, please find below the proposed management response to the AEB Draft Audit Report on the Early Warning System for Contracting Proposals (EWS CP).

Recommendation 1

The DG, PSSB should ensure that the IC Contracting Policy and related procedures regarding EWSCP are brought up-to-date, published and effectively communicated.

Management Response

Management agrees with this recommendation and will ensure appropriate and timely follow-up action.

Actions taken to date — The IC website contains a page devoted to contracting policy and PSB topics. Communication with IC staff is managed through the use of the on-line publication, "Staying in Touch". This newsletter explains subjects of interest to managers as well as those employees with contracting authority, such as policy changes or results of contract monitoring.

Next steps — The PSB Terms of Reference and the contracting web page will be updated to reflect the purpose of the EWS submission and instructions to proponents for its completion.

Recommendation 2

The DG, PSSB, in consultation with the Secretary to the PSB, should review and revise the current mandate and activities of EWSCP to ensure that all proposed procurement projects in excess of $84,000 are reviewed.

Management Response

Management agrees with this recommendation.

Action to date — In June 2008, IC lowered the PSB threshold on EWS and contract proposals from $84K to $76.5K to reflect the revised NAFTA limit. As of September 16, 2008, Task Authorizations (TA) are being reviewed by PSB. Previously and additionally, CIOB manages a comprehensive TA oversight function, in place since April 1, 2008, which includes ITSMC and POC governance and oversight.

ITSMC is responsible for IT business and investment allocation. This committee approves the annual Departmental IT plan and makes recommendations to the Deputy Minister on departmental IT expenditures. It also reviews and provides approval for new project proposals.

ITSMC's mandate is to:

  • Make IT investment decisions that are aligned with the Departmental mandate and strategic outcomes, and to allocate departmental resources to support these investment priorities;
  • Manage allocations for strategic IT activities within the set IT spending envelope;
  • Review and approve IT strategy and policies; and
  • Support the CIO in achieving cost-efficiencies for all departmental IT activities.

Project Oversight Committee (POC)

POC provides ongoing oversight of IT projects to help ensure that projects follow the Department's project management framework. This committee has functional control of all IT projects and reviews active projects at regular intervals (using the stage-gate process outlined below).

POC's mandate is to:

  • Enforce the use of standard project management processes;
  • Determine whether required deliverables are met at each gate of the stage-gate process;
  • Provide ongoing project oversight (and identify the need for third-party reviews) that will help to ensure that project outcomes are realized and costs are minimized;
  • Verify that the appropriate systems, processes and controls for managing projects are in place department-wide; and
  • Support the achievement of project and program outcomes while limiting the risk to stakeholders and taxpayers.

Recommendation 3

The DG, PSSB, in conjunction with the Secretary of the PSB, should implement a process and tracking system to ensure that all contracts that require EWSCP review are submitted and that all submissions are recorded and monitored to assess compliance with EWSCP Records of Decision.

Management Response

Management agrees to review the current process for monitoring and tracking EWS submissions tabled at PSB with a view to ensuring compliance with EWSCP records of decision. Management will ensure that additional tracking measures, procedures and a system are implemented quickly, while ensuring that the approach is practical, feasible and affordable, fully respecting departmental IT and project management protocols.

Action to date — The results of PSB views on EWS submissions are recorded in the minutes and proponents are advised of the required actions to be taken with respect to their submission as recommended by PSB.

Next steps — Management will assess options and develop an approach complete with action plan, milestones and dates/

Recommendation 4

The DG, PSSB, in consultation with the Secretary to the PSB, should ensure that the mandate and practices of EWSCP are supported by the results of a formal risk management process.

Management Response

Management agrees that a formal risk management process needs to support the EWSCP process.

Actions to date — The current Terms of Reference for PSB include a section on Risk Evaluation for each EWS submission. The proposed Terms of Reference, due to be presented to PSB October 2, 2008, require proponents to respond to the following:
"What is the impact of not performing or delaying the work? Additionally, specify for each risk the probability of occurrence (High, Medium or Low) and rate the impact if the risk occurs (High, Medium or Low). For those risk areas where either the probability of occurrence, or the impact is rated as High or Medium, a precis of the risk mitigation strategy must be included. Those proposals with a combined rating of high/medium or high/high would not normally proceed."

Next steps — A formal, documented risk management process will be put in place. This risk management process includes a system that compares EWS submissions with contracts awarded. Through this process, high risk procurement activities will be assessed in accordance with the EWS mandate in the PSB Terms of Reference.

Management response regarding TA's was previously noted in Recommendation.

top of page

Management Action Plan

Management Action Plan
Recommen-dation Planned Action or Justification for no action on the Recommendation Responsible Official Target Completion Date Revised Completion Date Current Status
1. The Director General (DG), Procurement, Stewardship and Security Branch (PSSB) should ensure that the IC Contracting Policy and related procedures regarding EWSCP are brought up-to-date, published and effectively communicated. The IC intranet website which contains IC's Contracting Policies, Staying In Touch newsletters, etc. needs to be more visible to the user community. Contracting and Procurement will have its own web page under the Comptrollership and Administration Sector's Web site, which will give needed visibility to Contracting and Procurement and will make it easier for IC employees to use.
Manager, CMM Mar 31, 2009
A special edition of the Staying In Touch newsletter will be written that will detail the EWS process, and when it applies. This special edition will be e-mailed to all staff with contracting authority. Nov 30, 2008
2. The DG, PSSB, in consultation with the Secretary to the PSB, should review and revise the current mandate and activities of EWSCP to ensure that all proposed procurement projects in excess of $84,000 are reviewed. Existing IC contracting policies will be updated to include the EWS internal process.
Manager, CMM Jan 31, 2009
The existing IC contracting training course will be modified to include the EWS process. Oct 31, 2008
3. The DG, PSSB, in conjunction with the Secretary of the PSB, should implement a process and tracking system to ensure that all contracts that require EWSCP review are submitted and that all submissions are recorded and monitored to assess compliance with EWSCP Records of Decision. The PSB's Terms of Reference (TOR) have been revised and include a detailed section on the EWS process. The EWS section clearly indicates that all proposed procurement over $76,500 must be reviewed by PSB, with the exception of certain procurements that are considered to be of minimal risk. These exclusions shall be clearly stated in PSB's TOR.
Manager, CMM Oct 2, 2008
CIOB will prepare quarterly Task Authorization (TA) reports for PSB that will describe the project, the date of project approval by the CIOB governance structure, the risk analysis, and the value and number of TAs issued. Completed
4. The DG, PSSB, in consultation with the Secretary to the PSB, should ensure that the mandate and practices of EWSCP are supported by the results of a formal risk management process. The EWS template instructions have a revised Risk Evaluation section to ensure that a complete and detailed risk assessment has been done prior to the submission of the EWS to PSB. The proponent will be required to include a précis of their risk mitigation strategy for risks that are considered medium to high.
Manager, CMM Oct 2, 2008
A formal, documented risk management process will be put in place. This risk management process includes a system that compares EWS submissions with contracts awarded. Through this process, high risk procurement activities will be assessed in accordance with the EWS mandate in the PSB Terms of Reference. Mar 31, 2009

Prepared By: A.J. Jones DG, PSSB

Approved By: Bill Merklinger A/CFO

  • Email
Help us improve
Back to "Help us improve" section.
  
Back to "What's the problem?" section.
Got it, thanks!
Um, you didn't enter anything.
Date modified: