Horizontal Audit of Project Costing

From: Innovation, Science and Economic Development Canada

February 2021

Horizontal audit of project costing [PDF - 747 KB]

Table of contents

• 1.0 Executive summary
  • 1.1 Introduction
  • 1.2 Audit background
  • 1.3 Overview of audit results
  • 1.4 Audit opinion and conclusions
  • 1.5 Management response
  • 1.6 Statement of conformance
• 2.0 Background
  • 2.1 Overview
  • 2.2 Previous audit engagements
• 3.0 About the audit
  • 3.1 Audit objective, scope and methodology
• 4.0 Findings and recommendations
  • 4.1 Introduction
  • 4.2 Costing process
  • 4.3 Costing tools, guidelines and support
  • 4.4 Cost monitoring and reporting
  • 4.5 Management of cost changes
  • 4.6 Management response and action plan
• 5.0 Overall conclusion
• Appendix A: Audit criteria

List of acronyms used in report

AEB

Audit and Evaluation Branch

CMS

Corporate Management Sector

DPMO

Departmental Project Management Office

DPPM

Departmental Project Portfolio Management System

DTSS

Digital Transformation Service Sector

IEOC

Investment and Experimentation Oversight Committee

IOC

Investment Oversight Committee

PSC

Project Steering Committee

RMC

Resource Management Committee

RPMC

Resource and People Management Committee

1.0 Executive summary

1.1 Introduction

Effective project cost management is essential to ensure stewardship of public funds. Cost management involves planning and estimating costs, determining the budget, establishing a baseline, controlling and monitoring costs, as well as managing changes to the cost baseline.

Depending on the nature, complexity and value of the initiative, costing estimates have different risk level. The spectrum includes, Treasury Board (TB) submission and Memorandum to Cabinet (MC), which are considered higher risk and attract significant resources; governed projects requiring internal oversight, which are medium risk; and internal reallocation projects, which are low risk.

For this audit engagement, the scope was limited to projects subject to the departmental Stage-Gate Framework and did not include MCs and TB submissions. The Department has identified these projects as representing a medium risk based on a combination of factors, including volume and materiality (both 5th largest grouping out of the 10 costing areas expressed in volume and dollar value), as well as complexity, availability of guidelines, and who is leading the costing.

The Department uses a Stage-Gate Framework, a six-step process from idea generation to post-project review, to guide and govern the project-development lifecycle. The Stage-Gate Framework allows for regular reviews and approvals of the projects, and ensures the monitoring of costs within the predefined constraints while achieving the planned outcomes. Depending on the total estimated cost, a project is subject to approval by:

• the Resource and People Management Committee (RPMC), which authorizes projects to move from the idea stage to the planning stage;
• the Investment and Experimentation Oversight Committee (IEOC), which oversees projects over $500,000 from planning to completion stages, and authorizes project change requests related to scope, schedule or cost; and
• the Project Steering Committees (PSCs), which review submissions to RPMC and IEOC prior to approval for all projects, and approve projects less than $500K at stages 3 to 6.

The Departmental Project Management Office (DPMO) is responsible for project and portfolio management functions for all departmental projects subject to the Stage-Gate process. The DPMO resides in the Corporate Management Sector (CMS) since April 2020 (formerly in the Digital Transformation Service Sector, DTSS). By doing so, the DPMO plays a pivotal role as a central resource for project managers by providing standardized processes, practices and tools across the Department and centralized oversight for project and portfolio management functions, as well as reporting mechanism for project costing activities.

The departmental guidelines and tools, developed by DPMO, are used to support the project gating approval through the Stage-Gate process, including approvals of initial cost estimate and cost changes throughout the project life cycle. DPMO provides support and guidance to project managers on these tools. While responsibilities related to developing cost estimates and implementing projects reside within each sector, all project management information, including cost information, is centralized in a system called the Departmental Project Portfolio Management System (DPPM). The DPPM is used for governance reporting to assist oversight bodies in the decision-making process, and for departmental reporting to Treasury Board Secretariat.

The audit team selected a sample of projects to examine based on a list of departmentally-governed projects extracted from the centralized project management tool (the DPPM mentioned above) on December 12 2019. As detailed project costing is provided to governance committees at stage 4 of the Stage-Gate process and beyond, the sampling population included active and closed projects at stages 4 to 6 in order to assess project costing practices. Ten projects were selected for the sample, valued at $110 million over 9 fiscal years, and representing 72% of the value of projects governed by the Stage-Gate process at that time. Five of the unfinished projects included in the sample have since been completed, and while cost estimates changed during the audit period of scope, final costs for these projects remained mostly close to their estimated value identified in the December 2019 costing documentation.

1.2 Audit background

The objective of this audit was to provide assurance that costing activities related to projects are in place, used consistently, and are operating effectively.

The audit scope included 10 multi-year projects subject to costing activities as defined by ISED's Stage-Gate Framework, and covers the period of April 1, 2018 to September 30, 2019, including:

• Governance and oversight;
• Internal controls;
• Effectiveness of processes and practices.

1.3 Overview of audit results

Strengths

Cost estimates are approved by key oversight committees. The process is applied consistently, sufficient information is provided to the committees to support approvals, and decisions are documented in meeting records.

The Departmental Project Management Office (DPMO) has developed departmental tools and guidelines for the approval of cost estimates via the Stage-Gate process and provides support on the use of them. These tools include documents presented to oversight committees to support approvals and a centralized Departmental Project Portfolio Management (DPPM) System used for recording governance decisions.

Project cost monitoring and reporting processes are in place in support of effective resource management. Costs are monitored at the departmental level by senior-level governance committees as projects move from one stage to the next, and internally with Project Steering Committees and project sponsors on a regular basis.

Change requests involving cost changes are approved by key oversight committees throughout the project lifecycle, according to a formal process based on predefined cost variance thresholds and the project's current stage. The approval process is applied consistently and decisions are documented.

The Stage-Gate process helps to refine cost estimates throughout the project lifecycle by requiring the review of project costs at each stage by governance bodies, as well as justification and approval of cost changes if costs exceed thresholds defined in the Stage-Gate process.

Areas for improvement

Some opportunities for improvement were identified by the audit. There is an opportunity to strengthen departmental project costing practices by ensuring that a formal project costing approach is used and documented for all projects.

In order to ensure consistent and accurate costing practices, departmental costing tools and guidelines should be developed based on the Treasury Board's Guide to Cost Estimating.

There is an opportunity for DPMO and CMS financial specialists to be involved in project costing efforts and validate the proposed costs prior to the approval process. The expertise from project management and financial specialists would be beneficial in selecting the appropriate cost-estimating technique to use, challenging the costing assumptions for completeness, and validating the accuracy of the estimate.

1.4 Audit opinion and conclusions

Effective governance processes are in place for the approval of cost estimates and cost changes throughout the project lifecycle, via the departmental Stage-Gate process. There are opportunities to strengthen practices, tools, and guidelines related to the development of project cost estimates to ensure a rigorous and consistent approach across the Department.

1.5 Management response

Management has agreed with the findings included in this report and will take action to address all recommendations by March 31, 2022.

1.6 Statement of conformance

This audit was conducted in accordance with the Internal Auditing Standards for the Government of Canada, as supported by the results of the Audit and Evaluation Branch's quality assurance and improvement program.

_________________________________________

Denis Martel
Chief Audit Executive
Innovation, Science and Economic Development Canada

2.0 Background

2.1 Project costing overview

Costing in the federal government

Costing is an essential element of project management that helps decision-makers plan and budget for their activities effectively. Cost information helps decision-makers understand the financial impact of the decisions they make and the initiatives they propose. The total estimated project cost also determines the approval and expenditure authorities, as well as the reporting process required for the project.

The objective of project cost management is to ensure that costs remain within budget and that exposure to the risk of cost overruns is adequately mitigated. As illustrated in Figure 1, project cost management involves planning and estimating costs, determining the budget, establishing a baseline, controlling and monitoring costs, as well as managing changes to the cost baseline. A variety of factors are to be taken into consideration when costing a project, including direct and indirect costs, project planned timelines, and the type of project outcome to be delivered.

Figure 1: Project cost management lifecycle^{Footnote 1}

Text version

The management of project costs is a continuous cycle that involves 4 key steps:

• Identify and plan resources;
• Estimate costs;
• Budget and allocate costs;
• Control and report costs.

In April 2019, the Treasury Board (TB) Directive on the Management of Projects and Programmes took effect, replacing the Policy on the Management of Projects, as well as the associated Standard for Organizational Project Management Capacity and the Standard for Project Complexity and Risk. The objective of the new Directive is to ensure that government projects and programmes are effectively planned, implemented, monitored and controlled, and closed to enable the realization of the expected benefits and results for Canadians. It introduces new requirements for the role of the Project Sponsor in terms of leadership, governance, and performance monitoring, as well as the role of the Chief Information Officer (CIO) with respect to digital projects. The Directive also clarifies the TB project approval process, increases the base departmental project approval authority limit, and introduces procedural changes for assessing departmental project management capacity.

Effective project costing is essential to ensure stewardship of public funds, and investment planning is a key element of project management. In April 2019, the TB Policy on the Planning and Management of Investments took effect, replacing the Policy on Investment Planning – Assets and Acquired Services. The policy applies to projects in terms of ensuring effective resource allocation and consideration of the full life-cycle cost of assets. The 2019 policy includes new requirements related to project management workforce, alignment of management processes with investment risk, consideration of input from key internal services functions (e.g. finance, procurement, HR, IT), approval authorities, project governance, content of the corporate investment plan, and procedures for concept case for digital projects.

Following the establishment of the new investment planning policy and the changes to the directive related to project management, the Treasury Board Secretariat (TBS) updated its costing expectations through its Guide to Cost Estimating in 2019. The objective of the Guide is to provide practical guidance for developing credible cost estimates in support of information to help decision-makers understand how the cost estimates are developed, improve the understanding of uncertainties, risks and sensitivities associated with the estimate that could impact cost, provide evidence that is validated, and in the case of Cabinet submissions, support the assertions and conclusions of the Chief Financial Officer (CFO), as required by the Policy on Financial Management and the supporting Guideline on Chief Financial Officer Attestation for Cabinet Submissions. In addition, the Guide suggests responsibilities related to cost estimates, as well as techniques and steps to estimate costs.

Project costing at ISED

ISED's Departmental Project Management Office (DPMO) defines a project as a temporary endeavour, with an activity or series of activities, undertaken to create a unique product, service or result, and is undertaken within specific time, cost, scope and performance parameters.

ISED's Corporate Plan includes the Department's Investment Plan, which summarizes the activities and long-term investments that ISED plans to undertake in order to achieve the department's strategic objectives. The Investment Plan provides a detailed description of planned investments, which includes major acquisitions, services and projects, above $500,000 over 5 years. In the 2018-19 Plan, the department listed 25 projects over $500,000 proposed or underway, with the majority being IT projects. The Department has delegated project approval and expenditure authorities for projects less than $5M. Projects over $5M require a formal submission to Treasury Board to assess project complexity and risk levels, based on which a project may be subject to Treasury Board approval processes.

The Department has identified these projects as representing a medium risk when assessing all costing activities carried out by ISED, including Treasury Board submissions, memoranda to cabinet, budget proposals, service fees, funding requests from Deputy Minister reserve (departmental and sector-specific), memoranda of understandings (internal and external), departmental reallocation exercises, and governed projects, such as the ones subject to the present engagement. This risk assessment was performed based on a combination of factors, including volume and materiality (both 5th largest grouping out of the 10 costing areas as expressed in volume and dollar value), as well as complexity, availability of guidelines and who is leading the costing.

The Department uses a Stage-Gate Framework to guide and govern the project-development lifecycle, which allows for regular reviews and approvals of the projects, and ensures the monitoring of costs within the predefined constraints while achieving the planned outcomes. Depending on the total estimated cost, a project is subject to approval by: the Resource and People Management Committee (RPMC), which authorizes projects to move from the idea stage to the planning stage; the Investment and Experimentation Oversight Committee (IEOC), which oversees projects over $500,000 from planning to completion stages, and authorizes project change requests related to scope, schedule or cost; and the Project Steering Committees (PSCs), which review submissions to RPMC and IEOC prior to approval for all projects, and approve Stage-Gates for all projects and authorizes project gating for projects with total costs less than or equal to $500K from project execution to project completion phases. The Digital Services Oversight Committee (DSOC) also governs all digital projects, and provides strategic advice and recommendations to ensure that ISED's digital improvement initiatives are coordinated and departmental services leverage digital technologies.

As illustrated in Figure 1, the Stage-Gate process is a set of five unique stages. Each stage is comprised of a process and set of deliverables, which must be met in order to ensure approval at the associated gate and to progress to the next stage.

Figure 2: ISED Project Management Stage-Gate Framework

Text version

This diagram provides an overview of the project management framework at Innovation, Science and Economic Development Canada (ISED) and the questions to consider throughout the project management process. The Project Stage-Gate Process consists of 5 unique Stages: Idea Generation (Stage 1), Concept Initiation (Stage 2), Project Planning (Stage 3), Project Execution (Stage 4), and Project Close-out (Stage 5). Each Stage is comprised of a process and set of deliverables, which must be met in order to ensure approval at the associated Gate and progress to the next Stage. The project is proposed at Stages 1 and 2. Rough order of magnitudes are provided for project cost and schedule in Stage 1, and the project formally begins once Stage 2 is approved. From Stages 3 to 5, the project is considered "active". If the cost or schedule estimate changes by a variance of more than 50% during Stage 3, or by a variance of more than 20% during Stages 4 and 5, a formal change request must be approved by a governance committee. The project formally ends once Stage 5 is approved, at which point it is considered "closed". A post project review is conducted after Stage 5. The following are questions to consider for each Stage-Gate:

For Stage-Gate 1:

• Is this project a priority?

For Stage-Gate 2:

• What is the rationale for undertaking the project?

For Stage-Gate 3:

• How do we plan to manage the project?

For Stage-Gate 4:

• What has been delivered?
• What remains to be completed?

For Stage-Gate 5:

• What are the lessons learned and how can we improve?

For Post-Project Review:

• Have the expected benefits been realized?

Depending on the total estimated cost, a project is subject to approval by various governance bodies at each Stage of the Stage-Gate Process.

• For projects with a cost greater than $500,000, Stages 1, 2 and the Post Project Review must be approved by RPMC. Stages 3, 4 and 5 must be approved by IEOC.
• For projects with a cost lower than $500,000, Stages 1 and 2 must be approved by RPMC. Stages 3, 4, 5 and the Post Project Review must be approved by PSC.

According to ISED's Stage-Gate Framework, a project officially begins at Stage-Gate 2 Approval and ends at Stage-Gate 5 Approval, or when the project is prematurely terminated due to its objectives not being met, met before the planned schedule (mainly for Agile projects), or the need for the project no longer exists.

All project management information, including cost information, is centralized in a system called the Departmental Project Portfolio Management System (DPPM) that is used for governance reporting to assist oversight bodies in the decision-making process, and for departmental reporting to Treasury Board Secretariat.

Key stakeholders within ISED

Corporate Management Sector

CMS provides secretariat support to the RPMC and IEOC, and is responsible for conducting the review of project documentation submitted to the RPMC and IEOC through the Departmental Project Management Office (DPMO), to ensure compliance with government and departmental legislative and policy requirements. CMS is also responsible for the policy development and sector coordination efforts to implement the requirements of the TBS Guide to Cost Estimating.

The DPMO, which resides within CMS since April 2020 (formerly in DTSS), supports the umbrella of project and portfolio management functions, including the Stage-Gate governance process for the department.

The DPMO acts at the departmental level for all projects subject to the Stage-Gate process. By doing so, the DPMO plays a pivotal role as a central resource for project managers by providing standardized processes, practices and tools across the Department and oversight for project and portfolio management functions, including a centralized oversight and reporting mechanism for project costing activities.

Sectors

Sectors are accountable for developing and planning cost estimates, securing funding for their projects, managing, monitoring and reporting on costs. Sectors leverage DPMO's expertise and standardized tools and processes to manage their projects, and must follow the Project Stage-Gate Process to report on progress and obtain approval at each stage of the project, including approvals for significant changes in cost estimates.

The Project Executive Sponsor is the Senior Executive of the sector responsible for funding the project. This function is usually exercised by the Assistant Deputy Minister (ADM) of the sector responsible for the project. If multiple sectors are involved, multiple ADMs will be considered joint executive sponsors. The Project Sponsor represents the stakeholders who will benefit from the project deliverables/objectives. The Sponsor is responsible for the projects costs and plays an important role in defining business requirements and in ensuring that the delivered product meets the needs of the business.

Projects are assigned to a Project Steering Committee (PSC), normally chaired by the Project Sponsor. The purpose of the PSC is to provide strategic guidance and advice on project business issues, including cost changes and budget authorization, prior to the project being submitted for departmental governance approval. Depending on the total cost of the project, the role of the PSC may range from a review governance body to an approval authority.

The majority of projects subject to the Stage-Gate process are IT projects, which are managed by the sector in collaboration with DTSS.

2.2 Previous audit engagements

While multiple audits have been conducted on specific projects or project portfolios, no previous audit engagement has been performed by the Audit and Evaluation Branch on project costing activities at ISED.

3.0 About the audit

3.1 Audit objective, scope and methodology

In accordance with the approved Innovation, Science and Economic Development (ISED) 2019-2020 Risk-Based Internal Audit Plan, the Audit and Evaluation Branch (AEB) undertook an audit of Project Costing.

Audit objective

The objective of the audit was to provide assurance that costing activities related to projects are in place, used consistently, and are operating effectively.

Audit scope

The audit scope included 10 projects subject to costing activities as defined by ISED's Stage-Gate Framework, and covers the period of April 1, 2018 to September 30, 2019.

The scope included an assessment of:

• Governance and oversight;
• Internal controls;
• Effectiveness of processes and practices.

Methodology

The audit was conducted in accordance with the Internal Auditing Standards for the Government of Canada.

Based on the risk assessment, audit criteria and sub-criteria, linked to the overall audit objective, were developed (see Appendix A).

The methodology used for this audit included performing various procedures to address the audit's objective, including a review of documentation, interviews, and walkthroughs.

In addition, a sample of ten projects was used to assess costing methodologies used, approval processes, as well as cost monitoring and reporting processes. At the time of the project selection, the value of these projects varied between $219,000 and $48,148,844, with the majority of projects being less than $25M. Factors for selecting the sample included project dollar value, coverage from a variety of sectors, and project health status related to planned costs and project schedule.

A debrief meeting was held with the Chief Financial Officer, the Director General, Resource Planning and Investments Branch and the Acting Director General, Corporate Planning and Governance Branch on January 27 2021 to validate the findings that form the basis of this report. This meeting also provided the auditee an opportunity to offer any additional information and clarification regarding the findings.

4.0 Findings and recommendations

4.1 Introduction

This section presents detailed findings from the audit of Project Costing. The findings are based on evidence and analysis from both the initial risk assessment and the detailed audit work.

4.2 Costing process

4.2.1 Cost estimate development process

The use of a formal costing process is essential to ensure appropriate and consistent costing practices across all projects, as well as completeness and accuracy of cost estimates. Based on Treasury Board Secretariat's Guide to Cost Estimating, projects are to have documented information regarding their costing approach, including the involvement of key stakeholders in the process, and details on the costing methodology used, assumptions, risks and related contingencies.

For IT projects, cost estimates are developed primarily by the IT Project Lead with input from the Project Manager. For non-IT projects, the Project Managers are responsible for determining cost estimates. DPMO requires Project Managers to enter a high-level breakdown of the total cost in the DPPM system but does not require them to provide details on the costing methodology, such as assumptions and cost components.

For five projects tested, evidence was found on cost components used to calculate the project costs, including spreadsheets documenting the cost categories used, such as salaries, consultant fees, and IT application costs. These projects also included a breakdown of costs by project deliverables or sector-specific costs versus IT costs incurred by the Chief Information Officer branch. However, for five projects tested, no evidence was found on cost components other than the cost plans recorded in the DPPM system, which provide a high-level breakdown of costs, such as totals for IT costs and non-IT recoverable costs. For all projects tested, there was insufficient evidence regarding costing assumptions, risks and contingencies used in the process. Some interviewees noted that the costing approach used was based on experience with similar projects and that relevant project stakeholders were included in cost discussions, although no related documentation was found regarding this approach. Turnover in Project Manager and/or IT Project Lead was also noted over the course of six projects tested, where interviewees could not provide sufficient documentation on the costing technique used.

Documenting the costing approach is essential to ensure that project decisions related to costs are made based on a sound information and to effectively manage turnover in project team members while ensure continuity of financial decisions as projects progress from one stage to the next via the Stage-Gate process.

Recommendation #1 (medium risk)

The Department should ensure that a formal project costing approach is used, and documented, including consideration of key stakeholders' input, costing assumptions, risks, and appropriate contingencies.

4.2.2. Cost approval

Project cost estimates are approved via the Stage-Gate process, as costs are included in documents used for approving a project to move on to the next stage. The Stage-Gate process is clearly documented, including roles and responsibilities. DPMO oversees the Stage-Gate process and requires specific documents to be provided such as a business case and project plan.

Project Managers are responsible for developing the required documents, which include cost estimates for governance approval, and for providing a high-level breakdown of cost estimates in the form of a Cost Plan in the Departmental Project Portfolio Management System (DPPM). Cost Plans are validated by DPMO and are kept in DPPM as records supporting the governance approval of costs. Cost Plans do not include information on the costing methodology used, as this level of detail is not required for the approval of cost estimates.

Initial cost estimates are presented for approval in stages 1 to 3, as the project scope and deliverables are refined and cost estimates become more specific. Documents are first approved by Project Steering Committees prior to approval by the following governance committees, based on the monetary value of the project (i.e. the tier category):

Figure 3. Stage-Gate governance approval for stages 1 to 3

In stage 1, a high-level cost estimate (Rough Order of Magnitude) must be presented for approval. In stage 2, a business case with the cost estimate is to be provided for all types of projects, and a Cost Plan is entered and validated by DPMO in DPPM. In stage 3, a Project Plan with a description of project scope, timeframe, risks, total cost and source of funding is to be provided, as well as a project dashboard showing actuals versus planned cost. A revised Cost Plan for Stage 3 is also entered and validated by DPMO in DPPM. At all stages, DPMO reviews the documents for completeness prior to approval by governance committees.

For all projects tested, it was found that the initial cost estimates were consistently approved via the Stage-Gate process. Documents containing cost information, such as business proposals and business cases, were validated by project authorities prior to approval by oversight committees. Cost Plans found in DPPM for the projects tested included high-level breakdowns of costs, such as IT vs non-IT recoverable, salaries, professional services, and other related costs.

All final versions of the documents were kept in the departmental records management system (GCDOCS) or in DPPM.

Lastly, Stage-Gate approval decisions, including the cost approved, were consistently recorded in DPPM and in the governance committees' meeting records of decisions.

4.3 Costing tools, guidelines and support

Cost development

The Treasury Board Directive on the Management of Projects and Programmes requires the implementation of processes that promote the coordinated management of related projects to reduce risk, contribute to shared outcomes, and realize efficiencies and benefits not available from managing the projects individually. To that effect, in order to ensure consistent costing practices across similar projects and a sound foundation for assessing cost estimates, it would be expected that departmental guidelines and tools, aligned with the Treasury Board Secretariat's Guide to Cost Estimating, are in place and applied consistently to assist project managers in developing cost estimates, and that support would be available on the use of these tools.

It was found that project managers typically use their own tools, such as custom spreadsheets, to develop cost estimates, as no departmental tools and guidelines exist for project costing. The spreadsheets at times included details such as per-diem rates and software costs, but generally provide insufficient information on the costing methodology used to gain a thorough understanding of how the estimate was produced. Without the use of departmental tools and guidelines for costing, it is difficult to ascertain the credibility of the cost estimate in support of effective decision-making.

Additionally, although DPMO reviews all documentation submitted at each gate and CMS ensures that TBS accounting standards are met, it was found that no support is provided by DPMO nor CMS financial specialists regarding the cost development process, approach and methodology used.

TBS' Guide to Cost Estimating suggests that the selection of cost-estimating techniques to use should be made in consultation with the Chief Financial Officer's organization. Hence, there is an opportunity for CMS financial specialists to provide their expertise in challenging the appropriateness of the costing technique and assumptions used, and provide suggestions to improve the estimate.

Cost approval

Cost estimates are approved when a project moves to the next stage of the Stage-Gate process. At each stage, high-level cost information, such as cost estimates and project spending to-date, is included in documents presented to governance bodies for gate approval. Departmental guidelines and templates are developed and communicated by DPMO, and are used consistently by project managers for the approval of cost estimates via the Stage-Gate process. DPMO provides support and guidance to project managers and sponsors on the Stage-Gate process by reviewing the readiness of projects that are proceeding for a Gate or a Change Request approval and ensuring that all documentation is complete to be presented to the appropriate governance committee. DPMO also provides training and support related to the departmental tools and templates used in the Stage-Gate process, including documents and the use of the DPPM system.

When seeking approval, project managers are required by DPMO to develop a Cost Plan in the DPPM system, which includes a high-level breakdown of total project cost (e.g., IT and non-IT recoverable costs, salaries, professional services). Testing demonstrated that Cost Plans have been entered mostly consistently in DPPM and approved by DPMO after governance approval.

Additionally, for IT projects, IT project leads provide support to project managers when requesting approval of costs from the Project Steering Committee prior to Stage-Gate submission. However, although DPMO reviews documents for completeness of project information, it was found that DPMO and CMS financial specialists are not involved in the process for validating cost estimates prior to approval for completeness and accuracy.

Recommendation #2 (medium risk)

The Department should develop departmental guidelines and tools based on the Treasury Board Guide to Cost Estimating to ensure consistency and transparency of the approach and methodology used for developing cost estimates.

Recommendation #3 (low tisk)

The Department should provide support on the use of tools related to the cost development process to ensure rigor and consistency in applying costing approaches across the Department.

4.4 Cost monitoring and reporting

As per the Treasury Board Directive on the Management of Projects and Programmes, processes for project monitoring, including performance measurement tracking, must be in place to ensure effective decision-making, accountability, and transparency. Regular monitoring and reporting of costs provide early insights and support proactive resolution of potential financial issues.

Costs are monitored at the departmental level via the Stage-Gate process. It was found that for all projects tested, cost information is included in the documents provided to governance bodies when a project moves through each stage of the lifecycle, or significant changes to costs are anticipated during a stage. Cost information includes cost estimates and a recent project dashboard, which allows oversight committees to review the health of a project from a financial perspective at each stage of the project or when changes are expected.

The DPPM system is used for reporting on costs as part of the Stage-Gate process. As per DPMO guidelines, project managers are required to provide project status information in DPPM to describe progress to-date on deliverables and explain cost variances, if any. DPMO uses this information to generate monthly dashboard reports in DPPM that include cost information, reviews the costs to ensure it aligns with the documents and provides the dashboard report to senior governance bodies for Stage-Gate or change request approvals.

Project managers (and IT Project Leads for IT projects) also monitor costs internally using custom spreadsheets, which include details on project expenditures to-date. Cost information is reported to Project Steering Committees and Sponsors as part of regularly-scheduled meetings on project status updates, during which an overview of spending to-date is provided and compared to cost estimates to identify potential opportunities to reallocate resources or the need to proceed with a change request approval via the Stage-Gate process.

4.5 Management of cost changes

As per the Stage-Gate process, when deviations to the project cost are anticipated, a project may be required to submit a change request for approval by key oversight committees depending on the project stage and the significance of the cost variance. For projects at stage 2, change requests are approved by RPMC if the cost estimate changes by +/- 100%. For projects at stage 3, change requests are approved by IEOC (for Tier 1 and 2 projects) or the Project Steering Committee within the Sector (for Tier 3 projects) if the cost estimate changes by +/-50%. For projects at stages 4 and 5, change requests are approved by IEOC (for Tier 1 and 2 projects) or the Project Steering Committee within the Sector (for Tier 3 projects) if the cost estimate changes by +/-20%.

Figure 4. Stage-Gate governance for change requests

Project managers are responsible for ensuring that cost changes exceeding thresholds are approved via the Stage-Gate process. DPMO also monitors projects to identify potential upcoming change requests by using the monthly report status and project dashboards in DPPM system, which provide cost variance information.

It was found that the change request approval process was communicated clearly on the DPMO website and applied consistently, as per the cost change thresholds defined in the Stage-Gate process. For the majority of projects, roles and responsibilities for approving cost change requests were clearly defined and understood. Change request approvals, including reset of the cost baseline, were recorded in DPPM and committee records of decisions, as required by the process defined by DPMO to that effect.

DPMO provides support to project managers on the departmental template for submitting change requests (change request précis). For all change requests tested, It was found that template was used consistently and provides sufficient information for effective decision-making, including: the cost baseline approved, proposed cost estimate, and history of previous change requests; the description and justification for the cost change; risks and benefits to the project outcomes; and the most recent project status dashboard, which includes cost variance information on planned vs. actual costs to-date. Depending on factors such as the size and stage of the project, a presentation on the project status and deliverables is occasionally provided.

Overall, it was found that the Stage-Gate process helps to refine cost estimates throughout the project life cycle, as the process requires the review of project costs at each stage of the project by governance bodies, and requires the approval and justification for cost changes if costs exceed specific thresholds defined in the Stage-Gate process.

Five of the projects included in the sample have been completed since the end of this audit engagement. While cost estimates changed during the audit period, it was found that for four out of these five projects, final costs remained within 1% of their latest approved cost estimate.

4.6 Management response and action plan

The findings and recommendations of this audit were presented to the Chief Financial Officer, the Director General, Resource Planning and Investments Branch and the Acting Director General, Corporate Planning and Governance Branch. Management has agreed with the findings included in this report and will take action to address all recommendations by March 31, 2022.

5.0 Overall conclusion

Effective governance processes are in place for the approval of cost estimates and cost changes throughout the project lifecycle, via the departmental Stage-Gate process. However, there are opportunities to strengthen controls related to the development of project cost estimates to ensure a rigorous and consistent approach across the department, and a sound foundation from which costs can be evaluated.

Appendix A: Audit criteria