Audit of the Canadian Intellectual Property Office IT Modernization Program

Audit and Evaluation Branch

November 2017

Recommended for Approval to the Deputy Minister by the Departmental Audit Committee on November 23-24, 2017

Approved by the Deputy Minister on January 8, 2018

Table of contents

• 1.0 Executive Summary
  • 1.1 Introduction
  • 1.2 Audit Background
  • 1.3 Overview of Audit Results
  • 1.4 Audit Opinion and Conclusion
  • 1.5 Management Response
  • 1.6 Statement of Conformance
• 2.0 Background
  • 2.1 Program Overview
  • 2.2 Previous Review Work
  • 2.3 Audit Objective, Scope and Methodology
• 3.0 Findings and Recommendations
  • 3.1 Introduction
  • 3.2 Governance
  • 3.3 Risk Management
  • 3.4 Internal Controls
  • 3.5 Management Response and Action Plan
• 4.0 Overall Conclusion
• Appendix A: List of ITM Projects
• Appendix B: Audit Criteria
• Appendix C: CIPO ITM Governance Structure
• Appendix D: ISED Stage-Gate Framework

List of acronyms used in report

AEB

Audit and Evaluation Branch

CEO

Chief Executive Officer

CIO

Chief Information Officer

CRWG

Change Request Working Group

CWR

Change Work Request

DPMO

Departmental Project Management Office

DPPM

Departmental Project and Program Management

D-SRO

Deputy Senior Responsible Owner

ECMWG

Enterprise Content Management Working Group

ESMO

Enterprise Services Management Office

FTE

Full Time Equivalent

IB

Investment Board

IOC

Investment Oversight Committee

IP

Intellectual Property

IPO

Intellectual property offices

ISED

Innovation, Science and Economic Development Canada

ITM

Information Technology Modernization

PMF

Project Management Framework

PMO

Program Management Organization

POWG

Program Office Working Group

PPRIMC

Project Risk & Issue Management Committee

RFC

Request for Change

SEC-PSC

CIPO Senior Executive Committee-Programs Steering Committee

SG

Stage-Gate

SLA

Service Level Agreement

SSC

Shared Services Canada

SOA

Special Operating Agencies

SRO

Senior Responsible Owner

1.0 Executive summary

Introduction

The Canadian Intellectual Property Office (CIPO) is a Special Operating Agency of Innovation, Science and Economic Development Canada (ISED) responsible for the administration of Canada's system of intellectual property (IP). This includes the granting of patents and the registration of trademarks, copyrights and industrial designs.

In order to update its aging and costly legacy IT systems, CIPO initiated in 2013 the IT Modernization (ITM) Program. The ITM Program aims to:

• Improve the online user experience;
• Simplify the application process;
• Provide enhanced access to IP information and records;
• Provide the ability to respond to changes in the legislative and global environment; and
• Improve operational efficiencies and reduce costs.

At the time of the audit, the ITM Program included 28 projects grouped as Client-Facing, Back-Office Services, or Foundational Projects, with a budget of $102.9 million. In January 2017, CIPO began a reprioritization exercise in order to meet international IP treaty obligations, which resulted in a reduction in the number of ITM projects from 28 to 25, and an extension of the Program lifecycle by two years (to FY2022-2023).

Audit Background

The objective of the audit was to provide assurance that an ITM Program management framework has been designed and implemented to support current and future IT projects.

The audit scope focused on activities and processes of the management control framework as it relates to CIPO's ITM Program from April 1, 2015 to September 30, 2016, including:

• Governance
• Risk Management
• Internal Controls

Overview Of Audit Results

Strengths

CIPO has established a governance framework to manage and monitor projects within the ITM Program, where an oversight structure is in place at the project level, and roles and responsibilities are defined and documented.

A risk management strategy has been defined and approved, and processes and tools to manage risks have been developed and implemented.  Further, a Program-level risk management committee has been established to identify, manage and control projects and project risks and issues that may have the potential to impact the Program.

At the departmental level, the Stage-Gate Process is used by the CIPO ITM Program to guide and govern the project development lifecycle, including change management, where control procedures are consistent with the department's Departmental Project and Portfolio Management (DPPM) tool.

Consideration of capacity and resource availability for project delivery is included in project planning, and is considered throughout both the projects' and the Program's lifecycle.  In addition, oversight committees provide guidance on the approach, priority and sequence of delivery in relation to initiatives and capacity requirements, including resource management.

Areas for Improvement

Some opportunities for improvement were identified by the audit:

• CIPO-Specific Governance: CIPO should identify project interdependencies, and ensure that interdependencies are updated as project risks evolve. 
• Departmental Governance: CIO should strengthen the Stage-Gate Framework to ensure that Program-level information is provided to departmental oversight committees, including accurate, up-to-date overall Program health, budget, priorities, milestones, and project interdependencies. Further, CIO should complete the planned enterprise architecture to encourage alignment of projects across the department.
• Risk Management: CIPO should strengthen its risk management processes by ensuring that all risks related to the overall ITM Program are identified, mitigated, and reported, and that a common risk universe is used across all projects.
• Internal Controls: CIO should update their change request guidance to reflect requirements for projects in Stage 2; defined thresholds for scope and schedule changes; and requirements to present change request histories to oversight committees.

Audit Opinion and Conclusion

The results of the audit revealed that an ITM Program management framework has been designed and is being implemented to support current and future IT projects. There are opportunities to strengthen the governance, risk management and change management of the overall Program, helping to ensure both that projects are well-managed individually, and also as an interconnected and coordinated program with collective outcomes.

Management Response

Management has agreed with the findings included in this report and will take action to address all recommendations by August 31, 2018.

Statment of Conformance

This audit was conducted in accordance with the Internal Auditing Standards for the Government of Canada, as supported by the results of the Audit and Evaluation Branch's quality assurance and improvement program.

Michelle Gravelle
Chief Audit Executive, Innovation, Science and Economic Development Canada

---

2.0 Background

2.1 Program Overview

Entity Background

The Canadian Intellectual Property Office (CIPO) is led by a Chief Executive Officer (CEO) who serves as the Commissioner of Patents. CIPO's CEO reports directly to the Deputy Minister of ISED and is responsible for CIPO's performance and strategic direction.

CIPO's areas of activity include:

• Patents, to protect new inventions  or any new improvement of an existing invention;
• Trademarks, to identify the goods or services of one person or organization and to distinguish these goods or services from those of others in the marketplace;
• Copyrights, to provide protection for artistic, dramatic, musical or literary works;
• Industrial designs, which are applied to a finished article of manufacture; and
• Dissemination of IP information to Canadians.

As a Special Operating Agency^{Footnote 1}, CIPO operates on a cost-recovery basis based on revenue received from applications and maintenance fees. Those fees, approved by Parliament, have remained largely unchanged since 2004.

For FY2016-17, CIPO's revenues were $158.3M, with expenditures of $151.8M. As of March 31, 2016, the organization had approximately 959 full-time equivalent (FTE) employees. 

Mission and Strategic Priorities

CIPO's mission is to contribute to Canada's innovation and economic success by:

• providing greater certainty in the marketplace through high-quality and timely IP rights;
• fostering and supporting invention and creativity through knowledge sharing;
• raising awareness to encourage innovators to better exploit IP;
• helping business compete globally through international cooperation and the promotion of Canada's IP interests; and,
• administering Canada's IP system and office efficiently and effectively.

CIPO's Business Strategy aims to strengthen the role of IP to support Canada's innovative capacity and output. The Strategy is comprised of three strategic pillars that include customers, access to innovative knowledge, and a modern IP framework. The enabling pillars include a highly-skilled workplace, a responsive infrastructure, and operational excellence.

IT Modernization Program

Intellectual property rights have become increasingly more central to the global business strategies of successful firms. To manage the growing demand for the protection of ideas, and recognizing that timely IP decisions help innovators quickly develop, monetize and globally commercialize their ideas, measures such as international work sharing and harmonization initiatives are being implemented across intellectual property offices (IPOs) globally. Efforts are underway in many IPOs to increase the functionality of their information technology services to respond more efficiently to modern business needs.

In 2013, CIPO initiated the IT Modernization (ITM) Program in order to update its aging and costly legacy IT systems. In December 2013, the CEO and Senior Executive team approved the ITM Program Brief, which defined, at a high level, the scope, objectives and outcomes of the Program. CIPO was required to plan, structure and oversee the implementation of the CIPO Portfolio of ITM Program projects throughout the expected seven- to ten-year span of the Program. A Program Management Organization (PMO) was established with dedicated resources led by a program manager who is directly accountable to the Director General of the CIPO Programs Branch.

The ITM Program aims to:

• Improve the online user experience;
• Simplify the application process;
• Provide enhanced access to IP information and records;
• Provide the ability to respond to changes in the legislative and global environment; and
• Improve operational efficiencies and reduce costs.

At the time of the audit, the ITM Program included 28 projects grouped as Client-Facing, Back-Office Services, or Foundational Projects with a budget of $102.9 million. The full list of projects along with their respective stage status as of December 30, 2016 can be found in Appendix A.

Figure 1: ITM Program, Projects by Type

In January 2017, CIPO began a reprioritization exercise in order to meet international IP treaty obligations by early 2019. The key principles of this exercise were to protect legislative delivery timelines, ensure continued business operations, and consider departmental capacity limitations. The expected outcomes are to eliminate potential ITM impacts on CIPO legislative projects, better align ITM projects with available resource capacity, ensure continued delivery of client service improvements, and ensure that capacity exists to continue to deliver the Program. This exercise resulted in a reduction in the number of ITM projects from 28 to 25, and an extension of the Program lifecycle by two years (to FY2022-2023).

Key Players in Program Delivery

There are three key players in CIPO ITM's program delivery model: CIPO's Programs Branch, ISED's Chief Information Officer (CIO) Branch, and Shared Services Canada (SSC).

CIPO's Programs Branch

The Programs Branch is responsible for coordinating the delivery of the IT Modernization Program and its subset of projects. The Branch is comprised of CIPO employees and consultants working together under the following six groups:

• The Project Management Office (PMO);
• Business Architecture;
• Legislative Implementation Group, which is responsible for the project management of CIPO's legislative projects;
• Enterprise Services Management Office (ESMO);
• Business Analysis Centre of Excellence; and
• Stakeholder Engagement and Change Management Office.

The Programs Branch works closely with the CIPO Lines of Business group (i.e. Patents, Trademarks, Copyrights and Industrial Design, Business Services and Corporate Services) which provides subject matter expertise to help define the CIPO ITM projects.

CIO Branch

With respect to CIPO ITM, the CIO Branch is responsible for providing technical expertise and resources for the integration of new technologies to replace the existing legacy systems within CIPO, including supporting the planning and scoping of all ITM projects. The Branch also owns and administers the Stage-Gate Framework through the Departmental Project Management Office (DPMO), and provides recommendations on enhancement requests and on project gating materials in preparation for oversight committee approval.

SSC

Shared Services Canada (SSC) is responsible for providing the underlying IT infrastructure, including all servers and networking infrastructure, as well as the supporting services required to function and maintain the operating environment. SSC is also responsible for the acquisition of supporting IT infrastructure for the ITM Program, under direction from the CIO.

2.2 Previous Review Work

A preliminary survey was conducted in 2010-11, with the objective of developing an appropriate level of knowledge of CIPO's Enterprise Business Renewal (EBR) initiative to enable the planning of effective, efficient and value-added audit engagements. The survey resulted in three recommendations in the areas of governance, project and change management, and risk management.

At the request of CIPO's CEO, a review of CIPO's ITM Program was conducted in 2014 by AEB with the objective of determining whether the findings identified in the 2010-11 preliminary survey had been addressed, and whether an effective management control framework had been designed and implemented to support future IT projects. The 2014 review resulted in four recommendations in the areas of documentation provided to the Project Steering Committee (PSC), project planning, consolidation of documentation related to the Program's framework and strategy, and assessment of the Program's ability to support  the achievement of established outcomes. All recommendations were closed prior to the commencement of the current audit engagement.  

2.3 Audit Objective, Scope and Methodology

In accordance with the approved Innovation, Science and Economic Development (ISED) 2016-2018 Multi-Year Risk-Based Internal Audit Plan, the Audit and Evaluation Branch (AEB) undertook an audit of Canadian Intellectual Property Office (CIPO) IT Modernization (ITM) Program.

The objective of the audit was to provide assurance that an ITM Program management framework has been designed and implemented to support current and future IT projects.

Audit Scope

The audit scope focused on activities and processes of the management control framework as they relate to CIPO's ITM Program, from April 1, 2015 to September 30, 2016, including:

• Governance
• Risk Management
• Change Management

The Service Level Agreement (SLA) between ISED and Shared Services Canada (SSC) was excluded from the scope of this audit, as the SLA broadly covers all departmental services to be provided by SSC, and is not specific to the ITM Program.

Methodology

The methodology used for this audit included:

• Review of CIPO ITM Program documentation and processes;
• Conduct of 37 interviews with CIPO, CIO and SSC personnel and management;
• Sample testing of eight ITM projects to assess the compliance with the Program management control framework.

Based on the identified risks, AEB developed the audit criteria and sub-criteria linked to the overall audit objective (see Appendix B).

A debrief meeting was held with the CIPO Director General, Programs Branch on August 14, 2017, and with the Director General, CIO Branch on August 15, 2017 to validate the findings that form the basis of this report. This meeting also provided the auditee an opportunity to offer any additional information and clarification regarding the findings.

3.0 Findings and Recommendations

3.1 Introduction

This section presents detailed findings from the audit of the CIPO IT Modernization Program.  The findings are based on evidence and analysis from both the initial risk assessment and the detailed audit work. In addition to the findings below, AEB has communicated to management findings that were non-systemic or of low materiality or risk.

3.2 Governance

CIPO-Specific Governance

CIPO has established a governance structure for the CIPO ITM Program where roles and responsibilities are defined, and decision-making mechanisms are in place, including two key senior level committees: the Senior Executive Committee-Programs Steering Committee (SEC-PSC) and the IM/IT Committee (see Appendix C).

The SEC-PSC, chaired by the CEO, approves all projects at each phase and Program changes, and the IM/IT Committee recommends projects to the SEC-PSC and approves change work requests. These committees are supported by four working groups: the Change Review Working Group, the Enterprise IT Working Group, the Enterprise Content Management Working Group and the Program Office Working Group (POWG). Terms of reference have been defined and documented for all working groups, and decisions are documented.

CIPO has developed project management processes and policies to manage and monitor ITM projects, including the CIPO Project Management Framework (PMF), which is comprised of a set of tools, techniques and templates to execute the activities and to produce the deliverables of a project. Project reviews are used by the Program to verify the health status of a project, and can be used to support changes to plans at the Initiation and Planning stages. Dashboards are presented to the IM/IT Committee Meeting on a monthly basis, and action items are raised and documented. Updates are provided on a monthly basis to the Business Transformation Steering Committee on the Program's status, and before each project is subject to the Stage-Gate process, it is discussed at the monthly Change Review Working Group.

However, key project interdependencies that may affect Program and project delivery have not been identified. A draft roadmap demonstrating relationships between ITM projects was developed in 2015 but did not provide information on how the identified project relationships will impact the projects concerned, or how these relationships will be managed. As of 2016, the CIO Branch had begun defining ISED's enterprise architecture, and CIPO's management committed to aligning the ITM projects to the architecture. At the time of the audit, however, the enterprise architecture had not been finalized.

Without identified interdependencies between ITM projects, as well as between ISED projects and other CIPO legislative initiatives, it will be difficult to manage the impact of these interdependencies on the Program and project delivery. Further, not identifying how they are aligned to the enterprise architecture could lead to uninformed decision-making.

Departmental Governance

The ISED Stage-Gate Framework is used to manage projects from idea to completion, and prescribes the deliverables to be completed in order to proceed to subsequent gates (see Appendix D).  This framework guides and governs the project-development lifecycle within ISED, and is consistent with TBS policies and best practices, including practices from the Project Management Book of Knowledge (PMBOK® Guide published by the Project Management Institute).

Two committees provide oversight on ITM projects through the Stage-Gate process: the Investment Board and the Investment Oversight Committee. The Investment Board (IB) is a senior committee mandated to provide strategic direction and oversight over ISED's investment portfolio, including projects, and approves projects to move from Idea Generation (Stage 1) to Concept Initiation (Stage 2) and to Project Planning (Stage 3). The Investment Oversight Committee (IOC) is responsible for providing assurance that proposals for large investments represent a sound use of resources for the Department, and authorizes projects to move from Project Planning (Stage 3) to Project Execution (Stage 4) to Project Close-out (Stage 5).

The Departmental Project Management Office (DPMO) establishes, maintains and monitors the management of projects within the Department. The DPMO functionally resides in the CIO but acts at the Departmental level for all projects. The DPMO is the gatekeeper of the Stage-Gate process to guide and govern the project-development lifecycle. The DPMO ensures that the required project management documentation is in place using the Departmental Project Portfolio Management (DPPM) online system, which is used to document, collect and report project information subject to the Stage-Gate Framework. It enables the centralization of project management information, as well as the recording of project management information across all sectors, which is then used to assist oversight bodies in the decision-making process.

Eight of the 28 projects were sampled to assess compliance with Treasury Board and departmental policies and practices. For all projects sampled, it was found that project reports were provided to governing bodies, that decisions to move forward were based on prescribed documentation reviewed by the governing bodies, and that all documents required at each Stage were completed and kept on file. It was also found that projects had been managed in accordance with the policies in place.

However, given that the Stage-Gate process has been developed specifically for projects, there is no requirement for CIPO to bring standardized Program-level information to departmental oversight committees (although over the past year, CIPO has presented the status of its project portfolio and its reprioritization exercise to various oversight committees). Further, it was found that project health information, including information related to budget, scope, and schedule, provided to the oversight committees was not always clear. The information was not presented in a consistent manner from one governance meeting to another, and change request histories were not always reflected in the status reports. Consequently, the ITM Program cannot leverage the Framework at the Program level, as there is currently no requirement for the ITM Program to obtain departmental approvals of program milestones and subsequent changes to it, such as program budget increases.

Providing incomplete information to oversight bodies could lead to uninformed decision-making, and complicate assessments of the Program's benefits against the total cost of the Program. In addition, if budget and scope information is not consistently capturing accurate information at the project level, the lack of clarity could compound for decision-makers when assessing progress against the achievement of the overall objectives of the Program.

3.3 Risk Management

CIPO's approach to risk management is established in its ITM Program Risk and Issue Management Strategy, which has been approved and communicated to the Program. This strategy defines the approach for how ITM project risks and issues are managed within the Program, including the frequency at which risks and issues will be reviewed, who will be accountable for performing the various risk and issue management roles, and the methods and controls to be used throughout the ITM Program's lifecycle. Further, the strategy includes common risk categories, including flexibility, implementation, interdependencies, organizational culture, resource availability, supportability, and technical.

The Program and Project Risk & Issue Management Committee (PPRIMC) is in place with defined terms of reference. Its mandate is to act as a forum to identify, manage and control program and project risks and issues that may have the potential to impact the Program. The committee meets on a monthly basis and records of decisions of meetings are documented.

Risks are identified by project teams over the course of the project and documented in the Departmental Project Portfolio Management (DPPM) online system by project managers. The risks are then categorized by project by the ITM Program Risk Manager, who is responsible for maintaining a risk register for the entire Program. These risks are discussed at the monthly PPRIMC meetings, and once formalized, are included in the respective project dashboards for monitoring and reporting purposes.

For all projects sampled, it was found that project documentation related to risks were prepared and communicated to stakeholders. The DPPM application and ISEDWiki demonstrated that risks and issues registers were developed for each project sampled.

Individual monthly dashboards are developed for each project and contain an executive summary, project risks and mitigation actions identified by the Program, along with a heat map containing each identified risk. Each project dashboard also contains information on the project schedule and budget. Although risk categories are identified in the ITM Program Risk and Issue Management Strategy, these categories were not used by the Program when performing individual project risk assessments. Risk management is therefore mainly project-driven, and there is no common risk universe to help guide the consistent and comprehensive identification of risks across projects.  

Further, risk categories used by the projects did not include reputational, financial, and legislative risks, which could prevent the Program from successfully mitigating issues that may arise in these areas, and potentially impact CIPO's ability to meet international IP treaty obligations on time.

Without a common approach to risk management across the Program, key risks may not be identified and managed, impacting the achievement of CIPO's objectives. Not actively managing these risks could lead to further unforeseen increased costs for the Program and delayed projects.

3.4 Internal Controls

Change Management

CIPO-Specific Change Management Controls

The Monitoring and Control, Quality Assurance Strategy & Plan was developed by the Program in 2014 with the objective of ensuring that all management aspects of the Program are working appropriately and that the Program stays on target to achieve its objectives and deliver the anticipated business capabilities, value and benefits. The plan includes control and monitoring activities at the Program level, control and monitoring activities at the project level, Program change control processes, and project change control processes.

CIPO's Change Review Working Group (CRWG) is responsible for recommending Change Work Requests (CWR) to the IM/IT Committee for approval; informing the IM/IT Committee on any changes to the management, administration and implementation of IT processes, guidelines, procedures and standards; and informing the IM/IT Committee of any risks, issues and urgencies that require executive attention, including recommendations for resolution.

Once the project's formal deliverables have been approved, they can only be changed after approval by the Senior Executive Committee-Programs Steering Committee (SEC-PSC) and/or by written authorization from the Senior Responsible Owner (SRO) / Deputy Senior Responsible Owner (D-SRO). However, the thresholds around submitting change requests defined by the CRWG have only been defined in October 2016.

Departmental Change Management Controls

The use of change control procedures is integrated with the Departmental Project and Portfolio Management (DPPM) system that is utilized for tracking portfolio, program, master projects, major departmental initiatives and project information. The DPPM system is designed to manage risks, issues, and changes at the project level.

The Stage-Gate Framework, administered by the CIO, is used by the CIPO ITM Program to guide and govern the project-development lifecycle. Descriptions of each Stage-Gate Framework Stage and their variance approval levels are as follows:

Stage-Gate Framework Stage	Project Phase	Esimate Variance as per Approved Program Plan
1	Idea Generation	ROM
2	Concept Initiation	±100%
3	Project Planning	±50%
4	Project Execution	±20%
5	Project Close-Out	±20%

The proposal officially becomes recognized as a formal project once it has passed Gate 2, and ends once it has passed Gate 5. As per the Stage-Gate Framework (see Annex D), for all projects past Stage 1 (Idea Generation), a change request is required to seek approval on a project's budget increase that exceeds the pre-defined Stage-Gate Framework variances, as well as for changes to a project schedule or scope.

Changes to a project's original budget within the pre-defined Stage-Gate Framework variances do not require a formal change request. Instead, the budget and/or schedule could be re-baselined and the project would then be evaluated against the new baseline and no history of changes is provided to governance committees.

While the Stage-Gate Framework states that change requests are required for projects past Stage 1, in practice, projects in Stage 2 are not asked to seek formal change request approval. Further, the Departmental Project Management Office (DPMO) has not clearly articulated the definition of significant changes for scope and schedule changes for ITM projects.

Not consistently documenting approved changes could impact decision-makers' ability to gauge progress on projects, and on the Program as a whole. Unforeseen cost variance can significantly impact the Program's ability to deliver on its committed projects within an approved timeframe. Further, not being able to assess the cumulative effects of the project changes could reduce the Program's ability to deliver its longer-term outcomes.

Capacity and Resource Considerations

The Program Resource Management Strategy and Plan was developed by the ITM Program in 2016 with the objective of addressing both business and IT components by outlining how the ITM Program will acquire, use, and manage the resources required to effectively achieve the required business changes. The strategy contains an ITM Program Office Resource Management Strategy and Plan, an ITM Program

Vendor Management Resource Management Strategy and Plan, an ITM Program Business Resource Management Strategy, and an ITM Project Resource Recommendations.

A CIPO ITM Business Resource Estimation Tool (PBRET) is in place and is used to track resource forecasting for each project and for the overall Program. This tool is one of the main elements of the Program's ITM Business Resource Management Strategy and Plan, and was developed by the ITM Program Management Office to provide the CIPO ITM Program and CIPO Lines of Business (LoBs) with a clearer understanding of the scope, effort, and resource demands of the CIPO Programs Branch and CIPO LoBs to support and deliver each IT-enabled project.

The IT Modernization Program Management Governance and Organization document establishes the framework within which the Program will be designed and delivered and  prescribes the guiding principles to oversee and manage planned ITM projects, including effective use of ISED and CIPO resources and monitoring resource availability for planning and delivery purposes. Consistent with this framework, the Program Office Working Group (POWG), along with the Change Review Working Group (CRWG) and the CIO, coordinates and provides guidance on the approach, priority and sequence of delivery in relation to initiatives and capacity requirements, including resource management. Further, the individual project management plans include information about project cost estimates, including those for hiring FTEs and anticipated professional services related spending per stage.

CIPO has recently undertaken a reprioritization exercise within the context of the ITM program delivery to support the commitment of the department to meet international IP treaty obligations by early 2019. While the reprioritization took place outside the scope of this audit, the exercise was intended to ensure the Program continues to have the resources and direction it needs in the Program's changing environment.

3.5 Management Response and Action Plan

The findings and recommendations of this audit were presented to the Commissioner of Patents, Registrar of Trade-marks and Chief Executive Officer of the CIPO, the Director General, Programs Branch of the CIPO, the Assistant Deputy Minister, Digital Transformation Service Sector, and the Chief Information Officer. Management has agreed with the findings included in this report and will take action to address all recommendations by August 31, 2018. Notably, the Commissioner of Patents, Registrar of Trade-marks and Chief Executive Officer and Director General of the CIPO, as well as the Chief Information Officer will:

• Define and present the current state architecture and the target state architecture, and map the specific ITM components to the end state departmental architecture;
• Review and update the ITM Program Blueprint and the ITM Program Plan to identify project interdependencies, as well as update the ITM Project Risk and Issue Management Strategy to include project interdependency as a risk category;
• Create a template, to be completed by program leads, that captures program health, budget, priorities, milestones and project interdependencies and include available information regarding the date of the next gate for projects when presenting project portfolio update to departmental oversight committees;
• Review and update its Program and Project Risk and Issue Management Strategy and update the risk categories, and communicate and ensure the revised risk categories are applied across all projects; and
• Assess scope change requirements and develop new guidance to ensure procedures and approvals are consistent across all projects.

4.0 Overall Conclusion

The results of the audit revealed that an ITM Program management framework has been designed and is being implemented to support current and future IT projects. There are opportunities to strengthen the governance, risk management and change management of the overall Program, helping to ensure both that projects are well-managed individually, and also as an interconnected and coordinated program with collective outcomes.

The audit identified some opportunities for improvement in the areas of:

• CIPO-Specific Governance: CIPO should identify project interdependencies, and ensure that interdependencies are updated as project risks evolve. 
• Departmental Governance: CIO should strengthen the Stage-Gate Framework to ensure that Program-level information is provided to departmental oversight committees, including accurate, up-to-date overall Program health, budget, priorities, milestones, and project interdependencies. Further, CIO should complete the planned enterprise architecture to encourage alignment of projects across the department.
• Risk Management: CIPO should strengthen its risk management processes by ensuring that all risks related to the overall ITM Program are identified, mitigated, and reported, and that a common risk universe is used across all projects.
• Internal Controls: CIO should update their change request guidance to reflect requirements for projects in Stage 2; defined thresholds for scope and schedule changes; and requirements to present change request histories to oversight committees.

Appendix A: List of ITM Projects

Project name		Project Core	StageFootnote 2
Client Facing Services: these projects primarily address elements visible to users not members of the CIPO organization.
1	Trade-Marks e-filing (PRJ00017K)	This project will consolidate four legacy web applications into a new, single, and e-service. It will provide additional web offerings in order to allow clients to send and receive necessary information required to process their trade-mark application over the internet.	Stage 3 (Hold)Footnote 3
2	CIPO Electronic Filing of a Patent Application (PRJ00017D)	Improvements to E-commerce services for filing a patent application in the following areas: data integration of the web interface to supporting backend systems, automated validation and confirmation of submitted applications.	Stage 3
3	CIPO Goods and Services Database Re-Engineering (PRJ000193)	This project is to expand the Canadian listing with internationally accepted terms to help better align CIPO with other intellectual property offices by supporting the application of Nice Classification as the standard for classifying goods and services in Canada.  It will also improve the manual's functionality and usability in support of Canadian businesses.	Closed
4	CDAS/ Merged with Patents Correspondence On the Web (PRJ000011)	This project will develop a client interface to provide access to CIPO's prosecution patent correspondence on the web. The project will use an approach that could be used horizontally across the CIPO product lines, enabling simpler public access to documents. Once the Patent documents are available online, CIPO Document Access System (CDAS) will be sending daily exports of complete search and examination documents to the WIPO CASE system.	Closed
5	CIPO e-Dossier (PRJ0001JB)	This project will provide comprehensive online access to all of CIPO's public IP case file documents and information, and restricted access to non-publicly disclosable patents and industrial design IP case file documents/ information. Application history and certificates on line. This project follows and elaborates on "CDAS/ Merged with Patents Correspondence On the Web" project.	Stage 2Footnote 4
6	My CIPO Online Services (PRJ 0001l4)	My CIPO Online Services delivers a secure, single point of entry to Departmental online services, allowing Innovators and Agents to view, download and upload IP information and interact with CIPO services.	Stage 2
7	My CIPO Online Services II	Project will build upon the foundational elements of Online Services project and introduce CIPO remaining services for accessibility via the portal.	Pre-Stage 1
8	Collaboration Capability (Intra CIPO and between CIPO and external parties)	This project delivers the collaboration services needed to enable the relevant aspects of the online user experience vision/strategy, and more specifically the mandatory requirements developed during this project. Required collaboration include: Online collaboration enablement between Clients and assigned Business line employees (Shared collaboration workspace, Webinars, instant messaging etc.); Online collaboration enablement between Agents and Business Employees. Also include collaboration enablement for e-learning and document sharing (webinars, online tutorials, case studies online, etc.)	Pre-Stage 1
9	TM Data XML conversion to ST.96	Currently TM bulk data is exported in a proprietary flat file format. The WIPO/industry standard for Trade-mark data is ST.96. This project will create a TM export module that produces a bulk export file in XML in accordance with the WIPO/Industry standard ST. 96.	Stage 5Footnote 5
10	CIPO Smart Search (PRJ 0001l3)	This project implements the ability to execute advanced searches of IP-related information (i.e. data and documentation) for both Departmental and external parties.	Stage 3
11	CIPO Agent Examination Management (PRJ0001JA)	This project will result in the ability to improve the management process for the Patents and Trade-marks agent examination process. All agent examination information will be contained within a single system, which will provide for a more unified approach.	Stage 2Footnote 6
12	CIPO Client Relationship Management Foundation (PRJ0001AS)	The CRM Foundation project will establish a fully functioning Client Relationship Management (CRM) solution within CIPO's Information Branch (IB), and key line of business stakeholders; It will develop the data definitions for a CIPO-wide Master Client Repository; and will also create a foundational service request framework to manage client requests and interactions with CIPO.	Stage 4
13	CIPO Client Relationship Management Self-serve (other LOB`s)	This project delivers the CRM services needed to enable the relevant aspects of the user experience vision/strategy.  The scope of this project includes the integration of the CRM services into one of the product line legacy solutions.	Pre-Stage 1
CIPO BackOffice Services: these capabilities are not accessible by external stakeholders directly, but may be made available through related client-facing services.  These services are required for CIPO to deliver on its mission and administer Canada's IP system.
14	CIPO Modernized IP Case and Workflow Solution for Patents (PRJ000151)	The CIPO Modernized IP Case and Workflow Project will deliver new technology based capability that will automate and track the business processes and activities specific to each Line of business (national & international). For maximum Business value and to reduce costs, a common solution is envisioned across all CIPO Business Lines. Consequently, this project will modernize CIPO's processing of patents, including appeals, and support ongoing regulatory and business changes.	Stage 2Footnote 7
15	CIPO Modernized IP Case and Workflow Solution for Trademarks and Trademarks Opposition Board (TMOB) (PRJ0001DD)	The CIPO Modernized IP Case and Workflow Project will deliver new technology based capability that will automate and track the business processes and activities specific to each Line of business (national & international). For maximum Business value and to reduce costs, a common solution is envisioned across all CIPO Business Lines. Consequently, this project will modernize CIPO's processing of trademarks including oppositions, and support ongoing regulatory and business changes.	Stage 3
16	CIPO Modernized IP Case and Workflow Solution for Copyrights/ Industrial Designs (PRJ0001DC)	The CIPO Modernized IP Case and Workflow Project will deliver new technology based capability that will automate and track the business processes and activities specific to each Line of business (national & international). For maximum Business value and to reduce costs, a common solution is envisioned across all CIPO Business Lines. Consequently, this project will modernize CIPO's processing of copyrights and industrial designs and support ongoing regulatory and business changes.	Stage 2Footnote 8
17	CIPO IP Document and File Management (IPDFM) (PRJ00014Z)	IPDFM is a centralized repository system used for all Lines of Business and Branches.	Stage 3
18	Backcapture (aka Implementing Scanning/ Imaging of Active IP Case File documents)	Includes digitizing, indexing, and making current case files available digitally.  As we migrate to the new solution, Patents process for dealing with paper correspondence will be leveraged by other Branches through a transition period leading to a common CIPO approach across all Branches (mail room and scanning).	OperationalFootnote 9
19	CIPO Integrated Financial Services (PRJ000181)	In support of the CIPO Revenue Management Strategy and the Operational Excellence pillar, this project will define and implement a solution to permit a single source of financial data and implement the concept of integrating financial and operational data to improve quality and timeliness of information for decision making.	Stage 3Footnote 10
20	CIPO Telework (Long Term IT Solution) (PRJ000013)	CIPO telework employees require an enhanced capability to securely work remotely from home with the equivalent access, services and applications available to them on their office computer, including the ability to collaborate in real time.	Stage 4Footnote 11
21	Corporate Business Intelligence	This project delivers the BI services needed to enable the relevant aspects of the ITM user experience vision/strategy on the new modernized IP Platforms.  Tool for Users to mine data in warehouse and development of standard management reports.	Pre-Stage 1Footnote 12
22	CIPO Collaboration Solution (On and off Premises)	Establish a capability for CIPO personnel (employees and contractors) to securely collaborate in real time with other CIPO personnel on premise or off premise (off premise personnel via the Teleworker solution).	Merged with Project 8
CIPO Foundational Projects: These projects/investments provide the basis to move forward toward the target state as outlined in the ITM Program Blueprint.   Key outputs will frame/focus detailed requirements analysis activities, enable service lifecycle activities (from definition through to implementation) and outline CIPO and ISED-CIO resource transitioning required to achieve the target state.
23	Image Plus Transition to CM8	This project has at its core to upgrade the ImagePlus tool set to the IBM Content Manager tools set version 8.	Operational Footnote 13
24	Departmental Business Intelligence (Licenses) CIPO BI Platform Upgrade	This project was mostly focused on upgrading CIPO's business intelligence platform from Cognos version 7.5 to version 10.2.	Closed
25	Business Intelligence and Analytics - MDM (Sub-Project 1) (CIPO Pathfinder) (PRJ0001GE)	This project will focus on person, address, and the organizational perspectives for common CIPO and Departmental clients, an effort to consolidate and eliminate duplication of client records through a single "golden record" that represents the most complete and accurate contact information regarding our clients.	Operational
26	Business Intelligence and Analytics - Enterprise Data Warehouse (MDM Sub-Project 2) (TM&CID) (PRJ0001GK)	This project will focus on harnessing enterprise data warehousing technologies in support of the master data management initiatives. Implement the second phase of Master Data Management corporate wide and additional Business Intelligence and Analytics capabilities that aligns with the 5 year roadmap.	Merged with Project 21
27	CIPO Infrastructure Environment (Test and Integration Environment Setup / Implementation (Tooling and Configuration))	This project will set up various environments for all projects requiring infrastructure within the IT Modernization Program; environment types such as development, test, integration, performance, UAT and production.	Costed under all projects
28	CIPO - PatStats Report Standardization	The scope of this CWR will be limited to the conversion of existing reports currently used within the Patents Branch, and will focus on the set of 84 "core" reports currently generated by the "PatStats" system.	OperationalFootnote 14

APPENDIX B: AUDIT CRITERIA

Audit of the CIPO IT Modernization Program
Audit Criteria	Sub-Criteria
Governance Structure
1. There is a governance framework to manage and monitor the projects, subprograms and program activities included in the IT Modernization Program.	1.1 The governance structure, including roles, responsibilities and authorities, are clearly defined, communicated, and include key stakeholders involved with the ITM Program.
	1.2 Policies, directives and tools to define the operational framework and program/project management practices are in place.
	1.3 Oversight bodies receive and review standardized program and project information to support program related decisions.
Risk Management
2. There are processes in place to manage risks.	2.1 Management has a documented approach to risk management at both the program and project level which includes all required business and IT stakeholders.
	2.2 Risk and issue registers are maintained at the program and project levels and are actively used to identify, monitor and report on risks.
	2.3 There is an effective process to mitigate program and projects risks.
Internal Controls
3. There is a control management framework to support current and future IT projects in order to realize the benefits of the ITM Program.	3.1 There are change management controls in place for the review and approval of project scope, schedule and budget changes.
	3.2 CIPO considers capacity and resource availability for project delivery throughout the program lifecycle.

Appendix C: CIPO ITM Governance Structure

Appendix C: Long description

Appendix C: CIPO ITM Governance Structure

Figure 1 depicts the governance structure of the Canadian Intellectual Property Office (CIPO).

There are seven key elements in CIPO's governance structure:

1. Enterprise Services Management Office
2. Information Knowledge and Management Directorate
3. Enterprise Information Technology Working Group
4. Program Office Working Group
5. Change Review Working Group
6. Enterprise Content Management Working Group
7. IM/IT Committee

The first and second elements of CIPO's governance structure, the Enterprise Services Management Office and the Information Knowledge and Management Directorate, fall under the responsibility of the Chief Information Officer (CIO) and play a role in the intake process for all projects subject to CIPO's governance approval process.

The third element of CIPO's governance structure, the Enterprise Information Technology Working Group, has for mandate to discuss IT-related issues with the goal of maintaining or raising the quality of IT services received from CIPO's service providers, provide opportunities for communication with respect to technology issues, and ensure standardization of hardware/software equipment.

The fourth element of CIPO's governance structure, the Program Office Working Group, has for mandate to provide integrated coordination for CIPO IT initiatives, activities and acquisitions (such as Change Work Requests, Business Proposals for SG projects) and make relevant links and identify interdependencies.

Both the Enterprise Information Technology Working Group and the Program Office Working Group play a role in the requirement gathering process for all projects subject to CIPO's governance approval process.

The fifth element of CIPO's governance structure, the Change Review Working Group, has for mandate to assess, impact and ensure alignment and integration of IT projects and IM/IT initiatives, with CIPO's strategic priorities and with current and future business needs, to recommend prioritization of change requests to IM/IT Committee, to recommend Change Work Requests to the IM/IT Committee for approval, and to recommend IT projects in Stage-Gate 1 to Stage-Gate 5, Post Project Review Stage and all Change Requests to IM/IT Committee for recommendation to the CIPO Senior Executive Committee-Programs Steering Committee (SEC-PSC) for approval.

The sixth element of CIPO's governance structure, the Enterprise Content Management Working Group, has for mandate to ensure a horizontal approach to information management (IM) projects, to identify compliance requirements, dependencies and opportunities to leverage efforts/resources for the development of consolidated IM plan, to assess IM readiness level based on established criteria to ensure integration of sound IM practices, and to ensure alignment with IM business needs and priorities.

Both the Change Review Working Group and the Enterprise Content Management Working Group play a role in the impact assessment, integration and alignment process for all projects subject to CIPO's governance approval process.

The seventh element of CIPO's governance structure, the IM/IT Committee, has for mandate to review and recommend IM/IT projects seeking Stage-Gate approval, monitor of progress and proactive management of risks associated with IM and IT projects, prioritize IM/IT projects and Change Work Requests, recommend and approve all change requests, and provide advice and guidance in respect to information management services and practices.

Appendix D: ISED Stage-Gate Framework

Appendix D image description

Appendix D: ISED Project Management Stage-Gate Framework

Appendix D contains the ISED Project Management Stage-Gate Framework, which lays out the Stage-Gate Process and gives details on the Framework's requirements.

The Project Stage-Gate Process is a set of 5 unique Stages. Each Stage is comprised of a process and set of deliverables, which must be met in order to ensure approval at the associated Gate and to progress to the next Stage.

Stages 1 and 2 involve Portfolio Management, where the project's status is "Proposed". During this time, Operational Costs are incurred.

Stages 3 to 5 involve Project Management, where the project's status is "Active". During this time, Project Costs are incurred.

During the Post-Project Review, the project's status is "Closed" and Ongoing Costs are incurred.

From Stages 2 to 5, Total Investment Costs are incurred. Also, project status reporting must be completed during the period and the assigned Project Steering Committee must be consulted prior to each Gating approval. 

Projects are categorized by Tier according to their cost:

• Tier 1 Projects have costs of >$1M
• Tier 2 projects have costs of >$500K  $1M
• Tier 3 projects have costs of  $500K

Definitions/Responsibilities:

Investment Board (IB)

• Provides strategic direction and oversight for ISED's project investment portfolio.
• Authorizes all projects to move from Stages 1 to 3.
• Reviews ORRs for Tier 1 and 2 projects.

Investment Oversight Committee (IOC)

• Ensures projects conform to government requirements and ISED processes, and support departmental priorities and objectives.
• Authorizes Tier 1 and 3 projects to move from Stages 3 to 5.

Project Steering Committee (PSC)

• Provides oversight, strategic direction and advice for individual projects in ISED's project portfolio.
• Assesses whether projects are effectively scheduled, managed and on track to realize expectations.

Sector Management

• Has the same responsibilities as a PSC.
• Authorizes Tier 3 projects to move from Stages 3 to 5 and the Post Project Review.
• Monitors projects throughout their lifecycle.
• Reviews the Business Benefits and Outcomes Report.

Treasury Board (TB) Submissions

• ISED must submit a TB submission for projects where the PCRA rating exceeds the Organizational Project Management Capacity Assessment (OPMCA) rating.
• TBS may recommend that the Department seek TB project approval authority for any project regardless of the PCRA level.

PCRA

• Project Complexity and Risk Assessment (applicable only to Tier 1 and 2 projects >$750K)

During Stage 1 (Idea Generation), the cost and schedule determine the Rough Order of Magnitude, which is +/-100% for this Stage. The Project Management deliverable required for this stage is the Business Proposal for all Project Tiers. For Gate 1, the Gating Committee for all Project Tiers is the Investment Board (IB).

During Stage 2 (Concept Initiation), the Project Management deliverables required are the Project Summary, Project Dashboard Report, Business Case, Project Charter, Business & Stakeholder Requirements, Project Complexity and Risk Assessment (PCRA—for Tier 1 and 2 only), Capital Checklist, and Privacy & Security Documents. The IT deliverable required is the High Level Solutions Architecture document. For Gate 2, the Gating Committee for all Project Tiers is the IB. At this Gate, the Rough Order of Magnitude should be +/-50%. The date of Gate 2 Approval is considered to be the Project Start Date. Upon Gate approval, the project is considered to be "Active".

During Stage 3 (Project Planning), the Project Management deliverables required are the Project Summary, Project Dashboard Report, Project Management Plan, Solution Requirements, updated PCRA (Tier 1 and 2 only), IM, Privacy & Security Documents. The IT deliverables required are the Detailed Solutions Architecture and the IT Project Cost Estimation documents. For Gate 3, the Gating Committee for Tier 1 and 2 projects is the Investment Oversight Committee (IOC), and for Tier 3 the Sector Management. At this Gate, the Rough Order of Magnitude should be +/-20%.

During Stage 4 (Project Execution), the Project Management deliverables required are the Project Summary, Project Dashboard Report, Transition Plan, and Security Documents. The IT deliverable required is the Project Architecture Alignment Review. For Gate 4, the Gating Committee for Tier 1 and 2 projects is the IOC, and for Tier 3 the Sector Management.

During Stage 5 (Project Close-Out), the Project Management deliverables required are the Project Summary, Project Dashboard Report, Project Close-Out Report including Lessons Learned, and Security Documents. For Gate 5, the Gating Committee for Tier 1 and 2 projects is the IOC, and for Tier 3 the Sector Management. The date of Gate 5 Approval is the Project End Date, when the project is now "Closed".

During Post Project Review, the Project Management deliverables required are the Project Summary, Project Dashboard Report, and the Outcomes Realization Results (ORR) Report (or the Tier 3 Business Benefits and Outcomes Report generated from the Departmental Project Portfolio Management or DPPM system). For Post Project Review, the Gating Committee for Tier 1 and 2 Projects is the IB, and for Tier 3 the Sector Management.

This Framework was developed by the Departmental Project Management Office (DPMO), which manages and incorporates best practices into Project Management processes, standards, tools and templates. For more information, a link to their intranet site is provided as well as their email address: ic.dpmo-bgpm.ic@canada.ca.

Management Response and Action Plan

A - For inclusion in the report

The findings and recommendations of this audit were presented to the Commissioner of Patents, Registrar of Trade-marks and Chief Executive Officer of the CIPO, the Director General, Programs Branch of the CIPO, the Assistant Deputy Minister, Digital Transformation Service Sector, and the Chief Information Officer. Management has agreed with the findings included in this report and will take action to address all applicable recommendations by August 31, 2018.

B - For follow-up purposes - Detailed actions to address the recommendations in the report