Language selection

Search

Evaluation of the Sustainable Development Technology Fund

From: Innovation, Science and Economic Development Canada

This publication is available online at www.ic.gc.ca/eic/site/ae-ve.nsf/eng/h_03866.html.

To obtain a copy of this publication or an alternate format (Braille, large print, etc.) please fill out the Publication Request Form at www.ic.gc.ca/Publication-Request or contact:

Web Services Centre
Innovation, Science and Economic Development Canada
C.D. Howe Building
235 Queen Street
Ottawa, ON K1A 0H5
Canada

Telephone (toll-free in Canada): 1-800-328-6189
Telephone (Ottawa): 613-954-5031
TTY (for hearing-impaired): 1-866-694-8389
Business hours: 8:30 a.m. to 5:00 p.m. Eastern Standard Time
Email: info@ic.gc.ca

Permission to Reproduce

Except as otherwise specifically noted, the information in this publication may be reproduced, in part or in whole and by any means, without charge or further permission from Innovation, Science and Economic Development Canada, provided that due diligence is exercised in ensuring the accuracy of the information reproduced; that Innovation, Science and Economic Development Canada  is identified as the source institution; and that the reproduction is not represented as an official version of the information reproduced, nor as having been made in affiliation with, or with the endorsement of, Innovation, Science and Economic Development Canada.

For permission to reproduce the information in this publication for commercial purposes, please fill out the Application for Crown Copyright Clearance at www.ic.gc.ca/copyright-request or contact the Web Services Centre mentioned above.

© Her Majesty the Queen in Right of Canada, as represented by the Minister of Innovation, Science and Economic Development Canada, 2018.

Cat. No. lu
ISBN 978

Aussi offert en français sous le titre Évaluation du Fonds Technologies du développement durable.

March 2018

Presented to the Performance Measurement and Evaluation Committee on March 29, 2018

Approved by the Deputy Minister on April 16, 2018

Table of contents

Evaluation of the Sustainable Development Technology Fund PDF - 590 KB


List of acronyms

AEB
Audit and Evaluation Branch
CA
Contribution Agreement
CTCGB
Clean Technology and Clean Growth Branch
BDC
Business Development Bank of Canada
EDC
Export Development Canada
GAC
Global Affairs Canada
GHG
Greenhouse Gas
ISED
Innovation, Science and Economic Development Canada
IT
information technology
JAM
Joint Account Management
MOU
Memorandum of Understanding
Mt CO2e
Megatonnes of Carbon Dioxide Equivalent
NRCan
Natural Resources Canada
OAG
Office of the Auditor General of Canada
PPCQ
Post-Project Completion Questionnaire
R&D
Research and Development
SCC
Standards Council of Canada
SDTC
Sustainable Development Technology Canada
S&E
Screening and Evaluation
SME
Small- or Medium-Sized Enterprise
SOI
Statement of Interest
SIPS
Strategy and Innovation Policy Sector
TRL
Technology Readiness Level

List of tables


List of figures


top of page

Executive summary

Program overview

SDTC is an arm's length, not-for-profit organization created in 2001 by the Government of Canada through the Canada Foundation for Sustainable Development Technology Act.  Its goal is to support projects that develop and demonstrate new technologies that address issues related to climate change, air quality, clean water and clean soil.

SDTC's SD Tech Fund provides non-repayable financial contributions and operational support for the development and demonstration of Canadian cleantech projects to facilitate the movement towards scale-up for commercialization.

The target population for the SD Tech Fund is Canadian companies:

According to SD Tech Fund administrative data, $893 million has been invested to support 341 cleantech projects since 2001.

Innovation, Science and Economic Development (ISED) is responsible for managing transfer payments and overseeing the administration of the SD Tech Fund Funding Agreement with SDTC to help ensure that government policies and regulations are observed.

Evaluation purpose and methodology

The objective of this evaluation was to address the core issues of relevance and performance in accordance with the Policy on Results and address certain issues identified by ISED's Clean Technology and Clean Growth Branch (CTCGB) and SDTC.  Specifically, the evaluation focuses on program relevance, the Fund's progress toward its intermediate and long-term outcomes, and efficiency and economy.

The scope of the evaluation included 218 projects categorized as completed or terminated between April 1, 2005 and March 31, 2017.  The evaluation employed seven data collection methods, including a literature review; document review; financial, administrative and performance data review; statistical analysis of administrative data; interviews; case studies; and a survey of SD Tech Fund recipients.

Findings

Relevance

Multiple lines of evidence show that there is a continued need for the SD Tech Fund to support cleantech producers to further develop and demonstrate their technologies, as it directly addresses the ongoing pre-commercial funding gap.  SDTC's due diligence process is valued by investors, who appreciate SDTC's process for evaluating technical and project management competence.

Performance

Evidence shows that recipient firms are forming collaborative partnerships across the innovation chain, due in large part to the consortium model of the SD Tech Fund.  In addition, SDTC is actively building and maintaining partnerships across the innovation chain including provincial and federal organizations to facilitate collaboration and sharing of information to support cleantech producers.

Approximately three-quarters of all projects that received SD Tech Fund support have made progress towards demonstration or have successfully demonstrated.  The key factors related to successful demonstration include the availability of financing and the formation of partnerships.  Numerous barriers toward demonstration exist, significantly impacting the progress of about one-third of projects.

Although the mandate of the SD Tech Fund ends at demonstration, at the time of the evaluation, about one-third of SD Tech Fund recipients subsequently reached the market (i.e. commercialize), with larger projects and those receiving other government support having a higher probability of success.  The ability to raise follow-on capital following successful demonstration remains the main barrier to commercialization.  The development of relationships with other partners may help to identify opportunities to enhance commercialization prospects for SD Tech Fund projects.

The SD Tech Fund plays an important role in the government's mission to achieve its GHG emission goals and enable Canadian firms to compete globally in the cleantech space.  Available data and primary research for this evaluation indicates that the SD Tech Fund has helped create environmental and economic benefits for the Canadian economy.

Efficiency and Economy

SDTC has a governance structure in place which supports the effective delivery of the SD Tech Fund.  It has recently implemented several improvements to address findings from recent audits and reviews, although further work is still required to fully implement the recommendations.

Evidence clearly shows that SDTC is efficiently and effectively delivering the SD Tech Fund, although some concerns exist over the timeliness of the application and contracting process.

Currently, there are a range of government support programs for cleantech projects across the innovation chain.  To access support from these programs, cleantech producers may be required to provide similar information that can take time and resources.  In addition, there are a range of organizations collecting post-demonstration information from cleantech producers.

Recommendations

  1. CTCGB should encourage SDTC to continue to work with other partners to develop relationships to enhance the identification of opportunities for cleantech producers to progress towards commercialization, including connecting firms with international opportunities/supply chains and sources of follow-on funding.
  2. CTCGB should encourage SDTC to work with the Clean Growth Hub to address some of the limitations of the administrative data, including developing standards to enable consistency of the data and collecting information related to key performance indicators, as well as environmental and economic benefits.
  3. CTCGB should encourage SDTC to fully implement the recommendations stemming from a recent review of program governance.
  4. CTCGB should work with SDTC to identify opportunities to streamline the application and contracting process to improve contracting timelines.
  5. CTCGB should encourage SDTC to investigate other information-sharing and streamlined reporting options, including collaboration with other data collection and funding entities, with the aim of minimizing duplication and reducing the administrative burden on companies.

top of page

1.0 Introduction

1.1 Report Overview

This report presents the results of an evaluation of the Sustainable Development Technology Fund (SD Tech Fund).  The SD Tech Fund is administered by the Canada Foundation for Sustainable Development Technology, commonly known as Sustainable Development Technology Canada (SDTC).  The purpose of the evaluation is to assess the relevance and performance of the SD Tech Fund.  The report is organized into four sections:

1.2 SDTC

SDTC was established in 2001 by the Government of Canada, through the Canada Foundation for Sustainable Development Technology Act, to fund the development and demonstration of new technologies that promote sustainable development.Footnote 1  It is an arm's length, not-for-profit (non-share capital) organization that was created to support projects that develop and demonstrate new technologies that address issues related to climate change, air quality, clean water and clean soil.
The objectives of SDTC are to:

To meet these objectives, SDTC strives to fill the gap between proven research and advancement to a commercial product or service.

1.3 SDTC Funding

Initially, $550 million was provided to SDTC as three separate up-front grants for the SD Tech Fund – $100 million in 2001, $250 million in 2003 and $200 million in 2005.  Subsequent funding for the SD Tech Fund was provided as conditional transfer payments in Budget 2011 ($40 million), Budget 2013 ($325 million) and Budget 2016 ($50 million), bringing the Government of Canada's total investment to $965 million.  Budget 2017 allocated an additional $400 million, specifically for the re-capitalization of the SD Tech Fund, starting in 2017-18.

According to SDTC reporting, since its launch in 2001, SDTC has invested an estimated $987 million to support 347 cleantech projects through the SD Tech Fund (341 projects, $895 million) and Next Generation Biofuels Fund (6 projects, $92 million), and has leveraged more than $2.7 billion from other project partners.Footnote 3 In total, these investments have contributed to 226 patents held by SDTC funded cleantech entrepreneurs.

1.4 SD Tech Fund

SD Tech Fund at a Glance

  • SDTC was established in 2001 through the Canada Foundation for Sustainable Development Technology Act.
  • SD Tech Fund supports the development and demonstration of technologies leading to eventual commercialization.
  • SD Tech Fund is the major source of government financial support to help bring cleantech projects to demonstration stage.
  • Target population for SD Tech Fund is Canadian companies developing technology with environmental benefits, commercial potential, and who form a consortium of partners.
  • $893 million in SD Tech Fund funding (representing 90% of all SDTC funding) was allocated to 341 cleantech projects through 2016-17.

The SD Tech Fund supports the development and demonstration of technologies to a scale that would allow the technology to move to commercialization, including the development of prototypes, demonstration of technology systems and processes, and the scaling up and piloting of technologies.  Projects with a technology readiness level (TRL)Footnote 4 of between 3 and 7 are considered eligible for SD Tech Fund support.  Although there are numerous supports for cleantech producers, the SD Tech Fund is considered to be the major source of government financial support to help bring cleantech projects to the demonstration stage.

The target population for the SD Tech Fund is Canadian companies:

To apply for SD Tech Fund funding, a company must first submit a statement of interest (SOI) for assessment by SDTC.  If the SOI is successful, the firm is invited to submit a detailed proposal for evaluation through the screening and evaluation (S&E) process, which includes due diligence, a site visit by SDTC's S&E experts, and a review by external experts.  Following this, the proposal is shared with and reviewed by SDTC and, if ultimately approved by the Board, the applicant is moved through to contracting and enters into a formal agreement with SDTC by signing the Contribution Agreement (CA).

As Table 1 shows, according to the administrative data, the SD Tech Fund alone has invested approximately $893 million to support 341 projects.

Table 1: SD Tech Fund Summary of Project Status since 2001 (up to March 31, 2017)
Status Count Percent Allocated SD Tech Fund Support
Active
Board approved projects that are finalizing a CA or have a signed CA and commenced project work.
123 29.1 $397,659,067.03
Completed
Board approved projects that have completed all required milestones in their CAs.
164 38.9 $445,852,746.47
Terminated
Board approved projects with a CA that was terminated before the completion of all milestones.
54 12.8 $49,763,793.28
Cancelled
Board approved projects that never completed a CA to start project activities.  Funds for these projects were released for re-allocation.
81 19.2 $0.00
Total 422 100.0 $893,275,606.78

Source: SDTC Administrative Data, September 2017

1.5 Governance and Administration

Governance of SDTC

SDTC is governed by a Board of Directors, which reflects the broad interests of the public, private and academic sectors in Canada.  The Board comprises 15 directors, seven of whom are appointed by the Governor in Council, including the Chair.  The remaining eight members are appointed by the SDTC Member Council which is comprised of individuals from the public, private and academic sectors.

There are five Board committees: the Audit Committee, Corporate Governance Committee, Human Resources Committee, Next Generation Biofuels Fund Project Review Committee, and SD Tech Fund Project Review Committee.  Both the Chair and the President/Chief Executive Officer serve on all Board committees as ex-officio, non-voting members.

The members of the Foundation consist of 15 industry leaders, who are appointed by other members in accordance with the Act.  Members provide informed and representative perspectives on the subject of sustainability.

Administration of the SD Tech Fund

SDTC is responsible for the administration of the SD Tech Fund in accordance with the investment guidelines per the Funding Agreement with ISED.  Among other things, SDTC reviews and selects project proposals and makes recommendations to the Board, and enters into contractual agreements with eligible recipients to provide funding for eligible projects.

ISED is responsible for managing transfer payments and overseeing the administration of the SD Tech Fund Funding Agreement with SDTC to help ensure that government policies and regulations are observed.

1.6 Stakeholders

The main groups of external stakeholders/consortium partners (i.e., those who have an interest in the projects supported by the SD Tech Fund) include the following:

1.7 Logic model

The logic model (Figure 1) depicts the activities to be pursued, the outputs to be produced and the expected outcomes resulting from the delivery of the SD Tech Fund and how its objectives are to be achieved.

Figure 1: SD Tech Fund Logic Model

Description of Figure 1

Figure 1 depicts a logic model for the SD Tech Fund.  A logic model shows how program activities are expected to produce outputs and in turn how these outputs are expected to lead to different levels of results or outcomes.
There are three program activities, each of which leads to outputs: 

  • The first program activity, "attract and nurture quality cleantech applications" leads to the outputs of "applications received", "support provided throughout the application process", "applications reviewed and assessed", and "board approval of applications". 
  • The second activity, "project management and monitoring" leads to the outputs of "Contribution Agreements signed", "project work plans implemented", "recipients coached through the project delivery process", "progress and close-out reporting", "post-project monitoring", and "overall SD Tech Fund performance measuring and reporting".
  • The third activity, "support launch of market-ready cleantech projects" leads to the outputs of "outreach and communication outputs", "participation in cleantech growth initiatives (e.g., Canadian Cleantech Advantage Leadership Summit), and "strategic partnerships to support commercialization and scale-up". 

All three activities/outputs lead directly to the following immediate outcomes (0-3 years out):

  • SD Tech Fund supported technologies make progress towards successful demonstrations.
  • Recipients form collaborative partnerships across the innovation chain.

All immediate outcomes lead to the following intermediate outcomes (4-6 years out):

  • SD Tech Fund supported technologies are successfully demonstrated.
  • SD Tech Fund supported technologies make substantial advancements towards commercialization.

All intermediate outcomes lead to the following long-term outcomes (7+ years out):

  • SD Tech Fund supported technologies are commercialized and diffused in target market sectors in Canada and internationally.
  • Environmental and economic benefits are realized from the adoption of SD Tech Fund supported technologies.

The above two long-term outcomes lead to the ultimate outcome: "Canadian businesses grow and become world class and globally competitive".

top of page

2.0 Methodology

This section provides information on the evaluation approach, objective and scope, the specific evaluation issues and questions that were addressed, the data collection methods, and limitations for the evaluation.

2.1 Evaluation Approach

This evaluation was managed and conducted by ISED's Audit and Evaluation Branch (AEB) and adopted a goal-based approach, addressing the program's stated outcomes. 

To maximize the use of evaluation resources, wherever possible, the findings of this evaluation built on findings reported in previous evaluations and reviews.Footnote 6  For this evaluation, data collection efforts focused on addressing the information gap relating to the progress toward the longer-term SD Tech Fund outcomes.

2.2 Objective and Scope

An evaluation of the SD Tech Fund is required in accordance with the Treasury Board Policy on Results and addresses certain issues identified by ISED's Clean Technology and Clean Growth Branch (CTCGB) and SDTC.  Specifically, the evaluation focuses on the progress of the SD Tech Fund toward its intermediate and long-term outcomes (although program relevance and efficiency and economy are also examined).  Given the focus of the evaluation, all 218 projects categorized as completed or terminated by March 31, 2017 were included in the evaluation scope.

2.3 Evaluation Issues and Questions

Based on the program's Performance Measurement Strategy,Footnote 7 and subsequent consultations with the program, the evaluation addressed the following questions:

Relevance

  1. To what extent does the SD Tech Fund address a demonstrable and unique need?

Performance

  1. To what extent are recipients forming collaborative partnerships across the innovation chain?
  2. To what extent are SD Tech Fund projects making progress towards demonstration and eventually demonstrating?  What factors influence demonstration?
  3. To what extent are SD Tech Fund projects commercializing?  What factors influence commercialization?
  4. To what extent are SD Tech Fund projects and/or technologies contributing to the realization of environmental and economic benefits?

Efficiency and Economy

  1. Does the governance structure support efficient and effective delivery?
  2. How efficient and effective is SDTC in the delivery of the SD Tech Fund? 
  3. Are there alternative approaches that would increase the cost-effectiveness of the SD Tech Fund?

2.4 Data collection methods

Multiple lines of evidence were used to address all evaluation questions.  The data collection methods included a literature review; document review; financial, administrative and performance data review; statistical analysis of administrative data; interviews; case studies; and a survey of SD Tech Fund recipients.

AEB conducted most of the lines of evidence, with contractors conducting the case studies and the survey.  The statistical analysis of the SD Tech Fund's administrative data on outcomes was conducted by ISED's Strategy and Innovation Policy Sector (SIPS).

Literature Review

The review of academic literature determined if the SD Tech Fund continues to address a demonstrable and unique need.  Trends and issues in sustainable development technologies in Canada and around the world were also examined.  Further, the literature review looked at how governments in other countries support cleantech in order to gain insight as to whether there are differences or alternative approaches to funding sustainable development technologies.

Document Review

The document review was conducted to gain an understanding of the SD Tech Fund, its alignment with government priorities, its role in Canada's Innovation and Skills Plan, and its achievement of expected outcomes.  Further, it provided insight into the achievement of program objectives and shed light on modifications made to the program since the previous evaluation in 2012. 
The document review included:

Financial, Administrative and Performance Data Review

The data review addressed questions on relevance and performance, as well as efficiency and economy.  In particular, it looked at the SD Tech Fund's contributions to the development of sustainable technologies, the number of projects making advancements towards successful demonstration, and how efficiently and economically the SD Tech Fund is delivered. 

The data review also focused on the progress of the SD Tech Fund toward expected outcomes, as well as the SD Tech Fund budgets, expenditures and leveraged funds.  This analysis looked at the efficiency of the internal processes of the program, as well as the burden placed on recipient organizations.

Statistical Analysis of Administrative Data

A quantitative analysis was conducted to identify factors that contribute to the successful completion and demonstration of cleantech innovation projects.  It also identified factors that served as barriers for projects that were not as successful.

Interviews

The objective of the interviews was to gather in-depth information including views, perceptions and factual information that address the evaluation questions.  The interviews were semi-structured and designed to obtain qualitative feedback from a range of respondents.  In total, 22 individual and group interviews were conducted with the following stakeholder groups:

Case Studies

As part of the evaluation, nine targeted case studies of SD Tech Fund recipients were conducted to collect in-depth information on SD Tech Fund projects that were completed or terminated early.  The case studies were derived from the 218 SD Tech Fund projects to be broadly representative of the population of projects based on criteria such as region, sector, completion date, size of recipient organization, and size of the SD Tech Fund contribution.  Each case study involved a review of the relevant project filesFootnote 10 and a telephone interview.

Survey of SD Tech Fund Recipients

A survey of past SD Tech Fund recipients was utilized to collect outcome information from as many firms that received support as possible.  The survey of recipients captured quantitative and qualitative evidence to complement existing data by asking recipients questions pertaining to program design and delivery and the achievement of outcomes.  The survey was conducted by telephone interview with an online option if preferred.  Based on a sample of 127 companies, representatives from 61 cleantech producers participated in the survey.

2.5 Limitations

Gaps in the Administrative Data

There are some gaps in the administrative data which inhibit a complete and accurate measurement of program outcomes in an efficient manner.  For example, the administrative database provided for this evaluation (for projects completed prior to March 31, 2017) did not contain any information on some of the key performance indicators required for the measurement of outcomes indicated in the Performance Measurement Strategy.  Further, the utility of the data was limited due to missing, incomplete, out-of-date, and/or contradictory information.  To overcome the limitations of the administrative data, other lines of evidence were used to help enhance the administrative data information as much as possible.

There are also challenges in calculating commercialization rates and environmental and economic benefits due to a lack of rigorous and reliable data.  One reason is that long-term tracking is a challenge, as it can take five years or more (after demonstration) before any progress on commercialization is made, which is beyond the three-year post-project tracking activities of SDTC.  Separately, the 2017 Office of the Auditor General of Canada (OAG) report noted that its audit of the SD Tech Fund did not find documented evidence of SDTC's Project Review Committee's consideration of environmental benefits when recommending projects to the Board for approval.Footnote 11  The report recommended that SDTC clearly document the assessment of potential GHG emission reductions that it undertakes when approving projects that have reductions as an intended long-term outcome – to which SDTC agreed.

The information shortcomings are known and recognized (Budget 2017 provided $14.5 million to NRCan and ISED to establish a Clean Technology Data Strategy) and work has started on developing more robust approaches and methods to enhance information.  In addition, collaborations are being established to facilitate data sharing across departments.  Further, the Clean Growth Hub will work with SDTC to identify best practices, ensure consistency of data and address barriers to sharing data and information.

Attribution

The ability to attribute long-term outcomes to SD Tech Fund activities or support is a challenge as it is difficult to control for intervening variables.  For example, the likelihood of recipient firms to reach demonstration is impacted not only by financial and non-financial SD Tech Fund support, but also by factors such as a firm's management team and their ability to execute.  Therefore, while the SD Tech Fund plays an important role in the ability of a firm to achieve their outcomes, full attribution cannot be assigned to the SD Tech Fund alone.

Survey Fatigue of SD Tech Fund Recipients

Cleantech producers receive many requests for information, but often have limited resources to respond.  The evaluation found that most provincial development organizations, federal departments and at least two cleantech-focused organizations regularly survey cleantech producers to monitor their progress/results.  Survey fatigue of SD Tech Fund recipients has an impact on one of SDTC's key data collection tools, the annual post-project completion questionnaires (PPCQs) required in each of the first three years post-project funding.  More than one-half of recipient firms did not submit any information via the questionnaires.  A key impact of this trend is that SDTC is not receiving all post-project information from recipient firms and must use a range of other techniques to report on economic and environmental impacts.

It is possible that the burden of having to respond to an evaluation survey may have negatively impacted survey response rates.  Several measures were taken to minimize this impact: (i) diversifying the questions from what was previously asked; (ii) keeping the survey questions to the minimum number required for addressing the evaluation questions; and (iii) inviting as many recipients in the sample as possible to help alleviate any non-response bias.

Access to past research data

SDTC and the SD Tech Fund have been evaluated on three previous occasions and audited twice to fulfill the requirements set out in past funding agreements.  AEB hoped to minimize duplication, build on past findings, and contribute to new knowledge.  However, the extent to which this could be undertaken was limited by the inaccessibility to previous research tools and data.  In order to ensure the rigour of this evaluation, similar questions/measures were used across methodologies to ensure consistency and minimization of any biases.

top of page

3.0 Findings

3.1 Relevance

3.1.1 To what extent does the SD Tech Fund address a demonstrable and unique need?

Key finding: There is a continued need for the SD Tech Fund to support cleantech producers to further develop and demonstrate their technologies, as it directly addresses the ongoing pre-commercial funding gap.  SDTC's due diligence process is valued by investors, who appreciate SDTC's process for evaluating technical and project management competence.

The evaluation examined whether there is an ongoing need for government support for pre-commercial development and demonstration in the cleantech space and whether the SD Tech Fund is meeting that need.

Cleantech is a critical space from an environmental standpoint and contributes to sustainable development, which the Federal Sustainable Development Strategy for Canada, 2016-2019 broadly defines as "development that meets the needs of the present without compromising the ability of future generations to meet their own needs".  In a 2016 report by Environment and Climate Change Canada, 13 goals to support sustainable development were identified, four of which are addressed by the SD Tech Fund (effective action on climate change, clean growth, clean energy, and clean drinking water).Footnote 12

The importance of these four SD Tech Fund sustainable development goals cannot be overstated.  For example, climate change can lead to rising sea levels that threaten coastal communities, more frequent and severe wildfires and pest outbreaks, and extreme weather events such as storms and heat waves.  Further, the impact on Indigenous communities can be particularly harsh, as melting sea ice, snow cover and glaciers can disrupt travel routes and reduce the quality of foods in the North.  As well, in terms of clean growth, the benefits of cleantech include lower GHG emissions and increased resilience to climate change, healthier communities, and more sustainable and competitive economic sectors.Footnote 13

While cleantech is clearly an important space from an environmental standpoint, it is also vital from an economic standpoint – it is a space that is estimated to grow to $3 trillion by 2020, making it the third largest economic sector in the world.Footnote 14   If Canada maintains its share of the global market, the cleantech space is projected to employ over 125,000 people in Canada by 2020.  As well, over 80% of Canada's cleantech small- or medium-sized enterprises (SMEs) are exporters and are nine times more likely to export than similarly-sized SMEs in other sectors.Footnote 13

Findings from the SD Tech Fund recipient survey underscore the importance of government support in meeting these sustainable development challenges.  In the survey, 93% of firms said SD Tech Fund support was extremely or very important for addressing financial challenges associated with the development and demonstration of pre-commercial cleantech projects.  Further, 85% reported that their project would not have gone forward without SD Tech Fund support (the remaining 15% indicated that aspects of their project would have changed in the absence of funding).  The Fund serves to de-risk the investment in cleantech and encourage/facilitate contributions from private investors and other partners.  Additionally, case studies determined that while there are many sources of public sector funding for research, there are few sources to support the development and demonstration of pre-commercial cleantech projects.

Findings from the case studies, survey and interviews noted that the SD Tech Fund helps companies progress from pilot scale to demonstration scale.Footnote 15  Interviewees agreed that the conditions that contribute to the difficulty in progressing from pilot scale to demonstration scale continue to exist and that if the SD Tech Fund no longer existed there would be significant short- and long-term impacts for Canadian cleantech.  The costs for development and preparation for demonstration (TRLs from 3 to 7) are very high in Canada for large-scale cleantech projects.  Funding for large-scale projects is not readily available from private sources (e.g., banks, venture capital, etc.) as it is perceived to be very high risk since the proposed cleantech innovations are untested in real-world/operational settings and would require additional investment/support for scale-up for commercial production.

Aside from providing funding for cleantech projects, another important need addressed by the SD Tech Fund is assessing a project's feasibility and probability of success.  Stakeholder interview findings suggest that receiving Board approval for a project is perceived by private investors and other government partners across the innovation chain to mean that a project is worth pursuing and merits funding – thereby providing companies with the necessary funds to help bring their project to demonstration and, hopefully, subsequent commercialization.  However, it was also noted that SDTC's reputation has the potential to hinder development in the sense that if a company is not successful in attaining SD Tech Fund support then that company may not be able to secure support from other investors.

The existence of a demonstrable and unique need is further evidenced by the number of countries which also help fund their respective cleantech ecosystems.  The literature review showed that Finland and Canada are similar in their national regulatory qualityFootnote 16 and government effectivenessFootnote 17, both of which contribute to "an institutional framework that attracts business and fosters growth by providing good governance and the correct levels of protection and incentives essential to innovation".Footnote 18  In Germany, much like the SD Tech Fund's consortium model requirement, the cleantech industry is heavily reliant on the extensive use of venture capital and private equity with a smaller proportion of public funding.  Further, Norway is a world leader in cleantech innovation – made possible by the number of incubator and cluster programs that are managed by the private sector, but are funded by the Government of Norway, as well as a rich network of government organizations that are actively engaged in helping start-ups in the clean energy industry, such as Innovation Norway.  Finally, the United States' approach shares many of the same features of the Canadian cleantech innovation model – several internal funding programs and others external to government.  However, in addition to public funding, the United States has a highly active network of cleantech hubs and holds the largest total sum of targeted venture capital in the world.

3.2 Performance

3.2.1 To what extent are recipients forming collaborative partnerships across the innovation chain?

Key finding: The consortium model of the SD Tech Fund plays an important role in facilitating collaborative partnerships between funding recipients and their suppliers and customers.  In addition, SDTC is actively building and maintaining partnerships across the innovation chain.

Pure Technologies

SD Tech Fund allocation: $2.2M
Leveraged Funding: $4.6M

  • Received funding in 2007 for the demonstration of a market ready system for inspecting pipe while the pipe was in use based on the PipeDiver technology.
  • Collaborative partnerships were formed with a pipe manufacturer and a number of municipalities.
  • The municipalities provided access to real-world infrastructure to test the prototypes.
  • Municipalities were also able to provide practical input related to their needs.

SD Tech Fund recipients are forming collaborative partnerships across the innovation chain due in large part to the consortium model employed by the program.  Several case study participants and stakeholders interviewed for the evaluation emphasized that the SD Tech Fund's requirement for collaborations is perhaps the most important feature of the program's design, which is missing in other government-funded programs that support project demonstration.  In most cases, the recipient firm identified at least two potential collaborators at the time of application to the Fund.  An important aspect noted in the interviews is SDTC's preference that at least one collaborator be a market validation partner (i.e., representative of the target customer).  In one case, a company was able to engage several municipalities to test prototypes of its technology.  This input was invaluable to developing the market-ready product through the provision of actual infrastructure for testing purposes.  The municipalities also provided advice on how to best commercialize the product.

A review of program administrative data on 341 completed, active or terminated projects indicated that each project has an average of 4.13 consortium partners (which includes the project firm as well).  There does not appear to be any difference in the number of consortium partners based on the size of a project – for example, of the 60 projects with funding of $15 million or larger there were an average of 3.90 consortium partners.  By comparison, of the 65 projects with funding of $2 million or smaller there was an average of 3.88 consortium partners.

Administrative data also indicate that for every dollar of SD Tech Fund support the total leveraged amount from all partners and collaborators equalled $2.80 – higher than the target of $2 outlined in the Performance Measurement Strategy.  By year, the amount leveraged varied from $2.07 to $3.42 between 2002 and 2017.Footnote 19

The survey found that 74% of firms agreed or strongly agreed that SD Tech Fund support was important to the formation of partnerships for their projects.  Almost one-half (46%) of survey respondents agreed or strongly agreed that SDTC's engagement activities/support helped to identify potential consortium partners.  The level of involvement of consortium members was reported to be highest during the project (77% of firms indicated that consortium members were often or always involved during the project) and lower pre- and post-SD Tech Fund support (43% and 46%, respectively).

Evidence clearly indicates that SDTC has been developing partnerships across the innovation chain.  For example, SDTC has (or has had) a memorandum of understanding (MOU) with several provincial partners.  The list includes the Province of British Columbia, Nova Scotia Innovacorp (expired agreement), Alberta Innovates, Transition Énergétique Québec, Ontario Centres of Excellence, and Emissions Reduction Alberta.  All MOUs specify the benefits of the agreement between the organizations as:

In terms of federal and other government departments and agencies, SDTC works collaboratively with EDC, BDC, GAC, NRCan, and the Standards Council of Canada (SCC) – as part of a larger innovation-funding ecosystem (along with private interests) to support demonstration and commercialization.  Together with these partners and other partners, SDTC works to improve access to financing, encourages investment, supports the demonstration of technologies, and builds the capacity necessary for the growth and prosperity of Canadian clean technologies.Footnote 20

Among SDTC partnership activities noted by interviewed stakeholders, the recent engagement in the Joint Account Management (JAM) meetings (launched in summer 2017) was mentioned most frequently.  The goal of the JAM is to facilitate discussion and collaboration among SDTC, EDC, BDC and GAC to share information on high-potential firms in order to understand the firms, the cleantech innovation, their financial needs, and any organizational opportunities or mechanisms that could be considered to support firms.  Recently, the JAM was working on developing a waiver that would allow SDTC to share information with its partners to minimize duplication of efforts in their funding processes for post-demonstration support.

In addition, SDTC is entering into partnerships with regional sector organizations, universities and colleges, as well as with regional economic development organizations.  These were described as partnerships that work to support cleantech across the spectrum – from research, development and demonstration through to commercialization (one of SDTC's long-term outcomes).  Overall, interviewees agreed that SDTC's formation of public and private partnerships is critical to support Canadian cleantech innovation.  Specifically, interviewees indicated that the development of partnerships, including among funding organizations, is key to addressing critical issues of cleantech producers such as application and reporting burden, identifying funding/financing opportunities for eligible firms, and, where possible, developing common data standards, definitions, and information sharing.

3.2.2 To what extent are SD Tech Fund projects making progress towards demonstration and eventually demonstrating?  What factors influence demonstration?

Key finding: Approximately three-quarters of all projects that received SD Tech Fund support have made progress towards demonstration or have successfully demonstrated.  The key factors related to successful demonstration include the availability of financing and the formation of partnerships.  Numerous barriers toward demonstration exist, significantly impacting the progress of about one-third of projects.

TRLs are a measure used to evaluate the maturity of an evolving innovation and are based on a nine-level scale (see Appendix A).  Levels 1 through 6 represent the early and progressive stages of R&D for innovations.  In contrast, levels 7 through 9 represent the pre-commercialization gap for innovations when projects are working towards demonstrating in an operational environment.  According to SDTC administrative data, almost 74% of the 164 completed SD Tech Fund projects eventually demonstrated (i.e. met or exceeded their TRL goal levels of 7 or higher by the end of the project).  An even greater proportion (88%) of survey respondents reported that their projects had successfully demonstrated.

Figure 2 shows that more than one-half of the firms that demonstrated were in the energy utilization or transportation sectors.

Figure 2: Sector Characteristics of Completed SD Tech Fund Projects that Demonstrated (n=164)

Description of figure 2

Figure 2 is a pie chart of 164 completed SD Tech Fund projects that eventually demonstrated. It is divided into seven different sector categories.  The largest slice is energy utilization at 31.4% followed by transportation at 22.3%, waste management at 12.4%, energy exploration and production at 9.9%, power generation at 9.1%, agriculture at 7.4%, and forestry at 7.4%.  Demonstration is defined as reaching TRL7 or higher by the end of the project.  The data source is the September 2017 version of the SDTC administrative database.

Note: Demonstration is defined as reaching TRL7 or higher by the end of the project.
Source: SDTC administrative data, September 2017

Xantrex Technology Inc.

SD Tech Fund allocation: $1.2M
Leveraged Funding: $2.5M
Revenues: $0 (for Xantrex)

  • Contracted in 2007 to develop an innovative power inverter technology and drive train for wind turbine manufacturers.
  • Designed to reduce operating costs, boost overall performance and efficiency, and generate no direct GHG gases or other pollutants.
  • Project was unable to secure a demonstration site and was therefore terminated.
  • Subsequently acquired by Schneider Electric in United States and commercialized.

The share of projects successfully demonstrating at TRL7 or higher (74%) is similar to the percentage of projects making progress towards demonstration (75%).  Progression towards demonstration can be measured by either: (1) the number of projects which completed all of the milestones as outlined in their CA; or (2) the share of projects reaching their goal TRL at or near the completion time of their CA.

According to several metrics that SDTC uses to track project progress, overall, projects tend to make good progress in the early stages, but momentum can wane towards project conclusion.  For example, the administrative data shows that between 74% and 100% of SD Tech Fund projects met their scheduled project milestones after years one and two.  However, the share of projects that met their milestones after year three ranged between 56% and 87%.  In addition, more recently developed measures used to track project progress by TRL shows that projects are making considerable progress but are not quite achieving defined targets.  Key program documentsFootnote 21 have set an overall target level of 75% of projects progressing two TRLs. However, the data shows that only about two-thirds (65.8%) are advancing at least two TRLs by the end of the project.

Several interviewees indicated that one of the main factors contributing to the successful demonstration of Canadian cleantech projects is the availability of financing and/or investment, as demonstrating is an expensive and time-consuming process.  As noted earlier, it is difficult for Canadian cleantech producers to access capital to finance their projects/innovations since cleantech projects are viewed as high risk investments.  SD Tech Fund support for "de-risking" projects to a level that private institutions and venture capitalists perceive as acceptable is seen as a key benefit.

Case study participants added that some of the key factors to successful demonstration included the comprehensive project management process that SDTC uses to ensure that projects are successfully demonstrating, including the detailed proposals outlining milestones and metrics to be achieved, as well as SDTC staff working closely with proponents through the project delivery process.  Survey respondents noted that SD Tech Fund support, collaboration and the formation of partnerships is key to successful demonstration.

Many other factors can affect the progress of cleantech projects towards demonstration.  Stakeholders interviewed for this evaluation noted that projects may be delayed for a range of reasons (e.g., technical advancement is taking longer than anticipated, issues with timeliness among key suppliers, availability of key equipment, management issues, etc.).  Overall, it was estimated that approximately one-third of projects experience issues that significantly impact progress towards demonstration.

3.2.3 To what extent are SD Tech Fund projects commercializing?  What factors influence commercialization?

Key finding: Although the mandate of the SD Tech Fund ends at demonstration, about one-third of SD Tech Fund recipients subsequently reach the market (i.e., commercialize), with larger projects and those receiving other government support having a higher probability of success.  The ability to raise follow-on capital following successful demonstration remains the main barrier to commercialization. The development of relationships with other partners may help to identify opportunities to enhance commercialization prospects for SD Tech Fund projects.

According to SDTC administrative data, about 34% of the 218 completed or terminated projects had commercialized or were deemed to be 'in the market' as of March 31, 2017 (see Table 2).

Table 2: Commercialization Status of SD Tech Fund Projects
Commercialization Status Count %
In the Market / Commercialized 75 34.4%
On path to market 65 29.8%
Failed to Reach Market 78 35.8%
Total 218 100.0%

Source: SDTC administrative data, September 2017

By sector, firms operating in the energy utilization, forestry or transportation sectors were far more likely to commercialize (see Figure 3).

Figure 3: Commercial Status of all SD Tech Fund Projects by Sector (n=218)

Description of figure 3

Figure 3 is a bar graph that illustrates the commercial status by sector of all 218 completed or terminated SD Tech Fund projects from 2001 to March 31, 2017 and is sourced from the September 2017 version of the SDTC administrative database.  The figure shows that:

  • Energy utilization: 49.2% of the 63 projects related to energy utilization are in the market/commercialized, 17.5% are on path to market, and 33.3% failed to reach market.
  • Forestry: 46.2% of the 13 projects related to forestry are in the market/commercialized, 15.4% are on path to market, and 38.5% failed to reach market.
  • Transportation: 43.6% of the 39 projects related to transportation are in the market/commercialized, 33.3% are on path to market, and 23.0% failed to reach market.
  • Waste management: 23.1% of the 26 projects related to waste management are in the market/commercialized, 34.6% are on path to market, and 42.3% failed to reach market.
  • Agriculture: 21.4% of the 14 projects related to agriculture are in the market/commercialized, 42.9% are on path to market, and 35.7% failed to reach market.
  • Energy exploration and production: 20.0% of the 30 projects related to energy exploration and production are in the market/commercialized, 40.0% are on path to market, and 40.0% failed to reach market.
  • Power generation: 18.2% of the 33 projects related to power generation are in the market/commercialized, 36.4% are on path to market, and 45.5% failed to reach market.

Source: SDTC administrative data, September 2017

In terms of the priority technology area (for some projects, more than one priority is identified), the commercialization rate was by far the highest among the 39 projects associated with enabling technology (51.3%).  For the other five priority technology areas (i.e., hydrogen economy, clean fossil fuel, biomass, biofuel, and renewable electricity) commercialization rates ranged from 15.8% to 27.3%.Footnote 22

Empirical analysis on the progress of projects backed by the SD Tech Fund suggests that larger projects have a higher probability of commercial success.Footnote 23  However, there is no clear statistical relationship between the share of total funding provided by the SD Tech Fund and the probability of commercial success.  Other findings indicate that projects led by firms who have accessed SDTC multiple times have a higher probability of commercial success, although this may have more to do with an enhanced firm-specific capacity, rather than firms experiencing better performance in subsequent projects through learning-by-doing.  As well, the presence of other government funding during a project appears to lead to a greater likelihood of commercial success – not only does other government funding add to the overall investment, it also incents SDTC to increase its own investment.

Interviewees indicated that one factor of success is the development of a customer base.  The ability to export to international customers was seen as being critical – firms must develop a pipeline or range of domestic and international customers for successful commercialization.  Further factors of success which were highlighted included the need for a strong project management team, the presence of consortium partners, and governance for building a talented management team for technical development and planning for demonstration and a path to market.  Interviewees indicated that these are areas that SDTC has provided support to develop.

In terms of barriers, interviewees identified factors that are generally outside the locus of control of SDTC and the SD Tech Fund.  For example, interviewees indicated that the key barrier to commercialization is the investment needed to ramp-up for commercial production, as it is expensive and difficult to secure the financing needed to scale-up for large-scale commercial production.  It was indicated that it is difficult to secure the funding for cleantech projects and projects that have demonstrated only a limited number of times.  Typically, investors would like to have the assurance associated with multiple demonstrations. 

The administrative data confirms these findings in that the majority of firms (particularly those with completed projects) identified financing as the key issue that prevented them from commercializing (see Figures 4 and 5).  Among terminated projects, technical issues and other issues were far more prevalent than among completed projects (54.2% versus 26.7% respectively), although financing remains the key issue.

Figure 4: Key Issues that Prevented Completed Projects from Reaching Commercial Market (n=30)

Description of figure 4

Figure 4 is a pie chart that illustrates the key issues that prevented some completed projects from reaching the commercial market and is sourced from the September 2017 version of the SDTC administrative database.  The figure shows that:

  • Financing/financial issues: 73.3% of the 30 completed projects failed to reach the commercial market due to financing/financial issues.
  • Technical issues: 10.0% of the 30 completed projects failed to reach the commercial market due to technical issues.
  • Other issues: 16.7% of the 30 completed projects failed to reach the commercial market due to other issues.

Source: SDTC administrative data , September 2017

Figure 5: Key Issues that Prevented Terminated Projects from Reaching Commercial Market (n=48)

Description of figure 5

Figure 5 is a pie chart that illustrates the key issues that prevented some terminated projects from reaching the commercial market and is sourced from the September 2017 version of the SDTC administrative database.  The figure shows that:

  • Financing/financial issues: 45.8% of the 48 completed projects failed to reach the commercial market due to financing/financial issues.
  • Technical issues: 29.2% of the 48 completed projects failed to reach the commercial market due to technical issues.
  • Other issues: 22.9% of the 48 completed projects failed to reach the commercial market due to other issues.
  • Policy issues: 2.1% of the 48 completed projects failed to reach the commercial market due to policy issues.

Source: SDTC administrative data , September 2017

Other barriers noted by interviewees included: identifying customers to take on cleantech solutions beyond immediate consortium partners in international and domestic markets; variations in procurement practices and regulations across Canada that make it difficult for cleantech producers to compete for contracts or gain entry into supply lists; and the need to make investments commercially viable, including producing cleantech at a price point that meets the needs of the end-users in Canadian and international markets.  In this final area, interviewees indicated that the challenge for cleantech producers is that their innovation must be able to replace an existing technology or service that clearly exhibits value to the potential customer through comparable unit pricing, unit/service duration or life, and/or savings in the long-term.

To provide some context to the potential barriers to advancement towards commercialization, the one firm in the case studies whose project had no progress to date indicated that their project did not progress beyond demonstration for reasons other than technology (most notably low oil prices) and that they would likely attempt to bring this technology toward commercialization again in the future.  For this firm, although the technology was perceived as being attractive, the risk of building an expensive commercial-scale facility using commercially unproven technology was seen as being too high.

The administrative data includes information on follow-on funding secured by SD Tech Fund recipients near or at the end of their SD Tech Fund funding and indicates that 40 of the 218 SD Tech Fund projects (or less than 20%) received a total of almost $2.1 billion in follow-on funding, with each project averaging almost $52.4 million in follow-on funding.  The smallest follow-on funding amount was $200,000 while the largest amount was $433.7 million.  The asset types of follow-on financing received are illustrated in Figure 6.Footnote 3

Figure 6: Types of Follow-On Financing Received by SD Tech Fund Recipients Since 2001 ($ millions)

Description of figure 6

Figure 6 is a pie chart that illustrates the types of follow-on financing received by SD Tech Fund recipients since 2001 (in millions of dollars) and is sourced from the SDTC Annual Report 2016-17 Supplement.  The figure shows that:

  • Public finance: $824.6 million in follow-on financing was received via public finance.
  • Private equity: $570.5 million in follow-on financing was received via private equity.
  • Venture capital: $445.7 million in follow-on financing was received via venture capital.
  • Private investment in public equity: $190.9 million in follow-on financing was received via private investment in public equity.
  • Debt/asset financing: $145.6 million in follow-on financing was received via debt/asset financing.

Source: SDTC Annual Report 2016-17 Supplement

However, receipt of follow-on funding is not a guarantee of success.  Although 55% of the 40 projects that received follow-on funding commercialized and 22.5% were in progress to market, an additional 22.5% failed to reach the market.  By comparison, 29.8% of the 178 projects that did not receive follow-on funding made it to market, 31.5% were in progress to market, and 38.7% failed to reach the market.

Additional findings from the survey revealed that 43% of firms agreed or strongly agreed that SDTC's engagement activities/support helped their firm secure follow-on financing from partners after SD Tech Fund support ended.

As noted above, there are many factors that can contribute to or detract from successful entry of cleantech projects and innovations into domestic and international markets.  The two case studies below provide an example of each.

Recommendation: CTCGB should encourage SDTC to continue to work with other partners to develop relationships to enhance the identification of opportunities for cleantech producers to progress towards commercialization, including connecting firms with international opportunities/supply chains and sources of follow-on funding.

3.2.4 To what extent are SD Tech Fund projects and/or technologies contributing to the realization of environmental and economic benefits?

Key finding: Despite the documented challenges in estimating environmental and economic benefits, the available data and primary research for this evaluation indicates that the SD Tech Fund has helped create environmental and economic benefits for the Canadian economy.

SDTC plays an important role in the government's mission to help grow Canadian companies into globally competitive ones.  Within this overall goal, the SD Tech Fund has two objectives: (i) contributing to achieving Canada's environmental objectives, including its GHG emissions reduction goals, and (ii) sustainable economic growth by enabling Canadian entities to compete globally in the cleantech space.Footnote 24

Environmental Benefits

A 2017 OAG audit examined how the SD Tech Fund was contributing to reducing GHG emissions.Footnote 25  The audit found that SDTC had a substantial review and challenge process for assessing proposed environmental benefits in project proposals – using internal experts and industry reviewers to assess intended benefits and emission reduction forecasts.  In addition, the audit found that SDTC had processes in place to estimate GHG emissions reductions as projects progressed.  SDTC had also modified its approach to calculating emission reduction estimates attributable to SD Tech Fund recipients to improve clarity in reports to Parliament.  However, the audit found that SDTC had difficulty in obtaining post-completion information from funding recipients.  Most of the expected post-project reports with environmental information were missing/not submitted.

Interviewees also indicated that environmental benefits are not easily defined or measured (e.g., while a new technology is thought to have environmental benefits, there may not be a measure presently available).  In addition, when estimates are developed, it is difficult to assess the level of benefits (e.g., reductions, etc.) that could be attributed to the investment by the SD Tech Fund versus other public and private investments. However, sometimes benefits are quantifiable, as evidenced by the Agrisoma case study.

Agrisoma Biosciences Inc.

SD Tech Fund allocation: $3.3M
Leveraged Funding: $7.8M
Follow-on Funding: $20.1M

  • Received SDTC funding in 2010 to develop and demonstrate Agrisoma's carinata-based renewable fuels.
  • Biojet fuel made from carinata powered the world's first 100% biojet flight that demonstrated a greater than 50% reduction in GHG emissions with better fuel efficiencies.
  • Demonstrated successfully multiple times.
  • Currently in progress toward market.

Despite these challenges, SDTC has gathered information that SD Tech Fund supported projects have contributed the following environmental benefits since 2001.Footnote 26

Economic Benefits

There are similar issues with the assessment of economic benefits yielded by SD Tech Fund projects, as it is difficult to obtain post-project information from recipients.  Further, the calculation of benefits and attribution to the SD Tech Fund is difficult.  However, recent Statistics Canada data shows that cleantech and environmental goods and services benefit the Canadian economy in many ways, including through increased gross domestic product (GDP) and employment.

According to SDTC reporting, SD Tech Fund projects have contributed to an estimated 9,437 new jobs (5,302 direct jobs and 4,135 indirect jobs) in the Canadian cleantech space since 2001.Footnote 13  Additional economic benefit estimates indicate that:

Primary research for this evaluation found that SD Tech Fund support for cleantech projects yielded economic benefits.  The majority of survey respondents who reported economic benefits (e.g., increase in the number of employees, sales, etc.) indicated that it was 'likely' or 'definitely' due to SD Tech Fund support.  Additionally, respondents also reported that the SD Tech Fund positively impacted their domestic (80%) and international (73%) sales.

The case studies provided more detailed, qualitative information.  Case studies showed that the SD Tech Fund enabled several Canadian cleantech producers to complete demonstration and to make advancements towards commercialization with significant economic impacts.

Recommendation: CTCGB should encourage SDTC to work with the Clean Growth Hub to address some of the limitations of the administrative data, including developing standards to enable consistency of the data and collecting information related to key performance indicators, as well as environmental and economic benefits.

3.3 Efficiency and Economy

3.3.1 Does the governance structure support efficient and effective delivery?

Key finding: SDTC has a governance structure in place which supports the effective delivery of the SD Tech Fund.  SDTC has recently implemented several improvements to address findings from recent audits and reviews, although further work is still required to fully implement the recommendations.

As described in Section 1.4, SDTC is governed by a 15-member Board of Directors with overall responsibility for the stewardship and strategic direction of the SD Tech Fund.  Seven Board Members, including the Chairperson, are appointed by the Governor in Council, while the remaining eight are selected by the SDTC Member Council which is comprised of individuals from the public, private and academic sectors.  Board Members' experience represents a wide range of sectors including cleantech, agriculture, sustainable community development, mining, life sciences, manufacturing, petroleum, finance, venture capital, engineering, environmental issues, entrepreneurship, chemical recycling, commercial wind and natural gas development, and public policy.

The Board and its five committees play a key role in the funding allocation process.  SDTC Board members sit on the Project Review Committee along with private sector investment principals to identify technologies with high potential and oversee funding processes.  However, the Board of Directors is the authority for final approval for SD Tech Fund support.  The Board now schedules at least five meetings per year to approve projects, aligned with the number of Project Review Committee meetings.

Overall, past reviews, audits and evaluations have found that SDTC and the SD Tech Fund have a sound governance structure and supporting processes in place to meet the program's objectives and mandate.  Past audits and evaluations have found that SDTC has strong business processes to screen, fund and support cleantech projects.  Activities and processes are in place to ensure that approved projects are consistent with objectives, transparent approvals and project monitoring.Footnote 27

A 2016 Governance Review assessed the effectiveness of the existing governance structures and processes and the extent to which current governance policies and practices have been implemented as intended.Footnote 28  Overall, the findings of this review were positive.  It found that SDTC has implemented a number of improvements to its governance processes and practices over the past two years, including expectations of the Board, Member Council, and Project Review Committee regarding integrity, values and ethics, as well as the roles and mandates of the governing bodies.

The review also identified some areas for improvement.  Eight recommendations were made in four areas:

Most recently, a 2017 OAG audit found that the SD Tech Fund has a comprehensive process in place for proposal review and approval that uses reviews by internal and external experts, several committees and final approval by the Board.Footnote 29  Further, it found that SDTC has measures in place to mitigate potential conflicts of interest for Board and committee members involved in the assessment and approval of proposals.

The evaluation followed up on progress made to address the recommendations outlined in the 2016 report.  While the evidence provided shows that some steps have been made toward addressing the action items, implementation is ongoing.

Recommendation: CTCGB should encourage SDTC to fully implement the recommendations stemming from a recent review of program governance.

3.3.2 How efficient and effective is SDTC in the delivery of the SD Tech Fund?

Key finding: SDTC is efficiently and effectively delivering the SD Tech Fund, although some concerns exist over the timeliness of the application and contracting process.

In examining the efficiency and effectiveness of the delivery of the SD Tech Fund, four metrics were considered:

Cost

An analysis of allocation and expenditure data provided by SDTC (covering the period from 2012 to 2016) was completed with the goal of analyzing the efficiency of the delivery of the SD Tech Fund.  Program expenditures which consistently exceed the allocation could indicate inefficiency in program delivery.Footnote 30  A 2011 performance audit flagged this as a risk, but evidence indicates this has been mitigated.  With the exception of 2015, program expenditures (i.e. salaries and benefits, non-salary operating expenses, and grants and contributions) were lower in every year than the allocated budget.  Over this five-year period, $355.6 million was allocated for the SD Tech Fund and $327.5 million (or 91.2% of the allocated amount) was spent.

In terms of the administrative costs of delivering the SD Tech Fund, administrative costs (i.e., salaries and non-salary operating expenses) as a percentage of total expenditures (i.e., project disbursements) have been declining since 2015.  In 2011, administrative costs were $11.3 million and project disbursements were $82.3 million, equating to an administrative cost ratio of 13.7%.  This ratio peaked at 21.5% in 2014, but fell significantly to 16.1% in 2015.  For the most recent reporting period in 2016-17, the administrative cost ratio was 14.3%.Footnote 31  For the $965 million in SD Tech Fund support available, SD Tech Fund operating expenses are capped at $167 million – implying an overall administrative cost ratio of 17%.Footnote 32

Administration of the Fund

SDTC has made a number of improvements to the administration of the fund in recent years.  The shift from a periodic call for applications to a continuous intake process in July 2016 was noted by case study participants and interviewees as a notable improvement, as it provides cleantech producers with greater flexibility over when to apply for SD Tech Fund funding.  In addition, SDTC has adopted a more interactive process throughout the application process, including the use of video conference presentations.  SDTC has also increased the frequency of final funding decisions made by the Board of Directors (from two times per year historically to five).

Further, providing funding up-front (the amount of which varies between projects based on the contract and the milestones schedule) was seen as a benefit.  SDTC's prepayment model is designed to allow firms to remain cash flow positive throughout the period that they are receiving SD Tech Fund support.  Survey respondents indicated that some government programs will only reimburse the firm against expenditures.  Having cash upfront is a benefit since cleantech is extremely cash-intensive and is beneficial when compared to the reimbursement model that many government programs employ.

Case study participants unanimously agreed that SDTC staff is helpful, technically competent, professional, and committed to the success of Canada's cleantech producers.  This was echoed in interviews, with special note made of the work of SDTC regional staff.  As well, stakeholders interviewed for the evaluation reported that the SD Tech Fund was being delivered efficiently to meet its goals.  It was identified as the correct model to meet the needs of firms.

Cleantech experts interviewed for the evaluation were asked for suggestions regarding how delivery could be improved.  The following areas were identified:

Timeliness

The timeliness of the application processes was consistently highlighted as an area for improvement by external stakeholders, as it is complex and requires multiple levels of approval.  The application process was noted by 35% of survey respondents to be slow and complicated (up to one year), requiring a lot of paperwork on the part of both applicant firms and SDTC.  Further, survey respondents thought that the application should be tailored to the size of the project.

Cleantech experts suggested that time requirements for the application and due diligence process was slow and misaligned with the pace of the private sector.  Several interviewees noted that the time from application to Board approval and receipt of funding is long.  Specifically, the application process would benefit firms if it was streamlined, simplified and processed faster.

Exploring further the timeliness of the approval of funds, a review of SDTC administrative data of the 218 completed or terminated projects (all of which started between 2002 and 2014 – prior to the change to the continuous intake of applications) found that 78.4% of projects received funding within 24 months of Board approval.Footnote 33  This percentage varied significantly from year-to-year, with the share being 100% in some years (for the 38 projects approved in 2002, 2003 and 2011)Footnote 34 and as low as 56.3% for the 16 projects approved in 2004.  The data also indicates that, on average, projects with a higher total cost were more likely to get approved and funded before projects with a lower total cost.

Further, a 2017 report by the Office of the Auditor General of CanadaFootnote 35 (based on 28 SD Tech Fund projects reviewed) found that the time elapsed from initial project submission to the first payment averaged 22 months.  This was partly attributable to SDTC's extensive due diligence process and to the time required for applicants to secure additional funding and meet other conditions.

It should be noted that there are a range of factors that could impact the timeliness of the application process that are outside of SDTC's control.  For example, many companies supported by SDTC are SMEs with limited staffing resources that may prevent them from responding quickly to requests for supplemental information from SDTC.

Reporting Requirements

Twenty percent of survey respondents suggested that reporting requirements were difficult, perceived as demanding, rigid and burdensome, particularly for smaller firms who may not be able to produce lengthy reports as easily as larger firms.  This was evident in an examination of the performance data, in that from 2010-11 to 2016-17, post-project questionnaire completion, a key reporting tool for economic and environmental impacts, ranged from only 9% to 34%. 

In the case studies, one company found the amount of reporting to be burdensome and suggested that SDTC should streamline its reporting requirements and take advantage of technology (e.g. online reporting).

Recommendation: CTCGB should work with SDTC to identify opportunities to streamline the application and contracting process to improve contracting timelines.

3.3.3 Are there alternative approaches that would increase the cost-effectiveness of the SD Tech Fund?

Key finding: Currently, there are a range of government support programs for cleantech projects across the innovation chain. To access support from these programs, cleantech producers may be required to provide similar information that can take time and resources.  In addition, there are a range of organizations collecting post-demonstration information from cleantech producers.

According to a 2016 report by the Working Group on Clean Technology, Innovation and Jobs, an alternative approach that could increase the cost-effectiveness of the SD Tech Fund (as well as other programs) is to create a "no wrong door approach" or a consolidation of programs to reduce the administrative burden on firms.Footnote 36  Currently, there is difficultly in navigating and accessing the appropriate government support programs for cleantech producers due to the large number of actors, programs and policies.  Federal, provincial and territorial collaboration on research priorities and policies could be achieved through alignment of collective outcomes across technologies and technology platforms.  Program delivery alignment could then help improve accessibility, and help technology developers negotiate public support through different stages of development.

Interviewed stakeholders identified the following approaches to improve the cost-effectiveness of the SD Tech Fund:

Recommendation: CTCGB should encourage SDTC to investigate other information-sharing and streamlined reporting options, including collaboration with other data collection and funding entities, with the aim of minimizing duplication and reducing the administrative burden on companies.

top of page

4.0 Conclusions and Recommendations

4.1 Relevance

4.2 Performance

4.3 Efficiency and Economy

4.4 Recommendations

Appendix A: Technology Readiness Levels Scale

Technology Readiness Level Description
TRL 1
Basic principles observed and reported
Basic principles of concept are observed and reported.  At this level scientific research begins to be translated into applied R&D.  Activities might include paper studies of a technology's basic properties.
TRL 2
Technology concept and/or application formulated
Technology concept and/or application formulated.  At this level invention begins.  Once the basic principles are observed, practical applications can be invented.  Activities are limited to analytical studies.
TRL 3
Analytical and experimental critical function and/or characteristic proof of concept
Analytical and experimental critical function and/or proof of concept.  At this level R&D is initiated.  Activities might include components that are not yet integrated or representative.
TRL 4
Component and/or breadboard validation in laboratory environment
Component and/or validation in a laboratory environment.  At this level basic technological components are integrated to establish that they will work together.  Activities include integration of 'ad hoc' hardware in the laboratory.
TRL 5
Component and/or breadboard validation in relevant environment
Component and/or validation in a simulated environment.  At this level the basic technological components are integrated for testing in a simulated environment.  Activities include laboratory integration of components.
TRL 6
System/subsystem model or prototype demonstration in a relevant environment
System/subsystem model or prototype demonstration in a simulated environment.  At this level a model or prototype is developed that represents a near desired configuration.  Activities include testing in a simulated operational environment or laboratory.
TRL 7
System prototype demonstration in an operational environment.
Prototype ready for demonstration in an appropriate operational environment.  At this level the prototype should be at planned operational level and is ready for demonstration of an actual prototype in an operational environment.  Activities include prototype field testing.
TRL 8
Actual system completed and qualified through test and demonstration.
Actual technology completed and qualified through tests and demonstrations.  At this level the technology has been proven to work in its final form and under expected conditions.  Activities include developmental testing and evaluation of whether it will meet operational requirements.
TRL 9
Actual system proven through successful mission operations.
Actual technology proven through successful deployment in an operational setting.  At this level there is actual application of the technology in its final form and under real-life conditions, such as those encountered in operational tests and evaluations.  Activities include using the innovation under operational conditions.

Source: Public Works and Government Services Canada description of Technology Readiness Levels.

top of page

Management Response and Action Plan

Management Response and Action Plan—Evaluation of the Sustainable Development Technology Fund PDF - 143 KB

A - For inclusion in the report

The findings and recommendations of the Evaluation of the Sustainable Development Technology Fund were provided to the Clean Technology and Clean Growth Branch of the Science and Research Sector (SRS) at ISED, as well as to Sustainable Development Technology Canada (SDTC).  Management of these partners have been consulted and agree with the findings included in this report and will take action to address all applicable recommendations by May 2020.

B - For follow-up purposes - Detailed actions to address the recommendations in the report

Recommendation Planned Action on the Recommendation Responsible Official (position) Target completion date
CTCGB should encourage SDTC to continue to work with other partners to develop relationships to enhance the identification of opportunities for cleantech producers to progress towards commercialization, including connecting firms with international opportunities/supply chains and sources of follow-on funding. The Clean Growth Hub, which includes SDTC as a member, will explore ways to better connect companies with commercialization supports, international markets and targeted sources of international project funding. Director General, Clean Tech and Clean Growth Branch July 2018
The Clean Growth Hub will identify strategies to help cleantech producers make progress towards commercialization.  Director General, Clean Tech and Clean Growth Branch February 2019
CTCGB should encourage SDTC to work with the Clean Growth Hub to address some of the limitations of the administrative data, including developing standards to enable consistency of the data and collecting information related to key performance indicators, as well as environmental and economic benefits. The Clean Growth Hub is undertaking a scan of a number of clean technology oriented programs to assess existing information being collected related to administrative data and performance indicators. This work is one of the components of the Clean Technology Data Strategy, established to provide the foundation for measuring the economic, environmental and social impacts of clean technology in Canada. Director General, Clean Tech and Clean Growth Branch May 2018
Using this information and with input from federal clean technology programs, including SDTC, the Clean Growth Hub will develop guidance and standards to enable consistent collection of comparable information across the federal government to support whole-of-government results reporting on clean technology. Director General, Clean Tech and Clean Growth Branch May 2019
The Clean Growth Hub will work with SDTC to implement the administrative data guidance and standards. Director General, Clean Tech and Clean Growth Branch May 2020
CTCGB should encourage SDTC to fully implement the recommendations stemming from a recent review of program governance. SRS will encourage SDTC to fully implement the outstanding recommendations. Director General, Clean Tech and Clean Growth Branch September 2018
CTCGB should work with SDTC to identify opportunities to streamline the application and contracting process to improve contracting timelines. SRS will encourage SDTC to identify steps and associated timelines to further streamline the application and contracting process, building on recent improvements, including moving from periodic calls for proposals to a continuous intake model, with a view to reduce the duration of the process and the administrative burden on companies.

This could include exploring opportunities and examining options for efficiencies, as well as implementing service standards.

Director General, Clean Tech and Clean Growth Branch October 2018
CTCGB should encourage SDTC to investigate other information-sharing and streamlined reporting options, including collaboration with other data collection and funding entities, with the aim of minimizing duplication and reducing the administrative burden on companies. SRS will encourage SDTC to identify actions and the associated timelines that could be taken to reduce duplication of requests for information from and other administrative burden on companies.

This could include exploring opportunities for collaboration with other federal programs and agencies and non-governmental agencies as appropriate, via the Clean Growth Hub and building on the work that the Hub has initiated under the Clean Technology Data Strategy. 

Director General, Clean Tech and Clean Growth Branch October 2018

top of page

Date modified: