Final Evaluation of the Hydrogen Early Adopters Program

6. Findings: Design, Implementation, and Cost Effectiveness

6.1 Program/Project Design

Conclusion

Participants agreed that collaborative arrangements are a key factor in technology and market development for the industry. They generally accepted the program requirement to establish a consortium; however, some noted they had been able to achieve a high level of technology development within the supply chain and with a broad range of firms in other projects without this requirement. The concept of the consortia was ultimately accepted by applicants, however, the liability clause, which was put in place in order to minimize the financial risk to the government, caused significant difficulties for some companies. The requirement for all members of the consortium to be joint and severally liable for the project was a financial risk that some companies were reluctant to accept.

While the consortium requirement and the related liability clause were put in place to both encourage program success and minimize the financial risk to government, the evaluation revealed other characteristics of the projects that also played a role in achieving results:

• Project size: The two larger projects with many demonstrations — the Hydrogenics and Sacré Davey projects — were better able to demonstrate results than the smaller, less complex projects.
• Type of end-user: The involvement of firms as end-users as opposed to institutions helped to showcase market applications, furthering the technology's development.
• Lead firm situation: Lead firms that were either on a sound financial footing or who had strong managerial skills tended to be in a better position to deal with the administrative requirements of the program.

Findings

The consortium requirement was intended in part to promote collaboration and learning within the supply chain, a goal which is widely lauded within industry. Collaboration allowed technology providers to work with suppliers and equipment manufacturers to ensure effective integration of their products. The consortia arrangement has also been a significant catalyst in lining up the supply chain, helping to bring together potential suppliers. This was important both from a technical standpoint where knowledge is shared up and down the supply chain and from a business standpoint where the projects have given the lead companies the opportunity to work out a business model with partners.

Firms that were not the lead but which did agree to be a part of the consortia typically did not behave differently than their contract-based counterparts. In other words, there were no noticeable behavioral differences between firms in or out of the consortia; all conducted their activities according to their particular contribution. The scope of the projects, which went beyond the capabilities of any one firm, appeared to have a more important impact on collaboration goals than a rigid consortium requirement.

The other intent of the consortium requirement was to minimize the financial risk to the government in funding these projects. This was done through the inclusion of a liability clause. Most interviewees commented that the liability clause was desirable in principle to the extent that it underpinned the creation of a consortia approach; however, in practice, conformity to the clause proved difficult as it required each company in the consortium to agree to be responsible for the actions of the others. Companies feared a partner might go under and leave the others liable to make up the loss to the project. As a result of these difficulties, negotiations between companies and with the H2EA program took longer than originally anticipated.

While the consortium and liability requirements were created in order to minimize project risks and maximize project success, other characteristics of the project design were also identified as key elements. Supporting several applications in one project, as was the case for the Hydrogenics and Sacré Davey projects, fostered inter-firm learning within the supply chain. Several of the applications, for example, were faced with similar learning curves in the area of code compliance and technology integration. The benefits of experience in one application could be transferred to other applications, helping to improve the chance of success of the demonstration project. Similarly, the Sacré Davey project has thus far successfully demonstrated several technology solutions, including H2ICE and Hydrogen-enriched Compressed Natural Gas (HCNG) technology and fuel cells, all of which are competing options to lowering greenhouse gases.

The ability of the lead firm to deal with delays and provide the administrative support required to effectively manage the project was also revealed to be a key success factor in project implementation. FCTL consumed a considerable amount of their resources in carrying out their project. Ford Canada, who had invested a significant amount of time in project administration, also reflected on the difficulty smaller companies must face in managing such a project. Having had one consortium led by a consulting firm specialized in project management (Sacré-Davey) underscores further the heavy burden of managing such projects.

Of the four consortia funded by H2EA, the Sacré-Davey consortium stands out in three important respects. First it is led by a consulting company with expertise in project management, as opposed to a technology developer who may not have the same capacity to deal with the administrative aspects of the program. Second, it is the only consortium to leverage funds up to the maximum of 75 percent under Treasury Board stacking rules. This situation, while beneficial from a financial standpoint, also resulted in additional administrative work, as the company had to attribute contribution dollars to specific elements of each demonstration in order to ensure compliance with the terms and conditions of each program. Third, the consortium relies heavily on contractors — five of the nine non-consortium firms involved — to provide hydrogen technology solutions for the demonstration. All of these factors have been important to the outcome of the project.

top of page

6.2 Program Implementation

6.2.1 Reaching Target Firms

Conclusion

The program reached its intended target firms. These target firms (most of whom were SMEs, exceptions being Ford Canada, Deere & Company, Purolator Courier and Bell Canada) consisted of a full supply chain of technology providers, users and suppliers as well as supporting government organizations.

Findings

The H2EA RMAF ¹⁹ indicates that, to be successful, the program must reach a range of firms and other organizations including those involved with hydrogen production, storage and distribution; development of hydrogen and hydrogen-compatible technologies; and users who will integrate hydrogen technologies into portable, stationary and mobile applications. Program data indicates that, in addition to the firms receiving funding through the program, another 37 consortia applied or expressed interest in the program. These consortia/partnerships represent 87 firms and organizations involved in the hydrogen and fuel cell sector in Canada.

Other organizations with an interest in having a successful H2EA program that promoted industry development included Canada's Hydrogen and Fuel Cell Committee, Hydrogen & Fuel Cells Canada, the NRC Institute for Fuel Cell Innovation, CTFCA and SDTC.

There were 20 different hydrogen applications demonstrated through H2EA involving over 50 companies representing all areas of the H2 industry noted above (the H2EA participants and their areas of interest are set out in the Key Performance Indicators ²⁰). In addition, CTFCA was a funding partner in the Hydrogenics, Sacré Davey and the Ford projects. SDTC was also a funding partner in the Sacré Davey project.

6.2.2 Application Process

Conclusion

The mechanism of a competitive application process was viewed as appropriate by most successful applicants that were interviewed, but many, both successful and unsuccessful, found the process to be rigid and time consuming. Interaction between the sponsoring program and companies in a developing sector like hydrogen and fuel cells is important at all stages of the application process, from contracting to reporting on results. It was suggested that sufficient resources at the departmental level were not available at the start of the program to deal effectively with the high level of uptake.

Findings

Many interviewees noted the widespread interest in the program at its launch and during the application process. This interest in the program generated a large number of applications (40 applications by February 16, 2004 ²¹). In a week, various companies sent in 10–12 proposals, and from these H2EA invited more detailed individual proposals. Overall, this was seen by these applicants as a good approach; they were given the impression that the program was highly competitive but with not enough money to go around. While these companies had no issue with the approach, they were concerned that program resources were insufficient to handle the load. The program was quickly oversubscribed and, with only three individuals working on the program at the beginning, the program had insufficient capacity to assess applications in a timely fashion.

The length of the application review process was expected to take between one and three months, but in practice took much longer, up to a year in some cases. An audit conducted in 2006 by Industry Canada notes that the length of the process was due in part to the information and documentation gathering from applicants ²². This had repercussions for applicants from both a technological and a financial perspective. For projects that were technologically driven, by the time they were approved, the technology had on occasion advanced and the projects were no longer viable unless the technology was updated. An industry respondent commented, "The industry is fast moving and the TPC approval process did not keep pace". In that instance, the approval process took a year, and, for business reasons, the company did not proceed, suggesting that if that time had been shorter, the company may very well have implemented the project. The delays also brought challenges in keeping consortium members committed to the demonstration and, in some cases, led to a change in partners and a consequent change in the cost sharing arrangements.

The 'batching of proposals' was also of concern to industry. TPC would allow proposals to be sent in at any time but would forward them in batches to the selection committee on specific dates posted on the website. This extended the length of the approval process for some projects and, according to some interviewees, undermined much of the momentum that was built up in the proposal stage among consortium members. On the other hand, from a due diligence perspective, this procedure was necessary in order for proposals to be compared to one another in a competitive and transparent process.

6.2.3 Due Diligence Processes

Conclusion

The H2EA program was generally considered to be too rigid in its development and application of procedures for an industry that is not yet fully mature and subject to rapid technological change. Terms and conditions of the program were appropriate for demonstrations of existing technologies (which was the intent of the program) but less so for those still undergoing development, which, while unanticipated, included some of the technologies demonstrated through the funded projects. The reassignment of files, necessitated by changes in staffing levels over the project lifecycles, further complicated the situation, resulting in difficulties in providing seamless support in the administration of the funded projects.

Findings

The program was run with an approval system similar to that of TPC's R&D program. As a result, the due diligence process, when applied to H2EA, did not consider the fact that demonstration projects were likely to require adjustments during the course of implementation. For example, once a project had already been approved, TPC could not make changes under their rules of procedure unless a request was made to amend the contribution agreement with justification that, for the companies, would further delay implementation of the project. Amendments to the contribution agreement involved the Program Services Board (PSB), an Assistant Deputy Minister level Committee in Industry Canada, a step that added to the complexity and consequent delays in implementation.

Other administrative issues that caused difficulties for the companies were the interpretation of eligible costs, ensuring stacking limits were adhered to, compliance with legislative requirements, and, as mentioned in section 6.1, the need to ensure that contributions coming from different sources could be traced to specific elements of the project. This situation became even more difficult to manage as program staff changes resulted in the reassignment of files during the course of the project lifecycle. As a result, new relationships between consortia members and program staff needed to be built, some of which had become strained due to the complexities of managing the projects.

---

¹⁹ Results Based Management and Accountability Framework (RMAF), H2EA Program, Industry Canada — Technology Partnerships Canada, 2003. (Return to Text)

²⁰ Key Performance Indicators, H2EA Program, page 5, February 20, 2007. (Return to Text)

²¹ H2EA Six Month Progress Report, Performance Management Network Inc., Industry Canada, November 23, 2004. (Return to Text)

²² Program Audit of the Hydrogen Early Adopters Program, Audit and Evaluation Branch, Industry Canada, June 2006. (Return to Text)