Evaluation of the Canadian Institute for Advanced Research—Final Report

PDF version

431 KB, 29 pages

April 2013

Recommended for approval to the Deputy Minister by the Departmental Evaluation Committee on April 2, 2013

Approved by the Deputy Minister on April 8, 2013

Table of contents

• Executive Summary
• 1.0 Introduction
  • 1.1 Program Overview
  • 1.2 Governance
  • 1.3 CIFAR Research Model
  • 1.4 Resources
  • 1.5 Expected Results and Logic Model
• 2.0 Methodology
  • 2.1 Evaluation Approach
  • 2.2 Objective and Scope
  • 2.3 Evaluation Issues
  • 2.4 Data Collection Methods
  • 2.5 Data Limitations
• 3.0 Findings
  • 3.1 Relevance
  • 3.2 Performance
• 4.0 Conclusions and Recommendations
  • 4.1 Conclusions
  • 4.2 Recommendations

• Management Response and Action Plan

List of acronyms used in this report

List of tables

List of figures

Executive Summary

Program Overview

The Canadian Institute for Advanced Research (CIFAR) was established in 1982 as an independent not-for-profit corporation that supports networks of eminent Canadian and international researchers. Its mission is to increase Canadian research capacity in areas of importance to Canada, and to strengthen the Canadian research environment through the promotion of excellence and engagement with the international research community at universities and research institutes worldwide. CIFAR has been funded continuously by the Government of Canada since 1987.

The Government of Canada's Budget 2007 allocated $25M over five years to CIFAR, which was provided under a conditional grant arrangement. The $25M represented approximately one-third of CIFAR's total revenues over the five-year period from 2007–08 to 2011–12. The conditional grant was not directed to specific CIFAR programs, but rather as support for the overall organization, contributing to the existing objective of strengthening academic global research networks in Canada.

Evaluation Purpose and Methodology

In accordance with the Treasury Board Policy on Evaluation and the Directive on the Evaluation Function, the purpose of this evaluation was to assess the core issues of relevance and performance of CIFAR. The evaluation findings are based on the analysis of multiple lines of evidence. The methodology included a document review, literature review, key interviews, and a comparative cost study.

A key input to the evaluation was the 2010 evaluation and performance audit of CIFAR, conducted by Hickling Arthurs Low (HAL). This study included a document, file and literature review; interviews; a survey of research program members; and a benchmarking exercise.

Findings

Relevance

Publicly funded fundamental research responds to a need to foster innovation and provide social and economic benefits to Canadian society. CIFAR responds to this need by developing and managing large-scale research programs designed to achieve knowledge breakthroughs that will contribute to improved health, environmental, societal and economic benefits for Canadians.

CIFAR is consistent with federal government priorities related to support for fundamental research and the attraction, training and retention of researchers as set out in the 2007 Science and Technology (S&T) Strategy, as well as Speeches from the Throne and Federal Budgets since 2007. CIFAR programs are also in line with IC's strategic objectives.

Further, support for CIFAR is consistent with federal roles and responsibilities to encourage the development of science and technology. CIFAR occupies a niche within the Canadian S&T ecosystem, offering a relatively unique approach to conducting both basic and applied research as well as enabling multidisciplinary work and funding for international researchers and collaboration.

Performance

CIFAR has largely achieved the expected results as set out in the Funding Agreement

CIFAR has been successful in building interdisciplinary research strengths in areas of importance to the long-term social and economic development of Canada. These research strengths are evident in the results achieved by the organization's researchers as well as in the national and international recognition inherent in the many awards they have received. However, assessing the effectiveness of related knowledge sharing activities would require more robust performance measures.

Further, closer links between Canadian and international researchers have been achieved through increasing the degree of international engagement in CIFAR's programs. This has been achieved through a practice of drawing new members and advisors from different countries and supplementing these efforts by engaging in visits to other nations and participating in collaborative forums and research. Additional insight into the effectiveness of international outreach activities would require more enhanced performance measures.

In fostering high-quality research in Canada, CIFAR aims to build a strong research community in the country by involving young researchers in its programs. Graduate students and postdoctoral fellows participate on a similar level as program members. CIFAR has also attracted high-calibre researchers from other countries to its work, some of whom have been recruited to positions in Canada. According to interviewees, among the draws of CIFAR is a degree of international collaboration that is not easily found elsewhere.

Another requisite of a strong research community is the presence of appropriate funds, and CIFAR has been working toward shifting its funding ratio to rely more substantially on contributions from the private sector and less on contributions from the public sector. Over the past five years, funding from provincial governments has decreased. However, revenues have been relatively stable over the past three years as the organization has taken in more funds from private sources.

With respect to the use of funds, employing a third-party organization to deliver research programs of the type supported by CIFAR is an effective delivery mechanism. Further, the cost comparison exercise shows that CIFAR's "virtual model" is efficient when compared to the more traditional models used by other advanced research organizations in terms of the ratio of non-program costs to total costs.

Recommendations

The conclusions of the evaluation led to the following recommendation:

Recommendation 1: The Science Partnerships Directorate should work with CIFAR to enhance their Performance Measurement (PM) Strategy. Particular attention should be placed on developing additional performance measures to better capture the effectiveness of CIFAR's activities and outputs with a view to assessing intermediate and longer term outcomes.

1.0 Introduction

This report presents the results of an evaluation of the Canadian Institute for Advanced Research (CIFAR). The purpose of the evaluation was to assess the relevance and performance of CIFAR. The report is organized into four sections:

• Section 1 provides the profile of CIFAR;
• Section 2 presents the evaluation methodology, along with a discussion of data limitations;
• Section 3 presents the findings pertaining to the evaluation issues of relevance and performance; and,
• Section 4 summarizes the evaluation's conclusions and provides a recommendation for future action.

1.1 Program Overview

Arms-length organizations have been used as tools to deliver federal public policy since the 1990s, particularly in areas such as research and development and education. Through its Science Partnerships Program, Industry Canada manages federal contributions to a number of third-party organizations, including the Canadian Institute for Advanced Research (CIFAR). These investments are expected to contribute to government commitments as set out in the federal Science and Technology Strategy. CIFAR has been partially funded by the Government of Canada since 1987.

CIFAR was established in 1982 as an independent not-for-profit corporation that supports networks of Canadian and international researchers who conduct long-term research on scientific, social and economic issues. Its mission is to increase Canadian research capacity in areas of importance to Canada, and strengthen the Canadian research environment through the promotion of excellence and engagement with the international research community at universities and research institutes worldwide. CIFAR supports research programs in 12 areas, drawing upon close to 400 eminent researchers around the world.

The research that is supported by CIFAR is fundamental in nature and includes a range of basic and applied research. The questions that are identified and developed into programs address critical gaps in understanding in areas identified as being fundamentally relevant to society, and where CIFAR's engagement can potentially make a large impact—be it through new technology, changes in policy, or in improving how we understand the world.

The purpose of continued Industry Canada support is to: attract and retain highly qualified researchers in Canada; build interdisciplinary research strengths in areas of importance to Canada; encourage closer links between Canadian and international researchers; and encourage greater private sector and provincial government investment in basic research activities.

1.2 Governance

CIFAR is governed by a Board of Directors, its associated committees, a Research Council, and a Council of Advisors.

The Board of Directors is responsible for the governance of CIFAR. The Board can have between 15 and 24 members^{Footnote 1} according to CIFAR's bylaws, and had 24 members at June 30, 2012 . Directors are distinguished individuals drawn from the Canadian business, research and professional communities. There are four committees of the Board: Governance; Advancement and Communications; Audit and Finance; and, Investment.

The Research Council is involved in the process of monitoring the overall direction and quality of CIFAR's research, advising the President on the approval and renewal of research programs, and providing insight into the Canadian and international research environment. The Council is appointed by and chaired by the President. At June 30, 2012 , it had 14 members^{Footnote 2} drawn mainly from the senior ranks of Canadian and more recently non-Canadian universities.

The Council of Advisors assists the Board of Directors and the President by providing advice as requested. The Council is composed of former members of the Board of Directors and the Research Council, and past participants of CIFAR's research programs. There were 27 members^{Footnote 3} of the Council as of June 30, 2012 .

1.3 CIFAR Research Model

The CIFAR research model is virtual in nature. The essence of CIFAR's approach is to develop a network of eminent researchers from different national and international institutions and to facilitate and support interaction among the researchers on research questions of broad interest at the interface of disciplines.

There are four components to CIFAR's research model:

1. Search for research questions: CIFAR starts with identifying the "big questions" and then formulates research teams best suited to tackle the questions. More traditional research organizations start by recruiting a team who then has some discretion in formulating the research questions. CIFAR does this by employing an active search and exploration process for new research topics using various forms of input from its stakeholders. Potential questions for CIFAR are reviewed by the President and senior staff of CIFAR, with advice from Research Council members, and questions of interest are then the subject of a series of workshops that refine the questions. A task force is struck to lead the initiative. Potential researchers, Program Directors and senior advisors are identified in the course of the search process that usually takes two to four years.
2. Approval process: Following a successful search process, a proposal is developed by the task force for the proposed program that outlines a research plan for the first five years, and proposes the identified researchers, Program Director and Advisory Committee members for the program. The Research Council assesses the program against the following criteria:
   • Adds significant value to its research area and answers big questions;
   • Does not duplicate an already well-understood area;
   • Addresses a research question that cannot be answered by a single researcher or any one discipline or by researchers at a single institution; and
   • Leads to advances that would positively impact Canada and the world, involve a significant number of Canadian researchers, and possibly draw researchers back to Canada
3. Research program: CIFAR establishes research programs as well as a research plan for a five-year term. It provides program members with five year, discretionary funds designed to facilitate their research as related to the program. CIFAR arranges and supports meetings of program members, usually two or three meetings a year for each program. Currently, CIFAR has 12 programs. Table 1 provides a brief description of these programs
4. Program review process: Programs are reviewed after four years into the five-year term through a formal peer review process by a panel of eminent researchers selected by CIFAR. The review considers the results of the program and the plans for the next five years as presented by the Program Director. In addition to assessing the programs against criteria of excellence and relevance, the peer review also evaluates the performance of individual program members both in terms of the excellence of their work and their overall contribution to the success of the program. The reviews can lead to a program's continuance or to its termination and the continuation or termination of the involvement of individual members.

1.4 Resources

The Government of Canada's Budget 2007 allocated $25 million over five years (2007–08 to 2011–12) to CIFAR, which was provided under a conditional grant arrangement. The federal grant represented approximately one-third of CIFAR's total revenues over the five-year period. The conditional grant was not directed to specific CIFAR programs, but rather as an addition to the total organizational budget, contributing to the existing objective of strengthening academic global research networks in Canada.

1.5 Expected Results and Logic Model

CIFAR's expected results, as outlined in the Funding Agreement, are to:

• Strengthen the preparation, attraction and retention of highly-qualified personnel in Canada;
• Encourage greater private sector and provincial government investment in basic research and development activities in Canada;
• Build interdisciplinary research strengths in areas of importance to the long-term scientific, economic and social development of Canada; and
• Encourage closer links between the research activities of Canadian researchers and international researchers.

The following logic model was developed by CIFAR and approved by its Board of Directors in 2008. The logic model (Figure 1) outlines the program's inputs, activities, and outputs, as well as the intended short-term, medium-term and longer-term outcomes.

Both the expected results and the relevant immediate and intermediate outcomes were used to guide the assessment of the core evaluation issue "achievement of expected outcomes".

Figure 1: CIFAR Logic Model, 2008

2.0 Methodology

This section describes the evaluation approach, the objective and scope of the evaluation, the specific questions that were addressed, the data collection methods used, and the limitations of the methodology.

2.1 Evaluation Approach

This evaluation builds on an evaluation and performance audit conducted by Hickling Arthurs Low (HAL) Corporation in 2010. The 2010 study reviewed CIFAR's activities from 2007 to March 2010 and included a limited assessment of two out of the three core issues of relevance (continued need and alignment with government priorities), as well as an assessment of performance. As this study was recipient-led there was no requirement to address the Treasury Board requirement to cover all five core evaluation issues.

The current evaluation study was managed by Industry Canada's Audit and Evaluation Branch (AEB) and updates the assessment and expands the scope to cover all five core issues of relevance and performance. Data collection and analysis was primarily conducted by Science-Metrix for core issues of relevance and by HAL Corporation for core issues of performance.

2.2 Objective and Scope

An evaluation of CIFAR is required under section 42.1 of the Financial Administration Act. In accordance with the Treasury Board Policy on Evaluation and Directive on the Evaluation Function, the purpose of this evaluation was to assess the core evaluation issues of relevance and performance^{Footnote 5}. The evaluation study covered the period from April 2007 to March 2012.

2.3 Evaluation Issues

Relevance

1. Is there a continued need for publicly-funded fundamental research? Do the research programs delivered by CIFAR respond to this need?
2. Does CIFAR align with federal government priorities and Industry Canada's strategic outcomes?
3. Is the funding of CIFAR consistent with federal roles and responsibilities?

Performance

4. To what extent has CIFAR-supported research contributed to building interdisciplinary research strengths in areas of importance to the long-term scientific, economic and social development of Canada?
5. Has CIFAR's international outreach encouraged closer links between the research activities of Canadian researchers and international researchers?
6. To what extent has CIFAR been able to strengthen the preparation, attraction and retention of highly-qualified personnel in Canada?
7. To what degree has CIFAR been able to encourage greater private sector and provincial government investment in basic research and development activities in Canada?
8. Is the delivery of research programs by CIFAR effective and efficient?

2.4 Data Collection Methods

This evaluation supplements the data collection methodologies that were employed in the 2010 evaluation and performance audit. The 2010 study included document, file and literature reviews; interviews (60); a survey of research program members; and a benchmarking exercise. The current study included a further document review, literature review, and interviews, and an updated analysis of the data collected through the benchmarking exercise.

2.4.1 Document Review

The review included documentation on: federal government priorities (e.g. Federal Budget, Speech from the Throne); Industry Canada priorities (e.g. Report on Plans and Priorities, Departmental Performance Reports), program documentation relating to CIFAR (e.g. Funding Agreement, TB submission); the 2010 CIFAR evaluation and performance audit; and other CIFAR documentation (e.g. annual reports, research reports, peer reviews). This methodology helped address all evaluation issues, in particular those issues relating to government priorities and federal roles and responsibilities. A list of the major documents reviewed is contained in Appendix 1.

2.4.2 Literature Review

This review focused on the need for fundamental research of the type CIFAR supports, the role of federal government in supporting research, and how CIFAR's research model differs from research funded through other federal sources (e.g. the granting councils). Included in the review were relevant reports of the Science and Technology Innovation Council, the Council of Canadian Academies (e.g. Report on the State of Science and Technology in Canada), Canadian scientific associations, reports of other countries on the rationale for funding fundamental research (including Canada, the US, Australia, Japan, the OECD, and Europe) and scholarly articles providing perspectives on fundamental and interdisciplinary research funding. Grey literature on the subject was also consulted. A list of the articles and reports reviewed is contained in Appendix 2.

2.4.3 Key Interviews

The objective of the interviews was to gather in-depth information including views, explanations and factual information that address the evaluation questions from a variety of stakeholder perspectives. In total, 20 interviews were conducted among the following stakeholder groups:

• CIFAR board members (2)
• Senior CIFAR management (4)
• CIFAR program directors (3)
• Federal granting councils (3)
• Selected provincial governments representatives (2)
• Finance Canada (1)
• Industry Canada (5)

The list of interviewees and interview guides are provided in Appendices 3 and 4 respectively.

2.4.4 Cost comparison study

To assess CIFAR's efficiency, a comparison was undertaken of CIFAR's operations with other institutes of advanced research that are similar to CIFAR's with respect to their size and research focus. The approach used in this evaluation was to update the information provided in the 2010 CIFAR evaluation and performance audit. Each of the participating institutes is recognized for producing high-quality research on complex matters. Accordingly, data for 2010-11 and 2011-12 (where available) was added to the 2003-2004 and 2008-2009 data reported in the 2010 study to provide a trend over a 7-8 year period. Information was gathered via the institutes' respective websites, financial statements and interviews. A summary of the financial information for each of the organizations included in the study is contained in Appendix 5.

2.5 Data Limitations

There were three key limitations that should be considered when reviewing the evaluation results. These limitations and mitigation steps are discussed below:

• Survey of CIFAR program members: A survey of CIFAR program members was not undertaken as there was insufficient time for administering one in the timeframe of the current evaluation. Additionally, program members had recently participated in a survey as part of the 2010 evaluation and performance audit, and the short interval between data collection periods would have placed a response burden on program members. With both these factors taken into consideration, performance data was updated to provide a longer trend analysis to mitigate against this limitation.
• Cost Comparison Study: A limitation of the cost study is that each institute's fiscal year is different, definitions varied between institutes (e.g., breakdown of operating costs into fundraising, program management), and comparable financial data for each institute was not always available for the same year. To mitigate against this, trends were analyzed over a seven to eight year period rather than a single data point, and when appropriate percentages were compared rather than absolute numbers. In addition, participating organizations all support a mix of researchers who are resident at the research organization whereas CIFAR does not have a dedicated research facility but rather it funds researchers who continue to work at their current facility. To mitigate against this, the evaluation considers the cost study as a comparison of research models rather than a comparison of research facilities.
• Performance Data: Overall, CIFAR collects high-quality and reliable performance data against their existing performance measures. This data was largely effective when used to report specifically against the expected results of CIFAR's funding agreement. However, the data from existing measures could not be used to fully assess the extent to which CIFAR's various activities and outputs contributed to the achievement of these results, particularly in the areas of knowledge sharing and international outreach. This limited the degree to which the evaluation study could fully capture the achievement of intermediate and longer term outcomes.

3.0 Findings

3.1 Relevance

3.1.1 Is there a continued need for publicly-funded fundamental research? Do the research programs delivered by CIFAR respond to this need?

Publicly-funded fundamental research responds to a need to foster innovation and provide economic and social benefits for Canadian society. Innovation performance is recognized by the OECD as a crucial determinant of competitiveness and national progress. By pursuing their best ideas, researchers help to build a broad base of knowledge, understanding and expertise that can seed longer-term social and economic opportunities. Many technological and social innovations, with deep and positive social and economic impacts, have had their roots in public research and came from findings that were impossible to foresee^{Footnote 6}.

The most recognized way in which fundamental research generates benefits and fosters innovation is through the creation of new knowledge, information and ideas that are available to firms and other 'users' such as public sector researchers and policy makers. However, a number of other significant socio-economic benefits are increasingly being recognized, including indirect, subtle, heterogeneous benefits. These benefits are difficult to track or measure, such as increasing a society's learning capabilities^{Footnote 7} or competence building.^{Footnote 8} Slater and Tang's review of studies on the benefits of publicly-funded basic research identified the following channels through which benefits from research flow into the economy and society:

• increase in the stock of useful knowledge
• supply of skilled graduates and researchers;
• creation of new scientific instrumentation and methodologies;
• development of networks and stimulation of social interactions;
• enhancement of problem-solving capacity;
• creation of new firms; and
• provision of social knowledge^{Footnote 9}.

Policy and scholarly literature, both in Canada and internationally^{Footnote 10}, supports the continued need for publicly-funded basic and applied research. This is largely due to the range of potential benefits derived from research activities, along with their recognized contribution to a strong knowledge economy. For instance, the Canadian government through the federal S&T Strategy^{Footnote 11} notes that public support for basic research is justified by the fact that the benefits to society are significant.

CIFAR responds to the continued need for fundamental research by developing and managing research programs designed to achieve knowledge breakthroughs. These programs focus on key issues which have the potential to contribute to improved health, environmental, societal and economic benefits for Canadians. Knowledge breakthroughs can lead to discoveries, inventions, and applications that play a key role in fostering technological and social innovation.

CIFAR supported fundamental research is largely aligned with "oriented basic research", which "is carried out with the expectation that it will produce a broad base of knowledge likely to form the basis of the solution to recognised or expected, current or future problems or possibilities."^{Footnote 12} For instance, research conducted in the Earth System Evolution program contributes to historical climate reconstructions that seek to boost the predictive power of climate models in the context of global climate change, and researchers in the Institutions, Organizations and Growth program investigate the "many factors that influence integration, cultural attitudes and ideologies" to help address issues of immigration and diversity.

Some CIFAR research programs also support the conduct of more "basic research", such as the Cosmology and Gravity program which aims to improve our understanding of the history and composition of our universe. This research is conducted without any specific application or use in view. Other research programs include a focus on more "applied research^{Footnote 13}". For example, both the Nanoelectronics and the Neural Computation and Adaptive Perception program involve research related to engineering applications, including the creation of new nanomaterials that could be used by the electronics industry and new algorithms and models that have applications for intelligent devices such as security surveillance systems.

The results of CIFAR's research are intended to lead "to new ways of thinking about and addressing central questions that confront humanity"^{Footnote 14}. This type of research is generally situated at the front end of the innovation pathway (i.e., conducting research leading to discoveries and inventions^{Footnote 15}). The timeframe for the practical use, adoption of new ideas, and/or commercial application of the research results of such work is often—but not always – long-term, and involves several additional steps and players along the S&T and innovation pathway.^{Footnote 16} Moreover, given the nature of any scientific explorations at the research frontiers, the results of such work are unpredictable, and potential uses and benefits may be both direct and indirect. As such, the main current models depict S&T innovation pathways towards value creation as being non-linear, complex processes involving inputs and activities by multiple players (institutions, sectors) and governed by multiple feedbacks (both positive and negative).^{Footnote 17 Footnote 18 Footnote 19}

To assess whether the design and features of CIFAR's research model continue to be based on current theory and practice for enabling knowledge breakthroughs, a literature review was conducted. The review confirmed that the CIFAR research model includes several features that are recognized as significant for enabling ground-breaking research. These features include:

• Long-term funding support for general research: CIFAR provides researchers annual funding for general research that is not tied to a specific project (e.g. direct salary support, teaching relief and support for graduate students and postdoctoral funding). Researchers can access this funding for up to five years (or longer if the program is renewed) and combine it with other sources of funding. Recent studies have confirmed that long-term explorative funding, combined with a variety of funding mechanisms, is better suited to enabling high-impact original research^{Footnote 20}. Further, a continuity of funding over the long-term is needed to develop innovative large-scale interdisciplinary collaborations.^{Footnote 21 Footnote 22}
• Facilitated and guided interdisciplinary collaboration: CIFAR's research programs focus on "important questions" and the research process is facilitated by CIFAR staff (e.g. regular meetings) and more formal oversight and advisory mechanisms (e.g. Advisory Committees). These features correspond with factors identified as contributing to creative research.^{Footnote 23} The literature also highlights the challenging domain within which CIFAR works, namely, the complexity of developing interdisciplinary programs that achieve academic excellence, escape disciplinary boundaries and fulfil a societal mission.^{Footnote 24 Footnote 25}
• International Collaborations: CIFAR identifies and brings together leading researchers from within and outside Canada, as well as providing program support for program members affiliated with institutions outside Canada. This also aligns with the findings of the literature which points to a need to collaborate on a global scale to enhance the quality of scientific research, to more effectively and efficiently address high-level global challenges, and more generally to reap the maximum benefit of research activities.^{Footnote 26}

CIFAR, through its research programs and activities, also responds to the need to foster innovation and provide social and economic benefits to Canadians by contributing to the development and attraction of highly-qualified personnel. Training skilled graduates has been identified as a significant way in which the benefits of research flow to society and foster innovation. CIFAR contributes to the development of the next generation of researchers in Canada through its activities in support of early career training. These include the development of programs for advanced study (winter and summer institutes), the Global Scholars program, and investments in graduate students and postdoctoral fellows made through funding provided to their supervising program members and opportunities to attend regular CIFAR program meetings.

3.1.2 Does CIFAR align with federal government priorities and Industry Canada's strategic outcomes?

The Government set its current agenda for supporting science and technology in 2007 with the introduction of its Science and Technology Strategy (S&T Strategy). Under this Strategy, the Government outlined its intention to foster three distinct Canadian S&T advantages: an Entrepreneurial Advantage, a Knowledge Advantage, and a People Advantage.

Historically, CIFAR has been aligned with the Knowledge and People Advantage of the S&T Strategy.^{Footnote 27} The Knowledge Advantage is based on the premise that Canadians must be positioned at the leading edge of the important developments that generate health, environmental, societal, and economic benefits. CIFAR's approach is consistent with supporting this advantage as it focuses on addressing globally important questions that will lead to advances that will positively impact Canada and the world. CIFAR provides Canadian researchers the opportunity to collaborate with leading international researchers and institutions in exploring and assessing questions of scientific, economic, and social importance to Canada. Further, CIFAR research programs are externally peer-reviewed every five years to ensure that the programs continue to conduct innovative and leading-edge research.

The People Advantage is based on the premise that Canada must be a magnet for the highly skilled people we need to thrive in the modern global economy with the best-educated, most-skilled, and most flexible workforce in the world. CIFAR is creating such an advantage by enhancing opportunities for graduates and attracting and retaining highly skilled individuals in Canadian universities. Specifically, CIFAR reports that each year it attracts into CIFAR programs 12 exceptional early-career researchers from Canada and internationally^{Footnote 28}. (Over the years, 26% have come from Canada and 74% from other countries.) This is recognized in the 2009 Progress Report on the S&T Strategy which identifies CIFAR as an organisation which brings together leading researchers from across Canada and around the world to work collaboratively on complex advanced research.

While CIFAR research is fundamental in nature, it is also making some contribution to supporting the Entrepreneurial Advantage as it "fuels entrepreneurial innovation through its international outreach, its industrial partnerships, its institutional spin-offs and its ongoing events and outreach"^{Footnote 29} Of particular note are the Institute's partnerships in the areas of nanoelectronics, computer sciences, machine learning, quantum materials and computing. For example, CIFAR researchers have developed technology and algorithms that involved collaboration/partnerships with high-tech firms (e.g. Google) to support potential applications.

Subsequent Speeches from the Throne and Budgets have reiterated the Government's commitment to these S&T Strategy priorities and to CIFAR specifically. Some examples of these references are outlined in Table 2:

The Program Alignment Architecture (PAA) for the department shows that CIFAR's activities are expected to contribute to Industry Canada's strategic objective of advancements in science and technology, knowledge, and innovation strengthen the Canadian economy under the program activity: Science, Technology and Innovation Capacity. CIFAR's vision, mission and expected results (see sections 1.2 and 1.6) are consistent with this program activity and strategic outcome. Further, through its research programs, CIFAR is clearly advancing science and technology in a number of areas (and being recognized for the quality of its researchers in these areas, see Sections 3.1.1 and 3.2.2), some of which are leading to innovations that will benefit the Canadian economy in the future.

3.1.3 Is the funding of CIFAR consistent with federal roles and responsibilities?

Industry Canada's mandate and responsibilities for Canadian S&T activities and policy goals stems from the Department of Industry Act, 1995, subsections 4(1), and specifically relate to "science in Canada". Section 5 states that these responsibilities include encouraging "the fullest and most efficient and effective development and use of science and technology" as well as fostering and promoting science and technology in Canada. In addition to this, section 6 of the Act states that the Minister shall "promote, assist and provide support services for, and investment in, Canadian industry, goods, services, science and technology". CIFAR's work in research and innovation, as well as its contribution to the development of highly qualified personnel contribute to the fulfillment of these duties.

The review of literature points to the role of the government in supporting research and innovation. S&T policy-related documents outlining the rationale for such support were observed in multiple jurisdictions, including Canada,^{Footnote 30} the US,^{Footnote 31} Australia,^{Footnote 32} Japan,^{Footnote 33} the OECD,^{Footnote 34} and Europe.^{Footnote 35} These policy statements and recommendations all identify a key role for direct and indirect public sector support for research and innovation in order to ensure the country/region achieves a strong and competitive economy. Additionally, many focused on key priorities, on excellence, on people, and on "openness" (including international cooperation and collaboration).^{Footnote 36}

While a comprehensive review of the theoretical rationale for public support of S&T is outside the scope of this review, recent studies^{Footnote 37} point to a number of rationales used for S&T and innovation policies, which can influence the objective and type of intervention and policy instruments that are used. For example, the traditional (neoclassical) justification for public funding of S&T is to correct "market failures." This is based on the assumption that firms have limited incentives to invest in S&T—particularly basic research—because of the difficulties, uncertainty and gestational lag (i.e., time from discovery to application) leading to the appropriation of returns from innovation, which produce a sub-optimal level of innovation. Moreover, the knowledge (information or technology) produced by research has more value as a public good rather than as a private good controlled by private interests.^{Footnote 38} More recently, evolutionary theories have been applied to this issue to understand the role and interplay of public and private support for S&T. These approaches generally adopt a broader perspective in that they consider that S&T activities provide wider economic benefits than simply through the production of knowledge (information or technology).

Specific to the Canadian context, the importance of government support for basic research was also clearly highlighted in the Expert Panel Report that reviewed federal support to R&D (referred to as the Jenkins report):^{Footnote 39}

The federal and provincial governments play an important role in fostering an economic climate that encourages business innovation–for example, by supporting basic and applied research and related training of highly qualified, skilled people […] the higher education and government sectors are key players in Canada's innovation system and complement the role of business.

The basic research performed by universities and other laboratories generates disruptive technologies that open up whole new industries. […] Given the foundational role that a strong post-secondary education sector plays in the Canadian system of innovation, this Panel (like the Expert Panel on Commercialization before it) urges the government to commit to investing in basic research at internationally competitive levels, and also to review and modernize the support for the total institutional costs of research.

The main federal government organizations involved in providing funding and support for research at post-secondary education institutions are the Canadian Foundation for Innovation (CFI) and the three granting agencies: the Natural Sciences and Engineering Research Council of Canada (NSERC), Social Sciences and the Humanities Research Council (SSHRC), and the Canadian Institutes of Health Research (CIHR). CFI supports infrastructure for research in all fields while the agencies support research in natural sciences and engineering, social sciences and humanities, and health, respectively.^{Footnote 40} Additionally, there are "tri-agency" programs such as the Networks of Centres of Excellence (in partnership with Industry Canada), the Canada Research Chairs Program, and others. Meanwhile, IC's S&T Partnerships sub-activity supports both organizations that conduct research and those that provide research services (i.e., CANARIE, Institute for Quantum Computing (IQC), Ivey International Centre for Health Innovation, Perimeter, Council of Canadian Academies (CCA)). The sub-activity also includes several third-party, non-profit organizations that administer funding to "ultimate recipients", namely research institutions and researchers (i.e., CIFAR, Genome Canada, Mitacs, Pierre Elliot Trudeau Foundation). A number of other government departments and agencies also have responsibilities for S&T and innovation-related activities, creating a complex S&T "ecosystem".

Within the federal ecosystem, CIFAR's research model is unique and complements programs offered by other organizations. For example, CIFAR is the only Canadian organization that offers funding and other types of support to implement a range of research programs involving interdisciplinary, international teams. The program's capacity to fund internationally-based researchers and make them full participants in research projects, as opposed to having to obtain funding from other sources to participate in work relevant to Canadian priorities, offers greater engagement within research teams.

CIFAR additionally accommodates a combination of different research approaches (i.e. basic and applied) and directs the study of a wide range of key questions that span health, social and natural sciences. Facilitating this work are longer timeframes for projects, allowing for more in depth study and more robust results. In many other cases, collaborative research programs explicitly target results in a relatively short term. For example, NCE Networks are required to include partners in other sectors (i.e., federal and provincial agencies, non-governmental organizations and private sector participants), and the NCE program review criteria specify that "Networks are expected to generate social and economic benefits by ensuring the rapid flow of ideas and innovations from researchers to Canadian receptors."^{Footnote 41}

CIFAR's provision of a wide range of support services for its research programs, including governance, networking, knowledge mobilization, communication and the development of new researchers, is also unique. For example, the Global Scholars Program (formerly the Junior Fellow Academy) offers paid fellowships, collaboration and mentorship opportunities with senior researchers, as well as ongoing participation in annual Global Academy meetings for three years following the fellowship. Additionally, program members meet in person two or three times a year and the associated interaction costs are covered by CIFAR.

Through these and other actions, CIFAR occupies a niche within the Canadian S&T ecosystem, offering a relatively unique approach to conducting both basic and applied research as well as enabling multidisciplinary work and international collaboration.

3.2 Performance

3.2.1 To what extent has CIFAR-supported research contributed to building interdisciplinary research strengths in areas of importance to the long-term scientific, economic and social development of Canada?

CIFAR brings together world recognized scientists and scholars from different disciplines to consider globally important complex questions that are of direct interest to Canada. The research conducted through these interdisciplinary collaborations is expected to contribute to the longer-term scientific, social and economic development of Canada. The potential impact of CIFAR's research in these areas is in part dependent on the dissemination and exchange of research results. As such, three areas were examined: the strength of CIFAR's research teams; the extent to which research results are contributing to social and economic impacts; and the effectiveness of CIFAR's knowledge sharing activities.

Strength of CIFAR research teams

The organization's focus on interdisciplinary research, coupled with its selection process for research questions, has led to achievements as well as international recognition in areas that are of importance to the long-term scientific, economic and social development of Canada.

Data collected from Essential Science Indicators by Thomson Reuters^{Footnote 42} demonstrates that over the period of 2002–2012, 43.1% of the CIFAR Fellows and Associates were ranked in the top 1% in their field, based on citations. In some programs, such as Child and Brain Development (formerly Experience-based Brian and Biological Development) and Integrated Microbial Biodiversity, 100% of the senior Canadian researchers are in the top 1%.

The strength of CIFAR's researchers is further enabled by their capacity to collaborate with experts around the world across different disciplines. External peer-review panels (see description of process in section 1.3) have unanimously noted the interdisciplinary strength of CIFAR's research programs, as well as the high value of the collegial relationships among researchers from different disciplines. The Social Interactions, Identity and Well-Being Program Panel in particular noted:

After reading samples of research papers produced by group members and conducting interviews with them over the span of two days, we agreed that none of us had ever seen an academic collaboration involving as many people of such high calibre working so enthusiastically together. The group's collegiality and dedication is impressive in light of its diversity, which includes scholars from markedly different backgrounds and career stages.

All nine panels reporting from 2007–2011 indicated that the programs reviewed were successful and indicated that the research teams led their fields internationally. For example, the panel for the Genetic Networks program notes that "There is no other group in the world that competes with the CIFAR Genetic Networks team in the analysis of genetic interactions at multiple levels. This is a defining challenge of genetics in this century. Canada leads the world in this area thanks to this team, and there is every reason to believe that it will continue to do so if properly supported." The panel further notes that the research attracts an extensive number of collaborators from around the world.

The work of other research programs was deemed to represent "extraordinary achievement" and to be critical to North American inquiry. Panels in general suggested that successful research is reported to further propagate the collaborative and interdisciplinary work that CIFAR seeks to foster. Interviews echoed this, indicating that CIFAR enables networking among experts that might not take place otherwise and that the program brings top researchers from around the world to Canada to meet on the organization's approved research topics.

This collaborative work is regarded as highly valuable by members, according to the survey results from the 2010 evaluation and performance audit. CIFAR research in both the natural and social sciences is viewed as having high recognition in other countries and it is the program model, that engages eminent researchers globally, that has afforded the program the international recognition across different disciplinary fields. Evidence of this recognition is in the number of awards and honours received by CIFAR members each year. For example, according to the Annual Performance Report for 2011–2012, CIFAR researchers received over 76 prestigious national or international peer-reviewed awards or recognitions during the year.

Impact of CIFAR research

CIFAR strives to ensure that its research projects are relevant to Canada through its project selection process. Specifically, one of CIFAR's criteria for approving a project is that it must be expected to lead to "advances that would positively impact Canada and the world."

The organization's major achievements, such as discoveries resulting in scientific advances and innovations, yield evidence that CIFAR-supported research is contributing to positive social and economic impacts as results are used outside of the research forum. Interviewees indicated that the research areas of health, cosmology and quantum physics have been strengthened because ofCIFAR. Further, they indicate that CIFAR's programs have an impact on scientific, economic and social development in Canada. For example, the neural interaction model of how the brain works, created by the Neural Computation and Adaptive Perception Program, led to the development of the pattern recognition functions that Microsoft is now applying to cell phones and other commercial applications.

While based in fundamental research, CIFAR does produce intellectual property as well, which has a direct benefit to Canada in such forms as spin off companies and advancements in health care. Additionally, some research programs have themselves made an impact in their respective fields. For example, the Institutions, Organizations and Growth program, launched in 2004, has made fundamental contributions to the understanding of which features of societies create environments that are conducive to investment in physical and human capital and R&D, and what explains the remaining variation in growth rates across countries and over time. The program is providing important insights into the role of technological progress in explaining the large variation among nations in levels and growth rates of income per capita.

The Successful Societies program seeks a better understanding of how social relations influence the capacities of individuals and communities for effective action underpinning collective development at both the national and local levels. While the Child and Brain Development program produced major advances in understanding the significance of early childhood development and how it affects health and developmental outcomes throughout a person's life. These findings have potential to significantly impact social policy development in Canada.

CIFAR is also making some contributions in areas of translating knowledge into commercial applications. Some examples of this are outlined in Table 3.

Knowledge Sharing

CIFAR's contribution to the economic and social development of Canada is enabled through dissemination and exchange of research results. In this regard, CIFAR holds a number of events each year that bring together CIFAR researchers and both public and targeted audiences drawn from business, government, NGOs and others in an interactive fashion to facilitate knowledge sharing and to showcase the value of CIFAR's research to society. From 2007 to 2012, CIFAR organized or co-sponsored 17 knowledge sharing events

An example of a recent event includes the Annual Symposium: The New Science of Child Development, held in November 2012. CIFAR and the Royal Society of Canada (RSC) presented a one-day Symposium entitled The New Science of Child Development, which explored the multi-faceted factors shaping the health, education, and psychological well-being of children. Another example is the Knowledge Circle evening held in March 2013. Entitled Why Nations Fail: The Origins of Power, Prosperity and Poverty, the event featured a debate focused on an alternative view on why the gap between rich and poor countries around the world continues to widen.

It is possible to further examine the participation of CIFAR's program members in knowledge sharing activities in general. For example, CIFAR reports that 162 program members reported active engagement in knowledge mobilization and public outreach and there were 1,190 invitations to speak at international meetings.^{Footnote 45} In addition, CIFAR publishes Reach Magazine, available online, that highlights the accomplishments of CIFAR researchers. The magazine is published approximately three times annually and has a circulation of over 4,200.

While there is ample evidence of a number of ongoing knowledge sharing activities, it is not possible to fully report on their effectiveness and impact without additional data. The development of performance measures designed to capture the impact and value of these events for participants and users of knowledge sharing mechanisms would assist in the measurement of CIFAR's success in this area.

3.2.2 Has CIFAR's international outreach encouraged closer links between the research activities of Canadian researchers and international researchers?

Evidence of closer links between the research activities of Canadian researchers and international researchers can be found in the demographics of program members. It drew program members and advisors from 13 countries in 2007 to 16 countries in 2012, representing 29 institutions in Canada and 75 internationally. This international engagement is prominent in the composition of program members, with 39% of members being based internationally and 48% of new members coming from outside of Canada in 2011–2012.

A total of 99 international members (30% of total members) work at institutions in the United States. The United Kingdom is the next largest collaborator with 13 researchers (4% of total members). Other nations have three or fewer program members. CIFAR is also advised by a significant proportion of international participants, with 79% of Advisory Committee members being appointed from other countries.

The program has engaged in international outreach activities through visits to other nations, collaborative efforts and through participation in academic forums over the past several years. CIFAR also engages in Canadian partnerships to better showcase Canada's research capacities and to attract attention on the international stage.

While the program does not collect information to measure the impact of these activities, the assessment did find some evidence pointing to effective practices. For example, in recent years, CIFAR set to work at fostering stronger relationships outside of Canada, particularly with India and China. The program undertook a series of visits to Indian advanced research agencies in October 2008, and collaborated with the Institute of Physics (IOP) of the Chinese Academy of Sciences on a three-day workshop in Beijing in 2010. The workshop brought together seven researchers from CIFAR's Nanoelectronics and Quantum Materials programs with their counterparts at the IOP. A number of IOP researchers have since been invited to attend CIFAR program meetings, and plans are underway to provide students supervised by both IOP and CIFAR researchers opportunities to collaborate in their host laboratories for short visits.

Further, in 2011–12, CIFAR held a number of events aimed at increasing the interaction between Canadian researchers and their international peers. This included the organization of a two-day Sackler Colloquium sponsored by the U.S. National Academy of Sciences in December 2011 on the Biological Embedding of Early Social Adversity whose results were published in the Proceedings of the Academy and featured a number of Canadian researchers. It also included significant participation of program members in the 2012 Vancouver meeting of the American Association for the Advancement of Science, with CIFAR partnering with over thirty Canadian organizations, including federal government departments to highlight Canada's commitment to science and innovation.

Additional analysis on CIFAR's performance in this area would be made possible by developing performance measures to track the continuum of activities intended to forge new relationships and the longer-term outcomes of these activities.

3.2.3 To what extent has CIFAR been able to strengthen the preparation, attraction, and retention of highly-qualified personnel (HQP) in Canada?

CIFAR aims to build a strong research community in Canada by involving young researchers in its programs. To this end, CIFAR established its Junior Fellow Academy in 2008, comprising 24 Junior Fellows, now called CIFAR Global Scholars, to support and celebrate gifted young researchers from different countries. These individuals, mostly based in Canadian universities, participate in CIFAR activities in a similar manner to members including attending research program meetings, interacting and collaborating with program members, and attending special events organized by the Junior Fellow Academy.

As of June 30, 2012 , eighteen Global Scholar alumni now have full-time positions as university faculty members or professional research scientists, demonstrating the value of the Academy to Canada's longer term research effort. Notably, three other alumni were already faculty members throughout their Global Scholar terms^{Footnote 46}. CIFAR notes that some Global Scholar Alumni have been elevated to program members.

CIFAR additionally supports the development of highly-qualified personnel in Canada via the supervision of graduate students. In 2011–2012, CIFAR program members supervised 1,165 graduate students (of which 208 graduated with their Ph.D. during the year) and 449 postdoctoral fellows^{Footnote 47}. In a survey of CIFAR members in 2010, over 65% rated CIFAR as having a notable positive influence on the quality of the training they provide to their graduate students and postdoctoral fellows^{Footnote 48}.

Apart from developing young researchers, CIFAR also seeks to attract and retain highly-qualified personnel from around the world. CIFAR indicates that 69 program members have been recruited to Canada since 1982 and 13 since 2007; between 2010 and 2012, seven program members were attracted to Canada from US universities (e.g. Harvard, MIT and Berkeley) and seven, who had offers from outside Canada, decided to stay in Canada. While these numbers may appear low in comparison to CIFAR's other figures with respect to numbers of researchers, it is important to recognize that these individuals include internationally recognized researchers, two of whom were awarded Canada Excellence Research Chairs. These individuals are in global demand and could have obtained positions in other prestigious universities elsewhere. Such decisions, to come to Canada or to remain here, reflect in part the international image of CIFAR in the global research community as well as the quality and reputation of Canadian research. Through CIFAR, researchers are able to forge collaborations that might not otherwise take place, thereby showcasing Canada's research environment to prospective experts.

Further, according to CIFAR data, 15 Global Scholars were attracted from abroad and were based in Canadian universities for their fellowship between 2008 and 2012. Over the same period, 17 Canadians chose to remain in Canada for their Global Scholar appointments.

The 2010 evaluation and performance audit found that the CIFAR model of facilitating interaction between eminent scientists and scholars in regular forums has drawn attention to the excellence of select groups at Canadian universities. The report suggests that this has been a factor in the decision of some outside researchers to accept appointments in Canada, or else for others already working in Canada, to remain.

Interviews conducted for this assessment confirmed this, indicating that CIFAR is a key factor in retaining and attracting scholars to Canadian institutions, and that it can be a rationale for staying in Canada as it enables international collaboration on a level that is not readily found elsewhere. One interviewee specifically stated that the networks CIFAR fosters could not be developed by academics on their own.

3.2.4 To what degree has CIFAR been able to encourage greater private sector and provincial government investment in basic research and development activities in Canada?

A distribution of CIFAR's revenue sources over the past five years is presented in Table 4. In brief, CIFAR's revenues have declined steadily from $20 million in 2007–08 to $13.3 million in 2011–12. This is largely due to a sharp drop in provincial contributions from the peak of $11.3 million in provincial contributions in 2007–08 from a one-time, additional allocation from Ontario, to $4.1 million in 2011–2012. This decrease may be attributable to the current economic context where governments (federal and provincial) are looking to reduce their expenditures.

With the exception of a slight dip in 2008–2009, private sector contributions have been trending upward over the past five years. This follows a conscious effort of the program to increase its funding ratio of public-private contributions from 80:20 to 70:30. Private sector contributions, ranging from $2.6 million to $3.6 million, have come from a variety of different sources (as is evident in Table 5). However, this increase in contributions from the private sector, and the corresponding shift in the ratio of public-private contributions, does not represent an overall increase in financial resources for the organization. While Table 4 shows that this target ratio has almost been achieved, this is more due to the decrease in provincial revenues than the increase in private sector revenues.

It is also important to note that not all private sector revenue sources have been trending upward. Revenues contributed by corporations fell from a high of $1.2 million in 2008–2009 to $0.52 million in 2011–2012. Over that same period, foundation contributions grew from $0.8 million to $1.49 million and individual donations ranged from $0.5 million to $0.8 million, partially offsetting the drop in corporate revenues. Other contributions, including donations made through campaigns and sponsorships, have varied significantly ranging from a low of $22,000 in 2008–09 to a high of over $1.0 million in 2011–12.

CIFAR has made an effort to increase the contribution of corporations, but this has proven to be a difficult task^{Footnote 49}. The decreases in corporate contributions since 2008–09 may be attributable to the economic downturn. The OECD reported in September 2012 that R&D investment by the private sector world-wide has decreased since that time^{Footnote 50}. CIFAR has determined that corporations are looking for greater brand awareness and public recognition for their support and the organization is responding by creating more opportunities for corporations to interact with researchers working in areas that overlap with corporate interests.

3.2.5 Is the delivery of research programs by CIFAR effective and efficient?

The effectiveness and efficiency of the delivery of research programs by CIFAR was assessed at two different levels: the effectiveness of using a third-party delivery mechanism; and the cost-efficiency of CIFAR's virtual delivery model compared to more traditional delivery models.

Effectiveness of Third-Party Delivery

Through the Science Partnership Directorate, Industry Canada manages federal contributions to a number of arms-length organizations to deliver on the government's commitments as set out in the S&T Strategy. The effectiveness of using CIFAR to support networks of Canadian and international researchers was assessed against the following criteria and principles for the use of foundations^{Footnote 51}: capacity to provide independent, non-partisan decision-making; capacity to enable the application of a peer review process; and capacity to provide guaranteed funding over the life of multi-year projects.

CIFAR is governed by a Board of Directors, composed of individuals from Canadian business, research and professional communities. An assessment of CIFAR's governance structures, conducted as part of the 2010 evaluation and performance audit of CIFAR, concluded that the Board is of high quality and provides CIFAR with a good corporate culture, and strong experience and capability to draw upon. This type of independent, non-partisan decision-making is seen as necessary to effectively support CIFAR's mission of framing and answering complex scientific, social and economic questions. Further, interviews with government officials noted additional advantages to this model of governance, which include the capacity to bring strong regional and sector-specific expertise to decision making and the opportunity for high-profile individuals to speak publicly about the benefits of the organization's work.

Scholarly peer review is generally viewed as essential to assess research quality and excellence throughout the research process. CIFAR has two mechanisms for peer review. The Research Council has a broad mandate to regularly monitor the overall direction and quality of the research programs and provide advice to the President. As well, using a more formal peer-review process, an external review of each program area is conducted after four years. CIFAR, largely because of its established relationships with relevant business, scientific and professional communities can effectively identify and draw on the direct experience and knowledge of experts in relevant fields.

Finally, each CIFAR research program is established for a five-year (renewable) term. Stable funding is necessary to support the long-term nature of fundamental research, which often takes several years to produce significant advances. As well, multi-year funding enables more thorough planning and assessment of research programs, provides stability for ongoing collaboration amongst team members, as well as increasing the ability of younger scholars to develop their research capacity through sustained interactions with more established scholars.

Cost-Efficiency of Delivery Model

While CIFAR maintains a head office in Toronto, Ontario and a staff of approximately 35 to support its programs, the research model used is "virtual" in nature. The term &virtual" refers to the fact that CIFAR does not have a physical location to house its researchers, as is the case for most other Canadian research institutes. In applying the virtual model, CIFAR develops a network of eminent researchers from different national and international institutions who remain at their home institutions. CIFAR then facilitates and supports interaction and collaboration amongst the researchers through in-person meetings.

In other programs of advanced research, the approach is different— researchers come to an institute and conduct research collaboratively. In comparison to this more traditional model, CIFAR's approach provides a high level of peer group interaction for the researcher, while allowing universities to retain their best and brightest. It is a cost-efficient approach, in that the amount that CIFAR pays to its researchers is relatively small, and leverages other sources of funding available to the researcher.

While CIFAR's model is unique both nationally and internationally, comparisons with other advanced research organizations could nonetheless be instructive for assessing cost-efficiency. The three organizations examined as part of the cost study each support leading-edge research and depend on non-government sponsors to varying degrees for financing their operations (see Appendix E for details). An assessment of total expenses compared to non-program related costs was conducted across these organizations. In 2012, CIFAR's proportion of non-program expenses was four to nine percent lower than that of the institutes with more traditional research support models.

4.0 Conclusions and Recommendations

4.1 Conclusions

Relevance

• There is a continued need for publicly-funded fundamental research as a means to foster innovation and to provide social and economic benefits to Canadian society. CIFAR responds to this need by delivering large-scale research programs designed to achieve knowledge breakthrough and activities in support of the attraction and development of highly-qualified personnel.
• The program is consistent with federal government priorities related to support for fundamental research and the attraction, training and retention of researchers as set out in the S&T Strategy. In addition, CIFAR's programs are aligned with Industry Canada's departmental strategic outcomes.
• Support for CIFAR is consistent with federal roles and responsibilities to encourage the development of science and technology. CIFAR occupies a niche within the Canadian S&T ecosystem, offering a unique approach to conducting both basic and applied research as well as enabling multidisciplinary work and funding for international researchers and collaboration.

Performance

• CIFAR has been successful in building interdisciplinary research strengths in areas of importance to the long-term social and economic development of Canada. These research strengths are evident in the results achieved by the organization's researchers as well as in the national and international recognition inherent in the many awards they have received. However, assessing the effectiveness of knowledge sharing activities would require more robust performance measures.
• International engagement in CIFAR's programs has increased. This has been achieved through a practice of drawing new members and advisors from different countries and engaging in international outreach activities. Additional insight into the effectiveness of these activities would require more robust performance measures.
• CIFAR has strengthened the preparation, attraction and retention of highly-qualified personnel in Canada by involving graduate students and post-doctoral fellows in its programs as well as by attracting and retaining high-calibre researchers in different disciplines to work and remain in Canada.
• Over the past five years, funding from provincial governments has decreased. Revenues have been relatively stable over the past three years as the organization has taken in more funds from private sources.
• Employing a third-party organization to deliver research programs of the type supported by CIFAR is an effective delivery mechanism. Further, the cost comparison exercise demonstrated that CIFAR's "virtual model" is efficient when compared with more traditional models used by other advanced research organizations.

4.2 Recommendations

Recommendation 1: The Science Partnerships Directorate should work with CIFAR to enhance their Performance Measurement (PM) Strategy. Particular attention should be placed on developing additional performance measures to better capture the effectiveness of CIFAR's activities and outputs with a view to assessing intermediate and longer term outcomes.

Management Response and Action Plan