Stakeholder Consultation Summary

Improving Performance and Data Collection of Business Accelerators and Incubators

Objective

This paper summarizes consultations with business and accelerator stakeholders that took place between late September and early November 2016. The paper is intended to support continued discussions with stakeholders as to how government, entrepreneurs, investors, and industry can work together to establish a national performance measurement framework for business accelerators and incubators (BAIs).

Background

The Government is engaging Canadians in developing an Innovation Agenda that can drive growth, create jobs, and allow Canadians to seize the opportunities of the transformative times we live in. The goal is to ensure that Canada will be globally competitive in promoting research, translating ideas into new products and services, accelerating business growth and propelling entrepreneurs from the start-up phase to international success.

A key element of the Innovation Agenda is to strengthen Canada's network of BAIs across the country. Most entrepreneurs now consider accelerators and incubators part of the start and scale journey. This has driven the demand for the launch of hundreds of BAI programs around the world, prompting questions on how differences across programs influence start-up performance.

The Department of Innovation, Science and Economic Development (ISED) invited BAI stakeholders to participate in three informal roundtables to discuss how government, entrepreneurs, investors, educators, and industry can work together to establish a national performance measurement framework for BAIs. The roundtables took place in Montréal (September 27th), Vancouver (October 4th), and Toronto (October 19th). In addition, ISED received input in writing or in conference calls as requested from stakeholders who were unable to participate in the roundtables.

A discussion paper was shared in advance with participants to frame the discussion into three themes:

• Objectives and Principles—Benefits of a performance measurement framework to governments, BAIs, and firms; and principles for its design and operation.
• Governance—Data exchange and consolidated collection
• Standardization of Data—Consistent performance metrics, data points, and definitions.

Overall 44 stakeholders provided input. There was diverse representation from regions, gender, sector, investors, academia, accelerators, and incubators.

Summary of Discussion

The Government consulted with BAI stakeholders actively collecting, tracking and reporting on data that enables evidence-based identification of gaps, opportunities and impact. Stakeholders provided input on: the benefits and challenges of a national performance measurement framework for BAIs including the sharing and standardization of data and, the viability of BAIs to share data to determine a) how BAIs influence start-up performance, and, b) how to establish an appropriate performance measurement framework for BAIs across Canada.

BAI stakeholders were forthcoming with suggestions on how performance collection and analysis could be used to most effectively support start-ups through accelerators and incubators. In relation to the three overarching themes of discussion, the following input was received.

Objectives and Principles

What should be considered so that key stakeholders in Canada's BAI ecosystem (BAIs, angel and venture capital investors, research centres, government programs, and entrepreneurs) could benefit from a national performance measurement framework?

Input Received:

Stakeholders can benefit from a national performance measurement framework.
Stakeholders agreed that there is opportunity to make better use of performance data on BAI and the firms they support. The goals of each stakeholder are different. BAIs could use this information to identify potential clients and benchmark their performance. Companies could use the data to identify BAIs to support their start and scale-up journey. Private investors could use the information to support funding decisions. Governments could use the aggregated information to inform policy direction to most effectively support innovative growth-oriented firms through business accelerators and incubators.

BAIs share similar principles but operate with different objectives.
Performance indicators that are relevant for some stakeholders are not relevant for others. Therefore, a national performance measurement framework would need to collect several categories of metrics. Factors specific to different BAIs need to be considered to appropriately benchmark and evaluate different organizations.

Consistent data indicators would help simplify reporting requirements by BAIs and firms.
Stakeholders suggested that a common set of indicators among the various government organizations supporting accelerators and incubators would help simplify reporting requirements. Some stakeholders suggested that they would like an online portal to enter, update and access their information.

Governance—Data Exchange and Consolidated Collection

What should be considered so that key stakeholders in Canada's business accelerator and incubator ecosystem would participate in data sharing as part of a national performance measurement framework and coordinated reporting on how BAIs influence start-up performance?

Input Received:

The BAI community is willing to work together towards a national solution.
There was consensus among all stakeholders that analysis and reporting on aggregate national performance of firms that participated in BAIs would be valuable. While there was willingness among stakeholders to participate in a data sharing as part of a national performance measurement framework, there was no obvious immediate solution for an organization or combination of organizations that could take on the role(s) of collecting, housing and analyzing the data on a national level.

Various data sharing approaches already in place should be taken into consideration.
Data collection and analysis efforts already in place across Canada, such as the Ontario Network of Entrepreneurs and British Columbia Innovation Council's data sharing and reporting initiatives, as well as experience within established reporting practices of some BAIs, should be leveraged and aligned to a national performance measurement framework.

The organization(s) responsible for consolidated collection and analysis would need to be neutral.
To support data exchange and consolidated collection, stakeholder concerns related to confidentiality and neutrality need to be addressed. The organization(s) responsible for collecting, housing and analyzing the data on a national level must be impartial given the potential impact on funding decisions. “Buy- in” of BAIs across Canada is critical to a national solution.

Stakeholders are interested in engagement with national counterparts.
Beyond performance measurement, BAI stakeholders from each region expressed a strong interest in engaging with national counterparts to exchange best practices on improving performance and data collection. Suggestions included having a content-driven conference for program managers of BAIs, an online forum for ongoing discussion, and a national workshop for performance metrics collection and analysis experts.

Standardization of Data

What metrics, data points and definitions should be considered to enable reliable data aggregation and comparison across Canada's business accelerator and incubator ecosystem?

Input Received:

Data aggregation and comparison across Canada should include metrics on firm growth.
Stakeholders agreed that the essential measures for success of start-up assistance programs are linked to the growth and competitiveness of the firms they support. If BAIs are successful in selecting and nurturing business ideas, incubated firms, on average should enjoy higher survival rates, grow revenues faster, employ more people, and attract more capital than non-incubated firms.  Ultimately, the long-term impact of BAIs benefits the overall ecosystem and the short-term impact is evident in firms successfully achieving business development milestones.

BAIs produce a variety of outcomes that are not captured in conventional firm performance metrics.
Performance indicators that are not typically reported on, specifically those related to company performance and ecosystem collaboration would show a more complete picture of how BAIs effectively support firms, and be more inclusive of BAIs with objectives specific to impacting demographic groups and niche sectors. These indicators include, for example: founder development, developing an entrepreneurship culture, collaboration among BAIs, and attracting new talent and investment to a region.

Reporting fatigue among BAIs and firms is increasingly become an issue.
As more and more firms participate in multiple start-up assistance programs during their start and scale up journey, reporting to different supporters is a challenge. A national performance measurement framework will need to explore options for one-stop reporting and alignment with business income tax reporting.

A common set of performance metrics could be reached given existing best practices.
There was consensus among stakeholders on the value of a common definition of metrics for BAIs. Stakeholders proposed developing a common set of a minimum viable set of metrics with consistent definitions with flexibility for additional metrics specific to the unique objectives of each BAI.

The following list includes suggested performance indicators that were consistent at each roundtable consultation.

Next Steps

In response to stakeholder advice and willingness to collaborate on a national scale to develop a performance measurement framework, the following next steps are proposed.

1.  Advance Discussion on Best Practices from a Regional to National Scale
Stakeholders across Canada demonstrated a strong interest for a forum for the BAI community to come together to share best practices on program content and reporting. There was consensus that efforts to share best practices be driven by the community itself. During consultations, stakeholders proposed convening BAIs through initiatives such as: an online forum for ongoing discussion, a content-driven conference for program managers of BAIs, and a national workshop for performance metrics collection and analysis experts.

2. Advance Discussion on a National Data Sharing Model
Several organizations demonstrated a willingness to share and report information on the performance of their organization and its clients with other BAIs across Canada. Participating organizations would explore a governance model and seek consensus on:

• A consistent set of data indicators,
• Definitions of data indicators,
• The nature of data sharing agreements,
• Data collection platforms, and
• Characteristics of a third party to compile and analyse this information.

3. Ongoing Data Collection, Reporting and Analysis
As ISED explores options for a long-term national solution for data collection and performance reporting of BAIs with stakeholders, it will continue to seek advice on a model to govern ongoing data collection and reporting, and a community-led, online resource of BAIs in Canada to report results to Canadians.

In the coming months, on behalf of the Government of Canada, ISED will engage with the BAI community to begin to implement these steps.