Evaluation of the Canada Business Network
This publication is also available online in HTML at www.ic.gc.ca/eic/site/ae-ve.nsf/eng/h_03696.html.
To obtain a copy of this publication or an alternate format (Braille, large print, etc.), please contact: The Audit and Evaluation Branch—Office of the Chief Audit Executive 613‑954‑5073.
Permission to Reproduce
Except as otherwise specifically noted, the information in this publication may be reproduced, in part or in whole and by any means, without charge or further permission from Industry Canada, provided that due diligence is exercised in ensuring the accuracy of the information reproduced; that Industry Canada is identified as the source institution; and that the reproduction is not represented as an official version of the information reproduced, nor as having been made in affiliation with, or with the endorsement of, Industry Canada.
For permission to reproduce the information in this publication for commercial purposes, please contact the:
Web Services Centre
C.D. Howe Building
235 Queen Street
Ottawa, Ontario, Canada
Telephone (toll‑free in Canada): 1‑800‑328‑6189
Telephone (Ottawa): 613‑954‑5031
TTY (for hearing‑impaired): 1‑866‑694‑8389
Business hours: 8:30 a.m. to 5:00 p.m. (Eastern Time)
© Her Majesty the Queen in Right of Canada,
represented by the Minister of Industry, 2014
Catalogue Number Iu4‑155/2014E
Cat. No. Iu4‑155/2014E‑PDF
Aussi offert en français sous le titre Évaluation du Réseau Entreprises Canada.
Presented to the Departmental Evaluation Committee on May 20, 2014
Approved by the Deputy Minister on June 3, 2014
Table of Contents
- Executive Summary
- 1.0 Introduction
- 2.0 Methodology
- 3.0 Findings
- 4.0 Conclusions and Recommendations
- Management Response and Action Plan
List of Abbreviations used in this Report
|ACOA||Atlantic Canada Opportunities Agency|
|AEB||Audit and Evaluation Branch|
|CanNor||Canadian Northern Economic Development Agency|
|CBN||Canada Business Network|
|CBO||Canada Business Ontario|
|CRA||Canada Revenue Agency|
|CED‑Q||Canada Economic Development for Quebec Regions|
|DPR||Departmental Performance Report|
|CFIB||Canadian Federation of Independent Business|
|FedDev||Federal Economic Development Agency for Southern Ontario|
|MSDA||Multi‑channel Service Delivery Assistant|
|PSSDC||Public Sector Service Delivery Council|
|RDA||Regional Development Agency|
|RPP||Report on Plans and Priorities|
|WD||Western Economic Diversification Canada|
List of Tables
|Table #||Table title|
|Table 1||Federal Funding Allocated to Managing Partners from 2008–09 to 2012–13|
|Table 2||National Website Trends, 2010–11 to 2012–13 FY|
|Table 3||Names Used by CBN and its Local Centres|
|Table 4||Funding Sources of all CBN Service Points by Category of Funds|
List of Figures
|Figure #||Figure title|
|Figure 1||Canada Business Governance: Program Articulation|
|Figure 2||Canada Business Network Logic Model (2013)|
|Figure 3||CBN Clients for Individual Assisted Interactions FY 2012–13|
|Figure 4||Web Visits for Online CBN Clients Including Regional Sites, FY 2012–13|
|Figure 5||CBN Officer‑Assisted Channels, Trends Overtime|
|Figure 6||CBN Service Area by Type of Centre|
|Figure 7||CBN Governance Structure, Evaluation Findings, 2012–13 Fiscal Year|
|Figure 8||Services Accessed During Officer‑Assisted Interactions, 2012–13|
|Figure 9||Value of CBN Funding in Today's Dollars, Including GOL Funds|
The Canada Business Network (CBN) dates back to the early 1990s. The current Network was formed in 2004 and renewed in 2009 with an ongoing $15 million annual budget. CBN's objective is to provide integrated information on federal, provincial and territorial government services, programs and regulations to established small businesses and new entrepreneurs in Canada. The Network is delivered across the country by six Managing Partners including Industry Canada, which is responsible for the National Office, and the Regional Development Agencies (Atlantic Canada Opportunities Agency, Canada Economic Development for Quebec Regions, the Federal Economic Development Agency for Southern Ontario, Western Economic Diversification Canada, and the Canadian Northern Economic Development Agency), which manage the program at the regional level.
Evaluation Purpose and Methodology
The purpose of this evaluation was to satisfy a commitment made at the time of the 2009 renewal. The evaluation addressed the core issues of relevance and performance in accordance with the Treasury Board Policy on Evaluation as well as issues identified by CBN program management. The evaluation focused on the five‑year period from 2008 to 2013.
An Evaluation Advisory Committee with representation from all Managing Partners was established to guide the evaluation process. The methodology included a document review, a literature review, an environmental scan, a review of program data in the client database, 72 interviews with program stakeholders, 15 focus groups with 109 CBN clients across Canada, online end‑user testing for the CBN web site, process mapping and financial data analysis.
The original rationale for the program is still valid. Small businesses are an important part of the Canadian economy and require assistance in navigating government complexity. In addition, CBN's objective of offering multi‑channel single window service is in line with how clients want to access information. The program's objectives align with the priorities and roles and responsibilities of the federal government to provide coordinated support for small business.
CBN has not been able to track client outcomes. Several lines of primary data collection were explored in order to measure client outcomes, but only one, a national series of focus groups, was feasible. As a result, there is insufficient evidence to conclude on some of the evaluation questions related to performance and CBN's impact on small business.
The Network is reaching an important subset of its target audience, micro businesses and business intermediaries; however, the evaluation is unable to conclude the extent to which the Network is reaching the broader small business population. Available evidence suggests low awareness of the program due in part to limited marketing activities, and confusion over the various names used to identify the Network across the country.
CBN reduces time and effort to access government information and services for many of those who find it; however, the need for improvements to the information and services provided to small businesses could be further explored.
Generally, clients express satisfaction with the information they receive from CBN. However, the evaluation cannot conclude the extent to which small businesses find the information and services CBN provides useful. Client surveys and other follow‑up, standardized training for CBN staff, quality control and a client complaint mechanism would potentially enhance the reliability and usefulness of CBN services.
The evaluation is unable to conclude on the impact of CBN on small business expansion and survival, but evidence suggests that the Network has helped some businesses start up or get on the right path quickly.
The CBN governance structure is complex considering the magnitude of the program. CBN's service offerings across the country are uneven and national performance measurement is a challenge for the network. As a network, CBN lacks the success factors common to integrated service delivery programs.
CBN has realized efficiencies in the face of declining resources. The decentralized governance and service delivery model have allowed for partner‑level efficiencies but prevented broader ones. These factors have also impacted the capacity to report on what federal funds are used for in areas with multiple funding partners, as well as the program's capacity to make strategic national decisions.
The environment in which CBN operates has evolved over the last 20 years. Best practices in integrated service delivery point to the need for leadership, governance, and integrated channel management. At the same time, the CBN has become increasingly regional in nature, with delivery models, service offerings, fees, and performance measurement activities varying by Managing Partner. As such, CBN is operating less as a Network, and more as a collection of regional programs providing services to small businesses. In this context, it is recommended that IC and RDAs:
- Review the design and delivery of the CBN, giving consideration to:
- Aligning service delivery with international best practices and client expectations; and,
- Re‑examining both the governance and service delivery models.
- Regardless of the outcome of this review, ensure that steps are taken to improve the federal government's ability to measure performance with standardized performance measures and financial tracking that enables reporting against the achievement of expected outcomes.
This report presents the results of an evaluation of the Canada Business Network (CBN).
The purpose of the evaluation was to satisfy a commitment made at the time of program renewal in 2009. The evaluation assessed the relevance and performance of the CBN in accordance with the requirements of the Treasury Board (TB) of Canada Policy on Evaluation. The report is organized into four sections:
- Section 1 presents the program profile;
- Section 2 presents the evaluation methodology;
- Section 3 provides the key findings related to the evaluation issues of relevance and performance; and
- Section 4 summarizes the study's conclusions and provides recommendations.
1.1 Program History
The program was originally launched as a pilot project in 1992 with three Canada Business Service Centres in Halifax, Edmonton and Winnipeg. In 1994, the Canada Business Service Centre Network, as it was known then, was launched across Canada by Industry Canada (IC) and the Regional Development Agencies (RDAs), including Atlantic Canada Opportunities Agency (ACOA), Canada Economic Development for Quebec Regions (CED-Q) and Western Economic Diversification Canada (WD). The network was founded on the principle of offering clients a one-stop-shop for information on government programs and services for businesses at both the federal and provincial level. Over the years, different centres brought their information holdings online and created individual web sites.
In 2000, under the Government On-Line initiative, the federal government provided funding to IC to create the Business Gateway, a federal Internet portal for business services. In 2004, Canada Business Service Centres came together with the Business Gateway to form CBN and to deliver "click," "call" and "visit" services to entrepreneurs across Canada. The CBN national web site was created and linked to each individual centre site.
The original service concept for CBN from the early 1990s was to have one walk-in centre, one telephone call centre, and one website in each province and territory. Following the 2009 renewal exercise, CBN was redesigned and provided with ongoing annual funding for each of the Managing Partners, which, at the time, included IC, ACOA, CED-Q and WD. The redesign included a new service concept that would: amalgamate the 14 federal-provincial/territorial websites into one national website; develop a new telephone management strategy including new service standards, protocols for telephone service as well as increased training for staff to deliver a more consistent quality of service. Other key changes from the 2009 renewal included the integration of BizPaL, an IC on-line information tool for business permits and licenses, and an increased focus on the growth of small businesses in addition to business start-ups.
This new service concept required stronger engagement by the four managing departments to establish shared federal priorities, greater collaboration among service centres and robust mechanisms for timely performance information and client input. Each Managing Partner for CBN was to carry out its own specific roles and responsibilities, but was ultimately committed to the CBN as a whole and the achievement of shared priorities.
The services provided by CBN are delivered through a decentralized governance structure. The objective is to ensure that evolving federal interests are met and managed to achieve uniform, cost-efficient service delivery with flexibility to adapt to specific regional needs as required.Footnote 1 The 2009 renewal documents outlined the roles and responsibilities of the CBN National Office and the Managing Partners. Specifically, the National Office is responsible for:
- Providing leadership for CBN;
- Developing new federal directions, service standards and policies in collaboration with the managing partners;
- Coordinating integrated planning and national performance reporting;
- Assessing national small business issues through research and consultations with key stakeholders;
- National marketing;
- Managing and delivering CBN's web presence;
- Hosting the content management system, the evolving suite of CBN online tools and national reporting/measurement tools;
- Coordinating centralized purchasing of electronic databases for research; and
- Developing a telephone management strategy with the Canada-Ontario Business Service Centre
Managing Partners, according to the 2009 renewal, are responsible for:
- Providing the strategic and operational direction to preserve and enhance the federal objectives of the CBN;
- Developing policies for the CBN;
- Delivering CBN in each of their respective regions;
- Developing federal/provincial/territorial agreements and collaborative arrangements, and negotiating the expansion of the Network in their respective regions;
- Overseeing communications and other support activities to ensure consistency across the Network;
- Ensuring fulfillment of federal objectives and agreed national priorities
- Ensuring centres comply with federal policies (such as the Official Languages Act and the Federal Identity Program) and national standards set for the Network;
- Ensuring sufficient funding is available to the centres to meet CBN objectives; and,
- Overseeing operating policies including performance evaluation and results reporting.
In 2011, the Canadian Northern Economic Development Agency (CanNor) joined the network as a Managing Partner, taking on responsibility for the three territorial-based CBN centres, previously managed by IC. In 2013, the Federal Economic Development Agency for Southern Ontario (FedDev) became a Managing Partner, taking on responsibility for the Canada Business Ontario Service Centre, also previously managed by IC.
1.2 Program Objectives
The objective of CBN, according to its foundational documents, is to provide integrated information on federal, provincial and territorial government services, programs and regulations to established small businesses and new entrepreneurs in Canada. It aims to provide small businesses with a single window or one-stop information service that is reliable, up-to-date, relevant, tailored to their needs and available in all regions of Canada. To achieve this objective, CBN aims to:
- Partner with other levels of government and business service providers in order to assist business clients across Canada, leverage opportunities, and minimize duplication and overlap;
- Reduce the burden, time and effort required by small businesses to find what they need by integrating information from multiple sources and levels of government;
- Focus efforts on the key priorities and current challenges to small business success and growth, and the currency and accuracy of information provided; and,
- Build awareness of products and tools to increase the number of clients accessing the CBN offerings.Footnote 2
1.3 Delivery and Governance of the Program
The delivery mechanisms for CBN include a national website, a toll-free telephone number, e-mail service, and a number of Service Centres across Canada. At the time the evaluation began in 2013, there were 16 Service Centres including four satellite centres. Of these centres, 11 were walk-in centres, four were call/e-mail centres, and one was a call centre that offered walk-in service by appointment (see Appendix A for a full list of CBN points of service). CBN is also moving towards providing services through alternate methods of access such as social media, video conferencing, webinars, YouTube and click to chat.
At the time of the evaluation, governance of the CBN was shared between IC (SBTMS) and the five other Managing Partners across Canada. Figure 1 below illustrates the governance structure for CBN as conceived by the program and includes the main or hub Service Centres in each region. Note that the governance structure changed in 2013 with the addition of FedDev and with the closure of walk-in CBN Service Centres in Atlantic Canada.
1.4 Other Stakeholders and Target Audience
In addition to the National Office and Managing Partners, key stakeholders for CBN include:Footnote 4
- Service Centres: The regional Service Centres are the main mechanisms through which CBN reaches out to businesses. The Service Centres are responsible for delivering core CBN services determined by the Managing Partners, other services negotiated with their respective partners, outreach efforts identifying regional business needs, promoting regional economic development, collaborating with the National Office, each other and other service providers to identify efficiencies and reducing duplication and providing the best client service.
- Provincial and territorial governments: Where partnership agreements have been reached with Managing Partners, some of the provinces and territories support the CBN Service Centres by providing funding and/or in-kind contributions and utilize the centres to deliver provincial-level programming and services.
- Other organizations, including non-governmental, not-for-profit and community organizations: The Network also works indirectly with other regional organizations including Community Futures Development Corporations, Business Economic Development Corporations, provincial enterprise/service centres and other community-based business support organizations to co-deliver certain services to small businesses or disseminate information to their clientele.Footnote 5
The CBN target audience comprises:
- Canadian small businesses: This includes entrepreneurs at various phases of their business cycle, including potential entrepreneurs, start-up businesses, and established Canadian small businesses.Footnote 6
- Business intermediaries: This includes service providers, business associations and community organizations. Usually, these organisations are listed on the CBN website and may work with CBN to provide services to the clients.
1.5 Program Resources
CBN has been funded since 1994 at an average of $15 million per year. Budget 2009 provided annual ongoing funding of $15 million for CBN, distributed between the Managing Partners as shown in Table 1. The original funding allocated to IC covered the operation of the National Office and Canada Business Ontario (CBO) Service Centre. In 2013-14, approximately $1.8 million of this funding was transferred to FedDev for the management of the CBO.
|Source: CBN foundational documents and 2011 Transfer Agreement of the CBN in Yukon, NWT, and Nunavut between IC and CanNor. IC received funds for Ontario and the North until CanNor became a managing partner in 2011-12 and FedDev became a managing partner in 2013-14. $1,828,314 was transferred from IC to Fed Dev in 2013-14 for the Canada Business Ontario Service Centre. The table is based on original funding allocations and does not include the 2012-13 reductions from the Deficit Reduction Action Plan.|
1.6 Logic Model
A logic model for CBN was developed in 2009 in consultation with the National Office and the other Managing Partners. This logic model was reviewed and revised for the evaluation to ensure that the program theory was accurately represented. The revised logic model was discussed and approved by the members of the CBN Evaluation Advisory Committee in June 2013 and is presented in Figure 2 below.
This section provides information on the evaluation approach, objectives and scope, the specific evaluation issues and questions that were addressed, the data collection methods, and data limitations for the evaluation.
2.1 Evaluation Scope and Objectives
The objectives of this evaluation were to address the core issues of relevance and performance in accordance with the TB Directive on the Evaluation Function and to address certain issues that CBN program management identified. The evaluation focused mainly on the most recent five-year period from 2008 - 2013.
2.2 Evaluation Questions
The evaluation addressed the following questions on relevance and performance. Note that questions on performance relate directly to the outcomes as expressed in the logic model.
- Is there a continued need for the CBN?
- Does the CBN align with the priorities of government and the strategic outcomes of the Managing Partners?
- Do CBN activities align with the roles and responsibilities of the federal government?
- Has the program reached its target audience?
- Has the consolidation of services under the CBN reduced the complexity, time and effort for small businesses when dealing with multiple levels of government requirements, programs and services?
- Is the information and service provided by CBN reliable, up-to-date, and useful?
- To what extent has CBN assisted small businesses to start up, survive and expand?
- How effective is the design and delivery of the CBN? Are there alternative delivery mechanisms that should be considered?
- Is the program approach (and its activities) the most efficient and economical way of achieving the expected results of CBN? Are there alternative approaches that should be considered?
2.3 Evaluation Approach
This evaluation took a goal-based approach focusing on the goals and expected outcomes of CBN as stated in the program logic model. The evaluation was managed by IC's Audit and Evaluation Branch (AEB). An Evaluation Advisory Committee (EAC) comprised of program and evaluation representatives of each of the Managing Partners assisted with the evaluation (with the exception of CanNor, which did not nominate an evaluation officer to the Committee). The EAC met five times during the course of the evaluation and approved the evaluation framework, data collection methodologies, data sources, and data collection instruments (e.g. interview and focus group guides).
2.4 Data Collection Methods
Evaluators used multiple lines of evidence to collect qualitative and quantitative data that was triangulated to arrive at findings and develop conclusions and recommendations.
The data collection methods included the following:
- Document Review
- Literature Review
- Environmental Scan
- Review of Program Data
- Focus Groups
- On-line End-user Testing
- Process Mapping
- Financial Data Collection and Analysis
The document review was conducted to gain an in-depth understanding of the program and to provide insights into both its relevance and performance. Key documents included federal Budgets and Speeches from the Throne, program foundational documents (e.g. Memoranda to Cabinet, TB Submissions, and program charter), relevant federal and provincial legislation, Reports on Plans and Priorities (RPP), Departmental Performance Reports (DPR), and previous audits and reviews. Program files were also reviewed including annual reports, performance reports, existing client survey reports, web metrics, marketing and communications strategies, and quality assurance reports.
A literature review was conducted to study the needs of small businesses, best practices in government services to small businesses, the program theory behind the "whole of government" or "no wrong door" approach to government service delivery, and innovation and alternatives to the delivery of services to small businesses.
The environmental scan addressed questions relating to the need for CBN. It reported on the demographics of the general business population in Canada, including geographical location, number of employees and type of industry using the North American Industrial Classification System. The scan also looked at business churn in Canada, including business births and deaths, and survival rate compared with other nations.
Review of Program Data
The review of program client data was led by AEB and conducted with the assistance of IC's Economic Research and Policy Analysis Branch in order to assess both relevance and performance. Data was retrieved from the Multi-channel Service Delivery Assistant (MSDA) database, the program's client data base designed to capture data on the assisted interactions with clients through the Service Centres. It is the program's main performance measurement tool. The data was cleaned and recoded (as per standard practice to ensure reliable, valid data for analysis) via extensive consultation and validation with program staff. The following data fields were processed for use in the evaluation research:
- Heard About (which is how clients heard about the Network)
- Client Type
- Method of Access
- Services Provided
Heard About, Client Type and Services provided were all used in the process mapping exercise described below, to produce a comprehensive picture of front line-client interface activities. Where possible, all data from centres have been included in the analysis using both micro data and summary reports, as required. Data on group interactions, outreach and web site usage were gathered separately.
MSDA was additionally used to recruit participants for client interviews and focus groups; develop a profile of CBN services and clients; and analyse the use of services and determine program outputs.
Interviews were conducted in order to gain a better understanding of the program, gather respondents' insights on relevance and performance, understand stakeholder and client perceptions on the effectiveness of the program, confirm information gathered by other methods, and gather references to additional documentation and data. Interviews with clients also assisted in the design of the focus groups. Interview guides were developed in consultation with the Evaluation Advisory Committee.
A total of 72 interviews were conducted with multiple stakeholder groups:
- Managing Partners including the IC National Office and RDAs (13)
- Service Centre Managers and Staff (15)
- Experts - academics, researchers, representatives of similar programs from other federal departments and provincial governments (9)
- Other Stakeholders – intermediaries, representatives of provincial and territorial governments, business associations, not-for-profit, community or non-governmental organizations (25)
- Clients (10)
Fifteen focus groups were conducted across Canada with CBN clients who had contacted a regional Service Centre. These focus groups were used to gauge client perception of the need for CBN's assisted channel services, the utility and quality of services received, level of satisfaction with the services, and the impact on clients' businesses.Footnote 8 Two focus groups were held in each of St. John's, Halifax, Quebec City, Montreal, Winnipeg and Edmonton. In each case, one group was held with new clients and the other with long-term clients in order to determine any difference in client experience or outcomes. In addition, one focus group was held in Toronto and one in Vancouver. Given that travel distances and expenses, individual phone interviews were conducted with clients in Northern Canada. Participants were randomly recruited from MSDA client contact data for all focus groups with the exception of the group in Vancouver and the four interviews in the North where the contact lists provided by the Service Centres were shorter. A total of 109 individuals participated in these client focus groups. Focus group participants were broadly representative of CBN's client base: 53% were business owners, 42% were business intenders (i.e. potential business owners) and 8% were third parties (i.e. intermediaries).
On-line End-user testing
Two additional focus groups were held using a quasi-experimental design to test potential client use of the internet to search for business information and to test their awareness of CBN. Half of each group was prevented from accessing the CBN website, with the other half having full access to CBN. Twenty participants (business owners and business intenders) were recruited from the general population and 14 attended (four were operating businesses and 10 were thinking about starting a business). One group was held in Toronto with six participants and the other in Ottawa with eight participants.
Process mapping was used to better understand the client experience when approaching the network as well as the partnerships and hand-off points at the various centres, which contributed to the assessment of performance of the network. Process mapping allowed evaluators to interpret the client data and map program activities to program outputs. Process maps were completed for 12 existing Service Centres and the national website service. The maps profile all the services provided by the centres and the numbers of clients served by each service. The maps were used in combination with centre profiles and other evaluation findings to create one service map for the entire network. A service blueprint was developed to outline the process clients follow, and the process the network follows backstage to support those client interactions. The network service blueprint may be found in Appendix E. Finally, a governance process map was created to accompany the service blueprint because the blueprint was insufficient to describe the flow of funding and decision making for the network.
Financial Data Analysis
Financial data was collected on all funding sources for the network (federal, provincial and private). Additionally, evaluators attempted to collect activity based costs from all the Service Centres and the National Office to develop a five-year profile of total salary dollars and operating expenditures by major activity category, as well as a one-year detailed profile of salary and operating expenditures by activity for 2012-13.
2.5 Limitations on the Data Collection
The following were limitations on the methodology.
1. Lack of client impact data
The evaluation was limited in its ability to assess the impact CBN is having on its clients. This is due a number of factors:
- There are many factors that can influence the program's ultimate objective (i.e. the success and growth of small businesses throughout Canada), making attribution of the impact of CBN information and services difficult.
- A survey to examine the impact CBN is having on its clients was not possible. Full client contact data was not sufficiently available to allow for a national survey of CBN clients, nor does a sample frame exist of "business intenders". This meant that only a general population survey would have been possible for this evaluation. AEB consulted with a number of survey research firms, who indicated that a general population survey would likely yield a very low response rate, and generate only a small amount of data that could be used in the evaluation. The EAC agreed that the evaluation would examine previously conducted national surveys to study client outcomes; however, these surveys did not yield significant outcome information.
- Evaluators also explored matching CBN client data with small business statistics using a common identifier (e.g. business number) with IC's Economic Research and Policy Analysis Branch. This was not feasible as many CBN clients are business intenders, and as such would not have had business numbers when they approached the Network.
In the absence of a national survey and data matching, the EAC agreed to use focus groups to assess client impact. A smaller recruiting sample frame for focus groups could be obtained more easily from the MSDA client database and richer data could be gleaned qualitatively, as evaluators could talk to clients about their experiences with CBN and its impact on their businesses. Evaluators sought to ensure coverage of all regions with CBN clients from almost all Service Centres across the country with 15 focus groups being conducted. Nonetheless, focus group samples cannot be considered representative of all CBN clients and participants' views are not generalizable. Findings and conclusions on the impact of the CBN on clients should be considered with this limitation in mind.
2. Data Quality Issues
The evaluation relied on data contained in the CBN's client data system (MSDA) to the extent possible; however, data limitations emerged:
- Data sets for Alberta and British Columbia were incomplete, as Service Centres had stopped using MSDA to record client interactions. In these cases, Service Centres had implemented their own client relationship management tools because MSDA did not meet their needs and there were no plans to modernize data collection on clients for the Network as a whole.
- While a significant amount of data could be retrieved from MSDA for most regions, there were no common definitions of variables across the network, resulting in inconsistent or multiple data entries for the same variables. As a result, significant time was required to prepare the data for analysis such that only one complete year of data was available for a number of variables.
To mitigate these limitations, evaluators worked closely with IC's Economic and Policy Research Branch, Service Centres and program staff to ensure high quality, accurate performance information on client interactions could be used to respond to the evaluation questions. In cases where standard variables existed across the network (e.g. method of access), up to five years of data could be analyzed for most Service Centres. The number of client interactions available for each data set is cited in the report. In addition, wherever summary data reports on client interactions were available for Alberta and British Columbia, evaluators used this data. The extensive work done on developing standard variables for collecting performance information on client interactions should be helpful to the program if a decision is made to modernize the program's client management system and database.
3. Recruiting for focus groups
The number of client focus groups that could be conducted in Vancouver was limited because the MSDA did not contain recent client contact data for British Columbia. Service Centres provided evaluators with a selected list of clients and intermediaries that had provided permission to be contacted by CBN. However, the list was not large enough to permit recruiting for two focus groups. Therefore, one focus group with clients (with six participants as opposed to ten) and a small number of interviews with intermediaries were conducted in Vancouver. Similarly, evaluators conducted four interviews with clients in the North, due to the lack of a larger client sample frame.
4. Incomplete data to conduct economy and efficiency analysis
The evaluators had planned to use activity-based costing to assess the costs of services and to produce some measure of efficiency such as costs per output (i.e. cost per client served) across the country. However, Service Centres were not able to provide evaluators with complete financial data by the five main activities as expressed in the logic model for one full year. Without full financial data on the network, the evaluation could not compare centres or assess the network as a whole. Further, those centres with funding from multiple sources were unable to differentiate the activities funded through federal support and other sources. Thus the evaluators are only able to provide a limited assessment of efficiency and economy.
3.1.1 Is there a continued need for the Canada Business Network?
Key Finding: The original rationale for the program is still valid. Small businesses are an important part of the Canadian economy and require assistance in navigating government complexity. In addition, CBN's objective of offering multi-channel single window service is in line with how clients want to access information.
The Policy Rationale for the Program Remains Valid
CBN was created to provide small businesses access to federal, provincial, municipal and private sector business information and services through a single window.Footnote 9 The need for the program was "based on extensive consultation with the business community…and aimed at solving a persistent problem namely low awareness and lack of access to government programs and services, coupled with the perception of complexity and run-around."Footnote 10
Since then, the policy rationale and justification for the program has been reiterated a number of times by government policy papers and reviews (e.g. IC, Growing Small Businesses, 1994; The Competition Policy Review Panel, 2008; foundational documents for CBN). A comprehensive review of CBN conducted for the renewal of funding in 2009 concluded that the program was still relevant and necessary to the small business community.
The Importance of Small Business to the Economy
With the small and medium-sized business population making up the vast majority of Canadian businessesFootnote 11, CBN's target audience is a vital contributor to the economy.Footnote 12 As of June 2013, there were 2.6 million businesses in Canada, over 1.1 million of which were small businesses with fewer than 100 full-time employees, and more than half of these were microbusinesses with less than 5 employees.Footnote 13 In total, small businesses represent 98% of all businesses with employees, and generate about 30% of Canada's Gross Domestic Product.Footnote 14 In 2011, small businesses were also responsible for $374 billion (approximately 30%) of Canadian exports.Footnote 15
In addition to these small businesses, there were about 1.4 million businesses that had an "indeterminate" number of employees. These micro businesses do not maintain an employee payroll, but may have a workforce consisting of contracted workers, family members or business owners. Thus the vast majority of businesses in Canada are micro businesses.
Small businesses also represent a growing share of Canadian employment. Between 2002 and 2012, small businesses created almost 91,000 jobs every year on average, and almost 78% of all private sector jobs created,Footnote 16 while medium-sized businesses and large business contributed 12.5% and 9.8% of all net new jobs respectively.Footnote 17 In 2012, small businesses employed over 7.7 million Canadians, which represented about 70% of the total labour force in the private sector.Footnote 18
While small businesses are an important part of the economy, they also have low survival rates. Specifically, the survival rates for small and medium enterprises for 2001 to 2006 were as follows:
- 85% of businesses survived one full year;
- 70% survived two full years;
- 62% survived three years; and
- 51% survived five years.Footnote 19
In general, micro-businesses (1 to 4 employees) have the lowest survival rates relative to other small enterprises. Each year, approximately 139,000 new small businesses are created, although only half will survive the first three years and only 25% are still operating after nine years. A high failure rate by the five year mark is common given the large number of challenges new businesses face during the first years of operation.Footnote 20
The low survival rates of small businesses point to the need for improving access to business information in Canada to make it easier for new entrepreneurs to start businesses. Efficient delivery of government programs and services is one factor that can contribute to small business growth and success. Assisting small businesses to get started or survive is thus a legitimate concern for government programs.
The Needs of Business Clients
Many studies have shown that small businesses need help finding government information and want government to improve its client services to business.Footnote 21 A 2007 survey of small businesses reported that participants said they looked to government for a variety of information, from market research to regulations, but had difficulty finding this information and knowing what level of government to deal with, and welcomed any steps to facilitate access.Footnote 22 The Canadian Federation of Independent Business (CFIB) reported in more than one study that a majority of their members expect government agencies to provide straightforward and consistent advice and they expect improvements to government customer service.Footnote 23 The interviews with stakeholders and focus groups with clients conducted for this evaluation confirmed that small businesses need assistance in finding government information and that they expect it to be integrated and consolidated in one place.
Interviewees said that a path finding service is essential to help small businesses, who are often overloaded with information, navigate government regulations and requirements. Several stakeholders said that government needs to play this path finding role because new businesses cannot afford to pay for this assistance. When asked about their initial search for information before discussing their views on CBN, most focus group participants said it was a struggle to find the information they needed to get a business started.
Evaluators also reviewed a number of studies, including some conducted by CBN Service Centres that surveyed small business people on their information needs.Footnote 24 The 2007 Ipsos Reid survey of small businesses reported that participants looked to government for a variety of information, ranging from basic market intelligence (e.g. competitors, labour market information, trends, market place demographics and statistics) to guidance on starting a business to information on regulatory requirements. Combined with the findings from the evaluation focus groups with CBN clients, the following are the most commonly cited information needs of small business people:
- Market research
- Licensing, permits and regulatory requirements
- Sources of financing
- Start-up information and tools
- How to expand and grow a business
The Need for Multi-Channel Single-Window Delivery
Most integrated service delivery programs in government focus on a no wrong door approach (i.e. there are multiple service channels as well as multiple locations) geared toward merging silos within either the municipal, provincial or federal governments.Footnote 25 Recently, two federal reviews of government support for business, the Aerospace ReviewFootnote 26 and Innovation Canada: A Call to ActionFootnote 27, recommended the use of a comprehensive, one-stop web portal to provide small businesses with high quality, timely advice and information and links on government programs. The value of these types of programs lies in their single-window design.Footnote 28 Typically, clients do not differentiate between government programs, departments, or jurisdictions, and find it difficult to navigate the multitude of government entities they must interface with on a regular basisFootnote 29. Interviews conducted for this evaluation confirm this view, underscoring that businesses do not distinguish between levels of government. Integrated service delivery reduces the complexity of government by allowing clients to access all or many required services through a single window.Footnote 30
The need for a single window approach to service delivery is further evidenced by the volume of federal, provincial, municipal, private sector and nonprofit programming for businesses across Canada. According to the latest draft of an IC inventory of federal programs and services to business, the Government of Canada delivers almost 300 programs with mandates that involve significant interactions with, or the provision of services to businesses. Many of these programs deliver more than one distinct service to their target client group.
The literature also suggests that single-window designs for government services provide multi-channel service delivery.Footnote 31 Single-window programs are expected to offer phone channels, e-mail, internet portals, and more to facilitate access for clients in both urban and rural areasFootnote 32 Footnote 33 Footnote 34 Footnote 35, as well as accommodate client preferences.Footnote 36 Footnote 37 For example, in the Institute for Citizen Centered Service Taking Care of Business 3 study, businesses demonstrated a preference for the telephone channel, despite the international trend of moving to online service channels. Furthermore, while Canadian businesses show a preference for using the web to get basic information, they then turn to the phone channel for clarification, verifying information, or for more complex inquiries.Footnote 38 This finding was reinforced by the evaluation focus groups where in most focus groups, clients told evaluators they usually start by looking at the information available on-line and when they "get stuck" they want to have access to a "live person" to answer questions.
In addition, the 2007 Ipsos Reid survey of small businesses found that the internet was seen as well suited to providing baseline market information, trends, and regulatory information, but was less capable of providing user-specific information and service. Telephone and in-person channels were the logical next step for many who access government information and services online.Footnote 39 CBN clients in most focus groups told evaluators they preferred to speak to a business information officer face-to-face rather than over the phone, although in a few focus groups, clients saw video communication through Internet platforms, such as Skype, as a good alternative to in-person. The desire for in-person service must of course also be compared with costs and level of use of such services.
In conclusion, the original rationale for the program is still valid. Small businesses are an important part of the Canadian economy and require assistance in navigating government complexity. In addition, businesses are looking for government assistance in finding different types of information, and CBN's objective of offering multi-channel single window service is in line with how clients want to access information.
3.1.2 Does CBN align with the priorities of government and the strategic outcomes of the Managing Partners?
Key Finding: CBN's objectives are well aligned with Managing Partners' strategic priorities to support small business, provide greater access to government information, encourage entrepreneurship and contribute to economic growth.
Alignment with Federal Priorities
The objectives of CBN are consistent with the priorities of the federal government as stated in recent federal budgets and speeches from the throne to: support small businesses, reduce red tape and administrative burden, and contribute to economic growth. The federal government has demonstrated a long-term commitment to small businesses through a number of programs, initiatives and funding commitments. Budget 2009 recognized that small businesses contribute to Canada's economic success by creating jobs and generating economic activity and growth.Footnote 40 The Budget further declared that the government would support small businesses through several programs, including Canada Small Business Financing Program, the National Research Council's Industrial Research Assistance Program and the Canadian Youth Business Foundation, in addition to providing $30 million over two years for CBN.
Since 2009, federal support for small business has been reconfirmed in multiple federal Budgets and Speeches from the Throne (see Appendix C). Most recently, in the 2013 Speech from the Throne, the federal government reaffirmed its commitment to creating jobs and securing economic growth, and recognized the key role that small businesses play in these areas.Footnote 41
Given its aim to improve access to information for small and medium sized businesses and make it easier for businesses to meet government requirements, CBN is also consistent with the government's goal of reducing administrative and regulatory burden on businesses in general. As early as 1994, the federal government identified administrative burden as one of the most important issues facing small businesses and launched a joint private/public-sector response to address a list of 100 "information burden irritants"Footnote 42. Building on the 1994 initiative, in 2004 the Government launched the Paperwork Burden Reduction Initiative (PBRI) to study and address the effects of administrative requirements on small businesses competitiveness in Canada. Reducing administrative burden remains an important objective for the federal government and appears throughout its strategic documents up to 2013.
Alignment with Managing Partners' Priorities
The objectives of CBN are consistent with the priorities of the managing partners related to improving access to government information, and encouraging enterprise, community development and economic competitiveness. The strategic outcomes, priorities and activities of all managing partners that relate to CBN are identified in Appendix D.
3.1.3 Do CBN activities align with the roles and responsibilities of the federal government?
Key Finding: The program's objectives align with the roles and responsibilities of the federal government to provide coordinated support for small business. While there is some overlap between the services offered by CBN and those of other federal, provincial and business service organizations, CBN has a unique role to play in integrating information on services for the small business community at large.
Evaluators found that the CBN objectives do align with the roles and responsibilities of the federal government under the Department of Industry Act of 1995. According to this legislation, the powers, duties and functions of the Minister of Industry extend to matters relating to "small business" and the Minister may exercise those powers to "initiate, recommend, coordinate, direct, promote and implement national policies and programs" to strengthen the national economy and promote sustainable development.Footnote 43 While support for small business falls within federal responsibilities, the Minister may also involve all levels of government (federal, provincial and territorial) in providing coordinated support for small businesses.
The legislation governing ACOA, WD and CED-Q also establish a role for them in "the establishment, development, support and promotion of enterprises, and more particularly small and medium sized enterprises" in each region.Footnote 44
Interviews completed for this evaluation also attested to the role of the federal government in providing support for small business. According to program stakeholders, while the provision of integrated business information and services is not exclusively a federal responsibility, it makes sense for the federal government to assume a leadership and coordination role. National Office staff and stakeholders saw the federal government as the only entity capable of providing the "global vision" needed to create a single window for small business information and services. Canada Business Service Centre staff and other stakeholders thought that federal leadership would allow for consistent and trusted source of information across the country.
While others are also providing programs and services to small business, including other federal departments, provincial governments and other service delivery organizations, CBN is unique in that it provides comprehensive information on government programs, services and requirements to the widest audience. Other federal government departments do offer a variety of services and programs to businesses (see Appendix B); however, these programs or services either have a specific focus or target specific segments of the business population (e.g. funding for aboriginal women's initiatives) and do not centralize access to this multiplicity of services. Service delivery experts interviewed for the evaluation saw a continuing role for the Network given the lack of integration of business programs and services at the federal level.
Several provincial governments are also providing service to business (e.g. Service BC, Service Nova Scotia, Service Ontario), but focus on their own jurisdictions. Some Service Centre staff and expert stakeholders thought that these services might overlap with the information and services provided by Canada Business Service Centres. Nevertheless, they also thought CBN's path finding/referral service was unique. Focus group participants also thought that the federal government was most suited to providing a service that integrates information from all levels of government.
3.2.1 Has the program reached its target audience?
Key Finding: The Network is reaching an important subset of its target audience, micro businesses and business intermediaries. However, the evaluation is unable to conclude the extent to which the Network is reaching the broader small business population. Available evidence suggests low awareness of the program due in part to limited marketing activities, and confusion over the various names used to identify the Network across the country.
This question relates directly to whether the Network has achieved its expected immediate outcome of "improved access to business related information" for its target audience. The CBN's foundational documents identify Canadian small businessesFootnote 45 and business intermediariesFootnote 46 as the target audiences for the Network.
CBN Client Reach
A review of the program's administrative data showed that the Network is reaching small businesses, potential businesses and intermediaries. In 2012-13, 81.2% of individual clients assisted by a CBN agent were businesses or potential businesses (see Figure 3 below). Another 8%, were third parties, including other government offices, CBN partner organizations, non-profit organizations and service providers, such as accountants or lawyers.
The MSDA client database does not track the number of employees in businesses interacting with CBN, making it difficult to get a clear picture of whether or not CBN is serving the small business audience at large. Random recruitment of focus group participants for this evaluation suggests that the bulk of CBN's clients are a subset of small businesses, micro-businesses with fewer than five employees. For example, all the clients in CBN client focus groups planned to start or were running micro-businesses. This is an important target group, since more than half of small businesses in Canada are microbusinesses. In addition, during interviews, Service Centre staff and stakeholders told evaluators that most of the Network's clients are people looking to start a micro-business in areas such as professional or personal services, small-scale manufacturing, and online or outlet retail.
- image note A
Note: This data includes single interactions recorded in the MSDA database and partial data from British Columbia. Data was not available for Nunavut and Yukon. No data for group interactions is available for this variable.
According to the administrative data compiled for the evaluation, in 2012-2013 there were 2.2 million visits to the CBN national website, and an additional 1.7 million visits to the eight additional regional websites maintained by the Service CentresFootnote 47 (See Figure 4).
In total, CBN Service Centres reported about 153,000 officer-assisted client interactions in 2012-13. About 36% of clients had accessed the network previously, while the remaining were new clients. Further, the channel by which clients are accessing services is changing. Despite the fact that clients often express a preference for in-person services, the program administrative data shows that overall there has been significant decline in the number of clients accessing CBN services in-person at its Service Centres or by phone since 2009 (See Figure 5). Across the country walk-in traffic is down by 56%, phone traffic has declined by about 40%; while use of e-mail has remained about constant.
- Footnote B
Note: British Columbia client data contained some gaps in this time series related to informatics changes. Due to the magnitude of client interactions in that centre, it is not included here as trends would otherwise be altered by these gaps.
By comparison, the number of clients accessing CBN services online appears to be increasing. As shown in Table 2 below, visits to the national site have increased since it was redesigned in 2009. Trends also show a transition to greater access of the website by mobile devices, although these still only account for about 6.2% of traffic to the site.
Other government departments serving businesses are seeing increased use of the internet and a declining number of in-person visits, mirroring the trends of the CBN.
Awareness of the CBN Program
There is some indication that awareness of the CBN is low. A national series of focus groups conducted by Ipsos Reid Public Affairs in 2008 concluded that awareness of Canada Business was low.Footnote 49 Interviews and focus groups conducted for this evaluation confirmed this is still the case. All stakeholder groups interviewed felt that awareness was low. Many focus group participants said that the services provided by CBN are not well known and clients in one focus group went so far as to say CBN is a well-kept secret. Still others, who were aware of their local Service Centre, were unaware of the www.canadabusiness.ca website or that a national Network of services existed.
In addition, in the two online end user tests conducted for the evaluation, business owners and business intenders unfamiliar with CBN were asked to search for a site that would provide them with comprehensive information for starting a business. Only two out of nine participants whose access to the national CBN web site was not blocked found a Network site. Note that one found the national site, www.canadabusiness.ca, and another found the local BC site, www.smallbusinessbc.ca. Four out of fourteen participants in total identified private sector sites selling publicly available information on government loan programs and business registration as the best source for business start-up information, unaware this is information that CBN provides for free.
In many focus groups, clients said that CBN needs to increase awareness of its services and in almost all groups clients said that the best way to do that would be through advertising, particularly on-line. Advertising campaigns had been successful in the past in raising awareness of the Network. In 2006, a research study found that a national horizontal "Services to Business" advertising campaign demonstrated some success in raising awareness about government services, including CBN.Footnote 50 In 2009-2010, the Network saw a spike in client interactions following publicity around program renewal and a modest online advertising campaign. Since that time there has been no national paid advertising for the network due to federal limits on advertising. The CBN National Office has mainly used its marketing budget to update and purchase promotional items for distribution by the local Service Centres in their outreach activities. It has also tried to use search engine optimizationFootnote 51, a blog and social media to raise the profile of the Canada Business website.
Most Service Centres have focused marketing and outreach efforts via their local centre web sitesFootnote 52, trade shows and outreach to other business service organizations due to restrictions on paid advertising by the federally-run centres and significant reductions in budgets for outreach and marketing. However, the staff at two non-profit centres, Small Business BC and the Business Link in Alberta, told evaluators they believe awareness of their centres is increasing. Both have developed comprehensive marketing strategies using partner outreach, their websites and social media. Small Business BC is one centre which has seen a significant increase in its outreach and marketing budget, largely due to private sector sponsorships for its small business awards campaign. In 2012, the centre raised $350,000 in sponsorships.
While regional marketing efforts by local centres may be showing some success, across the country, the lack of a national "Canada Business" brand is an impediment to effective national advertising. Table 3 below shows that the Network uses nine different names or naming conventions and four provinces do not use "Canada" in their name. During the conduct of the evaluation, evaluators found that few interviewees or focus group participants had heard of the "Canada Business Network" and many expressed confusion about the different names used by the Network. A single brand, such as that of Service Canada, would allow for the most efficient use of marketing dollars and a consistent on-line presence.
|National||Canada Business Network|
|Atlantic||Canada Business New Brunswick / Nova Scotia / P.E.I. / Newfoundland and LabradorFootnote 53|
|Quebec||Info Entrepreneurs (Montréal)
Ressources Entreprises (Québec)
|Ontario||Canada Business Ontario
Business Info Line (shared provincial phone line)
|Manitoba||Canada-Manitoba Business Service Center|
|Alberta||The Business Link|
|British Columbia||Small Business BC|
|Northwest Territories||Canada Business NWT|
|Nunavut||Canada-Nunavut Business Centre|
The evaluation was unable to conclude on the extent to which CBN is reaching the small business community at large. Available evidence from the interviews and focus groups shows the program is reaching micro businesses and intermediaries. However, the ability of CBN to achieve its expected outcome of "improved access to business related information" may be hampered by low awareness limits on advertising, marketing, and confusion over the various names used to identify the Network across the country.
3.2.2 Has the consolidation of information under CBN reduced the complexity, time and effort for small businesses when dealing with multiple levels of government requirements, programs and services?
Key Finding: Available evidence suggests CBN reduces time and effort to access government information and services for many of those who find it. However, CBN could further explore measures to improve information and services provided to small businesses.
This evaluation question relates to whether the Network is achieving its expected immediate outcome of reduced time and effort through improved integration of government services and information for clients.
A number of recent surveys found that most Canadian small business people think that government does not provide service in a timely fashion, that it does not effectively communicate new regulations and that customer service should be improved.Footnote 54 In addition, most small businesses are either unaware of government services for business or they still find it a problem to know who to contact.Footnote 55
For those who do find CBN, evidence showed the Network was easy to access and that it provided timely assistance. Recent CBN studiesFootnote 56 found the network offers good basic customer service by telephone and in-person. For example, most clients were served in the language of their choice, reached an agent on their first call (or were greeted by a staff person within five minutes of entering a Service Centre), and did not get transferred after reaching that agent. The studies also showed that the time it takes for clients to get responses to their questions, by telephone and e-mail or in-person is meeting expectations.
With respect to the Canada Business website, in the on-line user testing conducted for the evaluation, most business owners and business intenders who were introduced to the Canada Business website for the first time reacted positively, making it clear that they felt the national web site could save them time.
Usability testing conducted for the National Office found a 67% task completion rateFootnote 57 in 2010 and 74% rate in 2013. This rate falls just short of industry guidelines that suggest a general benchmark of 78%, though experts suggest organizations aim for a 90% task completion rateFootnote 58. Thus the time required finding information on the site has decreased, but there is still room for improvement.
In interviews, external stakeholders from most provinces and territories told evaluators that the Network significantly reduced the time and effort needed for small business people to access the information and services they require. Many focus group participants described the difficulties they faced in finding information relating to the government requirements, rules and regulations that applied to them. Some clients in many of the groups said CBN had helped them by expediting their search for information.
In addition, interviewees from Service Centre staff told evaluators that in the absence of CBN, clients would be required to spend more time searching for information. They said that small businesses would be frustrated and forced to pay consultants for similar services of uncertain quality. This was borne out by the results of the online user testing mentioned earlier, where participants identified sites selling government information that was available for free from CBN as the best sources of information they located online.
According to CBN's foundational documents, one of the ways CBN was to reduce the time and effort required by small businesses to meet government requirements was by integrating information from multiple sources and levels of government. The Network was to pull together information from federal and provincial/territorial government sources, while BizPaL, an on-line information tool for business permits and licenses which appears on the Canada Business site, was to provide some information on municipal requirements.
In most focus groups, clients told evaluators they definitely wanted to access information on federal and provincial/territorial requirements in the same place. Some added that this central location should also provide information on municipal government requirements.
The findings from the focus groups were mixed on the extent to which CBN was successful at providing consolidated information. In line with the results of the CBN national client surveys, some clients in every group felt that the referrals and information from CBN put them on the right track. Nevertheless, other clients in every focus group were critical of their experiences with the Network. They pointed to the lack of in-depth or specific information, the need for more experienced staff to respond to more complex questions, delays in getting a response to their questions, the lack of up-to-date statistical or other information. Clients in a few groups felt that they ended up in an endless referral loop where no one actually provided them with answers they needed to move their businesses forward. These clients felt that while the service was professional and courteous, it pointed clients in different directions which clients did not find helpful. These findings point to the need for broader, more in-depth surveys to better understand the experiences of CBN's clients as well as client outcomes.
Thus, evidence shows that CBN has made some progress in reducing the complexity, time and effort needed to access government information and services. CBN could, however, further explore measures to improve information and services provided to small businesses.
3.2.3 Is the information and service provided by CBN reliable, up-to-date, and useful?
Key Finding: Generally, clients express satisfaction with the information they receive from CBN. However, the evaluation cannot conclude on the extent to which small businesses find the information and services CBN provides useful. Client surveys and other follow-up, standardized training for CBN staff, quality control and a client complaint mechanism would potentially enhance the reliability and usefulness of CBN services.
This question examines whether CBN is achieving its expected intermediate outcomes of "better informed clients" and "improved client usage of business related information".
Many focus group participants were satisfied with the information and services they received. When asked by evaluators to write down their top three choices for small business information, nearly a third of all focus group participants listed CBN and most focus group participants said they would use the Network again.
In every focus group some clients said they received information they thought was accurate and useful, such as market research and referrals to further sources and they did not need to make further inquiries. On the other hand, as noted in section 3.2.2, there were also some clients who said the information was not useful because it was not specific, in-depth or up-to-date. These clients indicated they had to search elsewhere for the information they needed.
Findings from the interviews with centre staff and stakeholders and most focus groups suggested that market research is the Network's most valued service. The Network's National Office pays for licenses to market research databases and print reference materials that are used by the Service Centres. Some clients in most focus groups told evaluators that market information on potential buyers and competitors was the most important information that a business needs when starting out. Centre staff and stakeholders said that access to market research available through CBN is an important niche not being filled by other business service organizations.
As mentioned, the Network did undertake intercept surveys that assessed the functional aspects of client service. These surveys conducted in 2010Footnote 59 and 2011Footnote 60 showed that the majority of call-in clients (70% in 2010 and 78% in 2011) received all of the information or services they needed from CBN. Results were lower among walk-in clients. In 2011, only 54% of walk-in clients said they received all of the information or services they needed, down from 70% in 2010. Furthermore, surveys undertaken by local Service Centres show that the majority of clients were satisfied or highly satisfied with the information and services they received.
In 2013, Canada Business Ontario conducted an independent mystery shopper exercise to assess the quality of service provided by the joint federal-provincial Business Info Line. It found that call-in clients received responses quickly and felt the service was professional and easy to understand. The study found that 54% of clients received complete and accurate information; another 28% received accurate but incomplete information, while 17% of clients received incorrect information.Footnote 61 The percentage of incorrect information provided to callers was higher among calls answered by a provincial agent than a federal agent.Footnote 62
Overall, CBN has been unable to measure what happens to their clients following service. This was clear in a service blueprint evaluators developed to gain an understanding of the Network (see Appendix E). This service blueprint revealed that, as with many integrated service delivery initiatives, CBN cannot report on what happens to clients once they receive information or a referral. The Network has not been able to measure whether the client received the appropriate referral, whether client needs have been met or the outcomes of the services delivered.
Evidence from the interviews and best practices identified in the literatureFootnote 63 suggests that, in telephone interactions, a "warm hand-off" could go some way toward bridging this service gap. This involves a brief conversation between service providers while the client is on the phone to ensure the correct contact has been reached and to pass on information. However, such a change would require further study as it would have resource implications for the Network.
When CBN was renewed in 2009, one of the key initiatives was the development of a single national website that was to replace the 14 national/regional CBN websites. That website was to create a more consistent web presence for the network with centralized quality control where all content is reviewed and updated annually. Several of the centre managers and a service delivery expert told evaluators that canadabusiness.gc.ca provides a high quality path finder service for clients. Clients in the two on-line end user testing groups generally agreed. They found the site clearly and concisely presented the information that a person starting a business would need to know. Further, participants clearly stated a preference for having a single web site to work with. Several clients in the online end user tests also appreciated the fact that the main canadabusiness.gc.ca site is a government website. They felt that meant they could trust the accuracy of the content.
In 2012-2013, eight CBN Service Centres also maintained their own separate websites, each independent of national procedures and quality assurance. Service Centre staff told evaluators they require these sites for regional funding and identity, differing operational requirements and additional content provision to clients. The content on these sites varied from basic information on Service Centre services, partners and events, to sites providing detailed information and advice on the most common business questions such as business planning, export, and financing. Some sites are well integrated with the canadabusiness.gc.ca site, while others lack any prominent links to the national site. Thus, the on-line user experience with the Network varied depending on which of the nine Network sites a client visited.
A lack of common standards for quality control presents the Network with a considerable challenge to ensuring that clients are receiving consistent and reliable information across all channels. The training for front-line staff varies across the Network as each region or centre has developed its own orientation and training process. Canada Business Ontario, which runs a professional call centre, had roughly four weeks of training. A new agent had two weeks of formal in-class learning and testing, and then listened in on calls, did simulated calls and finally answered calls with an experienced officer by his or her side. In Manitoba, the local centre provided week-long one-on-one training with the Business Team Leader followed by two or more weeks of direct supervision. Small Business BC developed an employee handbook and provided a one-week orientation program including reading, job shadowing, attending its Education Centre seminars and an information quiz. Other centres did not have formal training programs but said they provided "ongoing" or "continuous training". Standard training of front-line staff coupled with adequate resources would assist their ability to deliver consistent and accurate information as well as to consistently track performance information.
Canada Business Ontario is also the only centre to develop a rigorous, formal process for assessing the accuracy and overall quality of its email and phone interactions with clients. A five-member team regularly reviews a representative sample of voice, email and research request interactions by individual officers to ensure they meet quality standards. If a call or email falls below "good" it will be reviewed with the officer in a one-on-one coaching session. Other Service Centres have implemented less systematic quality control practices such as occasional "secret shopping" and periodic monitoring of the content and quality of client interactions.
Further, the Network lacks a formal redress mechanism where clients can raise concerns about the service they receive. Service Canada, for example, has the Office for Client Satisfaction, which acts as a neutral organization that receives reviews and acts on suggestions, compliments and complaints regarding service delivery. Like the Office for Client Satisfaction, CBN could use the feedback it receives to regularly improve services, including accuracy and usefulness.
In sum, there is evidence from interviews and focus groups that CBN is contributing to its intermediate outcomes. However, the evaluation team was unable to confirm the extent of this contribution, due to a lack of client surveys or other follow-up activities to assess the usefulness of CBN information and services. Further, an inconsistent on-line experience, a lack of quality control mechanisms and the absence of standardized training impedes the program's ability to ensure it is delivering consistent, reliable and up-to-date information across the country. CBN also lacks a client complaint mechanism to allow for continuous improvement of services.
3.2.4 To what extent has CBN assisted small businesses to start up, survive and expand?
Key Finding: The evaluation is unable to conclude on the impact of CBN on small business expansion and survival, but evidence suggests that the Network has helped some businesses to start up or quickly get on the right path.
This question was to assess the extent to which CBN contributed to its ultimate outcome of "success and growth of small businesses throughout Canada". Some program staff and departmental managers questioned whether success and growth of small businesses was a realistic ultimate outcome of CBN, which provides information and services largely in the exploration and start-up phases of business development. Obtaining information on how to fulfill government requirements is only one of a myriad of factors influencing business success. Indeed, the logic model and performance measurement strategy developed by CBN in 2009 following program renewal cautioned that attributing the success and growth of small business throughout Canada to the program would be challenging considering other factors such as RDA investment, the state of the economy, other government policies, market conditions and firm capabilities.
That said, the evaluation focus groups did provide evidence of the impact of the program on clients. Some clients in every focus group said the assistance they received helped them move forward with their businesses or business ideas. Many said CBN's information and services provided a good starting point, put them quickly on the right track or helped them make an informed decision. Some said it encouraged them or made them more confident they were moving forward in a positive direction. The majority of focus group participants from the Edmonton and Winnipeg Business Service Centres said that CBN assistance had helped them move forward with their businesses or business ideas.
A small number of focus group participants said the assistance or information they received had a direct impact on getting their business started or in expanding their client base. A few others told evaluators that the information they received helped them to decide not to go into business thus saving them time and money.
While most interviewees did not know what impact CBN had on small businesses, some stakeholders provided anecdotal evidence. In Ontario, one small business owner who made repairs to precision tools was able to get a list of the 25 companies in the province that might use his service. In British Columbia, another stakeholder said that clients who went to Small Business BC for assistance with business plans were in a better position to access financing.
Some stakeholders, experts and centre staff believed if CBN was not there it would impact the success of small business. Concern was expressed that government complexity may dissuade some from starting up businesses.This is especially the case since other departments and levels of government are changing how they deliver small business services. For example, CRA closed all of its walk-in locations.
One small-scale on-line survey by a Service Centre attempted to assess the impact of CBN information and services on its clients. That study, conducted in Saskatchewan, found that just over half of respondents felt the assistance of CBN had an impact on starting their businessFootnote 64. The administrative data review also showed that among CBN repeat clients, one in six who contacted the network as a potential business later contacted the Network as businesses clients, showing they had gone on to start their own businesses.
While the evaluation cannot conclude on CBN's impact, there is some evidence that CBN has helped some clients move their businesses forward.
3.2.5 How effective is the design and delivery of the Canada Business Network? Are there alternative delivery mechanisms that should be considered?
Key Finding: The CBN governance structure is complex considering the magnitude of the program. CBN's service offerings across the country are uneven and national performance measurement is a challenge for the Network. CBN lacks the success factors common to integrated service delivery programs.
This question addresses the overall effectiveness of the program design and what aspects of the program could be redesigned or modified to improve effectiveness.
When CBN was launched in the 1990s service delivery via the internet in Canada was in its infancy. Walk-in resource centres with officer-assisted client service was the norm and supplemented with printed information provided by mail or fax. Technology, client expectations and corresponding best practices in service delivery have all evolved over the past twenty years—as has service delivery within the network. However, CBN has largely retained its original governance structureFootnote 65 while attempting to modernize its service delivery. This has resulted in varying service offerings across the country, barriers to tracking performance, and the program falling out of step with trends among other Integrated Service Delivery (ISD) programs.
CBN delivery models differ across the country
Three different models for delivery of CBN services have evolved across the country: federal, territorial and non-profit (see Figure 6). A total of five centresFootnote 66, in Ontario and Atlantic Canada, continue to be operated by the federal government and do not include formal walk-in/store front centresFootnote 67. Instead, clients are served either online or over the phone.Footnote 68 Call centre functions involve multiple service tiers in which simple inquiries and frequently asked questions are addressed by one level and more sophisticated inquiries, such as those requiring in-depth market research or special subject matter expertise, are answered by more qualified staff. Services offered tend to be those that involve the provision of information or referrals for business clients and market research.
Offices in the North are operated by territorial governments and jointly funded by the federal and territorial governments. In those centres, Business Information Officers respond to walk-in, phone and email inquiries. Since program activity is of a smaller scale representatives of CBN work for several different programs and services. CBN work typically involves provision of information or referrals for business clients and market research.
The remaining offices are non-profit or transitioning to non-profit centres, where CBN services are delivered by non-profit organizations. These centres all have walk-in/store-front locations, and respond to inquiries by phone and email. The non-profit organizations receive funds from a variety of sources for various tasks, some of which more closely resemble the work of business development organizations than integrated service delivery programs. For example, educational seminars, business counselling and networking events are commonplace in these centres. The non-profit organizations are governed by a Board of Directors and the federal government has a place on the board.
CBN Governance Structure
A review of the Network's governance structure demonstrated that it is complex, particularly for a program of low materiality with relatively straight forward objectives. This complexity is linked to the regional variation in the program that has emerged over time. At the federal level, the network is now overseen by six managing partner organizations. Decision-making for the network is vested in three, coordinated Managing Partner Committees (seen at the top of Figure 7) chaired by the IC National Office: an Assistant Deputy Minister (ADM) Committee, a Director General (DG) Committee and a Working Group comprised of middle-management-level staff. These groups brief up and down on matters pertaining to the network.
The National Office also leads another set of committees at the operational level (seen at the bottom of Figure 7) that involve Service Centre management and staff, as it seeks to implement national level initiatives determined at the Managing-Partner level. The committees touch on all aspects of the network, ranging from channel strategies to performance measurement and database subscriptions. For example, to make decisions on changing a variable in MSDA, all centres are consulted through these committees. Further, two of the RDAs, ACOA and WD, had additional management committees in 2012-13, geared toward addressing regional matters. In the West, this committee involves the provinces. In the East, the committee is comprised only of ACOA staff and includes the Managing Partner Working Group committee member as well as the four centre managers.Footnote 70
- Image note C
Note that the Ontario centre received a small financial contribution from FedDev to answer phone lines in 2012-13, but FedDev did not become a managing partner until the 2013-14 fiscal year.
According to the CBN Charter, regional-level partnerships are delegated to the RDAs while operational decisions are made by the Service Centres. Where operation of a Service Centre is contracted to a non-profit service delivery agent, a management board populated by a variety of partners and stakeholders may have the final say on operational changes that centre undertakes.
Interviewees from Managing Partners, Service Centre staff, experts and stakeholders said that the governance structure was overly complex and presents a significant impediment to the network's success. Some RDA staff said that the lack of a coordinated national focus has led to a lack of accountability and inconsistent services across the country. Similarly, National Office staff told evaluators that decentralized reporting and the network's complex governance structure have led to different delivery models and inconsistent branding across the country.
Uneven service offerings
While specific core services are not defined in CBN's foundational documents, the Network is generally to provide information and referrals on government programs as well as market research information, and outreach activities. A service mapping exercise conducted by evaluators showed that the program offers these services across the country. With respect to clients assisted by an officer, these "core" services (e.g. advising, research, promotion and other in Figure 8) represent about 73.1% of services delivered during in-person interactions in 2012-13, for which there was dataFootnote 71. It is of note that these activities comprise the full service offering of CBN's national web site. The remaining 26.9% of services are additional or non- core services and these are offered largely by the non-profit centres and differ from one region to the next (e.g. visiting lawyers offering advice, seminars on starting a business, hosting other government program outreach agents). Some regional web sites promote these value-added services. These services are defined in Appendix G.
- Image note D
Note that this chart represents about 120,000 services delivered but that clients may have received more than one service during an interaction. Some data from BC and Nunavut is missing due to differences in data collection.
The evaluation found that there is variation in the "core" government information, referral and market research services provided across the network. Interviews indicated that both Managing Partners and regional staff felt there was a lack of clear definition of the core services that local Service Centres are required to deliver. In practice, one significant difference in these "core" service offerings among centres is the fees charged by non-profit centres, particularly for market research information. The lack of standardization and consistency of what services may be charged to clients and at what price has led to uneven service costs to clients across the country.
The National Office makes available subscriptions to market research databases to each of the CBN Service Centres. In most provinces and territories, business service officers extract the relevant data from these databases and provide it to clients free of charge. Non-profit centres in Quebec, Alberta and British Columbia charge clients for market research services that use CBN databases. The Centres reported in interviews that they are providing clients with custom products or are charging for the time of an agent to retrieve and analyze the information obtained from the databases. This is intended to recover additional costs for human resources rather than the data itself. In all three provinces, clients in focus groups expressed dissatisfaction with having been charged for these services. In Quebec, both centres have developed agreements with federally-funded community economic development agencies whereby these organizations pay the centres to provide market research services based on these CBN-funded subscriptions.
Client dissatisfaction with respect to paying for CBN services was echoed in focus groups. Clients expressed frustration over being able to ask a few preliminary questions for free, but having to pay for more in-depth information.
In addition to the variation in services, there is some evidence that CBN clients received a different quality of information depending on how they access the Network. For example, a client intending to start a business may access a business start-up checklist from the national CBN website or one from the Small Business BC website. While the main CBN web site offers detailed information on how to complete each step and related links, the list on the Small Business BC site a one-page short description and a referral back to the centre's in-person/officer assisted services, as well as applicable costs for those services. There is no link from either checklist to the other.
Evaluators also observed that clients can receive different information depending on where they access the Network. For example, a client wishing to start a daycare would find ten different links on the national website, with links to information on programs in other provinces and territories. In Ontario and Alberta, a client would find a document identifying the relevant government programs, services and requirements. The same search on the BC web site yields a number of general documents that mention daycares. In the Quebec Service Centres, a search for "garderie" on Ressources Entreprises (Quebec City) yielded no results, while a search on Info Entrepreneurs (Montreal) yielded 36 results.
The presence of fees for other services is commonplace in centres within the non-profit model. These value-added services include teaching entrepreneurs how to run their businesses, business counselling, networking events, and in some cases, access to facilities such as physical space, IT equipment, and library services. Costs recovered for services are not standardized. While these value-added services and cost recovery through service fees were not part of the program's original mandate and service design, clients and staff alike expressed that they found value in many of these added service offerings. Nonetheless, the differentiation of program offerings across the country raises a question with respect to the magnitude of variation in regional needs and the universality and quality of services available for clients across the country.
Lack of national performance information
The complex governance structure, different service offerings, and differing service delivery models make it difficult for the Network to report on the performance of CBN as a whole. While the MSDA client database is capable of producing reports, these do not allow for in-depth analysis as definitions and activities varied across the country. In addition, data for two Service Centres and partial data for a third is not housed in the same application as all other centres. This data is, therefore, not easily accessible to the rest of the Network for performance measurement purposes (e.g. custom reports or comparison exercises).
In interviews, staff reported that the MSDA tool does not meet the needs of the Service Centres to manage client interactions. A few centres were moving towards regional client relationship management tools. However, most centre staff indicated that they would prefer a national-level client relationship management tool that would assist them in performance measurement and client service. It is also of note that there is general acknowledgement, both within the National Office and the Network's service centres that the application is dated and needs to be replaced.
Nevertheless, without common definitions, training for staff on its use, and buy-in from all managing partners and their centres, this presents a significant challenge. The lack of a definition of core network services and how they relate to funding agreements creates further issues for any common tool.
CBN and other Integrated Service Delivery (ISD) programs
A review of literature for approaches to integrated service delivery identified world-wide trends with respect to the centralization of service delivery management as well as integrated channel management. As outlined below, the evaluation found that there are success factors identified in ISD literature that are not fully present in CBN, nor is the network following citizen-centred service delivery evolution and trends.
ISD Governance and Partnership Models
In a study by the Public Sector Service Delivery Council (PSSDC)Footnote 72 Footnote 73, the majority of participating organizations identified leadership, governance and accountability as key to the success of their integrated service delivery and e-government programs. This is also underscored by the United Nations E-government Survey 2012Footnote 74 and other literature. For example, in a 2011 report, the Best Practices Sub-Committee of the PSSDC identifies the critical success factors of Service Canada, specifically referencing the organization's governance as a "key organizational consideration with respect to channel management."Footnote 75 The report states that:
Over a year ago, the management of the primary channel functions (In person, telephone and Web) were re-organized into a single new directorate….Although staff continues to be responsible for specific channel management; the new management paradigm has enabled channels to work together especially as it relates to channel migration and integration strategies and tactics.Footnote 76
One individual CBN centre is mentioned indirectly in the same study for its collaboration in the centralization of services to businesses via the government of Ontario. While the province and Canada Business Ontario are no longer in a funding agreement to answer calls, this same report does mention the relationship this program had with the centre at the time as a critical success factor as the "Business Info Line" integrated "over 160 fragmented numbers into a single 1-800 contact number for access to business information."Footnote 77
The literature review also showed CBN's partnership model differs from the majority of integrated service delivery initiatives internationally. Such programs normally have a centrally managed delivery window with partners participating insofar as they have a service that they want delivered through that service window. Partners do not necessarily participate in Tier-1 level work (basic requests, such as direct referrals or common services), but act as subject matter experts for more complex inquiries (as a Tier-2 or Tier-3 call centre for exemptions or matters requiring in-depth knowledge or research, for example). Interviews revealed that this model is common to other Canadian-wide ISD programs.
Further, Roy and Langford identify three possible organizing models for service integration: a single service provider, one or more "service integrators" operating markedly different programs for common clients under a common portal or a single window with a lead entity responsible solely for managing the initial interface or point of contact between a citizen and the government.Footnote 78 No literature was found in support of a model wherein individual government departments at the same level (e.g. federal) delivered the same ISD program and related government information different ways and under different brands.
Integrated Channel Management
In his 2008 report on Integrating Service Delivery, Kernaghan points out that CBN is a relatively small ISD, serving a small segment of the population. While he comments on the funding of CBN in the report, it is a neutral presentation of the model as something different than what is the norm for ISD initiatives. This report also addresses trends in service delivery and toward the end of the document, Kernaghan writes "Canada Business has experienced substantial movement to the online channel and has dropped its fax-back service for lack of demand. It is ironic that Canada Business also dropped an integrated telephone-web channel a few years ago because clients didn't seem to understand how it worked."Footnote 79 Considering this, the success of the National Office's current pilot to provide online chat services to clients on the national web site is a step in the right direction for the network.
Both locally and internationally, clients are moving to the Internet as their primary source for business information.Footnote 80 For example, one of Smart Service Queensland's critical success factors is its ability to encourage greater customer migration to the internet. Here in Canada the literature suggests that, "there is a substantial amount of evidence that Canadians want self-service delivery."Footnote 81 This is also echoed in the Taking Care of Business Studies mentioned elsewhere in this report. Canadian business clients specifically are increasingly turning to the Internet to access business information and then turn next to phone services to obtain more in-depth information.Footnote 82
Most ISD initiatives appear to be moving toward the use of one common phone number, website, and brand through which clients can expect to receive common services, for a common cost and at a common standard. By contrast, CBN's local Service Centres represent multiple service windows with unique brands and services, rather than a true single point of entry.
The United Nations E-government Survey 2012 identified the need for integrated channel management to ensure a consistent experience for clients accessing services by different channels such as web, phone and in-person.Footnote 83 The report asserts "that channels that ‘do not talk to one another' would result in a negative user experience" and eventually cause project failure due to low utilization. This is echoed in a Deloitte Research study, One Size Fits Few: Using Customer Insight to Transform Government, which states:
Without anyone managing the channel mix on an on-going basis, an organization ends up spending more than it should to maintain underutilized, or in some cases, outdated channels and misses out on cost-saving opportunities from actively migrating customers to lower-cost channels when they indicated a preference for using these channelsFootnote 84.
This is also stated by Duggan and Green who state that the Government of Canada has recognized that a true multichannel approach is required to ensure services meet the needs of CanadiansFootnote 85.
CBN lacks a national strategy that integrates all of its service channels. The web is the only channel with a national level strategy, a single, designated lead, and active, standardized quality assurance processes as well as clearly defined and well established performance measurement. The web channel also includes small page rating surveys that report on how useful content is for clients. However, as indicated earlier, local centres maintained eight other regional web sites during the 2012-13 fiscal year, citing other content needs and funding requirements by partners.
CBN's managing partners committed to developing and implementing a national telephone management strategy in 2009 at program renewal. It was to be implemented in 2010, but was never completed because Managing Partners could not reach an agreement on implementing such a strategy. While there has been some movement with respect to group training, channel management strategies remain largely unique between service models as the Network lacks the mechanism to facilitate coordinated national change.
Reflecting these trends, some interviewees from Managing Partners, Service Centre Staff and stakeholder groups thought that a national call centre would be an effective way to manage initial calls to the Network, with more detailed inquiries being passed on to more experienced agents in the regions. Evaluators were told that a national call centre would ensure consistent availability of bilingual services and allow for the extended service hours promised at program renewal.Note that Northern Service Centres and stakeholders did not see this as a viable solution in their region.
Implications for CBN
All this said, in interpreting best practice information on ISD programs, it is important to fully appreciate that there is a duality in CBN that presents a very clear challenge for the program with respect to managing its ongoing evolution. ISD initiatives are progressively moving toward a more centrally managed service delivery model, but the business development activities, such as offering seminars on book-keeping or access to business lawyers via CBN network partners (as identified in the service delivery models), fall outside the scope of this sort of national program activity and create a requisite for regional considerations moving forward.
Indeed, these activities seem to occur in regions where local partners are deeply integrated into in-person contact centres, most particularly in the non-profit model, which has developed partnerships around non-ISD programming. Business development activities have emerged within the network and are considered to be innovations within the program, often enabled by funding partners including provinces and private sector stakeholders. Further, focus groups and interviews indicate that these activities are valued by clients alongside CBNs core offerings. It is of note that it is the accommodation of such diversity that created the necessity for the previous performance measurement system—the MSDA—to have so much flexibility that the national analysis of its data holdings became so complex that robust reporting on individual channel costs is now impossible at the national level.
Further, the network reports that such activities are important undertakings with respect to local needs in the business community, outside of those responded to via the core ISD-style service offerings. Identifying a clear path forward with respect to any changes or amendments in service delivery strategy or even national performance measurement and quality assurance will be significant undertakings within the program's broader context and partnerships.
3.2.6 Is the program approach (and its activities) the most efficient and economical way of achieving the expected results of CBN? Are there alternative approaches that should be considered?
Key Finding: CBN has realized efficiencies in the face of declining resources. The decentralized governance and service delivery model has prevented broader efficiencies, the ability to consistently track costs across the Network, and make strategic national decisions.
This question examines the cost effectiveness of CBN and presents the funding and costing profiles of the program, the measures it has taken to improve economy and efficiency and some of the alternative approaches that should be considered.
CBN Funding: Declining Resources
The federal government has allocated approximately $15 million a year in funding for CBN since the 2000-01 fiscal year. In addition, the government provided a total of approximately $20 million in Government On-line funding to CBN from 2000-01 to 2005-06. However, since that time the value of federal funding the Network receives has gradually declined due to reduced program allocations. Further, taking inflation into consideration, the network had 25% less spending power in 2012-13 than it did in 2000-2001, (see Figure 9).
In 2012-2013, $12.9 million in federal funding was transferred to the Business Service Centres across the country and to the National Office. Funding sources of all CBN service points by category are shown in Table 6. In addition to this federal funding, the Service Centres have accessed other sources of funding including mixed federal/provincial/territorial funding (e.g. Business Development Canada Small Business Week funding), partnerships with other organizations, additional funding from the RDAs, fees for service, and private sector sponsorships. In 2012-13 this additional funding amounted to approximately $4.6 million. While in some regions, (British Columbia, Alberta, NWT and Nunavut), Service Centres continue to be closely partnered with their provincial/territorial governments, in others such as Newfoundland, Ontario, Manitoba and Saskatchewan there has been a loss of provincial support.Footnote 86
Within the non-profit service delivery model, federal funding counts for just over half of all funding. All other sources including provincial funding, other partner funding, sponsorship and revenue generation represent 44% of the Network's funding (see Appendix F, Table 2). This is compared with the Territorial model, where federal and territorial funding are almost 50/50, and the federal service delivery model, where federal funding counts for the majority of funding (81%). The non-profit centres have been able to raise additional funding, largely through partnerships, sponsorships and revenue generation activities. For the Funding profile of the IC National Office and each of the Service Centres, please see Appendix F, Table 3.
|Federal CBN Funding||$13,743,788||$13,487,373||$13,549,864||$13,536,577||$12,906,558||$13,444,832|
|Mixed Funding (Federal/Provincial/Private)||$124,500||$301,351||$349,242||$153,994||$270,244||$239,866|
|RDA Top Up||$2,458,783||$192,372||$82,500||$44,000||$78,600||$571,251|
|Other Partners, Sponsorship & Revenue Generation||$1,229,058||$1,859,622||$1,120,698||$1,175,038||$1,028,723||$1,282,628|
|Source: Information provided by RDAs and Service Centres. Figures include IC National Office. Federal CBN funding amounts exclude Employee Benefit Plans and 13% PWGSC accommodations cost for some centres, as well as funding retained by the RDAs.|
While CBN presents a relatively low federal materiality, a clear delineation between which of the network's activities were funded using federal dollars was not available to evaluators. The network should develop the capacity to demonstrate which resources and services federal investment in the program support.
Evidence of efficiencies
The evaluation found evidence that the program has worked to realize efficiencies despite declining resources. With the renewal of CBN in 2009, the National Office was able to reduce its costs with the transition to a new content management system for the national website. According to National Office staff, this change represented savings of more than a million dollars. Efficiencies elsewhere in the network vary regionally. In Ontario, the local centre contracts its call centre operations to a professional service, allowing for more efficient use of funds. The centre maintains management control by a federal office to ensure program objectives are adhered to in day-to-day operations.
In the Atlantic region, as previously mentioned, ACOA has closed its walk-in Service Centres. According to interviews and the document review, walk-in clients had been using the centres at a declining rate. Evidence from the literature review and interviews showed that storefront operations are by far the most expensive service channel, especially when Service Centres are located in urban areas where rental costs are high. Estimated average cost per client ratios using data for Atlantic Canada also indicated that the cost of in-person services in the region were high (e.g. over $300/client on average for the four provinces). ACOA has also moved to a regional call centre to answer Tier 1 calls; more complex inquiries are forwarded to CBN business service officers in each province.
In Saskatchewan and Manitoba, WD is moving to delivery of CBN services by a non-profit organization, as is already the case in Quebec, Alberta and British Columbia. A 2009 study that reviewed the options for possible service delivery models for the Canada/Manitoba Business Services Centre concluded that a non-profit model was the optimum model for the centre. It found that such a model would allow the centre to access funding for special projects, generate its own revenues, and allow for increased flexibility to realize operational efficiencies in staffing and procurement.Footnote 87
It is of note that a number of centres have either maintained funded partnerships with the provinces in their jurisdictions as well as forged new relationships that have generated income from other sources, bolstering the resources at the network's disposal in those areas. These contributions often bring with them additional services that are in turn offered locally by the centres.
Impediments to broader efficiencies
While the Network has realized efficiencies in different ways across the country, the evaluation found that the current program design stands in the way of further efficiencies at a national level and potential innovations.
Two measures to improve the Network's efficiency and effectiveness outlined in program renewal documents have not been realized. First, at program renewal, the plan was to move from 14 websites to a single national website. Part of the rationale behind this move was to reduce content duplication and reduce maintenance costs associated with documents that are rarely used, or do not correspond to business needs. This move was originally planned to be completed in 2010. However, as of 2014 the CBN's National Office and Service Centres continued to fund nine separate websites. It is of note that federal funds were not earmarked to produce these web sites in the 2009 program renewal but that regional service centres were unable to state definitively that no federal funds were spent on the additional sites. The assumption within the program is that the sites were created to respond to partnership requirements (and are therefore financed by those partners) or else to respond to the needs of the assisted service channels. It is known for certain that at least one additional site is financed using CBN federal funding.
The second measure envisioned at the time of renewal in 2009 was the implementation of a national telephone strategy. The strategy was in part designed to leverage the skills and experience across the Network to direct clients with specific needs to the appropriate expertise (regardless of where in the Network those expertise are located). The national telephone strategy was to be in place by 2010, yet as of 2014, the strategy had not been implemented.
In interviews, regional program staff highlighted the need for a new client relationship management tool because the current MSDA system was not meeting their needs. A couple of centre staff suggested a national client relationship management tool selected and managed by the National Office would be more efficient, as each region is now spending time and resources to purchase their own software/tools. Such individual systems also create inefficiencies by making it difficult to roll up and compare performance information on a national basis.
Program staff told evaluators that the ability to share best practices and local innovations between centres across the country was one of the most valuable features of the Network. However, evidence suggests that the complexity of the decision making processes hinders the Network's ability to innovate more broadly. A report was recently commissioned by the National Office to identify new trends and opportunities in services to business. It noted that, internationally, the trend is toward automation by putting more services on-line.Footnote 88 It also identified on-line chat and mobile service offerings as potential innovations for the network. The National Office had plans to partner with Canada Business Ontario to pilot an on-line chat option. However, the Network's multiplicity of websites would make this initiative difficult to roll out on-line chat across the country. Thus the National Office and local Service Centres have realized efficiencies in the face of declining resources and staff. However, the decentralized governance and service delivery model has prevented broader efficiencies, such as the realization of the single national website or telephone strategy. These issues also continue to impede the introduction of innovations across the Network.
4.0 Conclusions and Recommendations
The following are the conclusions to each of the nine evaluation questions.
Regarding the relevance of the program, the evaluation determined that:
- The original rationale for the program is still valid. Small businesses are an important part of the Canadian economy and require assistance in navigating government complexity. In addition, CBN's objective of offering multi-channel single window service is in line with how clients want to access information.
- CBN is well aligned with government priorities. The program's objectives are consistent with federal and Managing Partners' strategic priorities to support small business, provide greater access to government information, encourage entrepreneurship and contribute to economic growth.
- The program's objectives align with the roles and responsibilities of the federal government. While there is some overlap between the services offered by CBN and those of other federal, provincial and business service organizations, CBN has a unique role to play in integrating information on services for the small business community at large.
Regarding the effectiveness of the program, the evaluation determined that:
- The Network is reaching an important subset of its target audience, micro businesses and business intermediaries; however, the evaluation is unable to conclude the extent to which the Network is reaching the broader small business population. Available evidence suggests low awareness of the program due in part to limits on marketing activities, and confusion over the various names used to identify the Network across the country.
- Available evidence suggests CBN reduces time and effort to access government information and services for many of those who find it; however, the CBN could further explore measures to improve information and services provided to small businesses.
- Generally, clients express satisfaction with the information they receive from CBN. However, the evaluation cannot conclude on the extent to which small businesses find the information and services CBN provides useful. Client surveys and other follow-up, standardized training for CBN staff, quality control and a client complaint mechanism would potentially enhance the reliability and usefulness of CBN services.
- The evaluation is unable to conclude on the impact of CBN on small business expansion and survival, but evidence suggests that the Network has helped some businesses to start up or quickly get on the right path.
- The CBN governance structure is complex considering the magnitude of the program. CBN's service offerings across the country are uneven and national performance measurement is a challenge for the network. CBN lacks the success factors common to integrated service delivery programs.
- CBN has realized efficiencies in the face of declining resources. The decentralized governance and service delivery model have allowed for partner-level efficiencies but prevented broader ones. These factors have also impacted the capacity to report on what federal funds are used for in areas with multiple funding partners, as well as the program's capacity to make strategic national decisions.
The environment in which CBN operates has evolved over last 20 years. Best practices in integrated service delivery point to the need for leadership, governance, and integrated channel management. At the same time, the CBN has become increasingly regional in nature, with delivery models, service offerings, fees, and performance measurement activities varying by Managing Partner. As such, CBN is operating less as a Network, and more as a collection of regional programs providing services to small businesses. In this context, it is recommended that IC and RDAs:
- Review the design and delivery of the CBN, giving consideration to:
- Aligning service delivery with international best practices and client expectations; and
- Re-examining both the governance and service delivery models.
- Regardless of the outcome of this review, ensure that steps are taken to improve the federal government's ability to measure performance with standardized performance measures and financial tracking that enables reporting against the achievement of expected outcomes.
Management Response and Action Plan
Preamble: The environment in which CBN operates has evolved over last 20 years. Best practices in integrated service delivery point to the need for leadership, governance, and integrated channel management. At the same time, the CBN has become increasingly regional in nature, with delivery models, service offerings, fees, and performance measurement activities varying by Managing Partner. As such, CBN is operating less as a Network, and more as a collection of regional programs providing services to small businesses.
|Recommendation||Management Response and Planned Action||Management Accountability||Action Completion Date|
|It is recommended that the managing partners:
1. Review the design and delivery of the CBN, giving consideration to:
• Aligning service delivery with international best practices and client expectations; and
• Re‑examining both the governance and service delivery models.
|Response: Agreed. The CBN federal managing partners have taken first steps towards this recommendation through the formation of a team of CBN program representatives, partners and representatives from key federal service delivery organizations that were tasked to collaboratively explore future state and delivery options going forward for the program. Options were presented to managing partners March 2014.||CBN Federal Managing Partners Directors General/Assistant Deputy Ministers||October 31, 2014 for decision on recommended approach forward.|
|Action Plan: The federal managing partner will further explore future service delivery options in detail, including ways to improve governance, clarify roles and responsibilities, and respond to client needs. The federal managing partners will collaboratively develop and present a recommended approach forward. An implementation plan to address the recommended approach will follow.||CBN Federal Managing Partners Directors General/Assistant Deputy Ministers IC Assistant Deputy Minister, Small Business, Tourism and Marketplace Services to report on Implementation Plan.||Implementation plan prepared for March 31, 2015.|
|1. Regardless of the outcome of this review, ensure that steps are taken to improve the federal government's ability to measure performance with standardized performance measures and financial tracking that enables reporting against the achievement of expected outcomes.||Response: Agreed. Action Plan: The federal managing partners will review existing performance measures, tools, and systems to ensure we can better measure performance, report against expected outcomes, as well as to track costs. This will include a review and revision of the Performance Management Strategy for the CBN according to the Treasury Board Secretariat guidelines, the development of standard definitions, and revised reporting templates, processes and systems.||CBN Federal Managing Partners Directors General/Assistant Deputy Ministers IC Assistant Deputy Minister, Small Business, Tourism and Marketplace Services to report on implementation.||March 31, 2015|
- Date modified: