Language selection

Search

Archived — Final Evaluation of the Community Futures Program in Ontario

Final Report — October 2008

Prepared for: Industry Canada/FedNor
Prepared by: Government Consulting Services. Tabled and approved at DEC.


Table of Contents

Executive Summary

1.0 Introduction

2.0 Evaluation Methodology

3.0 Evaluation Findings

4.0 Overall Conclusions

IC/FedNor Management Responses to the 2008 Community Futures Program Final Evaluation

top of page

List of Tables

top of page

List of Figures

top of page

List of Acronyms

  • ACOA — Atlantic Canada Opportunities Agency
  • BDC — Business Development Bank of Canada
  • BR+E — Business Retention and Expansion
  • CED-Q — Canada Economic Development for Québec
  • CED — Community Economic Development
  • CF — Community Futures
  • CFDC — Community Futures Development Corporation
  • CFO — Community Futures Organization
  • EODP — Eastern Ontario Development Program
  • FedNor — Federal Economic Development Initiative for Northern Ontario
  • Gs&Cs — Grants and Contributions
  • GCRS — Grants and Contributions Reporting System
  • GCS — Government Consulting Services
  • GDP — Gross Domestic Product
  • IC — Industry Canada
  • NODP — Northern Ontario Development Program
  • NWOIP — Northwestern Ontario Investment Pool
  • O&M — Operations and Management
  • OACFDC — Ontario Association of Community Futures Development Corporations
  • OLMCs — Official Languages Minority Communities
  • OMAFRA — Ontario Ministry of Agriculture Food and Rural Affairs
  • R&D — Research and Development
  • RDA — Regional Development Agency
  • RED — Rural Economic Development
  • RMAF — Results-based Management and Accountability Framework
  • RPP — Report on Plans and Priorities
  • SC — Steering Committee
  • SEB — Self-Employment Benefits
  • SICEAI — Softwood Industry Community Economic Adjustment Initiative
  • SME — Small-and Medium-Sized Enterprise
  • TEA — The Exceptional Assistant
  • WD — Western Economic Diversification

Executive Summary

This report presents the findings of a final evaluation of the Community Futures (CF) program in Ontario. An evaluation of the CF Program is required by the Transfer Payment Policy and was undertaken to respond to the requirements outlined in the program's Results-Based Management and Accountability Framework (RMAF).Government Consulting Services (GCS) was engaged to undertake the evaluation of the CF program, and more specifically, the delivery of this program through the four Regional Development Agencies (RDAs): the Atlantic Canada Opportunities Agency (ACOA), Canada Economic Development for Québec Regions (CED-Q), Western Economic Diversification Canada (WD), and the Federal Economic Development Initiative for Northern Ontario (FedNor) under Industry Canada (IC).

The findings presented in this report focus specifically on the delivery of the CF program through Industry Canada and FedNor. The evaluation was conducted between November 2007 and September 2008.

The aim of the Community Futures program is to: support local rural communities and small and medium-sized enterprises (SMEs) in meeting their economic needs; help rural communities to develop and implement long-term community strategic plans leading to the sustainable development of their local economies; and provide resources to local CF organizations to build community capacity to adapt to and manage change. In Ontario, the CF program is delivered through a network of 61 non-profit Community Futures Development Corporations (CFDC) in seven regions (East, Southeast, South Central, Southwest, West, Northwest and Northeast).

According to budget information provided by FedNor, over the six-year study period, a total of $131.25 million in contributions was allocated to the CFDCs. With respect to operating costs, FedNor does not maintain separate budgets for each of its programs, therefore figures for O&M and capital included costs for the CF Program as well as the Eastern Ontario Development Program (EODP) and the Northern Ontario Development Program (NODP). Each CFDC receives roughly the same amount of operating dollars based upon base funding agreements, as proposed in the CFDC business plans.

Methodology

The evaluation methodology integrates the use of multiple lines of evidence and complementary quantitative and qualitative research methods as a means to ensure the reliability of results being reported, and the validity of information and data collected. The research methods included:

  • document review;
  • stakeholder interviews;
  • surveys of CFDCs and clients;
  • review of administrative and labour market data;
  • comparative analysis; and
  • case studies.

The evaluation focused on four main areas: program relevance, design and delivery, success, cost-effectiveness and alternatives. Data reliability varied across the methodologies employed in the evaluation. Consequently, reliability of findings related to Program Success and Cost-effectiveness is low.

Relevance of Program

The evaluation found that there is a continuing need for the CF program, primarily because there are limited sources of funding available for SMEs, and CFDCs are supporting businesses that may not have otherwise been able to start, expand or survive. This need for funding is more pronounced in smaller communities, which are predominantly located in Northern Ontario, many of which have higher rates of unemployment and lower earnings than the provincial averages. In recent years these communities have been affected by fluctuations in commodity prices and the CFDCs are helping to bring economic stability to the communities. The biggest impacts of CED work have been that it mobilized businesses or community groups and improved or developed business knowledge.

The objectives of the CFDCs are in-line with the objectives of the CF Program as they are working towards fostering economic growth and stability, fostering creation and maintenance of jobs, helping to create diversified and competitive local economies, and helping to build sustainable communities. Similarly, the objectives of the CF Program are aligned to Industry Canada's strategic objective of a competitive industry and sustainable communities. Based on interviews with FedNor senior managers and external stakeholders their view is that the Federal Government has a legitimate, continuing role to play in delivering the CF Program.

In addition to providing funding for SMEs, the CF Program offers a range of business services (e.g., counselling, business information, referrals) and provides assistance to communities with planning and implementing their economic development priorities. CFDCs have been successful in working with other programs and organizations to ensure that their services compliment, rather than duplicate one another. The evaluation raised a few issues with respect to potential duplication of administrative efforts and reporting of outcomes with the EODP and NODP programs.

Design and Delivery

The CF Program has been in operation for over 20 years and changes have been made to its design and delivery as issues have been identified. Therefore, the Program appears to function well and the evaluation identified few issues impacting on the success of the program. Based upon stakeholder interviews and surveys, funding was the biggest issue cited by stakeholders who felt there was a particular need for additional funding for operations, community economic development (CED) activities, and investment funds.

Overall, the investment funds of the CFDCs appear well-managed with the overall portfolio experiencing positive growth in the past four years. The growth of the investment fund is influenced by the level of loan activity of CFDCs, including the proportion of those funds that are in active investment, as well as the loan loss rates. There are large variations across CFDCs with respect to the level of funds in active investment and a large range in loan loss rates. Without targets for desired performance for loan portfolios, it is difficult to assess whether there should be concerns regarding the performance of investment funds for certain CFDCs.

Recommendation #1: FedNor should work with CFDCs to establish general target ranges related to desirable levels of funds in active investment and loan loss rates based upon local realities. This would allow FedNor to assess a CFDC's loan portfolio performance and assist with improvements where necessary.

top of page

Currently, the biggest challenge for the CF Program is in the collection and management of its performance information. While a structure is in place for collecting data related to CFDC performance and outcomes, the data is rarely used for the management of the program and does not provide appropriate information to measure the success of the Program. The current system in use for the tracking of performance information (i.e., The Exceptional Assistant, also known as TEA) is not used by all CFDCs and not all CFDCs are meeting the reporting requirements (i.e., submitting annual reports). TEA is also of limited use because it does not capture all performance indicators required for measurement purposes as per the RMAF. In addition, information to measure the longer-term outcomes of the Program is currently lacking, particularly with respect to Community Economic Development (CED) activities.

The evaluation found that the data entered into TEA is unreliable, making the resulting Quarterly Reports also unreliable. The outcome information currently reported by CFDCs (e.g., jobs created) is based on estimates at the time of the application and no verification is done to ensure that those estimates were actually achieved. There also seems to be a lack of verification of data entered by CFDCs, as there were many variables in the TEA data that appeared incorrect.

Recommendation #2: FedNor should complete a review of the performance data collected for the CF Program to ensure reliable and meaningful reporting and establish additional indicators to provide information for assessing the longer-term impacts of the program, particularly for CED activities. In addition:

  • FedNor needs to ensure that TEA is capable of capturing and reporting on all required performance information. It is possible that FedNor could reduce reporting requirements for CFDCs by maximizing its use of TEA and thereby improving its ability to manage the Program as well as make it more useful for CFDCs.
  • FedNor should ensure that all CFDCs are fulfilling all reporting requirements and should establish consequences for those CFDCs that do not comply with the requirements.
  • FedNor should ensure that CFDCs are not reporting activities undertaken and outcomes achieved through the use of EODP and NODP funds in CF Program reports (i.e., in the annual reports).
  • To ensure data reliability, FedNor should implement a system whereby TEA data captured and submitted in Quarterly reports would be reviewed on a regular basis; FedNor needs to work with the CFDCs to ensure data is corrected should issues be identified.

Program Success

Due to concerns with respect to the reliability of Quarterly Report data, especially with regards to reporting of business services provided, findings on program success are primarily based on soft data. It appears that CFDCs have provided a consistent level of business information, counselling and referral services to clients over a four-year period. The services provided by CFDCs have been deemed necessary by clients and there is a very high level of satisfaction with the services provided by CFDCs. The CF Program is also meeting the needs of OLMCs by providing advice, support and services in both official languages where required. There are however, on-going challenges related to maintaining skilled, bilingual staff and costs associated with providing bilingual information materials.

Through its loan portfolio, business services, and CED activities, the CF Program has been successful in improving clients' business skills and knowledge, creating new business start-ups, maintaining and strengthening existing businesses, increasing community capacity, and assisting communities to diversify their economies. The impacts of some of these activities, particularly CED work and the longer-term impacts of the program are more difficult to measure as indicators to measure these impacts have yet to be established.

Cost-Effectiveness and Alternatives

The current delivery model in place for the CF Program integrates a number of components that are important to successful community economic development which includes a grassroots approach where decisions are made at a local level; the involvement of local community volunteers; and a focus on partnership development and cooperation between community organizations. No other delivery alternatives were identified.

It was not possible to determine whether FedNor is cost-effective in administering the CF program compared to EODP and NODP, as program costs2 are not available separately for these programs.

Recommendation #3: FedNor should undertake a costing exercise to identify the costs for the operation of NODP, EODP, and the CF Program. This would be particularly useful because different delivery models are used to deliver these programs and it would allow FedNor to assess the extent to which the different delivery models are effective in achieving program outcomes.

Overall the CFDCs are cost-effective in administering the program. For the amount of funds received, CFDCs are providing a fairly high level of investment into the community, with approximately $42 million lent and $9 million contributed to CED annually province-wide, as well as offering a number of other services. The extent to which all CFDCs are equally effective in delivering the program raises a question about cost-effectiveness, as there are great ranges in the level of activity and outcomes of the program across CFDCs. While there may be reasons for some CFDCs to have varying levels of activity and outcomes equal to other CFDCs, such as geographic location, demographic composition of the community, the economy of the community and the level of demand for services, it may also reflect a lower level of effectiveness.

It seems that as the economic conditions of an area improve, the demand for CFDC services increases to the point where the private sector begins to compete for "bankable" clients. In these generally more populated areas, the CFDCs are serving more start-ups who do not have long credit histories. As there is more credit available in these regions, the loan clients are able to leverage the CFDC loan dollars to a greater extent and thus impact more jobs and businesses per investment fund dollar. However, each of these jobs in the larger communities represents a smaller percentage of the jobs in the regions. So while smaller communities may not demonstrate the same volume of job impact, percentage wise, they are impacting their community more profoundly. There may also be differences in cost-effectiveness related to the performance of the CFDCs. FedNor in concert with the CFDCs would be in the best position to determine whether certain CFDCs could be more effective in delivering the program given the many variables which influence effectiveness.

Recommendation #4: FedNor should work with CFDCs to establish target ranges for the level of activity for CFDCs (e.g., number of loans, investment fund growth). These ranges should be developed in consideration of factors that may influence the activities of CFDC (e.g., geographic location, demand for services). This would provide a guide for program offices and assist them in understanding whether CFDCs could be more effective in delivering the program or achieving program results.


2 Program costs are defined as the cost to FedNor to administer the program, not including funds provided to the CFDCs (i.e. FedNor salaries and O&M)

1.0 Introduction

This report presents the findings of the final evaluation of the Community Futures (CF) Program in Ontario. The evaluation was undertaken to respond to the requirements for an evaluation of the Program as outlined in the program's Results-Based Management and Accountability Framework (RMAF). Industry Canada / FedNor (IC/FedNor) engaged Government Consulting Services (GCS) to undertake the evaluation.

The objective of the study was to evaluate the Program in terms of relevance, design and delivery, success, and cost-effectiveness and alternatives. The research for this evaluation was conducted between March and July 2008.

The evaluation was overseen by a Steering Committee (SC) comprised of representatives from Industry Canada, FedNor and selected Community Futures Development Corporations (CFDCs) (see Appendix A for a list of Steering Committee members). The results of this evaluation study will be incorporated into a Pan-Canadian report, which will summarize findings from four CF evaluations being conducted concurrently across Canada.

The evaluation report is organized as follows:

  • Section 1 presents a description of the CF Program;
  • Section 2 presents the methodology for the evaluation;
  • Section 3 presents findings by evaluation issue and question; and
  • Section 4 presents the conclusions and recommendations.

1.1 Description of the Community Futures Program in Ontario

1.1.1 CF Program Context and Objectives

The CF Program is a cornerstone of the Government of Canada's support for rural economic development. It was first authorized in 1985 as part of the Canadian Jobs Strategy and the first community selection was announced in February 1986. The program introduced measures for the creation of, and support for, community-based development and/or adjustment initiatives in non-metropolitan areas of significant economic stress across Canada.

The program's aim is to support local rural communities' small-and medium-sized enterprises (SMEs) in meeting their economic needs, to help rural communities develop and implement long-term community strategic plans leading to the sustainable development of their local economies, and to provide resources to local Community Futures Organizations (CFOs) to build community capacity to adapt to and manage change.Footnote 3

The CF Program is currently administered by four Regional Development Agencies (RDAs): the Atlantic Canada Opportunities Agency (ACOA), Canada Economic Development for Québec Regions (CED-Q), Western Economic Diversification Canada (WD), and the Federal Economic Development Initiative for Northern Ontario (FedNor) under Industry Canada (IC).Footnote 4

The CF Program is one of three programs that IC/FedNor delivers to support its strategic objective for a competitive industry and sustainable communities. IC/FedNor also offers the Eastern Ontario Development Program (EODP) and the Northern Ontario Development Program (NODP).

1.1.2 Program Administration and Delivery

In Ontario, the CF Program supports a network of 61 CFOs, which are referred to as Community Futures Development Corporations (CFDCs). The CFDCs are organized in seven geographic regions (i.e., Northwest, Southwest, West, East, Northeast, South Central, and Southeast). The Southeast and East regions have recently been amalgamated, however for the purposes of this evaluation, they remained separate (These are shown amalgamated in Figure 1). Each region has between seven and fifteen member CFDCs (Table 1). The largest CFDC serves a population of over 176,000, while the smallest serves a population of only 4,680. CFDCs serve every part of Ontario, save for the larger cities in Ontario (i.e., Ottawa, Kingston, Greater Toronto Area, Barrie, Hamilton, Guelph, Kitchener-Waterloo, London, Windsor, and St. Catherines). In 2006 CFDCs served 3.68 million people or 30.3% of the population of Ontario. See Appendix B for a full listing of CFDCs, by Region.

Figure 1. Geographic Division of CFDC Regions

Geographic Division of Community Futures Development Corporation Regions

The Ontario Association of Community Futures Development Corporations (OACFDC) supports the 61 CFDCs through the provision of member support services, promotion of the CFDCs, and partnership development. The regions have also organized networks to support each other's loan and community economic development work. Nationally there is a pan-Canadian network which shares resources and best practices. The CF Program in Ontario is managed by FedNor's Program Delivery Managers and is delivered by program officers located throughout Ontario, who work directly with CFDCs.

Table 1. Number of CFDCs by Region
Region Number of CFDCs
East 7
Southeast 8
South Central 7
Southwest 7
West 8
Northwest 9
Northeast 15
Total 61

1.1.3 CF Program Areas

To accomplish the objectives of the program, the CFDCs undertake activities in four main areas: access to capital, business services, strategic community planning and socio-economic development, and community-based projects and special initiatives.

Access to Capital

CFDCs each have a local investment fund which is used to provide repayable loans, loan guarantees and equity to SMEs for the start-ups, expansion, and/or stabilization of businesses. CFDCs are to manage these funds in a way such that there is positive growth in the fund and thus ongoing investment in the community. CFDCs charge interest rates that are a minimum two percent above prime and are often based on a risk assessment of the SME (i.e., business history, credit rating, available security).

Loan applications are reviewed and approved or rejected by local CFDC volunteer boards. CFDC loans are intended for SMEs that have had difficulties accessing financing from traditional sources (e.g., financial institutions). As such, the CFDCs are financing riskier endeavours and have loan loss rates which are higher than traditional financial institutions. SMEs can receive a loan of up to $150,000 and the loans are repayable. Between April 2002 and March 2008, CFDCs provided a total of 6,963 loans to SMEs, totalling $294.1 million (Table 2).

Table 2. Loan Activity, by Region (2002-2008)
Region Year Total
2002-
2003
2003-
2004
2004-
2005
2005-
2006
2006-
2007
2007-
2008
East 96 75 104 98 110 81 564
Southeast 259 261 264 248 242 197 1,471
South Central 76 82 94 103 123 117 595
Southwest 153 127 106 91 137 68 682
West 124 142 122 126 126 99 739
Northwest 168 177 129 143 125 99 841
Northeast 275 326 457 352 348 313 2,071
Total number of loans 1,151 1,190 1,276 1,161 1,211 974 6,963
Total dollar value of loans (millions) $ 47.8 $ 44.4 $53.4 $48.9 $ 55.4 $ 44.1 $294.1

Some CFDC regions have started investment pools whereby, the CFDCs in a region pool their investment funds to support SMEs that require loans in excess of $150,000. The objective of these funds is to share the risk and the reward of larger loans, ensure accountability to Canada for the funds, and retain local decision making autonomy. They support projects or SMEs that benefit the region as a whole, rather than just one community. The first region to create an investment pool was the Northeast region, in 2001. Since its inception, the fifteen member pool has supported 102 loans for a total value of $36.6 million. The Northwest region created the Northwestern Ontario Investment Pool (NWOIP) in 2004. Seven of the nine CFDCs in this region are currently participating in the pool and since its creation, a total of 13 loans have been provided, totalling $5.25 million. The East region, which recently grouped the East and Southeast CFDCs together created an investment pool in 2008, with nine of the 14 CFDCs in this region participating. Since its creation, the Eastern Ontario Loan Pool has supported nine projects, totalling just over $800,000.

Business Services

CFDCs also provide a range of business information and services such as: on-site libraries, on-site guided access to Internet-based information, and referrals to other services and specialists as required. CFDCs may also provide counselling to support the start-ups, expansion and diversification of SMEs, and to improve business competitiveness. This counselling can take the form of assistance with a business plan, marketing or accounting.

CFDCs have been recording information on general inquiries and in-depth interviews since April 2004. Since that time, CFDCs have received 202,961 general inquiries and have conducted 32,759 in-depth counselling interviews (Tables 3 and 4).

Table 3. Number of General Inquiries Received, by Region (2004-2008)
Region Year Total
2004-2005 2005-2006 2006-2007 2007-2008
East 4,751 3,798 5,481 5,346 19,376
Southeast 6,212 5,817 5,523 5,771 23,323
South Central 8,151 8,634 9,543 6,885 33,213
Southwest 2,912 2,291 2,313 3,062 10,578
West 7,498 6,746 12,459 11,366 38,069
Northwest 1,935 4,048 1,841 2,152 9,976
Northeast 17,698 17,037 16,738 16,953 68,426
Total 49,157 48,371 53,898 51,535 202,961
top of page
Table 4. Number of In-Depth Interviews Conducted, by Region (2004-2008)
Region Year Total
2004-2005 2005-2006 2006-2007 2007-2008
East 1,140 1,406 1,012 960 4,518
Southeast 1,370 1,459 1,639 1,437 5,935
South Central 851 1,106 1,699 889 4,545
Southwest 344 506 660 455 1,965
West 2,256 1,890 2,057 2,032 8,235
Northwest 508 1,464 1,066 977 4,015
Northeast 1,045 1,102 886 513 3,546
Total 7,514 8,933 9,019 7,293 32,759
Strategic Community Planning and Socio-Economic Development

The CFDCs work with communities to assess local problems; establish objectives; plan and implement strategies to develop human, institutional and physical infrastructures; and promote entrepreneurship, employment and the overall economy. They do this through provision of funding, leadership and coordination of stakeholders. CFDCs do not systematically and completely track their involvement in strategic community planning and socio-economic development, therefore there is no information available on the number of community strategic plans with which CFDCs have been involved.

Community-Based Projects and Special Initiatives

As part of the CF Program, CFDCs are to support community economic development by collaborating with other partners in the public sector and civil society. The projects are intended to support the implementation of community strategic plans and can vary considerably from one community to another. Examples include tourism, entrepreneurship, economic opportunities for specific client groups such as women, youth, Aboriginal people and members of official language minorities, or projects which respond to specific challenges facing a community such as downturns in important industries. Similar to strategic community planning and socioeconomic development, CFDCs do not systematically track their involvement in CED projects. Limited information is available, through the CFDC annual reports, which is presented in section 3.3.5 (Intermediate Impacts of the CF Program).

1.1.4 Budget

According to budget information provided by FedNor, over the six-year study period, a total of $131.25 million in contributions was allocated to the CFDCs. With respect to operating costs, FedNor does not maintain separate budgets for each of its programs, therefore the figures for O&M and capital in Table 5 include costs for: the CF Program, the EODP, the NODP, and the Softwood Industry Community Economic Adjustment Initiative (SICEAI).

Table 5. CF Program Budget
Budget Item Fiscal Year
2002-2003 2003-2004 2004-2005 2005-2006 2006-2007 2007-2008
O&M (all programs) 13.01 11.74 11.45 12.83 12.27 11.79
Capital       0.12 0.06 0.12
G&C 20.49 25.12 24.26 20.26 20.26 20.86
O&M (CF Program)Footnote 5 1.28 1.28 1.28 1.28 1.28 1.28

Of the total $131.25 million, $126.3 million came from the CF Program. Of this amount, $99.8 million has been for the operation of the CFDCs, while the rest was for capitalization of investment funds, especially of new CFDCs, and other special projects. In 2002-2003 CFDCs received an average of $255,128 which increased to an average of $298,472 per CFDC in 2007-2008. These increases were the result of negotiations to increase base funding. No region or grouping of CFDCs had any notable differences in the amount of operating funds received.


Footnotes

  1. 3 back to footnote reference 3 Community Futures National Steering Committee. Community Futures Program Results-Based Management and Accountability Framework. May 2005.
  2. 4 back to footnote reference 4 Note that FedNor is not an RDA, however for the purposes of the evaluation; the four organizations are referred to as the RDAs.
  3. 5 back to footnote reference 5 Foundation documents for the CF Program indicate that $1.28 million has been approved for O&M. Note that this is incremental funding and does not reflect the actual O&M costs for the CF Program.

2.0 Evaluation Methodology

2.1 Evaluation Questions and Issues

A horizontal RMAF was developed for the renewal of the CF program in May of 2005, and was updated in the fall of 2007 in preparation for the evaluation. The pan-Canadian matrix outlined a set of common questions and indicators that all RDAs were to use as the basis for each of their evaluations. Since IC/FedNor conducted its evaluation in tandem with the evaluations conducted in the other regions of the country, IC/FedNor's evaluation used the common evaluation questions and indicators, and similar approaches and methodologies to the other RDA evaluations. Supplementary questions and methodologies were added to the IC/FedNor evaluation matrix to satisfy region-specific information needs. The evaluation was conducted according to the IC/FedNor-specific evaluation matrix, presented in Appendix C. Table 6 provides a summary of the evaluation issues and questions. Note that the evaluation questions in italics are those that are specific to the IC/FedNor evaluation. All other questions are common to all RDA evaluations. The results of this evaluation will be rolled up to provide evidence for a common Pan-Canadian evaluation of the CF Program.

Table 6. Summary of Evaluation Issues and Questions
Evaluation Issue Evaluation Question
Relevance
  • Is there a continued need for the CF Program?
  • Does the CF Program complement, duplicate or overlap other government programs? Other private sector services?
  • Are local CFDC objectives and activities consistent with the national CF Program? Are CF Program objectives consistent with departmental objectives?
  • Is the current role of the Federal Government appropriate?
Design and Delivery
  • Are the CFO networks (national, provincial, regional, and sub-regional) working effectively?
  • What factors impact or facilitate the achievement of program results?
  • Have communities been involved in developing strategic plans? To what extent are CFDC activities linked to those community plans?
  • Are the CFDC investment funds well managed? Are the number, level and loss rates of the loans meeting the needs?
  • Are the departments and CFDCs gathering the necessary data for evaluation and measurement purposes?
Program Impact Short-Term

  • To what extent has the CF Program provided appropriate information, referrals and counselling to clients?
  • To what extent has the CF Program improved business knowledge and skills of clients?
  • To what extent has the CF Program created new business start-ups or strengthened existing businesses?
  • To what extent is the CF Program serving the needs of Official Languages Minority Communities (OLMCs)?
Intermediate

  • To what extent has the CF Program: supported community economic development; assisted communities to develop and diversify their economies; and strengthened community capacity?
Long-Term

  • To what extent has the CF Program contributed to long term goals (i.e., economic growth and stability, diversification and development of local rural communities, sustainable communities, and survival of business assisted by CFDCs)?
  • Has the CF Program produced unintended positive and/or negative outcomes?
Cost-effectiveness and Alternatives
  • To what extent is the CF Program cost-effective?
  • Are there other more cost-effective/ efficient approaches or alternatives to be considered that would achieve CF Program objectives?
Other
  • What lessons have been learned from the CF Program either positive or negative?
  • To what extent have the recommendations of previous evaluations been implemented?
top of page

2.2 Methodology

The IC/FedNor evaluation integrated the use of multiple lines of evidence and complementary research methods as a means to enhance the reliability and validity of information and data to be collected. Data reliability varied across the methodologies employed in the evaluation. Consequently, reliability of findings related to Program Success and Cost-effectiveness is low. The following research methods were used to gather qualitative and quantitative data for the evaluation:

  • document review;
  • stakeholder interviews;
  • surveys of CFDCs and clients;
  • review of administrative and labour market data;
  • comparative analysis; and
  • case studies.

Each of these methods is described in more detail below.

2.2.1 Document Review

A review of relevant documentation was completed primarily to assess program relevance (e.g., need for the program, linkages with program and departmental objectives) and design and delivery. The following types of documents were reviewed:

Background and authority documents: including foundation documents for the CF Program and related policies.

Corporate/operational documents: including documents related to the delivery and management of the program, such as the RMAF, operational guidelines, manuals, handbooks, and administrative reports.

Audits, reviews, assessments, and evaluations: including previous evaluations, audits, surveys, and research studies.

The document review was conducted using a customized template that extracted relevant information from the documents and organized it according to indicators and evaluation questions. Appendix D contains a list of documents that were reviewed for the evaluation.

2.2.2 Stakeholder Interviews

Interviews served as an important source of information by providing qualitative input on relevance, design and delivery, program success and cost-effectiveness/alternatives of the CF Program in Ontario. A total of 39 interviews were completed, including FedNor senior managers, FedNor program officers, CFDC managers and chairs of boards of directors, and external stakeholders (Table 7). The interviewees were selected in collaboration with IC/FedNor representatives. Note that the CFDC interviewees were selected to ensure geographic representation and variation in the size of the populations served by the CFDCs (a list of interviewees is provided in Appendix E).

Table 7. List of Interview Groups
Interview Group Number of Interviews Conducted
FedNor senior managers 4
FedNor program officers 9Footnote 6
CFDCs (managers and chairs of boards) 21Footnote 7
External stakeholders 5
Total 39

Most interviews were conducted via telephone, with the exception of the FedNor senior manager interviews, which were conducted on-site at the FedNor office in Sudbury. Interviews were generally between one and a half and two hours in length, depending on the interviewee. All interviewees were contacted in advance of the interview to schedule an appropriate time and all received an interview guide in advance of the interview (see Appendix F for the interview guides). Note that a set of interview questions had already been developed to gather information as outlined in the pan-Canadian evaluation matrix. This set of questions was modified to gather information as per the IC/FedNor-specific evaluation matrix. The results of the interviews were summarized in an interview notes template and were then coded and analyzed to determine key themes. Note that in some cases, not all interviewees responded to all questions. Therefore throughout the analysis the number of interviewees who commented does not necessarily equal the number of interviewees noted in Table 7.

2.2.3 Surveys

As part of the evaluation two surveys were conducted with CFDC staff and chairpersons and with CFDC clients. Details of each of these surveys are included below.

Community Futures Development Corporations

The CFDC survey developed for the pan-Canadian CF evaluation was used as a basis for the IC/FedNor CFDC survey and some questions were modified or added to address IC/FedNor specific issues (see Appendix G for the CFDC survey). The survey was pre-tested with four FedNor program officers (two English and two French) and minor modifications were made following comments from the pre-test. The managers and chairs of all 61 CFDCs were invited to participate in the survey, which was administered via the webFootnote 8. To maximize the response rates, both GCS and the OACFDC sent reminders to the CFDCs.

The response rates for the survey were very high, with an overall response rate of 78.7 percent (95 percent confidence interval, +/-4.6%). When the responses from the two groups are separated, the results from managers were more representative than those of chairs (Table 8). The response rate for the managers was 88.5 percent (95 percent confidence interval, +/-4.6%); while the response rate from chairs was slightly lower at 68.9 percent (95 percent confidence interval, +/-8.5%).

Table 8. Response Rates for the CFDC Survey
CFDC Group Population Size Number of Responses Response Rate Confidence Interval
Managers 61 54 88.5% 95% (+/-4.6)
Chairs of the boards 61 42 68.9% 95% (+/-8.5)
Overall 122 96 78.7% 95% (+/-4.6)

From a regional perspective, the number of responses from CFDC managers is quite representative with two regions having responses from all managers and three regions having responses from all but one manager (Table 9). Responses from board chairs were not as representative by region as those of the CFDC managers. Only one region received responses from all board chairs, while two of the seven regions had fairly low response rates compared to the other regions, with just over 50 percent. Despite this, the overall results of this survey are viewed as reliable.

Table 9. CFDC Survey Response Rates, by Region
Region Number of CFDCs in Region Number CFDC Manager Responses Number of CFDC Board Chair Reponses
East 7 5 7
Southeast 8 7 6
South Central 7 6 5
Southwest 7 6 6
West 8 6 5
Northwest 9 9 5
Northeast 15 15 8
Total 61 54 42
Clients of Community Futures Development Corporations

As with the CFDC survey, the client survey developed for the pan-Canadian CF evaluation was used as a basis for the IC/FedNor client survey and some questions were modified or added to address IC/FedNor specific issues (see Appendix H for the client survey).

For the purposes of the client survey, CFDC clients were categorized into three groups.

  1. Financial: included those who applied to a CFDC for financial assistance, including both those who were accepted and refused.
  2. Non-financial: included those who accessed the information services of a CFDC (e.g., counselling, advice, training), but did not apply to receive financial assistance. These clients could have included those clients that called the CFDC or walked into the CFDC office with a general inquiry.
  3. Community partners: included those partners and organizations with which CFDCs may have engaged for the development and promotion of the CFDC service area, including the development of community strategic plans.

The client survey was to be administered via the web and GCS collected contact information for the survey through the CFDCs. The CFDCs were requested to update The Exceptional Assistant (TEA) with as much client information as possible, including e-mail addresses and send it to GCS. Following this data collection exercise, it was discovered that e-mail addresses were not available for enough clients in each of the regions and rural residents were under-represented, either because the client did not have access to high speed internet, use e-mail or the CFDC had not recorded the e-mail address Therefore, the SC recommended that the client survey be administered by telephone and a third-party firm was engaged to undertake that task.

The responses to the client survey were analysed by: the seven CFDC regions, client type, and size of business. In almost all instances, there were few differences between these groups. Where there were noteworthy differences, these have been identified within the body of this report. A total of 419 financial clients responded to the survey, 52 of whom were refused a loan. Unfortunately, only 20 of these refused clients completed all the required sections of the survey; this response rate is insufficient to merit inclusion as a survey population.

2.2.4 Administrative and Labour Market Data

The Exceptional Assistant (TEA)

Administrative data for the CF Program was obtained through TEA. CFDCs use this system to track activities and to manage their loan portfolio. Certain information from TEA is available through pre-programmed Quarterly Reports. For evaluation purposes, the developer of the TEA system downloaded all data reported in the Quarterly Reports (for April 2002 – March 2008) into a Microsoft Excel file. Some of the data was only available for the final four years (2004-2008) of the evaluation period. All data was then analyzed according to the appropriate performance indicators, including cross-tabs by CFDC region, size of population served, age of CFDC, access to EODP and NODP, and volume of loan activity.

Grants and Contributions Reporting System (GCRS)

Financial information for the CF Program was obtained from Industry Canada's Grants and Contributions Reporting System (GCRS). A FedNor representative generated the required information on the amount of contributions made to the CFDCs for operations and capitalization, including any EODP and NODP funds accessed by the CFDCs for the period April 2002 – March 2008. This financial information was analyzed for the appropriate performance indicators.

Labour Market Data

Labour market data was available through FedNor, who engaged Statistics Canada to develop customized data profiles for each of the CFDC regions. This data, compiled from the 1996, 2001, and 2006 censuses, provided key information required for the evaluation, including: labour force participation rates, unemployment rates, average household income, and employment by industrial sector. This information was used to assess the level of need for the CF Program across the seven CFDC regions as well as the impact of the CF Program.

2.2.5 Comparative Analysis

The cost-effectiveness of the CF Program was assessed by examining the inputs (e.g., level of investment), activities, and outcomes of the CFDCs and comparing those variables across the CFDCs. The following components were compared:

  • trends in services / activities of CFDCs;
  • investment activity (e.g., number and dollar value of loans, percent of portfolio in active investment);
  • number of businesses assisted;
  • program outcomes (e.g., jobs created); and
  • program contributions to CFDCs.

Note that the cost-effectiveness of the CF Program was also to be assessed through a comparison with two other similar programs administered by FedNor (i.e., the EODP and the NODP). To do this, the costs for program operationsFootnote 9 would be compared to program outcomes to calculate a cost per unit outcome. For the CF Program in Ontario, it was not possible to assess the extent to which FedNor is cost-effective in administering the program (i.e., costs for operation versus outcomes). This is due to the fact that FedNor does not separate the costs for the operation of the CF Program from the other programs it delivers (i.e., EODP and NODP). Significant attempts were made to extract the operating costs for the CF Program using existing cost data that was available for the EODP and NODP programs. However, it was not possible because the figures presented for these programs were also those for FedNor's programs as a whole. Discussions were also held with representatives of the Finance group at FedNor to determine whether the costs could be separated, however this was deemed infeasible.

2.2.6 Case Studies

Case studies were carried out as part of the evaluation to provide a more in depth understanding of the impact of the CF Program in three selected communities: Kirkland Lake, Belleville, and Brantford. The communities were selected in collaboration with the SC and were selected based on geographic location and size of population served (Table 10).

Table 10. Case Study Community Selection Criteria
Community CFDC Region Type PopulationFootnote 10
Kirkland Lake Kirkland and District Community Development Corporation Northeast Rural 8,248
Belleville Trenval Business Development Corporation Southeast Rural 48,821
Brantford Enterprise Brant Southwest Semi-urban 90,912

In carrying-out the case studies, the following activities were undertaken:

  • document review (e.g., community strategic plan);
  • review of program data provided by CFDCs (e.g., loan portfolio and community economic development projects);
  • review of economic data obtained through Statistics Canada;
  • an interview with the Manager of the CFDCs;
  • interviews with community representatives;
  • visits with loan clients; and
  • visits with community economic development partners.

Information gathered for each community was analyzed and results were summarized in a separate case study report (see Appendix J for the case studies). These reports were provided to each of the CFDC managers in those communities for their review and validation.

top of page

2.3 Limitations of Methodology

As with any evaluation, there are limitations to the evaluation methodologies, which are summarized below. Note that generally, the use of multiple lines of evidence helps to minimize the limitations of an evaluation; however, due to limitations associated with a few of the methodologies, particularly with respect to the Quarterly Reports data and CED information, GCS is less confident in the findings that are based on Quarterly Reports data and CED information. Where there are concerns about the reliability of the data, GCS has noted these concerns at the beginning of each set of findings.

Representativeness of data collected

The information gathered from case studies and interviews was not gathered on a representative basis; therefore it is not possible to make generalizations about the program based only on these findings.

The population size for the financial and non-financial clients was determined using the client data from TEA. Based on the number of survey responses received, the level of confidence in the survey findings is quite high (95 percent confidence interval, +/-5.0% for both groups) (Table 11). Due to the fact that the telephone lists received from CFDCs were incomplete, there is a risk that certain segments of the client population were not included in the survey.Footnote 11 Nonetheless, these lists were far more complete than the email lists discussed in 2.2.3 (Surveys). However, an examination of the survey demographics shows an even distribution of responses by CFDC region and by industry sector. The majority of survey respondents also owned very small businesses (i.e., four employees or less), which is consistent with the make up of SMEs in OntarioFootnote 12, and of CFDC clients. Therefore, it appears that the survey results are fairly representative of the client population. Note that based on the number of CFDCs per region, the Southeast region received a slightly higher proportion of responses than the other regions.

Table 11. Client Population and Response
Client Stream Population Size # of Responses Confidence Interval
Financial 7,158 419Footnote 13 95% (+/-5.0)
Non-financial 235,670 385 95% (+/-5.0)
Partners unknown 153 -
Total - 957 -

For the partners, it was not possible to determine the size of the population because CFDCs do not compile this contact information. The list of partners provided by CFDCs was compiled manually by CFDCs upon request; therefore, GCS had to assume that it was not a complete list of partners as the total number is unknown. Thus, the survey responses from partners cannot be considered representative of the whole population. It is important to note, however, that there is a good distribution of survey responses by region and by type of organization (e.g., municipal representative, non-profit organization). See Appendix I for demographic information for CED partners.

Administrative Data

The Exceptional Assistant

The evaluation identified some concerns with respect to the completeness and reliability of the Quarterly Report data and, in fact, many variables appeared to be outliers and had to be removed from the analysis because those variables were greatly skewing the overall numbers. In addition, there were some cases where fields in the Quarterly Reports were left blank, although it was not possible to determine whether there was either no data to report or whether data was missing. Due to the large volume of variables provided for each of the six years of the study period it was

14 not possible to assess the extent to which each variable provided by the CFDCs was reliable or not. However, the many outliers and blank fields raise concerns as to the reliability of the Quarterly Report data and therefore the level of confidence is low for findings where Quarterly Report data has been used as a methodology. Note that data pertaining to financial information (i.e., volume and value of loans, investment funds) is pulled directly from the CFDCs' accounting systems, so this data can be considered reliable. Overall, it should be noted that often the information and trends found in the Quarterly Report data were consistent with information gathered through other methodologies.

Annual Reports

The information on CED was obtained mainly through the annual reports submitted by CFDCs. Note that because all CFDCs have different agreement periods, annual reports do not have the same reporting period. For the purposes of this analysis, the reports were grouped into year 1 (generally including CFDCs with reporting periods between April 2005 and December 2006) and year 2 (generally including CFDCs with reporting periods between April 2006 and December 2007). CFDCs submitted 43 and 34 annual reports in year 1 and year 2 respectively (of 61 CFDCs). Of the reports submitted, some were not in the standard format and not all reports contained information on CED activity. Only 31 of 61 CFDCs provided CED information in year 1 (just over 50 percent of CFDCs) and 28 of 61 provided CED information in year 2 (fewer than 50 percent of CFDCs) (Table 12). In both years, the majority of CFDCs in the East included CED information in their annual reports, however, because this region is not typical of all regions (i.e., it has access to EODP) these results cannot be extrapolated to other regions. Therefore, GCS does not view these results as being representative.

Table 12. Representativeness of CED Information
Region Number of CFDCs Year 1 Year 2
Total # that submitted CED Information Percent Total # that submitted CED Information Percent
East 7 5 71.4 5 71.4
Northeast 15 6 40.0 8 53.3
Northwest 9 5 55.6 2 22.2
South Central 7 3 42.9 4 57.1
Southeast 8 4 50.0 6 75.0
Southwest 7 4 57.1 1 14.3
West 8 4 50.0 2 25.0
Total 61 31 50.8 28 45.9
Non-Clients

The evaluation methodology did not include any opportunities to speak with businesses that have not used services of CFDCs (i.e., non-clients). Without talking to non-clients, it is uncertain whether there are any differences between SMEs who accessed CFDC funds and those who did not. For example, do SMEs that receive support from a CFDC have better results or success than those who did not?

In the initial planning for the Pan-Canadian evaluation, the RDAs included a non-client survey. However, during the conduct of the evaluation in Ontario the SC raised concerns with respect to administering the client survey via e-mail, indicating that many clients do not have e-mail addresses, as mentioned in 2.2.3 (Surveys). Therefore, IC/FedNor decided to administer the client survey via telephone and use the funds allocated for a non-client survey to do this. Both the Steering Committee (SC) and FedNor agreed that this was the most efficient use of the evaluation budget, which had not allowed for telephone surveys of both the client and non-client groups.


Footnotes

  1. 6 back to footnote reference 6 There were 12 CF program officers interviewed, however officers located in the same city were interviewed together.
  2. 7 back to footnote reference 7 The Executive Director of the Ontario Association of Community Futures Development Corporations (OACFDC) was included in this group.
  3. 8 back to footnote reference 8 GCS administered the survey to four chairs via telephone, as they did not use e-mail.
  4. 9 back to footnote reference 9 Program costs are defined as the cost to FedNor to administer the program, not including funds provided to the CFDCs (i.e. FedNor salaries and O&M)
  5. 10 back to footnote reference 10 Statistics Canada, 2006 Census.
  6. 11 back to footnote reference 11 Three CFDCs did not provide client contact lists.
  7. 12 back to footnote reference 12 Industry Canada. Key Small Business Statistics. July 2008.
  8. 13 back to footnote reference 13 367 were accepted for a loan.

3.0 Evaluation Findings

This section of the report presents a summary of the evaluation findings, which are organized into the issue areas of relevance, design and delivery, success, and cost-effectiveness and alternatives. Under relevance, findings are discussed for: need for the program; complimentarity and duplication; linkages between CFDC, CF and Industry Canada objectives; and the role of the federal government in delivery the CF program. Design and delivery discusses findings regarding: the effectiveness of the CFDC networks; issues impacting the success of the program; CFDC involvement in strategic planning; management of investment funds; and performance measurement. Program success presents findings about: the appropriateness of information, referrals and counselling to clients; the impact on business knowledge and skills of clients; the impact on new and existing businesses; the needs of official languages minority communities; intermediate and finally long-term impacts of the CF program. Cost-effectiveness and alternatives explores findings on those two topics, while findings regarding other lessons learned and progress on previous recommendations are discussed subsequently.

3.1 Relevance

3.1.1 Need for the CF Program

Finding: SMEs are important to the development of local economies and the CF Program is supporting businesses that would otherwise not have been able to start, survive, or expand. Based upon independent research and other findings in this evaluation there is a continuing need for the CF Program as there are limited sources of funding for small businesses. Information from Statistics Canada also shows that areas served by CFDCs have consistently lower economic performance than the Ontario averages, although the gap varies by region, suggesting that the level of need for the Program may also vary by region.

Findings in this section are based on: interviews, research documents related to financing for SMEs, the client survey (i.e., loan clients), case studies, and Statistics Canada labour market data.

Contribution of SMEs to Rural Economies

As of December 2007, there were more than 2.3 million business establishments in Canada, with

37.6 percent of those businesses being located in OntarioFootnote 14. SMEsFootnote 15 in Ontario are small, typically with four or fewer employees and a recent report on small business statistics showed that over 50 percent of SMEs in Ontario employ one to four individualsFootnote 16.

According to the September 2007 report, Small and Medium Sized Enterprises in Ontario, SMEs account for 97 percent of all businesses in Ontario and are "important drivers of job creation and economic growth."Footnote 17 This same study shows that in 2004, the SMEs in Ontario accounted for 40 percent of the total GDP of the province. Similarly, a recent small business quarterly report by IC indicated that ‘rural-based entrepreneurship plays a significant role in the development and support of local economies.'Footnote 18 The report Small and Medium Sized Enterprises in Ontario also suggests that ‘access to financing is critical if SMEs are to develop and expand.'Footnote 19 Information from the evaluation shows that SMEs face challenges in accessing financing from traditional sources.

Availability of Financing for SMEs

All interviewees (39 of 39) agreed that there is a need for the CF Program, primarily because businesses cannot access other sources of funding in their regions. This notion was supported by the client survey, as almost all clients viewed access to capital as the CFDC service most needed by SMEs (94.4 percent). In fact 77.0 percent of clients surveyed were refused funding by a financial institution; with start-ups having a slightly higher refusal rate than existing businesses

(81.5 percent versus 72.1 percent). This rate of refusal is much higher than the 12.3 percent rate of refusal of rural Canadian businesses and 23.5 percent for all Ontario businesses, reported in the Survey on Financing of Small and Medium Enterprises.Footnote 20 This suggests that CFDC clients are less likely to receive bank financing than other rural or Ontario businesses. This could be an indication of the need for the program among CFDC clients. Note that 43.1 percent of clients suggested that they did not apply to a financial institution before approaching a CFDC, which is surprising given that CFDCs are targeting those who are not able to access traditional financing. Although, only two-thirds of loan clients interviewed for the case studies had first approached a bank, the other third had not done so primarily because they knew they would be turned down.

There is some existing research to support the notion that there are challenges for SMEs in accessing financing from financial institutions. The June 2008 Senate Report, Beyond Freefall: Halting Rural Poverty noted that "the challenges of accessing credit in rural Canada have long been recognized at the federal level." The report goes on to state that "arguably the most successful program to help address this concern has been the Community Futures program."Footnote 21 Also, in a November 2007 survey by the Canadian Federation of Independent Businesses, 61 percent of SMEs surveyed identified "securing term financing or loan from a bank" as the biggest financial barrier to establishing a business.Footnote 22

A 2007 Government study suggested that small firms are less able to obtain financing than large firms and that small firms are more likely to turn to informal sources of capital, including personal finances.Footnote 23 Also, according to the Survey of Suppliers of Business Financing, in 2001 only 12 percent of overall lending by chartered banks was for small authorizations (i.e., less than $1 million).Footnote 24 This is also supported by data from the SME financing data initiative, which found that there was a link between loan refusal rates and firm size with smallest firms having the highest refusal rates of all firms.Footnote 25 Therefore it appears that small SMEs are more likely to require non-traditional sources of financing than large and medium-sized SMEs. Information from the case studies is consistent with this, particularly with those loan clients interviewed in Belleville and Brantford. In these two communities, the CFDCs were supporting very small business with few employees (i.e., less than 5).

Need for Services Provided

Clients surveyed noted a great need for all the services provided by CFDCs, with access to capital seen as the greatest need (94.4 percent). The majority of clients also noted a need for business information (93.3 percent), business counselling (92 percent), referral services (90.3 percent), and training (90.1 percent). CFDCs surveyed felt very strongly that all services provided by CFDCs were needed and, in fact, for all activities CFDCs survey respondents indicated that there was either some need or a great need for the services, with the exception of one respondent who believed that there was little need for referral services. As for community economic development (CED) work, almost all clients believed there was either some or a great need for assistance with projects and almost all (99.4 percent) felt there was either some need or a great need for assistance with strategic community planning and socio-economic development. CFDCs were equally supportive of this with 92.7 percent of CFDC survey respondents saying there was either some need or a great need for assistance with strategic community planning and socio-economic development.

A recent study by Dr. Robert G. Roseheart also cites a need for a program similar to the CF Program, indicating that "overall, rural and regional economies are somewhat challenged to attract attention because of the current academic fascination with urban clusters, and the concentration of Research and Development (R&D) spending in cities."Footnote 26 He notes that in the northwest "much of the job creation is expected to come from small start-ups or expansion of small-and medium-sized companies ... There is a need to provide a business-friendly climate in the region to encourage business to start, expand and grow, and to encourage the creation of new business by new entrepreneurs."Footnote 27 While this is in reference to northwest Ontario, it likely has relevance throughout the province.

The CF Program is also needed because it has provided assistance to SMEs that would have had difficulties starting, expanding or maintaining their businesses. Clients surveyed generally agreed that had they not received CFDC support, they would not have been able to start/maintain/expand their business. However, of the survey respondents who were refused a loan, 45.0 percent indicated that they have been able to start/maintain/expand without the assistance. Information from case studies suggests that while some businesses would still exist in the absence of CFDC support, their owners suspect that the businesses would be smaller and their plans would have taken longer to complete. The case studies also highlighted the role CFDCs play in Northern Ontario towns whose economies are largely tied to commodities. As the demand for these commodities, such as steel, gold and lumber, rise and fall the main employers in the small towns can be seriously affected. CFDC loans help maintain significant players during downturns and help others grow rapidly as demand shifts.

Economic Indicators

Information from Statistics Canada shows that the regions served by CFDCs generally have lower economic performance than Ontario overall. The size of the labour force for the regions served by CFDCs grew 12.6 percent between 1996 and 2006.Footnote 28 Despite this, the labour force participation rate of the CFDC served areas has remained relatively constant at 63.6 percent over the same time period. This is between 2.7 percent and 3.8 percent below the provincial average (see Table 13). There is some variation in these numbers by region and by size of population served. In 1996 and 2006, two regions (South Central and West) had participation rates slightly above the overall Ontario rate. The CFDCs which serve populations of less than 49,999 had participation rates that were much below the provincial rate, while the larger communities approached, but did not equal, Ontario's rate.

Table 13. Labour Force Participation Rates in Ontario versus CFDC Served Communities
  1996 2001 2006
Ontario 66.3 67.3 67.1
CFDC regions (average) 63.6 63.5 63.7
East 64.9 64.4 64.2
Southeast 60.5 60.9 61.4
South Central 66.8 67.0 67.4
Southwest 65.5 65.1 66.8
West 66.6 67.0 67.5
Northwest 65.1 65.3 64.5
Northeast 59.8 59.3 59.2
< 10,000 61.8 61.7 63.7
10,000 – 49,999 61.1 60.5 59.7
50,000 – 100,000 65.1 65.5 65.7
100,000 + 65.5 65.9 66.0

The CFDC served regions, on average, also have consistently higher unemployment rates than Ontario overall (Table 14).Footnote 29 There is, however, some variation by region and by size of population served. The Northwest and Northeast regions have unemployment rates much higher than the overall provincial rates, while the East and South Central regions have unemployment rates lower than the overall Ontario rate. CFDCs which serve smaller populations also have much higher unemployment rates than the Ontario rate.

Table 14. Unemployment Rates in Ontario versus CFDC Served Communities
  1996 2001 2006
Ontario 9.1 6.1 6.4
CFDC regions (average) 10.7 8.1 7.1
East 8.8 5.8 5.6
Southeast 10.8 6.5 6.3
South Central 8.0 5.8 4.1
Southwest 8.8 7.0 5.7
West 9.6 6.8 6.6
Northwest 12.9 11.6 9.7
Northeast 12.9 10.5 8.7
< 10,000 14.8 12.3 10.2
10,000 – 49,999 12.4 10.2 8.7
50,000 – 100,000 8.8 5.6 5.5
100,000 + 8.7 6.0 5.9

The average earnings in the CFDC served areas were also below the overall Ontario average (Table 14). None of the regions had average earning rates as high as the provincial value. The smallest communities had the lowest average incomes and the largest communities had the highest.

Table 15. Average Earnings in Ontario versus CFDC Served Communities
  1995 2000 2005
Ontario 27,309 35,185 n/a
CFDC regions average 24,059 29,011 32,954
East 24,021 29,412 32,994
Southeast 23,296 27,867 32,073
South Central 25,115 31,191 36,041
Southwest 24,631 29,205 33,558
West 24,302 29,377 33,005
Northwest 25,039 29,955 33,110
Northeast 23,046 27,564 31,505
< 10,000 22,906 26,714 31,629
10,000 – 49,999 23,221 28,087 31,033
50,000 – 100,000 24,171 29,348 33,632
100,000 + 25,658 31,292 35,073

The labour market information and the research presented above suggest that small businesses in small communities (i.e., less than 50,000) have a higher level of need for the CF Program than medium and larger businesses in larger communities (i.e., more than 50,000). There may also be a larger need for the Program in certain geographic areas, such as the Northeast and Northwest, which have had consistently higher unemployment rates than Ontario. The Northeast region is also well behind Ontario in average employment earnings.

3.1.2 Complimentarity and Duplication

Finding: Information from the evaluation found that the CF Program, with the provision of access to capital, business services, and CED activities, is a unique program that is not wholly duplicated by any other program. There are other federal, provincial, and municipal programs that provide services similar in nature to certain components of the CF Program, including two FedNor programs, which are often complimentary, rather than duplicative.

Findings in this section are based on: interviews, review of CF, EODP and NODP program documents, the client survey, and the CFDC survey.

Many interviewees (29 of 39) were aware of other programs that compliment/duplicate the CF Program, although many believe that these other programs are complimentary (17 of 29). Note that CFDC board chairs were least aware of other programs. Clients and CFDCs surveyed were also aware of other programs that offer similar services to the CF Program (Figure 2).

Figure 2. Client and CFDC Awareness of Other Programs Offering Similar Services

Figure 2. Client and Community Futures Development Corporation Awareness of Other Programs Offering Similar Services

It is interesting to note that clients were more aware of other programs or organizations that offer financing than the CFDCs. It is also interesting to note that CFDCs were much more aware of other organizations or programs that offered business information and referral services. This may be a result of the CFDC having better access or being better connected to the other types of programs and organizations that offer these types of services.

With respect to strategic community planning and socio-economic development, less than half of CFDCs surveyed (44.8 percent) felt that there were other programs or organizations that offered this service. This was very similar to partners surveyed, with 43.6 percent indicating they were aware of other programs or organizations that offer strategic community planning and socioeconomic development.

Programs External to FedNor

Information gathered from the evaluation showed that the other sources of financing for SMEs include banks and the Business Development Bank of Canada (BDC) and the Canada Small Business Financing Program (CSBFP). With respect to other programs or organizations offering counselling, stakeholders identified the Enterprise Centres, the Ontario Small Business Centres, and the Self-Employment Benefits (SEB) program, which are all operated by the Provincial Government. CED partners surveyed indicated that some municipalities also undertake strategic community planning and socio-economic development and that there are a few other organizations offering this service.

CFDCs and FedNor staff believe there is good cooperation between programs and organizations offering complimentary services (24 of 28). For example, the CFDCs and the BDC have a referral agreement in place, whereby the CFDCs will provide loans to SMEs that are less than $150,000 and the BDC will provide loans to SMEs that are more than $150,000. Some organizations are also co-located, such as with Trenval BDC. Trenval is co-located with the Ontario Ministry of Agriculture Food and Rural Affairs (OMAFRA), Enterprise Quinte, and the Quinte Economic Development Commission. This allows for good cooperation and sharing of information between the organizations and offers clients one-window to business information and services, which minimizes the duplication of services. Information from the case studies also showed that the SEB and CF programs are often used by clients in conjunction with one another. Case study clients interviewed who obtained a loan from the CFDC also often took advantage of the SEB program, which offers participants a small salary for their first year of operation. Clients believe that the combination of the two programs made it possible to take the risk to start a business.

Programs Internal to FedNor

FedNor offers two other economic development programs in addition to the CF Program: the NODP and the EODP. NODP was first established in 1987 when FedNor was created. Its focus is on promoting economic growth in Northern Ontario in five areas: community economic development; trade and tourism; innovation, information and communications technology; business financing support; and human capital. NODP is delivered by FedNor program officers, who work with clients to complete the application process.

EODP was established in 2004 to promote regional economic development in rural Eastern Ontario by investing in projects in five priority areas: business development, skills development, access to capital, retention and attraction of youth, and technological enhancements. The program is delivered through the 15 CFDCs serving the Eastern Ontario region.

A single CFDC does not have access to both EODP and NODP. CFDCs in the West, Southwest and South Central regions have access to neither program.

EODP, NODP, and the CF Program are similar in that they have common objectives and desired outcomes and in some cases, offer similar services (Table 16). All three programs are generally focused on promoting growth and economic stability; diversification; and sustainable, self-reliant communities. While these programs are focused on similar objectives, each takes a different approach in fulfilling these objectives. The CF Program offers repayable financing to SMEs in local communities and provides support to communities for economic development activities (i.e., through technical or financial support). EODP offers contributions to projects that support program objectives and the CFDC can access the funding for its own activities or provide the funds to a third-party through a contribution agreement. The NODP offers repayable and non-repayable contributions and repayable loans for projects that support its program objectives.

The three programs also overlap in the geographic areas served. The CF Program targets all of rural Ontario, EODP targets rural Eastern Ontario and NODP targets Northern Ontario.

The CF Program focuses on local CFDCs; regional, provincial and national CF associations; incorporated, non-profit organizations working to establish a new CFDC; and incorporated, non-profit organizations establishing or administering an Investment Fund Pool. EODP has both primary and secondary recipients. The primary recipients of the Program are the CFDCs of Eastern Ontario and the Eastern Ontario CFDC Network Inc. The secondary recipients include: non-profit organizations, including municipalities and municipal organizations, community development organizations, and associations; legal commercial entities including individuals, corporations, partnerships, cooperatives and trusts; and Aboriginal organizations. NODP focuses on: capital providers; SMEs (less than 500 employees), including corporations, partnerships, cooperatives, proprietorships; and non-profit organizations, including Aboriginal organizations, community development organizations including CFDCs, post-secondary institutions, and municipalities and municipal organizations.

Table 16. Comparison of EODP, NODP, and the CF Program
Component Program
Community Futures Eastern Ontario Development Program Northern Ontario Development Program
Program Objectives Fostering economic stability, growth and job creation; helping to create diversified and competitive local rural economies; and helping to build sustainable communities. Promote socio-economic development; create a competitive and diversified regional economy; contribute to the successful development of business and job opportunities; and sustainable self-reliant communities. Promote growth, economic diversification, job creation, and sustainable, self-reliant communities.
Priorities
  1. Operate local investment funds that provide repayable financing for local businesses
  2. Provide business information and advisory services
  3. Develop and implement strategic community economic development plans
  4. Support community economic development projects
  1. Business and community development
  2. Access to capital
  3. Skills development
  4. Retention and attraction of youth
  5. Technological enhancement
  1. Information and Communications Technology
  2. Innovation
  3. Community Economic Development
  4. Business Financing Support
  5. Trade and Tourism
  6. Human Capital
Delivery Structure Delivered by 61 CFDCs Delivered by 15 CFDCs in Eastern Ontario Delivered by FedNor
Geographic Coverage Rural Ontario Rural Eastern Ontario Northern Ontario
Eligible Recipients
  • local CFDCs;
  • regional, provincial and national CF associations;
  • incorporated, non-profit organizations working to establish a new CFDC; and
  • incorporated, non-profit organizations establishing or administering an Investment Fund Pool

15 CFDCs in Eastern Ontario are the primary recipients. CFDCs may enter into agreements with third parties, including:

  • non-profit organizations, including municipalities and municipal organizations, community development organizations, and associations;
  • legal commercial entities including individuals, corporations, partnerships, cooperatives and trusts; and
  • Aboriginal organizations
  • capital providers;
  • SMEs (less than 500 employees), including corporations, partnerships, cooperatives, proprietorships; and
  • non-profit organizations, including Aboriginal organizations, community development organizations including CFDCs, post-secondary institutions, and municipalities and municipal organizations.
Funding Structure CFDCs may receive non-repayable contributions. CFDC investment funds and investment fund pools are provided in the form of repayable loans, loan guarantees, or equity investment to SMEs, including Social Enterprises. Contributions will normally be non-repayable, unless revenue streams are generated to allow for repayment Both repayable and non-repayable contributions
Complimentarity

FedNor and CFDC interviewees believe that EODP and NODP are complimentary to the CF Program because it allows the CFDCs to access funding to undertake activities to support their CF objectives that may not have otherwise been possible. Between April 2002 and March 2008, CFDCs accessed a total of $75.7 million in EODP and NODP funding (Table 17). These funds were used for capitalization of investment funds, operating costs, bilingual services and CED projects.Footnote 30 A total of $14.5 million was accessed by CFDCs through NODP for capitalization of investment funds. CFDCs accessed a much larger proportion of funds for operating expenses through EODP ($1.2 million) than through NODP ($127K). Ten different CFDCs accessed NODP funds to assist with providing bilingual services. CFDCs accessed both NODP and EODP funds for CED projects, totalling $59.3 million over the past six years.

Table 17. Summary of EODP and NODP Funds Accessed by CFDCs
  NODP EODP Total
Capitalization of investment funds $ 14,502,754 $ 0 $ 14,502,754
Operating expenses $ 127,458 $ 1,203,356 $ 1,330,814
Bilingual services $ 590,477 $ 0 $ 590,477
CED projects $ 24,778,935 $ 34,480,183 $ 59,259,118
Total $ 39,999,624 $ 35,683,539 $ 75,683,163
Duplication

The evaluation identified some potential duplication between EODP, NODP, and the CF Program. The first area of potential duplication is in the services provided. All three programs support CED and have an access to capital component. With respect to access to capital, according to information on these programs, it appears that the recipients are intended to be different types of businesses and/or organizations and therefore there is likely little duplication in this service. However, with respect to CED, it appears possible that similar organizations are being funded through EODP, NODP, and the CF Program to undertake similar kinds of activities.

Information from the annual reports also raises questions as to whether there is some duplicate reporting between EODP, NODP and the CF Program. In the annual reports for the CF Program, CFDCs are required to report CED activity. The CFDCs in the four regions that have access to EODP and NODP funds reported contributions to CED projects in excess of $17 million dollars over a two-year reporting period, while the CFDCs that do not have access to EODP and NODP reported contributions to CED projects of just under $500,000. It is unlikely that the CFDCs in the Northwest, Northeast, Southeast and East regions have contributed such a large amount of funding through their own operating funds for the CF Program and therefore are likely reporting activities done through EODP and NODP under the CF Program. This has been identified as an issue because this means that there is duplicate reporting occurring between NODP and the CF Program and EODP and the CF Program.

3.1.3 Linkages Between CFDC, CF and Industry Canada Objectives

Finding: CFDC objectives are in line with the objectives of the CF program as they both focus on economic growth, diversification and the sustainability of communities. The CF Program objectives are in line with Industry Canada objectives primarily through the priority related to building competitive and stable communities.

Findings in this section are based on: interviews with stakeholders, review of program documents, the CFDC survey, and case studies.

Consistency of CFDC Objectives with National CF Program Objectives

According to foundation documents for the CF Program, the objectives of the program are:

  • economic stability, growth and job creation;
  • diversified and competitive local rural economies; and
  • sustainable communities.

CFDC survey respondents believe that the objectives of the CFDCs are much in alignment with the overall objectives of the CF Program (93 percent). CFDCs indicated that their objectives are to foster economic growth and stability, foster creation and maintenance of jobs, help create diversified and competitive local economies, and help build sustainable communities. All FedNor and CFDC interviewees that were able to respond to this question (31 of 31) agreed that the objectives of the CFDCs are well aligned with the CF Program objectives. Note that two CFDC board chairs were unfamiliar with CF Program objectives.

Consistency with the Industry Canada objectives

According to its Report on Plans and Priorities (RPP), IC has three strategic objectives, which are:

  • a fair, efficient and competitive marketplace;
  • an innovative economy; and
  • competitive industry and sustainable communities.Footnote 31

The priorities under the final objective are to support industrial sectors which are important to the Canadian economy and help Canadians so that they may take advantage of economic opportunities, support business development, provide long-term growth and promote sustainable development.Footnote 32 These priorities are closely aligned with the objectives of the CF Program of economic stability, growth and job creation; diversified and competitive local rural economies; and sustainable communities. Information from FedNor interviewees confirmed this, saying there is a high level of consistency between CF Program objectives and those of Industry Canada (12 of 13). The areas of consistency included sustainable communities (cited by 5 of 12 interviewees); diversification (5 of 12); and improved access to capital (3 of 12).

3.1.4 Role of Federal Government in Delivering CF Program

Finding: The federal government has a legitimate role to play in the delivery of the CF Program and while the province does have a rural plan, there is a perception that the provincial government's involvement in rural economic development has been limited.

Findings in this section are based on: interviews with program stakeholders and document review.

In 2004, the province of Ontario developed its first rural plan for Ontario. This plan outlined the government's commitment to addressing the issues that face rural communities and set three priorities: strong people and strong economy; better health; and success for students. The plan was updated on an annual basis and progress relative to the plans goals and objectives is also reported annually.Footnote 33 In support of this plan, the province offers a number of programs that are available to individuals, businesses, community organizations, and municipalities in rural communities. For example, the Rural Economic Development (RED) Program offers funding for projects that will assist with overcoming barriers to economic development growth. The province also has programs that offer support for infrastructure (e.g., Ontario Biogas System Financial Assistance Program, the Canada–Ontario Municipal Rural Infrastructure Fund, and rural connections). The province also supports the Business Retention and Expansion (BR+E) Program. This program ‘promotes job growth by helping communities learn about issues and concerns of, as well as opportunities for, local businesses and set priorities for projects to address these needs. Ultimately, com munities will have greater success in attracting new business if existing businesses are content with local economic conditions and community support.'Footnote 34

All FedNor senior managers and external stakeholders interviewed (9 of 9) believe that there is a legitimate role for the Federal Government to play in the delivery of the CF Program. The main reason cited for this is a perceived lack of provincial investment in rural development (6 of 9). Similarly, the 2008 Senate report Beyond Freefall: Halting rural Poverty stated that the Community Futures program is "one of the few unequivocal success stories in federal rural policy and…one of the few visible signs of the federal government in rural Canada."Footnote 35 Furthermore, the 2007 Speech from the Throne noted the role of the federal government in "ensuring economic security for all Canadians."Footnote 36 Interviewees also believe that national consistency in the delivery of the Program is more attainable when it is delivered federally (5 of 9) and that it is important to have a federal presence in rural communities (4 of 9). It is interesting then that, while the province has a rural strategy in place, there is a perception that its involvement in rural economic development has been limited.

top of page

3.2 Design and Delivery

3.2.1 Effectiveness of CFDC Networks

Finding: The provincial and regional networks are seen as effective because they are relevant to local conditions. The national network is seen as less effective because of a lack of communication and lower participation due to the cost involved in maintaining the network.

Findings in this section are based on program documentation, stakeholder interviews, and the CFDC survey.

There are three networks in place for the CF Program. A pan-Canadian network was established in 2000 to provide a framework for regular collaboration and sharing of best practices throughout the country. This network works through provincial networks established in each of the provinces and territories, including the Ontario Association of Community Futures Development Corporations (OACFDC) in Ontario. The OACFDC was established in 1994 and works to provide a range of services to its member CFDCs (e.g., professional training and development, sharing of best practices, group benefits, government liaison). In Ontario, seven regional networks have been established (i.e., in each of the seven CFDC regions) with a representative of each of these regional networks liaising with the OACFDC.

Both interviewees and CFDC survey respondents believe that the provincial and regional networks are effective. The majority of FedNor and CFDC interviewees (21 of 25) and almost all CFDC survey respondents (95.7 percent) felt that the provincial network is effective and adds value to the CF Program. Similarly, FedNor and CFDC interviewees (20 of 26) and the CFDC survey respondents (84.8 percent) felt that the regional networks were effective particularly for sharing information and best practices.

There were mixed views on the effectiveness of the pan-Canadian network, with 11 of 21 FedNor and CFDC interviewees suggesting that it was ineffective and did not add much value to the overall program and only 9 of 21 suggesting that it was effective. It is interesting to note that FedNor interviewees rated the pan-Canadian network as less effective than CFDC interviewees. These results are similar to the CFDC survey, as 55.8 percent of respondents rated the national network as effective, while the remainder felt it was ineffective. Reasons for rating the network as ineffective include the cost involved in participating in conferences and a lack of communication. Overall it would appear that FedNor and CFDCs view the provincial and regional networks as more effective because they offer more opportunities for sharing information and are more relevant to local situations.

3.2.2 Issues Impacting the Success of the Program

Finding: The evaluation did not identify any issues that are impacting on the success of the program, and found that the program appears to be working well. However, the lack of performance data may hinder the program's ability to measure its success.

Findings in this section are based on interviews, the CFDC survey, and observations made by GCS during the analysis of available program data (i.e., Quarterly Reports and annual reports).

Due to concerns with respect to the reliability of Quarterly Report data, especially with regards to reporting of business services provided, findings on program success are primarily based on soft data. In examining the data gathered for the evaluation, no issues were identified as having a negative impact on the success of the program, which is likely related to the fact that the CF Program has been in place for a number of years and, over time, modifications have been made to address any previous issues that may have existed. FedNor and CFDC interviewees did, however, believe that funding is an issue impacting the program. Nineteen of 21 interviewees suggested that at a national level, there are issues related to funding, some of which include: restricted funding limits (i.e., maximum loan value of $150,000), insufficient funding available (e.g., for operations, for investment fund), and uncertainty surrounding program renewal. Similarly, when asked about program issues at the provincial level, many FedNor and CFDC interviewees (24 of 28) cited funding issues, particularly related to shortages in operating funds and unequal access to funds for CED projects.Footnote 37 Note that at both the national and provincial level, funding was equally cited as an issue by FedNor and CFDC interviewees. At the community level, both FedNor and CFDC interviewees (11 of 13) expressed some concern about issues related to the management of the CFDCs such as maintaining capacity of staff and recruitment challenges, access to technology and training resources, and board governance.

Forty-six (of 54) CFDC managers and 32 (of 42) CFDC board chairs cited program challenges in the CFDC survey. Funding was the biggest program challenge, cited by 40 CFDC managers and 22 CFDC board chairs. More specifically, there was an identified need for more operational and investment funds. Some respondents (16 of 46 managers and 6 of 32 board chairs) also suggested that administration requirements (e.g., reporting) were a challenge.

Only 21.6 percent of client survey respondents identified challenges in working with the CFDCs. Financial and non-financial clients identified more challenges than CED partners (41 percent or 84 of 203; and 45 percent or 92 of 203, respectively). Only 13 percent of CED partners (27 of 162) cited challenges, the most common of which was general challenges with the governance and/or management of the program (cited by 8 CED partners).

The most frequent challenges cited by financial clients related to limited funding and/or caps on funding levels (20 of 84 respondents). Other challenges which were listed by financial clients included:

  • structure of borrowing and loan conditions (15 of 84 respondents);
  • cumbersome and long application process (9 of 84 respondents); and
  • general challenges with the governance and/or management of the program (6 of 84 respondents).

Surprisingly, some non-financial clients cited limited funding and/or caps on funding levels as the biggest challenge (14 of 92). The survey information did not yield an explanation for this. Non-financial clients also felt that getting information (i.e., correct information, information on services) was a challenge (11 of 92).

While funding challenges were cited by many interviewees and some survey respondents, the evaluation did not provide any evidence to suggest that a lack of funding may be preventing the program from achieving its objectives. There is evidence however, that there are issues related to performance measurement (e.g., completeness and reliability of data) which may be impacting the program's ability to measure its success. This issue is discussed further in section 3.2.5 (Performance Measurement).

3.2.3 CFDC Involvement in Strategic Planning

Finding: CFDCs are involved in strategic community planning and socio-economic development, primarily as participants and not leads, and are aligning their activities with those plans. Where CFDCs are leading strategic planning, they are bringing together community groups in the development of the plans.

Findings in this section are based on the client survey, and interviews. Due to the limitations with respect to the client survey, particularly due to the limited number of partners surveyed that have partnered with a CFDC for strategic planning, GCS's confidence in the reliability of these findings is low.

The majority of partners surveyed indicated that their community has a strategic plan in place (99 of 153) and many of these partners (59 of 92) suggested that the CFDC was a participant in the process, not the lead. This is consistent with the CFDC survey, which showed that CFDCs have been involved in developing 688 community strategic plans, but they were the lead only 32.5 percent of the time (224 times). Information gathered from the case studies suggests that, in some communities, it is the municipality that retains the lead role in community strategic planning. In these communities, CFDCs would be invited as a community partner to participate in the process. CFDCs are more likely to initiate a strategic planning exercise in a community where municipal resources and/or capacity may not exist to do so.

Where CFDCs are taking the lead in strategic planning, interviewees indicated that community consultation, normally led by a third-party consultant, was the primary process used. This is consistent with information gathered from partners, as the majority (10 of 14) said they attended a consultation session.Footnote 38 Interviewees suggested that when CFDCs are leading strategic planning, they have brought together members of the community, with the majority indicating that normally more than 10 organizations and community groups are represented in the consultations. Information from the CFDC survey suggested an even higher level community involvement. The survey showed that the majority of CFDC respondents (43 of 96) consulted 20 or more organizations.

Partners surveyed agreed (17 of 19) that the level of community involvement in strategic planning was appropriate. CFDC interviewees and CFDC survey respondents were in slightly less agreement about community involvement. While many CFDC interviewees (22 of 32) believe that there is a good level of community involvement, 5 (of 32) suggested that it varies from CFDC to CFDC, and 3 (of 32) believe that community representation could be better. Similarly, while most CFDC survey respondents (64 of 93) believe that the level of community involvement has been appropriate, many (21 of 93) were uncertain, citing challenges such as: remoteness and lack of resources, difficulties in engaging the community, and the fact that the CFDCs may not have input if the municipality is leading the process.

Most interviewees (17 of 21), most partners surveyed (76 of 85), and almost all CFDC survey participants (89 of 92) agree that CFDC activities are linked to community strategic plans. Information from the case studies showed that CFDCs generally look for a link between an application / project and a community strategic plan, although they do not necessarily refuse a loan or a project if it does not fit.

3.2.4 Management of Investment Funds

Finding: Generally, investment funds appear to be well-managed as shown by the level of loan activity and growth in investment funds. However, there are some large variations among regions, particularly with respect to the percentage of investment funds in active loans and loan loss rates.

Findings in this section are based on: interviews, document review, Quarterly Report data, and financial information from GCRS.

In examining the management of investment funds, the evaluation aimed to assess whether the number and level of loans was meeting client needs. In TEA, CFDCs tracked the number of applications received from businesses from 2004 on. Information is limited however, because businesses that approach the CFDC about financing, but for various reasons, do not proceed to the application stage, are not captured in this system. Therefore, it is difficult to know the volume of loans provided to businesses relative to the demand for that service.

With respect to the number of loans provided to clients, since April 2002 the number of loans provided by CFDCs each year has been decreasing slightly, from a total of 1,151 loans in 2002-2003 to 974 loans in 2007-2008. However, as the volume of loans has decreased, the average dollar value of loans has been increasing slightly almost every year, rising from $45,362 in 2002-2003 to $52,551 in 2007-2008. It is interesting to note that generally a decrease in loan numbers is matched with an increase in the dollar value of loans. This would suggest that the level of loan activity has been fairly constant over that time period and there has been a trend towards providing fewer loans of higher value. Not surprisingly new CFDCs (i.e., those created since 2001) have lower levels of loan activity than established CFDCs, providing a lower number of loans that are of lower average dollar values. CFDCs that serve regions with a population of less than 10,000 have provided many more loans of smaller dollar values than areas with larger populations. The CFDCs in Southern Ontario, which tend to serve larger populations, have the opposite trend, providing SMEs with fewer loans, but of larger dollar value. This is likely related to the fact that SMEs in smaller communities would tend to be very small SMEs, thus with lower investment requirements.

To further assess the management of loan portfolios, the evaluation examined the proportion of the CFDC loan portfolio that was invested in active loans. Information from the Quarterly Reports shows that while there is a slight decreasing trend in the amount of funds that CFDCs have invested in active loans, there has been little variation over the years (average of 71.8 percent in 2004-2005 and 69.3 percent in 2007-2008). Therefore, CFDCs have maintained between 28.2 percent and 30.7 percent of their portfolio in financial reserves (between April 2004 and March 2008, the dates for which there is data). On a regional basis there is some variation in this figure, with the Northwest region having maintained the lowest amount of financial reserves (24.3 percent) and the East having maintained the highest amount of financial reserves (40.3 percent) (Figure 3). Based on the volume of loans and the financial reserves on hand, it would appear that the East region has the lowest level of loan activity of all other regions. While there are a number of factors that could explain this, such as changes in demand for services, it is more likely related to the fact that the CFDCs in the East are much more active in CED than most other regions. For more on this, see section 3.3.5 (Intermediate Impact of the Program).

Figure 3. Percent of Investment Portfolio Held in Financial Reserves

Figure 3. Percent of Investment Portfolio Held in Financial Reserves

An investment study commissioned by FedNor in 2003 inferred that CFDCs should maintain approximately 30 percent of their lending activity in cash reserve. However they also noted that due to the seasonal nature of loan transactions and capitalization provided by FedNor, the required amount of financial reserves that should be maintained by a CFDC is in the order of 50 percent of the total investment portfolio.Footnote 39 Most regions are within this 30 to 50 percent range. FedNor has not endorsed the recommendations made in this study and has not adopted an official policy with respect to CFDC cash reserve balances within the investment fund.

The management of the investment funds was also analysed through examination of the level of growth of the CFDC's investment portfolio. The general trend with the investments funds has been one of positive growth. The total dollar value of investments funds for all CFDCs grew from $217.2 million in 2004-2005, when data is first available, to $255.5 million in 2007-2008. This represents an overall growth rate of 17.7 percent. While the value of these funds has been growing since 2004-2005, the rate of growth has been smaller each year (Table 18). This is also the case on a regional basis with the majority of regions experiencing slower growth in their investment funds in the past three years, with modest growth in the last year of operation. In fact the Southwest region saw a slight negative growth in the value of its fund, with four of the seven CFDCs in that region recording negative growth. While loan loss rates may provide some insight into the slowed rate of growth, information on loan loss rates is not available for consecutive years. This does not allow for an analysis in the trend in loss rates compared to the trend in investment fund growth. The evaluation did not yield any further explanations for this trend.

Variations in this indicator and others related to investment funds may be attributable to many factors including the need within the community, local economic situation, competing priorities and Board interests.

Table 18. Yearly Growth Rate of Investment Fund (2004-2008)
Year Overall
2004-2005 2005-2006 2006-2007 2007-2008
Dollar value of investment fund ( millions) $217.2 $234.3 $250.4 $255.5 -
Percent growth (with capitalization) - 7.9% 6.9% 2.1% 17.7%

Some CFDCs have received capital injections from either the CF Program or through NODP. Between April 2002 and March 2008, 22 CFDCs received $20.6 million for capitalization through the CF Program, with the CFDCs in the West, Southeast and South Central regions receiving the largest proportion of this funding (26.9 percent, 23.7 percent and 21.2 percent respectively). Since 2001, there have been nine new CFDCs created, including three in the Southeast, two in the West, two in South Central, and one each in the East and the Northwest. The two new CFDCs in South Central received 88.4 percent of capitalization dollars that went to the region, while those in the West received 53.0 percent. In the case of the Southeast, only 20.7 percent ($1 million) of the total capitalization for that region can be attributed to the creation of one new CFDC in 2003.

During that same time period, 16 CFDCs received $14.5 million for capitalization through the NODP program. These injections have influenced the rate of growth of the Northern CFDC investment funds. Although the growth rate is still positive once those injections are removed from the fund, the rate of growth is affected, diminishing from 12.2 percent to 5.8 percent. With only one exception, injections from the CF Program were directed to CFDCs outside of the regions that can access NODP funds (i.e., Northwest and Northeast).

The investment funds in the South Central, West, and Northeast regions benefited proportionally more from capitalization dollars. When these dollars are removed from calculations of investment fund growth their funds grew much more slowly (Figure 4). In South Central and the West this can be most likely attributed to the new CFDCs. New CFDCs have investment funds which are about a third the size of an average CFDC fund when capitalization is not included. These added dollars represent a large proportion of their fund. The three regions which have access to neither EODP nor NODP (i.e., South Central, Southwest and West) all have below average loan portfolio values. These regions are not able to accrue as much investment income as a result, making them more reliant on capitalization to grow their funds.

Figure 4. Rates of Investment Fund Growth, With and Without Capitalization

Figure 4. Rates of Investment Fund Growth, With and Without Capitalization

There is limited information available on loan loss rates. While CFDCs provide information that could be useful for loan loss rates, this information is not compiled and therefore is not available for analysis. A survey administered by FedNor in April 2006 provided information on loss rates for fiscal year 2005-2006. This survey was not been replicated in subsequent years. According to this survey, the average loan loss rate for all CFDCs was 10.8 percent (Table 19). However, there are very large ranges in loss rates across regions and CFDCs, with three CFDCs reporting a zero percent loss rate and others reporting loss rates of 20 percent and higher (5 CFDCs). In comparison with traditional financial institutions, the loss rates on CFDC loans are significantly higher, although not surprisingly given that CFDCs are mandated to provide loans to higher risk clients, whereas financial institutions generally lend on a lower risk basis.

Table 19. Comparison of Loan Loss Rates
Organization Average Loan Loss Rate
CFDC (average for 2005-2006) 10.80 %
Financial InstitutionsFootnote 40 (average 2001-2006) 0.2 %
Business Development Bank of Canada (2006) 1.00 %

FedNor has not established a target range for loss rates and stakeholders had quite varied opinions. Some FedNor and CFDC interviewees suggesting loss rate levels are appropriate and some CFDC survey respondents suggesting that loss rates are too high. It is therefore difficult to determine whether or not these rates should pose a concern.

Recommendation #1: FedNor should work with CFDCs to establish general target ranges related to desirable levels of funds in active investment and loan loss rates based upon local realities. This would allow FedNor to assess a CFDC's loan portfolio performance and assist with improvements where necessary.

Overall, the CFDCs with the largest populations (i.e., 100,000 and up) had the smallest loan portfolios and investment funds, and the highest percentage of cash on hand. Based upon observations during the case study in Brantford, this may be as a result of increased availability of capital for small business generally in larger communities. Thus the CFDCs in these regions are serving smaller, less bankable, clients who are approved for smaller, shorter loans. These smaller, riskier loans may also be associated with higher loan loss rates. Unfortunately data on loss rates was not available for each CFDC, so cross-tabulation is not possible.

Interestingly the CFDCs with the smallest populations (10,000 or less) had the reverse, with large loan portfolios, large investment funds and low percentage of cash on hand. They also had above average number of closed deals annually, and above average value of new loans. Despite this, these CFDCs have well below average investment fund growth. Of the ten CFDCs with small populations, seven are found in the two northern regions, both of which have long standing investment pools. When a loan of up to $500,000 is made from one of these pools, the host CFDC has the associated loan and job statistics attributed to it. Thus, a sparsely populated CFDC may have a large loan portfolio on the strength of only one or two loans. On average though, these less populated CFDCs would have many small, loans, which would likely suffer from similar loan loss rates as the larger CFDCs.

3.2.5 Performance Measurement

Finding: While tools are in place for data collection and FedNor staff feel that performance data is sufficient, some CFDCs are not fulfilling reporting requirements, data provided is not reliable, and TEA is being under utilized. FedNor management of the program is not optimized.

Findings in this section are based on interviews, document review, and observations made throughout the data collection and analysis process.

Data Collection and Performance Reporting Requirements

FedNor has established data collection and reporting requirements for the CF Program (summarized in Table 20). This process begins with the development of a CFDC business plan, which outlines a CFDC's future goals, objectives, and planned activities and is essentially a proposal for funding for the operation of the CFDC. These plans have been required annually since program inception, although since April 1, 2002, some CFDCs operate under three-year operating agreements.Footnote 41 These plans are intended to form the basis against which CFDC performance will be assessed.

Beginning in April 1996, CFDCs requesting capitalization of their investment funds were required to submit an investment fund report. These reports provide historical and current information regarding a CFDC's investment fund status and activities, including growth of the fund. Around the same time, FedNor first purchased TEA, which is the information system in place to record administrative data concerning the program, including program outcomes.

Beginning April 1, 2002 CFDCs were required to provide annual reports to FedNor in a more standardized format. Previously, there was no consistent format for reporting performance. These reports provide an overview of the activities of the CFDCs and are mostly narrative with some quantitative statistics (e.g., number of loans, dollar value of loans, CED projects supported) included.

FedNor has also recently developed and distributed a business planning handbook (Spring 2008) to CFDCs which provides them with information on reporting requirements, including what data is required and timelines for reporting. The document has been distributed to CFDCs, but the extent to which it has been reviewed by CFDCs is not known.

Table 20. Summary of Performance Reporting Requirements
Requirement Effect Date Summary of Purpose Frequency
CFDC Business Plan Since program inception The business plans provide an overview of the CFDC region, economy, challenges, etc., and present goals, objectives and activities for the CFDC for the coming agreement period. Every year or every 3 years depending on the CFDC
CFDC Annual Report April 1, 2002 Provide an overview of the activities of the CFDC each year. Mostly narrative, with some quantitative statistics (e.g., number of loans, dollar value of loans, CED projects supported). Annual reports are intended as a way to measure performance against indicators in the business plan. Annually
Investment Fund Report April 1, 1996 Provides historical and current information regarding a CFDC's investment fund status and activities. This is a requirement for CFDCs requesting capitalization of their investment fund. Each year, as an attachment to the annual report
TEA Quarterly Reports First purchased for use in 1996/1997 (not mandatory) Provides information on the performance of each CFDC (mainly output-related, but some outcome indicators are reported). Quarterly
CFDC Compliance with Reporting Requirements

Information from the evaluation showed that some CFDCs are not complying with established reporting requirements. This is less of an issue with business plans because these plans provide the basis for funding arrangements between CFDCs and FedNor. According to a FedNor representative, CFDCs would not receive a funding agreement without that plan. The issue with respect to reporting compliance lies with the annual reports and TEA Quarterly Reports.

The first annual reports, using the new standard format, were submitted for the period starting April 2005 and generally include CFDC reporting periods between April 2005 and December 2006. FedNor currently keeps a log of the CFDCs that have provided an annual report. As discussed in the limitations of the methodologies, CFDCs submitted 43 and 34 annual reports in year 1 and year 2 respectively (of 61 CFDCs). Of the reports submitted, some were not in the standard format and not all reports contained information on CED activity. There have not been any consequences imposed on CFDCs that have not provided annual reports, thus there is no incentive for CFDCs to complete these reports.

With respect to TEA, the system is not mandatory and currently there are at least three CFDCs that do not use the system. A FedNor representative indicated that FedNor program officers in the field enter the information on behalf of those CFDCs.

Availability of Data

A review of TEA data showed that the system does not capture or generate all indicators required for performance measurement as per the CF RMAF. For example, it is not possible to determine the number of jobs created by sector, which is an indicator established to measure the impact of the Program on economic diversification. In addition TEA is limited to providing information related to program outcomes for loan clients (e.g., jobs created, businesses started) and does not track the longer-term impacts on businesses (e.g., impact on revenue, survival rate of businesses). There is also no system in place to capture the impact of CED work.

The system is also limited in providing information that would be useful to manage the program. While CFDCs are able to enter large amounts of information related to their loan portfolio, business services, and CED activity, it is limited in its ability to extract the information, being able to provide only the information that is in the pre-programmed quarterly reports. Note that the projects module (i.e., for CED) is not currently in use by the CFDCs. Also, through gathering information for the case studies, it was discovered that it is not possible to generate community-specific data. In one case, the CFDCs had been keeping separate records so were able to provide community-specific data; a second generated the data manually; and the third provided an estimate of the proportion of work in the community versus the region and the figures were extrapolated using that proportion.

Based on these findings, it appears that TEA has the capability to provide a complete data collection and reporting system for all CFDCs activities; however it is currently being under-utilized. TEA needs additional fields to capture data identified in the RMAF.

Use of Data

A few FedNor interviewees indicated that performance data is used primarily to measure program performance (3 of 13) and to make decisions about the program (3 of 13). According to a FedNor program representative, program officers in the field have access to all of the various data collected through the tools in place and use the information to ensure that CFDCs are on track with commitments set in their business plan and operating agreements.

Generally FedNor interviewees believe that they have enough information available to make decisions about the program. However, in reviewing the performance measurement data, the evaluation found that much of the information is not actually compiled in any way. FedNor has not yet been able to compile information gathered from the investment fund reports and it has only recently begun to scan in annual reports received and compile the information in a Microsoft Access database. This raises questions as to whether information being collected is used in any way to make decisions or to manage the program.

Data Reliability

Most FedNor interviewees (10 of 13) feel that performance information is reliable. However, observations made during data collection suggest that there are issues with respect to data reliability. In reviewing the TEA data, numerous outliers were found and were removed from the data set for analysis purposes. For example, in 2003, three CFDCs reported average loan values in excess of $800,000, which far exceeds the maximum allowable loan value. Those CFDCs confirmed that it must be an error, but were unable to identify the source of the error. Note that a senior FedNor representative indicated that verification of the data is conducted in the field; however this was not confirmed in the course of the evaluation.

With respect to the outcome information entered in TEA (e.g., jobs created), this information is estimated at the time of application and no verification is done to determine whether that was in fact the actual job impact. It is expected therefore, that the recorded impact information may be higher than the actual impact.

The lack of performance information and the unreliability of the information that is collected raises questions as to whether FedNor is able to effectively manage its program.

Recommendation #2: FedNor should complete a review of the performance data collected for the CF Program to ensure reliable and meaningful reporting and establish additional indicators to provide information for assessing the longer-term impacts of the program, particularly for CED activities. In addition:

  • FedNor needs to ensure that c is capable of capturing and reporting on all required performance information. It is possible that FedNor could reduce reporting requirements for CFDCs by maximizing its use of TEA and thereby improving its ability to manage the Program as well as make it more useful for CFDCs.
  • FedNor should ensure that all CFDCs are fulfilling all reporting requirements and should establish consequences for those CFDCs that do not comply with the requirements.
  • FedNor should ensure that CFDCs are not reporting activities undertaken and outcomes achieved through the use of EODP and NODP funds in CF Program reports (i.e., in the annual reports).
  • To ensure data reliability, FedNor should implement a system whereby TEA data captured and submitted in the Quarterly reports would be reviewed on a regular basis; FedNor needs to work with the CFDCs to ensure data is corrected should issues be identified.
top of page

3.3 Program Success

3.3.1 Appropriateness of Information, Referrals and Counselling to Clients

Finding: The provision of loans, business information, and counselling by CFDCs has been fairly consistent over the four-year period between 2004 and 2008. Clients believe there is a great need for these services and clients are very satisfied with the services provided.

Findings for this section are based on the Quarterly Report data and the client survey. Due to concerns with respect to the reliability of Quarterly Report data, especially with regards to reporting of business services provided, findings on program success are primarily based on soft data and hence the level of confidence in these findings is low.

As discussed in section 3.2.4 (Management of Investment Funds), the number of loans provided by CFDCs has decreased slightly, as the average dollar value of loans has increased. This suggests that the level of loan activity has been fairly constant over that time period. For the past four years, CFDCs have been tracking business activity (i.e., general inquiries and in-depth counselling) in TEA. According to this system, the number of general inquiries received by CFDCs has remained fairly consistent over that time period. As with the volume of loans, the newer CFDCs have had fewer general inquiries than established CFDCs, which suggests that newly established CFDCs may take a few years to fully establish their business services and loan portfolio.

Between April 2004, when data was first recorded, and March 2008, the CFDCs received a total of 202,961 general inquiries, and provided a total of 32,759 in-depth counselling interviews to clients. There are great differences between the numbers of general inquiries being reported, with some CFDCs reporting in excess of 6,000 annually, while others report fewer than 30 for all four years. Similarly, in-depth counselling ranges from three sessions per year to 896. These substantial differences call into question the validity of the data as some CFDCs may have different reporting practices for these services. However, demand for these services also likely plays a role.

The numbers of these services provided vary greatly by region, with the West providing the highest numbers for both of these services, with an average of 1,190 general inquiries and 254 in-depth counselling interviews per year (Figure 5). The Northeast and South Central regions also receive high numbers of general inquiries (average of 1,119 and 984 respectively), while the Northwest has received the fewest numbers of inquiries each year (average of 239). The West and Southeast regions have provided the highest number of in-depth counselling interviews (254 and 183 per year on average, respectively), with the Southwest and Northeast regions providing the lowest numbers of in-depth counselling (65 and 34 per year on average, respectively). There are few explanations for these variations other than that there are likely different levels of demand for these services depending on the region.

Figure 5. Regional Comparisons of Number of Services Provided

Figure 5. Regional Comparisons of Number of Services Provided

The CFDCs which have awarded the fewest loans had well below average general inquiries and above average in-depth counselling. This would suggest that there is less interest in the services provided by the CFDCs, but for the loans that they provide, they take a more active role in supporting the business. The CFDCs which awarded the most loans had well above average level of in-depth counselling, which implies they are working with each of their loan clients thoroughly. However, they had below average general inquiries. No reasonable explanation for this can be found, except to assume that it is related to demand.

The CFDCs serving the smallest populations (less than 10,000) each averaged double the number of general inquiries than their larger compatriots (1,468 versus 707) annually. As mentioned in 3.2.4, they also provide more loans than more populous CFDCs. Like other frequent lenders, they have much lower in-depth counselling figures. As the population in the region increases, more in-depth counselling is recorded. Larger communities may have more services in the community, so the CFDC can direct more small businesses to these alternative services. Larger communities also have more start-ups as clients, who would generally require more counselling, which would account for the increased number of counselling sessions.

Figure 6. Population Comparisons of Number of Services Provided

Figure 6. Population Comparisons of Number of Services Provided

As discussed in section 3.1.1 (Need for the Program), the results of the client survey showed that there is a high level of need for services offered by CFDCs, with access to capital services being rated as the services most needed within the community. Overall clients are very satisfied with all services provided by CFDCs, including access to capital and training seminars. Not surprisingly, access to capital services received the highest satisfaction rating, with 76.5 percent being very satisfied and 18.8 percent being somewhat satisfied (Figure 7). Overall, clients were least satisfied with the counselling services provided, although that service still received a very positive rating overall (73.0 very satisfied, and 19.2 somewhat satisfied). The general level of satisfaction was consistent across regions; with the West being least satisfied of all regions with the referral and counselling services and the Southwest being the least satisfied of all regions with access to capital services.

Figure 7. Client Satisfaction With CFDC Services

Figure 7. Client Satisfaction With Community Futures Development Corporation Services

3.3.2 Impact on Business Knowledge and Skills of Clients

Finding: CFDCs have improved clients' business skills and knowledge and clients are using these skills for financial management, marketing and applying for funding, although to a limited extent.

Findings in this section are based on the CFDC and client surveys. All CFDC survey respondents (managers and chairs) felt they had been successful in improving the business skills and knowledge of clients (67.4 percent said very successful, 32.6 percent said somewhat successful). This is consistent with what was reported by clients, who indicated that CFDCs had been very successful (49.3 percent) and somewhat successful (42.3 percent) in improving their business skills and knowledge, although CFDCs were slightly more positive in this respect than clients. Only a small proportion of clients did not agree that the CFDC had improved their business skills and knowledge (5.8 percent indicated not very successful and 2.6 percent indicated not at all successful). In examining the different client groups, CED partners believed that CFDCs were slightly more successful in improving their business skills and knowledge than financial and non-financial clients (Figure 8). Note that these responses were consistent between regions, with no noteworthy differences.

Figure 8. Success at Improving Business Skills and Knowledge of Clients

Figure 8. Success at Improving Business Skills and Knowledge of Clients

Financial and non-financial clients were asked to indicate to what extent they were using the information obtained from the CFDC. While most clients indicated they are using the information obtained, it appears to be to a limited extent. Few financial or non-financial clients are using information for human resource management, completing a funding application, or for project management (Figure 9).

CFDC services appear to be most useful for clients in assisting with the financial management of their business (28.9 percent said great extent, 24.0 percent said some extent). Clients are also using information to develop a business plan (17.0 percent said great extent, 22.0 percent said some extent), to market their business (12.6 percent said great extent, 25.6 percent said some extent), and to access other funding sources (10.5 percent said great extent, 24.1 percent said some extent). Not surprisingly, non-financial clients are using CFDC services to develop a business plan more than financial clients. This would likely be related to the fact that financial clients are more likely to have already completed a business plan (i.e., because they have already received a loan) than non-financial clients. Again, these responses were fairly consistent at a regional level, with no noteworthy differences.

Figure 9. Client Use of Information Obtained From CFDC

Figure 9. Client Use of Information Obtained From Community Futures Development Corporation

3.3.3 Impact on New and Existing Businesses

Finding: CFDCs have created new business start-ups and have helped to maintain and strengthen existing businesses.

Findings in this section are based on interviews, the CFDC survey, the client survey, and the Quarterly Report data. The Quarterly Report data related to the number of loans and their value is considered reliable given that this data is pulled directly from CFDC financial systems. Other data within the Quarterly Reports presents concerns with respect to reliability.

Almost all interviewees (35 of 39) believe that the CF Program has been successful in creating new business start-ups. CFDCs and clients surveyed have similar views, both suggesting that the Program has been successful in creating new businesses. CFDCs (managers and chairs) were slightly more confident with all respondents saying very successful (69.8 percent) or somewhat successful (27.1 percent). A few clients expressed some doubt that the CF Program had been successful in creating new business start-ups, with 7.1 percent suggesting it had not been very successful and a further 3.9 percent suggesting that it had not been successful at all (Figure 9).

Data obtained from the Quarterly Reports confirms what clients and CFDCs suggested. CFDCs provide loans to both new and existing businesses. As discussed in section 3.2.4 (Management of Investment Funds), over the six years 6,963 loans were awarded, with the trend being slightly negative in recent years. However, the average value of those loans has increased slightly over the years. Therefore, it can be said that the CFDCs are making fewer loans, but the value of those loans is increasing, resulting in a fairly constant level of lending over the six years. The only exception to this are the new CFDCs which have fewer loans and lower average loan value, suggesting it takes time to develop a clientele and build an investment fund.

Business Start-ups

Overall the number of businesses started as a result of both investment funding and business services is rising. Over the six years of study 2,888 new businesses were started as a result of loans provided by the CFDCs. Over the six years the trend has been fairly consistent in business start-up due to investment funding. The Southeast and the Southwest regions were very successful in starting new businesses, being at or above average every year with 13.8 and 11.0 businesses started per year. The East, West and Northwest were consistently below average with 4.6, 6.3 and 6.0 businesses started annually per CFDC, respectively. The number of businesses started seems directly linked to the loan activity. The least populated CFDCs helped start on average 11.8 businesses per year, which is above average. This seems reasonable given that they also provide more loans than average. Similarly, the CFDCs which provided on average 30 or more loans per year were well above the average for the number of start-ups businesses annually (average of 14.5). Those who averaged 10 or less loans per year helped start fewer businesses (3.8 on average).

If a business that had yet to have any sales revenue came to the CFDC for in-depth counselling, the efforts by the CFDC would be counted towards a business start-ups as a result of business services. Some of these business services clients might also be investment fund clients and would be counted twice in the overall trend in new businesses. Nonetheless, the trend in the number of businesses started as a result of business services is positive. In total 5,340 new businesses began as a result of business services. Three CFDCs, Huron and Sarnia–Lambton in the West region and Nottawasaga in South Central, reported being far more active in using business services to start businesses as they alone represent 49.1 percent of total new businesses. When all businesses impacted are counted these three represent 44.4 percent of the 11,542 businesses impacted through business services.

Information from the client survey also shows that the Program has had an impact on the creation of new businesses, as the majority of loan clients surveyed who received a loan for a start-ups (71.2 percent), suggested that they would not likely have been able to start their business without the loan from the CFDC. (Figure 10).

Figure 10. Client and CFDC Opinions on the Success of the CF Program in Creating New Businesses

Figure 10. Client and Community Futures Development Corporation Opinions on the Success of the Community Futures Program in Creating New Businesses
Existing Businesses

Over the six year study period, 10,010 existing businesses were assisted through both investment and business service activities, with a gradual increasing trend. CFDCs in the West and South Central regions provided more assistance to existing businesses than the provincial average, as would be expected by the presence of Huron, Sarnia–Lambton and Nottawasaga in these regions. The investment funds alone have assisted 3,808 existing businesses over the six years. CFDCs in the Southeast and Northeast impacted the most existing business (14.9 and 13.3 per year respectively). The CFDCs in the South (i.e., South Central, Southwest and West) awarded below average numbers of loans and assisted fewer existing businesses.

As the population served by a CFDC decreases, the more existing businesses, and businesses overall, it assists. This is a similar trend to the number of loans provided (i.e., as the population decreases, more loans are provided). This is most likely reflective of the demand for the loans and the lack of other sources of investment in the area. In other words, CFDCs in smaller communities have more clients.

Again, information from the client survey also shows that the Program has had an impact on existing businesses, as the majority of loan clients surveyed who had received a loan for an existing business, suggested that they would not likely have been able to maintain or expand their business without the loan from the CFDC (70.0 percent).

Overall, roughly 45.1 percent of businesses impacted by CFDC activity are new businesses and the remainder are existing ones. This percentage has not changed much over the six years. In the two northern regions the focus appears to be on maintenance of businesses, as the percentage of new businesses impacted drops to 39.3 percent. The case study in Kirkland Lake revealed that northern CFDCs may be more focused on helping to stabilize their communities in the face of fluctuating commodity markets. Assisting with starting new businesses is of secondary concern. In contrast, the CFDCs with the greatest populations (i.e., 100,000 or more) are more focused on start-ups, with 53.1& nbsp;percent of their business impact being felt in that group. These CFDCs are found predominantly in the South of the province and tend to have more diversified economies. They also have more options for credit in these regions, so an existing firm with a credit history would not have to rely as greatly on CFDCs for assistance.

The impact on businesses can also be measured by examining the ability of businesses to leverage other funds and whether they have experienced an enhanced ability to access capital from traditional sources. According to the Quarterly Report data, clients have reportedly leveraged funds in the amount of $497.5 million. This translated into a ratio of 1:1.63, meaning that for every dollar loaned by the CF Program, clients are obtaining $1.63 from other sources. The CFDCs with the fewest closed deals and below average value of new loans also had below average leveraging. However, the southern CFDCs (South Central, Southwest and West) had below average numbers of closed deals, yet they were above average in leveraging. While the evaluation did not yield any explanations for this, it could be related to a greater availability of other funding sources.

Information from the client survey shows that a small proportion of clients leveraged funds (16.0 percent of financial clients). Note that because clients were not initially asked whether they needed to leverage other funds, this is not necessarily an indication that clients are not successful in leveraging funds. Clients that leveraged funds indicated that the primary source of those funds was a financial institution (56 percent). Clients also leveraged funds through other federal programs (16. 0 percent) and from the BDC (8.0 percent). Of those clients that leveraged funds, 36.8 percent were able to leverage more than $150,000. Another 45 percent of clients leveraged between $10,000 and $74,999.

Both clients and CFDCs reported that the CF Program has enhanced client ability to access capital from traditional sources (Figure 11). Again, not surprisingly, CFDCs were more confident in their level of success in this respect, with only two CFDC respondents somewhat disagreeing. Clients cited the CF Program as being very successful (46.5 percent) or somewhat successful (33.4 percent) in enhancing their ability to access capital from traditional sources, with only a small percentage disagreeing (8 percent somewhat disagreed, 5.6 percent disagreed). Note that a fairly high number of respondents could not answer this question (29 percent). It is possible that these respondents had not yet tried to seek funding from traditional sources since obtaining the CFDC loan. Note that there was concern expressed during the case studies that the high collateral requirements from CFDCs made it difficult to obtain financing from other sources. However, collateral requirements are set by each CFDC and this may not be a concern across the province or the program.

Figure 11. Client and CFDC Opinion on Enhanced Ability to Access Funds from Traditional Sources

Figure 11. Client and Community Futures Development Corporation Opinion on Enhanced Ability to Access Funds from Traditional Sources

3.3.4 Needs of Official Languages Minority Communities (OLMCs)

Finding: CFDCs are serving the needs of Official Languages Minority Communities (OLMCs) by providing advice, support and services in both official languages where required, though there are some remaining challenges related to maintaining skilled, bilingual staff and costs associated with providing bilingual information materials.

Findings in this section are based on interviews, the client survey and the Quarterly Report data. Eighteen of the 61 CFDCs provide services to regions in which OLMCs are located. The majority of those CFDCs are located in the Northeast region (10 of 18), with none located in the Southeast region.

During the four years since the number of businesses assisted was tabulated in TEA (April 2004 to March 2008), 1,574 francophone businesses were assisted, which represents 7.7 percent of the total businesses assisted by all CFDCs in those years. During that same time period 5.9 percent of the new loans went to francophone businesses, which represented 6.6 percent of the loan value for Ontario. In looking just at the 18 bilingual CFDCs, there were 1,090 francophone businesses assisted which represents 19.5 percent of the total volume of business assisted by those CFDCs. In addition, francophone clients constituted 15.5 percent of the new loans provided by these CFDCs, representing 16.2 percent of the total dollar value of loans provided by bilingual CFDCs. Only two bilingual CFDCs had less than 5 percent of their loans going to francophone businesses. It is interesting to note that two CFDCs, which do not serve OLMCs, provided services to francophone businesses more than 5 percent of the time. One CFDC serving OLMCs did not record any services directed at francophone businesses. It is unclear whether it is not providing any bilingual services or whether that CFDC simply did not report those figures in TEA.

FedNor and CFDC interviewees generally agreed that CFDCs were able to provide bilingual services where required. Of the senior managers interviewed, all (4 of 4) suggested that the CFDCs are providing bilingual services. Of the FedNor program officers interviewed, seven of eight work with CFDCs that serve OLMCs. All but one indicated that there are no issues with CFDCs providing bilingual services, with the other officer suggesting that improvements are needed in this area. Similar responses were received from the CFDC interviewees that serve OLMCs (9 of 21). The majority indicated that they are providing bilingual services, with one suggesting that it is not fully capable (i.e., some materials are not available in French). This is consistent with information from the client survey, which showed that virtually all financial and non-financial clients received service in the language of their choice (99.5 percent). The remaining clients indicated that they were just served in English or that materials were not available in French.

Therefore, CFDCs appear to be successful in providing bilingual services where required, however there are on-going challenges. Interviewees agreed that there are challenges associated with maintaining bilingual staff (12 of 19); the cost involved in providing bilingual material (12 of 19); and the fact that in some regions, there is little demand for the services (12 of 19).

3.3.5 Intermediate Impact of CF Program

Finding: The CF Program has supported community economic development, although it appears to be by varying degrees between CFDCs. The Program has also assisted communities to develop and diversify their economies, and has strengthened community capacity, although the extent to which the program has contributed to these outcomes is difficult to quantify.

The findings in this section related to CED are based on information from the annual reports, case studies, the CFDC survey, and the client survey. Due to the limitations with the annual reports (i.e., they are incomplete), this information is considered unreliable. The findings with respect to economic diversification and community capacity are based only on the CFDC and client surveys and therefore GCS is limited in drawing conclusions in this respect.

Community Economic Development

Foundation documents for the CF Program state that CFDCs receive funding to, among other activities, support community-based projects and special initiatives by collaborating with other partners in the public sector and civil society to implement strategic community projects or deliver special initiatives targeted to communities. With restricted budgets for CED activities through the CF Program, CFDCs attempt to undertake this work through mobilizing partners and resources.

CED activities are reported in the CFDC annual reports. As discussed in section 3.2.5 (Performance Measurement), not all CFDCs have submitted annual reports and of those provided, some are incomplete and do not contain CED information. Nonetheless, the information from the annual reports received and information from the CFDC and client surveys showed that CFDCs are involved in CED and provide both financial and technical support to organizations and/or projects. Of the CFDCs that have reported CED information, it appears that CFDCs that have access to NODP and EODP funds are more involved in CED than other CFDCs.

Volume of CED Projects

According to the information compiled from the annual reports, CFDCs either led or participated in 815 CED projects in year 1 and 908 projects in year 2.Footnote 42 The information shows that the CFDCs in regions that have access to EODP and NODP funds (i.e., East, Northeast, Northwest, and Southeast) have been more involved in projects than the other CFDCs, with those regions accounting for 74.5 percent of the total CED projects in year 1 and 87.6 percent of the total CED projects in year 2 (Table 21). The CFDCs in South Central, Southwest and West regions have been the least involved in CED projects, although these regions do not have access to EODP and NODP. It is important to note that because not all CFDCs provided annual reports or reported CED information, these CED numbers do not necessarily represent the true number of projects supported and while it does appear that the regions with access to EODP and NODP are more involved in CED, that may not necessarily be the case.

Table 21. CFDC Involvement in CED Projects
Year 1 Year 2
# of CFDCs Reporting CED # of projects 3Percent of Total Projects # of CFDCs Reporting CED # of projects Percent of Total Projects
CFDCs that have access to EODP 9 of 15 374 45.9 11 of 15 416 45.8
CFDCs that have access to NODP 11 of 24 233 28.6 10 of 24 379 41.7
CFDCs that have access to neither EODP nor NODP 11 of 22 208 25.5 7 of 22 113 12.4
Total 31 of 61 815 100.0 28 of 61 908 100.0

According to the CFDC managers surveyed, CFDCs have varying degrees of involvement in CED projects, with 11.5 percent involved in less than five projects a year; 11.5 percent involved in five to nine projects a year; 17.3 percent involved in 10 to 14 projects a year; and 53.8 percent are involved in anywhere from 15 to 50 per year. Note that a small number of CFDCs (3) suggested that they are involved in at least 100 CED projects.

Dollars Contributed to Projects

CFDCs reported contributions of $11.1 million to CED projects in year 1 and $7.2 million to CED projects in year 2. Again, according to this information CFDCs that have access to EODP and NODP funding have provided the most funding for CED projects, with those regions contributing 97.6 percent of the total funding provided to all CED projects reported in year 1 and 96.8 percent of the total funding provided in year 2 (Table 22).

On average, CFDCs have contributed less than $10,000 per CED project supported (average of $10,000 per project in year 1 and $6,488 per project in year 2). The average contributions per project reported in the annual reports are consistent with the information provided by partners. The majority of partners surveyed (58.5 percent) received less than $10,000 in financial support for their project. This was also similar to information collected from partners during the case studies, many of whom received less than $10,000.

Table 22. Dollars Contributed to CED Projects by CFDCs
  Year 1 Year 2
# of CFDCs Reporting CED Dollars Contributed to Projects Percent of Total Dollars Contri-
buted by CFDCs
# of CFDCs Reporting CED Dollars Contributed to Projects Percent of Total Dollars Contri-
buted by CFDCs
CFDCs that have access to either EODP or NODP 20 of 39 $10,855,346 97.6 21 of 39 $6,985,403 96.8
CFDCs that have access to neitherEODP nor NODP 11 of 22 $ 266,468 2.4 7 of 22 $233,121 3.2
Total 31 of 61 $11,121,814 100.0 28 of 61 $7,218,524 100.0

Information in the annual reports showed that CFDCs are one of many possible partners contributing to a CED project and CFDCs are providing only a small portion of the total funding. In year 1, the contributions made by the CFDCs constituted only 31.7 percent of the total funding for the projects. In year 2, this proportion was even smaller, with CFDC project funding constituting only 17.4 percent of the total project funding. While CFDCs may not be contributing a large amount of funds to projects, partners interviewed during the case studies said that the CFDC contribution was important, leveraged other partner funding and that the project may not have proceeded without it. Similarly, the majority of partners surveyed (64.7 percent) indicated that they would not likely have been able to undertake the project without the support of the CFDC.

Technical Support

Information from the annual reports, case studies, and partner survey showed that in addition to providing financial support, CFDCs are providing a large amount of technical support to projects. Fifty-nine percent of partners surveyed said that they received technical support from the CFDC for their CED project (34 percent of those also received financial support), mainly in the form of business advice/planning/strategy and information support (55.4 percent) and human resource support/volunteer time (32.6 percent). This is consistent with case study information, as partners interviewed suggested that both board members and staff spend much time providing technical support to projects such as sitting on committees, advising on issues such as project and financial management, offering expertise for or carrying out studies and research, and actually working on-site for a certain number of hours per week.

CFDCs surveyed indicated that they are spending anywhere from a few hours per project to more than 250 hours. This is consistent with the information in the annual reports. CFDCs reported a total of 87,280 volunteer hours for CED projects in year 2 and 123,858 hours in year 2, with the number of volunteer hours spent per project ranging from 6 to 250. Unlike the number of CED projects and dollar values contributed, access to EODP and NODP does not seem to influence the amount of volunteer time contributed to projects as some regions without access to those programs recorded higher levels of volunteer time than those regions with access to those programs. This seems reasonable considering EODP and NODP are funding programs. Note that there does not seem to be a correlation between the numbers of projects supported, dollar value of projects, and volunteer time contributed. One might expect that if a CFDC is providing large financial support, it may not provide as much technical support and vice-versa; however this does not appear to be the case.

CED Outcomes

The CED information currently gathered in CFDC annual reports provides output-level information on CED (e.g., number of projects, dollar value of contributions), however, does not provide information on the impact of the work on the community. Therefore, this information was asked of partners surveyed. According to partners surveyed, the biggest impacts of CED work have been that it mobilized businesses or community groups and improved or developed business knowledge. Partners also agreed, to a slightly lesser extent, that CED work helped to revitalize a particular sector and created or maintained jobs (Figure 12). It is important to note that while regions with access to either EODP or NODP appear to be more involved in CED, there is no information to show whether they have a greater impact on their communities than the other regions.

Figure 12. Partner Opinion on the Impact of CED Work in Community

Figure 12. Partner Opinion on the Impact of Community Economic Development Work in Community
Economic Diversification

FedNor representatives, CFDCs and external stakeholder believe that the CF Program has helped communities diversify their economies (20 of 24). This is consistent with information gathered from the client and CFDC surveys. Not surprisingly, CFDCs were much more in agreement than clients and in fact no CFDCs surveyed suggested that the Program had not been successful in helping communities diversify their economies. Most clients were in agreement (74.2 percent); however some clients did not agree that the program had helped communities diversify their economies (11.3 percent), with non-financial clients being in slightly less agreement than the other two client groups (Figure 13). A small proportion of clients were neutral with respect to their opinions (14.4 percent) and it appeared more difficult for financial and non-financial clients to judge this than CED partners, as 19.7 percent of financial clients and 14.3 percent of non-financial clients were neutral, versus 11.8 percent of CED partners.

Figure 13. Client Opinion on the Success of the CF Program in Diversifying Economy of Communities

Figure 13. Client Opinion on the Success of the Community Futures Program in Diversifying Economy of Communities

TEA does not generate the number of jobs created or maintained by sector. While this information was collected through the client survey, the small numbers of respondents and the distribution of those responses by region and by sector, do not allow for analysis on the extent to which projects being supported are in a wide variety of sectors.

Strengthened Community Capacity

IC/FedNor defines capacity building as a ‘broad and complex set of activities that will, if successful, enable people of diverse backgrounds in communities to take a more active role in shaping their economies and determining their collective futures. It enables communities to set their priorities, identify and develop their own capabilities and resources and make the best investments.Footnote 43 Capacity building involves increasing the intellectual (e.g., organizational infrastructure and capacity, planning capability) and human capacity (e.g., leadership, skills development, partnerships) within a community through strategic economic development planning and providing support for businesses and organizations to implement those plans.

The CFDCs are structured such that they are providing services to businesses and organizations that are supportive of capacity building as defined above. They are mandated to lead or participate in strategic planning, support community economic development projects (both technical and financial support) and offer financing and business services to small businesses.

Information from the evaluation shows that CFDCs are improving the business skills and knowledge of clients and community organizations, mobilizing partners to undertake community economic development projects, and providing funding and business services to clients. Though the level of involvement in these activities varies by CFDC, information provided by interviewees and survey respondents suggests that overall CFDCs are having an impact on community capacity. Twenty-six of the 30 interviewees who responded to the question, indicated that CFDCs have strengthened community capacity, primarily through the merging of resources/partnerships (10 of 30) and by increasing skill sets and business knowledge of clients (9 of 30). Note that few external stakeholders and FedNor senior managers responded to this question.

Only one CFDC survey respondent felt that the CFDCs had not impacted community capacity, while the remaining respondents indicated that the CFDCs had been somewhat or very successful in doing so. Similarly, clients believe that the CFDCs have improved community capacity (48.3 percent very successful, 40.9 percent somewhat successful), with a few indicating this was not the case, and with non-financial clients agreeing slightly less than the other two client groups (Figure 14).

Figure 14. Client Opinion on CFDC Success at Building Community Capacity

Figure 14. Client Opinion on Community Futures Development Corporation Success at Building Community Capacity

3.3.6 Long-Term Impact of CF Program

Finding: The CFDCs have made investments in SMEs and are active in community economic development activities. There is limited information to assess the contribution of the activities to long-term impacts of the Program, although stakeholders agree that the CF Program is contributing to economic growth and stability, diversification and sustainability of communities, and survival of businesses.

Findings in this section are based on Statistics Canada labour market data, the CFDC survey, and the client survey.

Information gathered from the evaluation shows that the objectives of the CFDCs are in-line with CF Program objectives and that they are undertaking activities in support of these objectives. During the six-year study period, CFDCs have invested $294.1 million through 6,963 loans to SMEs. CFDCs reported assisting 6,696 businesses and creating or maintaining a total 48,533 jobs. CFDCs have also made both financial and technical contributions to CED activities, with each CFDC being involved in anywhere between 5 and 30 projects each year and each region contributing between $40,000 and $8.0 million to CED projects each year.

The long-term impact of the CF Program relative to all of the other community influences (e.g., other investments made, economic conditions) is difficult to assess. An analysis of Statistics Canada labour market reveals that on key economic indicators (i.e., labour participation rates, unemployment rates, and average family/household income), there has not been an increasing disparity between CFDC served regions and the overall province of Ontario.

There has been virtually no change in the labour force participation rate of CFDC served regions, with only a 0.2 percent increase between 1996 and 2006 (from 63.6 percent to 63.7 percent).

This is fairly consistent with the provincial figures, as in that same time period there was an increase in the labour force participation rate of 0.8 percent (from 66.3 percent to 67.1 percent). There have been more significant changes in the rates of unemployment, as discussed in 3.1.1 (Need for the CF Program). Overall, unemployment has dropped Ontario-wide and throughout CFDC served regions (Figure 15). These drops were largest between 1996 and 2001, and smaller between 2001 and 2006. Overall regions served by CFDCs saw a smaller decrease in unemployment rates (30.0 percent) compared to the province of Ontario (33.3 percent), although four of the seven regions actually saw larger decreases in unemployment rates than Ontario (East, Southeast, South Central and Southwest).

Figure 15. Unemployment rates, CFDC Regions Compared to Ontario

Figure 15. Unemployment rates, Community Futures Development Corporation Regions Compared to Ontario

Median household earning have been increasing both in CFDC served regions and in the province of Ontario (Figure 16). The rate of increase for CFDC regions was much faster than Ontario between 2000 and 2005, averaging 10.0 percent growth versus 1.4 percent for Ontario. Thus, the CFDC regions are beginning to approach the provincial medians.

Figure 16. Median Household Earnings, CFDC Regions Compared to OntarioFootnote 44

Figure 16. Median Household Earnings, Community Futures Development Corporation Regions Compared to Ontario

Analysis of the Statistics Canada data with respect to the impact of the CFDC activity and the CF Program as a whole is challenging. Any changes to these economic indicators cannot be directly attributed to the CF Program. What the labour market data can reveal is whether in general rural communities are remaining economically viable, improving or declining. A sharp decline or increase in disparity between rural and provincial averages would suggest that the suite of programs being offered is not effective or having the desired impacts. The labour market data does not indicate that there are any noteworthy differences between rural and provincial averages. It is therefore implied that the suite of programs active in rural communities is having the desired impact.

Both clients and CFDCs agree that the CF Program is contributing to its long-term objectives, although again, not surprisingly, CFDCs are more confident in this regard and in fact no CFDCs disagreed. Clients were slightly less confident that the CF Program is contributing to its long-term objectives, although for all four objectives, more than 70 percent of clients either agreed or strongly agreed (Figure 17). A much greater proportion of clients responded neutral to this question than CFDCs. This is likely due to the fact that that certain groups were better able than others to judge the long-term impact of the CF Program. If the responses are examined across the three survey groups, non-financial clients more than any other survey group, were least able to provide an opinion on the long-term impact of the CF Program. CED partners were most able to provide their opinion in this respect (i.e., they had the lowest proportion of neutral responses). This is not surprising, given that this survey group is likely more involved in economic development activities within the community than the financial and non-financial clients and thus likely in a better position to judge the impact of the Program.

Figure 17. Client Opinion on the Long-Term Impacts of the CF Program

Figure 17. Client Opinion on the Long-Term Impacts of the Community Futures Program top of page

3.4 Cost-Effectiveness and Alternatives

3.4.1 Cost-Effectiveness of the CF Program

Finding: It was not possible to determine whether FedNor is cost-effective in administering the CF program, as program costs are not available. CFDCs are cost-effective in administering the CF program, although the level of effectiveness appears to vary between CFDCs.

Findings in this section are based on document review and the Quarterly Report data. Due to the limitations with respect to the Quarterly Report data, the level of confidence in the reliability of this data is low.

To assess the cost-effectiveness of a progra m, generally the costs for program operationsFootnote 45 are compared to program outcomes to calculate a cost per unit outcome. For the CF Program in Ontario, it was not possible to assess the extent to which FedNor is cost-effective in administering the program (i.e., costs for operation versus outcomes). This is due to the fact that FedNor does not separate the costs for the operation of the CF Program from the other programs it delivers (i.e., EODP and NODP). Significant attempts were made to extract the operating costs for the CF Program; however, it was not possible.

Recommendation #3: FedNor should undertake a costing exercise to identify the costs for the operation of NODP, EODP, and the CF Program. This would be particularly useful because different delivery models are used to deliver these programs and it would allow FedNor to assess the extent to which the different delivery models are effective in achieving program outcomes.

In the absence of FedNor operating costs for the CF Program, the evaluation examined the cost-effectiveness of the Program based upon the operating dollars provided to the CFDCs and reviewed the activities of the CFDCs in relation to their outputs and outcomes.

Variations Between CFDCs

Information from the Quarterly Reports shows that there is much variation between the CFDCs with respect to level of activity (e.g., loan volumes, general inquiries), investment fund performance, and cost per outcome. Many outliers and blank fields in the Quarterly Reports raise concerns as to the reliability of the Quarterly Report data and therefore the level of confidence is low for findings where Quarterly Report data has been used as a methodology.

Loan and Business Activity

The average number of loans provided to SMEs over the six-year study period was 19.0, with one CFDC averaging 2.3 loans per year and another averaging 62.3 loans per year. Similarly, there is a very large variation in the average dollar value of loans, with one CFDC averaging $18,722 and another averaging $97,882.Footnote 46 As discussed previously, there is a correlation between the number and dollar value of loans and therefore, CFDCs with few loans offer higher dollar valued loans. These CFDCs tend to be those serving smaller communities in the North, with the reason for the higher dollar valued loans likely being the presence of loan pools between the region's CFDCs.

As with loan activity, the number of business services provided varies greatly between CFDCs (Table 24). Some CFDCs have had very low volumes of general inquiries, with one CFDC receiving an average of only 6.5 inquiries per year. Some CFDCs, on the other hand, have had extremely large volumes of inquiries (e.g., 6,567). Similarly, some CFDCs have conducted a limited number of in-depth counselling interviews (i.e., 2.5), while some CFDC have conducted hundreds of counselling interviews.

Table 23. Comparison of CFDC Loan and Business Activities
Service Average Per Year, Per CFDC Minimum Maximum
Number of loans 19.0 2.3 62.3
Average dollar value of loan $ 42,240 $ 18,722 $ 97,882
Number of general inquiries 832 6.5 6,657
Number of in-depth counselling interviews 134.3 2.5 896

The evaluation yielded few explanations for these variations; the level of demand for each of these services would vary according to region and community and would be influenced by geographic, demographic, and economic conditions. For example, if a community is experiencing an economic downturn, it may be difficult to attract new clients that would want to start a new business. Conversely, if a community is experiencing a boom, there may be many options for credit, so "less bankable" clients will be those applying to the CFDCs. Overall there appears to be increased demand for CFDC services as economic conditions in a community increase up to the point where sources of credit and business information external to CFDCs start to become more readily available. At this juncture, the demand for the services starts to decrease, something which is more common in the larger communities. However, as more sources of credit become available, more leveraging options arise, allowing each loan dollar and business service offered to go further. Nonetheless in the smallest communities the jobs which are impacted represent a greater percentage of the employment in the region, so the demand relative to the community size is greater than in larger populations. As well, since these jobs represent a larger percentage of total jobs in the community, these CFDCs may be impacting their communities more profoundly. It is also important to note, that with the data reliability issues with respect to the Quarterly Report data, the wide ranges in these numbers may also be due to data issues. Hence, throughout this evaluation little confidence has been given to the Quarterly Report data.

Investment Fund Activity

Overall, the investment funds for the CFDCs have been increasing and have seen a growth of 12.3 percent since 2004 when they were first tracked, although there is much variation by region.Footnote 47 Sixteen of the 61 CFDCs have experienced negative growth with one CFDC having a negative growth of 48.6 percent. Note that four of the five CFDCs which first received funding in 2003-2004 have seen negative growth rates, which is not surprising given that those CFDCs have not yet had the opportunity to build their fund (i.e., they have not generated much income through loans yet).

There does not appear to be a correlation between investment fund growth and the percentage of the fund in active investment, as there are CFDCs with negative growth and a high percentage of funds in active investment and vice-versa. This is more likely explained by the loss rates on loans. There is limited information available on loss rates, as there is no Quarterly Report data available. However, the 2005-2006 FedNor investment fund survey showed that the CFDCs had an average loss of 9.9 percent, although there is much variation in losses between CFDCs, with some (3) reporting a zero percent loss rate, while others reported loss rates as high as 29 percent (Table 25). Note that CFDCs with smaller growth in investment funds tend to have higher loss rates.

Table 24. Comparison of CFDC Investment Funds
Service Average Per Year Minimum Maximum
Investment fund growth (with injections removed) 12.3 % -48.6 % 68.9 %
Percent in active investment 67.8 % 41.7 % 92.0 %
Cash on hand 32.2% 8.0% 58.3%
Loss rates (April 2005-March 2006) 9.9% 0.0% 29.0%
Cost Per Outcome

CFDCs reported a wide range of jobs created and businesses started, with some CFDCs reporting zero jobs created per year and very few businesses started per year (i.e., 2.5). The number of jobs created and businesses started is influenced by the level of CFDC activity, the type of businesses being supported (e.g., size), and the nature of the loans (e.g., for start-ups, maintenance, expansion).

Because the CFDCs receive relatively similar funds for operations, it is the numbers related to jobs created and businesses started that influence the cost per outcome for the CFDCs. For example, CFDCs with fewer jobs created or fewer businesses started have higher costs per job created or businesses started/maintained. Some CFDCs have very low costs in this respect and are well below the average, with one CFDC creating a job for only $279. Conversely, there are some CFDCs that are well above the average with one CFDC creating a job for $15,637 (Table 26).

Table 25. Comparison of CFDC Costs per Outcome
Service Average Per Year Minimum Maximum
Jobs created 126.1 0 963
Businesses started 22.1 2.5 163.5
Cost per business started/maintained $ 4,997 $ 720 $ 27,167
Cost per job created $ 1,954 $ 271 $ 13,514
Investment cost per job created $8,623 $ 4,179 $ 53,672

All of the comparison information presented in this section raises questions as to whether some CFDCs have been more effective in delivering the CF Program than others. While their effectiveness may be influenced by factors such as geographic location, demographic composition of the community, the economy of the community and the level of demand for services, it could also be related to the performance of the CFDCs. FedNor in concert with the CFDCs would be in the best position to determine whether certain CFDCs could be more effective in delivering the program given the many variables which influence effectiveness.

Recommendation #4: FedNor should work with CFDCs to establish target ranges for the level of activity for CFDCs (e.g., number of loans, investment fund growth). These ranges should be developed in consideration of factors that may influence the activities of CFDC (e.g., geographic location, demand for services). This would provide a guide for program offices and assist them in understanding whether CFDCs could be more effective in delivering the program or achieving program results.

3.4.2 Alternatives to the CF Program

Finding: The current service delivery model in place for the CF Program integrates a number of components that are important to successful community economic development. Several studies and reviews have strongly supported the CF Program delivery model.

Findings in this section are based on interviews and document review. Almost all of those interviewed (36 of 39) believe that the current delivery model used for the CF Program is the most appropriate one to achieve program objectives. Some interviewees (21 of 39) provided reasons why the model is the most effective; the main reason being that it allows for local decision-making (13 of 21). Interviewees also believe that the fact that the Program is run by local volunteers (5 of 21), and that it is arms length to government (3 of 21) make the model effective. There were no suggestions for alternative delivery models.

A few recent studies support the notion that the current delivery structure for the CF Program is most effective. A study conducted by the University of Guelph in 2005 stated that:

  • "the Canadian Community Futures Program.....reflects the conscious crafting of a centrally supported program for endogenous development. It effectively joins state-to-community with a strong yet flexible governance model and is a good example of place-based policy"Footnote 48.

The 2008 Senate report entitled Beyond Freefall: Halting Rural PovertyFootnote 49 found that:

  • "The challenges of accessing credit in rural Canada have long been recognized at the federal level. Arguably the most successful program to help address this concern has been the Community Futures program, a federally funded but community-based and community-led initiative…"
  • "The Community Futures program success is due in no small part to the fact that it is locally run and suited to local conditions…"
top of page

3.5 Other Issues

3.5.1 Lessons Learned

Finding: The evaluation revealed a few key lessons which were mainly related to the current delivery approach for the CF Program and the key success factors with respect to this approach.

Interviewees (FedNor and external stakeholders) were strongly supportive of the current delivery approach being used for the CF Program and indicated there are components of this approach that are key to effective community economic development. Many (13 of 27) felt that a grassroots approach, where decisions are made at a local level is the most critical success factor for community economic development. Interviewees also noted that volunteerism is an important component (6 of 27) and that partnerships and good cooperation by community organizations are essential (3 of 27).

3.5.2 Progress on Previous Recommendations

Finding: FedNor has made progress on implementing recommendations made in the previous evaluation of the Community Futures Program (2002-2003).

A total of 13 recommendations were made as a result of the evaluation of the CF Program, completed in 2002-2003. Because that evaluation was formative in nature, many of the recommendations were related to design and delivery issues including: better identification of target groups for CFDCs, better recognition for volunteer contribution, a more well-defined reporting structure for CFDCs (i.e., performance information), continued support for training and development opportunities for CFDCs, negotiating multi-year agreements, and the development of more useful indicators for community economic development.

FedNor senior manager interviewees are satisfied with the progress made on these previous recommendations, noting that data collection has been improved; that more efforts have been put into promotional efforts; a revised RMAF and performance measures were developed; that more efforts have been made to target certain groups, particularly youth; and that FedNor is doing more to recognize volunteers. This is consistent with information obtained by a FedNor senior manager who provided a summary of progress against the previous evaluation recommendations (see Appendix K for this summary). Observations from the evaluation indicate that there are outstanding issues related to recommendation #9, concerning the availability, quality and reliability of data.


Footnotes

  1. 14 back to footnote reference 14 Industry Canada. Key Small Business Statistics. July 2008.
  2. 15 back to footnote reference 15 IC defines an SME as having less than 500 employees, but for FedNor's purpose, SMEs tend to be those with less than 10 employees.
  3. 16 back to footnote reference 16 Industry Canada. Key Small Business Statistics. July 2008.
  4. 17 back to footnote reference 17 Government of Canada. Small Business Financing Profiles. Small and Medium Sized Enterprises in Ontario. September 2007, page 1.
  5. 18 back to footnote reference 18 Industry Canada. Small Business Quarterly. Volume 9, Number 3, November 2007. Page 1.
  6. 19 back to footnote reference 19 Government of Canada. Small Business Financing Profiles. Small and Medium Sized Enterprises in Ontario. September 2007, page 1.
  7. 20 back to footnote reference 20 Statistics Canada. Small Business and Special Surveys Division. Survey on Financing of Small and Medium Enterprises, 2004. March 2006.
  8. 21 back to footnote reference 21 Senate Canada. Beyond Freefall: Halting Rural Poverty, Final Report of the Standing Senate Committee on Agriculture and Forestry. June 2008.
  9. 22 back to footnote reference 22 Canadian Federation of Independent Businesses. Banking Matters: Survey of Small Business Owners on Banking Issues. November 2007, page 2.
  10. 23 back to footnote reference 23 Government of Canada. Small Business Financing Profiles. Small and Medium Sized Enterprises in Ontario. September 2007.
  11. 24 back to footnote reference 24 Industry Canada. Key Small Business Financing Statistics. August 2005, page 11.
  12. 25 back to footnote reference 25 Industry Canada. Financing Small and Medium-Sized Enterprises: Satisfaction, Access, Knowledge and Needs. February 2002.
  13. 26 back to footnote reference 26 Roseheart, Dr. Robert G. Northwestern Ontario: Preparing for Change. February 2008, page.11.
  14. 27 back to footnote reference 27 Ibid. p.53.
  15. 28 back to footnote reference 28 All statistics from Statistics Canada 1996, 2001 and 2006 Censuses
  16. 29 back to footnote reference 29 Unemployment rates are being used because Statistics Canada did not report employment rates in 1996.
  17. 30 back to footnote reference 30 Capitalization means adding funds to the investment fund (i.e., a top-up of funds).
  18. 31 back to footnote reference 31 Industry Canada 2007-2008 Estimates: Report on plans and priorities. February 2007. p.10.
  19. 32 back to footnote reference 32 Industry Canada 2007-2008 Estimates: Report on plans and priorities. February 2007. p. 15.
  20. 33 back to footnote reference 33 http://www.omafra.gov.on.ca/english/rural/indruralplan.htm
  21. 34 back to footnote reference 34 http://www.reddi.gov.on.ca/pdf/3609787_bre_resource_manual.pdf
  22. 35 back to footnote reference 35 Final Report of the Standing Committee on Agriculture and Forestry. Beyond Freefall: Halting Rural Poverty. June 2008. p. 301
  23. 36 back to footnote reference 36 Government of Canada. Speech from the Throne: Strong Leadership. A better Canada. October 16, 2007
  24. 37 back to footnote reference 37 CFDCs in the Northern and Eastern regions of Ontario may access EODP and NODP funds to undertake community economic development projects. No such funding is available for CFDCs in Southern Ontario.
  25. 38 back to footnote reference 38 The respondents include only those partners that indicated the CFDC in their region took a lead role in the community strategic plan developed for their community and were consulted in the development of that plan.
  26. 39 back to footnote reference 39 KPMG. Community Futures Development Corporations Investment Fund Analysis: Summary Of Findings. June 2003
  27. 40 back to footnote reference 40 Based on the average loss provisions over a six year period for six large financial institutions.
  28. 41 back to footnote reference 41 Currently, 37 out of 61 CFDCs in Ontario are operating under three-year agreements.
  29. 42 back to footnote reference 42 28 of 61 CFDCs reported CED information in year 2.
  30. 43 back to footnote reference 43 For the purposes of this evaluation, FedNor developed this definition of community capacity from existing material.
  31. 44 back to footnote reference 44 For comparison purposes, all figures have been converted to 2005 dollars. For the 1996 Census, Statistics Canada did not provide median household earnings.
  32. 45 back to footnote reference 45 Program costs are defined as the cost to FedNor to administer the program, not including funds provided to the CFDCs (i.e. FedNor salaries and O&M)
  33. 46 back to footnote reference 46 Average loan values for three CFDCs were removed from the range, as they exceeded the maximum allowable loan contribution (i.e., >$150,000).
  34. 47 back to footnote reference 47 This is without capital injections from NODP or the CF Program.
  35. 48 back to footnote reference 48 University of Guelph. Community Futures Program in Canada, Good Governance in Successful Rural Development Programming, 2005.
  36. 49 back to footnote reference 49 Final Report of the Standing Committee on Agriculture and Forestry. Beyond Freefall: Halting Rural Poverty, June 2008, page 297.

4.0 Overall Conclusions

This section provides the overall conclusions for the evaluation of the CF Program.

Relevance

The evaluation found that there is a continuing need for the CF program, primarily because there are limited sources of funding available for SMEs, and CFDCs are supporting businesses that may not have otherwise been able to start, expand or survive. This need for funding is more pronounced in smaller communities, which are predominantly located in Northern Ontario, many of which have higher rates of unemployment and lower earnings than Ontario as a whole. In recent years these communities have been affected by swings in commodity prices and the CFDCs are helping to maintain the communities' economic infrastructures and bring economic stability to the communities.

The objectives of the CFDCs are in-line with the objectives of the CF Program as they are working towards fostering economic growth and stability, fostering creation and maintenance of jobs, helping to create diversified and competitive local economies, and helping to build sustainable communities. Similarly, the objectives of the CF Program are aligned to Industry Canada's strategic objective of a competitive industry and sustainable communities. The Federal Government has a legitimate, continuing role to play in delivering the CF Program, as there has been little investment in rural communities from the province and often the CFDCs offer the only Federal presence in rural communities.

In addition to providing funding for SMEs, the CF Program offers a range of business services (e.g., counselling, business information, referrals) and provides assistance to communities with planning and implementing their economic development priorities. There are other programs and/or organizations that provide some services that are offered by the CF Program, however no other programs provide the whole range of services that the CF Program offers. CFDCs have been successful in working with other programs and organizations to ensure that their services compliment, rather than duplicate one another. The evaluation raised a few issues with respect to potential duplication of administrative efforts with NODP and duplication in the reporting of outcomes with CFDCs reporting activities undertaken with EODP and NODP funds in CF reports.

Design and Delivery

The CF Program has been in operation for over 20 years and changes have been made to its design and delivery as issues have been identified. As a result, the Program appears to function well and the evaluation identified few issues impacting on the success of the program. Funding was the biggest issue cited by stakeholders who felt there was a particular need for additional funding for operations, CED activities, and investment funds.

Currently, the biggest challenge for the CF Program is in the collection and management of its performance information. While a structure is in place for collecting data related to CFDC performance and outcomes, the data is of limited use for the management of the program and for providing appropriate information to measure the success of the Program. The current system in use for the tracking of performance information (i.e., TEA) is not used by all CFDCs and not all CFDCs are meeting the reporting requirements (i.e., submitting annual reports).

TEA is also of limited use because it does not capture all performance indicators required for measurement purposes as per the RMAF. In addition, information to measure the longer-term outcomes of the Program is currently lacking, particularly with respect to CED activities.

The evaluation found that the data entered into the Quarterly Reports is unreliable. The outcome information currently reported by CFDCs (e.g., jobs created) is based on estimates at the time of the application and no verification is done to ensure that those estimates were actually achieved. There also seems to be a lack of verification of data inputted by CFDCs, as there were many variables in the Quarterly Report data that seemed incorrect.

Overall, the investment funds of the CFDCs appear well-managed with the overall portfolio experiencing positive growth in the past four years. The growth of the investment fund is influenced by the level of loan activity of CFDCs, including the proportion of those funds that are in active investment, as well as the loan loss rates. There is large variation across CFDCs with respect to the level of funds in active investment and a large range in loan loss rates. Without targets for desired performance for loan portfolios, it is difficult to assess whether there should be concerns regarding the performance of investment funds for certain CFDCs. This issue can be addressed as per recommendation #1.

Program Success

The CF Program has been successful in achieving its outcomes. Due to concerns with respect to the reliability of Quarterly Report data, especially with regards to reporting of business services provided, findings on program success are primarily based on soft data. CFDCs have provided a consistent level of business information, counselling and referral services to clients over a four-year period. The services provided by CFDCs have been deemed needed by clients and there is a very high level of satisfaction with the services provided by CFDCs. The CF Program is also meeting the needs of OLMCs by providing advice, support and services in both official languages where required. There are, however, on-going challenges related to maintaining skilled, bilingual staff and costs associated with providing bilingual information materials.

Through its loan portfolio, business services, and CED activities, the CF Program has been successful in improving clients' business skills and knowledge, creating new business start-ups, maintaining and strengthening existing businesses, increasing community capacity, and assisting communities to diversify their economies. The impacts of some of these activities, particularly CED work and the longer-term impacts of the program, are more difficult to measure as indicators to measure these impacts have yet to be established. This issue can be addressed as per recommendation #2.

Cost-Effectiveness and Alternatives

The current delivery model in place for the CF Program integrates a number of components that are important to successful community economic development including a grassroots approach, where decisions are made at a local level; the involvement of local community volunteers; and a focus on partnership development and cooperation between community organizations. No other delivery alternatives were identified.

It was not possible to determine whether FedNor is cost-effective in administering the CF program compared to EODP and NODP, as program costs are not tracked separately for these programs. This issue can be addressed as per recommendation #3.

Overall the CFDCs are cost-effective in administering the program. For the amount of funds received, CFDCs are providing a fairly high level of investment into the community as well as providing a number of other services. The extent to which all CFDCs are equally effective in delivering the program is a question raised by the evaluation, as there are great ranges in the level of activity and outcomes of the program across CFDCs. These ranges may be influenced by factors such as geographic location, demographic composition of the community, the economy of the community and the level of demand for services. It seems that as the economic conditions of an area improve the demand for CFDC services increases up to the point where the private sector begin to compete for "bankable" clients. In these generally more populated areas, the CFDCs are serving more start-ups who do not have long credit histories. As there is more credit available in these regions, the loan clients are able to leverage the CFDC loan dollars to a greater extent and thus impact more jobs and businesses per investment fund dollar. However, each of these jobs in the larger communities represents a smaller percentage of the jobs in the regions. So while smaller communities may not demonstrate the same volume of job impact, percentage wise, they are impacting their communities more profoundly. There may also be differences in cost-effectiveness related to the performance of the CFDCs. FedNor in concert with CFDCs would be in the best position to determine whether certain CFDCs could be more effective in delivering the program, as per recommendation #4.


IC/FedNor Management Responses to the 2008 Community Futures Program Final Evaluation

Recommendation #1: FedNor should work with CFDCs to establish general target ranges related to desirable levels of funds in active investment and loan loss rates based upon local realities. This would allow FedNor to assess a CFDCs loan portfolio performance and assist with improvements where necessary.

Agreed: FedNor will continue to review and assess current levels of funds in active investment and loan loss rates within CFDCs and develop a policy to establish target ranges, in cooperation with individual CFDCs, that reflect local realities. FedNor will continue to assess individual CFDC loan portfolio performance based on data available through our various reporting vehicles and work with CFDCs to improve performance, where necessary.

This review, assessment and establishment of target ranges will occur as CFDC agreements renew.

Recommendation #2: FedNor should complete a review of the performance data collected for the CF Program to ensure reliable and meaningful reporting and establish additional indicators to provide information for assessing the longer-term impacts of the program, particularly for CED activities. In addition:

  • FedNor needs to ensure that TEA is capable of capturing and reporting on all required performance information. It is possible that FedNor could reduce reporting requirements for CFDCs by maximizing its use of TEA and thereby improving its ability to manage the Program as well as make it more useful for CFDCs.
  • FedNor should ensure that all CFDCs are fulfilling all reporting requirements and should establish consequences for those CFDCs that do not comply with the requirements.
  • FedNor should ensure that CFDCs are not reporting activities undertaken and outcomes achieved through the use of EODP and NODP funds in CF Program reports (i.e., in the annual reports).
  • To ensure data reliability, FedNor should implement a system whereby TEA data would be reviewed on a regular basis and FedNor needs to work with the CFDCs to ensure data is corrected should issues be identified.

Agreed: FedNor will conduct a review of the performance measurement and reporting systems currently in use, including TEA, and take necessary actions to ensure that high quality and reliable data specific to the CF Program activities is being received. FedNor will make TEA mandatory for all CFDCs to ensure data integrity. By compiling data in a consistent format, FedNor can produce reports that will help in the management and evaluation of the program. The review will evaluate data verification processes, current practices with respect to compliance and establish consequences for delinquent and/or inadequate reporting.

The current reporting systems will be assessed and revised as necessary to ensure that performance measurement data is available for the assessment of longer-term impacts of the program.

This review and implementation of remediation measures will be completed by December 31, 2009.

Recommendation #3: FedNor should undertake a costing exercise to identify the costs for the operation of NODP, EODP, and the CF Program. This would be particularly useful because different delivery models are used to deliver these programs and it would allow FedNor to assess the extent to which the different delivery models are effective in achieving program outcomes.

Agreed.

FedNor will undertake a manual costing exercise to identify the costs for the operation of NODP, EODP, and the CF Program. FedNor utilizes, wherever possible, the same management, staff, administrative infrastructure and resources for the delivery of all three programs so that the overall delivery costs are minimized. By applying our resources to adjust to an ever-changing work environment, we have created an economy of scale and, therefore, efficiency in delivery that would not otherwise be achievable.

Given that the operations of the three programs are integrated to such a large degree and that FedNor systems, resources and processes are not in place for direct expenditure costing of current O&M expenditures, assumptions and estimates will be required in order to complete this exercise. This costing will be completed by September 30, 2009.

Recommendation #4: FedNor should work with CFDCs to establish target ranges for the level of activity for CFDCs (e.g., number of loans, investment fund growth). These ranges should be developed in consideration of factors that may influence the activities of CFDCs (e.g., geographic location, demand for services). This would provide a guide for program officers and assist them in understanding whether CFDCs could be more effective in delivering the program or achieving program results.

Agreed: FedNor will conduct an assessment of current levels of activity for a number of specific performance measures related to each of the four program lines of business and implement a process, in cooperation with individual CFDCs, to review their current activity levels and establish target ranges that take into consideration the many variables that can influence activity levels. This assessment, review and establishment of targets will help guide Officers in evaluating the effectiveness of CFDCs in the achievement of results.

This process will be completed by December 31, 2009.

Date modified: