Archived — Final Evaluation of the Student Connections Program

Final Report

November 2007

Prepared for:
Industry Canada

Prepared by:
Government Consulting Services

Tabled and approved at DEC on

This publication is available upon request in accessible formats.

Contact: 
Multimedia Services Section 
Communications and Marketing Branch

Industry Canada  
Room 264D, West Tower  
235 Queen Street  
Ottawa ONK1A 0H5

Tel.: 613-948-1554  
Fax: 613-947-7155
Email: ic.cmb-creative.ic@canada.ca


Permission to Reproduce

Except as otherwise specifically noted, the information in this publication may be reproduced, in part or in whole and by any means, without charge or further permission from Industry Canada, provided that due diligence is exercised in ensuring the accuracy of the information reproduced; that Industry Canada is identified as the source institution; and that the reproduction is not represented as an official version of the information reproduced, nor as having been made in affiliation with, or with the endorsement of, Industry Canada.

For permission to reproduce the information in this publication for commercial redistribution, please email: copyright.droitdauteur@pwgsc.gc.ca

Cat. No. Iu4-126/2008E-PDF
ISBN 978-0-662-47827-0
60431

Aussi offert en français sous le titre Évaluation finale du programme des Étudiants bien branchés


Table of Contents

Annexes (Separate document)

(Note: Annexes are available via an Access to Information)

  • Annex A: Steering Committee Members
  • Annex B:: SC Program Logic Model
  • Annex: C:: Colleges'/Universities'SC Start and End Dates and Amount of ICT Training
  • Annex: D:: Evaluation Matrix
  • Annex: E:: Document Review–List
  • Annex: F:: List of Interviewees
  • Annex: G:: Interview Guides
  • Annex: H:: Survey Distribution and Response Rates, by Survey Period
  • Annex: I:: Additional Data Tables

Note:

Minor editorial changes were made to this report in order to prepare the document for posting to the Industry Canada's Website (including removal of standard appendices such as list of interviewees and questionnaires). Readers wishing to receive a copy of the original version of this report should contact the Audit and Evaluation Branch at Industry Canada.

Tables

Executive Summary

The Student Connections (SC) program began in 1996 as part of the federal Youth Employment Initiative and formally became part of the Youth Employment Strategy (YES) in 1997. The objectives of the SC program are to:

  • Stimulate the adoption of Internet and E-business practices by Canadian small and medium-sized enterprises (SMEs);
  • Provide practical technology and business experience and skills for Canadian college and university students in IT-related fields;
  • Provide post-secondary students in IT-related fields with assistance in financing their education; and
  • Support Canadian seniors to take advantage of the benefits of the Internet.

A formative evaluation of SC was conducted in 2005–06. The Program's Results-Based Management and Accountability Framework (January 2004) calls for a final evaluation of the program in 2007–08. The terms and conditions of the program are scheduled to expire March 31, 2008 (although a one-year extension may be granted), and the findings of this evaluation are to provide information for a decision on the future of the program. This evaluation covers the period of April 2004 to March 2007.

During the first phase of the project, Government Consulting Services (GCS) developed an evaluation matrix and methodology report for the SC Program evaluation, which was approved by the SC Evaluation Steering Committee in June 2007. Using the finalized evaluation plan, GCS undertook four primary research activities: document review, interviews, a review of Program survey data, and a review of administrative data. The evaluation included an examination of relevance, success, cost-effectiveness/alternatives, and other issues.

As is the case in any evaluation, there were some limitations with respect to the methodologies employed. The primary issue had to do with the fact that most of those who provided input to the SC evaluation were those who operate, or participate in, the Program; and were therefore most likely to feel positively about the Program. Another potential issue with respect to the methodology related to the sample sizes. Although the overall response rates (for the survey) were good, sample sizes for SMEs and Seniors were relatively small, for the individual (6-month) survey periods included in the evaluation, when compared to the total numbers of SMEs and Seniors who participated in training during that period (i.e., distinct participants). The evaluation report, therefore, focuses on aggregate survey responses only.

Relevance

There continues to be a need, by SMEs, for practical Internet and E-Commerce training. SMEs agree that Information Communication Technology (ICT) is a key enabler for a successful business. Although a growing interest in Internet and E-commerce is noted, SMEs continue to lag behind larger businesses in their use of the Internet. As well as addressing this continuing need, the SC Program addresses IC, Human Resources and Social Development Canada (HRSDC), and more specifically YES priorities. IC's mandate is to help make Canadians more productive and competitive in the knowledge-based economy, thus improving the standard of living and quality of life in Canada. IC has committed to supporting participation in the digital economy to support its strategic objective of 'competitive industry and sustainable communities'. Additionally, SMEs are clearly an important driver to Canada's economy, as is access to and use of enabling technologies. With regards to HRSDC priorities, the SC program supports the department in its mission to help Canadians (including students and seniors) move through life's transitions. SC provides students with valuable and meaningful work experiences and abilities that will help them successfully transition into the workforce. This outcome is directly aligned with HRSDC's Youth Employment Strategy.

Success

The SC Program has been successful in meeting its performance targets of number of clients trained, number of youth hired, and ratio of SMEs to seniors trained. However, targets for the ratio of training on national products, ratio of training on E-commerce products, and ratio of sales revenue to the IC allocation of funding have not been met. A number of explanations for why these targets have not been met were put forward by Program staff. Confusion in classification of training sessions (as E-commerce or national products), following the implementation of a new content management system, may have led to some Centres incorrectly categorizing training sessions which should have been included as E-commerce or national products training. As well, as SMEs become more advanced in their skills and knowledge, the E-Commerce training offered to them takes a different form. The SC training delivered in the past often took the form of classroom training, whereas e-commerce training is now more often one-on-one training. As a result fewer clients are trained with the same amount of effort. Finally, IC Program staff indicated that while they have not investigated the reasons for Centres not reaching sales revenue targets, they believe that this may be due to the fact that IC, and some Centres, place less emphasis on revenue targets. IC focuses more on its other national targets and the Centres may focus instead on work experience provided to SBAs.

Recommendation

In response to the above findings, the following recommendation is made.

With regards to SC targets, IC should re-examine its targets relating to:

  1. ratio of training done on national products,
  2. ratio of training on e-commerce products, and
  3. ratio of sales revenues to the IC allocation of funding, to ascertain whether these targets are still useful and valid

If IC retains the national target relating to the ratio of training done on national products, guidance should be provided to the Centres as to how to determine if training is considered national product training. If IC retains the e-commerce target, IC may wish to adjust the target to examine what percentage of Student Business Advisor (SBA) training hours is spent on E-commerce training.

The SC Program is providing its expected benefits to its target groups—SBAs, SMEs, and Seniors. Benefits to SMEs relate to the flexible, affordable characteristics of the training, as well as to skills and knowledge gained through participation in SC training. The main benefits to SBAs are the experience they gain, which ultimately makes them more employable, and the knowledge and skills they develop through their participation in the Program. As well, the wages earned by SBAs help them finance their continuing education. Benefits to Seniors relate to improved ICT skills and, although not an intended benefit of the Program, additional benefits accrued through social interaction and a sense of connection that Seniors can gain as a result of their participation in the Program.

The final evaluation question related to success was whether or not the SC Program has addressed the requirements of the Official Languages Act. The SC Program has made numerous efforts to address the requirements of the Official Languages Act, through requirements articulated in its sub-contracts with community colleges and by providing opportunities for members of official-language minority communities to become informed of the initiative.

Cost-effectiveness / Alternatives

Funding for the SC Centres comes from IC in the form of funding for salaries, as well as through cash and in-kind contributions from the Centres and revenues raised through fees for training. IC funding accounted for approximately 56% of the total SC funding over the evaluation period. The cost of the Program was $8,585.45 per intern hired and $218.78 per client trained based on the total IC budget of $3.5 million. Based on only what IC paid to the Centres, the cost per intern hired was $7,060.46, and the cost per client trained was $179.92. While it appears that the SC Program is cost-effective, particularly given the benefits outlined in the success section of the report, it is not possible to say so definitively for several reasons. Firstly, IC has not identified cost targets. Also difficulties in identifying comparable programs mean that costing information is not available for comparative purposes. As well, IC examines the costs of the program at the national level only. An investigation into costs at the Centre level, for example to understand the differences in costs per student hired and costs per client trained, by Centre, may provide insight and lessons-learned that could be used to improve the delivery and cost-effectiveness of the Program.

Recommendation

IC should investigate other programs with an aim to identifying comparable programs (if they exist) that can be used as comparators in assessing the cost-effectiveness of the SC Program. In addition, IC should establish targets relating to costs, both at the national and Centre level, and monitor costs against these targets.

Numerous possible improvements to the SC program were identified by interviewees. However, a large majority of respondents agreed that there is no other more cost-effective or efficient approaches to achieving program objectives. None of the alternative approaches presented by interviewees were viewed by these interviewees as more cost-effective and efficient. Also, no existing programs identified have the same range of program objectives and scope. Suggestions for improvements tended to relate to increasing funding for coordinator salaries and marketing, and increasing the allowable student training hours.

Based on a review of documentation and the interviews conducted, in addition to the continued relevance of the Program, the SC Program is well-placed within the Government of Canada. IC has an objective to promote economic growth, and this Program supports that objective by providing essential ICT skills to strengthen SMEs. As well, according to interviewees, the Program cannot be self-sustaining. Finally, interviewees suggested the private sector would not be interested in delivering the Program, and the voluntary sector lacks the capacity to be able to deliver the Program.

Other Issues

Interviewees identified several unintended effects of the SC Program. These were: increased awareness on the part of SBAs of entrepreneurial issues and the unique challenges faced by SMEs; networking amongst colleges and universities; institutions leveraging funding for other programs as a result of SC success; and the use of the SC Program to attract students to participating academic institutions.

The final evaluation question looked at the progress made on issues identified in the past evaluation. It is clear that considerable efforts have been made to address the issues that were identified in the past evaluation. Of the four issues identified in the 2005–06 formative evaluation, some form of action has been taken on all four.

1.0 Introduction

This report presents the final evaluation of the Student Connections (SC) Program. The evaluation of the SC Program was conducted by Government Consulting Services (GCS), Public Works and Government Services Canada, in consultation with the Steering Committee established for the purposes of the evaluation, and managed by the Audit and Evaluation Branch of Industry Canada. (See Annex A for a list of Steering Committee Members.) This report is organized in four chapters: Chapter 1 provides the general background and objectives for the study; Chapter 2 presents the methodology followed in conducting the evaluation; Chapter 3 presents our findings, organized in relation to the Evaluation Framework; and Chapter 4 presents our overall conclusion.

1.1 Background

The SC program began in 1996 as part of the federal Youth Employment Initiative and formally became part of the Youth Employment Strategy (YES) in 1997. The objectives of the SC program are to:

  • Stimulate the adoption of Internet and E-business practices by Canadian small and medium-sized enterprises (SMEs), thereby making them more competitive in the knowledge-based economy;
  • Provide practical technology and business experience and skills for Canadian college and university students in IT-related fields, thereby making them more productive in the knowledge economy in support of the government's Youth Employment Strategy;
  • Provide post-secondary students in IT-related fields with assistance in financing their education through income earned during the work experience, as per the Youth Employment Strategy; and
  • Support Canadian seniors to take advantage of the benefits of the Internet, including on-line transaction services offered by governments, institutions and the private sector.

See Annex B for a copy of the Student Connections Program logic model, outlining the activities, outputs and outcomes of the Program.

Industry Canada uses a third-party approach to deliver its SC program and currently has one delivery organization, the Association of Canadian Community Colleges (ACCC). The program is administered by Industry Canada under the department's Small Business and Marketplace Services Sector. For the SC Program, IC manages research, Website management, national standards, national marketing, partnership development and ensures consistent program integrity. In addition, SC Program staff participate in IC directed activities such as reviewing policy to ensure SC adherence, and participating in audits and evaluations of the SC Program.

The Association of Canadian Community Colleges (ACCC) administers the program on behalf of Industry Canada. IC has a contribution agreement in place with ACCC, which sets out requirements of the program, including monitoring and reporting. The ACCC has sub-contracts (won through competitive processes) in place with select Canadian universities and colleges. Administrative Centres in these institutions are responsible for hiring, training and supervising students; marketing the program regionally; performing accounting functions, reconciling statistics and inputting financial and human resources data into a national database, and organizing training sessions with clients. In addition to students hired to deliver the training sessions, each centre also has a regional coordinator, who is funded, in part, through SC Program funding and client sales revenues.

The SC budget for each year of the evaluation period was $3.5 million ($2.9M program budget and $600,000 operating budget). There are typically 14 universities and/or community colleges, regionally distributed across Canada, with established SC Centres that employed approximately 300 youth annually.1 Over the course of the evaluation period, participating colleges and universities included:

  • Algonquin College
  • CCNB
  • Cégep de Jonquière
  • Cégep de Saint Hyacinthe
  • Collège Boréal
  • HEC Montréal
  • Holland College
  • Humber
  • John Abbott College
  • La Cité collégiale
  • Langara College
  • Memorial University
  • Okanagan College
  • Red River College
  • Southern Alberta Institute of Technology
  • University of Alberta
  • University of Calgary
  • University of Regina
  • Wilfrid Laurier University

See Annex C for a list of the participating colleges and universities, and their start and end dates (where applicable) with the SC Program. The table also illustrates the amount of client SC Information Communication Technology (ICT) training, by Centre, over the three years covered by this evaluation.

SC provides e-business and Internet training services under four general categories: E-business Basic Training, Customized Training and Presentations for Business, Computer and Internet Basics for Business, and Computer and Internet Basics for Seniors.

E-business Basic Training and Consultation services include: Internet Security, Distributing Electronic Newsletters, Online Business Research, Web Site Planning, Web Site Evaluation, On-Line Promotion, Selling on the Internet, and E-Business Blueprint (a review of technology and business issues, taking stock of current or potential e-commerce capabilities and making recommendations to guide businesses into the electronic marketplace).

Customized Training and Presentations for Business includes services that are tailor-made to meet the specific needs of a business. These services introduce entrepreneurs to a range of tools that can help improve their business. In addition SBAs will speak at, or conduct workshops, for groups relating to e-business practices.

Computer and Internet Basics for Business includes training targeted at business owners and employees with subject areas such as Computer Basics, Introduction to the Internet, Electronic Mail, and Internet Applications.

Computer and Internet Basics for Seniors includes training categories such as Computer Basics, Introduction to the Internet, Communicating Online, and Electronic Commerce and You.

A formative evaluation of SC was conducted in 2005–06 and the evaluation findings indicated that the program drew very positive responses from both its participants and its target beneficiaries, as well as meeting its national targets and benchmarks. The Program's Results-Based Management and Accountability Framework (January 2004) calls for a final evaluation of the program in 2007–08. The terms and conditions of the program are scheduled to expire March 31, 2008 (although a one-year extension may be granted), and the findings of this evaluation are to provide information for a decision on the future of the program.

1.2 Objectives and Scope

The objectives of this project were twofold:

  • to develop an evaluation plan for the final evaluation of the SC Program with oversight from Audit and Evaluation Branch officials; and
  • to carry out the evaluation using the methodologies established in the evaluation plan.

During the first phase of the project, GCS developed an evaluation matrix and methodology report for the SC, which was approved by the SC Evaluation Steering Committee in June 2007. The evaluation matrix is provided in Annex D. This evaluation matrix contained the detail required to undertake the evaluation, including evaluation questions, indicators, data sources, and detailed methodologies.

The evaluation covers the period of April 2004 to March 2007, which included a total budget of $8.7 million ($2.9 million per year).


1 In 2006–07 there were 15 universities and community colleges participating in the program. (Return to Text)

2.0 Methodology

2.1 Approach

Using the finalized evaluation plan, GCS undertook four primary research activities, as outlined below:

  1. Document Review: A variety of documents were reviewed (a detailed list is provided in Annex E), including founding documents, the Results-Based Management and Accountability Framework (RMAF), ACCC Annual Reports, financial reports, SC websites, and documents related to the broader government agenda with respect to SMEs.
  2. Interviews: Interviews were conducted with a wide number and variety of stakeholders, including IC Program staff, SC Coordinators, SC Senior Managers, ACCC representatives and business organizations. In total, 34 people were interviewed; a list of all interviewees is provided in Annex F. Interviewees in the National Capital Region (NCR) were interviewed in-person; those outside the NCR were interviewed by telephone. In all cases, interview guides were provided to interviewees in advance of the interviews. These guides can be found in Annex G.

    The notes from all interviews were summarized, reviewed by the interviewers and compiled in an Excel spreadsheet. This process allows for a systematic analysis of the responses to each question, and is undertaken to strengthen the accuracy and objectivity of the overall analysis.
  3. Review of Survey Data: ACCC conducts, through Prairie Research Associates (PRA) Inc., regular surveys of Student Business Advisors (SBAs) and individuals who participated in SC training, including business clients (SMEs) and seniors. ACCC retained the services of PRA, a private sector consulting firm, to administer the surveys over the period covered by the evaluation. Survey data was available, for the evaluation, for the years 2005–06 and 2006–07, and was incorporated into the evaluation findings. Survey data was not available for the year 2004–05 since, in response to the Privacy Act, IC and ACCC agreed to seek active consent from participants to participate in the survey. This had the effect of drastically reducing the size of the survey sample pool, to a point where survey data was not useful for the purposes of evaluation.

    Survey questionnaires were mailed to SMEs and seniors. In order to boost response rates, PRA attempted to contact SMEs and seniors, by telephone, who had not responded to the survey to remind them to return the questionnaire. PRA performed two sets of reminder calls. The survey instruments used to survey seniors and SMEs was unchanged over the period of the evaluation.

    With regards to the survey of SBAs, while the survey instrument remained unchanged over the period of the evaluation, the method in which SBAs were contacted was revised. Prior to October 2006, SBAs received an email invitation to participate in the survey. PRA then used telephone and email follow-ups to encourage SBAs to respond, in order to increase the response rate. Using this approach SBAs were often surveyed months after they participated in the Program; and, according to PRA, SBAs were often difficult to contact. However, beginning with the survey of SBAs for the period of October 2006 to March 2007, SBAs were invited to immediately complete the survey, after they completed their work as an intern, by visiting a website that housed the survey instrument. This approach was designed to increase the survey response rate and the quality of the information supplied, as work experiences were fresher in the SBAs' minds. Following this change to the process, the survey response rate increased by approximately 10 percent.
  4. Review of Administrative Data: A number of ACCC reporting documents were reviewed. These documents provided detailed information about program operations (e.g., number of clients, number of interns, financial information, training session offerings, etc.).

2.2 Limitations

As is the case in any evaluation, there are some limitations with respect to the methodologies employed. In this case, the primary issue has to do with the fact that most of those who provided input to the SC evaluation are those who operate, or participate in, the Program. As such, they are well-placed to provide feedback on the operations and outputs of the SC Program however, they may also be most likely to feel positively about the performance and the continued need for the Program. Ideally, our research would have extended further, to capture the views of others, such as SMEs not using the Centres and students not involved with SC.

Another potential issue with respect to the methodology relates to the sample sizes. Although the overall response rates (for the surveys) were good (see table 1 below), sample sizes for SMEs and Seniors were relatively small for the individual (6-month) survey periods when compared to the total numbers of SMEs and Seniors who participated in training during that period (i.e., distinct participants). (For response rates and margins of error by individual (6-month) survey period, see Annex H.)

The following table illustrates the distribution of surveys and the response rates for the surveys of the three populations (SMEs, SBAs and Seniors).

Table 1: Distribution of Surveys and Response Rates
Survey
Population:
Distinct
participants2
Total
Possible
Survey
Recipients3
# who
requested
no
contact
# of
names
provided
to PRA
# of
surveys
mailed
# of
surveys
returned
response
rate
Margin
of
Error4
SMEs 11,864 4,816 1,723 3,088 1,852 369 19.9% + 5.0%
SBAs 699 699 N/A 699 699 248 35.5% + 5.0%
Seniors 2,317 1,157 156 1,028 834 317 38.0% + 5.1%

With respect to the surveys, the number of respondents for all years (369 for SMEs, 248 for SBAs and 317 for Seniors) generated response rate of 19.9%, 35.5% and 38.0% respectively. When the survey samples are assessed in relation to a 95% confidence level, they provide margins of error of approximately +5%. Therefore, the survey results for the SC Program overall are deemed to be reliable.

With regards to the reliability of the individual (6-month) survey periods, the margins of error are higher than desired (see Annex H). Other indications however suggest, that the reliability may be higher than the margin of error would suggest. According to the Technical Services Manager at PRA, results from the surveys have been consistent over time, therefore diminishing the usual risks inherent to smaller sample sizes. If results varied greatly between surveys then the margin of error would be of concern. Given that this has not been the case, it suggests that the survey results are more reliable than the confidence level suggests. However, given the fact that there is some uncertainty regarding the reliability of the individual survey periods data, this report focuses on aggregate survey responses only.


2 As per IC Intranet. (Return to Text)

3 Only those SMEs who have taken E-Commerce and ITB training modules (not including information session) are surveyed by ACCC. Only Seniors who have taken training on any of the SC Seniors training modules are surveyed by ACCC. (Return to Text)

4 Margin of error with a 95% confidence interval. (Return to Text)

3.0 Findings

This chapter of the report presents our findings, organized according to the four general themes covered in the evaluation: program relevance, program success, cost effectiveness/ alternatives, and other issues such as unintended effects and progress following past evaluations.

3.1 Relevance

One of the major questions addressed in an evaluation is the extent to which the program or initiative under study continues to "make sense"; that is, are the objectives and mandate still relevant to government responsibilities and priorities. The SC Evaluation Framework included two questions related to this issue:

Q.1 Is there a continuing need, by SME'S, for practical Internet and E-commerce training?

Conclusion

There continues to be a need, by SMEs, for practical Internet and E-Commerce training. Based on surveys conducted over the past year, SMEs agree that ICT is a key enabler for a successful business. However, although a growing interest in Internet and E-commerce is noted, SMEs continue to lag behind larger businesses in their use of the Internet — particularly when it comes to selling and purchasing online. The continuing need for the Program is also supported by all interviewees.

Findings

Findings in this section are based primarily on a review of literature and documentation, and stakeholder opinions gathered through interviews.

A review of relevant documentation (e.g. web and newspaper articles, websites, etc.) was conducted in order to determine the extent to which ICT is important to SMEs and the extent to which SMEs have adopted E-business practices.

In October 2006, a writer for the Globe and Mail writes: "A recent survey of more then 1,000 businesses with fewer than 16 employees, conducted by Ipsos-Reid, reveals information technology is making a big difference — at least that's the perception of most respondents. These small businesses say technology helps make them more efficient and more competitive. In fact, 88 per cent of those surveyed characterize IT as being like "a silent partner" that helps them deliver better customer service and close more sales.Footnote 5

In 2006, Microsoft Canada mandated Ipsos-Reid to conduct a survey of SMEs. Highlights included:

  • 81% of Canadian SMEs with less than 16 employees say IT allows them to compete with bigger corporations.
  • 75% believe it increases their competitiveness by providing access to timely and relevant business info.
  • 71% use IT to manage and win new customers.

However, it seems that SMEs still lag behind in their use of the Internet — particularly for selling online. A survey conducted by Statistics Canada in 2006 provides the following relevant data regarding Internet access and use by firm size:

Table 2: Internet Access and Use by Firm Size (Percent), 2001–2005 Footnote 6
Source: Statistics Canada, Survey of Electronic Commerce and Technology ( SECT), 2006
  2001 2002 2003 2004 2005
Internet Access Small 68 73 76 79 79
Medium 91 92 94 96 96
Large 94 99 97 99 98
All Firms 71 76 78 82 82
Own Website Small 24 27 29 32 33
Medium 57 62 66 69 70
Large 74 77 77 79 82
All Firms 29 32 34 37 38
Sell Online Small 6 7 6 7 6
Medium 12 13 14 12 10
Large 15 16 16 13 16
All Firms 7 8 7 7 7
Purchase Online Small 20 29 35 40 40
Medium 30 47 50 59 63
Large 52 57 61 62 68
All Firms 22 32 37 43 43

As illustrated by table 2, small firms lag behind in all areas of Internet access and use, in comparison with medium and large firms. In particular, small firms lag significantly when it comes to selling and purchasing online. As well, in a letter to Members of Parliament prepared by the Canadian Chamber of Commerce in 2006, the following statements were made: "Canada is lagging internationally when it comes to adopting E-business sol utions, especially among small and medium sized enterprises (SMEs). Investment in Information and Communication Training (ICT) advancement by both business and the government would boost Canada's productivity and competitiveness and ICT advancement."

To continue, the October 2006 pre-budget submission by the Information Technology Association of Canada (ITAC) stated that, "there is a large amount of empirical research that indicates that firm size has an influence on ICT adoption… Regardless of what instrument is preferred, ITAC believes that incentives for the employment and training of skilled ICT workers to support small and medium businesses are equally important to facilitate the introduction of technology."

Interviewees agreed that SMEs have a definite need for IT training and that there is a continued need for the SC program. SMEs are becoming more and more aware of the importance and need for web presence in order to remain competitive in today's market. Many respondents stated that SC increases the competitiveness of Canadian SMEs by providing them with critical E-business skills. SC provides tailored IT training and advice at an affordable price; generating a constant demand for services by small and medium-size organizations.

In surveys of SMEs, respondents were asked to estimate how much they think they would be charged, per hour, for their IT training or service if it had been offered by a private organization. Average costs identified by respondents varied between $41.63 and $68.67. A limited investigation into private sector training was conducted for this evaluation, to identify the costs of private sector training similar to the training provided through SC. However, the identification of comparable training proved to be problematic. While there are many private sector firms that offer computer training, introductory training tends to be on basic computer skills or word processing packages. Alternatively, computer training provided by universities and colleges tends to be in computer programming. It was very difficult to find private sector training comparable to what is offered by SC. One company which was identified, and for which pricing information could be obtained, provides classroom training at a cost of $42.53 per hour, and on-site, tailored training at a cost of $100 per hour (with a minimum of 3 hours required).

Finally, the demand for SC training is impacted by the fact that there is a continuous evolution of SC products and services to meet the needs of clients. When first introduced in 1996, the program was centred on the Internet and connectivity. In 1999, Student Connections implemented Y2K consultations to respond to their clients' IT challenges. Since then, E-business and E-commerce have become an evolving trend. SC is therefore adapting training session curriculum and services to SMEs in order to reflect technology and business environment advancements.

Q.2 Does the SC Program address IC and HRSDC priorities?

Conclusion

The SC Program addresses IC, Human Resources Social Development Canada (HRSDC), and more specifically YES priorities. Under its 'competitive industry and sustainable communities' strategic objective, IC has committed to supporting participation in the digital economy. Additionally, SMEs are clearly an important driver to Canada's economy, as is access to and use of enabling technologies. With regards to HRSDC priorities, the SC program supports the department in its mission to help Canadians (including students and seniors) move through life's transitions. SC provides students with valuable and meaningful work experiences and abilities that will help them successfully transition into the workforce. This outcome is directly aligned with HRSDC's Youth Employment Strategy.

Findings

Findings in this section are based primarily on document reviews.

Industry Canada's mandate is to help make Canadians more productive and competitive in the knowledge-based economy, thus improving the standard of living and quality of life in Canada.Footnote 7 According to the IC website, Industry Canada aims to help Canadians contribute to the knowledge economy and improve productivity and innovation performance through its three strategic objectives:

  • a fair, efficient and competitive marketplace;
  • an innovative economy; and
  • competitive industry and sustainable communities.

Under the 'competitive industry and sustainable communities' objective, IC has committed to supporting participation in the digital economy. According to the IC website, "in a global, knowledge-based economy driven by rapid technological change, success is determined by the power to innovate. Connectedness is pivotal in empowering Canadians with the skills, competencies and tools necessary to innovate and take advantage of Canada's worldclass ICT infrastructure. It also promotes sustainable development and provides Canadians with access to education, knowledge, commerce and opportunities in the communities in which they reside, providing all Canadians with the means to participate in the creation and sharing of knowledge."Footnote 8

Further IC has indicated it will "continue to create opportunities for all Canadian individuals, businesses and communities to acquire the skills, competencies and tools required to fully participate in the digital economy."Footnote 9 According to Statistics Canada's Survey of Employment,

Payrolls and Hours (SEPH), on average in 2006 just over 5.1 million employees on payroll, or 48 percent of the total private sector labour force, worked for small enterprises (those with fewer than 100 employees. Nearly 1.7 million, or 16 percent, worked for medium-sized enterprises (those with 100 to 499 employees). In total, therefore, SMEs employed just over 6.8 million, or 64 percent, of private sector employees covered by SEPH. As well, in Canada, SMEs make up 97 percent of all firms that sell goods and services abroad and SME exporters contribute far more than their proportional share of job creation.Footnote 10

Based on the mandate of IC, the importance of SMEs to the Canadian economy, and the recognized importance of ICT in participating in the digital economy, the SC Program clearly aligns with IC priorities.

HRSDC's mission is "to build a stronger and more competitive Canada, to support Canadians in making choices that help them live productive and rewarding lives, and to improve Canadians' quality of life." In support of this mission the department "creates programs and support initiatives that help Canadians move through life's transitions—from families with children to seniors, from school to work, from one job to another, from unemployment to employment, from the workforce to retirement." The SC Program addresses this priority of HRSDC since it supports youth in developing skills, and gaining work experience, that will ultimately assist them in finding permanent employment. Through the SC program, post-secondary students obtain short-term, career-related employment which allows them to acquire valuable skills and experience that will increase their chances of successfully entering the workforce. As well, the Program supports seniors, improving their quality of life through the development of ICT skills.

More specifically, within Service Canada, the Youth Employment Strategy (YES) was created to help young people, particularly those facing barriers to employment, get the information and gain the skills, work experience and abilities they need to make a successful transition to the workplace. Under the YES, Service Canada offers three programs: Career Focus, Skills Link, and Summer Work Experience. Summer Work Experience provides wage subsidies to employers to create summer employment for secondary and post-secondary students, and support the operation of summer employment offices.Footnote 11 The Student Connections Program is one of many Summer Work Experience Programs. SC delivers on-site customized Internet and computer training and e-business services to small- and medium-sized enterprises and seniors. Annually, approximately 400 qualified post-secondary students are hired as Student Business Advisors (SBAs) and trainers to deliver these training and consultation services. As a result these students gain skills and work experience equipping them for future employment.

top of page

3.2 Success

Q.3 Is SC successful in at obtaining its performance targets?

Conclusion

The SC Program has been successful in meeting its performance targets of number of clients trained, number of youth hired, and ratio of SMEs to seniors trained. Targets for the ratio of training on national products, ratio of training on E-commerce products, and ratio of sales revenue to IC allocation of funding have not been met. However, the fact that the Program has not met targets on types of training provided (i.e. national products and E-commerce products) may be due to confusion relating to classification of training sessions.

Context

There are a number of performance targets established for the SC Program. The SC Program should be commended for having set performance targets. Targets provide a useful means of measuring, in a quantifiable way, the achievement of program objectives. The targets established for the SC Program were set keeping in mind factors such as IC and SC mandates (i.e., the focus on SME clients, and the focus on Internet or e-business training), as well as what is feasible for a centre, particularly in terms of revenue targets and clients trained. Targets are set to encourage Centres to strive for meaningful results and to challenge the Centres, while at the same time providing opportunities for them to achieve success with the Program. Most of the targets have remained the same over the evaluation period; with the exception of the e-commerce target, which increased 5% per year over the evaluation period. This target was increased since e-commerce is viewed as being key to the mandate of the program.

The SC Program targets for the evaluation period were:

  • Number of clients trained (national target = 15,000 / year);
  • Number of youth hired (national target = 300 / year);
  • Ratio of SMEs to Seniors trained (national target = at least 60% SMEs)Footnote 12;
  • Ratio of training done on National products (national target = 80% of clients trained);
  • Ratio of training done on E-commerce products (national target: for 2004–05 — 35% of clients trained; for 2005–06 — 40% of clients trained; and for 2006–07 — 45% of clients trained); and
  • Ratio of sales revenue to IC allocation of funding (national target = revenue equal to 40% of IC allocation).
Findings

The findings for this evaluation question are based primarily on administrative data for the Program, provided by the ACCC.

As illustrated by the following table, the SC Program met the targets relating to the number of clients trained and the number of SMEs trained, for each year of the evaluation period. With regards to the percentage of training done on National Products and E-Commerce Products, the Program met its target in FY 2004–05 but did not the following two years. However, according to IC SC Program staff, these numbers may not, in some cases, present a true picture, and in other cases can be explained by a variety of reasons. (Note: these reasons are described briefly later in this section)

Table 3: Targets in Relation to Number of Clients and SMEs trained
  2004–05 2005–06 2006–07
Target Actual % of Target Target Actual % of Target Target Actual % of Target
Total Clients 15,000 16,296 108.6% 15,000 15,060 100.4% 15,000 16,637 110.9%
Total SMEs  9,778 11,589 118.5% 9,036 10,825 119.8% 9,982 11,796 118.2%
National products 13,037 13,285 101.9%  12,048 11,298 93.8% 13,310 12,178 91.5%
E-Commerce Products 5703.6 6,647 116.5% 6024 5,668 94.1% 7486.65 5,228 69.8%

Table 3 above presents the number of instances of training for clients and SMEs (i.e., if the same individual took more than one training session, each instance of training was counted in the totals for actual total clients and actual total SMEs). The following table, however, presents the number of distinct participants (SMEs and seniors) for each of the years included in the evaluation period.

Table 4: Number of Distinct Participants, by Year
Notes: Total Distinct Participants does not equal the sum of SME distinct participants and Seniors Distinct Participants because Total Distinct Participants includes all participants including clients listed as "other" (neither SME nor Senior).
Year Total Distinct Participants Footnote 1 SME Distinct
Participants
Senior Distinct Participants
Total # % Total # %
2004–05 8903 6334 86.2 1016 13.8
2005–06 6922 4972 85.1 868 14.9
2006–07 8065 6566 83.4 1305 16.6
Total 23,890 17,872 84.9 3,189 15.1

The SC Program national target requires that at least 60% of clients trained be SMEs. As indicated by the above table, the SC Program has exceeded this target each year. However, at the Centre level, not all Centres have met this target each year. The following table presents the percentages of training provided to SMEs and Seniors, by Centre, by year. (Note: these percentages are for instances of training, not for distinct participants. Distinct participant figures were not provided at the Centre-level.) (For numbers of instances of training for SMEs and Seniors, by Centre, by year, refer to table I1 in Annex I.)

top of page

Table 5: Percentages of Training Provided to SMEs and Seniors, by Centre, by Year.
Centre 2004–05 2005–06 2006–07 All Years
% SMEs % Seniors % SMEs % Seniors % SMEs % Seniors % SMEs % Seniors
Algonquin College 26.7% 73.3%         26.7% 73.3%
CCNB 97.8% 2.2% 64.4%  35.6% 24.0% 76.0% 43.2% 56.8%
Cégep de Jonquière 94.2% 5.8% 69.9%  30.1% 55.6% 44.4% 72.7% 27.3%
Cégep de Saint Hyacinthe 99.5% 0.5% 76.0% 24.0% 54.4% 45.6% 86.0% 14.0%
Collège Boréal 93.9% 6.1% 91.9% 8.1% 48.1% 51.9% 90.8% 9.2%
HEC Montréal 99.3% 0.7% 99.3% 0.7% 99.8% 0.2% 99.5% 0.5%
Holland College 82.0% 18.0% 74.8% 25.2% 82.2% 17.8%  79.7% 20.3%
Humber 69.1% 30.9% 87.9% 12.1% 85.9% 14.1% 84.2% 15.8%
John Abbott College 47.8% 52.2%         47.8% 52.2%
La Cité collégiale     92.0% 8.0% 82.7% 17.3% 85.8% 14.2%
Langara College 32.8% 67.2% 49.7% 50.3%     41.6% 58.4%
Memorial University     11.8% 88.2% 20.7% 79.3% 18.2% 81.8%
Okanagan College         87.7% 12.3% 87.7% 12.3%
Red River College 18.6% 81.4%  82.3% 17.7% 72.6% 27.4% 48.6% 51.4%
SAIT 77.1% 22.9%         77.1% 22.9%
University of Alberta     100.0% 0.0% 59.4% 40.6% 59.9% 40.1%
University of Calgary     100.0% 0.0% 98.0% 2.0% 99.5% 0.5%
University of Regina 75.6% 24.4% 84.7% 15.3% 77.8% 22.2% 79.7% 20.3%
Wilfrid Laurier University 99.9% 0.1% 100.0% 0.0% 96.4% 3.6% 98.6% 1.4%

The highlighted cells in the table indicate where the Centres have fallen short in reaching the 60% target for SME training, over the previous three years combined. SC Program staff have pointed out that there is a certain amount of turnover in the Centres that participate in the SC Program, and relatively new Centres will, and do, have difficulty meeting the targets. It takes time for Centres to become established enough to meet the national targets. However, this rationale does not apply to the six Centres that did not meet the 60% target for the percentage of training provided to SMEs in FY 2006–07. None of these Centres were new Centres that year. (Note: cells which are blank indicate that the Centre was not part of the SC Program during that year.)

Other factors impacting on the ability of the Centres to reach targets include:

  • the location of the Centre within the institution (Centres located within a business centre tend to have greater success than those situated in a continuous education sector of the organization);
  • the ability of the Centre to hire, develop and retain a strong Coordinator, whether or not the Coordinator is full-time, and the degree of management experience of the Coordinator (Centres with full-time Coordinators tend to be more successful than those with part-time Coordinators);
  • the focus of the Centre (some Centres put greater focus on sales revenue, whereas others focus more on student experience);
  • support of senior management and the institution (especially with regards to resources for marketing, human resources, and operational administration of the Centre);
  • the extent to which the Program is integrated within the institution, and the extent to which it is perceived as being in competition with other offerings at the institution; and
  • the existence, or not, of a strong business network.

Four of the six Centres that did not meet the targets during 2006–07 (as illustrated in table 5 above) had hired new Coordinators, which may have impacted their ability to meet established targets. As well, although the Okanagan College did meet the SME target during its first year in existence, this Centre's targets were reduced since targets are directly linked to the number of interns which, at Okanagan, was 13 in the first year instead of the originally expected 20.

The SC Program target for training on national products requires that at least 80% of training provided be on national products. As indicated by the table on the previous page, the SC Program as a whole has exceeded this target in 2004–05, however has fallen short in the two following years. The following table presents the percentages of training provided on national products, by Centre, by year. (Note: these percentages are for instances of training, not for distinct participants. Distinct participant figures were not provided at the Centre-level.)

Table 6: Percentage of Training Provided on National Products, by Centre, by Year
Centre 2004–05 2005–06 2006–07
Total National
Products
Training
% Total National
Products
Training
% Total National
Products
Training
%
Algonquin College 263 88%        
CCNB 103 62% 325 86% 592 92%
Cégep de Jonquière (Total) 1009 68% 1313 73% 1310 82%
Cégep de Saint Hyacinthe 3361 93% 1386 85% 1009 75%
Collège Boréal (Total) 1231 85% 1081 89% 95 66%
HEC Montréal 1281 70% 1073 53% 1339 54%
Holland College (Total) 1200 74% 984 83% 597 73%
Humber  485 97% 1041 80% 849 73%
John Abbott College 889 97%        
La Cité collégiale     688 80% 1432 86%
Langara College 155 87% 221 74%    
Okanagan College         615 89%
Memorial University     516 90% 1122 84%
Red River College 592 93% 371 98% 127 58%
SAIT 825 80%        
University of Calgary     73 37% 24 47%
University of Regina 870 82% 1213 88% 1371 94%
Wilfrid Laurier University 1021 68% 994 55% 1120 54%
University of Alberta     19 100% 576 61%
TOTAL 13,285 82% 11,298 75% 12,178 73%

top of page

The highlighted cells in the table indicate where the Centres have fallen short in reaching the 80% target for national products. As indicated by the table, there are two Centres that have not met the 80% target in any of the previous three years. As well, as was the case above, it is not the new Centre (Okanagan) that had trouble meeting the target during the last fiscal year. During the 2006–07 fiscal year, nine Centres did not meet the 80% target for the percentage of training on national products. None of these Centres were new Centres that year.

However, SC Program staff have suggested that following the introduction of a content management system (Studio), it is sometimes difficult for SC Centres to categorize National products. Since new training offerings can now be made by pulling together parts of previously different training modules, it is not always clear what is considered a National Product. Therefore, Centre Coordinators may not be classifying some training as National Product training, when they should be.

The SC national target for E-Commerce training has changed each year over the evaluation period. In 2004–05, the national target for E-Commerce training was 35% of client training sessions; in 2005–06 the national target was 40%; and in 2006–07 the national target was 45%. As indicated by table 3 on page 13, the SC Program as a whole exceeded the target in 2004–05, however has fallen short of the target in the two following years. The following table presents the percentages of E-Commerce training provided, by Centre, by year. (Note: these percentages are for instances of training, not for distinct participants. Distinct participant figures were not provided at the Centre-level.)

Table 7: Percentage of E-Commerce Training Provided, by Centre, by Year
Centre 2004–05 2005–06 2006–07
Total E-Commerce
Training
% Total E-Commerce
Training
% Total E-Commerce
Training
%
Algonquin College 61 20%        
CCNB 39 23% 39 10% 4 1%
Cégep de Jonquière (Total) 286 19% 544 30% 212 13%
Cégep de Saint Hyacinthe 2,044 57% 544 33% 350 26%
Collège Boréal (Total) 555 38% 570 47% 18 13%
HEC Montréal 723 39% 801 40% 235 10%
Holland College (Total) 739 46% 660 56% 124 15%
Humber 321 64% 441 34% 262 23%
John Abbott College 223 24%        
La Cité collégiale     264 31% 3 0%
Langara College 40 22% 59 20%    
Okanagan College         47 7%
Memorial University     79 14% 91 7%
Red River College 80 13% 264 70% 35 16%
SAIT 461 45%        
University of Calgary     21 11% 10 20%
University of Regina 368 35% 779 56% 206 14%
Wilfrid Laurier University 707 47% 586 32% 282 13%
University of Alberta     17 89% 48 5%
TOTAL 6,647 41% 5,668 38% 1,927 12%

The highlighted cells in the table indicate where the Centres have fallen short in reaching the targets for E-Commerce products. As indicated by the table, none of the Centres met the 45% target in the last fiscal year. As well, eight Centres have not met the targets during the past two fiscal years.

As noted in Question 1, SC products have changed over time, and continue to change based on the needs of clients. E-Commerce training is complex and as SMEs become more advanced in their skills and knowledge, the E-Commerce training offered to them takes a different form (more one-on-one, as opposed to classroom training) and fewer clients are trained with the same amount of effort. This may explain, to some extent, the reasons for the difficulties experienced by the Centres in meeting the E-Commerce targets.

The SC program has a target of hiring 300 interns per year. At the national level, this target has been greatly exceeded each year of the evaluation period, with the number of interns hired being 426, 398 and 399 (in 2004–05, 2005–06 and 2006–07 consecutively). However, when looking at only those interns who completed their internships (i.e., worked 210 hours or more), the number of interns is closer to 300. Total completed internships per year were 300 in 2004/05, 319 in 2005/06, and 306 in 2006/07. (See Table I2 in Annex I, for a breakdown of the number of completed internships by Centre, by year.) The threshold of 210 hours of work, to be considered a completed internship, was established since this is thought to provide enough experience for the students to gain the knowledge and skills available to them through the SC Program. 210 hours equates to six weeks of work, if the student were hired full-time (note: SC interns do not work full-time).

The final performance target for the SC Program relates to the revenue generated by the Centres. Funding for the SC program comes from three main sources: the IC allocation; funding from host organizations, in the form of cash or in-kind contributionsFootnote 13; and, finally, through revenues raised by fees charged for training. The SC Program has a national target of sales revenue being equal to at least 40% of the IC allocation. The following table presents the sales revenues earned by each Centre, each year, and what percentage this equated to of the IC funding.

Table 8 : Percentage of IC Allocation Earned in Sales Revenue, by Centre, by Year
Administrative Centre 2004–05 2005–06 2006–07
Revenue % of IC allocation Revenue % of IC allocation Revenue % of IC allocation
Algonquin College $ 25,240.33 23.4%        
CCNB – Bathurst $ 22,620.00 22.8% $ 12,062.00 8.8% $ 11,554.25 8.8%
Cégep de Jonquière $ 62,949.19 39.0% $ 88,042.89 39.4% $ 82,535.38 37.9%
Cégep de St-Hyacinthe $156,313.71 71.9% $ 60,534.62 34.1% $ 14,306.36 9.3%
Collège Boréal $ 81,491.88 58.4% $ 65,939.55 48.2% $ 10,885.98 8.0%
HEC $ 99,532.49 48.4% $ 95,054.69  43.6% $117,186.84 65.1%
Holland $ 58,786.40 26.3% $ 47,699.00 22.8% $ 45,056.26 24.8%
Humber College $ 7,612.50 4.6% $ 16,184.55 9.4% $ 11,696.50 6.8%
John Abbott College $ 53,577.53 40.7%        
La Cité Collégiale     $ 12,982.45 12.5% $ 10,735.53 7.2%
Langara College $ 12,749.60 10.1% $ 10,494.10 10.9%    
Memorial University     $ 3,554.50 4.9% $ 14,827.80 11.3%
Okanagan College         $0.00 0.0%
Red River College $ 29,399.98 20.8% $ 9,555.00 8.2% $ 1,424.40 2.0%
SAIT $ 41,317.36 30.2%        
University of Alberta       0.0% $ 18,236.41 13.3%
University of Calgary     $ 13,904.00 28.4% $ 3,234.50 7.4%
University of Regina $ 79,000.78 53.2% $ 84,519.47 37.7% $ 62,220.12 30.5%
Wilfrid Laurier University $ 75,709.48 55.4% $ 65,699.06 41.1% $ 84,677.56 59.4%
Total $806,301.23 37.7% $586,225.88  27.5% $488,577.89 22.8%

top of page

The highlighted cells in the table indicate where the Centres have fallen short in reaching the targets for sales revenue. As indicated by the table, only two of the fifteen Centres met the revenue target last fiscal year, and only three Centres the year before. As well, the Program, as a whole, has not met the revenue target. When asked about the fact that Centres are not achieving their revenue targets, IC Program staff indicated that while they have not investigated the reasons for this, they believe that some Centres have different objectives within their institution, with some being more concerned about providing work experience to youth rather than focusing on generating revenue. As well, IC Program staff indicated that not as much emphasis from IC has been placed on the revenue target as on the e-business or SME targets. Finally, some Centres have been offering free introductory services to promote the program, which would have impacted on their revenues.

Recommendation

With regards to SC targets, IC should re-examine its targets relating to:

  • d) ratio of training done on national products,
  • e) ratio of training on e-commerce products, and
  • f) ratio of sales revenues to the IC allocation of funding, to ascertain whether these targets are still useful and valid.

If IC retains the national target relating to the ratio of training done on national products, guidance should be provided to the Centres as to how to determine if training is considered national product training. If IC retains the e-commerce target, IC may wish to adjust the target to examine what percentage of Student Business Advisor (SBA) training hours is spent on E-commerce training.

Q.4 Is SC providing its expected benefits to its target groups?

Conclusion

The SC Program is providing its expected benefits to target groups. Student Business Advisors benefit from the work experience, gaining new skills and knowledge, and increased self-confidence. As well, the financial compensation for their work assists them in financing their continuing education. Seniors also benefit from increased skills and knowledge. SMEs benefit from improved ICT skills which enable them to change how they use the Internet for their businesses. Finally, there is some evidence that the host institutions also benefit from their involvement in the SC Program, by using it to leverage funding for other programs, and to attract students.

Findings

Findings in this section are based, primarily, on information gathered through interviews and ACCC surveys.

The SC Program provides benefits to three main target groups: (1) students employed as SBAs, (2) SMEs, and (3) seniors who benefit from the SC training.

SMEs Benefits

Interviewees noted a number of SC benefits for SMEs. Most interviewees noted that SC training for SMEs is affordable, and is flexible with different teaching formats (one-on-one, groups, seminars, consultations) and locations. Interviewees also noted that the training is available during the days or evenings, and is relevant and tailored to the individual needs of the SMEs.

According to interviewees, SC training helps businesses increase their competitiveness by giving them an understanding of E-commerce and the Internet, and helping them get their businesses on-line. An additional benefit of the program that was noted by several interviewees is the fact that the SC program offers advice to SMEs that is independent and unbiased, as opposed to the advice they may receive from vendors of ICT products.

According to the surveys, SMEs feel they have gained knowledge and skills as a result of their participation in SC training. Two-thirds, or more, of respondents agreed or strongly agreed with three survey questions which asked about knowledge and skills acquired through SC. Specifically:

  • 73% of respondents agreed or strongly agreed that they "received practical instruction that has improved (their) use of information technology";
  • 65.6% of respondents agreed or strongly agreed that "the training (they) received has helped (them) take advantage of computer-related tools"; and
  • 70.7% of respondents agreed or strongly agreed that they "gained information or learned skills that are relevant to (their) business needs".

As well, when including those respondents who selected the "neutral" response, we are left with very few individuals who disagreed with these statements. (See Table I3 in Annex I for the table illustrating the extent to which SMEs feel they have gained knowledge and skills as a result of their participation in SC training.)

The survey of SMEs also asked questions to assess whether the behaviour of SMEs, with respect to ICT, has changed as a result of their training. The following table indicates, for the 2 year period of surveys covered by the evaluation, the percentage of respondents for each answer prior to and following training:

Table 9: SME Views on Extent to Which Their Behaviour Has Changed as a Result of SC
Percentage of Responses to the question: Prior to/After your Student Connections training, how often did you use the Internet for…
  Response: Total
Prior to After
… researching price and
availability of goods and services
for your business?
Never 23.7% 14.6%
Less than once per month 11.8% 9.6%
1 to 2 times per month 13.5% 17.5%
3 to 4 times per month 11.0% 13.8%
More than 4 times per month 32.4% 34.4%
no response 7.6% 10.1%
 
… placing an order or purchasing
business-related goods
and services
Never 31.8% 26.2%
Less than once per month 22.3% 20.3%
1 to 2 times per month 14.4% 15.5%
3 to 4 times per month 7.3% 9.9%
More than 4 times per month 15.8% 17.7%
no response 8.5% 10.4%
 
… web-based marketing and
promotion?
Never 41.7% 33.0%
Less than once per month 16.1% 13.2%
1 to 2 times per month 8.5% 12.7%
3 to 4 ti mes per month 4.2% 7.0%
More than 4 times per month 20.6% 22.5%
no response 9.0% 11.5%
 
… selling your business' goods
and services online?
Never 59.2% 47.9%
Less than once per month 9.3% 10.4%
1 to 2 times per month 4.8% 9.0%
3 to 4 times per month 2.3% 4.8%
More than 4 times per month 15.5% 16.6%
no response 9.0% 11.3%
 
… distributing E-newsletters? Never 58.9% 49.3%
Less than once per month 11.8% 12.7%
1 to 2 times per month 7.3% 9.3%
3 to 4 times per month 3.4% 6.2%
More than 4 times per month 9.9% 11.5%
no response 8.7% 11.0%
 
… online banking and
financial management?
Never 37.5% 31.8%
Less than once per month 4.5% 3.1%
1 to 2 times per month 6.8% 9.3%
3 to 4 times per month 8.7% 8.7%
More than 4 times per month 34.1% 36.3%
no response 8.5% 10.7%
 
… locating industry-specific
and specialized business
information?
Never 26.8% 17.5%
Less than once per month 11.5% 9.9%
1 to 2 times per month 11.5% 13.2%
3 to 4 times per month 11.8% 13.0%
More than 4 times per month 30.4% 33.0%
no response 7.9% 13.5%
 
… Canada Business Network
( www.canadabusiness.gc.ca)?
Never 72.7% 54.1%
Less than once per month 9.9% 14.9%
1 to 2 times per month 3.9% 9.6%
3 to 4 times per month 1.7% 1.4%
More than 4 times per month 2.5% 5.6%
no response 9.3% 14.4%
 
… Strategis
( www.strategis.gc.ca)?
Never 69.9% 57.7%
Less than once per month 12.1% 15.2%
1 to 2 times per month 5.1% 9.9%
3 to 4 times per month 2.0% 2.3%
More than 4 times per month 2.0% 3.9%
no response 9.0% 11.0%

top of page

As illustrated by the table, there was a significant decrease in the number of respondents who responded with "never" or "less than once per month" with regards to how often they used the Internet for various actions. When looking at only those survey respondents who indicated they "never" used the Internet for the various reasons, the greatest change came in the number of respondents who accessed the Canada Business Network. Prior to the SC training, almost three-quarters of respondents indicated they never used the Internet to access the Canada Business Network (www.canadabusiness.gc.ca). However, following the SC training the number of respondents who indicated they never accessed this site had dropped by 18.6%. As well, 11.3% fewer respondents indicated that they "never" used the Internet to sell their goods and services online, following SC training. Similarly, there were high decreases in the number of respondents who chose the "never" response with regards to their use of the Internet for: researching price and availability of goods and services for their business (9.1% decrease), web-based marketing and promotion (8.7% decrease), and distributing E-newsletters (9.6% decrease). Thus, we see, that the SC training provided to SMEs is benefiting the organizations through improved usage of the Internet for various purposes.

(For detailed survey responses, by year, from SMEs regarding their use of the Internet prior to and after the SC training, refer to Table I4 in Annex I.)

As indicated by the findings presented above, interviewees and survey respondents agree that the SC Program is providing benefits to SMEs. These benefits relate to the flexible, affordable characteristics of the training, as well as to skills and knowledge gained through participation in SC training, which has led to changes in how SMEs use the Internet.

SBA Benefits:

One of the main benefits for students, of the SC Program, is the work experience it provides to SBAs. Interviewees agreed that the SC Program provides key benefits to students including the opportunity to develop practical skills and real life experience in areas such as:

  • technical ICT skills,
  • teaching / presentation skills,
  • communication skills,
  • sales and administrative skills,
  • product development,
  • marketing,
  • human resources,
  • team-working,
  • problem-solving skills, and
  • financial skills.

In addition, interviewees frequently noted that one of the benefits of the Program to students is that it instils self-confidence. The SC Program also provides exposure to and a connection with diverse business clients, and allows students to build and develop networks. Students also gain practical experience in all aspects of running an entrepreneurial entity. Finally, a key benefit to students is the financial compensation they receive which provides them with assistance in financing their education.

The following table illustrates the responses of SBAs to survey questions asking them about their satisfaction level with their work experience as an SBA.

Table 10: SBA Satisfaction Levels with Regards to Their Work Experience
Q: I am satisfied with my overall work experience as a Student Business Advisor.
Response: Jan –
Sept 2005
Oct 05 –
Mar 06
Apr –
Sept 06
Oct 06 –
Mar 07
All Years
# % # % # % # % # %
Strongly disagree 1 1.9% 3 5.5% 0 0.0% 4 4.0% 8 3.2%
Disagree 4 7.5% 3 5.5% 3 7.7% 7 6.9% 17 6.9%
Neutral 0 0.0% 3 5.5% 3 7.7% 7 6.9% 13 5.2%
Agree 25 47.2% 23 41.8% 17 43.6% 38 37.6% 103 41.5%
Strongly Agree 23 43.4% 23 41.8% 16 41.0% 45 44.6% 107 43.1%
Total 53 100.0% 55 100.0% 39 100.0% 101 100.0% 248 100.0%
 
Considering my expectations in becoming a student business advisor,
Student Connections has been a valuable work experience.
Response: Jan –
Sept 2005
Oct 05 –
Mar 06
Apr –
Sept 06
Oct 06 –
Mar 07
All Years
# % # % # % # % # %
1–Strongly disagree 2 3.8% 5 9.1% 1 0.5% 3 3.0% 11 2.8%
2–Disagree 4 7.5% 5 9.1% 3 1.6% 5 5.0% 17 4.3%
3–Agree 25 47.2% 28 50.9% 18 9.9% 49 48.5% 120 30.7%
4–Strongly Agree 22 41.5% 16 29.1% 158 86.8% 44 43.6% 240 61.4%
No response   0.0% 1 1.8%  2 1.1%   0.0% 3 0.8%
Total 53 100.0% 55 100.0% 182 100.0% 101 100.0% 391 100.0%

As shown by the table, SBAs are highly satisfied by their work experiences. 84.6% of SBAs either "agreed" or "strongly agreed" with the statement "I am satisfied with my overall work experience as a Student Business Advisor". As well, 92.1% of SBAs either "agreed" or "strongly agreed" with the statement "Considering my expectations in becoming a student business advisor, Student Connections has been a valuable work experience".

The SC Program is also expected to lead to increases in SBA's ICT knowledge and business skills. Based on responses to survey questions, SBAs believe that the training they received, and their experiences as SBAs, has lead to increased knowledge and skills.

With regards to skills development, SBAs believed that the SC Program has "significantly" or "very significantly" helped them to:

  • become more confident in dealing with people in a professional setting (77.4%),
  • prepare presentations and effectively communicate in a business environment (71.4%)
  • gather and organize information and apply knowledge from a variety of disciplines (77.9%)
  • work independently as the job requires (84.3%)
  • work as part of a team as the job requires (69.8%)
  • enhance and develop IT skills (69.4%)
  • become better at identifying, analyzing, and solving problems faced by businesses (64.9%)
  • plan and carry out a project or task successfully from start to finish (76.1%), and
  • develop marketing and sales skills (58.0%).

(For detailed survey responses from SBAs regarding skills development, refer to Table I5 in Annex I.)

SBAs also indicated that as a result of their employment as an SBA, they acquired new information about various topics and tools. However, they were less positive about this than they had been about their development of new skills (as indicated above). The following table indicates the percentage of respondents who indicated they had acquired new knowledge. The two percentages noted for each item indicate the percentage who a) responded with either "a lot" or "extensive information" when asked about these specific tools, and b) responded with "some", "a lot" or "extensive information".

top of page

Table 11: Percentage of SBAs Who Indicated They Had Acquired New Knowledge
Q: As a result of your employment
as an SBA, how much new
information did you learn about…
% of 'A
Lot'/
Extensive
% of Some/
'A Lot'/
Extensive/
Internet search engines 30.7% 60.3%
SME's business needs issues 45.7% 73.6%
E-business tools 43.7% 74.1%
Internet security 37.6% 64.3%
Industry Canada 34.0% 68.0%
Strategis 21.9% 50.6%
Canada Business Network 21.9% 47.4%
The Youth Employment Strategy 23.9% 55.5%

(For detailed survey responses from SBAs regarding increased knowledge, refer to Table I6 in Annex I.)

SBAs were also asked various questions on the survey relating to whether or not the SC Program has impacted their marketability in their chosen field. SBAs were asked to indicate the extent to which they agreed with a series of statements relating to their marketability. The following lists the statements and the percentage of respondents who responded that they either "agree" or "strongly agree":

  • "I am a more advanced user of the Internet's resources, including E-business applications and technologies, as a result of my experience as a SBA." (73.4%)
  • "Student Connections provided me with an opportunity to showcase my skills to business clients." (81.4%)
  • "Student Connections provided me with an opportunity to network with potential future employers." (61.7%)
  • "I gained work experience that will be an asset when I look for work in the future." (92.8%)
  • "I gained new skills that will be an asset when I look for work in the future." (87.5%)
  • "I believe that one of my business clients would offer me a job in their company, if there was an appropriate opening." (61.3%)
  • "Student Connections provided me with professional experiences in my field of study." (72.6%)
  • "Student Connections will help me find employment in my field of study." (58.0%)

For the most part, SBAs were very positive about the impact their participation in the SC Program has had on their marketability to the labour force. There does seem to be a discrepancy between survey answers for the similar survey questions of "I gained work experience that will be an asset when I look for work in the future" (to which 92.7% agreed or strongly agreed) and "Student Connections will help me find employment in my field of study" (to which only 58.1% agreed or strongly agreed). However, the wording of the second question could have been interpreted by some to mean that the SC Program will play an active role in SBAs' job searches.

(For detailed survey responses from SBAs regarding their increased marketability, refer to Table I7 in Annex I.)

The final benefit of the Program to SBAs is the money earned, which helps SBAs to return to school. The following table indicates the number of interns hired, and the average salary paid to interns over the evaluation period. As the table indicates, interns were paid, on average $4,672.29 over the past three years. Table I8 in Annex I provides the average salary paid to interns, by Centre, by year. However, it is not possible to draw conclusions from this table, since the breakdown of the number of hours worked by interns is not available. Therefore, where a Centre has an average salary lower than the national average, it is possible that this figure is being artificially reduced by interns who were hired, but did not work their expected 210 hours, and therefore received less pay.

Table 12: Average Salary Paid to Interns, Per Year
Year # of Interns Hired Intern Salary &
Benefits Paid to Centres
Average Salary
Paid to Interns
2004–05 426 $ 1,900,770.76 $ 4,461.90
2005–06 398 $ 1,908,590.83 $ 4,795.45
2006–07 399 $ 1,904,843.81 $ 4,774.04
Total: 1,223 $ 5,714,205.40 $ 4,672.29

According to the ACCC, contracts with the Centres state that each intern must be paid a minimum of $11.50 per hour plus any applicable benefits. The ACCC pays the Centres $11.50 per hour plus 20% for benefits, for a total of $13.80 per hour per intern. In Canada, the current minimum wage ranges from a low of $7.00 per hour in Newfoundland, to a high of $8.50 in Nunavut.Footnote 14 The average minimum wage in Canada is $7.88. Thus, based solely on wage rates, the SC Program seems to offer an attractive opportunity to students to earn money for their continued education, although the ease with which the Program can attract students will vary across the country, based on other factors as well (e.g., the availability of other higher-paying jobs).

As indicated by the findings presented above, interviewees and survey respondents agree that the SC Program provides benefits to SBAs. The main benefits relate to the experience gained by the students, which ultimately makes them more employable, and to the knowledge and skills developed through their participation in the Program. A final benefit is the wages earned by SBAs, which help them finance their continuing education.

Seniors Benefits

All interviewees noted a number of benefits that seniors get from the SC program. The most commonly cited benefits to the seniors were improved ICT skills and social interaction. Interviewees noted that seniors are able to access training, at a low cost, that increases their computer literacy and improves their quality of life by giving them the skills to email family and friends, send and receive photos, etc. Several interviewees noted that seniors as a group can feel that they have been left behind, and the SC program provides them with a sense of belonging by giving them the skills to be able to take advantage of technology. A couple of interviewees also mentioned that they can access important information, in an increasingly electronic environment. Finally, interviewees noted that the SC training provided to seniors is tailored to them, offers flexibility in delivery locations, and is non-threatening since it includes only seniors.

Survey responses support the findings from interviews. The following table illustrates the extent to which seniors feel they have gained knowledge and skills as a result of their participation in SC training.

Table 13: Seniors' Views on Extent to Which They Have Gained Knowledge and Skills as a Result of SC
Thinking about the training you got through Student Connections, please indicate if you agree or disagree with the following statements:
  Response: Total
# %
I received practical instruction that has
improved my use of information technology.
1–Strongly disagree 4 1.3%
2–Disagree 21 6.6%
3–Neutral 37 11.7%
4–Agree 162 51.3%
5–Strongly Agree 81 25.6%
No response 11 3.5%
Total 316 100.0%
 
The training I received has helped me
take advantage of computer-related tools.
1–Strongly disagree 7 2.2%
2–Disagree 13 4.1%
3–Neutral 49 15.5%
4–Agree 145 45.9%
5–Strongly Agree 88 27.8%
No response 14 4.4%
Total 316 100.0%
 
I gained information or learned skills
that are relevant to my personal needs.
1–Strongly disagree 6 1.9%
2–Disagree 15 4.7%
3–Neutral 34 10.8%
4–Agree 158 50.0%
5–Strongly Agree 94 29.7%
No response 9 2.8%
Total 316 100.0%

As illustrated by this table, seniors are very positive about the skills acquired and knowledge learned as a result of their participation in the SC Program. Approximately three quarters of seniors agreed or strongly agreed that they had received practical instruction that improved their use of IT, and that the training they received had helped them take advantage of computer-related tools. (See Table I9 in Annex I for a breakdown of survey responses, by survey period.)

As indicated by the findings presented above, interviewees and survey respondents agree that the SC Program provides benefits to Seniors. The main benefits relate to improved ICT skills and the benefits accrued through social interaction and a sense of connection that Seniors can gain as a result of their participation in the Program.

As a final note, with regards to the benefits of the SC program, strengthened linkages between the Academic institutions and the business community was recognized as a potential benefit of the program (this was identified by the evaluation Steering Committee as a performance indicator for measuring success, as noted within the evaluation matrix in Annex D). During the interviews, Coordinators and business organizations were asked whether the SC program has provided benefits to the Centres participating in the program. Only two of the thirteen coordinators and one of the two business organizations interviewed indicated that this had been a benefit of the program. However, SC Program staff have suggested that it would be the SC senior managers who would be aware of this type of benefit; therefore, an email was sent to all senior managers (n=12) to ask about this issue. Four email responses were received from SC senior managers. Three of the four managers felt that strengthened connections between the academic institutions and the business community had been fostered by the SC program. The fourth senior manager indicated that SC is viewed by many businesses as a stand alone program, and therefore the business community does not always make the connection with the institution. However, this Centre is working on changing this perception.

Comments received from senior managers included:

.. business communities… view Student Connections as a point of contact with the institution to access students for work terms, part-time and full-time employment opportunities. Every year there is an increase in the number of small and medium sized businesses with whom we work.

SC provides us an opportunity to connect with newer and smaller businesses and to explore partnerships with business people who often have not otherwise had contact with the college.

Q.5 Has SC addressed the requirements of the Official Languages Act?

Conclusion

The SC Program has made numerous efforts to address the requirements of the Official Languages Act. Both through the development of sub-contracts with community colleges and by providing opportunities for members of official-language minority communities to become informed of the initiative, SC demonstrates clear support to English and French linguistic minority communities.

Context

The Official Languages ActFootnote 15 (1988) is an Act of Parliament that recognizes English and French as the official languages of Canada. The purpose of the Act is to:

  • a) ensure respect for English and French as the official languages of Canada and ensure equality of status and equal rights and privileges as to their use in all federal institutions, in particular with respect to their use in parliamentary proceedings, in legislative and other instruments, in the administration of justice, in communicating with or providing services to the public and in carrying out the work of federal institutions;
  • b) support the development of English and French linguistic minority communities and generally advance the equality of status and use of the English and French languages within Canadian society; and
  • c) set out the powers, duties and functions of federal institutions with respect to the official languages of Canada.

In accordance to section 41 of the Act, the Government of Canada is committed to:

  • a) enhancing the vitality of the English and French linguistic minority communities in Canada and supporting and assisting their development; and
  • b) fostering the full recognition and use of both English and French in Canadian society.
Findings

The findings in this section are primarily based on document reviews and a review of administrative data provided by ACCC and IC.

At this time, the following three (3) Administrative Centres are located in Official Language Minority Communities (OLMC):

  • La Cité Collégiale (ON)
  • Collège Boréal (ON)
  • Collège Communautaire du Nouveau-Brunswick (CCNB) (NB)

The SC Program makes an effort to promote the Program to students located in OLMCs and to other students whose first language is not that of their university or college, whether or not they live in OLMCs. Table 14 below indicates the number of students hired inside of Quebec whose first language is English, and the number of students hired outside of Quebec whose first language is French.

Table 14: Number of Students Hired in OLMCs Who Can Provide Training in Both Official Languages
Fiscal Year French-speaking students
outside Quebec
English-speaking students
in Quebec
2004–05 37 20
2005–06 49 2
2006–07 74 5

The above table illustrates the increasing availability of students that are capable of providing training services to French-minority clients located outside of the province of Quebec. However, the number of English-speaking students residing in Quebec has remained consistently low over the past two years. This is as a result of the locations of the Centres in Quebec, which are in areas where a lot of English is spoken, and where it is quite easy to hire bilingual students. At this time, the number of training sessions completed in French is not tracked; therefore no data could be assessed during this evaluation, with regards to the participation of Francophones in SC training.

In order to address the requirements of the Official Languages Act, section 3.13 of all sub-contract agreements between the ACCC and Colleges state: "Ensure that it respects the spirit of the Official Languages Act, and more specifically section 41 of that vitality of the English and French linguistic minority communities in Canada and supporting and assisting their development as well as fostering the full recognition and use of both English and French in Canadian society. To further this objective the Administrative Centre will ensure that all materials and services, including promotional materials, provided to clients and youth are made available in both official languages and plans are made for provision of services in both official languages."

According to the Small Business and Marketplace Services sector of IC, forty-five percent (45%) of students hired in 2003–04 reported French as their preferred official language. Furthermore, the Small Business and Marketplace Services sector indicates that SC provides greater opportunities for members of official-language minority communities to become informed about and gain a better understanding of the initiative and the services available in French. The following is a list of activities that have been undertaken to promote OL within SC:

  • Training in French during conference lunches & monthly meetings;
  • Promoting SC to young people through school boards serving minority communities;
  • Participating in the Francophonie week and meeting with students to discuss employment opportunities in the minority language;
  • Presenting the initiative to Regroupement franco-ontarien de développement économique et d'employabilité representatives so they can more effectively provide information to their clients about services provided by SC; and
  • Forging partnerships with other academic institutions in order to offer group sessions for members of official-language minority communities.

Persons interviewed commonly agreed that SC is successful in addressing its Official Languages requirements. Although requests for French training in English communities and vice versa are minimal, all administrative centers have stated that contingency plans are in place to meet ad hoc requests. In addition to hiring bilingual students in OLMCs, the administrative centers are provided with bilingual training and promotional materials.

top of page

3.3 Cost-effectiveness / Alternatives

Q.6 How cost-effective is the SC Program?

Conclusion

Funding for the SC Centres comes from IC in the form of funding for salaries, as well as through cash and in-kind contributions from the Centres and revenues raised through fees for training. IC funding accounted for approximately 56% of the total SC funding over the evaluation period. The cost of the Program was $8,585.45 per intern hired and $218.78 per client trained based on the total IC budget ($3.5M). Based on only what IC paid to the Centres, the cost per intern hired was $7,060.46, and the cost per client trained was $179.92. While it appears that the SC Program is cost-effective, particularly given the benefits outlined in the success section of the report, it is not possible to say so definitively for several reasons. Firstly, IC has not identified cost targets. Also difficulties in identifying comparable programs mean that costing information is not available for comparative purposes. As well, IC examines the costs of the program at the national level only.

Findings

The findings in this section are primarily based on a review of administrative data provided by ACCC and IC.

The following table presents the IC funding for the SC Program for each year of the evaluation period, the contributions to the Program by the Centres, and the sales revenues generated by the Centres.

Table 15: SC Funding, Contributions and Revenues, by Year
Year Funding Received from IC Cash Contribution from Centre In-Kind Contribution from Centre Sales Revenue at Centres
2004–05 $ 2,863,288.57 $ 864,103.00 $ 864,101.00 $ 806,301.23
2005–06 $ 2,871,651.95 $ 702,786.00 $ 941,328.00 $ 586,225.88
2006–07 $ 2,900,000.00 $ 757,246.00 $ 742,498.00 $ 488,577.89
Total: $ 8,634,940.52 $ 2,324,135.00 $ 2,547,927.00 $1,881,105.00
% of total funding: 56.1% 15.1% 16.6% 12.2%
% of IC funding: 26.9% 29.5% 21.8%

As illustrated by the table, there are significant contributions to the program from the participating Centres. Approximately 32% of the funding for the SC Program comes from the cash and in-kind contributions of the Centres. As indicated by the table, cash and in-kind contributions have fluctuated over the three year period covered by the evaluation. One explanation for this, suggested by IC, is that much of the cash and in-kind contributions provided by a Centre occur when the Centre is first starting up. Therefore the overall amount of cash and in-kind contributions to the Program is related to the number of new Centres joining the Program.

Approximately 12% of the funding over the evaluation period has come from revenues raised by the Centres through training fees. As indicated in question 3 above, the Centres have not met the revenue targets over the evaluation period; however, as shown in the above table, approximately 44% of the program costs have been covered by sources other than IC. Sales Revenue from the Centres has, however, fallen over the period of the evaluation, with the sales revenues in 2006–07 being just 60% of the sales revenue in the 2004–05 fiscal year. Possible reasons for this decrease were identified in question 3, and related to the emphasis placed, by both IC and the Centres, on revenue targets, and the possibility that some Centres may be providing free introductory training to promote the SC Program.

Table I10 in Annex I presents the breakdown of funding sources by Centre, for the three year evaluation period. Funding received from IC represents between 35% and 58% of total funding for the Centres. Similarly, there are wide differences in the percentage of Centres' funds that come from cash contributions from the Centres – ranging from 2% to 28% of total funding — and from in-kind contributions from the Centres – ranging from 9% to 37% of total funding. Sales revenue generated by the Centres makes up between 3% and 23% of total Centres' funding; and equal between 7% and 52% of the IC funding level.

(See Table I11 in Annex I for the breakdown of funding by Centre, by year.)

The following table presents the cost per student hired based on the IC funding allocation, and the total SC Program budget ($3.5M):

Table 16: Cost Per Student Hired
Year IC Funding # of
Interns Hired
Cost Per
Intern Hired
(Funding)
Cost Per
Intern Hired
(Program)
2004–05 $ 2,863,289 426 $ 6,721.33 $ 8,215.96
2005–06 $ 2,871,652 398 $ 7,215.21 $ 8,793.97
2006–07 $ 2,900,000 399 $ 7,268.17 $ 8,771.93
Total: $ 8,634,941 1,223 $ 7,060.46 $ 8,585.45

Over the 3 years covered by this evaluation, the average cost per intern hired based on the IC funding provided to the Centres was $7,060.46. The average cost per intern hired based on the $3.5 million program budget was $8,585.45.

Table I12 in Annex I presents the cost of the program per intern hired, by Centre, based on the IC funding allocation. When looking at only those Centres currently participating in the Program, and which have been a part of the Program for more than one year, the cost per intern hired ranges from a low of $2,991.42 to a high of $6,481.48.Footnote 16

The following table presents the cost per client trained based on the IC funding allocation and the total SC Program budget ($3.5M):

Table 17: Cost Per Client Trained
Year IC Funding* # of Clients
Trained
Cost Per Client
Trained (Funding)
Cost Per Client
Trained (Program)
2004–05 $ 2,863,289 16,296 $ 175.70 $ 214.78
2005–06 $ 2,871,652 15,060 $ 190.68 $ 232.40
2006–07 $ 2,900,000 16,637 $ 174.31 $ 210.37
Total: $ 8,634,941 47,993 $ 179.92 $ 218.78

Table I13 in Annex I presents the cost of the program per client trained, by Centre, based on the IC funding allocation. When looking at only those Centres currently participating in the Program, and which have been a part of the Program for more than one year, the cost per client trained ranges from a low of $83.31 to a high of $376.97.Footnote 17 Interestingly, the Centre with the lowest cost per intern hired (University of Calgary) is also the Centre with the highest cost per client trained. This suggests that the University of Calgary may place more emphasis on providing a beneficial work experience for students, than on providing training to SMEs and seniors.

As mentioned previously, it was very difficult to identify private sector training comparable to what is offered by SC. Introductory computer training offered by private sector firms tends to be on basic computer skills or word processing packages, and computer training offered by colleges and universities tends to be in computer programming. Due to the difficulties in identifying comparable programs, for the purposes of comparisons, benchmarking was not done for this evaluation. Similarly, targets have not been set by the Program for costs per student hired.

Finally, IC examines the costs of the program at the national level only. An investigation into costs at the Centre level, for example to understand the differences in costs per student hired and costs per client trained, by Centre, may provide insight and lessons-learned that can be used to improve the delivery and cost-effectiveness of the Program.

Recommendation

IC should investigate other programs with an aim to identifying comparable programs (if they exist) that can be used as comparators in assessing the cost-effectiveness of the SC Program. In addition, IC should establish targets relating to costs, both at the national and Centre level, and monitor costs against these targets.

In the absence of cost targets, or costing information from comparable programs, it is not possible to say definitively whether or not the SC program is cost-effective; however, given the benefits as outlined in question 4 above, IC seems to be getting significant benefits for a relatively small investment.

Q.7 Are there other more cost-effective or efficient approaches to achieving program objectives?

Conclusion

Numerous possible improvements to the SC program were identified by interviewees. However, a large majority of respondents agree that there are no other more cost-effective or efficient approaches to achieving program objectives. None of the alternative approaches presented by interviewees are viewed as more cost-effective and efficient. Also, no existing programs identified have the same range of program objectives and scope. Suggestions for improvements tended to relate to increasing funding for coordinator salaries and marketing, and increasing the allowable student training hours.

Findings

The findings in this section are based on interviews of Program staff, Centres' senior managers and coordinators, the ACCC and representatives from business organizations.

In order to evaluate alternative and/or more cost-effective approaches to SC, qualitative information collected through interviews was compiled and analyzed. All persons interviewed provided their comments regarding the following two (2) indicators:

  1. Views on other possible approaches
  2. Views on possible improvements to the SC program

When questioned, a majority of respondents were not able to identify another approach that would be capable of meeting the same objectives; especially for SMEs. Several respondents stated that they are unable to identify an alternative as flexible and affordable as the Student Connections program. Although a variety of comments were provided, the following is a list of the most frequent responses:

  • The provision of similar-type IT training by the private sector would result in high costs, often too expensive for SMEs.
  • Being profit-driven, IT firms are unlikely to concentrate on SMEs; rather they reach out to larger businesses that would generate higher revenue.
  • Private colleges offer introductory computer courses that do not provide the flexibility to customize materials and training to meet the specific needs of SMEs.
  • The SC program is the only program that provides student employment through the provision of tailored IT training to SMEs and seniors.

However, a few persons interviewed did provide potential alternatives, all the while mentioning that the mandate and/or outcomes are not identical to the SC program. Three (3) respondents compared SC to the Community Access Program (CAP) at Industry Canada. However, this program is very different from SC. CAP provides Internet access to those people, nationwide, who might not have computers or Internet access. Individuals staffing the CAP sites may provide some basic training to these users, whereas SC provides more advanced IT services and training and is aimed at SMEs. One interviewee made reference to FedNor's Youth Internship Initiative. This initiative, provides students with their first education-related job experience and, at the same time, continues to support business oriented, non-for-profit organizations. Although FedNor provides similar outcomes to the SC program, it is limited to the Northern Ontario Region and does not provide IT support and training to senior citizens.

As for training for seniors, several individuals stated that basic IT training is available to them through libraries. No other alternatives were identified for seniors.

All stakeholders interviewed provided suggestions as to potential improvements that could be made to the program. The most common suggestions (provided by more then 1/3 of respondents) are presented in the following list:

  • Increase coordinators salary to encourage stability. Additional salary dollars for the coordinators are even more crucial during the start-up period since the level of effort required to develop the program is quite substantive.
  • Improve funding for advertising and marketing, especially at the regional level (e.g. increasing the amount of information found in the IC pamphlets, dedicating a full-time student to marketing, etc)
  • Increase student terms to allow multiple year hiring. This would provide opportunity for a transfer of knowledge between experienced students and new recruits.

In addition, more isolated comments suggested a need for the following improvements to the SC program:

  • Increase or provide more flexibility to the current funding formula for Centres. Additional funding could benefit new Centres during their start-up period. The SC program is not economically self-sustaining for some of the smaller Centres; therefore a need to increase funding to subsidize the program is noteworthy.
  • Standardize and update training session curriculum in order to increase consistency in training nationwide. All materials and information should be available, in both official languages, in the content management system (Studio).
  • Lengthen the renewal period to provide opportunity for long-term planning.
  • Review the branding of the initiative.
  • Review the program's objectives. Targets should account for the quality of training and not solely on the quantity of training. Furthermore, targets should be flexible to accommodate newer Centres.
  • Network and develop national partnerships with other IC programs.

Q.8 Is it necessary for the Government of Canada (GoC) to operate this Program or would it be preferable to transfer it, or parts of it, to other levels of government or to the private or voluntary sector?

Conclusion

Based on a review of documentation and the interviews conducted, in addition to the continued relevance of the Program as established earlier, the SC Program is well-placed within the Government of Canada. IC has an objective to promote economic growth, and this Program supports that objective. As well, the positioning of this Program at the federal government level allows for national consistency and sharing of knowledge, expertise and training session curriculum across locations. Also, according to interviewees, the Program cannot be self-sustaining. Finally, interviewees suggested the private sector would not be interested in delivering the Program, and the voluntary sector lacks the capacity to be able to deliver the Program.

Findings

The findings in this section are primarily based on document reviews and interviews.

As illustrated in the relevance section of this report, there continues to be a need, by SMEs, for practical Internet and E-Commerce training; and SMEs are important to the Canadian economy. Given the IC mandate, and the relevance of the Program, as established earlier in this report, it is clear the Federal government has a role to play in providing some type of support to SMEs to gain the ICT skills and knowledge they are lacking.

The Canadian Bankers Association conducted a survey of Canadian SMEs in February 2002. Results reveal that 90% of respondents feel that the government and private sector should continue to help small business deal with E-business issues.

According to a large majority of persons interviewed, the SC Program should remain a federal government initiative. Recognizing that economic growth is an IC objective and that SC addresses this objective, all target groups interviewed agree that a link to the GoC exists. The current SC program allows for national consistency and sharing of knowledge, expertise and training session curriculum across locations, although as indicated under question 7 above, several individuals did suggest that improvements in this area could be made. Several persons also indicated the importance of program funding since the program is not self-sustaining. By operating the SC Program, a few persons interviewed stated that the Program provides visibility to the GoC, and t he GoC involvement is positively perceived as supporting communities in the regions.

Several individuals interviewed stated that transferring the program to the provincial government is not an appropriate option. They believe that the lack of national consistency would lead to a variety of individual programs as opposed to one program with common guidelines, management and administration.

When assessing various alternatives, several individuals mentioned that the SC Program should not be transferred to the private sector. Knowing that SMEs are not considered as being a lucrative market, it is fair to assume that IT professionals will not be inclined to invest both time and resources at this target group in order to generate limited profits. As well, the objectivity of private sector providers was called into question, since these companies often have a vested interest in selling their own products as opposed to guiding the SME to purchase a product that best meets the organization's needs.

All interviewed target groups stated that the voluntary sector would be unable to provide a similar service. It is believed that the high level of effort required to coordinate and administer a similar-type program would be too demanding for the volunteer sector. The potential lack in consistency and availability of services, along with the skill requirements do not lead to program delivery by the volunteer sector.

top of page

3.4 Other Issues

Q.9 Are there any unintended effects from SC?

Conclusion

Interviewees identified several unintended effects of the SC Program. These were: increased awareness of SBAs of entrepreneurial issues and the unique challenges faced by SMEs; networking amongst colleges and universities; institutions leveraging funding for other programs as a result of SC success; and the use of the SC Program to attract students to participating academic institutions.

Findings

The findings in this section are primarily based on interviews.

Several interviewees stated that, in addition to short-term employment, a portion of the students hired by the program are recruited by SMEs as a result of their work experience. However, this information was presented anecdotally with no evidence provided.

An unintended effect of the Program that was identified by interviewees was that through their work with SMEs, students acquire knowledge with regards to entrepreneurial challenges and challenges faced by SMEs.

Several interviewees mentioned, as an unintended impact, the amount of networking by Colleges and Universities that has resulted from the SC Program. As well, a few persons interviewed stated that colleges and universities have been able to leverage the SC Program's success to attract funding, from other sources, for other programs. In addition, institutions use the program in order to attract potential students.

Finally, competition with the private sector, which could have been an unintended effect of the Program, was not felt to have materialized since SC focuses on introductory ICT skills and as a result of SC training, SMEs are able to follow-up their introductory training with more advanced training offered through private sector training organizations.

Q.10 What progress has been made on issues identified in past evaluations?

Conclusion

Through information provided by ACCC and IC, it is clear that considerable efforts have been made to address the issues that were identified in the past evaluation. Of the four issues identified in the 2005–06 formative evaluation, some form of action has been taken on all four

Context

In order to assess program results and to identify any requirements for program improvement, Industry Canada retained the services of Hallux Consulting Inc. to conduct a summative evaluation of the Student Connections Program in 2004. Two years later, following changes to program delivery, this same consulting firm completed a formative evaluation of the SC Program.Footnote 18 The formative evaluation identified a total of four (4) issues that needed to be addressed by SC. According to the report, SC should:

  1. develop a contingency plan for product renewal in the event that the ACCC's current strategy fails;
  2. explore options for securing additional funding and getting greater flexibility in the terms and conditions governing youth employment in the program;
  3. examine its reporting requirements to determine if the reporting burden on AC, which increased when SC began to be delivered through a contribution agreement, can be reduced; and
  4. explore options for easing the pressures new Administrative Centres face when they join the program and must establish themselves in local markets for SC products and services.Footnote 19
Findings

The findings in this section are primarily based on discussions with ACCC and IC.

According to information received from the ACCC, numerous actions have been taken in order to address the issues raised in the past evaluations.

Issue 1
  • develop a contingency plan for product renewal in the event that the ACCC's current strategy fails.

In response to this issue, ACCC has developed a product renewal strategy to leverage the extensive knowledge that exists within Centres. As well, the ACCC developed a complete plan to accelerate the process of updating product modules, in consultation with the administrative Centres. By 2006, all E-commerce related materials and sites were updated and placed into an IT system used to track and record information. Internet training materials were transferred into the system, but they were entered "as is" (i.e., no updating of the materials was done). Finally, limited effort was placed on updating and incorporating Seniors-related information into the system, since ACCC believes that this service will diminish over time.

Issue 2
  • explore options for securing additional funding and getting greater flexibility in the terms and conditions governing youth employment in the program.

Due to the Terms and Conditions of the Program, little change can be done at the present time to funding parameters. However, SC Program staff have indicated that when they renew the program (after the one-year extension) they will be re-thinking the program in its entirety and hence its funding structure. As well, IC and ACCC have met to discuss options to provide greater flexibility for Centres in terms of the set-targets for 2008–09. Joint recommendations will be developed in the near future.

Actions which have been taken, to date, include:

  • In order to assume a portion of the administrative workload usually carried out by the Coordinators, the maximum duration of internships was increased to 1,600 hours.
  • Centres have been given the opportunity to renew a maximum of two students for a second internship. These students are valuable to the Centres since they provide new recruits with much needed guidance and mentoring. In addition, they are capable of assisting the Coordinator with various administrative tasks.
Issue 3
  • examine its reporting requirements to determine if the reporting burden on AC, which increased when SC began to be delivered through a contribution agreement, can be reduced.

In light of the above issue, a number of solutions/ actions have been completed. Specifically, changes have been made to some of the reports the Centres complete, to simplify Centres' reporting. As well several components of the Intranet site have been developed in order to facilitate data tracking.

Issue 4
  • explore options for easing the pressures new Administrative Centres face when they join the program and must establish themselves in local markets for SC products and services.

No new Centres have joined the program since the completion of the 2006 formative evaluation. However, the ACCC has initiated the following to assist in marketing the Centres in order to help Centres establish themselves in local markets:

  • Development of regional websites;
  • Development of a Web Linkage Strategy, to inform local organizations about SC, which further enable Centres to approach organizations for partnerships; and
  • Creation of the Google pay-per-click initiative.

Also, in support of the numerous new Coordinators, the following actions have been completed:

  • Delivery of two-day orientation sessions for new Coordinators;
  • Establishment of a mentorship program where new coordinators are paired with experienced and successful coordinators;
  • Centres are allowed to top-up the salary of Student Business Advisors (SBAs) who participate in coordination activities by using coordination dollars to supplement salaries.

In addition to the correctives measures that have been listed above, the ACCC has used the initial results of the formative evaluation to identify key priorities for the Centres. Although some of the priorities cannot be resolved by the ACCC (e.g. funding issues), the association recognizes the need to fulfill their requirements and to take corrective actions to respond to Centres' needs.


Footnotes

  1. 1 In 2006–07 there were 15 universities and community colleges participating in the program. (back to footnote reference 1)
  2. 5 Small business loves IT, but could be doing more, Dan McLean, Globe and Mail, October 12, 2006. (back to footnote reference 5)
  3. 6 Statistics Canada's Survey of Electronic Commerce and Technology (SECT), on which these data are based, defines small firms as having fewer than 20 employees, medium-sized firms as having between 20 and 99 employees, and large firms as having 100 employees or more for all industries except manufacturing. The upper limit for the medium-sized category in the manufacturing industry is 499 employees, while firms with 500 employees or more are defined as large. (back to footnote reference 6)
  4. 7 ic.gc.ca/eic/site/icgc.nsf/eng/h_00018.html (back to footnote reference 7)
  5. 8 ibid (back to footnote reference 8)
  6. 9 ic.gc.ca/eic/site/icgc.nsf/eng/h_00018.html (back to footnote reference 9)
  7. 10 Halabisky, David, Bryon Lee and Chris Parsley, Small Business Exporters: A Canadian Profile, Industry Canada, Small Business Policy Branch, 2005; as referenced on the Strategis site at http://www.ic.gc.ca/eic/site/sbrp-rppe.nsf/eng/h_rd01289.html (back to footnote reference 10)
  8. 11 http://www.servicecanada.gc.ca/eng/epb/yi/yep/newprog/yesprograms.shtml (back to footnote reference 11)
  9. 12 The SC Program targets for SMEs trained, training on National Products and training on E-commerce Products are stated as percentages of total clients trained. Therefore, this evaluation assesses whether or not the targets were achieved based on the actual total number of clients, in each of the three years covered by the evaluation. However, ACCC has indicated that they consider the targets to be the percentages based on the total clients target (i.e., 15,000). As a result, ACCC believes they have achieved their SME targets each year of the evaluation period. (back to footnote reference 12)
  10. 13 Cash contributions consist of money received from the Centre or host organization. In-kind contributions are other non-monetary contributions, such as office space, equipment, administrative services, etc. (back to footnote reference 13)
  11. 14 Source: http://canadaonline.about.com/library/bl/blminwage.htm. The minimum wage in Newfoundland is to rise to $7.50 on October 1, 2007; and to $8.00 on April 1, 2008. (back to footnote reference 14)
  12. 15 http://laws.justice.gc.ca/en/O-3.01/index.html (back to footnote reference 15)
  13. 16 The IC funding allocation at the Centre level includes only funding provided to the Centres for intern and coordinator salaries. Other program costs (e.g., management fees and overhead) are not included. While the salary amounts for interns and coordinators do not vary across Centres, the cost per intern hired varies due to differences in the number of hours worked by interns in the various Centres. Where interns work fewer hours, and drop out of the Program, additional interns are hired to replace them. Therefore, the number of interns hired, for the same total salary dollars, varies. (back to footnote reference 16)
  14. 17 The IC funding allocation at the Centre level includes only funding provided to the Centres for intern and coordinator salaries. Other program costs (e.g., management fees and overhead) are not included. (back to footnote reference 17)
  15. 18 Changes were made to program delivery, in that where the Program had previously been delivered through a contract with ACCC, beginning in 2003/4 the Program was delivered through a contribution agreement. As well, while the Program used existing SchoolNet funding authorities during 2003/4, it later sought, and obtained, its own funding authorities for the Program. (back to footnote reference 18)
  16. 19 Formative Evaluation of Student Connections, Hallux Consulting Inc., May 17, 2006. (back to footnote reference 19)

4.0 Conclusion

Based on the findings as outlined above, it is clear the SC Program is relevant. There is a continuing need by SMEs for a Program like the SC Program, and the Program is aligned to address IC and HRSDC priorities.

With regards to its success, the SC Program is not attaining all of its established performance targets, however, possible reasons for this have been identified by SC Program staff, and warrant further investigation. For example, the ability of Centres to meet performance targets may be affected by the Centres placement, integration and support within its host institution; as well as depending upon the emphasis placed, by the Centres, on revenues as opposed to providing experience for students. Also, following the introduction of the new content management system, confusion in classifying training may have led to underestimating of the amount of national and e-commerce training delivered. As well, the continued usefulness of some of the targets has been questioned. For example, e-commerce targets were originally developed when training most often took the form of classroom training. Now, however, e-commerce training more often takes the form of one-on-one training which impacts the ability of the Centres to meet the e-commerce training targets. The Program clearly is providing its expected benefits to SMEs, SBAs and Seniors; and the necessary mechanisms have been put in place to address the requirements of the Official Languages Act.

With regards to cost-effectiveness, while it appears that the SC Program is cost-effective, particularly given the benefits outlined in the success section of the report, it is not possible to say so definitively for several reasons. Firstly, IC has not identified cost targets. Also difficulties in identifying comparable programs mean that costing information is not available for comparative purposes. As well, IC examines the costs of the program at the national level only. While, suggested improvements to the Program were noted by interviewees, more cost-effective or efficient approaches to achieving Program objectives were not identified. Based on a review of documentation and comments from interviewees, and given the continued relevance of the Program, the SC Program is well-placed within the Government of Canada. As well, neither the private nor the voluntary sectors were viewed as viable or appropriate possibilities for delivery of the Program.

Several unintended positive effects of the Program were identified for SBAs and participating Centres. Finally, considerable efforts have been made to address issues identified in past evaluations. Of the four issues identified in the 2005–06 formative evaluation, some form of action has been taken on all four.


Management Response

Final Evaluation of the Student Connections Program (SCP) November 2007

Background

This evaluation study was undertaken by Government Consulting Services (GCS), on behalf of Audit and Evaluation Branch of Industry Canada and the Small Business Policy Branch of Industry Canada. The purpose was to provide Industry Canada with information pertaining to the relevance, the success and the value of the Student Connections Program (SCP).

The evaluation methodology included a document review, review of survey data and administrative data and interviews with 34 key stakeholders. Interview candidates were drawn from the following stakeholder groups: Industry Canada program staff, delivery organization senior officials and staff, school administrators and program coordinators. Two business organizations were also interviewed.

The following is a brief summary of the findings of the evaluation which are provided in more detail in the evaluation report:

  1. The Student Connections Program continues to be relevant and necessary.
  2. The program continues to meet its performance targets of number of clients trained and number of youth hired.
  3. The program is providing its expected benefits to its target groups.
  4. The program has produced a number of unintended effects, including increased awareness on the part of youth of entrepreneurial issues and the unique challenges faced by small and medium-sized enterprises.

Student Connections aims to help small and medium-sized businesses become more competitive in the knowledge-based economy by providing practical and affordable Internet and e-commerce training to allow them to adopt on-line business practices.

The program is funded under the Youth Employment Strategy (YES). The strategy and its programs’ terms and conditions have been extended for one year and are scheduled to be renewed before March 31, 2009. Findings of this evaluation will provide additional information that will be taken into account in the renewal.

Program management is committed to addressing each of the recommendations found within the evaluation study as set out in the detailed Management Response below. The recommendations include re-examination of some program targets and investigation of program design changes that could assist in assessing the cost-effectiveness of the program in the future.

Management Response to the Recommendations

1. Program targets

With regards to Student Connection Program targets, IC should re-examine its targets relating to:

  1. ratio of training done on national products,
  2. ratio of training on e-commerce products, and
  3. ratio of sales revenues to the IC allocation of funding, to ascertain whether these targets are still useful and valid.

If IC retains the national target relating to the ratio of training done on national products, guidance should be provided to the centres as to how to determine if training is considered national product training. If IC retains the e-commerce target, IC may wish to adjust the target to examine what percentage is Student Business Advisors (SBA) training hours are spent on e-commerce training.

Management Response : Agreed

Targets provide a useful means of measuring, in a quantifiable way, the achievement of program objectives. Student Connections Program targets are set to encourage centres to strive for meaningful results. Most of the targets have remained the same over the three years covered by the evaluation period.

Training targets relating to the ratio of training done on national products and on e-commerce products were developed as a means of ensuring emphasis on particular skill requirements. In 2004–05, centres delivered Student Connections training using paper training manuals that were difficult to update and keep current due to the rapid evolution of information technology and the need for frequent changes to the training modules. In 2005, an electronic content management system (CMS) was purchased. This enabled Centres and outside sources to update sections of the same training products faster and at a reduced cost. The content management system also allowed users to develop customized training modules by assembling sections of several different training products. These new practices may have lead to under reporting of training done on national and e-commerce products.

To ensure accurate reporting in the future, program management and the Association of Canadian Community Colleges (ACCC), the sole recipient of the contribution, will elaborate clearer parameters for centres to determine how to classify training modules when reporting types of training delivered. Program management is also reviewing the ratio for national and e-commerce training products to ensure these are appropriate, and will revise the wording of future contribution agreement to strengthen accountability for the achievement of targets.

The program target relating to ratio of sales revenues to the IC allocation of funding is primarily in place to help the recipient monitor the success of individual centres. However, other factors also influence the success of a Centre including its number of years of operation, its available market share and its ability to network with organizations that serve the small business community. Revenue generating activities vary in importance from one Centre to another. Some of the institutions sub-contracted by the recipient do not put significant emphasis on revenue generation as they see the program as an effective way to promote their institution and provide a meaningful work experience to their youth population. Program management will review this target, and if possible, will seek to incorporate other factors mentioned above into the assessment of the success of individual centres in 2008–09.

2. Cost effectiveness

IC should investigate other programs with the aim of identifying comparable programs (if they exist) that can be used as comparators in assessing the costeffectiveness of the Student Connections Program. In addition, IC should establish targets relating to costs, both at the national and Centre level, and monitor costs.

Management Response : Agreed

The evaluation report concludes that while it appears that the Student Connections Program is cost-effective in light of benefits generated, it is not possible to say so definitively in the absence of centre level cost targets and comparison to similar programs.

Program management, in conjunction with the ACCC, will establish centre level cost targets in the 2008–09 project description and will monitor them as a means of obtaining information to improve program delivery and cost-effectiveness.

The current program design makes it difficult to make cost-effectiveness comparisons with other similar programs (Student Connections is based on reimbursing student interns at an hourly rate, whereas other YES programs fund provide project funding). Should the Youth Employment Strategy be renewed beyond 2009, Program Management will ensure the re-design of the program incorporates measures to demonstrate cost-effectiveness.

Date modified: