First
Previous
Final Evaluation of the Community Access Program
4.2 Success and Progress Towards the Achievement of Objectives
This section of the report addresses issues and indicators pertaining to the achievement of program objectives. As stated earlier, the core objective of CAP has been to promote affordable public internet access, skills training and access to related services. Success (or progress) can be thought of as the extent to which the program has achieved its immediate, intermediate and ultimate outcomes.
4.2.1 To what degree have immediate outcomes of CAP been achieved?
CAP has been successful at funding and maintaining public internet access sites.
In 2004–2005, CAP's strategic direction was revised to refocus site locations on those most affected by the digital divide. The number of CAP sites was subsequently reduced from approximately 8,800 to 3,800. CAP sites have been located mostly in libraries, community centres and schools. Moreover, approximately seventy percent (70%) of CAP sites are located in rural, northern and remote communities while approximately three percent (3%) are located on First Nations reserves.
On the other hand, the most recent analysis of CAP site data (July, 2009), estimated that approximately sixty-eight percent (68%) of CAP sites are located within a twenty-five kilometer radius to other public internet access sites, while only thirty-two percent (32%) of CAP sites are not.30 Forty-two percent (42%) of CAP sites in rural areas do not have alternate public internet access within a twenty-five kilometer radius while the figure is much lower in urban areas, with only eight percent (8%) of CAP sites without alternate public internet access within a twenty-five kilometer radius. The same survey also found that approximately forty-seven percent (47%) of CAP sites have broadband connectivity (at least 1.5 Mbps) while fifty-one percent (51%) reported having high speed. The remainder, approximately three percent (3%) of CAP sites, reported offering only dial-up service.
Although CAP funding has fallen steadily from $25 million in 2004–2005 to $15.4 million in 2009–2010, surveys of CAP sites and administrative data suggests that the program has also been successful at maintaining a relatively consistent funding level per site and attracting funding from other sources. The average Industry Canada contribution per site over the 2005 to 2009 period was $4,227. Table 6 shows the average contribution by year, since 2005–2006.
| Year | 2005–2006 | 2006–2007 | 2007–2008 | 2008–2009 |
|---|---|---|---|---|
| Source: CAP Administrative Data. Note: Data on the median and ranges of IC contribution by CAP sites was unavailable at the time of the evaluation. |
||||
| Number of CAP Sites | 3,951 | 3,939 | 3,843 | 3,785 |
| Average IC Contribution | $4,306 | $3,850 | $4,944 | $3,809 |
CAP has also been successful at partnering with provincial/territorial or municipal governments and site organizations as well as non-profit/voluntary organizations to secure additional funding to operate CAP sites. According to self-reported data from CAP sites in 2008 and 2009, almost $15,000 in cash funding has been provided by other funding sources, on average, per CAP site.
CAP has demonstrated some success at providing access to demographic groups thought to be a part of the digital divide.
Digital divide groups who have tended to use CAP sites include those with low income, lower levels of education, and those without home access to the internet and the unemployed.31 A 2005 study reported that fifty percent (50%) of responding CAP sites identified serving target groups such as youth, seniors, job seekers, people with low income, women and people with limited education.32 However, analysis of the data from the 2003 survey of CAP users also suggests that the profile of CAP users differs little from the general population.
Key informants offered differing views regarding the program's success at serving digital divide groups. Almost all key informant interviewees felt that CAP has been successful in providing access to the internet to those most affected by the digital divide. Interviewees, representing external stakeholders and Industry Canada staff, agreed that CAP has contributed to addressing the digital divide by providing access to the internet and training to groups that would otherwise not have access. Specific groups mentioned by interviewees that have benefited include seniors, people with disabilities, low income Canadians, those living in rural/remote areas and First Nations communities. Although they felt CAP has been successful at providing access to the internet, a few interviewees, representing external stakeholders, noted that some communities (i.e. rural and First Nations) continue to be underserved and lag behind in terms of access. A few interviewees, representing Industry Canada staff, noted that although the program has been successful in this regard, much of the success was achieved early in the program and little new progress has been made in recent years to those most affected by the digital divide.
top of page4.2.2 To what degree have the intermediate outcomes of CAP been achieved?
While access to and use of the internet has continued to increase, attribution to CAP activities cannot be fully established.
The 2003 survey of CAP users suggested that CAP has had a positive impact on internet access and use. It reported that twenty-five percent (25%) of respondents first accessed the internet at a CAP site while forty-six percent (46%) of respondents planned to use the CAP site more often in the future. The survey found that sixty-nine percent (69%) of respondents felt the CAP site was very important. More recent data is not available as no other surveys of CAP users have been conducted.
The number of hours spent online each week by Canadians has continued to increase.
Table 7 below shows the change in online hours between 2004 and 2007 for select demographic groups. Popular internet activities have included online video, instant messaging, social networking sites, playing online games, streaming video and downloading music, among others.33 It is clear that as access to increased internet speeds has increased, internet users are increasingly using the internet for high data transfer activities such as viewing online video and playing online games.
| Demographic | 2004 Hours/Week | 2007 Hours/Week |
|---|---|---|
| Source: Canada Online 2007, Canadian Internet Project, 2008. Canadian Internet Project 2004 (n=2,168); Canadian Internet Project 2007 (n=2,098). * Differences in use between Anglophones and Francophones may, in part, be attributable to a lower amount of French language content on the internet. (Return to Table Reference 7) |
||
| Language Group* | ||
| Anglophone | 13.2 | 17.9 |
| Francophone | 10.7 | 13.7 |
| Age | ||
| 18–24 | 16.8 | 23.6 |
| 25–34 | 13.8 | 19.8 |
| 35–44 | 13.8 | 17.1 |
| 45–55 | 12.8 | 16.7 |
| 55–64 | 9.2 | 14.2 |
| 65+ | 9.0 | 9.4 |
| Education | ||
| High School Graduate or less | 11.7 | 13.7 |
| Attended College/University | 13.1 | 16.7 |
| University Degree + | 14.0 | 19.7 |
| Income | ||
| <$40,000 | 13.4 | 15.4 |
| $40,000–$59,999 | 12.6 | 16.1 |
| $60,000–$79,999 | 12.0 | 15.9 |
| $80,000+ | 14.5 | 19.9 |
What is clear from the table above is that the number of hours spent online is lower in 2007 for francophones (13.7) than Anglophones (17.9) and for high school graduate or less (13.7) compared to university degree holders (19.7). It is interesting to note that the differences between income levels is less significant (less than one hour per week) until the highest income group of $80,000+ is accounted for.
Most key informant interviewees believe that CAP has contributed to demand for IT/Internet among Canadians.
Most key informant interviewees, including external stakeholders and Industry Canada staff, believe that CAP has contributed to demand for ICT/Internet among Canadians by providing many Canadians with early exposure and training and thereby feeding demand. A few interviewees, representing external stakeholders and Industry Canada staff, noted that CAP has made more people aware of what the internet could do and thus fed interest and demand. A few additional interviewees, representing external stakeholders and Industry Canada staff, noted that CAP has resulted in an increased demand for broadband access in some communities. The Connecting Canadians: Canada's Community Access Program report contains some anecdotal evidence to suggest that CAP sites have helped increase the demand for internet service. This evidence suggests that CAP had early success with contributing to the demand for IT/Internet among Canadians.
Picton [Ontario] was also the first clear evidence that government investment would stimulate consumer demand for Internet service, and that the private sector would respond and build out the needed infrastructure. Picton had recently tried and failed to persuade telephone companies to lay in a line. That situation changed once Industry Canada announced its plans for a Picton CAP site. Even before the CAP site opened, an Internet service provider appeared. 300 Picton residents had apparently signed up for training, enough to convince the ISP [Internet Service Provider] that Picton was a market it wanted.34
Training accessed at CAP sites has emphasized internet skills development and use of a variety of online services.
Self-reported data from CAP site administrators in 2005, 2008 and 2009 have reported use of the internet at CAP sites to include internet/web searching, basic computer use, email, finding government services online, job searching, résumé preparation and academic/educational use. CAP site use has generally mirrored internet use by the general population, with general web searching and email access being the most popular types of uses. Table 8 provides results from 2008, showing the variety of training provided, as reported by CAP sites.
| Training Provided | Percentage of Sites Reporting |
|---|---|
| Source: Summary of Findings of CAP Networks and Sites for FY 07/08, Ekos Research, n=3,137. | |
| Using Internet/web searching | 97% |
| Basic Computer Use | 97% |
| Using Email | 96% |
| Finding Government Service On-line | 84% |
| Job Searching | 81% |
| Résumé Preparation | 80% |
| Academic/Education Use | 78% |
| Use of Multi-media | 66% |
| Applying for Social Benefits | 51% |
| E-banking | 44% |
| Web Design | 34% |
| Income Tax Filing | 33% |
| Creating a Business | 18% |
| Managing a Business | 16% |
4.2.3 To what degree have the ultimate outcomes of CAP been achieved?
Compared to other countries, Canada has fallen from 2nd place to 10th place in terms of broadband penetration since 2003.
This ranking is in part a reflection of the sharp increases seen in countries such as Denmark, the Netherlands, Norway, Switzerland and others. Moreover, Canada is currently ranked 25th among OECD countries in terms of advertised broadband speeds offered by service providers, with ranges from 256 kilobytes per second to 25,000 kilobytes per second. In contrast, first ranked South Korea has broadband speeds between 8,000 kilobytes per second to 100,000 kilobytes per second. A comparison of dial-up use shows that it accounts for only 0.2% of internet connections in South Korea, while in Canada it stands at seven percent (7%). By mid-2008, Canada ranked 8th in the world in terms of total broadband subscribers, at 9.2 million. Broadband is now the dominant method of internet access in OECD countries35. While the core objective of CAP has never been to explicitly expand broadband infrastructure, CAP's ultimate outcomes include strengthen ICT infrastructure, access and use.
The Atlantic Provinces, Quebec and Nunavut continue to experience lower than average internet access and use rates.
Despite increases in access to and use of the internet and efforts from programs such as CAP, some provinces and territories continue to experience lower than average internet access and use rates. Statistics Canada's 2007 Canadian Internet Use Survey found that less than seventy percent (70%) of survey respondents reported using the internet in the Atlantic Provinces and Quebec, while percentages were over seventy percent (70%) in other provinces like British Columbia (78%) and Ontario (75%). Findings from the Canadian Internet Project September 2008 report found similar levels of internet usage. The report goes on to note the following:
Lower levels of internet use in Atlantic Canada can be explained, in part, by a lag in technical infrastructure and deployment. Given its large rural territory, and as such, the difficulty in providing internet access, Quebec faces similar infrastructure issues.36
Despite statistics on access and use in the Atlantic Provinces, key informants familiar with the MOAs established with select provinces generally agreed that CAP was more successful in provinces where MOAs had been signed. This has been more so a reflection of the level of interest and engagement on the part of MOA partners prior to the establishment of MOAs, rather than a direct result of the MOAs. A few interviewees explained that CAP has been more successful in provinces where the provincial government has been engaged, specifically through the signing of MOAs. As one interviewee explained, MOAs have proven to be successful for two reasons: 1) greater commitment to the program on the part of the provincial government; and 2) more funding (i.e. provinces often match the funds the federal government is providing). At the local or community level, a few interviewees noted that the success of specific CAP sites is linked to the level of engagement of the community and volunteers. Interviewees were divided on whether CAP has been successful in rural areas with about equal numbers stating that CAP has been successful in rural areas and that CAP has not been successful in rural areas.
top of page4.3 Cost-Effectiveness
4.3.1 Is the program cost-efficient or cost-effective?
Dimensions of cost-effectiveness often found in evaluations include funds leveraged from other sources, ratio of operating expenses to grants and contributions and opinions of program representatives and stakeholders. Cost comparisons to other similar programs are often not possible as comparable options typically do not exist. Cost-benefit analysis is sometime included as a part of cost-effectiveness analysis as well. Generally, indications of cost-effective delivery of public programs include a high level of leveraged funds compared to program funds, falling or low ratios of operating expenses to grants and contributions and positive opinion from both program representatives and stakeholders regarding the cost-effectiveness of the program.
CAP appears to have been successful at leveraging funds from other sources.
Table 9 below shows the funds leveraged from provinces and territories with which the program has established Memorandums of Agreement to jointly fund CAP sites. Cash funds leveraged from these provinces and territories remained relatively constant between 2005 and 2008, at an average of $2.63 million. The province of New Brunswick significantly increased its contribution in 2008–2009 from $1 million to $3 million. Although the amounts leveraged by jurisdiction vary, the program has leveraged a total of $14,208,041 from MOA partners since 2005–2006, representing 110% of the Industry Canada contribution of $12,960,231.
| Province / Territory | Funding Leveraged By Year | |||
|---|---|---|---|---|
| 2005–2006 | 2006–2007 | 2007–2008 | 2008–2009 | |
| Source: CAP Administrative Data. * Saskatchewan contributions are in-kind. (Return to Table Reference 9) |
||||
| NL | $645,914 | $650,000 | $550,000 | $550,000 |
| PEI | $161,700 | $161,700 | $161,700 | $240,468 |
| NB | $1,000,000 | $1,000,000 | $1,000,000 | $3,000,000 |
| NS | $340,000 | $440,000 | $440,000 | $1,750,000 |
| MB | $492,000 | $300,000 | $200,000 | $470,000 |
| SK* | $- | $- | $- | $- |
| YK | $130,076 | $86,983 | $130,000 | $307,500 |
| Total | $2,769,690 | $2,638,683 | $2,481,700 | $6,317,968 |
According to analysis conducted by Ekos Research in 2008, CAP sites leveraged an average total cash funding of $14,880 from all sources, as well as $11,963 of in-kind funding in 2008. It was reported that IC CAP operational funding in 2008 represented 9% of overall cash funding provided to CAP sites. However, provinces with MOAs use some of their G&C funding towards the administration of CAP sites which could lead to the percentage represented by IC CAP operational funding to be higher than stated. The Ekos analysis also reported that forty-seven percent (47%) of a CAP site's cash funding was from a combination of municipal government and the site's host organization.
The ratio of Operating Expenses to Gs&Cs steadily declined between 2005–2006 and 2008–2009.
Table 10 shows the ratio of CAP operating expenses to Grants and Contributions (Gs&Cs) from 2005 to 2009. While Gs&Cs has fallen by eighteen percent (18%) since 2005, operating expenses have fallen by sixty-eight percent (68%). In 2005–2006, Gs&Cs totalled $17,482,323 while operating expenses totalled $5,061,000, yielding a ratio of .289. The following fiscal year saw a drop in the ratio to .206 as Gs&Cs fell to $12,247,246 while operating expenses dropped to $2,518,330. In 2007–2008, the ratio stayed constant with Gs&Cs rising to $14,588,861 while operating expenses also rose slightly to $3,000,000. Finally, 2008–2009 saw the ratio fall again as Gs&Cs stayed relatively stable at $14,417,714 but operating expenses fell to a low of $1,658,068.
| Year | Number of CAP Sites | Operating Expenses | Gs&Cs | Ratio |
|---|---|---|---|---|
| Sources: CAP Administrative Data and Industry Canada's Grants and Contributions Reporting System. | ||||
| 2005–2006 | 3,951 | $5,061,200 | $17,482,323 | .289 |
| 2006–2007 | 3,939 | $2,518,330 | $12,247,246 | .206 |
| 2007–2008 | 3,843 | $3,000,000 | $14,588,861 | .206 |
| 2008–2009 | 3,785 | $1,658,068 | $14,417,714 | .115 |
According to a 2005 cost-benefit analysis, the program's benefits slightly outweighed its costs with an estimated ratio of 1.06.
The 2005 study conducted a comparison of five IHAB programs active at the time, including the Community Access Program. The study used data collected between 1994–1995 and 2004–2005, just prior to the reduction in the number of CAP sites. The analysis framework adopted was based on cost-benefit analysis guidelines and principles as defined and outlined by the Treasury Board of Canada in their Benefit-Cost Analysis Guide. When compared to these programs, CAP showed the smallest benefit-to-cost ratio as shown in Table 11.
| Program | Present Value 2005 |
|---|---|
| Source: Bearing Point Cost Benefit Analysis of Information Highway Applications Branch Programs: Final Report, November 7, 2005. | |
| Community Access Program (CAP) | |
| Benefits | $474,260,206 |
| Costs | $447,139,823 |
| Net Benefits | $27,120,383 |
| Benefit-Cost Ratio | 1.06 |
| First Nations SchoolNet (FNS) | |
| Benefits | $57,152,570 |
| Costs | $38,945,491 |
| Net Benefits | $18,207,079 |
| Benefit-Cost Ratio | 1.47 |
| Smart Communities (SC) | |
| Benefits | $101,424,671 |
| Costs | $65,191,247 |
| Net Benefits | $36,233,424 |
| Benefit-Cost Ratio | 1.56 |
| Broadband Program (BRAND) | |
| Benefits | $87,102,534 |
| Costs | $33,871,371 |
| Net Benefits | $53,231,163 |
| Benefit-Cost Ratio | 2.57 |
| Computers For Schools (CFS) | |
| Benefits | $724,220,128 |
| Costs | $67,628,694 |
| Net Benefits | $656,591,434 |
| Benefit-Cost Ratio | 10.71 |
| Total for five programs | |
| Benefits | $1,444,160,109 |
| Costs | $652,776,626 |
| Net Benefits | $791,383,483 |
| Benefit-Cost Ratio | 2.21 |
30 2009 CAP Survey, July 2009. (Return to Reference 30).
31 Community Access Program 2003 Online User Survey, Industry Canada. (Return to Reference 31).
32 Highlights of June 2005 CAP Survey, Bearing Point. (Return to Reference 32).
33 CRTC Monitoring Report 2009, page 174. (Return to Reference 33).
34 Connecting Canadians: Canada's Community Access Program, circa 2000, page 19. (Return to Reference 34).
35 http://www.oecd.org/dataoecd/63/53/41551452.xls September 21, 2009. (Return to Reference 35).
36 Canada Online 2007, page 41. (Return to Reference 36).
- Date modified:
