Evaluation of the Radio Advisory Board of Canada Grant Program

This publication is also available online in HTML at www.ic.gc.ca/eic/site/ae-ve.nsf/eng/h_03759.html.

To obtain a copy of this publication or an alternate format (Braille, large print, etc.), please contact: The Audit and Evaluation Branch — Office of the Chief Audit Executive 343‑291‑2356.

Permission to Reproduce

Except as otherwise specifically noted, the information in this publication may be reproduced, in part or in whole and by any means, without charge or further permission from Industry Canada, provided that due diligence is exercised in ensuring the accuracy of the information reproduced; that Industry Canada is identified as the source institution; and that the reproduction is not represented as an official version of the information reproduced, nor as having been made in affiliation with, or with the endorsement of, Industry Canada.

For permission to reproduce the information in this publication for commercial purposes, please contact the:

Web Services Centre
Industry Canada
C.D. Howe Building
235 Queen Street
Ottawa, Ontario, Canada
K1A 0H5

Telephone (toll‑free in Canada): 1‑800‑328‑6189
Telephone (Ottawa): 613‑954‑5031
Fax: 613‑954‑2340
TTY (for hearing‑impaired): 1‑866‑694‑8389
Business hours: 8:30 a.m. to 5:00 p.m. (Eastern Time)

Email: ic.info-info.ic@canada.ca

© Her Majesty the Queen in Right of Canada,
represented by the Minister of Industry, 2014

PDF
Catalogue Number Iu4-162/2015E-PDF
ISBN 978-0-660-02292-5

Aussi offert en français sous le titre Évaluation du programme de subvention du Conseil consultatif canadien de la radio.

Final Report

February 2015

Presented to the Departmental Evaluation Committee on

Approved by the Deputy Minister on

Table of contents

Abbreviations Used in this Report

List of Acronyms used in this Report
Acronym Meaning
CRC Communications Research Centre Canada
CRTC Canadian Radio‑Television and Telecommunications Commission
OECD Organisation for Economic Co‑operation and Development
RABC Radio Advisory Board of Canada
SITT Spectrum, Information Technologies and Telecommunications
TBS Treasury Board Secretariat
TSAC Telecommunications Standards Advisory Committee

List of Tables

List of Tables
Table # Table Title
1 RABC Members
2 RABC Finances
3 Spectrum Consultations

List of Figures

List of Figures
Figure # Figure Title
1 Logic Model for Evaluation of the RABC Grant Program
2 Survey Responses on Questions Related to the RABC's Contribution
3 Survey Responses on Impact of Alternative Scenarios to the RABC

top of page


Executive Summary

Program Overview

Industry Canada provides a small annual grant to the Radio Advisory Board of Canada (RABC) to expert technical advice to the department on regulatory matters concerning radio spectrum management and use. Specifically, the Board brings together a wide range of radio spectrum stakeholders to provide input on federal spectrum standards, guidelines, procedures and policies. Consultation on technical standards comprises the main focus of the RABC's work.

Evaluation Purpose and Methodology

In accordance with the Treasury Board Policy on Evaluation, this evaluation assessed the relevance and performance of the RABC grant program. The evaluation findings, conclusions and recommendations are based on an analysis of multiple lines of evidence. The methodology included a document review, administrative data analysis, a literature review and website scans, a rapid impact assessment survey and interviews.

Findings

Relevance

The RABC is Industry Canada's most relevant source of expert, technical consensus advice on radio spectrum issues regarding the development of technical standards and related regulatory matters. Radio frequencies are the backbone of all radiocommunications, and their management and use are increasingly complex with rapidly evolving service demands and new technological applications. There continues to be a clear need for the federal government to receive expert stakeholder advice on regulatory matters concerning the radio spectrum.

The program is consistent with federal priorities regarding broadband access, the digital economy and having an efficient and competitive marketplace. It is in keeping with federal roles and responsibilities on two key accounts: 1) overseeing the orderly and efficient use of radiocommunications in Canada, in accordance with the Radiocommunication Act, and 2) contributing to the federal government's commitment to consult with stakeholders in establishing regulatory requirements.

Performance

The evaluation found that the RABC clearly contributes to the program's intended outcomes. For example, the RABC's advice has a direct bearing on the technical standards issued—the immediate expected outcome. The RABC's consultation process and procedures ensure Industry Canada receives representative, well‑balanced, timely expert advice from spectrum stakeholders within Canada. A critical part of this involves the RABC's outreach to include non‑members in consultations on technical standards when appropriate, in addition to its members, although this is not reflected in the grant agreement.

As a result of the RABC's efforts, Industry Canada is assured it is receiving consolidated input on technical standards from virtually all spectrum stakeholders likely to be affected. As such, the advice received carries considerable weight, with Industry Canada seldom deviating from it. The RABC's input, therefore, contributes to the intermediate outcome, which is effective and efficient radio spectrum use. It was also clearly evident that the RABC's input contributes to minimizing spectrum interference, a long‑term outcome. In addition, the Board's input is one of several factors that contributes to fair and equitable access.

The RABC provides an efficient, economical means for spectrum stakeholder consultation. The resulting value of work and advice received far exceeds the annual cost of the grant, which has remained unchanged since 2006. Altogether, the evidence indicates that the RABC is an effective model for stakeholder consultation.

In accordance with the Directive on Transfer Payments, a performance measurement strategy is required, which should be commensurate with the program's low materiality.

Conclusions and Recommendations

The RABC grant program directly affects and benefits telecommunications use within Canada, from broadcast and emergency services to fixed wireless and mobile communications. Stakeholder consultation is an important part of regulatory development, and the RABC provides a critical, efficient and economical focal point for Industry Canada to consult with spectrum stakeholders within Canada. The inclusion of non‑members in consultations on technical standards is necessary for the RABC to ensure it has as full stakeholder representation as possible and serves as a true focal point of expert technical advice for Industry Canada.

The long‑term relevance of the program has been well established by this evaluation. Given this and the small cost of the program, it would be appropriate to streamline the next evaluation to focus on performance only (contribution to intended outcomes, efficiency and economy) and omit an examination of relevance. A performance measurement strategy is required to facilitate assessment of the program's performance going forward.

Recommendation 1: In accordance with the Directive on Transfer Payments, SITT should ensure there is a performance measurement strategy for the RABC grant program, taking into consideration the low materiality of the program.

Recommendation 2: SITT should update the grant agreement to reflect the RABC's actual practice of inviting non‑members to participate in consultations when appropriate, so that the RABC's advice represents the radiocommunications community as fully as possible.

top of page


1.0 Introduction

This report presents the results of an evaluation of the relevance and the performance of the Radio Advisory Board of Canada (RABC) grant program. The evaluation complies with Treasury Board of Canada's Financial Administration Act and Policy on Evaluation regarding evaluation of Transfer Payment programs. The evaluation, covering the period 2009–10 through 2013–14, was conducted in‑house in 2014–15 by Industry Canada's Audit and Evaluation Branch, in accordance with the Departmental Evaluation Plan. The RABC grant program has not been previously evaluated.

The report is organized into four sections:

  • Section 1 provides an overview of the RABC grant program;
  • Section 2 presents the evaluation methodology;
  • Section 3 presents the findings pertaining to the evaluation issues of performance and relevance; and
  • Section 4 summarizes the study's conclusions and provides recommendations.

1.1 Program Purpose

The RABC, formed in 1944, is an advisory body to Industry Canada. Its purpose is to provide unbiased, technical engineering advice and recommendations on federal standards, guidelines, procedures and policies regarding spectrum management and use. The Board's mandate has remained essentially the same since its inception seven decades ago.

The radio spectrum serves a diverse range of needs, including emergency and security telecommunication services, broadcasting, wireless communications and so forth. Industry Canada and RABC members, along with other interested non‑member parties who participate in the RABC consultations, are the immediate stakeholders and beneficiaries of the RABC grant program.

As of 2013–14, the RABC members included 12 associations, five large corporate members and three government stakeholders, two federal and one provincial (see Table 1).Footnote 1 These members represent a broad segment of the radiocommunications industry and users of the radio spectrum in Canada. The RABC also includes non‑members in its consultations on technical standards, in an effort to ensure Industry Canada receives consolidated input from as many interested stakeholders as possible. Ten RABC member representatives, plus an ex‑officio Industry Canada representative, form the RABC's Executive Committee, which oversees the RABC's standing and ad hoc committees and supervises the administrative work.

Table 1: RABC Members as of 2013–14

Organizations
Associations
  1. Association of Public‑Safety Communications Officials, Canada
  2. Broadcasters' Technical Advisory Committee
  3. Canadian Association of Chiefs of Police
  4. Canadian Association of Broadcast Consultants
  5. Canadian Wireless Telecommunications Association
  6. Canadian Electricity Association
  7. Canadian Satellite and Space Industry Forum
  8. Electro‑Federation of Canada
  9. Model Aeronautics Association of Canada
  10. Radio Amateurs of Canada
  11. Railway Association of Canada
  12. Utilities Telecom Council of Canada
Corporations
  1. Bell Telecom Group
  2. Canadian Broadcasting Corporation
  3. NAV Canada
  4. Rogers Wireless
  5. TELUS
Government
  1. Government of Ontario, Solicitor General, Ontario Provincial Police
  2. Department of National Defence
  3. Royal Canadian Mounted Police

1.2 Program Resources

Industry Canada provides an annual grant to the RABC. Board membership fees represent the other main source of funding. In the years in which the Board has held a conference (e.g., Spectrum 20/20), events represented a third source of revenues. The following table presents the Board's revenue sources and total expenditures for the five‑year period evaluated.

Table 2: RABC Finances
Revenue Sources 2009–10 2010–11 2011–12 2012–13 2013–14
Industry Canada Grant 85,000 85,000 85,000 85,000 85,000
Membership Fees 79,200 74,950 76,750 76,750 82,250
Events 59,145 40,710 60,589 ­ ­
Scholarship Grant 5,000 4,000 ­ ­ ­
Interest 906 1,283 2,183 2,313 2,332
Other ­ 1,460 ­ ­ ­
Total Revenues 229,251 207,403 224,522 164,063 172,582
Total Expenditures 222,532 199,413 192,656 164,296 141,310
Surplus (Deficit) 6,719 7,990 31,866 (233) 31,272

Source: RABC Annual General Meeting document, (2009–10 to 2012–13); RABC 2013–14

The RABC has two employees who manage the organization, while members undertake consultation work at their own expense. Their contribution represents on average about 2,000 hours annually, plus there are travel expenses related to attending some of the meetings. In addition to the grant, Industry Canada invests staff time and related travel costs to attend RABC meetings and engage in the ongoing work of the RABC's various working groups.

top of page

1.3 Program Design and Delivery

The grant agreement with Industry Canada outlines three primary goals for the RABC:

  1. To provide technical advice on standards, guidelines, procedures and policies to the federal government relating to spectrum management and usage and the administration of national radiocommunications.
  2. To facilitate consensus views across all of the Canadian radio communication industry.
  3. To have a membership that fully represents the radiocommunications community in Canada, including broadcasters, carriers and commercial service providers, experimenters, manufacturers, national security and public safety agencies, public utilities and the scientific communities.

The federal grant is used to assist in attaining these goals. As stipulated in the agreement, the grant is to pay, in part, for office rental and equipment, staff salary and support for technical committee advisory meetings. The Board provides advice primarily on technical standards and to a more limited extent on proposed policies and regulations. The process for advising on each differs, as described below.

Technical Standards Consultation

Industry Canada's Spectrum, Information Technologies and Telecommunications Sector (SITT) directly requests RABC advice on technical standards, notably its Radio Standards Specifications (of direct interest to manufacturers) and Standard Radio System Plans (of direct interest to operators). SITT provides its draft technical standards to the RABC for discussion and response.

This is the main focus of the RABC's work, which is facilitated by the Board's four technical committees: Broadcasting, Electromagnetic Compatibility, Fixed Wireless Communications, and Mobile and Personal Communications. Sub‑committees and working groups further support committee work. Industry Canada representatives attend these meetings to provide information as necessary and to understand stakeholder perspectives. Non‑members are invited to participate when appropriate, in an effort to ensure Industry Canada benefits from the combined input of all affected stakeholders.

The RABC submits to Industry Canada revised proposed standards that have been arrived at by consensus among all stakeholders. Where there are differences of opinion on given points, the RABC submits the differing opinions.

Industry Canada may or may not make subsequent changes to the proposed standards before formally issuing finalized technical requirements through the Canada Gazette. This process does not preclude any member or other stakeholder discussing points of specific interest to them directly with Industry Canada.

Public Consultation on Policies and Regulations

The Board responds at times, on its own initiative, to public consultations on draft spectrum policies and regulations undertaken via the Canada Gazette. In these instances, the RABC's Executive Committee and its General Manager determine which of these postings warrant the Board's attention, as not all postings are suitable for the RABC's consideration. For example, the RABC does not participate in consultations concerning spectrum licences. Industry Canada representatives do not participate in these meetings on draft policies and regulations, nor do non‑members given that this is a public consultation process, whereas consultation on technical standards is via a request directly from Industry Canada to the RABC (i.e. not a public posting).

A Board committee develops the proposed response, which is then submitted to the Executive Committee for consideration and voting. RABC members formally vote by ballot on proposed submissions (approve, approve with comment, abstain, disapprove, or disapprove with comment). Ballot results are included in the RABC's Annual General Meeting documentation, with each member's vote shown separately. Only those proposals that are supported by a majority of RABC members are submitted to Industry Canada. A submission from the RABC does not preclude RABC members, and the organizations and individuals they represent, from making their own submissions.

Other RABC Consultation

From time to time, the RABC initiates consultation on its own with its members, generally to support engineering regulations, standards and plans. Information to industry at large is normally provided through Board publications posted on the RABC's website. For example, technical studies may be issued to explain why the RABC supports certain government standards. At times, Industry Canada also requests informal advice from the RABC on spectrum matters other than on specific technical standards.

1.4 Expected Results

Program inputs, activities, outputs and outcomes are presented in Figure 1. Although the grant program is longstanding, no measurement strategy or logic model had been put in place for the grant program prior to this evaluation. Therefore, the Evaluation Directorate created a logic model and identified performance indicators for the evaluation which were discussed and approved by Industry Canada program staff and the RABC.

In the logic model, 'requirements' is used to refer to polices, regulations and technical standards collectively, inasmuch as these items stipulate requirements to be followed. In the Activities and Outputs section, the activity boxes are indicated by asterisks. The other boxes are outputs.

Figure 1: Logic Model for Evaluation of the RABC Grant Program

Image of the Logic Model for Evaluation of the RABC Grant Program (the long description is located below the image)
Description of Figure 1

The logic model presented in figure 1 identifies inputs, activities and outputs, and expected outcomes. There are three inputs as follows: Industry Canada Engineering, Planning and Standards personnel; Industry Canada grant for RABC operations; and RABC members. The activities and outputs are as follows: Industry Canada drafts requirements and guidelines which leads to the activity of RABC discussions and the subsequent output of consolidated advice for Industry Canada. This output feeds into the activity "Industry Canada finalizes requirements and guidelines", which results in the output of federal requirements and guidelines. There are two intended immediate outcomes, one being a streamlined, efficient regulatory review process, and the other is spectrum requirements and guidelines based on consultation with the radiocommunications community. These in turn lead to the intended intermediate outcome, which is the RABC contributes to a regulatory environment that supports effective and efficient radio spectrum use within Canada. There are two long-term intended outcomes: first, fair and equitable access to the radio spectrum within Canada; and second, spectrum interference is minimized. Finally, the program is intended to align with Industry Canada's strategic outcome of an effective and competitive Canadian marketplace.

top of page


2.0 Methodology

This section covers the evaluation approach, objective and scope, core issues addressed, data collection methods, and data limitations for the evaluation.

2.1 Evaluation Approach and Design

The evaluation assessed the program from a contribution analysis perspective, the purpose being to form conclusions about the program's impact on the regulatory environment for spectrum management and use. Given the low materiality of the grant expenditure, the evaluation focused on the usefulness of the grant to Industry Canada. The evaluation used a logic model and multiple lines of evidence, drawing on both qualitative and quantitative data, to ensure an appropriate level of rigour necessary to arrive at valid findings and conclusions.

2.2 Objective and Scope

The evaluation assessed the relevance and performance of the RABC grant program in accordance with Treasury Board's Policy on Evaluation. Given an annual grant of $85,000, the evaluation was calibrated for low materiality and low program risk and hence is modest in design. At the same time, the evaluation needed to be sufficiently rigorous to arrive at valid findings and conclusions, for example, by triangulating multiple lines of evidence.

2.3 Evaluation Core Issues

The evaluation addressed the following questions on relevance and performance:

Relevance

  1. Is there a continued need for Industry Canada to receive expert advice from RABC?
  2. Does the RABC grant program align with federal government priorities and the strategic outcomes of Industry Canada?
  3. Does the program align with federal government roles and responsibilities?

Performance

  1. Has the program ensured that the views of Canada's radiocommunications community are represented and that balanced advice is provided to IC?
  2. To what extent does the RABC's input have an impact on Industry Canada spectrum policies, technical rules, regulations and guidelines?
  3. Does the RABC contribute to a regulatory environment that supports effective and efficient radio spectrum use within Canada?
  4. Does the consultation process contribute to fair and equitable access to the radio spectrum within Canada and to minimizing spectrum interference?
  5. To what extent has the RABC grant program demonstrated efficiency and economy?

2.4 Data Collection Methods

The evaluation used multiple lines of evidence, described below, to gather quantitative and qualitative data to address all evaluation questions.

Document Review

The document review afforded an understanding of the program and contributed to addressing both relevance and performance questions. Key documents and website material included RABC and related Industry Canada material (e.g., grant agreement, RABC website, SITT and Engineering, Planning and Standards presentations, Departmental Performance Reports, Reports on Plans and Priorities) and federal documents (e.g., federal budgets, Speech from the Throne, legislation, policies and Treasury Board Secretariat documents).

Administrative Data

Administrative data and related analyses contributed to addressing effectiveness and efficiency questions. The evaluation used financial data from the RABC's 2013 Auditor's report and RABC estimates of members' time devoted to consultations in 2012 and 2013. Statistics on spectrum consultations via the Canada Gazette and technical standards issued from 2009–10 through 2013–14 were compiled based on postings on Industry Canada's Spectrum Management and Telecommunications webpages. Input from SITT staff was used to verify the statistics.

Literature Review and Website Scans

This input contributed to addressing relevance questions, specifically continuing need, and federal roles and responsibilities. It also contributed to assessing efficiency and economy, specifically whether there are alternatives that should be considered by providing insight on consultation best practices and spectrum regulatory consultation processes used in some other jurisdictions.

Rapid Impact Assessment Survey

Using a survey, subject matter experts were asked to assess the program's effectiveness and two plausible alternative scenarios. Unlike surveys that aim to collect data from a large number of respondents, rapid impact assessment surveys are used to assess overall program impact as well as the impact of counterfactual scenarios with input from a limited number of subject matter experts. Thirty‑two RABC and SITT respondents completed the survey resulting in an 86% response rate (37 invited). This information contributed to assessing the program's effectiveness and the likely impact of the two alternative scenarios that could be employed in the absence of the RABC.

Interviews

Interviews provided insight into the program's relevance and performance not otherwise available, by drawing on input from people knowledgeable about the program. Sixteen interviews were held, either in‑person or by phone, with program staff, RABC representatives and a few other external stakeholders: SITT (4), RABC (9), non‑RABC representatives (2) and the United States' Federal Communications Commission (1).

2.5 Limitations and Mitigation Strategies

The key evaluation design challenge involved ensuring sufficient methodological rigour while keeping the scope of evaluation effort commensurate with the small size of the program relative to other departmental programs. This challenge was addressed by employing a rapid impact assessment which draws on the insight of subject experts. The approach is advantageous in that it is fast and cost effective without compromising rigour or validity.

Interviews present the possibility of bias on the part of interviewees, interviewers or both, which could skew findings. Given the small size of the RABC grant program, it was appropriate to limit the number of interviews, although bias can be more problematic. Use of an interview guide developed in accordance with the evaluation matrix, interviews with both program staff and external stakeholders, and triangulation of interview findings with other lines of evidence combined to counter any bias effect.

top of page


3.0 Findings

3.1 Relevance

3.1.1 Is there a continued need for Industry Canada to receive expert advice from the RABC?

Key Findings: Industry Canada needs to continue receiving expert technical advice. The regulatory environment has a direct bearing on radio spectrum use, the spectrum being the basis for all wireless telecommunications. The RABC is Industry Canada's most relevant source of technical consensus advice on spectrum issues and is the only existing mechanism that provides a means for consulting with a diverse group of spectrum stakeholders.
Radio Spectrum Use is Essential and Rapidly Changing

All forms of wireless telecommunications use radio spectrum frequencies. As noted in SITT and Communications Research Centre (CRC) documents and presentations, the digital economy, WiFi and the rapid proliferation of smartphones, tablets and other wireless devices, as well as other technological developments, all contribute to a huge demand on spectrum use. As technology continues to evolve rapidly, user capabilities and services are changing. For example, machine‑to‑machine (M2M) communications and fifth generation (5G) of mobile broadband technology are expected to become mainstream in the 2020s. Such rapid technological development and the growing global digital economy require significant changes in spectrum use.

Spectrum Use Requires Regulation

Departmental documents on spectrum use highlight the need for a well‑managed regulatory environment. A 2014 presentation by SITT's Engineering, Planning and Standards Branch states that radiocommunication regulations are necessary to avoid two critical situations: 1) overloaded frequencies and severe interference; and 2) harm to people (i.e., radio frequency radiation exposure by electronic devices). In addition, regulations and standards are needed to have a well‑functioning market. A 2011 Branch presentation, "Radio Spectrum 101", notes that spectrum management is grounded in technical rules "because the laws of physics first dictate spectrum sharing conditions". In short, the spectrum is a finite resource with technical limitations. Canada's regulatory framework governing spectrum use must also take into account the fact that Canada is part of a larger North American market and global markets. According to an OECD May 2014 report, New Approaches to Spectrum Management, regulators face considerable work to keep spectrum policies and regulations up to date.

Expedient Focal Point for Technical Advice

Interview respondents noted that technical standards need to be continually updated. This is driven by increasing changes in telecommunications and the need to harmonize radio spectrum use in North America, to allow the telecommunications market to thrive. They see the RABC as a forum through which Industry Canada obtains unbiased, consensus advice on technical standards. The RABC provides a means for stakeholders, both RABC members and non‑members, to discuss their interests with a view to providing a consensus position on proposed technical standards. The process enables competitors and different stakeholder groups to understand each other's position and clarify where there is agreement and where opinions differ.

Without the RABC, Industry Canada would have to consult stakeholders individually and deal with multiple points of view. As a few respondents pointed out, the RABC was specifically created by Industry Canada to inform its regulatory responsibilities.Footnote 2 Many respondents emphasized that the RABC is the quickest and most efficient means for Industry Canada to consult and obtain a consolidated response from a broad and diverse group of stakeholders.

Interview responses from all stakeholders confirm that the RABC is the most relevant source of technical advice and, indeed, a focal point for consolidating stakeholder advice and feedback to Industry Canada. The first order of business for each RABC consultation is to determine if everyone needed is at the table. If not, the RABC invites non‑members to attend meetings, possibly small industry players, non‑member government entities or key stakeholders from outside Canada. These organizations participate and contribute to the discussions, but they do not vote on any issues put to ballot.

Respondents identified a few other sources of technical advice available to Industry Canada, for example, the policies and standards of other countries such as the United States or other organizations such as the International Telecommunication Union. Technical consultants and retired experts might be other sources, as well as organizations such as the Information Technology Association of Canada and the Canadian Advanced Technology Alliance. According to respondents, these sources provide limited points of view.

SITT material identifies several partners that Industry Canada works with to foster a reliable and resilient telecommunications network. These include, for example, other federal government departments; two other advisory bodies‒the ICT Standards Advisory Council of Canada, and the Canadian Security Telecommunications Advisory Committee; research institutions; and various individual industry organizations. However, according to interviewees, the RABC is the only advisory body providing consolidated, consensus‑driven technical advice on the radio spectrum. This consensus input, identified as a goal in the grant agreement, is a significant and distinguishing feature of the RABC.

3.1.2 Does the RABC grant program align with federal government priorities and the strategic outcomes of Industry Canada?

Key Findings: The program aligns with federal government priorities to strengthen Canada's digital economy and to contribute to a well‑functioning radiocommunications environment. Spectrum regulations align with Industry Canada's strategic outcome of an efficient and competitive marketplace.
RABC Aligns with Federal Priorities

The federal government's Digital Canada 150 initiative, launched in April 2014, is a reflection of the importance the government places on digital technologies and their role in people's lives, businesses, the economy, employment and distance education. As stated in Industry Canada 2014 news releases, the intent of Digital Canada 150 is to "strengthen Canada's digital economy" and "position Canada as a leader in the global digital economy". Although technical standards are not mentioned, they have an impact on spectrum use.

The government also underscores in key documents, such as the 2013 Speech from the Throne and its 2013 and 2014 national budget plans, that access to broadband networks is an important part of Canadian daily life. The government in recent years has committed funds to enhance high‑speed Internet access for First Nations, rural and Northern communities. Such access is yet another example of increasing demand on spectrum use. The government also emphasized the importance of guarding against spectrum interference in its 2014 budget plan, indicating it would introduce legislative amendments to clarify "prohibitions against the manufacture, sale or use of jamming devices (e.g. devices that block cell phone transmissions by creating interference), to better protect Canadians."

SITT interviewees and a Cabinet directive reinforce the fact that the RABC grant aligns with federal priorities. The interviewees noted that the RABC's work contributes to a well‑functioning radiocommunications environment and supports job creation. The Cabinet directive speaks to the regulatory development process. Specifically, the 2007 Cabinet Directive on Streamlining Regulation requires federal regulators to develop regulations that, among other requirements, promote a fair and competitive market economy and ones that are based on inclusiveness, transparency, accountability and public scrutiny. This directive remains central to federal regulatory development, given its prominence in the Treasury Board Secretariat's current Guide to the Federal Regulatory Development Process.

Aligned with Industry Canada Priorities

As noted in Departmental Performance Reports, advancing the marketplace is a top priority for Industry Canada. The 2009–10 report notes that the department continued to strengthen the regulatory environment necessary to support a digital economy, as part of efforts under the strategic outcome of an efficient and competitive marketplace. Spectrum regulations were identified as one of four corporate risks in the 2011–12 report, given the challenges associated with addressing the continuing evolution of telecommunications and a wireless infrastructure in Canada. The reports note the increasing demand for mobile services, with access to spectrum of critical importance. The department's 2013–14 Report on Plans and Priorities indicates that the department will continue to develop spectrum standards in partnership with the RABC.

SITT's mandate is to accelerate Canada's economic growth and innovation through the development and use of information and communication technologies. SITT relies on the RABC's input to assist in developing regulations in support of this. Part of SITT's mission is to maximize economic and social benefits derived from radio spectrum use. These in turn align with Industry Canada's strategic outcome of an efficient and competitive Canadian marketplace.

top of page

3.1.3 Does the program align with federal government roles and responsibilities?

Key Findings: The grant to the RABC is directly aligned with ministerial powers delegated to the Minister of Industry. Stakeholder consultation is an integral part of regulatory development.
Departmental Responsibility and Ministerial Powers

Industry Canada is the federal department responsible for regulating radiocommunications, which includes developing technical standards. The Minister of Industry is delegated this responsibility through the Radiocommunication Act (R.S.C., 1985, c. R‑2), s. 5 (1), which stipulates that ministerial powers include, among other things, ensuring "the orderly development and efficient operation of radiocommunication in Canada."

Obligation to Consult Stakeholders

Stakeholder consultation as part of regulatory development is a cornerstone of good governance according to OECD documents. It is the only means by which policy makers can truly understand and assess the potential impact of proposed regulations. As a key factor contributing to effective regulations, the process should be as inclusive as possible while being neither complex nor costly. Advisory bodies are the most widespread form of public consultation among OECD countries. Circulation of regulatory proposals for public comment is another common form of consultation within OECD countries, and public notice and comment (e.g., the Canada Gazette process) has become more widespread in recent years.

A few interviewees thought that the government has an obligation in accordance with legislation to consult with stakeholders and to be fair and transparent. A review of the Department of Industry Act (S.C. 1995, c. 1), the Radiocommunication Act (R.S.C., 1985, c. R‑2) and the Telecommunications Act (S.C. 1993, c. 38) revealed no requirement for stakeholder consultation relevant to the RABC's work. However, there is an obligation for federal departments to consult stakeholders. A November 2007 Treasury Board Secretariat (TBS) presentation, Applying the Federal Cabinet Directive on Streamlining Regulation, notes that the Cabinet Directive  requires federal regulators to address, among other factors, the following aspects as part of their regulatory development process:

  • Consider the views of stakeholders and Canadians;
  • Understand competitiveness impacts; and
  • Minimize the burden on business.

The Cabinet Directive specifically states under "Regulatory Impact Analysis, (A) Consultation", that "Publishing proposed regulations in the Canada Gazette is not a substitute for meaningful consultations on the development of regulatory proposals."

The above requirements are reiterated in TBS' Guide to the Federal Regulatory Development Process, under the heading "The Importance of Consultations". Those who may be interested in or affected by a proposed regulation must have the opportunity to participate in meaningful consultations.

3.2 Performance

3.2.1 Has the program ensured that the views of Canada's radiocommunications community are represented and that balanced advice is provided to Industry Canada?

Key Findings: RABC consultations are representative of Canada's radiocommunications community, involving both members and non‑members, and RABC procedures ensure both balanced and timely advice to Industry Canada. Although not reflected in the grant agreement, inclusion of non‑members in RABC consultations is a critical component in achieving as full representation of stakeholders as possible.
Representation

One of the RABC's goals, according to the grant agreement, is to develop and maintain a membership that "fully represents the radiocommunications community in Canada". This is to include broadcasters, carriers, commercial service providers, experimenters, manufacturers, national security and public safety agencies, public utilities and the scientific community. The RABC's membership of 20 organizations suggests it has achieved this to a considerable extent, although not fully according to other findings.

There are other stakeholders who participate from time to time in consultations but who are not looking to join the RABC or one of its member associations. Some interviewees noted that small organizations do not always have the resources to afford RABC membership, which is currently $4,250. Other spectrum stakeholders may have a limited range of interest in spectrum matters and only wish to participate infrequently. The RABC ensures all stakeholder views are captured by inviting non‑members to participate, sometimes at the suggestion of Industry Canada.

Survey respondents assessed various aspects of the RABC's importance and contribution to an effective and efficient regulatory review process, as shown in Figure 2. Almost all survey respondents (97%) consider the RABC to be an important or extremely important part of the regulatory review process for obtaining high‑quality input. All respondents consider it to be important or extremely important for obtaining input that is representative of radiocommunication stakeholders. Three‑quarters (74%) consider the RABC to be effective to a great extent in obtaining both high‑quality and representative input. These findings were reinforced in the interviews.

Both SITT and RABC interview respondents see the RABC as adequately representing the diverse range of radio communication stakeholders in Canada and being as inclusive as possible. As well, the interviews and document review indicate active involvement of RABC members in the consultations. There may be five to 50 or even 80 people involved in a working group. While some Board members are more engaged than others, and some (such as public safety organizations) may only be active on certain issues, all respondents thought that everyone takes these consultations quite seriously.

Figure 2a: Survey Responses on Questions Related to the RABC's Contribution
How important is the RBAC to the following?
Bar chart of Survey Responses on Questions Related to the RABC's Contribution - How important is RBAC to the following? (the long description is located below the image)
Description of Figure 2a
Figure 2a. Survey Responses on Questions Related to the RABC's Contribution
How important is the RABC to the following? (%)
Factors Extremely important Important Moderately important Of little importance Unimportant
Representative input 69 31 - - -
High-quality input 69 28 - - 3
Minimizes spectrum interference 69 19 9 3  
Effective, efficient spectrum use 63 28 9 - -
Minimizes time required 59 31 9 - -
Minimizes workload 53 38 9 - -
Fair, equitable access 38 34 19 6 -
Figure 2b: Survey Responses on Questions Related to the RABC's Contribution
How effective is RABC in contributing to the following?
Bar chart of Survey Responses on Questions Related to the RABC's Contribution - How effective is RBAC in contributing to the following? (the long description is located below the image)
Description of Figure 2b
Figure 2b. Survey Responses on Questions Related to the RABC's Contribution
How effective is the RABC in contributing to the following? (%)
Factors To a great extent To a moderate extent To a small extent To a very small extent Not at all
High-quality input 74 23 3 - -
Representative input 74 19 3 3 -
Minimizes time 68 26 7    
Minimizes spectrum interference 68 23 7 - 3
Effective, efficient spectrum use 68 19 10 - -
Minimizes workload 58 29 7 3 -
Fair, equitable access 32 36 23 7 -

These two graphs provide a comparison of the importance survey respondents place on various aspects of the RABC's work and how effective, in their opinion, the RABC is in achieving them.

These findings indicate that the RABC is effective in satisfying the intent of the grant agreement for full representation of Canada's radiocommunications community. Such full representation, though, is not achieved exclusively through RABC's membership. It is only realized as a result of the RABC's practice of involving non‑members in the consultation process. The need to involve non‑members to achieve as full representation as possible is not reflected in the grant agreement.

The groups that appear to be missing in consultations from those listed in the grant agreement are experimenters, national security agencies, such as the Canadian Security Intelligence Service or the Communications Security Establishment, and perhaps parts of the scientific community, such as academics, although 'scientific community' is not defined. A few interviewees spoke to this. Industry Canada directs innovators to the RABC as appropriate, but experimenters as a group are not focused on regulatory issues. SITT obtains input from a separate security advisory group, as security standards are not part of spectrum management and discussions are sensitive in nature. Academics have participated in RABC Spectrum 20/20 events, but they do not participate in the consultations.

Balanced Advice

Industry Canada and RABC stakeholders indicated that the RABC provides balanced advice on proposed technical standards, as it brings together competing interests and seeks to arrive at a consensus position. Respondents reported that when there are dissenting views, these are identified along with the rationale and provided to Industry Canada. In this way, Industry Canada receives consensus and consolidated input. When there is no agreement, which occurs infrequently, decisions are not forced and no response is provided.

When the RABC responds to public consultations issued via the Canada Gazette, the balloting process ensures that the RABC's response is supported by the majority of its members. The RABC responded to three public consultations in 2013, for example, and the vote cast by each member is recorded in the RABC's 2013 Annual General Meeting document.

top of page

3.2.2 To what extent does the RABC's input have an impact on Industry Canada's spectrum policies, regulations, technical standards and guidelines?

Key Findings: The RABC's input has considerable impact on spectrum technical standards, as well as on policies and regulations. The Board's advice carries considerable weight, with Industry Canada usually accepting it without the need for further revision.

Based on Industry Canada data shown in the table below, the RABC provided input on 53 technical standards issued during the five‑year period 2009–10 through 2013–14. It also provided input on 15 public consultations concerning spectrum policy and regulations. According to both SITT and RABC interviewees, the RABC's input carries considerable weight. It helps Industry Canada make decisions reflecting what a majority of stakeholders think, having a definite impact on spectrum technical standards, policy and regulations.

Table 3. Spectrum Consultations
2009–10 2010–11 2011–12 2012–13 2013–14 Totals
Consultations with RABC: Technical Standards
# Technical Standards Issued 18 13 4 8 10 53
Public Consultations via Canada Gazette: Policies and Regulations
# Consultations Initiated 7 5 4 9 4 29
# RABC Submissions 4 4 2 3 2 15

The RABC submits revised drafts of proposed technical standards to Industry Canada. The input not only reveals points of agreement but also more difficult or controversial aspects, important information in developing technical standards. RABC members are willing to pay dues and participate because they see the impact they have through the RABC reflected in published standards. The Board's input assures that much of the necessary research for appropriate technical standards has been done. According to one respondent, RABC members would be able to tell if the department chose not to follow the Board's advice, and the respondent had not seen any indication of this. An Industry Canada respondent likewise said they seldom deviate from the RABC's advice.

The RABC responds to Industry Canada public consultations issued via the Canada Gazette only if its members can arrive at a consensus on some aspect of the proposed policy or regulations. For Industry Canada, a broad perspective on policies and regulations is as important as for technical issues. The department values this input, in addition to that for technical standards, because it highlights common points of agreement otherwise not available to Industry Canada.

3.2.3 Does the RABC contribute to a regulatory environment that supports effective and efficient radio spectrum use within Canada?

Key Findings: While Industry Canada is ultimately responsible for a regulatory environment that supports effective and efficient radio use within Canada, the RABC is a positive contributor to this outcome through its expertise and advice.

Both survey and interview responses confirm the RABC's role with respect to effective and efficient spectrum use. Most (91%) survey respondents consider the RABC to be important or extremely important in ensuring effective and efficient spectrum use. Two‑thirds (68%) see the RABC as contributing to this to a great extent and 19% to a moderate extent. As noted in the interviews, the RABC's role is to recommend changes needed for effective and efficient use. Technical standards define such aspects as, among many others, maximum power levels, energy levels transmitted outside a band and types of antenna to be used.

A couple of interviewees identified specific ways in which the RABC contributes to this outcome. One noted that industry tends to be ahead of government with respect to emerging technology and evolving spectrum use. Industry is also aware of what is happening in American and Asian markets, which tend to be ahead of Canada. Sharing this technical knowledge with the department contributes to effective and efficient spectrum use. Another said that the RABC's advice often mentions the possibilities of ineffective or inefficient spectrum use. Inappropriate technical rules would result in inefficient use, cause interference problems or have an impact on industry costs. The RABC can have a direct impact on decisions made by Industry Canada that in turn have a big impact on Canadians and the economy.

3.2.4 Does the consultation process contribute to fair and equitable access to the radio spectrum within Canada and to minimizing spectrum interference?

Key Findings: Consultation with the RABC contributes to minimizing spectrum interference. It also is one of several factors contributing to fair and equitable access.

Close to three‑quarters (72%) of survey respondents consider it important or extremely important that the RABC contribute to fair and equitable access. A third (32%) see the RABC as achieving this to a great extent, and another third (36%) to a moderate extent. Interviewees noted that such factors as policy, spectrum auctions, licencing, enforcement and CRTC's role (with respect to broadcasting) have considerable impact on fair and equitable access.

SITT interviewees noted that the RABC contributes to fair and equitable access by having a very transparent process that gives all stakeholders a chance to provide input. This prevents unfair advantages and results in broad, unbiased, balanced advice to Industry Canada. Without the RABC, it would be easy for technical standards to favour one company over another.

Survey and interview responses indicate the RABC has more of a role in minimizing spectrum interference. Most (88%) survey respondents said the RABC is important in contributing to this, and two‑thirds indicated it was effective in achieving it to a great extent. According to all interviewees, the RABC plays a clear role in minimizing interference, a subject that preoccupies its members. Discussions often address how well a system will reduce interference and allow equitable access. Pre‑empting interference can be very difficult technically, but if interference is not managed it reduces spectrum quality, and maximizing spectrum use depends on minimizing interference. Policies and standards need to be geared for interference‑free spectrum use, which is part of having fair access to the spectrum. Program management observed, though, that it would be difficult to prove that a low number of interference cases was a direct result of the RABC's input.

3.2.5 To what extent has the RABC grant program demonstrated efficiency and economy?

Key Findings: Stakeholder consultation through the RABC is efficient and economical, although there are some suggestions for program improvement. The value received greatly exceeds the cost of the grant. SITT needs to ensure that a performance measurement strategy commensurate with the low materiality of the program is in place to facilitate future assessment of the program's performance.
Efficient and Economical

The evaluation looked at whether the RABC streamlines engagement government and RABC members. The majority of interviewees noted that the RABC streamlines the process for both partners. Industry Canada benefits because RABC stakeholders often conduct research related to proposed spectrum standards that the department would otherwise have to undertake by engaging consultants or hiring more engineering staff. RABC members benefit from an efficient consultation process that costs them less time and money than if they had to develop responses on an individual basis.

Most (91%) survey respondents consider the RABC to be extremely important or important in minimizing their organization's workload when reviewing proposed regulatory requirements and providing advice. Over half (58%) indicated that the RABC is effective in minimizing workload to a great extent and 29% indicated it is effective in this to a moderate extent. Unsolicited comments from a few respondents emphasize the view that the RABC saves both industry and the department time and money.

The RABC grant is an efficient means of leveraging considerable effort on the part of radio spectrum stakeholders. Findings from the administrative data review show that RABC members together contributed an average of 1,977 hour annually, or an average of 13.2 days per RABC member organization (based on 20 members). The survey results indicated that a median of two senior staff and one junior staff person from each organization are involved in a typical year in preparing for and attending RABC consultations and contribute roughly 12 to 20 days annually. However, some organizations contribute much more. For example, one company dedicates 1.5 full‑time employees solely to RABC work, and one of the smaller association members has a dedicated full‑time volunteer regulatory affairs officer. Some companies might devote additional time to specific items of interest to them. Through the survey, we obtained some estimates of the dollar value of senior and junior staff time devoted to RABC work. The combined total amounts to close to $1.1 million annually. This is only a fraction of the total value of effort leveraged, as RABC members collectively represent hundreds of organizations and individuals.

Most (90%) survey respondents consider the RABC to be extremely important or important in minimizing the time required to finalize regulatory requirements. Two‑thirds (68%) consider the RABC to be effective in achieving this to a great extent and 26% to a moderate extent. Unsolicited comments from 10 of the respondents reflect a common view of the RABC as a highly efficient think‑tank, one that makes an invaluable contribution to spectrum management.

Both SITT and RABC interviewees consider RABC processes to be efficient. It was noted that the process for consultation on technical standards typically takes months not years. The RABC has achieved operational efficiencies over the years by adapting to changing technology, which has contributed to cost efficiencies as well. For example, the use of emails and teleconferencing has reduced meeting time and turnaround.

One plausible alternative to the RABC process would be to conduct stakeholder consultation on technical standards via the Canada Gazette. Evidence from the evaluation indicates that this would be a much lengthier process. In contrast to the RABC's short turnaround on technical standards of a few months, Industry Canada public consultations on spectrum policy and regulations via the Canada Gazette take considerably longer. According to information on Spectrum Management and Telecommunications webpages, 17 of 29 public consultations initiated from 2009–10 through 2013–14 took roughly 15 months on average from initial notice of consultation to decision. The average length of time is presumably considerably longer given that the results for the other 12 consultations had yet to be posted as of November 2014: published decisions were still outstanding for four initiated in 2009, one from 2010, one from 2011, four from 2012 and three from 2013. During the same period, Industry Canada issued 53 technical standards following consultation with the RABC.

Many interviewees highlighted the RABC as a best practice. It results in a government‑industry shared perspective, with both contexts understood by everyone. The RABC contributes to good government‑industry relations. One RABC respondent has promoted the RABC process to two other federal departments. Another noted that multinational companies want Canada at international spectrum regulatory events because Canada has a better system of consulting with stakeholders, and as a result, its positions on spectrum issues are better thought out than in many other countries.

Implications of Having No RABC

The evaluation investigated the implications of not having the RABC. According to many interviewees, if RABC did not exist stakeholder consultation would be less efficient and less economical for both Industry Canada and all other spectrum stakeholders. The process would take much longer, for example, at least a year or two via the Canada Gazette. Such delays have significant cost implications for industry and the department. For example, according to program management, the equivalent of roughly an additional two or three full‑time staff would likely be needed to undertake the research now done by RABC members that informs the technical standards, manage the process via the Canada Gazette (or organize and chair stakeholder consultations), and analyze and consolidate all of the input from either process. This in essence would double Industry Canada's current workload devoted to developing spectrum technical standards and the RABC. According to a SITT respondent, the salary for these positions would be about $100,000 each. According to some external stakeholders, using the Canada Gazette process for technical standards would take considerably longer and also be more costly for them as they would have to devote more time and effort to monitoring the Canada Gazette for notices, and researching and developing responses to proposed standards on an individual basis.

Interviewees identified key drawbacks, but no benefits, to not having the RABC. For example, without the RABC, stakeholders have no reason to openly discuss their positions and arrive at a consensus. If Industry Canada led the consultations instead of industry, stakeholders would focus on promoting their separate positions. Industry Canada would more likely encounter unforeseen circumstances without a mechanism that brings stakeholders together to discuss matters. That is, the RABC provides a unique open forum for competitors and other stakeholders to collectively shape a well‑informed perspective on spectrum issues, which otherwise would not be available to Industry Canada. RABC members often flag directions that IC should not pursue, saving time and effort. Interviewees believe that Industry Canada would not have the advantage of such industry insight without the working relationship it has with the radiocommunications community through the RABC.

Interviewees indicated that if the RABC did not exist, lobbying could play a more central role in the development of spectrum technical standards. The RABC process does not preclude individual organizations from lobbying Industry Canada on matters of interest to them, but where consensus has been reached, individual stakeholders are less likely to lobby their position. The process is transparent and achieves a balanced perspective which OECD documents note is a prerequisite for effective policies.

top of page

Alternative Scenarios to the RABC

Interview and survey respondents addressed two plausible alternative scenarios, either one of which Industry Canada might have to employ if the RABC did not exist. Scenario 1 involves Industry Canada managing the spectrum stakeholder consultation process by sending proposals for technical standards to interested parties who have publicly registered their interest in certain domains. Each stakeholder would submit its own response independently to Industry Canada, which would then need to collate and assess the input. Scenario 2 would be to use the Canada Gazette public consultation process. As in the first scenario, each organization would submit its response separately, and Industry Canada would have to collate and analyze the input. That is, there would be no source of consolidated, consensus input for Industry Canada to draw on if the RABC did not exist.

Neither alternative is recommended according to interview and survey responses. (See Figure 3 below.) Either one would result in an increased workload for Industry Canada and RABC members, and more time would be needed to finalize the standards. Industry Canada is a significant beneficiary of the RABC. Without it, Industry Canada would have to fund research and standards development work that RABC members currently contribute. According to one interviewee, input from RABC members addresses about 80% of all research necessary to develop appropriate standards. According to some interviewees, the process would be more litigious if Industry Canada undertook the work done by the RABC, and according to one, public consultation is not a suitable option given the technical nature of the issues.

The majority of survey respondents expected to see no change or a decrease in stakeholder representation, effective and efficient spectrum use, and fair and equitable access under either scenario. The majority of respondents also thought that there would be no change or an increase in spectrum interference under either scenario. Respondents were about equally divided on whether either scenario would increase or decrease the availability of high‑quality input on all spectrum regulatory requirements. For scenario 1, 47% thought the quality would increase and 47% thought it would decrease. Further details on the survey responses to these scenarios are presented in Figure 3.

Figure 3a: Survey Responses on Impact of Alternative Scenarios to the RABC
Scenario 1: Consultation via Circulation to Intrested Parties
Respondents thought that Scenario 1 would have an overall negative effect on quality and efficiency.
Bar chart of Survey Responses on Impact of Alternative Scenarios to the RABC (the long description is located below the image)
Description of Figure 3a
Figure 3a. Survey Responses on Impact of Alternative Scenarios to the RABC
Scenario 1: Consultation via Circulation to Interested Parties (%)
Respondents thought that Scenario 1 would have an overall negative effect on quality and efficiency.
Factors Moderately to significantly decreased No change Moderately to significantly increased Don't know
Workload – Industry Canada - 3 97 -
Time required to finalize regulatory requirements - 3 93 3
Workload – RABC stakeholders 13 7 80 -
Spectrum interference 13 37 47 3
High-quality input 47 3 47 3
Representative input 43 13 37 7
Effective and efficient use 37 33 30 -
Fair and equitable access 33 40 23 3
Figure 3b: Survey Responses on Impact of Alternative Scenarios to the RABC
Scenario 2: Consultation via the Canada Gazette
Respondents thought that Scenario 2 would have an even greater overall negative effect on quality and efficiency.
Bar chart of Survey Responses on Impact of Alternative Scenarios to the RABC (the long description is located below the image)
Description of Figure 3b
Figure 3a. Survey Responses on Impact of Alternative Scenarios to the RABC
Scenario 2: Consultation via the Canada Gazette (%)
Respondents thought that Scenario 2 would have an even greater overall negative effect on quality and efficiency.
Factors Moderately to significantly decreased No change Moderately to significantly increased Don't know
Time required to finalize regulatory requirements - 3 97 -
Workload – Industry Canada 3 3 93 -
Workload – RABC stakeholders 7 3 90  
Spectrum interference 10 35 52 3
High-quality input - 55 45 -
Representative input 52 10 38 -
Effective and efficient use 45 21 34 -
Fair and equitable access 35 41 21 3

In asking interviewees about alternatives that should be considered, it was suggested that requiring the RABC to do more and expand its mandate was not an appropriate option. Changing the working relationship between Industry Canada and RABC to an industry‑driven body would shift the RABC's mandate from being an advisory body serving government to one serving industry interests alone. However, a survey respondent suggested that the RABC could play a larger role in educating all stakeholders, including Industry Canada, on the needs and requirements of spectrum users.

Mechanisms in Some Other Jurisdictions

The use of advisory bodies to facilitate spectrum regulatory development is not unique to Canada. A web search revealed examples of spectrum technical advisory bodies in three other jurisdictions, although there may be other examples as well. The information that is readily available provides some insight, albeit limited, and it appears that other advisory bodies function somewhat differently.

For example, spectrum management in the United States is more complex than in Canada, with two federal organizations involved in oversight served by a total of three advisory groups. In contrast, Canada has one federal department providing oversight and one technical advisory body. Two of the American advisory bodies are described as providing input on spectrum policy, with no mention of regulations or standards, and the third is a federal interdepartmental committee that provides advice on policy and regulations governing federal government spectrum use. Nevertheless, stakeholder consultation on technical standards in the United States is considerably lengthier than in Canada as it includes public consultation via the Federal Register, the United States' equivalent of Canada's Gazette process. As well, lawyers are central to regulatory consultations rather than engineers. According to a representative of the United States' Federal Communications Commission, it is rare for the American process to take less than a year and it can be considerably longer. Once rules are published, petitions are common.

Singapore's Telecommunications Standards Advisory Committee (TSAC) and Hong Kong's Radio Spectrum and Technical Standards Advisory Committee membership bear close resemblance to the RABC's. However, according to posted information, their mandates appear to be broader than the RABC's. TSAC develops and recommends specifications, standards, information notes, guidelines and so forth for approval and adoption by the regulatory authority. Hong Kong's committee formulates policies and procedures in addition to technical standards.

top of page

Resource Allocation

The Directive on Transfer Payments requires federal departments to ensure grants are the minimum level required, with other available sources of funding taken into account. The RABC's total membership fees and Industry Canada's grant are about equal in size. On average, membership fees accounted for 40% of total annual revenues for the five‑year period 2009–10 through 2013–14. Industry Canada's grant accounted for 44%, ranging from 39% in the first three years to 51% in the last two when no Spectrum 20/20 conferences were held, thus there were no conference revenues.

Two interviewees commented that a fifty‑fifty split between Industry Canada and RABC membership dues as key revenue sources is fair, as it reinforces both being equal partners. Raising the current membership rate is not a viable option, as it would likely result in losing some of the smaller stakeholder organizations, skewing the membership in favour of large organizations. The RABC used to have staggered dues, but found this did not contribute to all members having equal rights and an equal voice, whereas one rate does. If larger organizations paid more, they would want more influence, which would be disadvantageous to achieving a consensus position and providing Industry Canada with a balanced response. The associations representing individuals and small businesses have indicated they find the current fee of $4,250 high and challenging enough.

As with RABC members, Industry Canada's resource contribution includes both a financial investment and staff time. The current grant amount of $85,000, which has not changed since 2006 according to the Public Accounts of Canada, is at a minimum level, if not below, according to interviewees. The grant helps defray operating costs, and the RABC's operating costs are tightly controlled. In addition to the grant, Industry Canada has the equivalent of two to three full‑time employees dedicated to developing spectrum standards and attending meetings. It costs an estimated $5,000 annually for staff to attend the RABC meetings.

In the absence of a grant, the RABC's mandate as an advisory body to Industry Canada, its critical raison d'être, would no longer exist. If the RABC continued, it would no longer be a forum bringing everyone together, airing all opinions and aiming for consensus. Some of the large stakeholders participate primarily because of the federal government's presence. According to one interviewee, these stakeholders would not participate in the RABC's working groups if Industry Canada was not present.

A few interviewees said that Industry Canada would be very well served if it increased the grant. An increase would contribute to addressing inflation, help the RABC maintain its membership fee at the current rate, facilitate outreach efforts to be as inclusive as possible and give the RABC room to organize an event like Spectrum 20/20 annually. The event facilitates an exchange of new ideas and discussions by technical experts on the future of spectrum management and topical issues. Another interviewee thought that an increase would signal to industry that the department continues to take the RABC's work seriously. A few interviewees commented that the value received considerably exceeds the cost and the return on investment is immediate.

Suggestions for Program Improvement

An intended immediate outcome of the RABC grant program is a streamlined, efficient regulatory review process. This in turn contributes to achieving the strategic outcome of an effective and competitive Canadian marketplace. Some interviewees noted that reducing consultation to less than 60 days is insufficient time to work on issues and compromises the quality of advice.

Another aspect for consideration is recognition of federal funding. According to the grant agreement, the RABC is required to acknowledge the federal government's financial support and provide reasonable visibility in its public relations and communications activities, where appropriate. Both the document review and interviews confirmed that the RABC recognizes federal funding in some instances. However, Industry Canada is notably absent on the RABC's website, whereas all other RABC funders (i.e., its members) are identified on the site's homepage with their corporate logos.

A few other points emerged that interviewees thought would be beneficial to consider:

  • Spectrum senior management at Industry Canada would benefit from greater insight into multi‑faceted spectrum issues facing Industry Canada by being more involved in the work being done by the RABC.
  • It would be helpful to the RABC members if Industry Canada could provide a schedule of planned consultations in advance.
  • It might be advantageous for SITT to undertake other forms of consultation with the RABC on spectrum matters. For example, discussions with interested stakeholders via the RABC before launching public consultation, or prior to publishing a policy or regulation to ensure no new developments are overlooked, particularly if a few years have passed since consultation took place.

The Directive on Transfer Payments requires federal departments to ensure a performance measurement strategy is in place for any transfer payment to facilitate assessment of a program's performance. In the absence of such a strategy, corrective action is to be recommended. The logic model and performance indicators developed for this evaluation provide a basis for a performance measurement strategy for the RABC grant program. Findings from the evaluation indicate some revisions to these elements are required. For example, minimizing spectrum interference is the primary long‑term outcome of the RABC's work. It should therefore be the first long‑term outcome identified in the logic model. The findings also indicate that the RABC's work contributes to fair and equitable access, although other factors have considerable bearing on this. Again, these findings should be reflected in the logic model.

top of page


4.0 Conclusions and recommendations

4.1 Conclusions

Relevance

The RABC grant program serves a significant need with respect to spectrum use within Canada. The program aligns with federal government and Industry Canada priorities for a fair, competitive market, digital economy.

Stakeholder consultation is an essential part of regulatory development in Canada, as in other OECD countries. To this end, the RABC has a unique mandate as a consensual technical advisory body to Industry Canada on radio spectrum regulatory issues.

The program's continuing relevance over the long term has been well established by this evaluation. Given this and the small cost of the program, it would be appropriate to streamline the next evaluation to focus on performance only and omit an examination of relevance.

Performance

The RABC approach is effective, efficient and economical. It effectively brings together a wide range of spectrum stakeholders, both members and non‑members. It provides valuable and timely input to the federal government at very low cost. It provides a forum for open discussion and for understanding the perspectives of various stakeholders, a key benefit for everyone involved. These discussions help stakeholders clarify their positions, determine areas of agreement, highlight where there are differences, and provide consolidated advice to Industry Canada. The standards ultimately issued by Industry Canada seldom deviate from the advice submitted by the RABC. The inclusion of non‑members is a critical aspect in ensuring the RABC's consultation process is truly a focal point in providing technical advice to Industry Canada. This aspect should be addressed in the grant agreement.

The RABC contributes directly to minimizing spectrum interference, essential for efficient and effective spectrum use. It also contributes to fair and equitable spectrum access, although other factors have a critical bearing on this. The process is relatively expedient and cost‑effective compared to alternatives such as the public consultation process via the Canada Gazette.

The RABC is an effective model for federal consultation with stakeholders on spectrum technical standards, delivering significant value to Industry Canada at low cost. A roughly fifty‑fifty split between the amount of Industry Canada's grant and RABC membership revenues is reasonable, as it reinforces the partner relationship.

The evaluation found no suitable alternatives to the RABC model for stakeholder consultation on technical spectrum regulatory issues. The program is functioning very well, although some suggestions for program improvement emerged.

A performance measurement strategy is required. The logic model, with some refinement, and possibly some of the performance indicators used for this evaluation provide a starting point. The resulting strategy, performance indicators and collection of data should reflect the low materiality of the grant and therefore be streamlined as much as possible without compromising the relevance and quality of the resulting information in providing insight on the program's performance.

Outreach to and inclusion of non‑members in the consultation process is an essential part of the RABC's role, to ensure Industry Canada benefits from broad representation and consolidated advice from all interested stakeholders. The agreement speaks to the RABC achieving full stakeholder representation through its membership only, whereas the evaluation revealed that the RABC can only achieve such representation by including non‑members, a fact that should be reflected in the agreement. As well, the agreement should not require membership by any groups that are not likely candidates for membership, such as experimenters and national security agencies. This would not preclude involving such groups in a consultation as non‑members if warranted.

4.2 Recommendations

The findings and conclusions of the evaluation lead to the following recommendations:

  1. In accordance with the Directive on Transfer Payments, SITT should ensure there is a performance measurement strategy for the RABC grant program, taking into consideration the low materiality of the program.
  2. SITT should update the grant agreement to reflect the RABC's actual practice of inviting non‑members to participate in consultations when appropriate, so that the RABC's advice represents the radiocommunications community as fully as possible.

top of page


Appendix A: Forms of Stakeholder Consultation

According to OECD documents, five forms of public consultation are typically used by OECD countries:

  1. Informal consultation ‒ initiated by government with selected stakeholders; includes discretionary, ad hoc, and non‑standardised contacts;
  2. Circulation of regulatory proposals for public comment – among the most widely used; generally more systematic and structured than informal consultation; the regulator decides who to include; responses are usually written, but may be oral;
  3. Public notice and comment – more formal, structured than the foregoing; much more widespread in OECD countries in recent years, although levels of participation tend to be low; in Canada, this is the Canada Gazette public notice and comment process;
  4. Public hearings – usually supplements other consultation procedures; tends to be formal, with limited opportunity for dialogue or debate among participants; and
  5. Advisory bodies – the most widespread form of public consultation among OECD countries, with its use greatly expanded in recent years; role varies from reacting to a regulator's proposals to acting as a rule‑making body, i.e., advice is only one of several regulatory functions, e.g., some advisory bodies carry out extensive consultation processes themselves.

Few OECD governments measure the quality of their regulatory consultations, although some refer to it in their regulatory impact assessments. Surveys with stakeholders could provide feedback on the quality and quantity of consultation. As of 2012, no OECD country had addressed this in any survey according to OECD's 2012 report Measuring Regulatory Performance.


Appendix B: Bibliography

Document Review

Canada.National budgets: The Road to Balance: Creating Jobs and Opportunity. February 2014.
Jobs, Growth and Long‑term Prosperity: Economic Action Plan 2013. March 2013.
Jobs, Growth and Long‑term Prosperity: Economic Action Plan 2012. March 2012.

--- Speech from the Throne. Seizing Canada's Moment: Prosperity and Opportunity in an Uncertain World. October 2013.

Industry Canada. Departmental Performance Reports. 2009–10; 2010–11; 2011–12; 2012–13; 2013–14

--- Digital Canada 150. Last modified April 2014.

--- Engineering, Planning and Standards (DGEPS) Overview – Collaboration with CRC. DGEPS, February 2014.

--- Grant Agreement 2010–11 [RABC]. DGEPS, 2010.

--- News releases: "Strengthening Canada's digital economy and increasing high‑speed Internet access." Toronto. .
"Industry minister James Moore highlights Measures to position Canada for success in the digital age." Vancouver. .
"Industry minister discusses the Harper government's commitment to a connected, digital Canada with i‑Canada Alliance." .

Office of Consumer Affairs, Participating in the Standards System – What are Standards?. Last modified March 2013.

--- Radio Spectrum 101: Usage in Canada and Abroad.DGEPS, 2011.

--- Report of the Industry Canada International Telecommunication Union Stakeholder Consultation. July 2013.

--- Reports on Plans and Priorities. 2009–10; 2010–11; 2011–12; 2012–13; 2013–14.

--- SITT Snapshot Deck. SITT,September 2014.

--- The Mobile Data Demand Challenge – Canada's Action Plan. SITT, November 2013.

--- "The Spectrum Crunch." [factsheet] Communications Research Centre Canada, December 2013.

--- Wireless Technology Insights. Communications Research Centre Canada, March 2014; July 2014.

Justice, Department of. Broadcasting Act (S.C. 1991, c. 11)

--- Department of Industry Act (S.C. 1995, c. 1)

--- Lobbying Act (R.S.C.1985, c44 (4th Supp.))

--- Radiocommunication Act (R.S.C., 1985, c. R‑2)

--- Telecommunications Act (S.C. 1993, c. 38)

Radio Advisory Board of Canada. Annual General Meeting . 2013.

--- Committee quarterly meeting notes and minutes. 2013.

--- Constitution. Approved 1999, effective 2000.

--- PowerPoint presentation to CanWISP. n.d.

Radio Advisory Board of Canada and Industry Canada. Spectrum 20/20. 2012.

Telecommunications Policy Review Panel. Telecommunications Policy Review Panel: Final Report ‑ 2006. Industry Canada, 2006.

Treasury Board of Canada Secretariat. Applying the Federal Cabinet Directive on Streamlining Regulation (November 2007)

--- Directive on Transfer Payments. Effective October 2008.

--- "Our Commitment to Canadians". Cabinet Directive on Regulatory Management. Last modified October 2012.

--- Guide to the Federal Regulatory Development Process, Part 2: Overview of the Federal Regulatory Development Process, last modified April 2014.

--- Policy on Transfer Payments, updated April 2012.

Literature Review and Website Scans

Congressional Research Service. The Federal Rulemaking Process: An Overview. June 2013.

Federal Communications Commission. Accessed: What we do, Technological Advisory Council, Technological Advisory Council Members.

Global News. "Spectrum auction 101: What consumers should know about Ottawa's sale of cellphone airwaves". .

Infocomm Development Authority of Singapore. Accessed: Standards and Quality of Service.

--- Telecommunications Standards Advisory Committee webpages. Accessed: Organisation Structure of TSAC, TSAC Membership List, TSAC Standards‑Setting Framework & Procedures (2012–2014) [logic model], Terms of Reference for TSAC (2012–2014).

Office of the Communications Authority, Hong Kong Special Administrative Region. Radio Spectrum and Technical Standards Advisory Committee (SSAC). www.ofca.gov.hk/en/about_us/advisory_committees/SSAC/index.html. Accessed: Terms of Reference, Members, Papers.

Organisation for Economic Co‑operation and Development. Background Document on Public Consultation. [PDF document], n.d.

--- "Enabling regulatory reform." [PDF document], Gabriella Meloni, Consultant to the Regulatory Reform Division, OECD Directorate for Public Governance and Territorial Development. n.d.

--- [website] Fighting Corruption in the Public Sector: Lobbying

--- Lobbyists, Government and Public Trust: Increasing Transparency Through Legislation, Volume 1. 2009.

--- Measuring Regulatory Reform: Evaluating Regulatory Management Tools and Programmes. [PDF document], Claudio Radaelli and Oliver Fritsch. Expert Paper No. 2, July 2012.

--- "New approaches to spectrum management", OECD Digital Economy Papers, No. 235, OECD Publishing.

--- Transparency and Integrity in Lobbying. [PDF document], October 2013.

Rixon, Daphne. "Stakeholder engagement in public sector agencies: ascending the rungs of the accountability ladder," International Journal of Public Administration, 33 (2010): 347‑356.

Shepherd, Tamara, Gregory Taylor, and Catherine Middleton. "A tale of two regulators: telecom policy participation in Canada," Journal of information Policy, 4 (2014): 1–22.

Simpson, Genevieve, and Julian Clifton. "Consultation, Participation and Policy­Making: Evaluating Australia's Renewable Energy Target," Australian Journal of Public Administration, vol. 73, no. 1 (2014): 29–33

Trichur, Rita. "Telus pulls out of wireless industry lobby group," The Globe and Mail. .

US Department of Commerce. Charter of the Spectrum Management Advisory Committee. 2013.

--- "Spectrum management". National Telecommunications & Information Administration. Also accessed: Interdepartment Radio Advisory Committee, Commerce Spectrum Management Committee

top of page


Evaluation of the Radio Advisory Board of Canada Grant Program
Management Response and Action Plan

Management Response and Action Plan
Recommendation Management Response and Planned Action Management Accountability Action Completion Date

1. Recommendation 1

In accordance with the Directive on Transfer Payments, SITT should ensure there is a performance measurement strategy for the RABC grant program, taking into consideration the low materiality of the program.

Response: Agreed

Action Plan:
- SITT will work with the RABC to develop such a performance measurement strategy and to begin tracking of performance indicators.

Director General, Engineering, Planning and Standards

2. Recommendation 2

SITT should update the grant agreement to reflect the RABC's actual practice of inviting non‑members to participate in consultations when appropriate, so that the RABC's advice represents the radiocommunications community as fully as possible.

Response: Agreed. In order to ensure that the RABC is able to provide balanced and unbiased advice, it is crucial for all interested stakeholders, particularly those with opposing points of view, to be represented.

Action Plan:
- SITT will implement such a provision in conjunction with the next grant agreement.

Director General, Engineering, Planning and Standards

Date modified: