Language selection

Search

Five-Year Evaluation Plan 2014-2019

Table of Contents (November 2014)

Introduction

The Financial Administration Act and Treasury Board Policy on Evaluation requires deputy heads of small departments and agencies to have coverage of all ongoing grants and contributions over a five year period.

Western Economic Diversification Canada’s (WD) Policy and Strategic Direction (PSD) has developed a five year plan to fulfil the evaluation coverage requirements of the Policy. This five-year evaluation plan covers the fiscal years 2014–15 to 2018–19.

The evaluations included in the five-year plan are based on a combination of consultations, analysis and prioritization process as described below.

Evaluation in the Government of Canada

In the Government of Canada, evaluation is the systematic collection and analysis of evidence on the outcomes of programs. This evidence is used to make judgments about the programs’ relevance, performance and alternative ways to deliver them or to achieve the same results.

Evaluation provides Canadians, Parliamentarians, Ministers, central agencies and Deputy Heads an evidence-based, neutral assessment of the value for money (i.e., relevance and performance) of federal programs.

The main purposes of evaluation are to:

  • Support accountability to Parliament and Canadians by helping the government to credibly report on the results achieved with resources invested in programs;
  • Inform government decisions on resource allocation and reallocation;
  • Support Deputy Heads in managing for results by informing them about whether their programs are producing the outcomes that they were designed to produce, at an affordable cost; and
  • Support policy and program improvements by helping to identify lessons learned and best practices.

Planning Context

Objective and Scope of the Planning Process

The planning process is intended to produce a five-year evaluation plan for approval by the Deputy Minister.

The five-year evaluation plan is designed to achieve evaluation coverage of all ongoing grants and contributions programs, and direct non-grants and contributions program spending as appropriate to the department, in accordance with Treasury Board policy. For the period of 2014–19, the department currently does not have any major statutory spending, any specific evaluations requested by the Secretary of the Treasury Board, or WD-related evaluations outlined in the Government of Canada evaluation plan. If that circumstance changes during the life of the plan, the plan will be amended accordingly. The plan is reviewed and updated annually or when a significant change in programming has occurred.

The evaluation plan was developed taking into account the broader government context within which WD is functioning. The following sections describe the context and a brief methodology used to develop the five-year evaluation plan.

Environmental Scan

Overall Government Priorities and Drivers

Through the Federal Accountability Act and Action Plan in April 2006, the Government of Canada enhanced its commitment to responsible spending and accountability by introducing a new focus on value for money and broadening the scope of evaluation beyond grant and contribution programs. The Financial Administration Act now requires that all grant and contribution programs be evaluated over a five-year life cycle.

The Government of Canada’s Expenditure Management System (EMS) requires clarity on expected results, sound logic models, feasible performance frameworks and periodic evaluations to inform program decision-making. The EMS focuses on results and value for money for all spending.

The Government of Canada’s Policy on Management, Resources and Results Structures (MRRS) sets out the requirement for departments to have performance measurement information on which allocation and reallocations decisions can be based. This performance measurement framework should include integrated financial and non-financial information. The development and application of performance measures has long been an integral part of the program evaluation function in government. Evaluation units have traditionally used measures of program performance to assess the impact and effectiveness of various government programs.

Central Agency Priorities and Drivers

The Treasury Board Policy on Evaluation requires deputy heads of small departments and agencies to evaluate all ongoing programs of grants and contributions every five years, as required by section 42.1 of the Financial Administration Act; and evaluate direct program spending, excluding grants and contributions, as appropriate to the needs of the department.

Departmental Program and Priorities

WD’s basis for reporting to Parliament is its Program Alignment Architecture (PAA). The PAA describes how the department manages its resources to achieve intended results/outcomes. The PAA focuses on programs and sub-programs designed to develop and diversify the western Canadian economy. The PAA states WD’s strategic outcome as:

  • A growing and diversified western Canadian economy

The following four program activities contribute to the achievement of this strategic outcome:

  • Business Development and Innovation: Western Canadian SMEs are engaged in international business, are competitive, and are innovative;
  • Community Economic Growth: Western Canadian communities have strong businesses, the capacity for socio-economic development, and the necessary public infrastructure to support economic growth;
  • Policy, Advocacy and Coordination: Policies and programs that strengthen the western Canadian economy;
  • Internal Services: Effective and efficient support for advancing the department's strategic outcome.

WD identified seven organizational priorities for 2014–15: Innovation, Skills, Aboriginal Economic Growth, Trade and Investment, Federal Defence Procurement, Modernization of the Western Canada Business Services Network (WCBSN) and Managing for Excellence in a Changing Environment.

Call for Proposals – The department launched a Call for Proposals (CFP) process to select projects for funding. The CFP process is a fair, open, transparent and widely accepted business process whereby interested parties develop project proposals and apply for funding to carry out activities that support objectives and priorities identified by the department. There are currently two streams under WDP and two application processes:

  • Stream one provides support to not-for-profit (NFP) organizations for projects aligned with WD's priorities.
     
  • Stream two is the WINN initiative that offers repayable contributions to small- and medium-sized for-profit enterprises (SMEs) to move innovative technologies from the later stages of research and development to the marketplace.

The CFP will be issued up to two times per year for both streams. Projects involving all industry sectors will be considered. Through the CFP process, the department aims to strengthen its responsiveness to economic trends and industry needs and ensure alignment to Government of Canada plans and priorities.

Linking the Five Year evaluation Plan to Government, Central Agency, and Department Priorities

This evaluation plan is directly linked to current departmental priorities, the PAA, MRRS and is designed to achieve coverage of ongoing grants and contributions over the next five years as specified in the Treasury Board evaluation policy. The plan outlines targets and timeframes for each project evaluation. Evaluation engagements included in this plan will provide objective, timely and relevant information around program spending in support of WD’s strategic outcome.

Overall Approach

In order to develop this plan, the PSD took the following into account:

  • Resources dedicated to evaluation projects and engagements each fiscal year;
  • A review of the departmental PAA, RPP, DPR, and Corporate Business Plans;
  • A review of Government of Canada priorities, including any government wide evaluation priorities;
  • Consultation with program staff to determine their evaluation and consultation needs;
  • A review of previous year’s evaluation plans to see if there was incomplete work that needed to be addressed in the five-year plan;
  • A review of past transfer payment authority renewals and their associated evaluation commitments; and
  • A review of other departments’ evaluation plans for programs delivered by WD on behalf of other federal departments.

Previous Evaluations

Previous evaluations conducted within the past five fiscal years based on approved evaluation plans include:

  • Urban Development Agreements in 2009–10
  • Centenaries Program in 2009–10
  • Loan and Investment Program in 2010–11
  • Mountain Pine Beetle Program in 2010–11
  • Entrepreneurs with Disabilities Program in 2010–11
  • Innovation Activity in 2011–2012
  • Western Economic Partnership Agreement in 2010–12
  • Women’s Enterprise Program in 2012–13
  • Trade and Investment Activities in 2012–13
  • Community Futures Program in 2013–14

The Five-Year Evaluation Plan

Based on the overall approach, a list of planned evaluations over the next five years was developed for inclusion in this plan (Table 1). The evaluation plan will guide the use of evaluation resources in the department and contribute to effective and efficient program delivery.

Key features of the plan include:

  • 100% coverage of G&C spending.
  • Alignment with the PAA to the extent possible (Table 2).
  • Recognition of the need to separate out the WINN initiative from regular WDP funded projects under the CFP process for not-for-profit organizations (even if both contribute to innovation related areas of the PAA).

All evaluations in this five-year plan will provide conclusions about the relevance and performance (effectiveness, efficiency and economy) of departmental activities. The evaluations will assess the core issues identified below. The department may choose to address additional issues in each evaluation.

  • Issue #1: Continued and demonstrable need.
  • Issue #2: Alignment with Government priorities.
  • Issue #3: Alignment with Federal roles and responsibilities.
  • Issue #4: Achievement of expected outcomes.
  • Issue #5: Demonstration of efficiency and economy.

Subject to the requirement to address all core issues, the department has the flexibility to determine the evaluation approach and level of evaluation effort in accordance with the program's risks and characteristics and the quality of performance information available for each individual program.

Updates to the Approved Plan

The five-year plan was approved by WD's Deputy Minister on May 20, 2014. WD regularly reviews its commitments in the evaluation plan to ensure that it reflects current departmental priorities. The following updates have been made to the current approved plan:

  • The Evaluation of the ABSN/FEDOs/EDP has been removed;
  • The Evaluation of Policy, Advocacy and Coordination Functions is moved from 2015–16 to 2014–15 and 2015–16 fiscal years;
  • The department's WDP Call for Proposal (CFP) process is currently undergoing a review to determine best practices and lessons learned to improve upon the process. The mid-term Evaluation of the CFP process for 2015–16 is dropped from the evaluation plan to minimize duplication of work; and
  • The department committed to a mid-term review in 2015 and a final review in 2018 of the $35.5 million contribution agreement signed with the Rick Hansen Institute in 2013. The agreement states that the reviews will be led by the Rick Hansen Foundation with the department providing an oversight function. This has been included in the evaluation plan. Results from the reviews will feed into the next departmental evaluation of Business Productivity and Growth.
Table 1: Evaluation Projects for 2014-2019
Project Name Project Description / Background Resources Year Date of Last Evaluation
FTEs Contracted 2014-2015 2015-2016 2016-2017 2017-2018 2018-2019
Evaluation of Business Productivity and Growth Impact evaluation focused on relevance, and performance 0.8 $70,000 X         2008 WDP Evaluation
Evaluation of Trade, Investment and Market Access Impact evaluation focused on relevance and performance 0.8 $60,000       X   2013
Evaluation of Innovation Capacity Building and Technology Commercialization Impact evaluation focused on relevance and performance 0.8 $80,000     X     2012 Innovation Evaluation
Evaluation of the Western Innovation (WINN) Initiative Impact evaluation focused on relevance and performance 0.8 $60,000       X    
Evaluation of Community Futures Program Impact evaluation focused on relevance and performance 1.5 $80,000         X 2014
Evaluation of Policy, Advocacy and Coordination Impact evaluation focused on relevance and performance 0.8 $40,000 X X       2009
PROGRAMS AND INITIATIVES LED BY OTHER DEPARTMENTS AND ORGANIZATIONS
Evaluation of the Building Canada Fund – Communities Component (BCF-CC) (WD to support this evaluation led by Infrastructure Canada) Impact evaluation focused on relevance and performance     X          
Evaluation of the Economic Development Initiative (WD to support this evaluation led by Industry Canada) Impact evaluation focused on relevance and performance           X    
Evaluation of the Canada Business Network (WD to support this evaluation led by Industry Canada) Impact evaluation focused on relevance and performance           X   2014
Review of the Agreement with the Rick Hansen Institute (WD to provide oversight on the reviews led by the Rick Hansen Foundation) Mid-term and Final review 0.2   X       X 2012

Return to the top of this pagetop of page

 

Table 2: Linkage to PAA
Project Name Business Development & Innovation Community Economic Growth Policy, Advocacy & Coordination
Trade, Investment and Market Access Business Productivity and Growth Innovation Capacity Building Technology Commercialization Infrastructure Programming Community Development Community Futures Program Targeted Economic Initiatives Advocacy & Coordination Economic Analysis
Evaluation of Business Productivity and Growth   X                
Evaluation of Trade, Investment and Market Access X                  
Evaluation of Innovation Capacity Building and Technology Commercialization     X X            
Evaluation of the Western Innovation (WINN) Initiative       X            
Evaluation of Community Futures Program             X      
Evaluation of Policy, Advocacy and Coordination                 X X

 

Date modified: