Language selection

Search

Evaluation of the Western Economic Partnership Agreements

Table of Contents (May 2012)

Executive Summary

Purpose of the Evaluation

Western Economic Partnership Agreements (WEPAs) are a multi-year federal-provincial decision-making tool designed to select projects that strengthen economic activity. Through individual agreements signed with each provincial government in western Canada, Western Economic Diversification Canada (WD) and the respective provincial government provide a matching contribution, enabling significant investments to fund projects that support national economic priorities while recognizing the unique characteristics of each provincial economy. The term of the current round of WEPAs (WEPA III) is from 2008 to 2012. The expected completion date for funded projects is September 30, 2013. The department provides $25 million in funding through the Western Diversification Program authority for each of the western provinces, matched by the provincial governments, for a total investment of $200 million to stimulate opportunities for western Canadians.

The purpose of this project was to conduct an evaluation of WEPA. In accordance with the 2009 Treasury Board Policy on Evaluation, the evaluation focused on relevance and performance (efficiency, effectiveness and economy).

Method of Study

This project was undertaken in two phases. The first phase focused on development of a detailed evaluation work plan. The field research undertaken in the second phase of the project included a detailed document and literature review, interviews with 143 key informants (including 25 WD representatives, 9 representatives of provincial governments involved in WEPA, 6 economic development experts, 15 other stakeholders, 4 representatives of other programs in Western Canada, 4 representatives of similar agreements in other regions, proponents of 50 projects, 19 representatives of spin-off projects, and 9 proponents of applications that did not receive funding), case studies of 8 projects (42 representatives were interviewed as part of the case studies), and focus group sessions in Winnipeg, Saskatoon, Edmonton and Vancouver involving 37 participants.

The evaluation focused on two key components of WEPA: WEPA as a decision-making tool; and the economic development projects funded under WEPA using the Western Diversification Program (WDP) authority.

Major Findings and Conclusions

The key conclusions arising from the review of WEPA are as follows:

Relevance

1.  The need exists for economic development projects in western Canada. Economic development projects, like those that the department has funded under the Western Diversification Program authority, are still needed because they support critical areas of the economy. Programming focused on promoting research and development, commercialisation of new technologies, expanding investment, and supporting trade can help western Canada become more competitive and reduce its dependence on natural resources. The department can continue to fund similar projects in the future using its existing authorities, whether or not the WEPAs are renewed.

2.  The projects funded were aligned with departmental strategic outcomes and government-wide priorities, and were consistent with the priorities of the federal government. The department works to promote innovation, business development and community economic development in its mandate. The projects funded under WEPA contributed to each of these priority areas. The Speech from the Throne in 2011 highlighted the federal government’s commitment to supporting business development, productivity, innovation, trade, and economic competitiveness. All key informants agree that the projects funded were consistent with the federal government’s agenda in western Canada and departmental priorities.

Performance (Efficiency, Effectiveness and Economy)

3.  Although most of the projects are still ongoing, there has been significant progress made already towards meeting their objectives and generating expected impacts. At the time of this evaluation, 60% of projects were still on-going, 38% had only been recently completed and 2% had not started yet. That being said, the projects and the agreements overall are viewed as having made significant progress towards achieving their objectives. More specifically:

  • The WEPA decision making process has contributed in promoting collaboration and strengthening the partnership between the federal and provincial governments.
     
  • The projects funded have generated a wide range of impacts. Reflecting the large number of projects funded, the most significant impact was perceived to be in terms of strengthening innovation, improving business productivity, and promoting trade and market development.
     
  • Funded projects by the department have already made significant progress towards achieving their performance targets.
     
  • The impact of departmental funding extends beyond the direct impact of the funded projects to include the impacts of follow-on projects, investments, and developments. Fifty-two percent of project proponents (26 of the 50) reported that their project have already led to other projects, investments or developments. The proponents estimate that these developments will involve new investments of nearly $84 million. Proponents of spin-off projects reported that their projects generated a variety of impacts including attracting follow-on investments totalling $11 million.

4.  The WEPA decision making process had both some benefits and challenges. The key informants agreed that the WEPA decision making process had a clearly defined governance structure. It was well-structured given what it was trying to accomplish, the intended outcomes and performance measures were clear and consistent with departmental reporting requirements.

The major issues regarding the WEPA decision process include the time required for project approvals, the complexity of the project approval process, a lack of coordination in performance reporting requirements between the federal and provincial governments, and a lack of dedicated provincial funding in some provinces. Other issues arose related to the challenges in defining new priorities, administration of the process, and the establishment of communication protocols.

5.  The funded projects leveraged significant funds from other sources. The budgeted costs associated with the 65 projects approved as of March 31, 2011 total $374 million, of which departmental assistance represented 22%. In other words, WEPA III leveraged $3.48 from other sources for every dollar contributed by the department which is significantly higher than $2.24 leveraged under the previous WEPA II. Project proponents noted that departmental funding of projects signals government support, which helps to provide market credibility as well as attract additional private sector funding and participation.

6.  Key informants provided a variety of suggestions for improvement. Key informants suggested that consideration should be given to improving the application and review process, reducing the time required for approval of applications or at least providing a better indication of how long a decision may take, streamlining the reporting requirements, encouraging the provincial governments to establish a dedicated source of funding, and developing mechanisms to communicate success stories and better share lessons learned as well as best practices across projects and provinces.

In conclusion, the projects funded under WEPA align with the department’s priorities, they are demonstrating early positive results, and there is an ongoing need for economic development funding. The department has the existing program authorities to make similar investments in the future. WEPA, as a decision-making model, has some strengths and weaknesses. The department has some options as to what decision-making model it will choose to make its economic development project decisions in the future. Regardless of what model the department chooses, this report offers up some suggestions for improvement.

Introduction

This chapter provides a brief overview of the Western Economic Partnership Agreements (WEPAs), the purpose of the evaluation, and the structure of the report.

Western Economic Partnership Agreements

WEPAs are a multi-year federal-provincial decision-making tool designed to select projects that strengthen economic activity. Through individual agreements signed with each provincial government in western Canada, the department and the respective provincial government provide a matching contribution, enabling significant investments to fund projects that support national economic priorities while recognizing the unique characteristics of each provincial economy.

WEPAs are based on the following principles:

  1. Cost-sharing with provincial governments on a 50-50 basis.
  2. Partnership and joint federal provincial planning and decision-making.
  3. Equal allocation of resources among the four provinces.
  4. Sustainable growth, not short-term relief.
  5. Building on strengths and enhancing competitiveness to achieve a more sustainable economic base.
  6. Recognition of unique characteristics of each provincial economy while at the same time being consistent with national economic development policies.
  7. Encouraging incrementality, private and institutional participation.
  8. Coordinating and enhancing ongoing federal and provincial government and industry activities.
  9. Reducing overlap and duplication by enhancing and coordinating economic development activities of the federal and provincial governments.
  10. Flexibility of delivery terms (i.e. unilateral, joint or third-party delivery). Both parties may fund individual projects equally, entirely by one of the parties, or some combination in between.

The term of the current round of WEPAs (WEPA III) is from 2008 to 2012. The expected completion date for funded projects is September 30, 2013. The department provides $25 million in funding for each of the western provinces, matched by the provincial governments, for a total investment of $200 million to stimulate opportunities for western Canadians. The program is governed by a management committee consisting of federal and provincial representatives. Various types of non-profit, Aboriginal, government and industry organizations (excluding individual businesses) are eligible to participate. Funded projects focus most commonly on innovation, business development, and market development.

Purpose of the Evaluation

The purpose of this project was to conduct an evaluation of WEPA. In accordance with the 2009 Treasury Board Policy on Evaluation, the evaluation focused on relevance and performance (efficiency, effectiveness and economy). A list of the specific evaluation questions under each evaluation issue is provided in the following table.

Table 1: Evaluation Questions by Evaluation Issue
Relevance: The extent to which the program addresses a continued need, is aligned with Government priorities, and is aligned with federal roles and responsibilities
  • Is there a continued need for the WEPAs? Are the needs documented and complete?
  • Is there a continued demand for the WEPA?
  • Is the program aligned to WD and federal government priorities?
  • Is the program consistent with federal roles and responsibilities?
Performance: The extent to which the program has achieved its expected outcomes
  • To what extent is the WEPA effective in meeting its objectives? What has happened as a result of implementing the WEPAs in western Canada?
  • What factors facilitate or impede the achievement of program results?
  • Did the WEPAs promote collaboration and strengthen partnerships between the federal and provincial governments? Do the agreements promote key aspects of the federal government agenda in western Canada?
  • INNOVATION: Has the WEPA strengthened innovation in western Canada?
  • BUSINESS DEVELOPMENT AND ENTREPRENEURSHIP: Has business development and entrepreneurship been strengthened as a result of implementing the WEPAs in western Canada?
  • COMMUNTY ECONOMIC DEVELOPMENT: Have communities been strengthened and made economically viable through implementation of the WEPAs?
  • PROVINCIAL PRIORITIES: Did the implementation of the WEPAs lead to the achievement of results in both federal and provincial strategic priorities? Do any of these provincial priorities conflict with the intended federal priorities?
  • Have there been any unintended (positive or negative) outcomes?
Performance: The extent to which the program demonstrates efficiency and economy
  • Is the program design appropriate for achieving expected program results?
  • Is the program undertaking activities and delivering products in the most efficient manner?
  • Are administrative and operational costs reasonable and funds being distributed at the lowest possible cost and risk?
  • How could the efficiency of the program’s activities be improved?
  • Did all parties contribute their share of resources?
  • Are there alternative, more efficient ways of delivering the program?
  • Is the program flexible in responding to funding opportunities presented?
  • How does program delivery vary from province to province?
  • Are outcomes and performance measures clear, quantifiable and consistent with departmental reporting requirements?
  • Is the program achieving its intended outcomes in the most economical manner?

Structure of the Report

Chapter II presents an overview of the current round and previous WEPAs while Chapter III provides a description of the methodology employed to conduct this evaluation. Chapter IV describes the key findings of the evaluation and Chapter V presents the major conclusions.

Western Economic Partnership Agreements

This chapter provides an overview of WEPA in terms of its priorities, eligible recipients and costs, review of applications, governance, differences in delivery across the four provinces, budget and expenditures, and logic model. It also compares the current round of WEPAs to previous agreements.

Overview of the Existing Agreements

1.  Priorities

These cost-shared agreements promote co-operation in planning and decision-making, recognizing the unique characteristics of each provincial economy while supporting national economic priorities. The current round of WEPAs focuses on the following federal priorities:

  • Supporting the creation and growth of knowledge-based research and development, business clusters; and the commercialization of new products, technologies and services out to market;
  • Supporting trade and investment promotion;
  • Enhancing business productivity and competitiveness; and
  • Increasing value-added production in traditional industries.

Each of the agreements also reflects priorities identified by the signing provincial government. In Manitoba, the Canada-Manitoba WEPA integrates the above strategic priorities as well as the promotion of economic development through tourism opportunities – promoting Manitoba as a tourist destination across the globe. In Saskatchewan, the WEPA objectives include community and regional development and diversification to support projects that sustain and enhance economic opportunities for Saskatchewan communities and geographic regions including developing opportunities for greater Aboriginal participation in the economy.

In Alberta, the WEPA is focused on expanding the knowledge-based economy through technology commercialization in priority sectors. The aim is to attract investment, support entrepreneurs and increase the number of high-tech companies in Alberta. The priorities for funding under the Canada-British Columbia WEPA include Asia Pacific, labour market and trade, Aboriginal inclusion, 2010 Olympics and Paralympics Winter Games and knowledge based industries.

2.  Eligible Recipients

Eligible recipients include non-profit organizations, post-secondary institutions, hospitals and regional health centres undertaking research and new technology development, First Nations band councils, individuals undertaking research, co-operatives, small and medium-sized commercial not-for-profit enterprises, federal crown corporations, provincial departments and agencies, and municipal governments.

3.  Eligible Costs

Funding can be contributed towards eligible costs, including operating and capital costs, that are incurred by the recipient and which, in the opinion of the department, are reasonable and are required to achieve the results to which they relate. Examples of eligible costs are:

  • Operations, maintenance;
  • Personnel: salary and benefits;
  • Equipment purchase and lease;
  • Information management and information technology acquisitions;
  • Legal, administrative, accounting, licensing, permits, consulting fees;
  • Rents, leases, leasehold improvements;
  • Acquisitions of proprietary processes;
  • Interest charges, insurance, fees, taxes;
  • Supplies, transportation;
  • Capital improvements related to projects; and
  • Other costs that are reasonable, incremental and directly attributable to activities necessary to achieve desired results, as specified by the agreement.

4.  Review of Project Proposals

Projects are assessed against a range of investment criteria including relevance to the WEPA strategic priorities, economic development impacts, feasibility/sustainability, strategic fit, range of support, leveraged funding, rural impact and commercialization potential. Project proposals are assessed independently, according to established procedures, by each partner prior to consideration for final approval by the Management Committee.

5.  Governance of the WEPA Decision Making Process

Each WEPA is managed by a management committee established by the federal and provincial ministers and co-chaired by senior federal and provincial representatives for each region. The Management Committee develops administrative guidelines for the application, assessment, approval, reporting, implementation and monitoring of projects as well as measurement of results achieved. Decision-making for project selection is achieved through consensus. While the federal and provincial governments share the overall contributions to the agreement equally, there is flexibility regarding the contribution of each partner to specific projects or initiatives. The contribution funding is provided by each partner directly to the project proponents to carry out projects that have been approved by the Management Committee.

6.  Differences in Decisions across the Four Provinces

While the structures of the agreements are largely the same across the four provinces, there are differences. In particular, differences exist with respect to:

  • Whether specific dedicated provincial government funding was set aside for WEPA. In Manitoba and Saskatchewan, the provincial contribution was formally approved by the Treasury Board and established as a specific line item in the provincial budget. In Alberta and British Columbia, there is no dedicated funding but rather funding is secured on a project-by-project basis from a variety of ministries and programs. The absence of dedicated funding can make WEPA more vulnerable to shifting priorities, provincial cutbacks or budget freezes.
     
  • The approval authority for the provincial funding. In Manitoba, each project receiving provincial funding requires Treasury Board approval. In Saskatchewan, no further approval is required beyond the department’s input to the Management Committee. In Alberta and British Columbia, once projects are approved by the Management Committee, the provincial contributions need only be signed off by the Deputy Minister of the funding ministry.
     
  • The level of coordination in negotiating and administering agreements. In Manitoba, each project is assigned to one of the funding partners which then take the lead in negotiating and administering the contribution agreement. In Saskatchewan, while one of the funding partners take the lead in working with a particular proponent, separate funding agreements are negotiated and administered. In Alberta, while there is a common application form used by both the department and the lead provincial ministry, applicants must work directly with both orders of government in securing funding. In British Columbia, applicants must also work directly with both orders of government in securing funding with some path-finding services provided by the department and the lead provincial ministry.

7.  Budget and Expenditures

Under the Agreements, from 2008 to 2012, the department through the Western Diversification Program authority provided $25 million in funding for each of the western provinces. The funds were matched by the respective provincial governments and expected to leverage significant investment to stimulate economies of the western provinces. The status of the existing budget, as of March 31, 2011, is summarized in the following table.

 

Table 2: Summary of WEPA Budgets and Expenditures
Province Alberta BC Manitoba SK Total
Federal Status
Budget $ 25,000,000 $ 25,000,000 $ 25,000,000 $ 25,000,000 $100,000,000
Value of Projects Approved to March 31, 2011 $26,413,249 $12,802,917 $24,862,104 $19,347,495 $83,425,765
Expenditures to March 31, 2011 $16,753,000 $6,635,000 $13,670,000 $11,063,000 $48,121,000
Provincial Status
Budget $ 25,000,000 $ 25,000,000 $ 25,000,000 $ 25,000,000 $100,000,000
Value of Projects Approved to March 31, 2011 $18,685,000 $18,994,754 $24,642,100 $21,104,288 $85,795,596

 

As of March 31, 2011, 65 projects have been approved under the current round. Additional projects will be approved in fiscal year 2011-12. The activities funded under the current WEPAs were in:

  • Innovation: including activities leading to technology adoption and commercialization, technology linkages, applied research and development, community innovation, skills development and knowledge infrastructure.
     
  • Business Development: including activities contributing to improve business productivity, enhanced market and trade development, industry collaboration, foreign direct investment and access to capital.
     
  • Community Economic Development: including activities contributing to planning and strategies, development and viability, and economic adjustment.

The number and value of projects approved by province and sub-activity are summarized in Table 3. As indicated, the 65 projects involved assistance of $83.4 million from the department. Projects related to technology adoption & commercialization account for nearly one-half of this amount.

Table 3: Total Number of Projects and Departmental Funding by Province and Sub-Activity as of March 31, 2011
Province Alberta BC Manitoba Saskatchewan Total
Sub-Activities Number Value ($) Number Value ($) Number Value ($) Number Value ($) Number Value ($)
Technology Adoption & Commercialization 12 22,106,000 4 5,441,253 1 5,803,804 9 7,264,920 26 40,615,977
Improve Business Productivity 3 3,519,249 6 2,679,138 3 4,980,000 4 2,302,280 16 13,480,667
Market / Trade Development     3 1,713,776 7 7,112,600 3 1,380,295 13 10,206,671
Technology Research & Development         1 999,500 2 8,000,000 3 8,999,500
Foreign Direct Investment 1 788,000     1 400,000     2 1,188,000
Technology Skills Development     1 337,500 1 366,200     2 703,700
Knowledge infrastructure         1 5,200,000     1 5,200,000
Community Planning             1 400,000 1 400,000
Community Economic Adjustment     1 100,000         1 100,000
Not Noted in the Database     1 2,531,250         1 2,531,250
Provincial Total 15 26,413,249 16

12,802,917

15 24,862,104 19 19,347,495 65 83,425,765

8.  WEPA Logic Model

A formal logic model was not developed during the negotiations for the current round of agreements. The logic model for the WEPAs was developed in two parts. The first umbrella logic model (Figure 1 provided on the following page) and its narrative were developed by the department to link the priorities, activities and outcomes of the WEPAs to the departmental Program Activity Architecture. The second logic model (Figure 2) and its narrative reflect provincial priorities and their linkages to the departmental Program Activity Architecture.

Comparison to Previous Agreements

The first formal type of federal-provincial agreement for economic development in western Canada - the General Development Agreements (GDAs) - was ratified in 1974. In 1984, the GDAs were replaced with a new "umbrella" agreement - Economic Regional Development Agreements (ERDAs) between 1984 and 1994. The responsibility for the ERDAs was transferred to the department in 1987.

1.  WEPA I: 1996 to 2001

The first WEPAs covered the period from 1996 to 2001. In this round, the department contributed $20 million which was matched by $20 million in funding from each of the four western provinces for a total investment of $160 million which was invested in more than 192 projects. An evaluation of the first WEPAs showed that the funding leveraged an additional $456 million from other levels of government and the private sector. In other words, every dollar contributed by WEPA leveraged another $2.86 from other sources ($5.75 for every dollar contributed by the department). Additionally, an estimated 120 new businesses were started as a result of WEPA projects. Nearly $500 million was invested in buildings, equipment and projects creating over 1,300 full-time jobs.

2.  WEPA II: 2003 to 2008

The WEPAs were renewed from 2003 to 2008 with the dollar value increasing to $25 million per region. The total funding of $200 million was invested in 162 projects. The 2003 to 2008 round of agreements (WEPA II) focused on the following federal priorities:

  • Supporting the creation and growth of knowledge-based research and development, business clusters and the commercialization of new products, technologies and services out to market;
  • Supporting trade and investment promotion;
  • Enhancing business productivity and competitiveness; and
  • Increasing value-added production in traditional industries.

An interim evaluation of the 2003 to 2008 WEPAs concluded that, for every dollar invested by the department for WEPA initiatives, another $2.24 was leveraged from other sources.

Text Version, Figure 1: WEPA Logic Model

In this figure, a logic model for the Western Economic Partnership Agreement program is outlined.

 

Text Version, Figure 2: WEPA Logic Model - Provincial Priorities

In this figure, a logic model for the Western Economic Partnership Agreement’s Provincial Priorities is outlined.

 

3.  WEPA III to Previous Agreements

The following table summarizes the available information on the three rounds of WEPA with respect to the terms; level of funding; number of projects funded; and leveraged funding. There is only limited data available on the cost and sources of funding for projects funded under WEPA I. There was a significant structural shift in funding between WEPA II and WEPA III. In particular, the funding allocated to technology adaptation and commercialization, foreign direct investment, and market/trade development activities significantly increased, while the funds spent on community economic adjustment, community planning, access capital, technology linkages, industry collaboration and community innovation decreased.

Table 4: Characteristics of WEPA I, II, and III
Characteristics WEPA I WEPA II WEPA III

Overview of the Agreement

Term

1996-2001 2003-2008 2008-2013
WD Funding Per Province $20 million $25 million $25 million

Total Budget

$160 million $200 million $200 million
Project Funding 1
Projects Funded 192 147 65

Average WD Funding/Project

$ 417 $ 645 $ 1,283

Leveraged WD Funding/Project

$ 2,375 $ 1,447 $ 4,470

Leveraged/WD Dollar

$ 5.70 $ 2.24 $ 3.48

Another notable difference is the decision to focus on larger projects under WEPA III, both in terms of the level of funding provided by the department and the total project costs. The average funding provided by the department increased from $644,886 under WEPA II to $1.28 million under WEPA III while the average total project costs increased from $2.09 million under WEPA II to $5.75 million under WEPA III. Of the 65 projects approved under WEPA III, 29 involved departmental contributions of $1 million or more as compared to 30 of the 147 projects approved under WEPA II. The following table compares the distribution in projects by level of funding under WEPA II and WEPA III.

Table 5: Comparison of Distribution in Departmental Funding and Total Project Costs between WEPA II AND WEPA III
Level of Funding WD Funding (number of projects) Total Costs (number of projects)
WEPA II WEPA III WEPA II WEPA III
0-49,999 28 0 8 0
50,000-99,999 20 1 12 0
100,000-499,999 48 20 42 4
500,000-999,999 21 15 30 12
1,000,000-4,999,999 29 22 33 30
5,000,000-9,999,999 1 7 16 13
10,000,000-49,999,999 0 0 6 5
50,000,000-99,999,999 0 0 0 0
100,000,000-299,999,999 0 0 0 1
Total Projects 147 65 147 65

Apart from the size of projects and distribution of funding by sub-activity, the model for WEPA III generally follows that of its predecessor. A few other notable differences between WEPA II and III are summarised as follows:

  • The department introduced a new performance measurement system for all of its projects part way through WEPA II. Reflecting its Program Activity Architecture (PAA), the department established a series of standard performance indicators for each of its sub-activities. Beginning in April 2005, the department incorporated at least one standard performance indicator into each contribution agreement. In addition to one or more standard indicators, each project may also have unique indicators on which they are required to report.
     
  • Certain provincial priorities have evolved. In Manitoba, for example, the focus shifted from community development to supporting tourism. In Saskatchewan, the focus shifted from increased productivity and diversification to supporting community development while, in British Columbia, the focus shifted from tourism, innovation, entrepreneurship, and sustainable communities to Asia Pacific, labour market and trade, Aboriginal inclusion, and 2010 Olympics.
     
  • Project funding tended to become less flexible. According to key informants, there were more rigid restrictions as to what each order of government could fund as well as the ability of proponents to transfer funding between various line items without a formal amendment.

 


[1] The funding data presented in this table is as of the time of the respective evaluations of WEPA I, II, and III and as such, may not represent the final totals.

Evaluation Methodology

This section describes design and the methodology of the evaluation.

Major Lines of Evidence

This project was undertaken in two phases. The first phase consisted of initial interviews as well as a file and document review leading to the development of a detailed Evaluation Work Plan. The Plan outlined the strategies and methodologies which were then implemented in the second phase of the project. The field research undertaken in the second phase of the project included document and literature review, key informant interviews, case studies, and focus groups.

1.  Document and Literature Review

A detailed review was conducted of WEPA documents and files as well as literature relevant to the activities of the agreements. The document and literature review included:

  • A detailed review of relevant documentation including the WEPA Results Based Management and Accountability framework, briefing notes, project databases, project final reports, and the 2008 WEPA Evaluation Report.
     
  • Developing a profile of WEPA in terms of program objectives, activities, outputs, intended outcomes, delivery model, budget, and logic model outlining the causal linkages between the objectives, activities, outputs, and outcomes.
     
  • A review of the Evaluation Framework prepared by the department which outlines the evaluation issues, questions, indicators, and data sources to be used in conducting the evaluation.
     
  • A review of literature on other similar agreements that have been established in other regions. A particular focus was placed on agreements involving the federal government and provincial or territorial governments with objectives related to economic development or diversification.
     
  • A review of literature on other programs in western Canada that share similar objectives as WEPA. These programs were based on the results of a literature review as well as interviews with the key informants including federal and provincial government representatives.

2.  Interviews with Key Informants

As part of this evaluation, interviews and surveys were conducted with 143 key informants including WD representatives, representatives of provincial governments involved in WEPA, economic development experts, other stakeholders, representatives of other programs in Western Canada, representatives of similar agreements in other regions, project proponents, representatives of spin-off projects, and proponents of applications that did not receive funding.

The following table provides the target number and number of interviews completed for each key informant group as well as a short description of each key informant sample. As demonstrated in the table, the target was to complete 121 to 135 key informant interviews.

Table 6: Sample Key for Key Informant Interviews
Key Informants Completed Target Number Description of Sample
WD Representatives 26 25 Purposive Criterion sampling. Departmental Representatives were selected based on their knowledge and involvement in WEPA activities as well as regional distribution. 28 representatives were selected and were 26 interviews completed. Of the 26 departmental representatives, 4 were members of the management committees and 22 were WD officers or program leads working in regions. Of the 26 representatives, were Alberta, 7 British Columbia, 6 Manitoba, and 5 from Saskatchewan.
Provincial Government Representatives 10 9 Purposive sampling. The participants were selected based on their involvement in WEPA. Of the 10 representatives interviewed, 3 representatives were from Alberta, 4 from British Columbia, and 2 each from Saskatchewan and Manitoba.
Economic Development Experts 6 8 to 10 Purposive sampling. Through internet research and document review, we identified 10 major academia and research institutions, and economic think-tanks in Canada. We contacted each organization and asked about their knowledge and expertise in economic development in western Canada overall and specifically about WEPA. We interviewed 6 experts who considered themselves sufficiently knowledgeable to provide input.
Other Stakeholders 15 13 to 16 Purposive Criterion sampling. We identified 73 industry associations, economic development groups, research organizations and non-profits that are active in regions, sectors, or projects relevant to WEPA. We contacted all 73 organizations to determine the level of familiarity with the department and WEPA as well as their willingness to participate.
Representatives of Other Programs in Western Canada 4 4 to 5 Purposive/Snowball Sampling. Through internet research, document review and referrals from key informants, we identified programs in western Canada that fund similar initiatives. We then surveyed programs representatives who were involved in delivering or managing these programs.
Representatives of Similar Agreements in Other Regions 4 4 to 5 Purposive Sampling. We identified similar federal/provincial programs and selected five which shared goals most similar to those of WEPA. 4 interviews were completed with representatives involved in these programs.
Project Proponents 50 35 Census. We attempted to contact all project proponents and completed interviews with representatives of 50 of the 65 projects. Of the 50 proponents, 9 represented projects funded in Alberta, 14 represented projects in British Columbia, 12 represented projects in Manitoba, and 15 represented projects in Saskatchewan.
Spin-Off Projects 19 15 to 20 Purposive/Snowball Sampling. During the interviews with project proponents, we asked them to identify other projects and initiatives created as a result of the project activities or organizations that significantly benefited from the project implementation. Based on the interviews we were able to identify 20 such organizations and projects and completed 19 interviews.
Proponents of Projects That Did Not Receive Funding 9 8 to 10 Census. We were provided by a list of 19 representatives associated with applications to WEPA that did not received funding (i.e. were rejected or withdrew their application). We contacted each representative (excluding those who were turned down for one application but approved for another and therefore were interviewed as project proponents) and were able to complete 9 interviews.
Total 143 121 to 135  

All key informant interviews were conducted by telephone. Where possible, each individual was provided a copy of the questionnaire in advance of the interview.

Return to the top of this pagetop of page

3.  Case Studies

Case study reviews were one of the critical lines of evidence for this evaluation. Case study reviews were conducted of 8 projects funded by the department. The case studies were selected to include a cross-section of projects by province, type of project, size of project, and type of proponent organization. The selection of specific projects for case studies also took into consideration the stage of project development (i.e. completed or close to completion); the willingness of the proponent to participate, and input provided by departmental representatives regarding the projects.

More specifically, the eight case studies:

  • Include two projects from each province;
     
  • Had made substantial progress in implementation by the time of the evaluation. Of the eight projects, four were known to be completed, two had received 90% to 100% of their funding from the department, one had received 70% to 80% payout, and one had been paid 50% to 60% of the approved funding;
     
  • Include a mixture of technology, market/trade development, and business productivity projects. Of the eight projects, three were categorized as Improve Business Productivity; one was categorized as Market/Trade Development and four were categorized as Technology Adoption & Commercialization.
     
  • Include one non major-urban project (Northern Manitoba) and two large dollar projects (over $2 million in departmental funding). In terms of departmental funding, two projects were approved for $2 million to $6 million, four were approved for $1 million to $2 million, one was approved for $500,000 to $1 million, and one was approved for less than $500,000.
     
  • Include a mixture of different types of proponents including three university-based projects, one service organization, one public private partnership, one sector council, one provincial tourism agency, and one provincial research organization.

In conducting the case studies, the evaluation team collected and reviewed background information including project proposals, progress reports, completion reports and other project outputs, and conducted an interview with the departmental project officer responsible for the file. Other methods include conducting a site visit to the location of six of the eight projects to review the work undertaken as well as examining documentation regarding the impact and outcomes generated to date. The evaluation team also conducted interviews with the project proponents who have been involved in the project, and interviews with a sample of other representatives involved in or impacted by the project. As demonstrated in the following table, a total of 42 interviews were conducted for the case studies.

Table 7: Case Study Interviews by Nature of Involvement
WEPA III Case Study Interviews
Nature of Involvement Frequency
Project Proponents 8
WD Project Officer 8
Others Involved 14
Users of Project Outputs 12
Total 42

The information obtained from the case study reviews was analyzed and summarized and the key findings included in this report.

4.  Focus Groups

As part of the evaluation four focus groups were conducted. Focus group sessions were staged at departmental offices in Winnipeg, Saskatoon, Edmonton and Vancouver. Representatives were from the department, the respective provincial government, and other selected stakeholders. An outline of possible questions was forwarded to participants in advance to help them prepare for the session. The focus of the discussions varied somewhat from site to site, depending upon the composition of the groups, the interests of the participants, and relevance of particular questions to that jurisdiction. In total, 37 representatives participated in the focus groups.

Table 8: Focus Group Participants by Type of Key Informant
Type of Key Informant Winnipeg Saskatoon Edmonton Vancouver Total
Proponents/Stakeholders 5 5 3 4 17
WD Representatives 1 4 5 4 14
Provincial Representatives 1 1 2 2 6
Total 7 10 10 10 37

The participants were selected based on their knowledge and involvement in WEPA activities. Most were also interviewed as part of the key informant interviews. All participants were invited by e-mail and telephone. The sessions were two hours in length.

5.  Data Analysis and Reporting

The data from each of the evaluation methodologies was summarized to address each of the relevant evaluation issues/questions. The data analysis strategy includes the triangulation of multiple lines of evidence. This involves the extraction of the results from each line of inquiry that relate to each evaluation issue and cross validation of the findings. This step took into account the strengths and limitations of each line of inquiry.

Evaluation Challenges and Limitations

During the process of conducting this evaluation, a number of challenges were encountered; however several techniques were employed to overcome each challenge. The main challenges included:

  • Timing of the evaluation. In terms of effectiveness, the evaluation focused on immediate and intermediate outcomes. However, most of the supported projects were newly funded, which means that sufficient time has not passed to measure the progress made against intended intermediate outcomes. Many project activities were still underway or only recently completed and the impacts not yet measurable. The impact of many of the lessons learned, strategies tested, models developed and results achieved can only be measured over a longer-term.
     
  • Staff turnover in some of the proponent organizations and organizations which applied unsuccessfully for funding. Some of the lead staff responsible for developing proposals which were ultimately not approved for funding as well as some lead staff associated with funded projects were not available for an interview due to staff turnover. To overcome this challenge, the evaluation team tried to interview other representatives of the organizations who were most familiar with the project activities. Before conducting interviews, the team made sure that the representatives had adequate knowledge of the project activities.
     
  • Lack of knowledge and awareness of the department and WEPA among stakeholders and economic development experts. Many of the stakeholders and economic development experts contacted for interviews were not aware of WEPA. To overcome this challenge and reach the targeted number of interviews, the evaluation team expanded the contact list to include a large number of organizations. Of the 73 stakeholder organizations contacted, the team were able to complete 15 interviews. Of the 10 economic research organizations and think-tanks contacted, 6 interviews were completed.
     
  • Potential for respondent bias. The evaluation findings are based, in part, on the views of those with a vested interest in the program and potentially biased in their responses regarding program outcomes. Several measures were taken to reduce the effect of respondent biases and validate interview results including (i) interviewers communicated the purpose of this evaluation, its design and methodology, and strict confidentiality of responses clearly to participants; (ii) interviews were conducted by telephone by skilled interviewers; (iii) the respondents were asked to provide a rationale for their ratings including a description of specific activities which contributed to the reported outcomes; and (iv) impacts generated by particular activities were confirmed through document review, case studies, and learning circles.
     
  • Identifying representatives of similar programs and scheduling interviews. The evaluation team had to conduct extensive internet research and several telephone calls to identify representatives of similar programs who had adequate knowledge of their respective programs and are able to participate in an interview. To overcome this challenge, the team sent the representatives information about WEPA and this evaluation, explained the importance of their participation and assured confidentiality of the interviews.
     
  • Limited information available about similar programs. Finding adequate information about similar programs was a challenge. In particular, it was difficult to gather program specific data (e.g., description of program design and delivery, budget and administrative cost, performance measurement system, assessment and selection criteria, etc.). Most programs have limited data on the Internet and some programs have never conducted an evaluation (or evaluation results are not publicly available). To overcome this challenge, during interviews the evaluation team asked representatives of the similar programs to provide materials and resources which include more detailed information about the programs.
     
  • In reviewing the results of the focus groups, it is important to note that the discussion reflects the opinion of those who were in the room. The participants in the focus groups are not necessarily representative of all those who have been involved in the program. Given the nature of focus groups, the main conclusions do not necessarily mean that all participants spoke to the issue or, if they had, would necessarily agree with the opinions which were expressed by others.
     
  • Difficulty obtaining additional information from project proponents for case study reviews. The case study reviews required multiple communications with proponents to validate findings, request additional information and clarify certain aspects and conclusions. With the busy schedules of most project proponents, it was difficult to arrange times to request additional information. To overcome this challenge, the evaluation team made consistent and multiple attempts to contact, used various communication methods (e.g. phone, email), and explained the importance of their contributions in this review.

Major Findings

This chapter summarizes the key findings of the evaluation gathered from all lines of evidence, grouped by evaluation issue and question.

Relevance

This section explores the relevance of the WEPA decision-making process in terms of its consistency with departmental and governmental priorities, and the need for project activities.

1.  The need exists for economic development projects in western Canada, and current departmental program authorities can be used to fund such projects.

When asked to rate how much of a need exists for the type of projects provided through WEPA, on a scale of 1 to 5, where 1 is no need at all and 5 is a major need, the average rating ranged from 3.7 amongst WD representatives to 4.9 amongst provincial representatives, project proponents, and proponents of applications which did not receive funding.

The key informants who provided high ratings noted that projects such as those delivered using the WEPA decision mechanism is needed because it supports critical areas of the economy (e.g. innovation, research and development, and commercialization); funds projects that often could not be funded through other initiatives (i.e. may not be eligible for funding or for which there is a shortage of funding available); levers funding from multiple sources; facilitates wider and more knowledgeable input into the selection of projects; promotes increased understanding between the two orders of government regarding development opportunities, priorities and needs; coordinates development activities; and results in projects that are relevant to the needs of the local economy.

Some key informants noted that, because funding come from existing Western Diversification Program (WDP) authority or provincial government funds, WEPA does not complement other funding as much as it simply duplicates or adds a level of complexity to structures that are already in place. In the opinion of some, WEPA complements the WDP only in that it allows the department to provide funding to provincial agencies without a specific Order in Council.

The departmental mandate allows for the funding of economic development projects using decision making mechanisms developed under the Western Diversification Program authority. Some key informants, mostly WD representatives, who provided a lower rating, believe that the federal and provincial governments would continue to cooperate, communicate and invest in similar projects in the absence of a WEPA decision making agreement. Furthermore, direct funding by the department may be more efficient and effective to the extent that the approval process would be simpler (i.e. only one order of government would be involved), and the department could focus more specifically on its priority areas and perhaps more easily adjust those priorities over time without an intergovernmental decision-making process as WEPA.

2.  A review of the literature confirms a need for projects that can facilitate economic development and diversification of the economy in western Canada.

Despite some improvements in recent years, the economies of the western provinces remain heavily dependent on the resource sectors. Western Canada's economic output from mining, forestry, agriculture and oil and gas activities accounts for about 15% of the region's Gross Domestic Product, compared to only 3% for the rest of Canada2. Over the next several years, further developments in resource sectors are predicted to be the primary source of economic growth in the region. Growth is expected in the forestry, mining and national gas sectors in British Columbia, the oil and gas sector in Alberta, the potash and crude oil sectors in Saskatchewan, and the mining sector in Manitoba3. While growth in these sectors is welcomed, heavy dependence on the resource sectors can create challenges and make the economy more vulnerable to fluctuations in global markets and volatile commodity prices.

The rate of economic diversification in western Canada will be influenced by a variety of economic drivers such as the level of investment in research, development and commercialization, business productivity, access to key markets, and access to investment capital. The available literature indicated that:

  • The level of business investment in innovation in western Canada is low. Compared to other developed nations, industry in Canada and particularly the western provinces lags behind in terms of investment in research and development activities, and the commercialization of research and development products4. Alberta, Manitoba and Saskatchewan have the lowest levels of business expenditures on research and development5. The Coalition for Action on Innovation in its recent Action Plan for Prosperity stresses the need for government action to build a more innovative Canada6.
     
  • From 1981 to 2007, increases in labour productivity contributed to 57% of per capita income growth, meaning it was the largest contributor of the improved standard of living in Canada7. However, compared to other Organization for Economic Cooperation and Development (OECD) countries, Canada and particularly western provinces rank very low in terms of labour productivity and productivity growth. Canada ranks 17th among 20 OECD countries and 6th among the G7 countries in terms of productivity increase over the last 30 years8. In 2007, the level of business sector labour productivity in Canada was only 75% of the level in the United States9. Low levels of business productivity in Canada is associated with weak investment by firms in innovation and building their knowledge capital, a failure to realize the opportunities from increased specialization, and weak business practices and governance mechanisms10. Some of the strategies, which can be implemented by government to boost business productivity, include investments in research and development, innovation, programs to facilitate development of technical, management and entrepreneurship skills, and adoption and diffusion of the latest state-of-the-art technologies11.
     
  • Small and medium-sized enterprises (SMEs) play a significant role in the Canadian economy. SMEs contribute approximately 41 percent of overall Canadian Gross Domestic Product, and employ a majority of the Canadian work force12. The role of SMEs in economies of some western provinces is especially important. For example, 98 per cent of all businesses in British Columbia are SMEs13. SMEs are also the driving force behind Canada's productivity and productivity growth. According to research, over the last 10 years, SMEs have led larger firms in terms of productivity growth14. However, the recent economic downturn, increasing labour costs, and the high Canadian dollar have created many challenges for SMEs and increased the need for programming, particularly programming that enables SMEs to upgrade and modernize facilities, equipment and processes, revise their business process, build new partnerships, connect with suppliers and expand into new markets15.
     
  • The economy in western Canada is heavily dependent on export trade. The continuing growth of Asian economies provides significant trading opportunities for Canadian business, especially for businesses in western Canada. Despite recent declines in international trade due to the recession, Canadian trade, especially trade with Asian countries, is demonstrating a steady increase16. Almost 75% of Canada's exports to Asian countries are shipped through ports in British Columbia17. The value of trade through British Columbia ports is expected to increase from $35 billion in 2005 to $75 billion by the 202018. By strengthening the supporting infrastructure, establishing partnerships, and supporting trade activities (e.g. trade shows, business trips etc.) governments can help business to enter into new markets and expand trade activities.
     
  • Competition for foreign investment is increasing. Although Foreign Direct Investment in Canada overall and particularly in western Canada is on the rise, Canada faces increasing competition from developing countries which are taking advantage of trade liberalisation and infrastructure improvements19. Canada's share of the global stock of inward Foreign Direct Investment fell from 9.8% in 1980 to 6.3% 1990 and to 3.2% in 200620. There is an important role for government in branding and communicating western Canada's advantage as a place to invest and do business21.

3.  The specific objectives, priorities and activities of WEPA are aligned with departmental strategic outcomes and government-wide priorities.

The mandate of the department focuses on the development and diversification of the economy of Western Canada. Towards that end, the department works to promote innovation, business development and community economic development22. The activities of WEPA contribute to each of these priority areas. More specifically, WEPA supports investments:

  • In innovation that promote the growth of a stronger, sustainable knowledge-based economy. As noted in Chapter II, WEPA has supported investments in technology adoption and commercialization, research and development, and building the knowledge infrastructure.
     
  • In business development that helps small and medium-sized enterprises become more innovative, grow faster, create value-added jobs and compete in global markets. WEPA has supported investments in improving business productivity, developing new markets, and attracting foreign direct investment.
     
  • That helps rural and urban communities adjust to changing economic circumstances, invest in public infrastructure and sustain their local economies. More specifically, WEPA has supported a small number of investments that facilitate improvements to community infrastructure. In addition, investments in innovation and business development also contribute to community economic development.

WEPA is consistent with the operational priorities of the department. In 2010-2011, these priorities included supporting technology commercialization (facilitating the translation of knowledge and technology into commercial opportunities); expanding trade and investment (enhancing the participation of SMEs in global markets, creating value-added opportunities connected to western Canada's gateways and corridors; raising western Canada's visibility as a competitive investment location); and facilitating business productivity and competitiveness23. Of the 65 WEPA projects approved as of March 31, 2011, 26 focused specifically on technology adoption and commercialization, 13 focused on market/trade development while 2 focused on foreign direct investment, and 16 focused specifically on improving business productivity.

In addition, the focus of WEPA is consistent with federal government priorities. The Speech from the Throne in 2011 highlighted the federal government's commitment to supporting business development, productivity, innovation, trade, and economic competitiveness. The Speech stated that government will support innovation and new technologies; "continue to make targeted investments to promote and encourage research and development in Canada's private sector", universities, colleges and polytechnics; and will facilitate "opening new markets for Canadian businesses and work towards attracting foreign investment to our economy."24

All WD representatives agreed that WEPA is consistent with the federal government agenda in Western Canada as well as with departmental priorities. These key informants noted that WEPA contributes to government priorities by supporting technology development and commercialization, trade expansion and investment attraction; improvements to business productivity, and strengthening Canada's competitive advantages, all of which contribute to economic development and diversification in Western Canada.

4.  The specific projects supported under WEPA are consistent with the priorities of the federal government.

During the negotiation of WEPA III, each provincial government was invited to define specific provincial priorities. To the extent that the priorities of the two orders of government may conflict, this raised the possibility that projects could be funded under WEPA which were not consistent with federal government priorities. However, this has tended not to occur for two reasons. First, the priorities of the federal government tended to be defined so broadly that they could be seen as encompassing all of the provincial government priorities. Secondly, the project review and approval process provided a mechanism through which the funding partners could discuss current priorities, select appropriate projects and, at times, even shape projects in such a manner that they would better contribute to the priorities of both orders of government.

It should be noted that several WD representatives did perceive that certain negotiated provincial priorities were either not consistent with departmental priorities or at least currently represent a relatively low priority for the department. The provincial priorities mentioned most frequently were promoting Manitoba as a tourism destination across the globe and promoting community and regional development and diversification in Saskatchewan (since the agreements were negotiated, community economic development has become much less of a priority for the department). Very few community economic development or tourism projects were approved.

It was more common for provincial representatives to note examples where they saw possible projects as being consistent with the stated federal government priorities yet not supported by the department because they were considered to be a low priority at the present time.

5.  WEPA is an appropriate role for the federal government given its responsibilities related to economic development. However, WEPA is not a necessary role given the possibility of using other models to promote economic development.

Key informants were asked whether WEPA is an appropriate and necessary role for the federal government. Given the shared responsibilities of the federal and provincial governments with respect to economic development, most key informants saw participation in a joint decision making process such as WEPA to be an appropriate role for the federal government.

Text Version, Figure 3: Is the funding of WEPA an appropiate and necessary role for the federal government?

In this figure, key informants indicated their response to the question of whether the funding of WEPA is an appropriate and necessary role for the federal government.

However, some key informants from the department disagreed that WEPA is a necessary role. It was viewed as not necessary for the department to promote development through a bi-lateral agreement such as WEPA; rather, the department could and does use other strategies such as direct funding to promote development.

6.  WEPA is seen as a decision-making tool that allows the department to deliver on projects linked to objectives under the Western Diversification Program authority. WEPA’s share many of the advantages and disadvantages of intergovernmental agreements.

The WEPA is funded under the program authority of the Western Diversification Program (WDP). Contributions under the WDP support activities that develop and diversify the western Canadian economy and activities where economic and/or employment benefits accrue primarily within western Canada. Funding is focused on activities that: support innovation; promote a competitive and expanded business sector in Western Canada; and, develop sustainable communities that improve the competitiveness and quality of life in western Canadian communities. As a decision making tool, WEPA allows the department to fund projects that support WDP’s broad objectives using a governance model that incorporates provincial and federal management committees.

The comparative analysis undertaken for the evaluation revealed that the decision making processes of WEPA share many of the advantages and disadvantages of intergovernmental agreements. In particular, the agreement facilitated joint planning and coordination, reduced the risk of duplication and overlaps, enhanced intergovernmental partnership and relations, increased the profile of funded projects, helped to create better referrals to other sources of funding, and greater leveraging of financial and in-kind contributions.

The disadvantages included decision-making processes which tended to be long and bureaucratic as federal and provincial government approach to making decisions differed significantly. In addition, less attention was given to areas where government priorities do not align (e.g. community economic development and tourism). Also coordination and administration of the process required significant efforts and resources; and the money spent on the programming activities may not be necessarily new as both governments would have spent similar amounts through other avenues if WEPA did not exist.

Return to the top of this pagetop of page

Achievement of Expected Outcomes

This section describes the effectiveness of projects implemented using the WEPA decision making tool with respect to achieving departmental objectives.

1.  It is too early to determine the ultimate impact of WEPA III projects given that most projects are still in the implementation phase or have only recently been completed.

Significant time is required for projects such as those funded through WEPA to fully implement their activities and generate expected outcomes. Most WEPA projects, however, were still being implemented at the time of this evaluation. According to project proponents, 60% of projects were still on-going, 38% had only recently been completed and 2% (one project) had not yet started.

Text Version, Figure 4: What is the current status of the projects?

In this figure, 15 Project Proponents indicated the current status of their projects.

2.  WEPA projects were viewed as having made significant progress towards achieving their objectives.

The project proponents most commonly view the objectives of their projects as developing new products, technology, innovation or infrastructure; developing new and expanding existing markets; improving business processes, productivity and competitiveness; engaging in research, development and testing; preparing a workforce and addressing labour issues; and supporting local economies.

The project proponents and other key informants were asked to rate how successful WEPA projects have been to date in achieving their objectives. On a scale of 1 to 5, where 1 is not successful at all and 5 is very successful, project proponents provided an average rating of 4.6. Project proponents noted that their intended project activities were or are being implemented as intended. Projects have produced outputs such as research and technology products, and training that have been utilized or purchased by other organizations, and that projects have met or are on track to meet their performance indicators. Amongst the key informants, the average ratings regarding the success of WEPA projects in achieving their objectives varies from 3.5 amongst stakeholders to 4.1 amongst departmental representatives.

Text Version, Figure 5: On a scale of 1 to 5, where 1 is not at all successful and 5 is very successful, how successful do you think WEPA/the WEPA project is to date in achieving its objectives?

In this figure, key informants provided a rating response to the question of how successful WEPA/WEPA projects have been in achieving its objectives.

While noting that it is premature to determine the eventual impacts as many projects were still ongoing, key informants highlighted the success of various projects, the effectiveness of the projects in leveraging funding from various sources, and the level of coordination and communication between the federal and provincial governments. Those who provided a lower rating noted delays in project approvals which slowed implementation and resulted in some projects not going ahead. In addition, a few projects were not implemented as planned and/or did not generate desired outputs or outcomes due to problems with the WEPA decision making process. A review of 8 case study projects demonstrated that, although most projects experienced at least some delays and one project had not yet made any progress towards meeting performance targets due to delays, all projects still expect that they will achieve their objectives and meet their performance targets.

2.  In most provinces, the WEPA decision-making process was viewed as successful in promoting collaboration and strengthening the partnership between the federal and provincial governments.

Government representatives were asked to rate the extent to which WEPA has been successful in promoting collaboration and partnership between the two orders of government using a scale of 1 to 5, where 1 is no impact at all and 5 is major impact. As demonstrated in the following figure, WD representatives provided an average rating of 4.2 and provincial representatives provided an average rating of 4.4.

Text Version, Figure 6: How much of an impact do you believe that WEPA has had in terms of promoting and strengthening the partnership?

In this figure, key informants provided a rating response to the question of how much of an impact WEPA has had in strengthening relationship on a scale of 1 (no impact at all) to 5 (major impact).

Government representatives noted that partnership is the central premise of WEPA. They highlighted the importance of the joint federal-provincial funding in:

  • Coordinating the activities of the two orders of government by facilitating joint planning and decision-making within an area of joint responsibility. The decision making process under WEPA contributed to greater communication and coordination at both the project officer/program level and across senior management. The agreements resulted in increased understanding between the two orders of government regarding the needs of the respective provinces, key strategic directions and existing initiatives.
     
  • Facilitating a longer-term, more strategic approach to development focused on series of related projects and initiatives. The coordinated, strategic approach extended beyond projects funded under WEPA by facilitating greater coordination in other support provided by the two orders of government.
     
  • Making more informed decisions in the selection of projects. One of the strengths of the existing model was that it facilitated input from a wide variety of federal and provincial representatives who were familiar with the sectors, markets, technologies, and proponents.
     
  • Supporting the development and implementation of incremental, industry-driven projects which meet the development needs of the province. By working in collaboration with industry organizations and provincial governments, WEPA helped to better align the priorities of the proponent, the federal government and the provincial government. In addition, it enabled the department to provide funding to provincial crown agencies without a specific Order in Council.

Some representatives who provided lower ratings noted that the joint decision-making was not necessarily an easy task and, at times, generated strong disagreements regarding the selection of projects.

4.  Reflecting the diverse nature of the project activities supported, WEPA projects generated a wide range of impacts. WEPA projects generated their most significant impacts in terms of strengthening innovation, improving business productivity, and promotin trade and market development.

Government representatives were asked to rate the impacts that WEPA projects have had with respect to each ofo its expected outcomes using a scale of 1 to 5, where 1 is no impact at all and 5 is major impact. The results are summarized below.

Text Version, Figure 7: How much of an impact do you believe WEPA has had in terms of:

In this figure, key informants indicate their perception of the impact of WEPA on expected outcomes.

Return to the top of this pagetop of page

Government representatives provided the following rationale for their ratings and examples to illustrate the nature of the impact.

  • Strengthening innovation (e.g. increasing research and development, strengthening technology linkages, facilitating technology adoption and commercialization, and/or strengthening the knowledge infrastructure). Strengthening innovation was the major area of focus, accounting for 64% of the WEPA funding approved to date.
     
  • Improving business productivity. WEPA projects facilitated business investment in technology to improve productivity, supported general and entrepreneurship training, and improved business operations and processes. Sixteen percent of the funding and 25% of the projects funded to date focused on improving business productivity.
     
  • Promoting trade and market development. Twelve percent of the funding and 25% of the projects funded through WEPA was categorized as market and trade development. Several projects implemented in Manitoba were highlighted as having a significant impact in terms of promoting international trade and market development in the province.
     
  • Addressing the priorities negotiated with the province(s). Each Agreement defines specific provincial priorities. Each province was viewed as having made progress towards achieving provincial priorities as a result of implementing projects, with the rating being highest in Alberta where the focus was on innovation and technology transfer.
     
  • Increasing value-added production in traditional industries.Compared to other priority areas, few projects were implemented which focused directly on value-added production in traditional industries. Approximately 7% of funding directly focused on increased value-added production in traditional industries. However, several other projects in the forestry, agriculture and mining industries (e.g., Composites Innovation Centre, Canadian Manufactures and Exporters Manitoba Division Centre of Excellence, and FPInnovations) impact on value-added production. In addition, it is anticipated that some of the technology projects will lead to increased value-added production in traditional industries over the medium to longer-term.
  • Promoting attraction of foreign direct investment.Only two WEPA projects directly focused on promoting foreign direct investment in Western Canada, including a project related to Vancouver Olympics 2010. In addition, several projects such as YES Winnipeg and CentrePort Canada Corporation were not categorized as investment projects but are expected to generate foreign direct investment in the future.
     
  • Facilitating community economic planning, development and adjustment. According to WD representatives and provincial representatives, projects specifically aimed at community development were not a high priority for WEPA. Only two community planning/economic adjustment projects were funded through WEPA. Nevertheless, many projects funded through the department have supported business and job creation and have already or will contribute to community development.

5.  Although many are still on-going, the WEPA projects have already made significant progress towards achieving their performance targets25.

The performance measurement system used by the department requires project proponents to collect and report data on a number of agreed upon standard and unique performance indicators (usually up to a maximum of five indicators per project). When the projects are approved, targets are established for each indicator. Throughout project implementation, the proponents may provide periodic updates to the department on performance indicators and targets. During the interviews conducted as part of this evaluation, project proponents were asked to provide updates on their respective performance indicators.

The following table provides a list of all 22 performance indicators which are associated with two or more of the 65 WEPA projects, the number of projects using each indicator, the number which have reported their performance to date (with the number of those surveyed provided in brackets), the aggregated target numbers, the progress previously reported to the department on each indicator (prior to the survey), and the revised totals after updating the progress based on the results of the survey.

As demonstrated in the table, on an aggregate basis, seven of the targets for the 22 indicators have already been met and three are very close to being met. In particular, the WEPA projects have met or exceeded targets in terms of introducing technologies to market, participating in export and market development initiatives, preparing export ready companies, creating industry association partnerships, training people, facilitating investments, and achieving client satisfaction. The WEPA projects are very close to meeting their targets in terms of identifying products and processes for further research and development, demonstrating viable technologies, and creating and expanding businesses. The indicators requiring much more progress to be reported to achieve the targets include facilitating investment, creating spin-off companies, and executing licenses. Each of these indicators would require considerable time after completion of the project to be realized.

Table 9: WEPA Performance Measures
Common Performance Indicators Projects with Indicator Reporting* Aggregated Target Previously Reported Updated Total
Innovation
Technology demonstrations 15 12(8) 91 19 33
Prototypes developed 11 10(8) 82 33 36
Technologies to market 8 7(5) 203 402 1,018
Technologies adopted 5 4(2) 49 8 14
Patents filed/issued 5 5(4) 29 8 6
Products or processes that are identified for further R&D 3 1(1) 19 4 14
Demonstrations of viable technology 2 1(1) 13 - 8
Licenses executed 2 1(1) 12 - 0
Export and Market Development
Co's participating in export and market development initiatives 8 8(7) 5,105 5,138 5,268
Increase in sales 8 6(4) $ 100,400,000 $ 46,610,000 $ 59,896,000
Export ready companies 4 3(3) 110 97 132
Employment and Staffing
Jobs created or maintained 20 20(12) 7,033 903 2,780
People trained 4 4(3) 593 132 596
Skilled personnel 2 2(0) 52 12 12

Investment

Venture capital invested 2 2(2) $ 16,500,000 $ 1,600,000 $ 6,970,000
Indirect investment facilitated by WD 5 4(0) $ 22,200,000 - $ 29,100,000

Business Creation

Businesses created/maintained/expanded 16 15(8) 1,356 471 1,341
Spin-off companies formed 4 4(2) 19 - 1
Other
Projects promoting participation in major international events 6 5(5) 63 17 26
Industry association partnerships created 4 3(3) 57 42 57
Client satisfaction with business services provided 3 2(1) 3 1 1
Client satisfaction as measured at a minimum of two events 2 2(2) 80% 80% or 92% 89% or 92%

* Provides the number of projects that we were able to obtain performance data either through surveys or by reviewing WEPA performance files. The numbers in brackets indicate the number of projects with whom the evaluation team completed surveys

From the sample of 50 projects which were the focus of proponent interviews, eight projects were selected for further case study. The document and follow-up interviews with various representatives associated with or impacted by these eight projects confirmed the data reported by the proponents. Of the eight projects, one has already fully met their performance targets, six have made significant progress, and one has yet to report any progress due to delays in receiving security certification. All eight projects expect to fully meet their performance targets over time.

Although still early in implementation, the case study projects have already generated significant impacts in terms of facilitating development of technology, innovation and skills, creating jobs, and increasing revenues. The case studies highlight the diverse nature and success of the projects supported under WEPA. These include:

  • Installation of underground mining stimulators and mining training programs developed to better prepare workers for the mining industry; and raising the profile of Manitoba in international markets and increasing tourist volumes and revenues in Manitoba;
     
  • Establishing a new mineral analysis laboratory in Saskatchewan, to conduct mineral analysis for the mining industry and researchers. This also resulted in creating jobs, increasing revenues and contributing to local economic development;
     
  • Availability of a cutting edge neuroscience technology in Alberta for researchers and industries helping to create and commercialize new prototypes, drugs and biological implants.
     
  • The attraction of 49 students from India to attend internship programs at various universities in British Columbia and build their skills. Some of these students have already returned to pursue a graduate degree in various fields. Representatives of various industries expressed interest and support in the program, which may lead to more skilled people working in British Columbia in the future.

6.  The impact of WEPA projects extends beyond the direct impact of the projects to include the impacts of follow-on projects, investments, and developments.

The impacts were not limited to the direct impacts of each project. Projects often lead to follow-on projects or investments by the proponent organization or by other organizations which may utilize the technology, knowledge, products or services created by the project. Even though most WEPA projects are still in the implementation phase, the results of interviews with project proponents as well as others involved in spin-off projects indicated that the projects have already generated significant follow-on projects and investments. Fifty-two percent of project proponents (26 of the 50) reported that their WEPA project have already led to other projects, investments or developments. The proponents estimate that these developments will involve new investments of nearly $84 million. A further 26% of the proponents expected their projects to lead to other projects, investments or developments in the future.

Of the 26 project proponents who indicated that their projects have generated new projects and investments, 14 were able to elaborate on the impact of these projects. Examples of such impacts include the provision of an in-school science program to 9,000 students from grades 7 to 12 in Alberta, and the participation of people in various related employment training programs. Other examples were pilot projects to demonstrate technology developed or staging of events (e.g. a film festival in Manitoba), implementation of new communication and network technologies in several remote communities.

As part of the evaluation, representatives were interviewed from 19 other organizations identified by the project proponents as having undertaken spin-off projects, investments or developments related to the original WEPA project. The objectives of these follow-on activities varied widely, from developing and testing new technology and software to commercializing new products, exploring new markets, raising awareness of the new technology, undertaking business development and expansion, and building partnerships. When asked to rate the success of their follow-on activities on a scale of 1 to 5, where 1 is not successful and 5 is very successful, these representatives provided an average rating of 4.5. The following figure indicates the impacts mentioned by proponents of spin-off projects.

Text Version, Figure 8: Has the project had an impact on your organization in terms of:

In this figure, 19 Proponents of Spin-off Projects indicated the impact WEPA projects have had on their organizations.

Return to the top of this pagetop of page

The proponents of spin-off projects provided the following rationale for their ratings and examples to illustrate the nature of the impact.

  • Facilitating partnerships, strategic alliances, and collaborations. The proponents of spinoff projects entered into a range of partnerships and collaborations. The partnerships included establishing agreements over joint project implementation, publishing research reports and articles, obtaining new investments and product developments, building various facilities and infrastructure projects, and participating in various working groups and advisory committees.
     
  • Skill development. The projects contributed to the development of skills and capacities of the staff members of the spin-off organizations as well as helped other industries, businesses and students learn about and utilize various products and services.
     
  • Developing new markets Four proponents noted that their new markets included various regions in Canada and six proponents were able to expand to international markets such as United States, Poland, Russia, Mexico, Brazil, China, Australia, United Kingdom, Brazil, Spain, and Germany.
     
  • Increasing revenues. Of the ten proponents who noted an impact in increasing revenues, three were able to quantify the revenue increase on a dollar basis and two indicated a percentage increase. According to three proponents, the project helped them to increase revenues by $44 million.
     
  • Creating or maintaining employment. The proponents of 11 spinoff projects, who reported an impact in creating or maintaining employment, noted that the project created or maintained 187 employees. Most of the created/maintained employment was for high-skilled positions in technology, research, Information Technology and marketing sectors.
     
  • Developing, commercializing or adopting new technologies. 10 proponents reported developing or adopting 18 different technologies such as iPad and iphone software and applications, networking device for computers, data downloading smart meters, communications equipment, fibre-optic technology, storage devices, lithium technology, device controllers and switches, and GPS technology. They also reported commercializing five products.
     
  • Encouraging or attracting investments. The proponents of spin-off projects noted that new investments came mostly through connections and partnerships that they built through project implementation. According to the proponents, four projects generated an investment of $10.8 million for their organizations.

The follow-on projects, investments or developments may themselves generate further follow-on activities. Of the 19 proponents of spin-off activities interviewed, 21% (4 proponents) indicated that their projects have already led to other projects, investments or developments and an additional 37% noted that will happen in future. The secondary spin-off projects included a demonstration of products in other regions, and expanding project activities to other regions. The investment generated by new initiatives totalled $5.4 million, with the funding provided by governments, industry groups and private companies.

Of the 8 projects reviewed as case studies, 4 also generated spin-off projects, follow-on investments or developments. For example, after participating in international sales trips organized through a WEPA project, an industry partner was able to build business partnerships with 15 tour operators from five different countries, attract $1.3 million new investments and increase their customer base by additional 400 clients. A nanotechnology research facility specialized on nerve repair engaged in partnership with private sector companies to develop neuron-science drugs and train research fellows. An infrastructure innovation cluster helped partner companies to attract significant new investment for developing and commercializing new technologies. A world class electron microscopes established in Alberta through a WEPA project led to a new project to introduce nanotechnology to students in the province.

7.  The success of WEPA activities in producing expected outcomes is attributed to the collaboration and partnership between governments, the ability to leverage other funds, the flexibility of selecting projects, and the skills of program staff.

This evaluation explored key factors that may have contributed to the success of WEPA programming. Factors identified by key informants include:

  • Collaboration and cooperation between two orders of governments, which helped to set common goals and priorities and identify worthy projects.
     
  • Ability to leverage substantial amounts of funds from other sources, including the federal and provincial governments as well as the private sector.
     
  • Clearly defined and agreed upon activity goals and priorities.
     
  • The flexibility of selecting various projects across different sectors and regions.
     
  • Involvement of skilled staff members and capable project partners. The expertise, skills and commitment of the people involved in WEPA activities, including both federal and provincial government representatives, played a significant role in the success of the projects.

According to the project proponents, the success of projects can be attributed to skilled and committed project staff and the capacity of proponent organizations to deliver quality project activities and develop innovative products and technologies. The ability of the projects to reach out and involve community members, clients and other stakeholders in the project activities and partnerships and support from other stakeholders, and communities also played an important role in success of the WEPA projects.

8.  Several project-specific factors such as delays in project approvals and implementation, lack of stakeholder support and difficulties in accessing additional funds served as obstacles to the success of WEPA.

While projects have generally been successful, they have faced some constraints. Key informants indicated that the following obstacles related to the decision-making process under WEPA served as an impediment to achieve results in projects.

  • Changes in provincial governments can cause a shift in priorities and delay project approvals. There was the concern that the lack of dedicated funding makes provincial funding more vulnerable to government freezes and cutbacks.
     
  • Project development and approval was cited as a complex process, which require significant administrative efforts. Each WEPA project needed to be approved by at least two government bodies (federal and provincial). The application process and project requirements may differ significantly among funders, which created a significant administrative burden on applicants. This burden includes eligible expenditures and funded budget lines which differ among funders.
     
  • Involvement of multiple players in the decision making process which can constrain coordination and communication. Coordination of activities with multiple stakeholders required additional time and effort. In particular, when the priorities and funding objectives of the two governments do not exactly align require significant coordination. This slowed down the process and increased bureaucracy in making decisions.

Other constraining factors most commonly identified at the project level included resource constraints related to difficulties in accessing other sources of funding to continue or expand project activities; limited interest from stakeholders and clients in participating in project activities or using outputs; internal issues related to project administration and management; the economic downturn which affected the business environment in the region; and issues with respect to meeting the product or certification standards. Constraining factors identified with respect to the design and implementation of WEPA is discussed in the following section.

Performance: Efficiency and Economy

This section reviews the findings of the evaluation regarding the efficiency and economy of the WEPA decision-making process.

1.  The WEPA decision-making process had both some benefits and challenges.

Government representatives were read a series of statements regarding the design of WEPA and then asked if they strongly disagreed, somewhat disagreed, neither agreed nor disagreed, somewhat agreed or strongly agreed with each. The responses are illustrated below.

Text Version, Figure 9: For each of the following statements would you like to know whether you strongly agree, somewhat agree, neither agree or disagree, somewhat disagree or strongly disagree.

In this figure, key-informants indicated provided their ratings of on a series of statements as to whether they strongly agree (rating of 1) neither (rating of 3), or strongly disagree (rating of 5).

Return to the top of this pagetop of page

The federal and provincial government representatives generally agreed that:

  • WEPA has a clearly defined governance structure. The members of the WEPA Management Committees tend to have been involved in the agreement for a number of years and, in some cases, for multiple generations of agreements. The management committees have developed administrative guidelines for the application, assessment, approval, reporting, implementation and monitoring of projects as well as measurement of results achieved.

    The time required to review and approve applications can be delayed by the need for parallel reviews (i.e. applications are reviewed by both the federal and provincial governments), the time that may be required to get a proposed project on the agenda of a management committee meeting, and the necessity to obtain further approvals once a project is recommended by the management committee.

    WEPA was well-structured given what it has been attempting to accomplish. The WEPA model contributed to effective government support for economic development by increasing understanding between the two orders of government. The WEPA model was viewed to have facilitated joint planning and decision-making within an area of joint responsibility. Most project proponents and a majority of proponents of applications who did not receive funding consider WEPA to be well-structured. The project proponents and other applicants indicated that WEPA decision-making process was supported by qualified project officers who provide useful guidance to proponents and were committed to successful implementation, and have an effective reporting and performance measurement mechanism which ensures accountability of results and achievement of objectives.

    Those who felt that WEPA was not well-structured focused primarily on the application and reporting process, which they felt was cumbersome and bureaucratic. In particular, the proponents and other applicants noted that the requirements often vary between the federal and provincial governments, and the process can take a long time and much effort to complete.
     
  • The intended outcomes and performance measures were clear, quantifiable and consistent with departmental reporting requirements. The performance measurement system had numerous strengths including:
     
    • Performance indicators were clearly defined in each contribution agreement and specific targets were established for each;
    • The inclusion of standard indicators allowed for the aggregation of certain data and the carrying forward of that data into the department’s performance reporting; and
    • Proponents demonstrated a strong commitment to collect and report on the performance indicators.

There were challenges in the WEPA decision making process. WEPA was viewed as flexible by federal government representatives, in large part because the federal priorities were defined quite broadly Provincial government representatives were somewhat less likely to view WEPA as flexible in responding to funding opportunities, citing some disagreements regarding the selection of actual projects within the priorities.

Several project proponents and other representatives who have been involved in previous rounds noted that WEPA III is somewhat less flexible than previous agreements. The proponents noted that restrictions regarding the types of expenditures that each government can fund and the allocation of project funding by line item has tended to become more rigid over time.

Some members of the management committee from the department expressed several challenges in the WEPA decision-making process that related to:

Priorities:

  • Requirement to negotiate defined priorities with the provinces which reduces the department’s ability to pursue evolving federal priorities.
  • The degree of generality that was necessary in the agreement which created a dynamic of broadly defined priorities rather than flexible, targeted approaches.
  • Engagement of industry under WEPA which was a priority for the department but not to the same extent for all provinces.

Administration:

  • The requirement for formal engagement with the provinces on the selection of projects, financial considerations, communications, and approvals which increased the complexity in many cases and, at times, delayed projects.

Communications:

  • The establishment of different communications protocol which often made coordinating announcements a challenge and negatively impacted federal visibility. Establishing equal representation for announcements was challenging in cases where proponents had a reporting relationship with the province.

2.  Departmental funding was necessary to the implementation of the projects. Without funding, most projects would not have gone ahead.

Although the projects received contributions from other sources, funding through the Western Diversification Program authority played a major role in the implementation of most projects. As demonstrated in the following figure, 72% of project proponents indicated their projects would not have gone ahead without the funding and an additional 10% mentioned that the likelihood were low (25% likelihood). These proponents indicated that their projects required significant investments and it would not have been possible for them to obtain the necessary funding from other sources.

Only 14% of the project proponents reported a significant likelihood that their project would still have gone ahead (50% likelihood or higher). On average, there was only 11% likelihood that the projects would have been implemented in the absence of any departmental funding.

Text Version, Figure 10: How likely is it that the project would have gone ahead even without the assistance under WEPA?

In this figure, 50 Project Proponents indicated how likely their projects would have gone ahead without assistance from WEPA.

The projects that reported very little or no likelihood of being implemented in the absence of any funding were proportionally distributed across provinces and industries. The proponents noted that the projects required significant investments and it would not have been possible for them to generate the necessary funding from other sources without the department’s support and endorsement. Those projects which may have still proceeded would have reduced in scope (e.g. certain important project activities would have been eliminated) or delayed.

Of the nine proponents of applications submitted to WEPA which did not receive funding, four (44%) indicated that their project are in the implementation phase, four (44%) reported that their projects were cancelled, and one (11%) is still looking for alternative funding. After applying unsuccessfully through WEPA, three proponents applied to other sources of funding to fill the budget gap. At the time of this evaluation, only one of those proponents was successful in accessing funds from other sources. Of the four projects which are underway without any funding, three reported a reduction in the project budget, a corresponding scale down of project activities, and delays to the project schedule and timing.

3.  The funded projects leveraged significant funds from other sources.

The budgeted costs associated with the 65 projects approved as of March 31, 2011 total $374 million, of which the departmental assistance represents 22%.

Table 10: Value of Departmental Assistance Approved as a Percentage of Total Project Costs, March 31, 2011
Province AB BC MB SK Total
Total Project Funding $ 70,196,724 $ 51,650,239 $ 86,733,511 $ 165,097,476 $ 373,677,950
Total WD Assistance $ 26,413,249 $ 12,802,917 $ 24,862,104 $ 19,347,495 $ 83,425,765
WD Funding/Total Funding 37.6% 24.8% 28.7% 11.7% 22.3%
Total Funding Per Dollar of WD Funding Provided $ 1.66 $ 3.03 $ 2.49 $ 7.53 $ 3.48

The budget data indicate that WEPA III projects leveraged $3.48 from other sources for every dollar contributed by the department, which is significantly higher than $2.24 leveraged under the previous WEPA II projects. The degree of leverage varies from $1.66 from other sources for every dollar contributed by the department in Alberta to $7.53 from other sources for every dollar contributed by the department in Saskatchewan. The leverage figures in Saskatchewan benefit from a large contribution from other federal government departments for one large project.

Table 11: Budgeted Sources of Funding for WEPA Projects as of March 31, 2011
Province AB BC MB SK Total
WD $ 26,413,249 $ 12,802,917 $ 24,862,104 $ 19,347,495 $ 83,425,765
Provincial $ 19,675,000 $ 18,994,754 $ 27,011,554 $ 21,104,288 $ 86,785,596
Other Federal $ 705,000 $ 9,334,000 $ 213,000 $ 103,958,000 $ 114,210,000
Other Non Stackable $ 19,337,816 $ 2,420,547 $ 28,786,823 $ 10,924,383 $ 61,469,569
Operating Revenue/ Working Capital $ 2,065,659 $ 10,489,271 $ 2,897,080 $ 8,155,005 $ 23,607,015
Equity $ 2,000,000 --$ $ 2,175,950 --$ $ 4,175,950
Other Stackable --$ --$ $ 250,000 $ 1,508,575 $ 1,758,575
Municipal --$ $ 140,000 $ 45,000 $100,000 $285,000
Total $ 70,196,724 $ 51,650,239 $ 86,733,511 $ 165,097,476 $ 373,677,950

Return to the top of this pagetop of page

Project proponents noted that departmental funding signals government support for particular projects, which helps to provide market credibility as well as attract additional private sector funding and participation.

The leveraging of departmental funding for WEPA projects is further extended if we consider the value of investments associated with follow-on projects and developments. As indicated in the previous section on project outcomes, the projects have generated at least $94.8 million in follow-on investments.

4.  The administration of WEPA was viewed as a cost-effective process.

When asked to rate how cost-effective the WEPA decision making process has been in implementing projects and producing its expected outputs and outcomes on a scale of 1 to 5, where 1 is not at all cost-effective, 3 is somewhat, and 5 is very cost-effective, provincial representatives provided an average rating of 4.6 and departmental representatives provided an average rating of 4.3.

Key factors which contributed to the cost-effectiveness include:

  • Key informants described the administrative and operational costs of the program as reasonable – although the department does not track the total cost of internal resources allocated to WEPA. Both Government representatives agreed with the statement that the administrative and operational costs of WEPA are reasonable. However, data on actual Full Time Equivalents administrative expenditures were not available as there were no dedicated staffs specifically to WEPA. Data is not tracked regarding the actual time that staff members spend on WEPA activities relative to other activities.
     
  • The decision making process was seen as having the resources needed to support effective implementation of approved projects. Government representatives agreed that WEPA process provided adequate resources and capacity to support effective implementation and achieve the intended outcomes of approved projects. According to the representatives, the funding allocated from the department to support implementation of WEPA projects was adequate. The timing of the cash flow was often coordinated between the two orders of government. For example, if one funder has expended their budget for the fiscal year while the other has funding left over, one partner may accelerate payments while the other delays payment on their share until the next fiscal year. On the other hand, the lack of dedicated funding from provincial governments in British Columbia and Alberta negatively affected delivery and created uncertainty.

5.  There are several options to fill the gaps in the absence of WEPA.

The comparative analysis demonstrated that even in the absence of intergovernmental agreements, governments can still work to facilitate communication and coordination by:

  • Engaging in regular communications such as consultations, meetings and information exchange to coordinate program activities with each other and avoid overlap in programming and duplication of efforts. The focus of this coordination can be on specific projects or be more general in nature;
     
  • Organizing advisory committees, where representatives of various stakeholder organizations and groups come together and review, discuss and advise on programming activities; and
  • Developing a strategic plan for the region or specific sectors. Such strategic plans could outline the development priorities for the region/sector and roles and responsibilities for each government involved in it.

6.  The major issues identified as affecting or potentially constraining the effectiveness of efficiency of WEPA as a decision making process include the time required for project approvals, the complexity of the project approval process, a lack of coordination in performance reporting requirements between the federal and provincial governments, and a lack of dedicated provincial funding in some provinces.

Based on results of key informant and project proponent interviews, case studies, focus groups and document and file reviews, a number of constraining factors have been identified which are discussed below:

  • The time required for project approvals can be long and unpredictable. The existing system was criticized both in terms of the length of time that can be required to obtain approval as well as the unpredictability of the time required, which can affect the credibility of project officers amongst applicants. There are a variety of application, project and proponent factors that can slow approval. It is very difficult for proponents to plan activities when timing is very uncertain.

    A review of project approval times indicates that the average length of time required for approval increased from 189 calendar days under WEPA II to 245 days under WEPA III. The following figure shows the distribution of the time required under WEPA II and WEPA III. As demonstrated in the table, about 50% of projects were approved in 160 days or less under WEPA II as compared to over 200 days under WEPA III.

Text Version, Figure 11: Time required for the project approvals.

This figure shows a line chart of the distribution of time required for WEPA II and WEPA III project approvals.

Return to the top of this pagetop of page

However, further analysis indicates that the difference in project approval times is largely attributable to an increase in the average size of projects. Larger projects tend to require longer time periods for approval. 29 of the 65 projects funded under WEPA III have involved departmental contributions of $1 million or more as compared to 30 of the 145 projects funded under WEPA II. The average time required to approve large projects has not changed (259 days under WEPA III as compared to 255 days under WEPA II).

  • Project development and approval is a complex process for applicants, which often require significant administrative efforts. The process is inherently complex in that WEPA projects need to be approved by at least two funding bodies (federal and provincial) and the application process and criteria may differ significantly between those bodies (requiring separate applications and processes). Sometimes applications receive approval from one order of government must wait several months or more to be approved by the other, which delays project implementation. Furthermore, the types of expenditures (e.g. certain missions or infrastructure) that the federal government and provincial government can and cannot fund vary between the two orders of government.
  • Performance indicators and reporting requirements may vary significantly between funders, adding to the administrative burden on applicants. The major weakness of the existing performance tracking system is that, in most provinces, performance indicators and reporting schedules are not coordinated between the federal and provincial government. This raises the possibility that proponents will need to track different indicators and produce separate reports for each order of government. Another weakness is that the performance indicators vary significantly from project to project which makes it difficult to meaningfully aggregate data on the performance of the agreement overall.
     
  • The lack of dedicated provincial government funding set aside for WEPA. The federal government had a dedicated budget through the Western Diversification Program authority. In Manitoba and Saskatchewan, the provincial contribution was formally approved by Treasury Board and established as a specific line item in the provincial budget. However, in Alberta and British Columbia, there is no dedicated funding but rather funding is secured on a project-by-project basis from a variety of ministries and programs. The lack of designated provincial funds added complexity to the project approval process and creates extra challenges for implementation. As evidenced in British Columbia, the absence of dedicated funding can make the agreement more vulnerable to shifting priorities, provincial cutbacks or budget freezes.

7.  There were a number of suggestions for improvement.

Key informants and project proponents provided suggestions and recommendations with respect to how a future departmental decision making process could be improved (also applicable to other departmental decision making process), which are summarized below.

  • Improve the application and review process by developing a more standardized structure for proposal documents; increase transparency by providing clearer guidelines on what information is needed and what criteria are used in reviewing applications as well as by providing clear feedback on the rationale for the decisions made; and providing for a more seamless review of applications between the two orders of government (e.g. have one department responsible for shepherding the proposals through the system). Consideration could also be given to issuing a formal call for proposals to complement the current system.
     
  • Reduce the time required for approval of applications (e.g. by devolving more of the funding decisions to the regional level) or at least provide a better indication of how long a decision may take. It was noted that the existing system is not very predictable.
     
  • Streamline the reporting requirements by developing a common group of performance indicators for each project, agreed to by the two orders of government and the proponent. Under such a system, proponents would not be required to report separately to the two sources of funding.
     
  • More narrowly define the priorities to better target the program funding and to ensure that projects are consistent with federal government priorities.
     
  • Ensure that the provincial governments establish a dedicated source of funding for any future agreements.
     
  • Consider varying the level of funding by province based on demographic and economic characteristics as well as geographic size. Larger provinces should be allocated more funding.
     
  • Establish a program advisory group. The planning process could be broadened to include input from industry as well from the federal and provincial government.
     
  • Develop mechanisms (e.g. meetings, events, and documentation) to better share lessons learned and best practices across projects, proponents, and government.
     
  • Communicate the success stories to raise awareness of WEPA. Both government and proponents should place a higher priority on finding opportunities (e.g. the results of research or specific activities) to get the word out about the successes of particular projects and the important role played by government in facilitating the development and implementation of those projects.

Representatives of other programs in western Canada noted that the key to successfully delivering this type of agreement is on-going and open communication with all program partners and stakeholders as well as having a flexible design and delivery mechanisms that allow changes to accommodate the needs of various organizations and projects.

 


[2] WD Value Capture Strategies in Key Sectors and Potential for Foreign Direct Investment in Western Canada.

[3] Canada West Foundation. An Extraordinary Future: A Strategic Vision for Western Canada.

[4] Conference Board Canada. 2009. How Canada Performs: A Report Card on Canada.

[5] The Conference Board of Canada. 2009. Western Canada Productivity, Competitiveness, and Potential

[6] Coalition for Action on Innovation. Action Plan for Prosperity, October 2010

[7] Boothe P. & Roy R. Business Sector Productivity in Canada: What do we know. International Productivity

[8] OECD Productivity Database

[9] Boothe P. & Roy R. Business Sector Productivity in Canada: What do we know. International Productivity Monitor

[10] Boothe P. & Roy R. Business Sector Productivity in Canada: What do we know. International Productivity Monitor

[11] Conference Board of Canada. 2011. Labour Productivity Growth
http://www.conferenceboard.ca/hcp/Details/Economy/measuring-productivity-canada.aspx

[12] Industry Canada. 2011. Key Small Business Statistics

[13] WD. 2011. Small Business Profile

[14] RBC, "Small and Medium-Sized Businesses are Driving Productivity Gains," October 2006

[15] BDC’s Entrepreneurial Insight, February 2008

[16] Canada Pacific Gateway. 2010. Asia-Pacific Gateway and Corridor Initiative Reaches A Milestone.

[17] Roslyn Kunin and Associates Inc. 2007. Final Report: Situation Analysis of Projected Asia-Pacific Gateway Investment in the Western Provinces with a Focus on Human Resources.

[18] British Columbia Ports Strategy.

[19] Conference Board 2004. Open for Business? Canada’s Foreign Direct Investment Challenge.

[20] United Nations Conference on Trade and Development. World Investment Report 2007.

[21] Conference Board 2004. Open for Business? Canada’s Foreign Direct Investment Challenge.

[22] WD Report on Plans and Priorities, 2011-2012.

[23] WD Report on Plans and Priorities, 2010-2011.

[24] Speech from the Throne. 3 June 2011, Ottawa, Ontario

[25] The funded projects are expected to be completed by September 30, 2013

Major Findings and Conclusions

The evaluation focused on two key components of WEPA: WEPA as a decision-making tool; and the economic development projects funded under WEPA using the WDP authority. The key conclusions arising from this review are as follows:

Relevance

1.  The need exists for economic development projects in western Canada. Economic development projects, like those that the department has funded under the Western Diversification Program authority, are still needed because they support critical areas of the economy. Programming focused on promoting research and development, commercialisation of new technologies, expanding investment, and supporting trade can help western Canada become more competitive and reduce its dependence on natural resources. The department can continue to fund similar projects in the future using its existing authorities, whether or not the WEPAs are renewed.

2.  The projects funded were aligned with departmental strategic outcomes and government-wide priorities, and were consistent with the priorities of the federal government. The department works to promote innovation, business development and community economic development in its mandate. The projects funded under WEPA contributed to each of these priority areas. The Speech from the Throne in 2011 highlighted the federal government’s commitment to supporting business development, productivity, innovation, trade, and economic competitiveness. All key informants agree that the projects funded were consistent with the federal government’s agenda in western Canada and departmental priorities.

Performance (Efficiency, Effectiveness and Economy)

3.  Although most of the projects are still ongoing, there has been significant progress made already towards meeting their objectives and generating expected impacts. At the time of this evaluation, 60% of projects were still on-going, 38% had only been recently completed and 2% had not started yet. That being said, the projects and the agreements overall are viewed as having made significant progress towards achieving their objectives. More specifically:

  • The WEPA decision making process has contributed in promoting collaboration and strengthening the partnership between the federal and provincial governments.
     
  • The projects funded have generated a wide range of impacts. Reflecting the large number of projects funded, the most significant impact was perceived to be in terms of strengthening innovation, improving business productivity, and promoting trade and market development.
     
  • Funded projects by the department have already made significant progress towards achieving their performance targets.
     
  • The impact of departmental funding extends beyond the direct impact of the funded projects to include the impacts of follow-on projects, investments, and developments. Fifty-two percent of project proponents (26 of the 50) reported that their project have already led to other projects, investments or developments. The proponents estimate that these developments will involve new investments of nearly $84 million. Proponents of spin-off projects reported that their projects generated a variety of impacts including attracting follow-on investments totalling $11 million.

4.  The WEPA decision making process had both some benefits and challenges. The key informants agreed that the WEPA decision making process had a clearly defined governance structure. It was well-structured given what it was trying to accomplish, the intended outcomes and performance measures were clear and consistent with departmental reporting requirements.

The major issues regarding the WEPA decision process include the time required for project approvals, the complexity of the project approval process, a lack of coordination in performance reporting requirements between the federal and provincial governments, and a lack of dedicated provincial funding in some provinces. Other issues arose related to the challenges in defining new priorities, administration of the process, and the establishment of communication protocols.

5.  The funded projects leveraged significant funds from other sources. The budgeted costs associated with the 65 projects approved as of March 31, 2011 total $374 million, of which departmental assistance represented 22%. In other words, WEPA III leveraged $3.48 from other sources for every dollar contributed by the department which is significantly higher than $2.24 leveraged under the previous WEPA II. Project proponents noted that departmental funding of projects signals government support, which helps to provide market credibility as well as attract additional private sector funding and participation.

6.  Key informants provided a variety of suggestions for improvement. Key informants suggested that consideration should be given to improving the application and review process, reducing the time required for approval of applications or at least providing a better indication of how long a decision may take, streamlining the reporting requirements, encouraging the provincial governments to establish a dedicated source of funding, and developing mechanisms to communicate success stories and better share lessons learned as well as best practices across projects and provinces.

In conclusion, the projects funded under WEPA align with the department’s priorities, they are demonstrating early positive results, and there is an ongoing need for economic development funding. The department has the existing program authorities to make similar investments in the future. WEPA, as a decision-making model, has some strengths and weaknesses. The department has some options as to what decision-making model it will choose to make its economic development project decisions in the future. Regardless of what model the department chooses, this report offers up some suggestions for improvement.

Date modified: