Western Economic Diversification Canada
Symbol of the Government of Canada

Common menu bar links

Section 5: Performance: Demonstration of Efficiency and Economy

5.1 Economy

Efficiency Indicator

In 2013-14, WD implemented a revised Program Alignment Architecture (PAA) with Business Productivity and Growth as a separate sub-program within the Business Development and Innovation program. This revision to the PAA enabled the department to track operating expenditures related to the delivery of programming for Business Productivity and Growth. Since then, the department also introduced a common efficiency indicator at the sub-program level - “operating costs per $1,000 in Grants and Contributions (Gs&Cs) expended”.

With respect to the Business Productivity and Growth sub-program, the total Gs&Cs invested by the department was $23,390,759 in 2013–14 with $3,060,317 in operating and maintenance (O&M) expenditures. This results in an efficiency indicator of $131. The department therefore spends roughly 13.1% of its budget to deliver $1000 of programming in Business Productivity and Growth. Going forward, the department will be able to report on trends related to this new indicator.

O&M expenditures include staff costs related to G&C project management and the cost of activities related to stakeholder engagement and pathfinding. Although full-time equivalent (FTE) data by sub-program was not available for each year, FTEs dedicated to Business Productivity and Growth decreased from 43 (2009–10) to 39 (2012–13) and 29 (2013–14). For the 2013–14 fiscal year, G&C expenditure per FTE was $806,578 and O&M expenditure per project approved was $35,176.

Comparison to other programming.

WD’s efficiency indicator for the Business Productivity and Growth sub-program compares favourably to other sub-programs under WD’s Business Development and Innovation program. Efficiency indicators for sub-programs ranged from 2.2% (Innovation Capacity Building) to 13.1% (Business Productivity and Growth) to 14.3% (Technology Commercialization) to and 48.2% (Trade, Investment, and Market Access). Sub-program cost differences are primarily attributable to variations in G&C expenditures (for example, Business Productivity and Growth projects are usually of lower dollar value, however, the project assessment, management and stakeholder engagement can require the same level of effort as for much higher valued innovation projects).

Key informants and survey respondents were also asked for their opinions on programming economy. Seventy-nine percent of project proponents and 37% of representatives of unfunded projects agreed that the Business Productivity and Growth projects provide good value for money; 83% of project proponents could not identify opportunities to generate the intended outputs and outcomes at a lower cost. Of the 57% of key informants that responded, most agreed that the programming provides good value for money with respect to the use of public funds. Forty-two percent of representatives of unfunded projects did not know or did not answer the question on value for money.

Design of the programming.

Among key informants who were aware of the design of the department’s support for Business Productivity and Growth activities, most agreed that it was appropriate (i.e., target groups, types of activities funded, delivery mechanisms). Focus group participants were also generally satisfied with the approval, monitoring, reporting and claims processes. However, participants expressed some concern about the department’s recent change in the proposal intake process from continuous to call for proposals (CFP) process. Although the new CFP process improved response times, participants raised a few disadvantages in terms of unpredictable intake periods and more limited communication between staff and applicants. Another issue raised in the focus groups related to limitations with respect to transferring project budgets from one year to the next given annual budget allocations.

Although the department is addressing client needs, a few key informants suggested design and delivery improvements such as strengthening communication with proponents regarding program offerings (requirements, priorities and approval processes) (9 key informants) and strengthening and leveraging partnerships in delivering programming (6 key informants). The focus group discussions led to a variety of suggestions including supporting projects which promote diversification and strategic investments and developing new mechanisms of support (e.g., for smaller projects in the $100,000 range).

5.2 Efficiency

Leverage Rate

A total of $90.3 million in departmental G&C funding was approved for the 87 projects and $150.6 million was leveraged from other sources, yielding a leverage rate of $1.67 leveraged per departmental G&C dollar of funding approved. This leverage rate compares favourably with several similar federal programs but is lower than for others24 as illustrated in the table below.

 

Table 5.1. Dollars Leveraged Per Dollar G&C Approved Compared to Other Similar Programs
Organization Program $ G&C Approved $ Leveraged from Other Sources $ Leveraged Per
$ G&C Approved
Business Development and Growth Programs
WD Business Productivity and Growth $90,305,313 $150,612,063 $1.67
CedQ25 Business and Regional Growth Program $231,990,752 $911,723,655 $3.93
FedNor Northern Ontario Development Program $148,000,097 $409,548,680 $2.77
WD Western Diversification Program $219,897,65226 $399,155,151 $1.82
Trade and Investment Programs
WD Trade and Investment Sub-Program $39,436,84027 $56,009,170 $1.42
ACOA Trade and Investment Program $217,918,002 $194,520,911 $0.89
Innovation Programs
WD Innovation Activities $306,000,00028 $764,000,000 $2.50
Business and Employment Skills Programs
ACOA Entrepreneurship and Business Skills Development $51,934,688 $61,092,361 $1.18
ESDC Aboriginal Skills and Training Strategic Investment Fund $40,200,000 $12,300,000 $0.31

 

Return on Investment

WD’s return on investment compares favourably with similar federal programs reviewed. WD projects that reported results at the outcome level during the study period cost $7,953 in G&C funding per business created, maintained, or expanded and $2,206 per job created or maintained; these projects also generated $26.00 in increased sales per dollar cost. These figures compare favourably to cost per job estimates for FedNor’s Northern Ontario Development Program ($5,229 in G&C cost per job created or maintained29 ) and for the Federal Economic Development Agency for Southern Ontario (FedDev Ontario) CME-SMART Program, which provides assistance to businesses to undertake activities to improve their productivity and competitiveness, ($7,751 per job created and $3,545 per job maintained30).

Key informants and survey respondents were asked for their opinions on programming efficiency. Forty-two percent of proponents and unfunded project representatives had no suggestions for improving the programming efficiency or economy. Thirteen key informants regarded the programming financial and staff resources as commensurate with its objectives. Most key informant respondents perceived that the resources allocated to Business Productivity and Growth were being used efficiently (i.e., delivery costs are minimized) since funding was leveraged, the due diligence process is rigorous and staff costs have already been minimized. A few suggestions were provided to further improve efficiency such as allowing regional approval for smaller projects and easing rules around types of expenditures covered. Key informants provided a few suggestions with respect to alternative models or methods that would result in more efficient delivery of the activities, such as leveraging federal, provincial, and industry partners and other third parties in delivering programming. A few survey participants suggested improvements to the timeliness and communication of decisions and the department’s online presence (i.e., claims reporting).

5.3 Quality and Usefulness of Performance Measurement

The Performance Measurement Strategy for the Western Diversification Program Authority (WDP) outlines the performance measures, accountability and evaluation strategy. There is also a Data Collection Strategy for the WDP that defines each performance indicator and measurement approach which, for the Business Productivity and Growth performance indicators, involves program officers entering the data into the departmental database (Project Gateway). The performance measurement team routinely checks and corrects for data entry errors as part of their reporting process.

Access to accurate data is, at times, difficult given the challenges of the department’s project management system. However, this problem should improve as the department moves to a new, more accessible system.

In terms of data completeness, the database contained at least one sub-program level Program Alignment Architecture (PAA) indicator for each project. At the program level, however, only four projects identified a PAA indicator. There were 165 unique indicators, which cannot be captured systematically and used for the purposes of evaluation or decision-making. For example, almost one quarter (39) of the unique indicators pertained to number of people trained or job creation, however, because the indicators were made very specific (e.g., highly qualified personnel, Aboriginal, students), this information cannot be generated by the system or captured at the departmental level.

The majority of surveyed project proponents (71%) agreed that the performance measures and reporting requirements set by the department were appropriate. Key informants, and particularly proponents, agreed that the performance measurement system was effective in tracking and reporting on outcomes. Some departmental key informants noted that the performance measurement data captured by the PAA indicators has been useful for decision-making, particularly in terms of informing investment decisions and risk management, undertaking departmental reporting and planning and identifying needs. Focus group participants noted that attribution can be a challenge since it is difficult to isolate project impacts resulting from the department’s funding from what would have occurred in the absence of funding. It is also difficult to capture longer-term impacts since some impacts do not occur until well after the project is complete.

5.4 Risk Management

The department assesses individual project risk as part of the due diligence process. There is also a “Control Based Monitoring (CBM) Framework” for the Western Diversification Program. This CBM Framework outlines a risk based approach to manage projects for compliance to the Contribution Agreement and for maintaining appropriate financial controls. Risks are monitored throughout the lifecycle of the project and the adequacy of the controls, last assessed in 2011, were found to be appropriate.

Most departmental representatives perceived that an appropriate risk management strategy exists and that the main risks related to projects not achieving results or not being implemented as planned. Risks are addressed primarily through monitoring and due diligence processes. The department also tends to fund low risk projects. Risk ratings in the due diligence reports for the 87 projects showed 68 (78%) were low risk, 17 (20%) were moderate risk and two were high risk.

 


[24] Includes the following evaluations: 1) Economic Development Agency for the Regions of Quebec (CedQ). 2012. Summative Evaluation of the Business and Regional Growth Program; 2) Industry Canada. 2011. Final Evaluation for the Northern Ontario Development Program; 3) Western Economic Diversification Canada. 2008. Evaluation of the Western Diversification Program; 4) Western Economic Diversification Canada. 2013. Evaluation of Trade and Investment Activities; 5) Atlantic Canada Opportunities Agency. 2010. Evaluation of the Atlantic Canada Opportunities Agency Trade and Investment Program Sub-activity; 6) Western Economic Diversification Canada. 2012; Evaluation of the Innovation Activity; 7) Atlantic Canada Opportunities Agency. 2010. Evaluation of the Atlantic Canada Opportunities Agency's Entrepreneurship and Business Skills Development Program Sub-activity; 8) Employment and Social Development Canada. 2013. Evaluation: Aboriginal Skills and Training Strategic Investment Fund.

[25] CedQ: Economic Development Agency for the Regions of Quebec.

[26] Only includes programs that leveraged funds: AB/SK Centenaries and Canada Celebrates SK (ASC/CCS), Western Economic Partnership Agreements (WEPA), Urban Development Agreements (UDA) in Regina, Saskatoon, Winnipeg, Vancouver, and Canada/SK Northern Development Agreement.

[27] Includes: Conference Support Payments (2 projects), WEPA Round III (2008-12) (16 projects), WDP (41 projects), and WEPA Round II (2003-08) (4 projects).

[28] Includes: WEPA Round II and III, and WDP.

[29] Industry Canada. 2011. Final Evaluation for the Northern Ontario Development Program. The evaluation stated: "With respect to NODP costs per outcome, 23 of the 40 projects reviewed as part of the file review resulted in temporary and/or permanent jobs created and/or maintained. Those 23 projects created or maintained a total of 1,637 temporary and permanent jobs at a total NODP funding of $8,560,499, and a total project cost of $30,138,317. Therefore, based on the sample files, the cost to the NODP per temporary and permanent job created and maintained was $5,229."

[30] Federal Economic Development Agency for Southern Ontario. "Interim Evaluation of FedDev Ontario Programs. 2012. Cost per job calculated based on $38.85 million in FedDev Ontario contributions for the CME-SMART Program from 2009 to 2012 which resulted in 5,012 jobs created and 10,960 jobs maintained.