Western Economic Diversification Canada
Symbol of the Government of Canada

Common menu bar links

Performance: Achievement of Expected Outcomes

This chapter summarizes the key findings of the evaluation regarding the incrementally, outputs, immediate, intermediate and ultimate outcomes of CFP activities, as well as factors contributing to and constraining effectiveness. The chapter also examines the performance measurement activities of the CFP.

Incrementally

Most CF Managers and Chairs agreed that the absence of the CFs in their community would have a highly negative impact on start-up, survival, growth of new businesses, and community strategic planning and community economic development in their service area. About 95 percent of respondents indicated an absence of CFs would results in fewer start-ups, 83 percent indicated there would be fewer businesses growing and 81 percent indicated fewer businesses in operation. 62 percent of respondents indicated the absence of CFs would result in fewer community strategic planning and community economic development activities in their service areas.

The client survey noted that the absence of the CF in their community would negatively impact their businesses to a great extent as shown in the figure below.

Text Description, Figure 4: Proportion of clients who believe the absence of CF funding would have an impact on their businesses

In this figure, CF clients provided their opinion on the impact of the absence of CF funding would have on their businesses.

The majority of clients (70 percent) indicated that it was unlikely or very unlikely that they would have been able to start, maintain, or expand their business without the financing they received from the CFs. Most clients (76 percent) indicated that it was very likely that they would have delayed starting or expanding their business if they had not received a CF loan. About 53 percent of clients indicated that it was very likely their businesses would be smaller if they had not received the CF loan. Clients also indicated that they would likely have reduced the quality of their services or been unable to secure funding from other sources in the absence of a CF loan.

These findings are consistent with a 2009 research study of the Canada Small Business Financing Program (CSBFB).15 The study concluded that loan recipients under the CSBFB scored more poorly on indicators of credit worthiness than counterparts who did not receive loans from the program. This would affect their ability to secure loans in the absence of the program and thus impact negatively on business performance.

The CF Partners surveyed for the evaluation supported the findings and suggested that a lack of CF funding would negatively impact businesses to a great extent as shown in the figure below.

Text Description, Figure 5: Proportion of CF Partners who believe the absence of CF funding would have an impact on businesses.

In this figure, CF Partners surveyed provided their opinion on the impact of the absence of CF funding would have on businesses in their communities.

The majority of key informants (63 percent) stated that the absence of CFP funding would have a significant negative impact on their community. More specifically, respondents identified the impact as reduced access to capital and business development services for start-ups.

Outputs and Immediate Outcomes

The immediate outcomes as identified in the logic model for the CFP are:

  • More effective implementation of community economic development (CED) through projects, partnerships and CED initiatives. Improved access to capital;
  • Enhanced business services;
  • Strengthened community strategic plans; and
  • More effective implementation of community economic development (CED) through projects, partnerships and CED initiatives.

The analysis of performance data from the CFs provided information on the achievement of outputs and immediate outcomes.

Improved Access to Capital

As shown in the table below, the CFs issued a total of 3,533 new loans during the period of the evaluation. The highest numbers of new loans were issued in the 2008–09 to 2009–10 fiscal years. This may be reflective of the greater demand for access to capital as a result of the global economic recession. Compared to the findings in the 2008 evaluation, the average number of loans per CF per year has risen from 15.8 in 2002–07 to 16.8 in the current evaluation period. The average size of loans remained fairly constant over the period. The highest value of loans was realised in the 2009–10 fiscal year.

Summary of the CFP Loans by Fiscal Year
Fiscal Year Number of New Loans Number of Existing Loans Total Number of Loans Average Number of Loans Value of Loans Average Size of Loan
2008–09 746 771 1,517 16.9 $77,875,409 $51,335
2009–10 761 838 1,599 17.8 $79,354,060 $49,627
2010–11 651 823 1,474 15.7 $71,430,285 $48,460
2011–12 652 791 1,443 16.0 $67,680,302 $46,903
2012–13 723 874 1,597 17.7 $78,728,223 $49,298
Average/year 707 819 1,526 16.8 $75,013,656 $49,157

Enhanced Business Services

A total of 284,691 clients (average of 633 clients per CF per year) accessed business development services from the CFs over the period of the evaluation. As shown in the table below, a greater number of clients accessed CF business development services in the fiscal years 2008–09 to 2009–10, which may have been a result of the global economic recession. This is consistent with the greater number of new loans issued by the CFs in that year. The table also shows the various types of services accessed by clients.

Summary of CFP Business Development Services
Fiscal Year Number of Clients Number of Advisory Services Number of Information Services Number of Website Information Services Number of Training Sessions Number of Training Session Participants
2008–09 66,931 41,238 193,414 923,064 4,008 28,370
2009–10 66,840 50,237 225,065 733,907 4,473 31,594
2010–11 58,531 45,908 271,544 789,959 4,049 27,580
2011–12 58,749 43,545 255,057 746,871 3,402 32,866
2012–13 33,640 31,153 202,348 800,682 2,819 19,352
Average/CF
per year
633 471 2,550 8,877 42 311

Results from the client survey indicated that clients were selective in accessing specific business development services offered by the CFs. The services respondents reported accessing the least often were referrals to other organizations (10 percent) and business skills training services (16 percent).

More effective implementation of CED through projects, partnerships and CED initiatives

Information from the performance data indicates that the CFP averaged 34.3 CED projects per CF per year over the period of the evaluation. This is higher than the 2008 CFP evaluation which indicated an average of 17 CED projects per year. The performance data indicated the number of community strategic plans to be an average of 4 per CF per year which is consistent with the 2008 CFP evaluation where CFs reported being involved in the development of 1 to 4 community strategic plans per CF per year. The number of partnerships increased from an average of 30 per CF per year (2008 Evaluation result) to an average of 38.4 partners per CF per year within the period of the current evaluation.

Summary of Community Strategic Plans and Community Economic Development Activities
Fiscal Year Number of CED Projects Number of Projects of Leadership and/or Expertise Number of Community Plans Number of Partnerships Developed and Maintained
2008–09 2,721 3,523 387 3,360
2009–10 2,958 3,392 367 3,535
2010–11 3,247 3,343 341 3,423
2011–12 3,202 3,271 352 3,626
2012–13 3,292 2,987 340 3,336
Total 1,5420 16,516 1,787 17,280
Average/year 34.3 36.7 4.0 38.4

Community strategic plans involve activities where the CF led, facilitated or participated in the development of community plans. Case study respondents indicated that CFs are consistently called upon by the by the community to help in the developments of community plans. Community economic development projects included events such as youth camps, trade shows and school achievement projects. An example of such an event is the Youth Business Excellence Awards hosted by the Community Futures Newsask in Saskatchewan. The CF has been hosting this event for the past twelve years to recognise the achievements of young entrepreneurs in Northeast Saskatchewan.

The CF partner survey described the types of partnership engagements with the CFs. Partnership projects spanned a range of sectors. The following describes the nature of the projects:

  • Promotion of the community for tourism or investment;
  • Sponsorship of business management and entrepreneurial training courses and seminars;
  • Development of infrastructure to support economic development;
  • Special initiatives to support entrepreneurship among specific groups such as women, youth; Aboriginal people and official language minorities; and
  • Support for communities when there is a downturn in the economy.

Partners reported benefits of partnership with the CFs, which included enhanced linkages between partners to address community needs, increased sharing of knowledge and experience, added capacity to service delivery to extend reach and accessibility of services and increased coordination of services for clients.

Return to the top of this pagetop of page

Intermediate Outcomes

The intermediate outcomes identified in the logic model for the CFP are improved business practices and increased entrepreneurship, strengthened and expanded businesses and strengthened capacity for socioeconomic development.

Improved Business Practices and Increased Entrepreneurship And Strengthened and Expanded Businesses

The table below indicates the number of businesses assisted by the CFs through lending activities and business services reported by the CFs in the performance database. The numbers indicate an increasing trend in the number of businesses assisted through lending activities and a decreasing trend in the number of businesses assisted through business services.

Summary of Number of Businesses Served by CFs
Fiscal Year Number of Businesses created/maintained/expanded through Lending Activities Number of Businesses created/maintained/expanded through Business Services
2008–09 1,272 1,447
2009–10 1,515 1,388
2010–11 1,438 1,239
2011–12 1,493 1,191
2012–13 1,478 792
Total 7,196 6,057

The increasing trend in lending activities of the CFs is consistent with Industry Canada’s research on SME financing16. As shown in the figure below from the report, as the economy started to recover in 2010, credit request rates for SMEs increased to 21 percent, up from 16 percent in 2009.

Text Description, Figure 6: Small business financing request rates

The figure shows the financing request rates for small businesses from 2000 to 2010.

The Industry Canada research report on SME credit conditions17 states that the request rate for start-up and young businesses (2 years old or younger) at 36 percent was significantly higher than for older more established businesses (23 percent). However, the approval rate was much lower at 79 percent versus 91 percent respectively. This indicates higher financing needs for start-up and for younger businesses, which usually lack retained earnings, solid credit histories, and collateral to pledge for financing from traditional sources. Such start-up and young businesses may therefore seek out programs such as the CFP which provide loans to more risky clients.

CF clients surveyed for the evaluation indicated that the CF services they received improved their business skills and helped them keep the business open. About 15 percent of CF clients surveyed indicated that the services they received contributed to increasing their revenue and improving their business productivity. These clients reported that on average their sales revenue increased from $234,000 in 2008–09 to $436,000 in 2012–13.

Strengthened Capacity for Socioeconomic Development

The CF performance data also provides an indication of the level of strengthened capacity for socioeconomic development through community economic development activities (table below)

Summary of outcomes in CED
Fiscal Year Number of Instances Facilitating Community Involvement Number of Instances of Increased Capacity Number of Development of Economic Stimuli Number of Enhanced Community Services/Facilities
2008–09 899 936 533 405
2009–10 974 996 534 450
2010–11 905 1,032 565 492
2011–12 968 1,013 647 554
2012–13 1,034 1,007 513 613
Total 4,780 4,984 2,792 2,514

Facilitating community involvement includes CF facilitated events such as town hall meetings and public strategic planning sessions which contribute to strategic planning. Increased capacity involves activities such as workshops, regional roundtables and mentorship programs. Economic stimuli involve activities such as development of tourism marketing strategy, development of business plan for business incubators and establishment of cluster agreements. Enhanced community services/facilities include physical infrastructure and cultural/recreational infrastructure.

The surveyed CF Managers and Chairs reported that CF involvement in their core activities contributed to each of the immediate outcomes to a great extent. The most significant impacts were reported for improved business practices and increased entrepreneurship and strengthened and expanded businesses at 97 percent. Impacts on strengthened capacity for socio-economic development were reported to a lesser extent at 71 percent. Most CF partners surveyed indicated that their partnership with the CF in their community contributed to strengthened capacity for socio-economic development (75.4 percent), strengthened and expanded businesses in the community (73.3 percent) and improved business practices and increased entrepreneurship activities (72 percent).

Ultimate Outcomes

The ultimate outcomes of the CFP from the logic model are economic stability, growth and job creation; diversified and competitive local and rural economies and economically sustainable communities. These outcomes link to the departmental outcomes of having western Canadian communities that have strong businesses, capacity for socioeconomic development and the necessary public infrastructure to support economic growth. The objectives of the CFP therefore contribute to the departmental strategic outcome of a growing and diversified western Canadian economy.

Using business tracking data compiled by Statistics Canada, the long-term performance of CF-assisted firms was compared in terms of employment growth, business survival rate and revenue growth to a group of comparable firms that did not receive assistance. The data was grouped by sector, using the North American Industry Classification System (NAICS)18 codes and/or firm size using Average Labour Units (ALUs).19 The crude petroleum and liquid natural gas extraction sectors, the utilities sector and the finance and insurance sector were excluded from the analysis. Larger enterprises with more than 100 ALUs are also excluded. This is to ensure that that the data set captures the results from SMEs in the CF service areas.

Analysis of the data demonstrates that CF activities in providing access to capital and business development services were making a difference as CF-assisted firms perform better than the comparable group in terms of employment growth and revenue growth in addition to having a better business survival rate after five years of entry into business. The data also shows that CFs assist firms in a variety of industry sectors. See the rest of this section for details.

Key informants and survey respondents agreed that CFP activities contributed to the ultimate outcomes of the program. Key informants indicated that the CFP also had a greater impact on economic stability, growth and job creation than it contributed to economically sustainable communities or a diversified and competitive local rural economy. Survey findings suggested that the CFP contributed to growth and job creation (88 percent), economic sustainability (79 percent), economic stability (78 percent) and diversification and competitiveness (66 percent). The lower rating for impacts in economically sustainable communities or a diversified and competitive local rural economy may be partially attributable to the fact that these outcomes are long-term.

Size and Concentration of CF-assisted and Comparable Firms

The table below presents the distribution of firms by industry for the reference year (2010) for both CF-assisted firms and the firms from the comparable group. The table demonstrates the diversity of industries that CF-assisted firms are involved in and the relative importance of such industry.

Share of Firms by Industry for Western Canada 2010
Industry Enterprise size (ALU) CF-assisted Firms Comparable Firms
Total (All selected NAICS) All sizes 100.0% 100.0%
Less than 20 ALU's 95.4% 96.6%
20 to 100 ALU's 4.6% 3.4%
Agriculture, Forestry, Fishing and Hunting All sizes 7.0% 11.7%
Mining and Oil and Gas All sizes 1.6% 3.1%
Construction All sizes 10.3% 16.6%
Manufacturing All sizes 8.8% 3.4%
Wholesale Trade All sizes X 3.3%
Retail Trade All sizes 19.1% 9.6%
Transportation and Warehousing All sizes 8.1% 6.3%
Information and Cultural Industries All sizes 1.8% 0.7%
Real Estate and Rental and Leasing All sizes 1.4% 4.7%
Professional, Scientific, and Technical Services All sizes X 13.0%
Management of Companies and Enterprises All sizes X 1.8%
Administrative and Support, Waste Management and Remediation Services All sizes X 5.2%
Arts, Entertainment, and Recreation All sizes X 1.8%
Accommodation and Food Services All sizes 11.8% 6.7%
Other Services All sizes X 12.2%

(X) Data suppressed for confidentiality purposes
Source: Business Special Surveys and Technology Statistics Division, Statistics Canada, 2013

Nearly all CF-assisted firms (95.54 percent) were small businesses with less than 20 employees. Comparable firms (96.6 percent) were mostly small business also with less than 20 employees. The table indicates the diversity of industry sectors that CF-assisted firms are involved. Retail trade comprises the largest share of CF-assisted firms (19.1 percent) followed by Accommodation and Food Services (11.8 percent). Construction (10.3 percent), Manufacturing (8.8 percent) and Transportation and Warehousing (8.1 percent) also rank high.

The Herfindahl Index (HI) is a measure of economic diversity as it indicates the extent to which a regional economy is dominated by a number of industries. The index ranges from 0 where several industries have small and equal shares of the market economy to 100, where one industrial sector has perfect market composition (full specialization). Thus, a lower HI measure indicates a highly diversified economy. In 2010, the HI for the group of CF-assisted firms measured 10.47. This was a slight decrease from previous years, where the index measured 10.55 and 10.52 for the years 2008 and 2009 respectively. Over three years, the HI has been trending downwards, demonstrating higher economic diversity amongst CF loan clients.

Return to the top of this pagetop of page

Employment Growth

The table below shows the compounded annual employment growth rate for CF-assisted firms over a five-year period and compares the results to a group of comparable firms. The following table presents the results of net job creation from 2005–10.

Compounded Annual Growth in Employment by Industry in Western Canada 2005–10
Industry Enterprise size (ALU) CF-assisted Firms Comparable Firms
Total (All selected NAICS) All sizes 8.2% 3.9%
Less than 20 ALU's 9.5% 4.2%
20 to 100 ALU's 5.4% 3.0%
Agriculture, Forestry, Fishing and Hunting All sizes 8.7% 1.7%
Mining and Oil and Gas All sizes X 9.7%
Construction All sizes 7.2% 5.8%
Manufacturing All sizes 6.3% 2.2%
Wholesale Trade All sizes 0.8% 2.2%
Retail Trade All sizes 10.3% 3.7%
Transportation and Warehousing All sizes 10.1% 2.9%
Information and Cultural Industries All sizes 10.5% 2.4%
Real Estate and Rental and Leasing All sizes -3.1% 3.7%
Professional, Scientific, and Technical Services All sizes 7.6% 4.4%
Management of Companies and Enterprises All sizes X 0.5%
Administrative and Support, Waste Management and Remediation Services All sizes 7.5% 5.2%
Arts, Entertainment, and Recreation All sizes 3.3% 3.9%
Accommodation and Food Services All sizes 9.6% 4.3%
Other Services20 All sizes 9.8% 2.4%

X) Data suppressed for confidentiality purposes
Source: Business Special Surveys and Technology Statistics Division, Statistics Canada, 2014.

The table shows that between 2005 and 2010, CF-assisted firms were much more successful in creating jobs than the group of comparable firms. CF-assisted firms grew by an average of 9.5 percent annually compared to 4.2 percent for non-assisted firms. The figure below shows the industrial sectors where the difference between CF-assisted firms and non-assisted ones was particularly large.

Text Description, Figure 7: Compounded annual employment growth in Western Canada for firms with less than 100 ALUs (2004–09)

This figure shows the difference between the compounded annual employment growth rates for CF-assisted firms versus non-assisted firms.

Business Survival Rates

Analysis of the data from Statistics Canada indicates that CF-assisted firms were more likely to have survived following the fifth year of entry into business than comparable firms. The business survival rate for CF-assisted firms was 76 percent after five years following start-up compared to 60 percent for comparable firms. The figure below shows the survival rate by year for start-up firms.

Text Description, Figure 8: Year-Over-Year Sales Growth for Firms Across all Industries in Western Canada with less than 100 ALUs (2005–10)

This figure shows the difference between the compounded annual employment growth rates for CF-assisted firms versus non-assisted firms.

For both groups, total sales rose consistently from 2005 to 2008. However, in 2009 total sales for comparable firms declined by over 13 percent from the previous year, while total sales for CF-assisted firms declined by 0.3 percent. Sales figures for comparable firms grew at approximately the same rate as CF-assisted firms between 2009 and 2010. In general, it appears that CF loan clients were able to withstand shocks to economic activity better than those firms without CF assistance.

Revenue Growth

Analysis of the data from Statistics Canada show that CF-assisted firms had a compounded annual growth rate of 13.8 percent growth rate in sales as compared to a 6.1 percent growth rate in sales for comparable firms. The figure below shows the year over year sales growth for CF-assisted firms and comparable firms from 2005–10. Between 2001 and 2011, CF-assisted firms outperformed comparable firms in terms of sales growth.

Text Description, Figure 9: Year-Over-Year Sales Growth for Firms Across all Industries in Western Canada with less than 100 ALUs (2005–10)

This figure shows the difference between the business survival rates for CF-assisted firms versus comparable group of non-assisted firms from 2000 to 2010.

For both groups, total sales rose consistently from 2005 to 2008. However, in 2009 total sales for comparable firms declined by over 13 percent from the previous year, while total sales for CF-assisted firms declined by 0.3 percent. Sales figures for comparable firms grew at approximately the same rate as CF-assisted firms between 2009 and 2010. In general, it appears that CF loan clients were able to withstand shocks to economic activity better than those firms without CF assistance.

Contributing and Constraining Factors to Success

Key informants, survey and case study respondents identified several barriers to the achievement of outcomes. The barriers most commonly identified by key informants include:

  • The limited amount of funding and resources available to CFs which impact the capacity of CFs to undertake projects in community economic development;
  • The length of the contract between CFs and the department (one-year as opposed to multi-year);
  • The limited marketing and outreach conducted by CFs to promote their activities;
  • Loan limits set by the CFP; and
  • The lack of skilled workers especially in Alberta and Saskatchewan.

In terms of mitigation strategies, WD key informants representatives referred to the CFP revitalization exercise which is expected to promote further uptake of training of CF staff and board members and strengthen the delivery of the CFP. The department has set new targets for value and number of loans approved so that the CFP will have a more measurable impact. As part of the revitalization exercise, the department is working on identifying opportunities to achieve economies of scale (e.g., through sharing of resources) for the efficient delivery of the program. Some CFs mentioned having taken steps to share resources such as using the same loans officer or CF managers.

Performance Measurement

As mentioned earlier, the CFs use The Exceptional Assistant (TEA) software, hosted by CommonGoals to report data on a quarterly and/or annual basis. The client survey required that the CFs generate a list of their clients over the period of the evaluation from this software. Several challenges were encountered as accessing data was cumbersome for the CFs. The Evaluation Team encountered some resistance from some of the CFs as a result.

Data from TEA are reported into a database hosted by CommonGoals for use by the department. Information from this database could not be easily imported to common software programs used by the department for analysis in an aggregate format. The 2008 Evaluation recognised this issue and suggested that the department devise a strategy to resolve system inconsistencies and data definitions.

Based on recommendations from the 2008 Evaluation, the department, together with the other Regional Development Agencies, worked on implementing the national Performance Measurement Strategy (PMS). The work with Statistics Canada on gathering data to address the long-term outcomes of the CFP was implemented as part of the PMS. This is recognised as a best practice by the Evaluation Team.

Key informants noted that reporting was not consistent across CFs due to lack of well-defined definitions and the subjectivity of some of the indicators. Some respondents mentioned that there were too many indicators to allow for effective reporting. With respect to the use of performance information, the survey identified areas CFs commonly use performance information including reporting to partners, clients, funders and the community, as well as in promotional materials (e.g., website, newsletter and proposals presentations).

Some WD representatives interviewed indicated that the department has addressed issues regarding data validity and reliability by streamlining indicators and improved definitions, which have been embedded into TEA. The WD respondents anticipated this will contribute to more timely and useful performance information.

 


[15] Canada Small Business Financing Program: Updated Analysis of Incrementality. Allan Riding, Canada Works Ltd, 2009

[16] Small Business Access to Financing: Request and Approval Rates, Interest Rates and Collateral Requirements. Published by Industry Canada on July 2013

[17] Credit Condition Trends 2009–2012. Published by Industry Canada in November 2003 http://www.ic.gc.ca/eic/site/061.nsf/eng/02829.html

[18] The NAICS is an industry classification system developed for the North American Free Trade Agreement (NAFTA). It provides common definitions of the industrial structure and a common statistical framework to facilitate the analysis of the economies of the countries of NAFTA.

[19] The ALU is calculated by dividing a firm's annual payroll by the average annual earnings of a representative worker in the same industry.

[20] Includes service industries that support oil and gas extraction.