Western Economic Diversification Canada
Symbol of the Government of Canada

Common menu bar links

Performance (Achievement of Outcomes)

The objective of the EDP is to provide western Canadians with disabilities with easy access to business information, training and development, mentoring and one-on-one counselling services, and financing in their pursuit of self-employment and entrepreneurship. In general, the interview data indicate that entrepreneurs with disabilities are benefiting from the program, are satisfied with the program and are achieving success as a result of the program. After the 2005 evaluation of the program, a series of performance indicators were developed to capture the results of the program. The link between these performance indicators and current departmental Program Activity Architecture outputs and outcomes is not clear. Urban Delivery Agents reported on the number of business created/expanded/maintained; however, this indicator was not reported by the Community Futures Organizations. In addition, some of the indicators such as number of hours spent in coaching activities, are not linked to any of the outcomes in the Program Activity Architecture.

Some of the Community Futures Organizations are involved in delivering other initiatives for the provincial and federal governments; as a result they indicated that data in the database on the performance of the EDP includes results from other initiatives. Focus groups participants validated that client counting has been very inconsistent from region to region. In addition, the operation of the EDP has been integrated into the normal operations of the delivery agents in most areas. In British Columbia, urban EDP operations have been integrated into the operations of the Loan and Investment Program. Other Urban Delivery Agencies and the Community Futures Organizations have largely integrated the EDP loan funds with their core loan portfolios. Because of the issues with data integrity, it is exceedingly difficult to assess actual performance indicators of the EDP. For future renewal or extension of the EDP, the department should develop clear quantitative performance indicators for the program.

Factors that Impeded the Measurement of Results

The most common challenges identified by key informants on the achievement of results for the EDP were:

  • Program accountability:
    • Performance measures gathered through the indicators did not reflect relevant outcomes (especially those relating to loans);
    • Integration of EDP services with the core services of the delivery agents, which made data reporting challenging;
    • Performance measures not defined adequately (service providers counted differently); and
    • Overall expectations of the EDP as described in relation to performance measure are not clear.
  • Achieving consistent levels of service; eligibility criteria; definitions; terms and conditions:
    • The EDP is viewed as somewhat fragmented and lacking consistency;
    • The urban delivery agencies have wide ranging approaches and usage of funds, even though their contracts with the department look very similar;
    • The Provincial Community Futures Associations reported that engaging all Community Futures Organizations in EDP-related activities is a challenge;
    • Differences in the terms and overall management of the fund from region to region;
    • Many organizations no longer have loan funds, so their commitment to the EDP may be lacking as a result of the depletion of the loan fund; and
    • Reluctance to disclose disabilities. Many interviewees identified a reluctance to disclose disabilities as a barrier to providing more direct service to entrepreneurs with disabilities.

Achievement of Outputs and Outcomes of the EDP

Key informants indicated that activity levels and the types of supports for the EDP vary across components of the EDP and this variation occurs between the rural and urban delivery agents of the program. As a result, the EDP meets needs, achieves outputs and outcomes to a different degree in different locations. From the key informant interviews, it was estimated that about half of Community Futures Organizations have had very little or no activity related to the EDP and just 1 of the 7 Urban Delivery Agents has had a significant level of EDP specific loan activity. Key informants from the Community Futures Organizations indicated that the original vision of the EDP of providing a loan portfolio that would be self-sustaining has not been achieved for those organizations that have depleted their EDP loan fund.

Key informants also indicated that the reporting and accountability systems currently in place do not support ease of analysis and reporting on the results of loan funds. Many of the delivery agents have largely integrated the EDP loan funds with their core loan portfolios. The success stories of the EDP were viewed as significant in their impact, however relatively few in number by key informants. The incremental impact of the EDP overall is viewed as mainly in the area of pre and post care by providing extra counselling/advisory support, and working with other community organizations to support entrepreneurs with disabilities.

When asked about attribution, or what would have occurred without the EDP, interviewees describe clients who have been successful, and who have benefited greatly from the services provided as a result of the program, primarily in the area of pre and post care. Key informants reported that it is a challenge to attribute success to the provision of the EDP loan fund, as many of the businesses that were funded were not successful, and the loans were not repaid.

The comparison with other initiatives (Table 4–1) revealed the following relating to the EDP and its achievement of objectives:

  • Low awareness of the EDP. Overall, the programs and services that target people with disabilities appear to have a low awareness of the EDP. This would limit the EDP’s impact on the community, and its outreach.
  • Pockets of innovation. Some other programs and services appear to be more innovative in their approaches, for example in the usage of the Internet, or in reaching out to new markets (geographic reach). Examples are innovative websites offered by some of these initiatives which include courses specifically for entrepreneurs.
  • Lack of and inconsistent information, definitions, and program design features. The lack of information makes comparison, and overall evaluation, challenging.
  • Performance targets not in alignment; not met. Performance targets, for example relating to loan activities, may not align with needs or priorities for specific organizations.

Return to the top of this pagetop of page

Number of Clients Served

Within the evaluation period, the EDP provided services in all areas to 15,471 entrepreneurs with disabilities8. Table 5–1 provides a breakdown of the number of clients served by the program as captured in departmental and CF databases.

Table 5–1 Number of Entrepreneurs with Disabilities Served
Rural - CF British Columbia* Alberta Saskatchewan Manitoba Total
2006-07 1,913 152 393 510 2,968
2007-08 1,697 139 723 178 2,737
2008-09 1,105 219 196 832 2,352
2009-10 1,985 231 370 337 2,923
2010-11*** 1,296 289 185 312 2,082
Total 7,996 1,030 1,867 2,169 13,062
Urban British Columbia Alberta Saskatchewan Manitoba** Total
2006-07 77 252 116 - 445
2007-08 191 168 104 - 463
2008-09 104 305 125 193 727
2009-10 103 227 158 316 804
2010-11*** 73 145 122 135 415
Total 548 1097 625 644 2409
Grand Total 8,544 2,127 2,492 2,813 15,471

Source: TEA System of Community Futures (Rural) and departmental project database (Urban)
*Number of entrepreneurs for BC includes numbers from other initiatives. **Data for Manitoba’s Urban delivery Agent was absent in the departmental database for 2006 and 2007. ***Data includes the first three quarters of Fiscal 2010-2011

Number and Value of Loans

Table 5–2 and 5–3 below indicates the number of EDP loans issued over the period covered by the evaluation.

Table 5–2 Number and Value of Loans Issued by the Community Futures Organizations from 2006-2010
Year British Columbia Alberta Saskatchewan Manitoba Number of Loans Value of Loans
2006-07 42 30 5 8 85 $2,880,164
2007-08 39 20 5 17 81 $2,602,594
2008-09 37 28 3 19 87 $2,742,192
2009-10 45 20 5 15 85 $2,799,544
2010-11* 39 20 4 6 69 $2,088,474
Total 202 118 22 65 407 $13,112,988

Source: TEA System of Community Futures Organizations
* Data includes the first three quarters of Fiscal 2010-2011.

Table 5–3 Number of Loans Issued by the Urban Delivery Agents from 2006 to 2010*
Year British Columbia Alberta Saskatchewan Number of Loans Value of Loans
2006-07 33 2 1 36 $479,385.35
2007-08 4 1 1 6 $133,748.82
2008-09 40 12 0 52 $838,245.06
2009-10 46 8 2 56 $679,655.00
2010-11** 16 2 0 18 $82,150.00
Total 139 25 4 168 $2,213,184.23

Source: Departmental project database
*These do not include numbers from Urban Delivery Agent in Manitoba. It was not possible to reconcile and generate the data on an annual basis from the information in the department’s project database.
** Data includes the first three quarters of Fiscal 2010-2011.

Overall, in the period covered by the evaluation, the delivery agents issued a total of 575 loans (excluding the Urban Delivery Agent in Manitoba) from 2006 to 2010. From the Community Futures Organizations database, as at the end of 2010, the net cash on hand for the EDP after all loan commitments in 2010 was estimated to be about $7,165,049. About 41 (45%) of all Community Futures Organizations reported no loan activity in 2009. The cash on hand for the Urban Delivery Agents in British Columbia, Alberta, and Saskatchewan was estimated to be about $281,551 as at March 31 2010. Most of the Urban Delivery Agents reported low or no loan activity in the key informant interviews. Reliable data on loan default rates was not available for the evaluation; however, key informants and focus group participants indicated a high default rate of about 60%. Only 39% of survey respondents expressed a positive opinion that the EDP prevents clients from failing at their business ventures.

Return to the top of this pagetop of page

Impact on Employment

The EDP had jobs created/maintained/expanded as an indicator to help measure the impact of the program on employment. The following table indicates the number of jobs created by the EDP lending within the evaluation period for both urban and rural delivery agents.

Table 5–4 Numbers of Jobs Created by EDP Lending
Year Urban Rural
2006-07 82 364
2007-08 33 171
2008-09 129 179
2009-10 170 265
2010-11* 66 155
Total 480 1,134

Source: Urban Numbers from departmental project database and Consultations. Rural numbers from the Community Futures Organizations TEA System
* Data includes the first three quarters of Fiscal 2010-2011.

Overall, the EDP was estimated to have created 1614 jobs over the period covered by the evaluation. The client survey indicated that the EDP increased the number of people with disabilities who are self employed full time and reduces numbers who are unemployed and looking for work. The EDP also has the ancillary effect of creating additional employment for other individuals. The majority of respondents who started a business reported employing at least one person in addition to themselves, with an average of 3.4 employees. The majority of those who expanded their business were also able to employ at least one additional person, with an average of 4.9 employees.

Impact on Businesses

The urban delivery agents reported on an indicator for the number of business created/expanded within the period from 2007-2010 (Table 5 – 5). This indicator was part of the indicators reported by the rural delivery agents in their database.

Table 5–5 Number of Business Created/Expanded
Year Urban
2007-08 33
2008-09 87
2009-10 132
2010-11* 21
Total 273

Source: Numbers from departmental project database and Consultations.
* Data includes the first three quarters of Fiscal 2010-2011.

The client survey indicated that over 40% of respondents were unemployed or derived their income from other sources prior to accessing the EDP. Following participation in the EDP, roughly half of respondents indicated they either started or expanded on a business which continues to operate.

Survey respondents indicated the EDP played a significant role in allowing them to reach their business goals. 37% of respondents indicated the EDP had a significant impact on their business with a total of 63% assigning a positive value of 4 or higher (on a scale of 1-5) on the level of impact (Table 5 – 6).

Table 5–6 Responses to Question on Impact of EDP on Businesses
   (n=81) Percentage
1 - No impact at all 15 19%
2 3 4%
3 10 12%
4 21 26%
5 - Significant impact 30 37%
Don't know/No response 2 2%

Figure 5-1 shows the description and percentage of impacts on the business of client survey respondents. There were three areas in particular where the EDP had a particularly notable impact on the client:

• Providing moral support and encouragement (34%);
• Increasing opportunities and removing barriers, including market entry (33%); and
• Providing business planning and education (34%).

Return to the top of this pagetop of page

Text Version: Impact of EDP on Clients

Figure 5-1 Impact of EDP on Client

In this figure, the percentage of respondents indicate the areas the EDP impacted on their lives.

Most Significant Impact of EDP

The “pre and post care” services offered by the EDP was viewed by almost all key informants as a very valuable aspect of the EDP supported advisory/training/ business services. This service was described as being very much needed and highly beneficial. Focus group participants supported this and further stated that as a result of this service, a high number of clients that applies for loans and goes through with their applications do receive the loans. The approval rate for loans, as indicated by client survey respondents was quite high at 91%.

Data from the databases indicate that quite a significant amount of time is spent by the delivery agents in “pre and post care” services for clients. The total number of coaching hours recorded by the Urban Delivery Agents from 2007 to the third quarter of 2010-11 was 13,525 hours. The number of events sponsored in support of this “pre and post care” for the Urban Delivery Agents was 494 from 2007 to 2010. The Community Futures Organizations did not report results on these indicators.

As a result of the pre and post care services of the EDP, client survey respondents reported strong positive opinions regarding most aspects of the program. One a scale of 1 to 5 where 1 represents “Strongly Disagree” and 5 indicates “Strongly Agree”, over 50% of respondents assigned a positive value of 4 or higher when asked to rate the following features of the EDP:

  • Letting people know what the program offers (66%);
  • Providing confidence and motivation needed to start a business (58%);
  • Facilitating access to financing (59%);
  • Cooperating with other related programs and services (53%);
  • Providing people with disabilities specialized services not provided by other organizations (50%);
  • Supporting their preference to access business-related services specialized for people with disabilities (70%); and
  • Providing better quality of service relative other individuals and organizations (60%).