Western Economic Diversification Canada
Symbol of the Government of Canada

Common menu bar links

Observations and Recommendations

Based on interviews, analysis, and evidence gathered through the audit, each audit criterion was assessed by the audit team and a conclusion for each audit criterion was determined. Where a significant difference between the audit criterion and the observed practice(s) was found, the risk of the gap was evaluated and used to develop a conclusion for each audit criterion and to document recommendations for future improvement initiatives. Details of these observations and recommendations are provided below.

IMT Strategic Planning and Risk Management

Expectations

We expected to find formal IMT strategic planning processes in place to ensure that IMT planning is integrated with the department’s overall planning processes, includes formal input from stakeholders across the organization, and considers the current IMT capacities and capabilities of the department. Furthermore, we expected to find an IMT risk management framework in place to manage IMT strategic, operational and projects risks, and that is aligned with the department’s overall risk management processes.

Observations

There are no formal processes to ensure that IMT planning is fully integrated with the department’s planning processes, and IMT planning is currently performed with limited input from stakeholders across WD (i.e. regional stakeholders, senior management, Planning and Reporting Committee). IMT capacity and capabilities have not been formally assessed to determine if they are appropriate to achieve the IMT priorities of the department. Furthermore, IMT risk management is performed informally at WD, as an IMT risk assessment framework is not in place, and, for example, a risk assessment is not formally performed as part of annual IMT planning.

Annual IMT planning is performed by the CIO / Director of IMT , communicated within the IMT section of the Finance and Corporate Management (FCM) Business Plan, and approved by the Executive Director – Finance and Corporate Management. IMT planning is currently performed with limited input from relevant stakeholders: regional IMT contacts, senior departmental management, or the Planning and Reporting Committee.

While the Corporate Management Business Plan is prepared by the Planning and Reporting Committee and approved by the Executive Committee, the department as a whole does not have an integrated planning schedule as functional area business plans are completed at different times during the year. The Planning and Reporting Committee has had limited exposure to IMT priorities / planning in the past; the CIO is not a member of the committee, and IMT has not frequently been discussed.

The audit team performed some analysis of the IMT organizational structure (the audit was not intended to provide a full organizational review), and did not observe any clear gaps in this area; however, WD has not performed a formal assessment of IMT human resource capacity and capabilities to determine if appropriate staffing levels, competencies and organizational structure exist to help achieve IMT objectives and priorities.

Corporate risk management is performed by the Planning and Reporting Committee who leads the development of the Corporate Risk Profile. It was noted that the IMT Directorate does not have direct representation on the Planning and Reporting Committee, and there is limited consideration of IMT risks that affect the department within the Corporate Risk Profile. Furthermore, specific IMT risks / risk mitigation strategies within the Corporate Business Plan or within the IMT portion of the FCM Business Plan have not been identified. Risk management processes are performed informally by IMT management for large IMT projects; however, a formal IMT risk management framework has not been established at WD.

WD and IMT management have identified integrated strategic planning and IMT risk management as areas that can be improved and have taken initial steps to improve processes:

  • The department is in the early stages of developing a department-wide integrated planning schedule for the preparation departmental plans, including regional and functional business plans.
  • The CIO has prepared an IMT Business Plan for 2011 – 2014 that includes sections on Corporate Alignment and Risk Management; however, this plan has not yet been finalized or approved.

Impact

IMT planning processes should be integrated with departmental planning processes and consider input from all relevant stakeholders. Absence of formal IMT planning processes that are fully integrated with departmental planning increases the risk that IMT planning and resource allocation will not be aligned with the broader priorities of the department.

Without a formal assessment of IMT human resource capacity and capabilities, gaps may not be well understood and addressed in a timely manner to achieve the IMT priorities of the department. It is also expected that the Federal Government’s Strategic and Operating Review, and the Shared Services Canada initiative will have an impact on IMT capacity and capabilities, which further supports the need to perform a formal assessment as the impact of these initiatives becomes clearer.

Without having an IMT risk management framework and formalized IMT risk management processes that are aligned to risk management processes of the department, there is an increased risk that IMT risks will not be appropriately identified and managed to help the Department achieve its objectives.

Recommendations

1. A formal IMT planning process should be established that allows for integration with departmental planning. Furthermore, planning processes should engage all relevant stakeholders (including regional IMT contacts, senior departmental management, and the Planning and Reporting Committee), and should be periodically monitored by management.

2. An IMT risk management framework should be developed that allows for WD to identify strategic and operational risks related to IMT , develop mitigation strategies/controls for identified risks, and regularly report on the status of mitigation measures. Furthermore, IMT management should be formally engaged in the departmental planning process to help ensure that IMT risks are considered as part of departmental planning, and that the IMT Directorate is aware of risks at the departmental level that are relevant to IMT activities.

IMT Performance Measurement

Expectations

We expected to find appropriate IMT performance measures that are clearly defined, and a formal IMT performance monitoring processes established to allow for adequate evaluation of IMT service delivery and IMT ’s contribution to the department by management and relevant governance committees.

Observations

Formal performance measures for IMT have not been defined. Some informal monitoring of IMT performance with regards to IMT projects and ongoing operations has been performed within the IMT Directorate; however, there has been limited monitoring / reporting on IMT performance to governance committees.

Some IMT performance monitoring is performed by IMT management, IMT Council, and the Executive Committee; however, a comprehensive IMT performance measurement process has not been established.

IMT operational statistics (i.e. service calls received / completed) and IMT project status updates are prepared and provided to the Executive Committee as part of the IMT quarterly update. Beyond these operational statistics, however, performance measures have not been defined at a more strategic level. For example, the IMT portion of the FCM Business Plan does not include formal performance indicators to measure the achievement of IMT priorities within the Plan. Furthermore, there is limited evidence of IMT management or relevant committees (IMT Council, Executive Committee) performing ongoing monitoring of the achievement of priorities within the Plan; a performance measurement framework has not been defined to clarify what should be measured and how performance should be monitored by management and governance committees.

IMT management has identified IMT performance measurement as an area that can be improved and have taken initial steps to improve processes; the CIO has prepared an IMT Business Plan for 2011 – 2014 that includes a section that defines formal IMT performance indicators; however, this plan has not yet been finalized or approved.

Impact

Lack of comprehensive IMT performance measures limits the ability of IMT management and governance committees to effectively monitor IMT initiatives and take corrective action as necessary. As reporting of IMT performance has not been defined, there is an increased risk that IMT management and relevant committees will not have appropriate information to exercise oversight and make informed decisions.

Recommendation

3. Formal performance measures should be established for all key IMT activities, including: operational service delivery, key IMT projects, and priorities identified within IMT annual plans. IMT performance reporting processes should be established that define IMT performance reporting to IMT management and relevant committees at scheduled periodic intervals.

IMT Project Priority Setting Processes

Expectations

We expected to find a process for prioritizing IMT project and ensuring they are aligned with the objectives of the department; we also expected to find that prioritization processes would include analysis of costs, benefits, dependencies and alternatives.

Observations

Processes over prioritizing IMT projects are maturing as the IMT Project Priority Setting Framework has recently been established; however, evidence of justification of IMT project rankings is not retained, and cost analysis performed as part of prioritization processes does not consistently include all costs associated with IMT projects.

A new IMT Project Priority Setting Framework has recently been utilized at WD, and is a significant improvement for prioritizing IMT projects to align to IMT strategies and business requirements of the department; however, the use of the Framework is still maturing and could be improved to optimize the effectiveness of the process.

The Framework provides detailed guidance over the roles and responsibilities of IMT management and relevant committees with regards to prioritizing IMT projects. The Framework was used by the IMT Council to prioritize and rank 29 IMT projects, and was approved by the Executive Committee in August 2011; however, justification for IMT project rankings was not compiled by IMT Council and provided to the Executive Committee.

IMT projects are ranked using the Framework according to 5 factors: business value, delivery value, economic value, business risk, and technical risk. In considering value of IMT projects, only incremental costs of IMT projects were considered, and full time employee labour (at headquarter and regional locations) to develop / implement IMT projects were not considered. It was also noted that ongoing maintenance costs (post development / implementation) were not consistently allocated to project costs.

Impact

While WD has a limited number of large projects, not retaining justification for IMT project rankings limits the ability of the Executive Committee to perform effective oversight before approving IMT project rankings, and increases the risk that ineffective IMT resource allocation decisions may be made.

As not all relevant costs are allocated to IMT project cost (i.e. full time employee labour at headquarters and regions, and ongoing maintenance costs), effective cost-benefit analysis is limited, which increases the risk that IMT projects may not be appropriately prioritized.

Recommendations

4. Some level of justification of IMT project rankings should be provided to the Executive Committee to facilitate the effective oversight and approval of IMT resource allocation decisions.

5. Projects costs that are used to perform cost-benefit analysis and produce IMT project rankings should consistently include all relevant costs, including: full time employee labour (at headquarters and regional locations), and post development maintenance costs.

IMT Project Oversight Processes

Expectations

We expected to find an IMT project oversight framework that defines monitoring processes for IMT management and relevant committees to ensure that appropriate oversight is performed consistently for IMT projects. It was also expected that change management processes would be established that ensure significant project changes also follow project oversight processes.

Observations

While WD has a limited number of large projects, oversight processes over IMT projects are not currently formalized at WD. Some oversight of IMT projects has been performed by IMT Council and the Executive Committee; however clarity could be improved over the Committees’ oversight expectations for large, medium or small projects. In addition, change management processes have not been formally defined to provide clear guidance on ensuring significant changes go through appropriate project oversight processes.

WD has recently improved IMT project oversight processes with the establishment of the IMT Project Priority Setting Framework and implementing quarterly IMT reporting to the Executive Committee (which includes providing IMT project updates); however, ongoing oversight processes and change management processes can still be better defined to ensure effective oversight is consistently performed, and changes are appropriately managed and approved for IMT projects.

The recently implemented IMT Project Priority Setting Framework includes a description of each stakeholder’s (management and relevant committees) responsibilities in exercising the Framework, including high level responsibilities over monitoring projects; however, the Framework does not include guidance on how these responsibilities should be exercised by relevant committees, or how projects should be measured in terms of quality, cost, and meeting project timelines.

It was noted that WD does use standardized forms to record the approval of IMT changes; however, formal change management processes do not exist at WD.

The first IMT quarterly update was provided to the Executive Committee in August 2011 and included some evidence of IMT project monitoring, including high-level status of key projects, and a snapshot of the IMT project portfolio; there is an opportunity to further improve project oversight reporting to make it more efficient. For example, it is difficult to gather if projects that are not on track are high priority projects, which would better allow the Executive Committee to determine if re-allocation of resources (potentially from low priority projects) is required. When presenting information, the input/decision that is being sought from the governance committee should be made clear.

Ultimately, project oversight processes have not been formalized to detail, based on the importance (e.g. value, risk) of the project, the frequency and performance measures to present to Executive Committee, the IMT Council and project-specific governance committees.

Impact

Without having formally established IMT project oversight processes, there is an increased risk that effective monitoring of IMT projects will not be consistently performed by relevant oversight committees. This also increases the risk that corrective actions may not be taken in a timely manner.

Informal change management processes could result in, amongst other things, changes being implemented that have not received approval from an appropriate level of management.

As specific oversight processes for IMT projects have not been formalized based on the importance of IMT projects (i.e. value, risk), there is also a risk that management and relevant committees may spend too much time monitoring low value / low risk projects, and not enough time monitoring high value / high risk projects..

Recommendation

6. A formal IMT project oversight framework should be established that helps ensure that adequate oversight is performed consistently for IMT projects. The framework should provide guidance on ongoing monitoring of the performance of IMT projects with regards to cost quality and schedule and be supported by a documented change management process. Furthermore the framework should clarify the reporting requirements to each governance committee depending on the importance of a project (i.e. large, medium, low value/risk).