Correctional Service Canada
www.csc-scc.gc.ca
Common menu bar links
Performance Assurance
Offender Management System Renewal Project Closure Review
Internal Audit Branch
378-1-234
Approved by Audit Committee
September 25th, 2007
Table of Contents
- Executive Summary
- 1.0 Background
- 2.0 Objective of the Review
- 3.0 Scope and Approach of the Review
- 4.0 Observations
- 5.0 Conclusion and Recommendation
- Appendix A - List of Interviews
- Appendix B - Issue Tracking Analysis
- Appendix C - Management Action Plan
Executive Summary
Background
This project closure review has been performed by the Centre for Public Management in conjunction with Internal Audit of Correctional Service Canada (CSC) to respond to Treasury Board Secretariat requirements under the Enhanced Management Framework1. The objective of the review was to assess the project management processes that were used to implement the OMSR project in order to identify lessons learned and ensure that future projects will benefit from the experience gained in this implementation.
The Offender Management System Renewal (OMSR) project commenced in 2001 with three streams: Database and User Interface Migration, Application Renewal and Information Sharing. The objective of the renewal project was to update the Offender Management System (OMS), CSC's database of information about Canadian offenders, to provide enhanced technology to support and improve information sharing between criminal justice partners. The information sharing stream was completed prior to March 31, 2006, while the migration stream was completed in November 2006 and the renewal stream is on-going.
Conclusions
Overall, we found that the OMSR project had sufficient project management processes in place to help ensure its success. Our interviews indicated that users are satisfied with the system, and would not wish to revert to the legacy system. This is an important endorsement of the project and, by association, its project management processes.
As part of this review we have also identified project strengths and opportunities for improvement which can be carried forward to future projects and they are as follows:
Project Strengths:
The project benefited from senior management support and a strong project manager. The initial lack of large development project experience was bolstered through the use of an external project management consultant. While the project had its challenges and experienced delays, an effective communication plan kept the various internal and external stakeholders up to speed on progress. The training program resulted in very few support calls after the go-live of the migration project. Project team members felt that issues, once raised to the project manager, were dealt with on a timely basis. Finally, the decision was made to build an internal project team, rather than hire an integrator, in order to build internal capacity. This investment, which resulted in some delays at the beginning of the project, has now paid off with a trained project team in place for maintenance and future development.
Opportunities for Improvement:
The project steering committee met every three months, and had other responsibilities in addition to OMSR. For this reason, many of the issues that was resolved along the way happened outside of the formal meetings, with no benefit of minutes or a record of decisions. In the future, a dedicated project steering committee should meet regularly to capture these issues and decisions. In addition, the Project Management Office (PMO) should to be strengthened in order bring best practices to each project undertaken, and allow centralized planning and control. This would also address a number of other opportunities for improvement, such as centralized logging and tracking of issues and risks, and maintaining an up to date project charter and business case which reflects the changes in the project over time. We noted that a project steering committee and a more rigorous PMO were in place prior to the merger of the project with IMS in 2004, but were not maintained post-merger.
The financial information provided by the Assistant Commissioner Corporate Services shows that the final project costs were within its established budget of $42M. Given the magnitude of this project, this is a notable point. Nevertheless, our review revealed opportunities for improvement with respect to the overall financial tracking and monitoring process.
These items are covered in more detail in the body of this report.
Senior management has reviewed, and agrees with, the findings contained in the report. The Management Action Plan which addresses the recommendation is included in Appendix C.
1 The Enhanced Management Framework is designed to ensure that government information technology projects fully meet the needs of the business functions they are intended to support, deliver all expected benefits and are completed on time and on budget.
1.0 Background
The Offender Management System (OMS) is a database of detailed information about Canadian offenders. It is used daily by Correctional Service Canada and National Parole Board officers to track offenders and ensure their safe re-entry into society.
In 2001 the Government of Canada contributed $50 million to renew the OMS and improve CSC's information sharing methods with its partners. CSC received $42.4M, the National Parole Board (NPB) received $4.6M and $3M were given to Department of Justice (DOJ). CSC's plans for the OMSR initiative were as follows:
Fiscal years 2001 - 2005: |
$31.0M; |
Fiscal year 2005 - 2006: |
$4.4M; |
Fiscal year 2006 - 2007: |
$7.0M. |
Total |
$42.4M |
The Offender Management System Renewal (OMSR) initiative was necessary to provide users with better quality information. As well, the OMS Legacy system was no longer able to handle its growing number of users. Originally created for 900 users, it now serves more than 10,000 users and 12,000 are projected in a near future.
The purpose of the OMSR project was to provide enhanced technology to support and improve information sharing between criminal justice partners. The final Effective Project Approval, drafted in December 2002, listed the project as containing three streams:
- Stream 1 - Database and User Interface Migration
- Stream 2 - Application Renewal
- Stream 3 - Information Sharing
The objective of the Database and User Interface Migration stream, also referred to as the OMSM, was to migrate the legacy OMS functionality to a more user-friendly interface and robust database technology, and to make appropriate changes to the software and hardware platform architectures. This was implemented in November 2006.
The purpose of the Information Sharing stream, subsequently referred to as the connectivity project, was to better collect and share offender information with a large number of criminal justice partners at all levels in support of safer reintegration and an effective, integrated criminal justice system. In addition to the technical challenges, this represented a major effort in communication and coordination and was completed prior to March 31, 2006.
The Application Renewal is on-going, and will take advantage of the migrated platform to renew applications using an evergreening approach to ensure improvements in the flow of critical information on offenders within CSC and among all stakeholders in the criminal justice community. The use of the evergreening approach, which implies on-going improvement projects, will eliminate the need for another large development project.
The initiative was launched in coordination with connectivity initiatives with external partners, such as the National Parole Board (NPB), to reduce duplication and ensure information sharing.
This project closure review is being undertaken at the request of Treasury Board Secretariat, as required under the Enhanced Management Framework (EMF), with a focus on project management effectiveness. The Centre for Public Management Inc. was contracted by Internal Audit, CSC, to work with Internal Audit staff to complete this review.
2.0 Objective of the Review
The review had the following objective: To assess the project management processes that were used to implement the OMSR project in order to identify lessons learned and ensure that future projects will benefit from the experience gained in this implementation.
3.0 Scope and Approach of the Review
The OMSR project has been underway since 2001, and has been subject to a number of external reviews over that timeframe. In September 2004, the project was merged with the IMS branch which resulted in changes in the way the project was managed. For this reason, the scope of this project closure review focuses on the OMSR project from the time of the merger to go-live of the migration in November 2006. In addition, we reviewed the original project planning documents from 2001 as well as the project reviews undertaken throughout the project to ensure the proper context and background for this review. It should be noted that decisions made at the outset of this project such as platform, development approach, user requirements, etc., were specifically excluded from the scope of this report.
As part of the review, we examined key aspects of the management of large IT projects, including:
- Project planning;
- Project governance; and
- Project management capacity.
For the purpose of this review and report, we use the term OMSR to include the three components of the project, as described in the Effective Project Approval (EPA)2 from Treasury Board: the renewal, the connectivity and the migration components.
The review began in January 2007, and we carried out our examination work from January to April, 2007. This work involved:
- Interviews,
- Documentation review,
- Reviewing project management practices against those of recognized organizations such as the Project Management Institute's (PMI) PMBOK and the IT Governance Institute's Control Objectives for Information & Related Technology (COBiT); and,
- Detailed examination of selected project management processes.
As part of the review, we performed 15 interviews, including key executives, OMSR project members and other key stakeholders. We also obtained relevant documentation to validate the information gathered during interviews (See Appendix A for a list of interviewees). The team also examined supporting documentation to determine whether specific processes or controls were in place.
2 EPA: This is Treasury Board's approval of the objectives (project baseline), including the Cost Objective, of the project implementation phase and provides the necessary authority to proceed with implementation. Sponsoring departments submit for EPA when the scope of the overall project has been defined and when the estimates have been refined to the substantive level.
4.0 Observations
4.1 Project Planning
In the context of project planning, the team reviewed a number of elements.
4.1.1 Business Case
Under project planning, we expected to find that the OMSR project team had developed a business case that was comprehensive and complete, and which identified and justified the OMSR project and indicated how it related to CSC and government-wide priorities. At a minimum a business case should identify:
- the opportunities for improvement;
- the benefits of such an undertaking;
- the technical solution at a high-level;
- indicators against which to measure any improvement in program performance;
- costs (up-front, direct ongoing, indirect and potential user/client costs); and
- the risks and steps to be taken to manage those risks.
The team found that the OMSR business case had covered all significant elements.
4.1.2 Project Charter
We found that the OMSR project team had a project charter which defined the scope of the project. It also set out the overall project management framework and standards to be used. The project charter also clearly defined who the internal and external stakeholders were, described roles and responsibilities for the members of the project team and established the overall project governance structure. Furthermore, we found that the project charter had set out the timeframes, deliverables and financial commitments for the project.
Project Planning: Strengths
- The project charter and business case were comprehensive documents.
- On the outset of the project, the decision was made to build an internal project team, rather than hire an integrator, in order to build internal capacity. This investment, which resulted in some delays at the beginning of the project, has now paid off with a trained project team in place for maintenance and future development.
Project Planning: Opportunities for Improvement
- We noted that although the project evolved over the years, the project charter and business case documents were not updated. While the incremental changes are documented in various status reports, meeting minutes and other documents, it is useful to maintain these original project planning documents as a central "road-map" to the project and single point of reference. In the future these documents should be kept up to date as the project evolves.
4.2 Project Governance
We expected to find that a steering committee had been established and was operating and, in particular, that individuals at the appropriate levels had been identified and were participating on the committee. The project governance approach changed on the merger of the OMSR project with IMS in 2004. We have presented this structure in the following diagram:
OMSR GOVERNANCE
For the purposes of this review, we have examined the post-merger structure.
We found that the steering committee for this project is the Information Management and Technology Committee (IMTC). The committee is made up of permanent and rotating members and chaired by the Assistant Commissioner, Corporate Services (CIO). Meeting on a quarterly basis its mandate is to serve as a key focal point for information management/information technology/technology issues of concern to CSC. As such, the OMSR project was just one aspect of this committee's mandate. The next level of governance is the weekly manager's meeting, which is an operational meeting chaired by the Director General of Information Management Services and Offender Management System. We noted that there were regular informal meetings between the DG IMS/OMS and the Assistant Commissioner, Corporate Services (CIO) and others, outside of the formal meetings.
Project Governance: Strengths
- The IMTC is a senior level committee which is able to resolve issues, and if necessary, raise them to the appropriate levels of CSC,
- In our interviews, we consistently heard the message that Senior Management was very supportive of this project. This is an important factor in the success of any project, and particularly one that faced as many technical challenges as OMSR; and,
- Important issues were raised with the appropriate individuals as required, they did not wait until the next regularly scheduled meeting.
Project Governance: Opportunities for Improvement
- In a project of this size, best practices suggest a project steering committee that focuses solely on the project, and would meet at least on a monthly basis. The IMTC met on a quarterly basis and discussed all issues related to IMS at CSC. As noted above under strengths, based on our review, it appears that an appropriate level of governance was provided to the project, but on an informal basis. As a result, there are no minutes of these meetings or a record of decisions. For future projects, a project steering committee should be formed which would meet on a regular basis (monthly at a minimum) to discuss the issues facing the project, propose resolutions, and identify the issues which should be raised at the IMTC.
4.3 Project Management Framework
The objective was to assess the soundness of project management processes in place and their contribution to the project's success by ensuring that an effective management control framework existed. The project management processes that we examined as part of this review included the following areas: project planning, tracking and control; risk and issue management; internal and external training and communication; project management capacity; and, project management office.
4.3.1 Project Planning, tracking and control
As noted in the section under project planning, the initial project plans were in place for this project at the outset. We reviewed the project master plans, and found that they contained the details required and were updated on a regular basis. As noted in the subsequent section on Project Management Office, these plans were maintained at the stream level and not maintained or integrated by the PMO.
Project Planning, tracking and control: Strengths
- The project manager of the migration stream is undertaking a formal lessons learned review. Although this initiative was in process at the time of this OMSR closure review, it represents a best practice in the area of project management. In order to benefit from the findings of this project initiative, it will be incumbent on management to incorporate them into future projects.
Project Planning, tracking and control: Opportunities for Improvement
- The OMSR project received Treasury Board EPA approval in 2003 for
a total budget of $42.4M. The reconciliation provided by the ACCS indicates
that the actual costs of $41,907,851 were within the approved this approved
budget. Given the magnitude of this project, this is a notable point.
As part of this review, we requested the financial reports in order to review the actual costs of the project against budget over its life. While responding to our request, it became evident that there had been coding errors made over the life of the project by the PMO and that there had not been regular reconciliation of the PMO costing and corporate financial records. The resulting reconciliation process required three months to complete.
For future projects, the PMO and corporate finance should work together to develop an integrated approach to tracking project expenditures, which includes regular reconciliations and variance reporting to enable errors to be detected and corrected on a timely basis and to ensure that the costs are within the approved budget.
4.3.2 Risk and Issue Management
Risk and issue management was handled differently through the lifecycle of the project. Prior to the merger of the OMSR project with IMS, issues were captured and tracked in an MS ACCESS database, and risks were managed using a separate process and committee, the Risk Identification Committee (RIC). On the merger of the project with IMS, these processes were rolled into the weekly management meeting and dealt with on an action items basis. Risks and issues would be raised in the project status report, and dealt with at the team level and raised at the management meeting.
Risk and Issue Management: Strengths
- Tabling of risks and issues at the management meeting ensures that issues and risks receive attention on a timely basis. Our interviews indicated that pressing issues were resolved on a timely basis.
Risk and Issue Management: Opportunities for Improvement
- Best practices in this area dictate that a log be used to track risks and issues in order to ensure tracking and traceability. To this end, we believe that the issue database that was used prior to 2004 represents a best practice, and would benefit future projects. It could be expanded to include risks. As an example of the benefits of the log approach, we performed an analysis of the management meetings for the month of February, 2006. The results of this analysis is presented in Appendix B, and demonstrates that while issues are likely being resolved off-line, in many cases there is no record of this resolution. A log provides a continuous record of the issues and risks raised, who has been assigned to the task, and the ultimate resolution.
4.3.3 Internal and External Training and Communication
The connectivity and migration streams of the OMSR project involved thousands of users both within and outside of CSC. These users were not homogenous, requiring a training and communication approach which was able to respond to their different needs in a cost effective manner. The project had a communication plan in place from August 2000, which was updated in response to an independent review which took place in March 2003. As part of this review we reviewed a sample of training and communication materials, as well as the training and communication plans.
Internal and External Training and Communication: Strengths
- The external communication program (to partners) involved attendance at conferences, submission of articles to trade publications and direct communication as required,
- We examined a sample training program for InfoPol (External) and the participant manual from the OMSR training Kit (Internal) and found both to be effective,
- Our interviews indicated that internal stakeholders found the project to be transparent, with good communication (newsletters, emails, daily reports and conference calls); and,
- Although some end users felt that the internal training could have been more customized to their needs, the project followed an accepted and cost effective approach by utilizing standard screens to teach navigation skills, and utilizing the beta test environment for users to gain experience in their specific modules. The success of this approach was proven by the low number of support calls to the OMS specialists in the weeks after go-live.
4.3.4 Project Management Capacity
On the outset of the project, in recognition of CSC's lack of experience in managing large IT projects, an external advisor was hired to assist the project manager. This advisor, entitled OMSR Information Technology Project Management Integrator Advisor (ITPMI), provided regular reports on various issues facing the project over its life, with a focus on the project management framework. In our review of his reports, we found that he maintained his independence from the project, and provided value added advice to the project manager on project management issues.
Project Management Capacity: Strengths
- The implementation of the ITPMI role, and the service provided, strengthened the overall project management approach of the project. The consultant was able to maintain his independence and provide unbiased advice over the life of the project.
4.3.5 Project Management Office
Subsequent to the merger of the OMSR project with IMS, the project management office (PMO) tracked financial expenditures, prepared all the TBS submissions and kept track of meeting minutes for the various governance committees for the project. The PMO did not maintain central project work plans, project management processes, methodology or knowledge objects, issue and risk logs nor did it coordinate or prepare status reports. These PMO functions were set up at the outset of the project, but not maintained in the merged structure, instead undertaken by the various project streams and managers.
Project management Office: Opportunities for Improvement
- IMS should maintain a project management office which would act as a central
resource for all IT projects. The tasks that should be performed by the PMO
include:
- Maintain the master project schedule,
- Track issues and risks,
- Coordinate status reporting,
- Act as a repository for best practices and knowledge objects developed on projects,
- Perform resource management,
- Develop and maintain development policies and standards; and,
- Perform regular financial tracking and monitoring in conjunction with the Comptroller's Branch.
- The PMO staff should be free of IMS administration tasks, and focus solely on project work.
- A formal PMO, performing the tasks noted above, will improve resource allocation and utilization while also improving quality through the enforcement of development standards and the transfer of best practices from one project to another. This role will become even more important with the renewal projects which will be undertaken in the coming years.
5.0 Conclusion and Recommendation
Overall, we found that the OMSR project had sufficient project management processes in place to help ensure its success. Our interviews indicated that the users were happy with the system, and would not wish to revert to the legacy system. This is an important endorsement of the project and, by association, its project management processes. As part of this review we have identified project strengths and opportunities for improvement which can be carried forward to future projects.
More specifically, for CSC to continue to improve its IM/IT project management practices, we are recommending the following:
The Senior Deputy Commissioner3 should enhance IM/IT project management practices, in particular:
- ensure project planning documents are kept up to date as the project evolves;
- establish a project steering committee for the specific project;
- strengthen financial tracking and monitoring;
- enhance tracking of risks and issues;
- formalize and enhance the role of the project management office.
3 Prior to July 2007, the IMIT division fell under the responsibility of the ACCS. It is now the responsibility of the Senior Deputy Commissioner.
Appendix A: List of Interviews
Appendix B: Issue Tracking Analysis
Appendix C: Management Action Plan
RECOMMENDATION
The Senior Deputy Commissioner should enhance IM/IT project management practices, in particular:
- ensure project planning documents are kept up to date as the project evolves;
- establish a project steering committee for the specific project;
- strengthen financial tracking and monitoring;
- enhance tracking of risks and issues;
- formalize and enhance the role of the project management office
MANAGEMENT ACTION PLAN
Actions have been taken to address the findings of the OMSR review as they relate to all current and future Treasury Board approved IM/IT projects, as follows:
- Project planning documents will be kept up to date as each projects evolve;
- A project steering committee will be established for all future IM/IT projects;
- Monthly reviews will be established in order to strengthen financial tracking and monitoring;
- Risk registers will be completed and updated monthly (at a minimum).
In addition, action has been taken to formalize and enhance the role of the Project Management Office. In particular a project control function has been added for the HIMM Project for risk analysis, financial reporting and the maintenance of project plans.
