Audit of Training Activities

Table of Contents

• EXECUTIVE SUMMARY
• STATEMENT OF ASSURANCE
• 1.0 Introduction
• 2.0 Audit Objectives and Scope
  • 2.1 Audit Objectives
  • 2.2 Audit Scope
• 3.0 Audit Approach and Methodology
• 4.0 Audit Findings and Recommendations
  • 4.1 Management Framework for Training Activites
  • 4.2 Compliance with Policies and Guidelines Pertaining to NTS
  • 4.3 Implementation of Management Action Plans
• 5.0 Overall Conclusion
• Annex A
• Annex B
• Annex C
• Annex D
• Annex E
• Annex F

EXECUTIVE SUMMARY

Background

The Audit of Training Activities is being conducted as part of Correctional Service Canada's (CSC) 2011-2014 Risk-Based Audit Plan (RBAP), and is linked to numerous CSC initiatives such as the CSC Corporate Risk Profile, the Report on Plans and Priorities, and the CSC Human Resource Strategic Plan.

Despite its small budget in comparison to overall departmental spending, training is material as the Learning and Development Branch plays a critical role in assuring CSC meets its public safety objectives. Learning and Development (L&D) initiatives and training activities are of particular interest to CSC management due to the anticipated increase in staff and subsequent training requirements resulting from legislative changes such as the Truth in Sentencing Act, and the resulting increase in the number of offenders entering the criminal justice system.

The objectives of this audit, as outlined in the RBAP, are:

• to provide assurance that a management framework is in place for training activities and that it is implemented and monitored; and,
• to provide assurance that CSC is complying with relevant legislation, policy directives and guidance pertaining to CSC's National Training Standards (NTS).

To achieve these objectives, the audit team reviewed key documentation and policies, conducted interviews with institutional, regional and national staff, and performed file reviews. The audit was national in scope and focused on compliance with CSC's NTS, including Treasury Board required training, regulated training, and CSC-driven training.

Conclusion

Overall, the audit found that a management framework to support training activities is in place. CSC employees are receiving mandatory training as per the NTS, and these training activities are monitored regularly by way of compliance reports provided to senior management. The audit also found that CSC is in compliance with legislation and policy with regard to the NTS examined in this audit.

That stated, our examination has revealed that NTS courses are inconsistently administered across CSC. CSC Guidelines and CSC's Management Control Framework for Staff Training require updating to reflect the current governance structure and to define the roles and responsibilities of key personnel to ensure the equitable distribution of duties and responsibilities. Duplicate monitoring systems exist at the institutional level, and reporting processes vary across regions, resulting in inefficiencies across the organization and documentation being managed inconsistently across the country.

The impact of these findings is that CSC's administrative control over some NTS courses is limited and opportunities to ensure consistency of implementation and efficiencies are missed. Further, risk management of critical courses is required due to the significant number of NTS courses. Inconsistent documentation practices render the monitoring of course compliance for individual CSC staff challenging.

Recommendations have been made in the report to address these areas for improvement. Management has reviewed and agrees with the findings contained in this report and a Management Action Plan has been developed to address the recommendations (see Annex F).

STATEMENT OF ASSURANCE

This engagement was conducted with a moderate level of assurance.

In my professional judgment as Chief Audit Executive, sufficient and appropriate audit procedures have been conducted and evidence gathered to support the accuracy of the opinion provided and contained in this report. The opinion is based on a comparison of the conditions, as they existed at the time, against pre-established audit criteria that were agreed on with management. The opinion is applicable only to the area examined. The evidence was gathered in compliance with Treasury Board policy, directives and standards on internal audit and the procedures used meet the professional standards of the Institute of Internal Auditors. The evidence gathered was sufficient to provide senior management with proof of the opinion derived from the internal audit.

__________________________________ Date: __________________
Sylvie Soucy, CIA
Chief Audit Executive

1.0 INTRODUCTION

Background

The Corrections and Conditional Release Act, 1992 (CCRA) states that "the purpose of the federal correctional system is to contribute to the maintenance of a just, peaceful and safe society".¹ The principle within this Act related to training that guides the CSC in achieving this purpose is listed under s.4(j)(1) of the Act, and requires the Commissioner ensure "that staff members [are] properly selected and trained and [are] given appropriate career development opportunities".²

The TB Policy on Learning, Training and Development sets out additional requirements for training, and provides Deputy Heads with the authority, pursuant to section 12(1)(a) of the Financial Administration Act, to "determine the learning, training and development requirements of persons employed in the Public Service and fix the terms on which the learning, training and development may be carried out".

Under this authority, CSC developed the Learning, Training and Development Guidelines, 2007 (the Guidelines) which illustrate CSC's commitment to continuous learning, training and development, and also outline organizational priorities, such as ensuring compliance with CSC's approved NTS, as well as Public Service required training.

National Training Standards (NTS) - Mandatory Training

CSC has developed a suite of training courses, previously referred to as the National Training Standards (NTS), now defined by the Learning and Development Branch as Mandatory Training. For the purpose of this audit, these courses will be identified as NTS.

These courses "represent the fundamental learning and development requirements an employee will be provided in order to perform certain aspects of their roles and responsibilities. All training activities are compulsory by the deadline indicated in the individual trainings, in order to meet the various legal requirements".

It should be noted that while all NTS courses are mandatory, this suite of courses does not include all mandatory regulated and legislated training, such as Workplace Hazardous Materials Information System (WHMIS).

During the time this audit was conducted, the number of NTS courses totalled 62; an increase from 39 in 2005-06.

There are three types of NTS courses:

1. CSC driven training, which is developed by CSC, e.g. the Correctional Training Program (CTP) for correctional officers;
2. Regulated training, e.g. First Aid and CPR; and,
3. Government of Canada Required Training, e.g. Canadian School of Public Service (CSPS) Orientation to the Public Service.

These courses are delivered by internal or external instructors, at various locations such as institutions, staff colleges or Canada School of Public Service sites. In instances where availability of resources is an issue, as with institutions that do not have a firing range, training is conducted off-site at privately run locations.

Every employee³ at CSC has a career plan that identifies which NTS courses are required to perform their job, based on their classification, position and work location. Each NTS course has at least one competency associated with it; they are created and assigned by CSC and are unique to the organization. These competencies are used to track employee compliance with the mandatory training requirements in the Human Resource Management System (HRMS). Employee career plans are determined and manually entered into HRMS by the Staff Training Coordinators (STC), who are responsible for monitoring all NTS training activities at the institution/site/district level.

Organizational Structure

The STC generally works on-site at the institution and reports to the Manager of Management Services at the regional staff college, who, in turn reports to the Director of the Regional Staff College.

The five Regional Staff College Directors report to the Director General of Learning & Development (L&D) at NHQ, who reports to the Assistant Commissioner Human Resources Management (ACHRM). The L&D Branch also includes three divisions at NHQ: Policy, Planning and Performance; Training, Design and Development; and, Implementation of National Learning Programs, Partnerships and Infrastructure.⁴

Risk Identification and Analysis

A risk analysis to identify the most important risks facing CSC training was completed by the audit team. The assessment was based on an examination of the management framework and organizational governance structure, a review of past audits, and interviews with L&D staff.

Training demands are expected to increase significantly as hiring is underway in response to the projected increase in offender population resulting from newly passed legislation. As such, the audit team concluded that CSC's main risk relates to ensuring compliance with mandatory training requirements for those staff working directly with offenders.

It was also brought to the attention of the audit team that reporting information pertaining to training may be inconsistent and unreliable.

These risks were considered in establishing audit objectives and developing supporting audit tools for testing and gathering of evidence.

2.0 AUDIT OBJECTIVES AND SCOPE

2.1 Audit Objectives

The audit objectives are:

• to provide assurance that a Management Framework is in place for training activities and that it is implemented and monitored; and,
• to provide reasonable assurance that CSC is complying with relevant legislation, policy directives and guidance pertaining to the National Training Standards (NTS). Specific criteria related to each of the objectives are included in Annex A.

2.2 Audit Scope

The audit was national in scope and focused on compliance with CSC's NTS, including TB required training, regulated training, and CSC-driven training. It also focused on substantive CX, WP and AS employees⁵, as these staff classifications represent more than 60% of CSC's employees and require mandatory courses from each type of NTS.

Non-mandatory training, developmental training, or professional certifications and designations, all of which are not included in the NTS were considered out of the scope of this audit. Also out of the scope, was how individual mandatory training courses are developed, maintained, or delivered. Additionally, the audit did not assess the quality of individual courses, or their adherence to legislative or policy requirements.

The audit examined the controls in place to ensure that training is completed in compliance with the NTS, and that compliance is monitored and reported in an accurate manner. The audit focused on NTS training activities for 12 selected courses as of September 30, 2011 and employee career plans as of January 6, 2012. These courses were selected as they represented courses from CSPS training, CSC driven – training, and regulated training.

However, the audit did examine how the NTS suite as a whole is approved, updated and amended.

3.0 AUDIT APPROACH AND METHODOLOGY

The approach included site visits to three regions, interviews with institutional, regional and NHQ staff, and a review of relevant policies, legislation and documentation such as strategic plans and compliance reports. It also included a review of a sample of 372 employee files from across all regions.

Annex B lists and describes the techniques used to gather evidence to complete this audit.

4.0 AUDIT FINDINGS AND RECOMMENDATIONS

4.1 Management Framework for Training Activities

The audit team assessed the extent to which a management framework for training activities is in place. This included an examination of directives and guidelines, organizational structures, and roles and responsibilities. The audit team also examined planning processes and reporting and monitoring systems in place to support training activities.

4.1.1 Policy and Legislative Framework

We expected to find that CSC policies and guidelines were consistent with government legislation and policies relating to training, and that these policies and guidelines were communicated and implemented consistently across CSC.

CSC's policies and guidelines for training activities are consistent with legislation and TB policies; however, updating of CSC's Learning, Training and Development Guidelines and Staff Training Management Control Framework checklist is required.

Under the authority granted by the Treasury Board policy on Learning, Training and Development, CSC developed the Learning, Training and Development Guidelines, 2007 (the Guidelines). The objectives of the Guidelines are to outline CSC's commitment to continuous learning, and to clarify roles, recognizing that learning is a shared responsibility.

The audit found that training is occurring in accordance with TB policies and CSC guidelines. CSC has a Management Control Framework in place for staff training, however, the framework still references the TB Policy for Continuous Learning in the Public Service, which ceased to be in effect as of December 2005. The Guidelines currently in place outline objectives, priorities, roles and responsibilities of some CSC officials, as well as monitoring and reporting requirements. However, roles and responsibilities and monitoring and reporting requirements require clarification and updating to ensure training activities and accountabilities are applied and implemented consistently across CSC.

Specifically, the Guidelines do not address the roles of other key positions involved in learning and development activities, such as: NHQ L&D directors; regional staff college personnel such as: staff training coordinators and managers of management services; institutional heads and the unfunded position of Correctional Manager, Scheduling and Deployment. Additionally, the Guidelines have not been updated to address the closure of the Correctional Management Learning Centre or the establishment of the National and Regional Learning and Development Committees, which were created as a result of a review performed in 2008 and included in the CSC Transformation Agenda.

The Terms of Reference for the National and the Regional L&D Committees clearly outline their mandate, membership and meeting requirements. To reinforce and strengthen the centralized management role of the L&D function, the National and Regional Committees and the L&D Governance Board should be referenced in the Guidelines as well.

The need for greater clarity on the subject of roles and responsibilities and monitoring requirements will be examined in further detail in sections 4.1.2 and 4.1.3 of this report.

Policies and guidelines regarding training are communicated and implemented across CSC; however, clarification with regard to the targeting of employees for certain NTS courses is required.

All employees interviewed were aware of, and able to, identify CSC policies and guidelines used to direct training activities at CSC. Information regarding these policies and guidelines is communicated at the national, regional and institutional levels, through various formats such as GEN-COMM emails, the L&D website, management meetings, conference calls, and staff briefings.

When asked during interviews if policies and procedures related to NTS were clear, 83% (79/95) answered yes. More than half of staff (58%) interviewed commented that improvements could be made to the NTS, with the majority of these responses pertaining to the application of specific NTS courses to certain employees. Specifically, they were sometimes unclear as to which employees should be included in a course's target group. While 54% stated clarification of target groups is required for some NTS courses, examples of other concerns noted during the interviews include the following:

• 38% had difficulty tracking NTS changes, such as target groups or recertification cycles, from year to year;
• 34% believe there are too many NTS courses;
• 27% expressed concerns with instructor /trainer issues such as availability;
• 20% said the NTS planning cycle/ process needs to be improved; and,
• 16% voiced a need for a more centralized approach to scheduling, such as block training, as opposed to an individual institutional approach.

During interviews, STCs confirmed that when faced with specific NTS questions, they seek and receive guidance from numerous different sources, be it STC meetings with the regional staff college or L&D at NHQ. Some STCs noted that an electronic system called Sharepoint has been used for posting questions and receiving answers, though it appears this system has been inconsistently used and monitored in past months. The inability of the STC to obtain information from one central point of contact poses a risk to CSC that national training standards will not be interpreted and applied consistently across all regions and institutions.

For this reason, L&D branch should promote the use of a central point of contact, such as SharePoint, to ensure questions are received, responded to, and monitored to support the sharing of best practices among CSC employees, and the consistent application of national training standards across CSC.

Planning Processes

We expected to find that planning processes were in place to ensure that CSC meets its learning and development requirements.

Planning processes are in place at the institutional, regional and national level, and are supported by committees; however, there is limited national coordination in some key areas.

Institutional Planning

All institutions have training plans and processes in place, which are developed though various committees such as Institutional Personnel Committees and Institutional Management Team Meetings. Ninety percent of institutions included in the audit have these plans formally documented, and are shared with regional management only.

The audit team noted that 18 of 28 sites (64%), including staff colleges and institutions, had addressed risks associated with the implementation of new legislation such as the Truth in Sentencing Act, in their planning processes; the learning and development needs of the remaining sites had not yet been impacted by new legislation.

Regional Planning

As per the Guidelines, regional staff college directors are responsible for developing, implementing and monitoring an annual regional training plan to ensure NTS are met. The audit found that each regional staff college develops an annual training plan and calendar based on the number of targeted employees and the number of training sessions required. One of the principle functions of the Regional L&D Committee is to determine and or approve the regional training plan. This plan is then provided to regional management for further approval, however, regional plans are not presented to national management for approval.

Sector-based Planning

Nationally, the L&D Branch works with all Sectors of CSC to respond to CSC's overall learning and development needs. The Learning and Development Integrated Business Plan 2011-2012 outlines these ongoing business activities. It includes strategic objectives and an HR plan for NHQ and the Regional Staff Colleges. L&D needs are also addressed in CSC's Strategic Plan for Human Resource Management.

Specifically related to the planning for NTS, the L&D Governance Board is responsible for approving the suite of NTS courses on an annual basis. In support of this approval process, the L&D branch issues call letters to all Sector heads in the fall, requesting submissions for changes and additions to the NTS for the upcoming fiscal year. All submissions for new training require sector head approval and must identify a source of funding and impact of training days; changes to existing NTS must identify amendments to target groups and frequency. This system of requesting and approving NTS was instituted as a control measure to address the issue of the growing number of courses being added to the NTS each year.

Correctional Training Program (CTP) Planning

In contrast to NTS courses that are delivered by the institutions, the Correctional Training Program (CTP) is controlled by the L&D branch, through the Regional Staff Colleges. The CTP is the principal induction NTS course that is administered by all regional staff colleges. To effectively plan and monitor this course, the L&D branch has a national training program schedule of all CTP sessions across all regions to facilitate the overall planning and administration processes.

Conclusion

With the exception of compliance reports, the audit team found no evidence of links between institutional plans and the L&D branch. This lack of coordination between the institutions which administer some NTS, such as firearms training, with the L&D branch responsible for reporting on compliance with all NTS, creates an information gap. The result is, the L&D branch is unable to easily determine how, where and when courses are scheduled at the institutional level, and at what cost and level of efficiency to the organization. A centralized approach to planning and administering NTS courses would strengthen the controls and accountability of the centralized governance structure already in place.

Monitoring and reporting requirements are discussed in greater detail in section 4.1.3.

4.1.2 Roles and Responsibilities

We expected to find that roles and responsibilities for training were clearly defined and consistent with CSC policies and guidelines. We also expected to find that roles and responsibilities were consistent across the regions.

Roles and responsibilities of training staff are documented and understood; however, clarification of duties and alignment of work descriptions would support the equity of duties at a national level.

The majority of employees interviewed stated that they had work descriptions for their positions, and that they understood their roles and responsibilities regarding training.

The generic work description for Staff Training Coordinators (STCs) (AS-02) was created in April 2006. STCs are responsible for the coordination and monitoring of all training activities at the institutional level such as: determining employee career plans; scheduling training activities; inputting training data into HRMS; and, monitoring compliance.

Most STCs are physically located away from the college, on-site at an institution or district office, where they work closely with the Correctional Managers, Scheduling and Deployment (CMSD), and functionally report to the institutional head. Interviews revealed that this work environment can be challenging as STCs are required to balance priorities and respond to the operational requirements of both the regional staff college and the institutions despite the fact that STCs report only to the staff college. Interviews further revealed that this environment can be increasingly challenging if an STC is responsible for multiple sites or is remotely located.

Of the 23 STCs interviewed:

• 7 provide support to more than one site; 2 perform the additional role of registrar;
• 6 provide remote service to a site;
• The average number of CSC employees per STC responsibility per site is 344 (ranges from 200 – 520); and
• The average years' of experience per STC is 2.1; and
• 8 have been in the position for less than 1 year.

Interviews also established that the responsibilities and workloads of STCs vary according to the region and the site, resulting in a marked lack of consistency in duties from a national perspective. For example: the STCs in two regions are required to manage institutional training budgets; in some institutions the STCs attend all management meetings pertaining to training, whereas some STCs attend no meetings; and in a few institutions, the audit team noted that STCs also hold operational duties such as identifying and scheduling employees in the Scheduling and Deployment System (SDS).

STC roles and responsibilities also vary depending on the type of course offered. For example: CSPS registration, tracking and reporting is managed by the National Required Training Coordinator at NHQ; induction courses, such as Correctional Training Programs and New Employee Orientation Program courses are run by the Regional Staff Colleges, and coordinated by the registrars. As institutional employees participate in courses offered by CSPS and the staff colleges, STCs are responsible for monitoring training activities that they do not administer, making it difficult for the STC to keep track of all NTS training activities.

The generic work description for Manager of Management Services (MMS) (AS-04) was created in December 2005. The MMS, located at the Regional Staff College, supervises the STCs and serves as the link between the STCs and the college. The MMS also manages the provision of all administrative services, facilities, and maintenance at the staff college.

On average, the MMS manage nine STCs. Two of four MMS' interviewed had been in the position for less than one year.

As with the STCs, the responsibilities of the MMS differ from region to region. For example, the staff college building in Atlantic is leased; therefore, the MMS has the responsibility of managing contracts with PWGSC, which is not required in regions where the staff college buildings are owned.

Roles and responsibilities of training staff should be consistently articulated across all regions to ensure equity of duties and responsibilities.

4.1.3 Monitoring and Reporting

We expected to find monitoring and reporting practices were in place to ensure that CSC meets its reporting obligations under TB policy, and that management receives accurate and timely information.

Monitoring and reporting practices exist at the institutional, regional and national level; however, the frequency and content of monitoring is inconsistent across institutions and regions, and reporting efforts are duplicated at most sites.

As per the TB policy on Learning, Training and Development, CSC provides information to the Canadian School of Public Service as requested in order to fulfil its annual required training monitoring obligations.

As stated in the Guidelines, CSC is required to monitor and report on compliance with all mandatory training requirements, and this responsibility is further assigned to the Director General, L&D and the Regional Staff College Directors. The Guidelines do not provide guidance as to how often monitoring should occur, or to whom reports should be provided. Despite a lack of formal guidance, all sites examined in the audit have monitoring and reporting practices in place. Of the 23 STCs interviewed 21 maintain their own spreadsheets to compare against compliance reports generated from HRMS. The most cited reasons for keeping a separate spreadsheet was the perception of outdated or incomplete information in HRMS due to employee movement. Objective 2 of this report examines HRMS data as it pertains to training requirements.

The audit team observed that spreadsheets are also used to monitor employee anniversary dates for some NTS courses that require recertification. While some NTS, such as Creating a Harassment Free Work Environment are "one time only" training, other NTS such as firearms competencies require yearly recertification. As such, the monitoring of numerous anniversary and expiry dates for multiple NTS and hundreds of employees was reported to be a challenging and time consuming endeavor for the STCs. The audit team noted in the Pacific region that anniversary dates were reported to be easier to manage with the introduction of the Block Training⁶ pilot program.

Not only is the content of the reporting inconsistent, but the frequency of reporting at the institutional and regional levels varies widely, as does the direction of reporting. Institutional heads receive reports from the STC on a weekly to bi-weekly basis; regional management receives reports on a weekly to monthly to quarterly basis, depending on the region. As an example of this inconsistency, the audit team found that in one region, RHQ provides compliance reports to the staff college and institutions for verification, yet in other regions the STC from each institution provides data to the staff colleges, which in turn provide compliance reports to RHQ.

Generating reports for different audiences with varying timelines and formats is time-consuming for STCs and leads to multiple reports with sometimes contradictory results, reinforcing the STCs' need to maintain a separate tracking system.

At the time of the audit, L&D was in the process of developing a training "dashboard" which will allow CSC management at all levels to access training reports as required, thereby lessening the reporting burden and duplicate monitoring of the STCs. To ensure consistent, reliable reporting results, NHQ should implement a standardized/centralized monitoring and reporting process, such as the aforementioned "dashboard". However, the success of this "dashboard" will be dependent on the accuracy and timeliness of the data in HRMS, and CSC staff willingness to use HRMS as the sole system for tracking training activities.

As per the Guidelines, the L&D Branch is responsible for reporting on compliance with NTS, and does so by generating and providing mid-year and year-end NTS compliance report updates to senior management.

In 2010-2011, the L&D Branch implemented a new compliance reporting methodology in order to provide senior management with a more accurate view of employee training compliance. Under the previous compliance reporting methodology, employees who had not taken training, but who were pending or scheduled for training, were recorded as being compliant and meeting the required competency. Compliance is now represented by the number of current active staff meeting the competency required divided by the number of current active staff that requires the competency, therefore those pending employees are now considered non-compliant. This new methodology illustrates the true rate of compliance at the operational level. The most recent L&D compliance report presented to senior management was the 2011-2012 mid-year review in January 2012.

CSC strives for 100% compliance with all NTS courses and their associated competencies. When asked, 89% of interviewees stated that there was a stated/ expected compliance rate for NTS, and 78% stated that 100% was the expected rate. The audit team noted that institutional staff was more likely to respond that compliance was 100%, than staff located in regional administration and at NHQ. When the response was less than 100%, staff generally made reference to the need to balance the risk between achieving operational readiness against achieving an overall compliance rate.

In July 2011, to support the implementation of the new compliance reporting methodology, the L&D branch undertook a risk analysis of NTS courses that focused on key operational training that had the greatest impact on the organization with regard to operational risk, operational readiness, and legal risk. While this exercise identified 14 NTS as having a greater impact on operational readiness than the remaining 48 NTS, the audit team noted that a 100% compliance rate was expected for all 62, regardless of the assessment of their lower associated risks.

Conclusion

• Training is taking place in accordance with CSC policies, and these policies are in keeping with Government of Canada legislation and TB policies; however, L&D Guidelines and CSC's Management Control Framework require updating.
• Formal planning of training activities takes place at all levels; however, the overall planning and administration of NTS is not centralized as is set out in the current governance structure.
• Roles and responsibilities are for the most part understood; however, the responsibilities of key positions such as the STC and MMS differ among regions and institutions.
• Monitoring and reporting processes are established at institutional, regional and national levels; however, duplicate monitoring systems exist at the institutional level, and there is a lack of overall coordination between all three levels.
• NTS are not formally risk-ranked to ensure courses with the highest operational risk are planned for, monitored and reported on accordingly.

4.2 Compliance with Policies and Guidelines Pertaining to NTS

The audit team assessed whether CSC is compliant with relevant legislation, policy directives and guidance pertaining to the National Training Standards. This included a review of employee files and training documentation, verification of data in the Human Resources Management System (HRMS), data entry practices, and quality assurance controls.

4.2.1 Quality Assurance and Controls

We expected to find controls in place to ensure that data related to training is entered in HRMS is accurate, consistent, and occurs in a timely manner.

Quality control is being performed on the data entered and the reports generated; however, cyclical monitoring and analysis is required to ensure discrepancies are being corrected and continuous improvement is occurring.

Nationally, the L&D Branch recently implemented a Career Plan Validation and Spot Checking Business Process for the monthly monitoring of career plans. Career plans are the baseline against which employee compliance with NTS is measured. All employees, with the exception of those in term positions of less than four months and students, should have a career plan that aligns with their classification, job title and personal learning plans.

A career plan for each employee is determined by and entered into HRMS by the STC at the beginning of each fiscal year. Although L&D has provided guidance documents (pivot table and mandatory training requirement checklist) to facilitate this process, it remains a time-consuming exercise, as each career plan must be entered manually. For example, the average CX employee career plan has 13-16 required competencies, and the average STC is responsible for tracking the activities of 344 staff.

The Career Plan Validation and Spot Checking Business Process includes reports of missing career plans, duplicate competencies and open training sessions, which are sent to each staff college to be actioned by the appropriate STC. As a follow-up, NHQ performs random spot checks on the affected career plans to confirm if corrective action has been taken.

The audit team asked 37 staff (including staff college personnel and L&D) if quality assurance was performed on the data entered into HRMS. 25/37 (68%) said yes, and the balance either said that it was not performed (27%) or that they did not know (5%), which suggests that the current QA business process in place is not clearly communicated to, or understood by, affected staff.

The audit team did not find any evidence of a formal follow-up process on identified quality assurance issues, and is therefore unclear if the QA process is successful in improving career plan data entry from month-to-month. While QA is being conducted on the data, NHQ should include a formalized follow-up and reporting cycle to the existing QA business process as a means of continually improving career plan and compliance reporting data.

4.2.2 Documentation

We expected to find that financial management policies and authorities are established and communicated.

Substantiating documentation is kept on file; however, files are not managed consistently.

The audit team planned to conduct a file review of 108 employee training files; however, individual employee training files do not exist at all institutions. While most STCs manage training activities by employee, some STCs manage training activities by course title and delivery date. Therefore, obtaining documentation to determine if substantiating documentation was kept for 12 selected NTS was more challenging than expected (Annex E).

Interviews confirmed that there are no formal guidelines for document retention or file management. Document review demonstrated that STCs obtain and maintain different documentation for various courses. The document examination also revealed inconsistencies with document storage, as some documents are held with the STC on-site, while other documents are filed with central registries, or with the registrars at the staff colleges, or with the National Coordinator at NHQ.

Of the 108 employees reviewed, 31% had all required documentation to substantiate that training had occurred for the 12 NTS selected. This number rose considerably when the audit team looked at documentation pertaining to courses administered at the institutional level. For instance, 92% (54/59) of CXs files examined had all documents to confirm firearms training had occurred.

During interviews, the STCs were asked how long training documentation should be kept. The answers were wide-ranging between institutions and regions, with responses varying from two years to an indefinite period of time.

A complete training record for each employee should consistently exist to enable the transferring of training documents when an employee relocates to another institution. Moreover, the lack of a complete training record could complicate the retrieval of documentation for employees, or the confirmation of training history, in the event that information is required for an incident or investigation.

To ensure training documents are managed consistently, national Standard Operating Procedures that include detailed business processes about document storage, retention, location and file management should be developed, particularly for those courses required to fulfill legislative requirements and ensure operational readiness.

Data is captured in HRMS

We expected to find that mandatory NTS training activities have been captured in HRMS, as per L&D Guidelines.

Data is entered into HRMS, with minor inconsistencies detected.

As per the Guidelines, all learning activities are tracked in the Human Resources Management System (HRMS).

The audit team examined a statistically valid random sample of 372 employee career plans and determined that 99% of the employees reviewed had career plans. The audit team performed a secondary examination of 108 career plans to verify their accuracy and determined that 70% of career plans contained the NTS courses required to meet the requirements of their positions.

This secondary examination also revealed that between September 30, 2011 and January 6, 2012, 29 of 372, or eight percent of the employees in the audit sample had changed classifications. Of these, 11 of 29 (38%) took longer than one month to enter into the system. This is important because comments from staff indicated that they believe there is a significant number of staff whose information is not accurately documented due to changes in classification.

As for training data that is entered into HRMS, the audit team verified that training documentation kept by the STCs, such as dates of training and competencies granted, was consistent with the data entered into HRMS. As the audit team detected only minor inconsistencies with data that is entered into HRMS, CSC staff concerns with regard to the quality of the HRMS data appear, from this audit, to be unfounded.

CONCLUSION

• Quality assurance is being performed on the data entered and the reports generated. However, a formalized follow-up and reporting cycle is required to ensure continuous improvement.
• Substantiating documentation is kept on file but there are inconsistencies in the type of documentation kept on file and training documentation is not managed consistently. This poses a challenge in ensuring that a history of training for each CSC employee is maintained and is accessible should the need arise.
• Data is entered into HRMS accurately with only some minor inconsistencies detected. There was no evident reason as to why CSC staff cannot rely on this data.

5.0 OVERALL CONCLUSION

Overall, the audit found that CSC employees are receiving mandatory training as per the NTS, and that these training activities are monitored regularly by way of compliance reports provided to senior management. The audit also found that CSC is in compliance with legislation and policy with regard to the National Training Standards examined in this audit.

The results of the audit demonstrate that a management framework to support training activities is in place. More specifically, the audit team found that:

• training is taking place in accordance with CSC guidelines, and these guidelines are in keeping with GoC legislation and TB policies;
• formal planning is occurring at all levels;
• roles and responsibilities are for the most part understood;
• monitoring and reporting processes are established at institutional, regional and national levels, and governance structures exist to support these processes;
• training activities are documented and recorded in the Human Resources Management System (HRMS), and are subject to quality assurance activities.

The audit identified areas that require attention, as follows:

• L&D Guidelines and CSC's Management Control Framework for Staff Training require updating;
• the overall planning and administration of NTS is not centralized as per the current governance structure;
• roles and responsibilities of key positions need to be clarified because they differ among regions and institutions causing inequitable duties and responsibilities;
• duplicate monitoring systems exist at the institutional level, and there is a lack of overall coordination between all three levels;
• NTS are not formally risk-ranked to ensure courses with the highest operational risk are planned for, monitored and reported on accordingly;
• training documentation is managed differently across regions and institutions, which could lead to disorganized file transfers and training histories ; and,
• quality assurance activities performed on data do not include a formal follow-up process to ensure continuous improvement is occurring.

ANNEX A

AUDIT OBJECTIVES AND CRITERIA

ANNEX B

AUDIT APPROACH AND METHODOLOGY

Audit evidence was gathered through a number of methods as follows:

Institutions were selected for the audit based on CSC's Internal Audit Branch Analysis of Site Coverage for Internal Audit/Review Engagements FY 2007-08 to 2010-11 to prevent over-auditing of sites, and to ensure equal coverage across regions and institutions.

Interviews: 97 interviews were conducted either in person, by videoconference or by teleconference with staff from Learning and Development (NHQ), all five Regional Staff Colleges, and from selected institutions (see Annex D for a list of interviewees).

Review of documentation: Relevant documentation such as strategic plans, meeting minutes, process documentation, performance and compliance reporting, training files, procedure manuals and training material was reviewed.

Testing: File review was performed to provide assurance that training compliance data is entered, monitored and reported accurately. A statistically valid random sample was selected (using a 95% confidence level and 5% confidence interval) from an HRMS employee report for all CX, WP and AS employees across Canada, resulting in a population of 372 career plans. From this sample, a sub-sample of 20 files per Region (10 from NHQ) was selected for a secondary documentation review of career plans and substantiating documentation for 12 NTS courses (Annex E).

ANNEX C

LOCATION OF SITE EXAMINATIONS/VIDEOCONFERENCE INTERVIEWS

ANNEX D

LIST OF INTERVIEWEES

ANNEX E

MANDATORY COURSES (NTS) INCLUDED IN SECONDARY DOCUMENTATION REVIEW

The following courses were selected as they apply to the greatest number of CX, WP and AS employees:

CSC-Driven:

Correctional Training Program (CTP), 2008

Parole Officer Orientation

New Employee Orientation Program (NEOP)

Firearms 9mm - initial

Firearms 9mm - refresher

Firearms 9mm - transition

Regulated Training:

First Aid/CPR - basic

First Aid/CPR - standard

Anti-Harassment (JLP) - for employees

Anti-Harassment (JLP) - for management

Required Training:

CSPS Orientation to the Public Service

CSPS Essentials of Managing in the Public Service

ANNEX F

AUDIT OF TRAINING ACTIVITIES MANAGEMENT ACTION PLAN (MAP)

** PLEASE NOTE THAT THE ABOVE ACTION PLAN IS BASED ON PROPOSED L&D MODEL FUNDING LEVEL**

Recommendation:	Recommendation 112 The Assistant Commissioner, Human Resources Management should update fundamental documents pertaining to NTS training activities to accurately reflect the current governance structure and define the roles and responsibilities of key personnel.
Management Response / Position:	Accepted Accepted in part Rejected

Recommendation:

Recommendation 213 The Assistant Commissioner, Human Resources Management should assess and rank the full complement of the NTS courses to identify mission-critical courses to ensure that NTS courses address the operational risks, fulfill legislative requirements and meet the operational readiness of CSC in an effective and efficient manner. The Assistant Commissioner, Human Resources Management should examine the current L&D service delivery model and identify opportunities to fully centralize planning, monitoring and reporting processes to ensure the consistent application and administration of NTS courses across CSC.

Management Response / Position:

Accepted Accepted in part Rejected

Recommendation:	Recommendation 314 The Assistant Commissioner, Human Resources Management should assess and rank the full complement of the NTS courses to identify mission-critical courses to ensure that NTS courses address the operational risks, fulfill legislative requirements and meet the operational readiness of CSC in an effective and efficient manner.
Management Response / Position:	Accepted Accepted in part Rejected

Recommendation:	Recommendation 415 The Assistant Commissioner, Human Resources Management should include a formalized follow-up and reporting cycle to the existing QA business process as a means of continually improving career plan and compliance reporting data.
Management Response / Position:	Accepted Accepted in part Rejected

Recommendation:

Recommendation 516 The Assistant Commissioner, Human Resources Management should, in consultation with the Senior Deputy Commissioner, develop and implement national Standard Operating Procedures that include detailed business processes for document storage, retention, and file management, particularly for those courses required to fulfill legislative requirements and ensure operational readiness.

Management Response / Position:

Accepted Accepted in part Rejected