Audit of Policy Development Processes

Internal Audit Report

October 20, 2017

PDF

Executive Summary

What We Examined

The Audit of Policy Development Processes was conducted as part of the Correctional Service Canada (CSC) Internal Audit Sector’s 2016-2019 Risk-Based Audit Plan (RBAP).

The objectives of this audit were to:

  • assess the extent to which a management framework is in place to support the effective management of policy development;
  • assess the extent to which policy development processes support the achievement of CSC’s objectives, including the management of its risks; and
  • assess the innovation maturity of CSC’s policy-related management framework and development processes.

For the first objective, the audit examined whether: roles and responsibilities of employees involved with policy development are clearly defined, communicated, understood, and accepted; the organization considers its current and future human resource requirements related to policy development; employees are provided with the necessary training, tools, and resources to develop policies; and policy development activities are monitored to help ensure the processes are effective.

For the second objective, the audit examined whether: external and internal environments are monitored to identify issues with CSC policies, and these issues are analyzed to determine policy solutions; consultations occur with key stakeholders in a meaningful manner; policy instruments are coordinated and aligned both internally and externally; policy instruments are sufficiently reviewed and approved by an individual with the appropriate level of authority; and new policy requirements and policy changes are effectively communicated to all stakeholders.

For the third objective, CSC’s innovation maturity was examined based on six factors: strategy, people, measures, methods, communication, and breadth. It should be noted that the work performed for this objective was categorized as review level assurance and was thus not commensurate with the high level of assurance provided through the first two objectives of this engagement.

The audit was national in scope and included policy development processes for policy instruments in place at the national, regional, and local levels. The audit did not include an assessment of the development processes for internal services (human resources, finance, communications, etc.) policy instruments.

Why it’s Important

This audit links to CSC’s corporate priority of "efficient and effective management practices that reflect values-based leadership in a changing environment" and the following corporate risk: "there is a risk that CSC will not be able to implement legislative changes and ensure the financial sustainability of the organization."

The emphasis that CSC places on effective and clear policy is evident through its 2016-2017 Report on Plans and Priorities. The report touches upon several areas within CSC where strong policy development processes are paramount. For example, CSC indicated it would continue to enhance its policy framework to address any systemic barriers to effectively manage Indigenous offenders. In addition, as a key activity, the Health Services Sector identified the development and implementation of policy and programs to ensure patient safety and improve program delivery. Innovation has been included in this audit because it is a government priority and was identified in the audit risk assessment as a means to mitigate risk.

What We Found

With respect to the first objective, we found that a management framework is in place to support the effective management of policy development. We found that roles and responsibilities are clearly defined, communicated, understood, and accepted for most of the key players within the policy development process. We also found that from a capacity standpoint: CSC identified and assessed its current and future human resource requirements; sufficient human resources are in place to meet policy development needs; and although training is limited, a clear need was not identified. Still, we noted a few areas that require consideration by management in order to help ensure that the management framework fully supports CSC’s policy development processes. Specifically, CSC should:

  • establish and communicate roles and responsibilities between the Offices of the Primary Interest (OPIs) and the Policy Sector to help ensure sufficient oversight of the national policy development process;
  • continue to establish tools and resources for policy development and promote them to all levels;
  • develop performance expectations and monitor indicators to determine if the national policy development process is operating as intended; and
  • reinforce the responsibility for reviewing standing orders, and monitor results to determine if local and regional policies are aligned with national policies.

For the second objective, we found that policy development processes support the achievement of CSC’s objectives, including the management of its risks. We found that environmental scans are conducted externally and internally to identify policy issues, and in general, policy issues are formally analyzed and considered by management. We also found that key stakeholders are identified and engaged at the appropriate time and feedback from key stakeholders is reviewed for inclusion in policies. As well, CSC policy instruments are aligned with external requirements and are internally coordinated. CSC policies are also reviewed prior to approval and are approved by an individual with the appropriate level of authority. Finally, policies are readily accessible to all stakeholders. Still, we noted a few areas that require consideration by management in order to help ensure that risks are better managed. Specifically, CSC should:

  • provide guidance around lead-time assessments, human/financial resource assessments, and disposition requirements; and
  • assess the amount of time provided to stakeholders for consultation in order to determine its appropriateness.

For the third objective, we found that CSC is a mature organization with respect to its policy-related management framework and development processes, and seeks opportunities to become more innovative. In particular, the consultation work that CSC undertakes with a broad range of stakeholders with multiple points of view facilitates innovative policy development. Further, CSC demonstrates that it has a strategic focus on innovation by establishing means for gathering new ideas and approaches, communicates its strategies, and encourages its employees to be innovative. Finally, although CSC has established methods that support innovation, including processes and tools, within its policy development processes, it is encouraged to develop measures to track the extent to which innovation supports policy development, and further enhance its use of technology.

Management Response

Audit findings and recommendations are well received and will allow further refinement and communication of roles and responsibilities as they relate to the policy development process. As such, the Assistant Commissioner, Policy, will seek concurrence of EXCOM on key changes to clarify roles and responsibilities and collaborate with stakeholders to enhanced policies, monitoring and tools supporting CSC’s policy development process. Full implementation of the Management Action Plan is scheduled by March 31, 2018.

Glossary

Alignment:
consistency in the direction provided by individual policy instruments. For example, local direction should be consistent with national direction.
Coordination:
the integration of subject matter across CSC sectors.
Disposition:
a comparison between a draft document (or an approved document) with its previous version(s) and stakeholder comments on the previous version(s).
Intranet:
CSC’s intranet is commonly known as the "InfoNet" or "The Hub".
Lead time:
the amount of time between promulgation of policy and its expected implementation.
Policy:
for the purposes of this audit, policy refers to all formal direction issued by CSC.
Policy instruments:
documents that contain formal direction issued by CSC; which include, but are not limited to: Commissioner’s Directives, guidelines, regional procedures, standing orders, post orders, manuals, protocols, and handbooks.
Promulgation:
issuance of an approved policy instrument.

1.0 Introduction

1.1 Background

The Audit of Policy Development Processes was conducted as part of the Correctional Service Canada (CSC) Internal Audit Sector’s 2016-2019 Risk-Based Audit Plan (RBAP). This audit links to CSC’s corporate priority of "efficient and effective management practices that reflect values-based leadership in a changing environment" and the following corporate risk: "there is a risk that CSC will not be able to implement legislative changes and ensure the financial sustainability of the organization."

The emphasis that CSC places on effective and clear policy is evident through its 2016-2017 Report on Plans and Priorities. The report touches upon several areas within CSC where strong policy development processes are paramount. For example, CSC indicated it would continue to enhance its policy framework to address any systemic barriers to effectively manage Indigenous offenders. In addition, as a key activity, the Health Services Sector identified the development and implementation of policy and programs to ensure patient safety and improve program delivery.

As per CSC’s 2015-18 Corporate Business Plan, National Headquarters (NHQ) is responsible for the overall planning and policy development for CSC. Further to this, the Policy Sector at NHQ assists management in understanding, analyzing and responding to risks at all levels of the organization. The Corporate Business Plan also highlights that CSC will continue to monitor and participate in the development of government initiatives and legislation tabled in Parliament to ensure that the organization is aware of possible legislative changes and is able to develop and implement plans to ensure that policies are updated as required.

CSC issues a range of policy instruments that are designed to articulate rules to guide its activities. The policy instruments that are developed at NHQ, include, but are not limited to, Commissioner’s Directives, guidelines, and policy bulletins. Further, policy instruments such as regional procedures, standing orders and post orders are developed and maintained at the regional and local levels to support, and provide further direction, for implementing national policy instruments.

In June 2013, the Government of Canada launched Blueprint 2020, which is "a vision for a world-class Public Service equipped to serve Canada and Canadians now and into the future."Footnote 1 The Blueprint 2020 vision is guided by four principles that help examine how work is done in the Federal Public Service:Footnote 2

  • an open and networked environment that engages citizens and partners for the public good;
  • a whole-of-government approach that enhances service delivery and value for money;
  • a modern workplace that makes smart use of new technologies to improve networking, access to data and customer service; and
  • a capable, confident and high-performing workforce that embraces new ways of working and mobilizes the diversity of talent to serve the country’s evolving needs.

The following five themes were identified to guide the Public Service’s actions:Footnote3

  • Innovative Practices and Networking;
  • Processes and Empowerment;
  • Technology;
  • People Management; and
  • Fundamentals of Public Service.

Under the "Innovative Practices and Networking" theme, expectations have been placed on departments, where appropriate, to adopt new approaches that will enable employees to generate, shape and move forward on innovative ideas. Further, departments and agencies are adopting new perspectives and ways of working together to respond to policy and service delivery challenges.Footnote4 Innovation has been included in this audit because it is a government priority and was identified in the audit risk assessment as a means to mitigate risk.

1.2 Legislative and Policy Framework

Deputy heads are accountable for good public sector management. Ministers and deputy heads have authority to manage the people, resources and activities of their departments towards the objectives set out in legislative mandates and government policy. In particular, deputy heads manage by exercising authorities assigned to them by a person (for example, a minister), by a body (for example, the Public Service Commission), or by statutory instrument (such as a departmental act, the Financial Administration Act, or the Corrections and Conditional Release Act (CCRA)). Deputy heads are responsible for applying Treasury Board's mandatory policy instruments within their organization, monitoring and auditing their application, taking corrective action in cases of non-compliance, reporting to the Treasury Board Secretariat (TBS) on matters regarding compliance, and providing advice on the development of policies, often in committees and special studies coordinated by TBS.Footnote 5

Section 97 of the CCRA authorizes the Commissioner of CSC to make rules for the management of the Service, and Section 98 specifies that these rules can be designated as Commissioner’s Directives.Footnote 6 The Corrections and Conditional Release Regulations (CCRR) also authorizes the institutional heads to develop standing orders.

1.3 CSC Organization

National Headquarters

Policy Sector

To ensure the coherence of CSC’s policy framework both internally and externally, the Strategic Policy Division (SP) within the Policy Sector manages and facilitates the Service’s participation in horizontal government policy initiatives and in the development of CSC strategies related to these initiatives. SP partners with CSC’s sectors and members of the Public Safety Portfolio, and other government agencies to ensure consistent and cohesive direction. SP also manages the national policy framework and, working in partnership with subject-matter experts, plays a key role in the development of Commissioner’s Directives. SP does this by ensuring coherence and alignment of CSC's policy framework, and by providing expert policy advice. Specifically, SP is to:

  • provide support to sectors and the CSC Executive Committee on policy development;
  • ensure policies are consistent with legislation and internal policy frameworks;
  • ensure policies are understood and appropriately applied; and
  • ensure the national policy development process is respected.Footnote 7

Further, SP is to contribute to larger government priorities by ensuring CSC's position is understood and included in horizontal initiatives. In this regard, SP is to:

  • manage the development and approval process of CSC's role in interdepartmental memoranda to cabinet and other interdepartmental policy initiatives;
  • liaise with Public Safety Canada to ensure consistency with portfolio policies and priorities;
  • contribute to material required for Standing Committees of Parliament/Senate and Ministerial and government transitions; and
  • contribute to Public Safety-wide priority setting and policy development.Footnote 8
Offices of Primary Interest (NHQ Sectors)

CSC sectors are responsible for developing and revising their respective policy instruments (Commissioner’s Directives, guidelines, etc.) in consultation with SP. Each sector is responsible for providing corporate expertise on its subject matter and provides leadership on policy development and implementation.

Legal Services

Legal Services at NHQ is consulted during the creation and revision of national policy instruments (i.e. Commissioner’s Directives and guidelines) to ensure compliance and alignment with the spirit of applicable legislation and current jurisprudence. Generally, Legal Services is not involved in regional and local policy development processes.

Regional Headquarters (RHQ)

RHQs are responsible for developing regional procedures when further details are needed to ensure application of national policy. The regional deputy commissioners (RDCs) are responsible for ensuring that regional staff is consulted in the development of national policy. The regional assistant deputy commissioners correctional operations and the regional assistant deputy commissioners integrated services coordinate the involvement of both institutional and community staff. Further, when national policy is developed and updated, RDCs are responsible for ensuring that regional policy instruments, affected by the national policy, are updated to ensure they align and are consistent with the national direction.

Local Levels

The institutional heads and district directors have the responsibility to develop and maintain policy instruments (standing orders, post orders, and community protocols) that provide direction to staff to meet the requirements set forth in related legislation (CCRA, CCRR, etc.) or national policy (Commissioner’s Directives and guidelines). Standing orders and protocols are created by the respective site (RHQ, institution, district office, etc.) and approved by the institutional heads and district directors, respectively.

Other Stakeholders

Internal Stakeholders

There are a number of CSC stakeholders whose activities affect and support the development of policy instruments. These include, but are not limited to the:

  • Internal Audit Sector;
  • Research Branch;
  • Incident Investigations Branch;
  • Evaluation Division; and
  • Learning and Development Branch.

Reports generated by these groups provide insight into the successes and issues associated with policy instruments, generate recommendations for consideration by senior management, as well as information to be considered when creating and revising these instruments.

External Stakeholders

CSC is required to respond to recommendations and reports that are issued by outside organizations which may impact CSC policy. These stakeholders may provide CSC with policy recommendations in its reports. Some of these external stakeholders include:

  • Department of Justice Canada - the Department "supports the Attorney General as the chief law officer of the Crown, both in terms of the ongoing operations of government and of the development of new policies, programs and services for Canadians. The Department provides legal advice to the Government and federal government departments and agencies, represents the Crown in civil litigation and before administrative tribunals, drafts legislation and responds to the legal needs of federal departments and agencies."Footnote 9
  • Office of the Auditor General of Canada (OAG) - the OAG "serves Parliament by providing it with objective, fact-based information and expert advice on government programs and activities, gathered through audits. Parliamentarians use OAG reports to oversee government activities and hold the federal government to account for its handling of public funds."Footnote 10 OAG reports may include recommendations related to CSC’s policy framework.
  • Office of the Correctional Investigator (OCI) - the OCI produces The Annual Report of the Correctional Investigator, which makes recommendations that may trigger a need for a policy amendment or the need for a new policy, as well as ad-hoc investigations and thematic reports.
  • Coroners’ Inquests and Fatality Inquiries - every time there is a death in custody, the province's coroner is to be notified. It is the responsibility of each province to conduct a death investigation as per their own rules and territorial/provincial legislations. An inquest reviews a death and makes recommendations where necessary.Footnote 11

1.4 Risk Assessment

The Audit of Policy Development Processes was identified as an audit priority in the 2016-2019 CSC RBAP. An engagement-level risk assessment was completed by the audit team using the results of interviews, questionnaires, research, and knowledge obtained through previous audits to assist in determining areas that the audit should cover. Policy documents, audits completed by other jurisdictions and other available information related to policy development were also considered.

Overall, the assessment identified key risks associated with the management framework that is in place to support CSC policy development and the effectiveness of policy development processes. These risks have been incorporated into this audit. Innovation has been included in this audit because it was identified in the audit risk assessment as a means to mitigate risk.

2.0 Objectives and Scope

2.1 Audit Objectives

The objectives of this audit were to:

  • assess the extent to which a management framework is in place to support the effective management of policy development;
  • assess the extent to which policy development processes support the achievement of CSC’s objectives, including the management of its risks; and
  • assess the innovation maturity of CSC’s policy-related management framework and development processes.

Specific criteria have been established to assess these objectives and are included in Annex A.

2.2 Audit Scope

The audit was national in scope and included policy development processes for policy instruments in place at the national, regional, and local levels.

The audit did not include an assessment of the development processes for internal services (human resources, finance, communications, etc.) policy instruments. In many cases, these policy instruments provide direction on the implementation of Treasury Board policy, which ultimately limits the control (and associated risk) that CSC has over their development. For the file review, the audit assessed a selection of policy instruments that were created and/or revised between April 2015 and September 2016.

3.0 Audit Findings and Recommendations

3.1 Management Framework

The first objective of this audit was to assess the extent to which a management framework is in place to support the effective management of policy development. The management framework was examined from three perspectives: roles and responsibilities, capacity, and monitoring.

The following sections provide findings for each of the audit criterion and recommendations where management attention is required. Overall results for each criterion can be found in Annex A.

3.1.1 Roles and Responsibilities

We expected to find that roles and responsibilities of employees involved with policy development are clearly defined, communicated, understood, and accepted.

This criterion was partially met as there are two areas where we identified opportunities for improvement.

Roles and responsibilities between the OPIs and the Policy Sector are not clear.

Through interviews and documentation collection, it was evident that there is a lack of clarity with regard to which group is responsible for oversight of the development process when a Commissioner’s Directive and/or guideline is being created or revised. For example, it was not clear which group was responsible for: establishing and monitoring milestones and timelines; ensuring that mandatory steps are followed; determining how documentation will be shared (between the groups), stored, and retained for future reference; and monitoring the process to ensure it is working efficiently and effectively. Although SP has created a detailed guide to policy development that outlines roles and responsibilities, this guide has been in draft form for quite some time and is not published. Without clear establishment of roles and responsibilities between the OPIs and the Policy Sector, there is an increased risk that the national policy development process is not operating as efficiently or effectively as it could. 

Responsibilities related to the review of standing orders are not fully communicated or accepted.

CSC Guidelines 005-1 Institutional Management Structure: Roles and Responsibilities includes a matrix that defines and communicates the responsibility for reviewing standing orders related to operations to the correctional manager operations (male maximum and male medium security institutions) and correctional managers (male minimum security and women’s institutions), and for reviewing standing orders related to interventions to the assistant warden interventions. This guideline is published on CSC’s intranet and is therefore available to all managers who are prescribed this responsibility. However, it was unclear how these managers were informed of this responsibility. A review of national generic work descriptions and a sample of performance agreements revealed that this responsibility is not clearly communicated via these mechanisms. In effect, standing orders were not sufficiently aligned with national policy (refer to section 3.2.3 for results) which in turn increases the risk of front line non-compliance with national policy.

3.1.2 Capacity

We expected to find that the organization considers its current and future human resource requirements related to policy development; and employees that are responsible for developing policy are provided with the necessary training, tools, and resources to develop policies.

The following areas met the audit expectations for this criterion:

  • current and future human resource requirements are identified and assessed;
  • based on interviews with management at all levels and analytical reviews of human resource plans (including organizational charts), we found that sufficient human resources are in place to meet policy development needs; and
  • although training is limited, a clear need for more training was not identified.

Overall, this criterion was met with one exception.

Tools and resources are not sufficiently established or communicated.

SP publishes various tools and reference material on its intranet site, including: templates for creating policy instruments; a link to the Report of the 2008 Policy Review Task Force; and a policy development process diagram. As such, there are some tools and resources available to assist staff in the policy development process. However, we heard through interviews with staff and management at the regional and local levels that there is limited awareness of these tools and resources, and as such, they are not being used to their full potential. Further, staff and management at all levels felt that there is a knowledge void with respect to "how to" develop policy instruments and how the policy development process works at all levels. As presented in Section 3.1.1, SP has created a draft guide for national policy development that has the potential to fill this void if approved and sufficiently communicated. The limited awareness of existing tools and resources can be attributed to a breakdown in communication in their availability and an indirect relationship between SP and the regional and local levels. Insufficient establishment and communication of these tools and resources has an impact on the efficiency and effectiveness of the policy development process.

3.1.3 Monitoring

We expected to find that policy development activities are monitored to help ensure the processes are effective. This criterion was not met.

Although CSC effectively monitors policy issues and related performance information, there is a lack of monitoring to determine if the various steps in the policy development process are operating effectively.

SP published a policy development process diagram on its intranet site in January 2017 (see Annex C). This diagram outlines the key steps in the national policy development process, and assists SP when it is tracking progress of individual policies that are undergoing development. However, no clear performance expectations or indicators have been established for the key steps throughout the process. As a result, CSC is not able to determine through sufficient analysis if the amount of time provided for consultation and review is adequate, if lead time for policy promulgation is sufficient, or if communication of new policy is done effectively. Further, a process map has not been established for local or regional policy development, and as such, the same result has been found at these levels.

Conclusion - Objective 1

Overall, we found that a management framework is in place to support the effective management of policy development. From the three perspectives for which this objective was examined, we found that roles and responsibilities are clearly defined, communicated, understood, and accepted for most of the key players within the policy development process. We also found that from a capacity standpoint, CSC has identified and assessed its current and future human resource requirements; sufficient human resources are in place to meet policy development needs; and although training is limited, a clear need was not identified.

Still, we noted a few areas that require consideration by management in order to help ensure that the management framework fully supports CSC’s policy development processes. Specifically, CSC should:

  • establish and communicate roles and responsibilities between the OPIs and the Policy Sector to help ensure sufficient oversight of the national policy development process;
  • continue to establish tools and resources for policy development and promote them to all levels;
  • develop performance expectations and monitor indicators to determine if the national policy development process is operating as intended; and
  • reinforce the responsibility for reviewing standing orders, and monitor results to determine if local and regional policies are aligned with national policies.
Recommendation 1

The Assistant Commissioner, Policy should:

  • establish and communicate roles and responsibilities between the OPIs and the Policy Sector;
  • continue to establish tools and resources for policy development and promote them to all levels; and
  • develop performance expectations and monitor indicators to determine if the national policy development process is operating as intended.
Management Response

Agree. The Assistant Commissioner, Policy will further clarify with EXCOM the roles and responsibilities of OPIs and the Policy Sector, and will review, update and provide relevant policy development tools and ensure these are communicated at all levels across the organization. As well, in collaboration with stakeholders, the Assistant Commissioner, Policy will ensure performance measures are established and monitored.

Recommendation 2

Regional Deputy Commissioners should:

  • ensure that institutional heads reinforce the responsibility for reviewing standing orders.

Regional Deputy Commissioners and the Assistant Commissioner, Health Services should:

  • establish a process to ensure regional and local policy instruments are updated whenever national policies (including guidelines) are promulgated.
Management Response

Agree. In collaboration with the Assistant Commissioner, Policy and Assistant Commissioner, Human Resource Management, the Regional Deputy Commissioners will clarify and reinforce the responsibilities of regional and institutional staff related to the review of standing orders. In collaboration with the Assistant Commissioner, Policy, the Regional Deputy Commissioners and the Assistant Commissioner, Health Services will establish a process to update regional and local policy instruments to ensure compliance with national policy.

3.2 Development Processes

The second objective was to assess the extent to which policy development processes support the achievement of CSC’s objectives, including the management of its risks. Policy development processes were examined from five perspectives: issue identification and analysis; consultation; coordination and alignment; review and approval; and communication.

The following sections provide findings for each of the audit criterion and recommendations where management attention is required. Overall results for each criterion can be found in Annex A.

3.2.1 Issue Identification and Analysis

We expected to find that external and internal environments are monitored to identify issues with CSC policies, and that these issues are analyzed to determine policy solutions.

The following areas met the audit expectations for this criterion:

  • environmental scans are conducted externally and internally to identify policy issues; and
  • generally, identified policy issues are formally analyzed and considered by management.

Overall, we found that this criterion was met with two exceptions.

Lead time for implementation is not fully considered.

Through file reviews and interviews with staff at the local and regional levels, we found that lead time for implementation of new policies is not fully considered during the policy development process. While it is generally understood at the national level that lead time can (and should) be provided depending on the complexity of the policy issue and/or the extent of the policy change, this tends to be the exception rather than the norm. We reviewed corporate communications ("This Week at CSC") that announced 56 new/revised Commissioner’s Directives and guidelines between June 2016 and June 2017. Of these, 48/56 were expected to be "in effect" the day that the announcement was made, and the other 8/56 were expected to be "in effect" before the announcement was made. As such, no official lead time was provided to staff responsible for implementing the new policies. Further, the local and regional levels indicated that they often do not have enough time to fully understand new policy requirements, brief staff on these new requirements, receive clarification on these new requirements, and/or ensure local policies are aligned with the new requirements. Although proper implementation without lead time could be expected for simple policy changes, recognized change management principles require provision of lead time for policy changes that are more complex.

Human resource and financial impacts are not fully assessed.

Through file reviews and interviews with staff at all levels, we found insufficient assessments of human resource and financial impacts that would result from new policies. Through interviews, we heard an assumption that additional human and/or financial resources will not be committed for implementation of new policy requirements, and therefore, analysis of such would not add significant value to the policy development process. Whether or not additional resources are made available, these assessments are critical for: providing clear role definition for the staff that is required to implement the new policy; allowing managers to reallocate funds if necessary; and increasing the likelihood of early implementation of policy changes. Further, proper assessments of human resource and financial impacts go hand-in-hand with effective change management principles.

3.2.2 Consultation

We expected to find that consultations occur with key stakeholders in a meaningful manner.

The following areas met the audit expectations for this criterion:

  • key stakeholders are identified and engaged at the appropriate time; and
  • feedback from key stakeholders is reviewed for inclusion in the policy.

Overall, we found that this criterion was met with two exceptions.

Consultation timeframes are not always met.

For the national policy development process, key stakeholders are identified by the OPI at NHQ and are asked to provide comments on the draft policy. For the sample of national policies that we reviewed, the consultation timeframes (average of 17 calendar days) were met only 54% (26/48Footnote 12) of the time. In addition, we heard through interviews that some stakeholders, particularly at the local and regional levels, felt that the timeframes were too short and did not always allow for meaningful input. If consultation comments are not received, or if quality is lacking, there is a risk that stakeholder needs are not fully considered and this has an impact on policy effectiveness.

Disposition is not provided systematically.

We expected to find that disposition (of comments provided and revised drafts) would be provided to stakeholders to help support effective change management. With the exception of the disposition provided by the Women Offenders Sector during the creation of Commissioner’s Directive 768 Institutional Mother-Child Program, we found through both interviews and file review that disposition was not provided systematically. Further, we heard that the lack of feedback effects the credibility of the consultation process as some questioned if their comments would be taken seriously. Lack of feedback also presents a risk that staff may not fully understand why policy-related decisions were made, and in turn, could impact their acceptance of the revised policy.

3.2.3 Coordination and Alignment

We expected to find that policy instruments are coordinated and are aligned both internally and externally. The following areas met the audit expectations for this criterion:

  • policy instruments are aligned with external requirements; and
  • policy instruments are internally coordinated.

We found that this criterion was partially met.

Policy instruments are not sufficiently aligned internally.

We expected to find that local policy instrumentsFootnote 13 would align with national policy direction (i.e. Commissioner’s Directives and guidelines) where applicable. In total, we tested 180 national policy requirements to determine if they were aligned with local direction and found that only 64% (115/180) of these requirements were aligned. Further, this alignment testing was performed on national policy that had been in place for such a period of time that strong alignment results were expected. The audit team performed this testing starting in February 2017.

Table 1 - Internal Policy Alignment
National Policy (Publication Date) Internal Alignment Results
Commissioner’s Directive 768 Institutional Mother-Child Program and guideline (2016-04-18) 32% (9/28)
Commissioner’s Directive 800 Health Services (2015-04-27) 93% (27/29)
Commissioner’s Directive 760 Social Programs and Leisure Activities (2016-05-02) 53% (47/89)
Commissioner’s Directive 566-14 Perimeter Security (2015-06-08) 94% (32/34)
Total 64% (115/180)

Local standing orders accounted for the highest volume and rate of alignment errors amongst the local policy instruments that were tested. As described in Section 3.1.1 of this report, CSC Guidelines 005-1 Institutional Management Structure: Roles and Responsibilities assign the responsibility for reviewing standing orders to the correctional manager operations or the correctional managers, and the assistant warden interventions, but this responsibility is not further articulated through their work descriptions or through a sample of their performance agreements that we reviewed. In effect, when local policy instruments are not sufficiently aligned with national policy, there is an increased risk of front line non-compliance with national policy.

3.2.4 Review and Approval

We expected to find that policy instruments are sufficiently reviewed and approved by an individual with the appropriate level of authority. This criterion was met.

The following areas met the audit expectations for this criterion:

  • policy instruments are reviewed prior to approval; and
  • policy instruments are approved by an individual with the appropriate level of authority.

3.2.5 Communication

We expected to find that new policy requirements and policy changes are effectively communicated to key stakeholders.

The following area met the audit expectations for this criterion:

  • policies are readily accessible to all stakeholders.

Overall, we found that this criterion was met with one exception.

New policy requirements and changes are not communicated on a timely basis.

At CSC, communication of new policy requirements and changes is done through various means such as shift briefings, town hall meetings, e-mails from managers to staff, and conference calls. However, we found through documentation review and interviews with staff at all levels that the primary mechanism for communicating national policy direction to all CSC staff is through a weekly e-mail to all staff titled "This Week at CSC". In general, staff receives this e-mail on the date in which the policy is put in effect, and therefore, sufficient time is not provided to read the new policy documentation and understand its content prior to implementation. As described in Section 3.2.1, lead time for implementation of new policy is not fully considered when analyzing new policy issues, and as a result, communication is generally not done on a timely basis. When communication is not timely, there is an impact on front-line compliance, proper alignment of policy instruments, staff engagement with the new policy, and effective change management principles.

Conclusion - Objective 2

Overall, we found that policy development processes support the achievement of CSC’s objectives, including the management of its risks. From the five perspectives for which this objective was examined, we found that environmental scans are conducted externally and internally to identify policy issues, and in general, identified policy issues are formally analyzed and considered by management. We also found that key stakeholders are identified and engaged at the appropriate time and feedback from key stakeholders is reviewed for inclusion in the policy. As well, CSC policy instruments are aligned with external requirements and are internally coordinated. CSC policies are also reviewed prior to approval and are approved by an individual with the appropriate level of authority. Finally, policies are readily accessible to all stakeholders.

Still, we noted a few areas that require consideration by management in order to help ensure that risks are better managed. Specifically, CSC should:

  • provide guidance around lead-time assessments, human/financial resource assessments, and disposition requirements; and
  • assess the amount of time provided to stakeholders for consultation in order to determine its appropriateness.
Recommendation 3

The Assistant Commissioner, Policy should:

  • provide guidance around lead-time assessments, human/financial resource assessments, and disposition requirements; and
  • as part of its monitoring of the national policy development process, assess the amount of time provided to stakeholders for consultation in order to determine its appropriateness (taking into account the nature and extent of the policy change).
Management Response

Agree. The Assistant Commissioner, Policy will further clarify roles and responsibilities between OPIs and the Policy Sector and will seek to obtain EXCOM concurrence, as it relates to lead-time assessments, human/financial resource assessments, including amount of time provided to stakeholder consultations and disposition requirements.

3.3 Innovation

The third objective was to assess the innovation maturity of CSC’s policy-related management framework and development processes. CSC’s innovation maturity was examined based on a model (see Annex D) created by the innovation consulting firm, "OVO InnovationFootnote 14". We chose six factors from OVO’s model to assess CSC’s innovation maturity: strategy, people, measures, methods, communication, and breadth. It should be noted that the work performed for this objective is categorized as review level assurance, and is not commensurate with the high level of assurance provided through the first two objectives of this engagement.

Table 2 - Innovation Maturity of CSC’s Policy Development Processes
Factor Results Innovation Maturity Rating
Strategy From a strategic perspective, we found that CSC closely resembles a "journeyman" when it comes to innovation in its policy development processes. A strategy for innovation has been established, documented, and communicated. Examples of how this strategy is put into action are as follows:
  • CSC’s Executive Committee Sub-Committee on Effectiveness and Efficiency has a focus on innovation;
  • CSC’s Blueprint 2020 sub-groups; and
  • Consultations with a broad range of stakeholders that go beyond what is required by law are a step within the national policy development process.
Journeyman
People From a people perspective, we found that CSC closely resembles a "journeyman" when it comes to innovation in its policy development processes. Although we could not find evidence of training that is consistently delivered that is specific to innovation, we did find that: management and staff are provided with opportunities to focus on developing innovative approaches to policy development; incentives and rewards are in place for innovative ideas; and there are central teams that focus on innovation in policy development. Examples of these good practices include:
  • Central teams: the Executive Committee Sub-Committee on Effectiveness and Efficiency and Blueprint 2020 sub-groups;
  • Training: CSC training such as Gender-based Analysis Plus (GBA+) and the Leadership for Managers Program; and
  • Rewards: CSC Guidelines 265-1 Administration of the CSC Recognition Program.
Journeyman
Measures In terms of measures for innovation within its policy development processes, we found that CSC closely resembles an "apprentice". Similar to our monitoring results (Section 3.1.3), we could not find clear and consistent measures in place for tracking the extent to which innovation is embedded in its policy development processes.
However, the Public Service Annual Survey is a measure that CSC uses to monitor employees’ views related to innovation, and the 2017 survey results indicate that there is an opportunity for CSC to further encourage staff to be innovative and to take initiative in their work.
Apprentice
Methods In terms of innovative methods within its policy development processes, we found that CSC closely resembles a "journeyman". We found that innovative tools and techniques are used consistently when developing policies. Examples of the methods we found are:
  • The consultation process facilitates and promotes innovative approaches; and
  • Tools used in the development of policies include: the site orders tool, the strategic policy generic e-mail account; WebEx meetings; the Management Action Plan Integration Tool; GCconnex; CSC’s "Headstarter"; and the CSC "Hackathon".
Journeyman
Communication From a communications perspective, we found that CSC closely resembles a "journeyman" when it comes to communicating its innovation goals for policy development. In this regard, we found that innovation goals are communicated to staff albeit on an ad-hoc basis. Examples of these communications include:
  • Internal: e-mails from senior management, "Gen-Communiqués" to all staff, and CSC intranet postings.
  • External: "Let’s Talk Express", use of social media, and reporting on Blueprint 2020 progress.
Journeyman
Breadth In terms of the breadth of innovation in its policy development processes, we found that CSC closely resembles a "master". We found that innovation teams and communities from local institutions and offices, regions, and sectors are in place to focus on policy development; and external stakeholders and information is leveraged to gather ideas and to develop new concepts and technologies. Examples include:
  • Formal consultations as part of the policy development process (as applicable) with a broad group of stakeholders such as: inmate committees, citizens’ advisory committees, unions, the Office of the Correctional Investigator, the Canadian Association of Elizabeth Fry Societies, and the John Howard Society of Canada; and
  • There are opportunities for staff at all levels to provide input through formal consultations and other means, including the aforementioned tools (see the "Methods" factor above).
Master

Conclusion - Objective 3

Overall, we found that CSC is a mature organization with respect to its policy-related management framework and development processes, and seeks opportunities to become more innovative. In particular, the consultation work that CSC undertakes with a broad range of stakeholders with multiple points of view facilitates innovative policy development. Further, CSC demonstrates that it has a strategic focus on innovation by establishing means for gathering new ideas and approaches, communicates its strategies, and encourages its employees to be innovative. Finally, although CSC has established methods, including processes and tools, within its policy development processes that support innovation, it is encouraged to develop measures to track the extent to which innovation supports policy development, and further enhance its use of technology.

4.0 Conclusion

For the first objective, we found that a management framework is in place to support the effective management of policy development. The audit noted a few areas that require consideration by management in order to help ensure that the management framework fully supports CSC’s policy development processes.

With respect to the second objective, we found that policy development processes support the achievement of CSC’s objectives, including the management of its risks. Still, the audit noted a few areas that require consideration by management in order to help ensure that risks are better managed.

For the third objective, we found that CSC is a mature organization with respect to its policy-related management framework and development processes, and seeks opportunities to become more innovative. CSC is encouraged to develop measures to track the extent to which innovation supports policy development, and further enhance its use of technology.

Recommendations have been issued in the report based on areas where improvements are required.

5.0 Management Response

Audit findings and recommendations are well received and will allow further refinement and communication of roles and responsibilities as they relate to the policy development process. As such, the Assistant Commissioner, Policy, will seek concurrence of EXCOM on key changes to clarify roles and responsibilities and collaborate with stakeholders to enhanced policies, monitoring and tools supporting CSC’s policy development process. Full implementation of the Management Action Plan is scheduled by March 31, 2018.

6.0 About the Audit

6.1 Approach and Methodology

Audit evidence was gathered through a number of methods:

Interviews
In total, 107 interviews were completed during the conduct phase of this audit. Interviews were conducted with senior management and key staff at NHQ, RHQ, and the local levels. Interviews took place in person, by teleconference, and by videoconference.
Review of Documentation
Documentation that was reviewed included: legislation; CSC policy instruments; corporate documents such as process maps, policy development guides, tools, work descriptions, and policy development analysis; and electronic communications such as e-mails and intranet sites.
Analytical Review
Analytical review was completed in relation to the criteria on issue identification and analysis; consultation; coordination and alignment; review and approval; communication; and the objective on innovation.
Sampling
The policy instruments that were selected for testing were chosen to ensure adequate coverage of:
  • corporate priorities;
  • areas of risk identified through CSC’s Corporate Risk Profile; and
  • areas of risk identified through the audit risk assessment.

The population was stratified by CSC sector, and then by date of creation or amendment. Random samples were chosen from the population. The sample of national policy instruments that we tested were as follows:

  • Commissioner’s Directive 760 Social Programs and Leisure Activities (2016-05-02), Correctional Operations and Programs Sector;
  • Commissioner’s Directive 768 Institutional Mother-Child Program (2016-04-18), Women Offender Sector;
  • Commissioner’s Directive 800 Health Services (2015-04-27), Health Services Sector; and
  • Commissioner’s Directive 566-14 Perimeter Security (2015-06-08), Correctional Operations and Programs Sector.

6.2 Past Internal Audit Work Related to Policy Development Processes

Audit assurance and consulting engagements typically examine the policy framework for the particular program or activity being assessed to ensure that it is compliant with external requirements (legislation, central agency policies, etc.) and to ensure that policy instruments are aligned. Specific engagements that were used in planning the work for this audit include:

Review of the Framework Governing the Management of CSC’s Responses to Internal and External Reports

The review found that:

  • CSC did not have an integrated central repository or single body responsible for ensuring consistency, timeliness, and quality of all responses to recommendations from all reviews or any other reports;
  • CSC’s employees operate within a framework of policy, legislation and public service values, and in so doing, are subject to scrutiny by other entities (Office of the Correctional Investigator, National Joint Boards of Investigation, Coroner’s Inquests/Medical Examiner’s Investigations, Parole Board of Canada, the Office of the Auditor General, and the Office of the Commissioner of Official Languages);
  • CSC has requirements to respond to legislative and policy-generated recommendations that sometimes require the policies to be changed; and
  • there was no systematic mechanism in place that ensured the consistency and congruence of all CSC’s responses across the various frameworks to manage internal and external reports.

Status Report on Administrative Segregation

This review found that:

  • despite the fact that the consultation process occurred over several months, staff indicated that the roll-out of the changes was too fast, and did not provide enough time to have fulsome discussions to ensure everyone understood the requirements;
  • given that electronic distribution of the revised Commissioner’s Directive and guideline was the primary medium of communication, staff felt that it was unrealistic to read these documents in their entirety with such short notice; and
  • staff questioned the practicality of some of the changes. For example, some staff indicated that the revised framework is too restrictive with respect to segregation placements, and others indicated that they are risk averse when it comes to release.

6.3 Statement of Conformance

In my professional judgment as Chief Audit Executive, sufficient and appropriate audit procedures have been conducted and evidence gathered to support the accuracy of the opinion provided and contained in this report. The opinion is based on a comparison of the conditions, as they existed at the time, against pre-established audit criteria that were agreed on with management. The opinion is applicable only to the first two objectives as work performed for the third objective (innovation) is categorized as review level assurance, and is not commensurate with the high level of assurance provided through the first two objectives of this engagement.

The audit conforms to the Internal Auditing Standards for Government of Canada, as supported by the results of the quality assurance and improvement program. The evidence gathered was sufficient to provide senior management with proof of the opinion derived from the internal audit, with the exception of the evidence gathered to support the third objective (innovation).

Annex A: Audit Criteria

The following table outlines the audit criteria developed to meet the stated audit objectives and audit scope:

Objective Audit Criteria Met/
Met with Exceptions/
Partially Met/
Not Met
1. Management Framework
Assess the extent to which a management framework is in place to support the effective management of policy development.
1.1 - Roles and Responsibilities
Roles and responsibilities of employees involved with policy development are clearly defined, communicated, understood, and accepted.
Partially Met
1.2 - Capacity
The organization considers its current and future human resource requirements related to policy development; and employees are provided with the necessary training, tools, and resources to develop policies.
Met with Exceptions
1.3 - Monitoring
Policy development activities are monitored to help ensure the processes are effective.
Not Met
2. Development Processes
Assess the extent to which policy development processes support the achievement of CSC’s objectives, including the management of its risks.
2.1 - Issue Identification and Analysis
External and internal environments are monitored to identify issues with CSC policies, and these issues are analyzed to determine policy solutions.
Met with Exceptions
2.2 - Consultation
Consultations occur with key stakeholders in a meaningful manner.
Met with Exceptions
2.3 - Coordination & Alignment
Policy instruments are coordinated and are aligned both internally and externally.
Partially Met
2.4 - Review and Approval
Policy instruments are sufficiently reviewed; and approved by an individual with the appropriate level of authority.
Met
2.5 - Communication
New policy requirements and policy changes are effectively communicated to all stakeholders.
Met with Exceptions
3. Innovation
Assess the innovation maturity of CSC’s policy-related management framework and development processes.
Maturity Model Elements
3.1 - Management Framework Innovation Maturity
Strategy Journeyman
People Journeyman
Measures Apprentice
3.2 - Development Processes Innovation Maturity
Methods Journeyman
Communication Journeyman
Breadth Master

Annex B: Site Selection

Three of CSC’s five regions were selected by the audit team due to the breadth of the subject matter and the impact this breadth had on the audit team’s ability to deliver results in a timely manner. Although on-site visits were not completed, interviews and document review were conducted for these sites. Sites were selected to ensure coverage of the following:

  • Indigenous offenders (including pathway healing initiative);
  • Women offenders;
  • Remote institutions and healing lodges;
  • Mental health and physical health services; and
  • Community corrections.
Region Sites
Quebec
  • Port-Cartier Institution
  • Archambault Institution
  • Joliette Institution for Women
  • Montréal Metropolitan District
  • Regional Headquarters
Ontario
  • Grand Valley Institution for Women
  • Bath Institution
  • Beaver Creek Institution
  • Central Ontario District Office
  • Regional Headquarters
Prairie
  • Regional Psychiatric Centre
  • Willow Cree Healing Lodge
  • Okimaw Ohci Healing Lodge
  • Grande Cache Institution
  • Saskatchewan, Manitoba, and North Western Ontario District Office
  • Regional Headquarters
National Headquarters
  • Policy Sector
  • Correctional Operations and Programs Sector
  • Health Services Sector
  • Women Offender Sector
  • Communications and Engagement Sector
  • Legal Services Sector
  • Aboriginal Initiatives Directorate
  • Learning and Development Branch

Annex C: CSC's National Policy Development Process

CSC's National Policy Development Process

CSC's National Policy Development Process

Planning

  1. Policy trigger
  2. Appropriate policy instrument selected
  3. Strategic policy (SP) provides OPI with controlled version/template of policy instrument

Drafting

  1. Policy instrument and bulletin drafted
  2. Informal consultation
    Consultation required, go to Consultations
    Consultation not required, go to steps 6-8
  3. Controlled version of policy and bulletin created/updated (translation)
  4. Formal consultation
    Consultation required, go to Consultations and follow steps 4-8
    Consultation not required, go to step 8a
    8a. Quality control of policy instrument
    8b. Draft policy instrument updated

Consultations

  1. OPI select stakeholders to consult
  2. Stakeholder provides feedback to OPI
  3. Feedback assessed by OPI and incorporated where necessary. Then go to step 6 of drafting section.
  4. OPI/SP identify stakeholders to consult
  5. Consultation package sent to stakeholders
  6. Stakeholder provide feedback to OPI
  7. Feedback summary table prepared
  8. Feedback assessed by OPI/SP and incorporated where necessary. Then go to steps 8b and 8a of Drafting

Approval

  1. Policy instrument signed by Sector Head
  2. EXCOM approval required
    If yes, go to steps 10a then step 11
    If no, go to step 10b then step 10a
    10a. Commissioner approval required
    10b. Approval package sent to EXCOM members for approval
    If yes, go to step 11
    If no, go to step 11a
  3. Approval package sent to Commissioner for approval
    11a. Translate final policy instrument if changes made during approval process

Promulgation

  1. Policy instrument promulgated on Infonet & CSC website

Notes

  1. Internal and External consultations occur on most policy documents unless the change is technical in nature or if consultations are restricted due to legislative/legal or security considerations.
  2. The approval level is determined based on the nature of the change and the document. A technical change does not require EXCOM approval. A Guideline is approved by the Sector Head (OPI).

Strategic Policy Division, January 2017

Annex D: OVO’s Innovation Maturity Model

OVO Innovation (2010)

Innovation maturity progression:

  • Involves consistent, focused effort by engaged executives
  • Over a period of several years
  • Focused on near team results (ideas becoming products) and longer term cultural change
  • Investment in people and methodologies
  • Expectation of a formal approach – distinct process, established metrics
  Novice Apprentice Journeyman Master
Strategy
  • No expectation for Innovation
  • Relieve an existing issue
  • Completely reactive
  • Innovation as a desire
  • Short term focus
  • Incremental goals
  • Innovation as a focus
  • Incremental and disruptive
  • Becoming proactive
  • Innovation as expectation
  • Innovation leadership
  • Disrupt firms and markets
People
  • Part-time on a project basis
  • No formal training
  • Few/no rewards or incentives
  • No central team
  • Part-time commitment
  • Some innovation training
  • Ad-hoc rewards
  • Recognition of central team
  • Full time commitments
  • Detailed training
  • Incentives and rewards
  • Identification of central team
  • Initiated by individuals
  • Autonomy/ experimentation
  • Corporate incentives
  • Engagement by central team
Methods
  • Few innovation tools
  • Haphazardly applied
  • No consistent process
  • Introduction of new tools
  • Carefully applied
  • Defining a core process
  • Wide range of tools
  • Applied by trainees
  • Following a defined process
  • Creating new techniques
  • Applied by experts
  • Defined methodology
Funding
  • No planned funding
  • Borrowed from other sources
  • Very limited
  • Small planned funding
  • Combined with other funds
  • Limited to idea development
  • Funds in annual plan
  • Set aside for team and ideas
  • New research possible
  • Team fully funded
  • Innovation R&D dollars
  • Available for all activities
Measures
  • No established metrics
  • Focus on cost/time
  • Short term
  • Few established metrics
  • Based on ideas and ROI
  • Short term
  • Multiple metrics
  • Incremental/ Disruptive
  • Short term & long term
  • Many metrics
  • Focus on differentiation
  • Mid term/Long term
Communication
  • Vague goals
  • Little communication to team
  • None to organization
  • Communicated goals
  • Some communication to team
  • Little to organization
  • Active communicator
  • From engaged leader
  • To organization as a whole
  • Consistent communication
  • From all management
  • Internally and externally
Breadth
  • Isolated teams
  • No concurrent projects
  • No external involvement
  • Some collaboration
  • Few concurrent projects
  • Little external involvement
  • Broad collaboration
  • Many concurrent projects
  • Some external involvement
  • Innovation Communities
  • Multiple projects
  • Internal and external
Rewards
  • Not designed
  • Project based
  • Not meaningful extrinsic
  • Some consideration
  • Project based
  • Aligned to strategies
  • Carefully considered
  • Based on participation
  • Based on goals/plans
  • Incorporated into plans
  • Based on outcomes
  • Intrinsic and extrinsic
Outcomes
  • Frustrated teams
  • Incremental ideas
  • Rarely implemented
  • Little impact
  • Satisfied teams
  • Reasonable ideas
  • Occasionally implemented
  • Some new products/ services
  • Engaged teams
  • Good ideas
  • Regularly implemented
  • Predictable flow of ideas
  • Engaged organization
  • Disruptive ideas
  • Regularly implemented
  • Innovation leadership
Date modified: