Office of the Auditor General: Report on Audit Follow-ups, Tabled at Audit Committee – October 27, 2016, Français au verso

Office of the Auditor General

October 27, 2016

Mayor, Members of Audit Committee and Council,

I am pleased to present this report on follow-ups of audits carried out by the Office of the Auditor General of the City of Ottawa.

This report includes an overview and an executive summary for each of the follow-ups conducted.

Respectfully,

Ken Hughes

Auditor General

Office of the Auditor General – Report on Audit Follow-ups

October 27, 2016

Staff of the Office of the Auditor General

Suzanne Bertrand

Sonia Brennan

Ken Hughes

Vivien Kaye

Ed Miner

Janet Onyango

Louise Proulx

Ines Santoro

Ensen Xie

Office of the Auditor General – Report on Audit Follow-ups

October 27, 2016

Table of Contents

Progress toward improvement .............................................................................. 1

Summary and assessment of overall progress made to date on audit recommendations ..........................................................................................

Office of the Auditor General – Report on Audit Follow-ups

October 27, 2016

2

Executive Summaries – Audit Follow-ups .............................................................4

Follow-up to the Audit of Client Service Centres ...................................................5

Introduction ....................................................................................................5

Summary of the Level of Completion .............................................................6

Conclusion .....................................................................................................7

Acknowledgement .........................................................................................7

Follow-up to the Audit of Corporate Credit Cards .................................................8

Introduction ....................................................................................................8

Summary of the Level of Completion .............................................................8

Conclusion ................................................................................................... 10

Acknowledgement ....................................................................................... 10

Follow-up to the Audit of Ontario Works Eligibility Assessment Process ............ 11

Introduction .................................................................................................. 11

Summary of the Level of Completion ........................................................... 13

Conclusion ................................................................................................... 14

Acknowledgement ....................................................................................... 15

1

Progress toward improvement The Office of the Auditor General (OAG) conducts audit follow-ups 2 to 3 years after an audit is complete to afford management time to implement the recommendations. The OAG adheres to the best practices and professional standards of the international audit community by including the practice of audit follow-ups. The Audit Process includes the Planning Phase, the Fieldwork Phase, the Reporting Phase and finally the Follow-up Phase. In the follow-up, the OAG evaluates the adequacy, effectiveness, and timeliness of actions taken by management on reported observations and recommendations. This evaluation ensures that the required measures, promised by management and approved by Council, have been implemented. Accordingly, the follow-ups in this report were conducted according to the OAG’s 2014 Work Plan.

The audit follow-ups contained in this report include:

· Audit of Client Service Centres · Audit of Corporate Credit Cards · Audit of Ontario Works Eligibility Assessment Process

As can be seen in the next section, it is clear from the results of these follow-ups that management is committed to the audit process and has done a great deal to address issues raised by the audits and improve management practices in these areas. Management should be proud of these improvements and their progress should be acknowledged. The OAG wishes to draw attention to the Audit of Ontario Works Eligibility Assessment Process where the implementation of recommendations faced unique challenges. Due to the ongoing problems with the provincial implementation of SAMS, certain key processes and procedures are in a temporary state. We will consider returning to review the Ontario Works Eligibility Assessment Process after the system and procedures have normalized. In addition, the Follow-up to the Audit of Client Service Centres found that improvements have been made with respect to theefficiency and effectiveness of cash handling and compliance with the cash handling policy and procedures, however, improvements to procedures continue to be required to provide security and accountability around cash handling.

2

Summary and assessment of overall progress made to date on audit recommendations Audits are designed to improve management practices, enhance operational efficiency, identify possible economies and address a number of specific issues. The follow-up phase is designed to identify management’s progress on the implementation of recommendations from the audit reports. This report is not intended to provide an assessment of each individual recommendation. Rather, it presents our overall evaluation of progress made to date across all completed audits. Should Council wish to have a more detailed discussion of specific follow-ups, OAG staff are available to do so.

The table below summarizes our assessment of the level of completion of each recommendation for the above-noted audit follow-ups.

Table 1: Summary of level of completion of recommendations

Action Percent

complete Number of

Recommendations % of Total

Recommendations

Little to no action 0 to 24 3 6%

Action initiated 25 to 49 0 0%

Partially complete 50 to 74 4 8%

Substantially complete 75 to 99 4 8%

Complete 100 37 77%

Total 48 100%

We have categorized each of the audit follow-ups based upon the following criteria:

· Solid Progress = 50% or more of the recommendations evaluated at 75-100% complete.

· Little or No Progress = 50% or more of the recommendations evaluated at 0-49% complete.

· Gradual Progress = All others.

Solid Progress:

· Audit of Client Service Centres · Audit of Corporate Credit Cards · Audit of Ontario Works Eligibility Assessment Process

3

With these audit follow-ups now complete, no further work to review the implementation of these recommendations is intended by the OAG, with the exception of Ontario Works Eligibility Assessment Process. As discussed above, we will consider revisiting this area once the system and procedures have normalized post-SAMS implementation. However, as a result of the annual work plan and/or Council requests, new audits in any of these areas may occur in the future.

4

Executive Summaries – Audit Follow-ups The following section contains the executive summary of each of the audit follow-ups.

5

Follow-up to the Audit of Client Service Centres

Introduction The Follow-up to the 2012 Audit of Client Service Centres was included in the Auditor General’s 2014 Audit Plan.

The key findings of the 2012 audit included:

· The City could streamline operations in its seven Client Service Centres to reduce staff and save money.

· There were opportunities to increase automated transactions for service delivery and potential savings from these alternative delivery methods.

· Based on an analysis of the level of activity at each centre, consideration should have been given to closing some centres given their low usage rates. Resources were not fully utilized and as many as 13 positions could be eliminated. This would have resulted in an annual savings of $824,000. Total potential savings identified in the audit were $860,000.

· There had been no updated operational risk assessment at the Client Service Centres unit level which was required by the April 2010 Council approved Enhanced Risk Management Framework and Policy.

6

Summary of the Level of Completion The table below outlines our assessment of the level of completion of each recommendation as of September 2015.

Table 2: OAG’s assessment of level of completion of recommendations

Category %

Complete Recommendations Number of

Recommendations

Percentage of Total

Recommendations

Little or no action

0 – 24 n/a 0 n/a

Action Initiated 25 – 49 n/a 0 n/a

Partially Complete

50 – 74 9, 10 2 12%

Substantially Complete

75 – 99 5, 6 2 12%

Complete 100

1, 2, 3, 4, 7, 8, 11, 12, 13, 14, 15, 16, 17

13 76%

Total 17 100%

7

The table below outlines management’s assessment of the level of completion of each recommendation as of May 9, 2016 in response to the OAG’s assessment. These assessments have not been audited.

Table 3: Management’s assessment of level of completion of recommendations

Category %

Complete Recommendations Number of

Recommendations

Percentage of Total

Recommendations

Little or no action

0 – 24 n/a 0 n/a

Action Initiated 25 – 49 n/a 0 n/a

Partially Complete

50 – 74 n/a 0 n/a

Substantially Complete

75 – 99 5, 9 2 12%

Complete 100 1, 2, 3, 4, 6, 7, 8, 10, 11, 12, 13, 14, 15, 16, 17

15 88%

Total 17 100%

Conclusion At the time of this follow-up work, we found that 88% of the recommendations could be considered substantially complete or complete. The remaining 12% of the recommendations were partially complete.

One of the partially complete recommendations was to comply with all aspects of the Cash Handling Policy and Procedures. In the follow-up, we assessed compliance in six areas (i.e., segregation of duties, coin order processing, slip printing, access to the CLASS Cash Register drawer, refund processing and adequacy of monitoring) and found that in three of these six areas the CSCs were not in full compliance. Therefore, we assessed this recommendation as 50% complete.

Acknowledgement We wish to express our appreciation for the cooperation and assistance afforded the audit team by management.

8

Follow-up to the Audit of Corporate Credit Cards

Introduction The Follow-up to the 2012 Audit of Corporate Credit Cards was included in the 2014 Audit Plan of the Office of the Auditor General (OAG).

The key findings of the original audit included:

· The City’s credit card program, with 2011 purchases valued at $16.6 million, is well administered and well supported by Finance department.

· The City has clear processes, heightened supervision and oversight over reconciliations verification, as well as satisfactory compliance against applicable policies and procedures.

· Based on both a judgmental and random sample, error rates were low and controls were functioning as intended.

· The audit recommends that the City increase the use of credit cards versus petty cash or cheques which could yield annual savings of $250,000. Having staff at the lower operational level carry out the purchasing instead of more senior level employees could save approximately $10,000 per year. Total potential savings identified in the audit are $260,000.

Summary of the Level of Completion The table below outlines our assessment of the level of completion of each recommendation as of October 2015.

9

Table 4: OAG’s assessment of level of completion of recommendations

Category %

Complete Recommendations Number of

Recommendations

Percentage of Total

Recommendations

Little or no action

0 – 24 n/a 0 n/a

Action Initiated 25 – 49 n/a 0 n/a

Partially Complete

50 – 74 2 1 8%

Substantially Complete

75 – 99 n/a 0 n/a

Complete 100 1, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13

12 92%

Total 13 100%

The table below outlines management’s assessment of the level of completion of each recommendation as of July 2016 in response to the OAG’s assessment. These assessments have not been audited.

10

Table 5: Management’s assessment of level of completion of recommendations

Category %

Complete Recommendations Number of

Recommendations

Percentage of Total

Recommendations

Little or no action

0 – 24 n/a 0 n/a

Action Initiated 25 – 49 n/a 0 n/a

Partially Complete

50 – 74 n/a 0 n/a

Substantially Complete

75 – 99 2 1 8%

Complete 100 1, 3, 4, 5, 6, 7, 8, 9,

10, 11, 12, 13 12 92%

Total 13 100%

Conclusion Management has been proactive in addressing the recommendations of the 2012 Audit of Corporate Credit Cards. At the time of this follow-up, 12 out of 13 recommendations are complete and 1 is partially complete.

Acknowledgement We wish to express our appreciation for the cooperation and assistance afforded the audit team by management.

11

Follow-up to the Audit of Ontario Works Eligibility Assessment Process

Introduction The Follow-up to the 2012 Audit of Ontario Works Eligibility Assessment Process was included in the Auditor General’s 2014 Audit Plan.

The key findings of the original 2012 audit included:

· The initial eligibility assessments for assistance are granted in accordance with the Ontario Works Act, Regulations and policy directives. For the most part, the receipt, verification and appropriate analysis of required information are documented accordingly for the application process.

· Within the selected sample, a total of $3,840 in payments of benefits to recipients was identified as being ineligible. This represents approximately 0.6% of the population tested. Although the sample is not necessarily statistically valid, if this error rate held consistent across the entire population of payments, the total ineligible payments during the period from January 2011 to June 2012 would be approximately $834,000.

· The audit found that by increasing the number of cases assigned to each case worker, the City could decrease the number of generic case workers to be more in line with the provincial median. This would result in a staffing reduction savings of approximately $1,549,200 annually. Total potential savings identified in the audit are $2,567,750.

· The audit also recommends that the Community and Social Services Department work with Human Resources on an Attendance Management program with the objective of reducing sick leave of case workers to the rate currently achieved in the East Service Centre (i.e., 16.5 days). Other Centres are as high as 21 days per year whereas the City-wide average of all employees is 11.

In November 2014, a new case management software - Social Assistance Management System (SAMS) was implemented by the provincial government to administer Ontario Works, Ontario Disability Support Program, Assistance for Children with Severe Disability and Emergency Assistance cases. While the process to implement functionalities is managed by the Ministry of Community and Social Services, a provincial/municipal SAMS working group was created to deal with SAMS’ challenges and associated business processes.

12

The following three changes made by the Ministry in relation to the implementation of SAMS had a significant impact on the review of eligibility. Specifically:

· The requirement to verify income changed from being a standard monthly requirement, to being a risk-based decision (i.e., the caseworker checks the income "verification required” checkbox or there are new earnings);

· As a temporary workload reduction measure, the Ministry waived the requirement to update expired Outcome Plans (Participation Agreements have been renamed Outcome Plans in SAMS); and,

· The Enhanced Verification Program (EVP) is a primary mechanism to monitor the completeness and accuracy of pay information submitted by OW clients. EVP activities were also suspended by the Ministry.

The launch of SAMS compounded with changes in how business is conducted has impacted on our follow-up of some of the audit’s recommendations. For these recommendations, management took actions to address them in the pre-SAMS environment; however the actions are no longer relevant in the SAMS environment. As well, due to on-going problems with the provincial implementation of SAMS, certain key processes have been deferred and workaround procedures are in place. As such, there is less value in assessing the procedures in place in this temporary state. We may return to review the OW eligibility assessment process at a later date after the system and procedures have normalized. For the purpose of this follow-up, in these cases where management implemented the recommendation in the pre-SAMS environment, we have assessed them as 100% completed.

13

Summary of the Level of Completion The table below outlines our assessment of the level of completion of each recommendation as of November 30, 2015.

Table 6: OAG’s assessment of level of completion of recommendations

Category %

Complete Recommendations Number of

Recommendations

Percentage of Total

Recommendations

Little or no action

0 – 24 3, 10, 14 3 17%

Action Initiated 25 – 49 n/a 0 n/a

Partially Complete

50 – 74 12 1 6%

Substantially Complete

75 – 99 2, 13 2 11%

Complete 100 1, 4, 5, 6, 7, 8, 9, 11, 15, 16, 17, 18

12 67%

Total 18 100%

The table below outlines management’s assessment of the level of completion of each recommendation as of September 16, 2016 in response to the OAG’s assessment. These assessments have not been audited.

14

Table 7: Management’s assessment of level of completion of recommendations

Category %

Complete Recommendations Number of

Recommendations

Percentage of Total

Recommendations

Little or no action

0 – 24 n/a 0 n/a

Action Initiated 25 – 49 n/a 0 n/a

Partially Complete

50 – 74 n/a 0 n/a

Substantially Complete

75 – 99 2 1 5%

Complete 100 1, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18

17 95%

Total 18 100%

Conclusion The City’s Community and Social Supports (CSS) unit has substantially or fully implemented 14 of the 18 recommendations from the 2012 audit as they pertained to the former SDMT system only. One recommendation is partially completed while the remaining three have not been addressed. Management indicated that this was due to ongoing technical challenges, requested enhancements to SAMS and continued provincial administrative deferral of business rules.

The SAMS implementation continues to have functionalities deficiencies and challenges for stakeholders and municipal partners. Additional effort is required from management to fully implement the remaining recommendations and address the underlying issues. Specifically, additional communication and/or monitoring is needed in the areas of eligibility documentation, using more detailed case information when monitoring workload, attendance management, using SAMS’ functionalities to create efficiencies and reduce FTEs, and, reviewing the potential of using SAMS’ functionalities to replace spreadsheets for ERP reporting.

15

Acknowledgement We wish to express our appreciation for the cooperation and assistance afforded the audit team by management.