Predictive or Retroactive Audit?

25 WCASCONTECSI 9

Sao Paulo, May 31, 2012

Miklos A. VasarhelyiKPMG Professor of AIS

3

The Present and The Future

• Yes -> there is a need for total revamping of business measurement and assurance schemata– The past measurement and assurance compromises and

tradeoffs do not work any more under extant information technology

– A conceptual revolution is needed, better that is not forced– Measurements that are full cycle and preemptive– Audit Automation: Progressive (P1), preventive (p2) and

predictive audits (P3)

• A dual frame of standard setting must be put in place to allow for the progressive development and implementation of measurement and assurance in the digital era

4

Continuous Online Audit Outline

• Definition and background• A conceptual revolution• Audit automation and continuous audit• Progressive (P1), preventive (p2) and predictive audits

(P3)• Implementation of continuous audit

5

DEFINITION AND BACKGROUND

6

Continuous Audit Definition

• Continuous auditing is a type of auditing which produces audit results simultaneously with, or a short period of time after, the occurrence of relevant events. (CICA/ AICPA 1999)

• The Institute of Internal Auditors' (The IIA) defines continuous auditing simply as "any method used to perform audit-related activities on a more continuous or continual basis," without further defining what "more" means.

• IIA (GTAG #3 ), continuous auditing is defined as the automatic method used to perform control and risk assessments on a more frequent basis.

• ISACA issued guidance in 2010

777

An evolving continuous auditframework

• Automation• Sensoring• ERP• E-

Commerce

Continuous

Audit

ContinuousControl

Monitoring

Continuous

Audit

Data

Continuous RiskMonitoring and

Assessment

9

Continuous Assurance Model

10

Today

• Although several surveys state that a substantial percentage of large firms are doing continuous audit this is not at all true

• There are several initiatives but manly outside the mainstream process

• The real time economy is here but there is substantive lack of progress in the assurance process

• Tools and technology exist but the necessary socio-technical factors are not yet here

CRMA

CCMCDA

Itaú-Unibanco

P&G

PPP Insurance

Inventory Dashboard

Siemens

Continuous Control Monitoring

Audit Automation P&G: Order to Cash Auditor JudgmentSiemens- AAS AutomationAICPA – ADS / APS

Audit Methodologies• Multidimensional Clustering • Process Mining• Continuity Equations • Predictive Auditing • Visualization• Analytic Playpen

Itaú- UniBanco

P&G

HCA

Met-Life

Duratex

J+J

CA Technologies

Supply Chain

Inventory

FCPA Sales Commission

IDT

Claims Wires

FCPA

Duplicate Payments

PPP Credit Card InsuranceA/P

A/P

HP

GLKPIs/KRIs

Sigma Bank

Process Mining

KPMG

American Water /

Caseware

Verizon

Talecris / ACL

AT&T

12

The old tradeoffs fail with new technology• They are not only inefficient they can be delusory or

plain wrong– Sampling vs full population testing– Manual confirmations vs confirmatory extranets (database to

database pinging) – Annual audit opinions– Business measurement

• Focus on financial numbers (after the horse left the barn)• Accounting rules

– LIFO and FIFO» Depreciation

» Owners Equity» Goodwill

13

The old tradeoffs (2)

• Must learn to live with a hybrid environment where rules and processes are migrating– Is the FASB able to set standards on an disruptive

environment?– Can you change the tires of a car in movement?

• Yearly reporting• Assurance• Financial operations

– The dire scenario of not changing• False sense of assurance• Progressive loss of value of the financial reporting and

auditing functions in exchange to alternate approaches• Substantive societal costs

14

The role of assurance providers

• A much more pervasive presence on the wealth producing landscape– Assurance coordination needs -> the profession did not manage to

capitalize on enormous assurance needs in a digital society– Technology validation on a big data / analytic model environment– Boundary validation and interpretation in a multi-source process

environment

• RER (Real Electronic Reporting) provides a wealth of opportunities that will be fulfilled by someone/ something– Goodbye bill by hours /economic model of the profession– Goodbye yearly audit / continuous assurance– Goodbye manual audits– Goodbye retroactive audit / predictive/ preventive audits

1515

Electronic measurement and reporting (XBRL)• XBRL although a very positive step on the route

towards automation perpetuates some of the weaknesses of the “paper oriented” reporting model– Audits to improve their social agency function should be of

corporate measurement and databases not of financial reports– As most substantive regulatory based changes XBRL presents a

series of unintended consequences including• Pressure toward standardization of reporting• Facilitation of more frequent reporting• Evolutionary force towards the standardization of the semantics of

accounting reporting• A poor conduit to represent corporate transactions (XBRL/FR)

• XBRL/FR will eventually lead to XBRL/GL –great societal effects

16

CONCEPTUAL R->EVOLUTION

17

Where the conceptual revolution lays• A forward looking audit (predictive)

– Models predict levels and flows– Variances establish aberrations– Evidence is weighted and evaluated– New forms of evidence arise– Preventive prediction models are also to be

used

– BUT

1

18

Conceptual revolution (2)

• Retroactive (but for very recent period) is still needed– Models cannot capture the unexpected– PCAOB requirements are not only anachronistic

but counterproductive and expensive– There is a natural interlinkage between CDA /

CCM / and CRMA; they are complementary and related

– ALSO

2

19

Conceptual revolution (3)

• Control Monitoring has to be Automated– Controls in ERPs are not observable and they are user

configurable– PCAOB requirements are not only anachronistic but

counterproductive and expensive– There is a natural inter-linkage between CDA / CCM / and

CRMA; they are complementary and related• E.g. although controls may be active and effective

there is never certainty that all risk are covered and that no new form of fraud has been invented

– ALSO

3

20

Conceptual revolution (4)

• CRMA– Bear Sterns collapsed weeks after a clean audit

opinion– Auditors stated that conditions changed

dramatically in a short period of time– PCAOB has been pressuring for a “risk based

audit” but this has not been clearly specified and is held back by ridiculous (for example sampling) requirement

4

21

AUDIT AUTOMATION AND CONTINUOUS AUDIT

22

• There is no real time economy audits if at least parts of the assurance process is not automated

• The major obstacle that internal auditors face is the availability and access to data

• The second major obstacle is the multiplicity of audit-like organizations with splintered needs and objectives

• Audit automation modules have to ultimately be built-in to production processes and cooperate with these although having different lords (owners)

• They have to closely interact with analytic models and human interaction modules

23

PROGRESSIVE, PREVENTIVE AND PREDICTIVE AUDITS

24

Continuous Audit and Audit Automation (P1 P2 P3)• The progressive audit (P1)

– Where actual audit processes are formalized / broken down into small steps and parts automated

– Coherent with the proposed ASEC – Audit Data Standards– Teeter (2013) automates steps in the Siemens and P&G audits

through breaking them down in Audit Actions and then creating automation for them

• The location of the auditor• The procedure adopted• The timing / frequency of the procedure

Audit Data Standards & Apps

AICPA’s Assurance Services Executive Committee – June 2011

Audit apps(based on assertions)

Platform developers

Classify

Query

Ratio

Data acquisition

ERP vendors

Data access

Cloud/data providers

Relationship Between Audit Apps and Common Data

Common Data

Repository

Dashboard

Data matchin

g

Black box logActivity

logs

Production data

Trend

Analytic Query

Dashboard

Dimensions of data/procedures

• Method– Data generation– Audit procedure

• Timing– Data generation– Data frequency– Audit frequency

• Location– Data storage/access– Audit steps

Introduction

Data &

Procedures

Framework

Conclusion

Discrete Continuous

Man

ual

Au

tom

ated

Remote

Local

Dimensions of Assurance

• Automation– Manual vs Automated

• Timing– Discrete vs continuous

• Location– Local vs remote

• Focus– retroactive vs

predictive

• Procedure– Confirmation– Physical verification– Aging of receivables– Cutoffs– Etc

Discrete Continuous

Man

ual

Au

tom

ated

Remote

Local

Retroactive -> Predictive

Procedures

Audit Focus

Retroactive

PredictivePreventive

Not Preventive

30

Continuous Audit and Audit Automation (P1 P2 P3)• The preventive audit (P2)

– Where based on forensic models preventive (and hopefully adaptive) filters are created and are placed in the actual process preventing transactions with high discriminant loadings to be processed and deflecting these to a review process

– This is a preventive control / associated with a review and analytic audit process

– See Yong Bum Kim’s dissertation (2011) for basic work on the approach

31

INCORPORATING FORENSIC INTO AN CA/CM PHILOSOPHY

YONG BUM KIM

32

Operations

controls

Forensicmodels

Forensic

analysis

archives

Filtering byForensicmodels

AuditBy

Exception

Incorporating Forensics into an CA/CM philosophy

Forensicmodels

33

Continuous Audit and Audit Automation (P1 P2 P3)• The predictive audit (P3)

– Where based on data mining and other models you can predict the value of operational parameters and use this prediction for audit operational purposes. This audit is complementary to P2 but does not actually places filters in operations.

– For example fourth quarter results can be predicted from quarterly and monthly data in an external audit to issue an opinion and not wait to perform year-end data verification

– Another example would be (Kuenkaikaew, 2012) predicting fraudulent service cancelations to detect employees violating corporate policies.

INTERNAL AUDIT ANALYTICS:EVOLVING TO A NEW ERA

Siripan Kuenkaikaew

Results and analysis

• Models comparison of data weighted approach with 1:14 ratio

35

Model/ Measurements (%)

Accuracy Error rate Specificity

Recall Precision False alarm rate

J4864.23 35.77 51.72 65.12 94.98 48.28

Logistic regression

70.16 29.84 50.30 71.58 95.28 49.70

Support vector machine

79.36 20.64 37.20 82.37 94.84 62.80

36

Contact Information:Miklos A. Vasarhelyi

KPMG Professor of AISmiklosv@rutgers.edu

Rutgers Business School1 Washington Park, Room 919

Newark, NJ 07102-3122

201-4544377

37