Auditing Utility (On-Demand) and Service Organization Applications

Utility Computing: Auditing a Disruptive Innovation Practicum: Evaluating a Prospective Audit Client – Ocean Manufacturing

ScheduleWeek Topic Readings Practicum

12-Sep-05 Identifying Computer Systems Chapter 2 Evaluating IT Benefits and Risks

Jacksonville Jaguars

19-Sep-05 IS Audit Programs Chapter 3 The Job of the Staff Auditor

A Day in the Life of Brent Dorsey

26-Sep-05 IS Security Chapter 4 Recognizing Fraud The Anonymous Caller

3-Oct-05 Utility Computing and IS Service Organizations

Chapter 5 Evaluating a Prospective Audit Client

Ocean Manufacturing

10-Oct-05 Physical Security Chapter 6 Inherent Risk and Control Risk

Comptronix Corporation

17-Oct-05 Logical Security Chapter 7 & 8 Evaluating the Internal Control Environment

Easy Clean

24-Oct-05 IS Operations Chapter 9 Fraud Risk and the Internal Control Environment

Cendant Corporation

31-Oct-05 Controls Assessment Chapter 10 IT-based vs. Manual Accounting Systems

St James Clothiers

7-Nov-05 Encryption and Cryptography Chapter 11 Materiality / Tolerable Misstatement

Dell Computer

14-Nov-05 Computer Forensics Chapter 12 Analytical Procedures as Substantive Tests

Burlington Bees

21-Nov-05 New Challenges from the Internet: Privacy, Piracy, Viruses and so forth

Chapter 13 Information Systems and Audit Evidence

Henrico Retail

28-Nov-05 Auditing and Future Technologies Chapter 16 Flowcharting Transaction Cycles

Southeast Shoe Distributor

Old and New

Service Organizations like EDS Are in the business of running IS shops Only the transactions are handled by the client

They are being replaced by Utility Computing Which is an outgrowth of software vending

business models Particularly those of Oracle, SAP and

Salesforce.com

What is Utility Computing? Utility-based computing provides a mix of the following businesses:

Storage and server virtualization. Software that can contribute to higher utilization of IT resources.

Automated infrastructure provisioning. Software capable of improving manageability of the data center while eliminating many manual and error-prone procedures and saving costs.

Grid tools. Software capable of providing for geographically distributed processing for a range of compute-intensive applications.

Blade servers. A server packaging concept that emphasizes lower space and power requirements while promising greater manageability in conjunction with automated infrastructure provisioning software.

IT and systems management software. Software solutions that contribute to greater manageability of utility-based computing technologies and provide for metering and billing of IT resources for the purpose of chargeback.

Business applications on demand . The delivery of preconfigured business applications form a remote location over an IP network on a subscription-based outsourcing contract.

IT and business service providers. Providers of IT and business services that offer their solutions on a pay-as-you-go basis, including not only providers of IT services such as outsourcing and web hosting, but also emerging providers of business process outsourcing services.

Why do firms choose Utility computing?

Utility computing offers greater flexibility in the creation of

computing environments when they are needed.

It opens up usage-based pricing and reduces users' use of

capital.

Utility Computing allows an organization to have the ability to

harness latent computing power and resources, regardless of

application or other physical or organizational boundaries.

It allows an organization to virtually repurpose operating

systems, application mix, processing power, and storage to the

immediate needs of the corporation, to meet new demand or to

rapidly create computing environments for projects.

When to Use Utility Computing Utility computing should be used

to bypass IT when it stands in the way of the business for any number of reasons

To serve as a temporary innovation fix if functionality is not available from a large suite vendor

When the underlying process is outsourced such as call center support applications.

Utility computing should not be used when you are dealing with transactional-intensive applications

such as in a warehouse management system when data is exceptionally sensitive when on-demand service providers don’t have the deep

functionality or provide the level of customization required,

Pervasiveness of Utility Computing Recent moves like

Oracle's acquisition of Siebel, And The growing popularity of software-as-a-service vendors like

Salesforce.com are indicators that the software industry is tilting toward an on-

demand future

Still, on-demand services are likely to account for less than 10 percent of business application use through 2010 (Gartner)

The reason why the on-demand model is not suitable for complex business uses like

logistics support and order handling nor for large complex companies requiring business process support

But the "complexity constraint bar" will rise over time since on-demand vendors can add functionality easily

Consequences: License Fees

Previously, hardware and software were purchased, and budgeted for, in large, predictable chunks.

For software licensing, the most common way today was for the customer to pay a fixed fee according to the processing power of the machine or machines being used

Or for the licensee to pay a fixed fee according to number of users (or seats) accessing the software.

With utility computing, processing power is purchased and paid for according to demand. The emergence of the service-oriented architecture (SOA),

and the development of virtualised computing, have introduced the notion of almost complete flexibility in which systems or services are used

That creates all kinds of problems. If something is not used, for example, then, increasingly, customers do not expect to be charged for it. But if something is used, how is it measured? And what if resources are allocated on a provisional basis, but not used?

Consequences: Control of Data and Programs

Copies of data outside the organization Accounting transactions (fraud, loss, alteration) Personnel and customer records (privacy, theft)

Operation of programs may be less well understood since there are no in-house experts This may lead to more audit exceptions

Example:

Salesforce.com Salesforce.com's products fall into a broad category of software called customer relationship

management, or CRM They help companies manage all sorts of customer relations, such as letting salespeople keep

track of leads or helping execs judge the success of marketing campaigns Allows customers and software makers to turn Salesforce.com into a platform for others to build

upon -- much like Microsoft Corp.'s (MSFT ) Windows.

Last month introduced AppExchange, Concept: provide an eBay of corporate software. an online marketplace where software makers and customers can swap and sell applications they

develop

could eventually change the structure of his industry. Software over the Web -- commonly called on-demand -- accounted for less than 10% of the $46

billion in corporate software sold last year.

creating an open marketplace for on-demand software will help cause the decline of the big, complex, and expensive corporate applications sold by the likes of SAP (SAP ) and Oracle Corp. (ORCL ).

Example:

Oracle

Oracle is promoting “Grid systems” the grid is treated as a utility like electricity

It is one of the various approaches to on-demand computing, pool storage and other resources across the whole network

so that complex programs can harness huge amounts of power, and

applications can draw on resources from anywhere on the system as they need them.

Example:

Oracle Oracle picks out various trends that it believes make grids

"unstoppable": * Blades: low cost computing blades can be assembled into 'blade

farms' that can then be interconnected, for scalable commodity computing clusters costing up to 80% less than conventional systems.

* Linux: Oracle is firmly behind Linux as an enterprise system and claims that blades enable Linux, with all its cost advantages, to play in grids. Linux' main disadvantage is that it does not scale far in symmetric multiprocessing environments, but it can work efficiently an blades, which are typically only two to four processors each, this making it suitable for mass computing.

* Virtualization: Virtualization techniques, especially in storage, make the grid a reality by creating 'virtual' servers and storage farms regardless of where the resources are physically located.

* Standards: As well as Globus, which drives grid developments in their original academic home, there is now the Grid Computing Forum, a formal standards body.

Example:

OracleEnterprises implement grids in 3 stages

1. Scavenging resources: This is attractive because it involves reclaiming unused

resources to carry out computing tasks for instance, PCs lying idle at night.

2. Sharing resources: With a shared grid, applications and data are moved around to

use any available resources on the grid, with schedulers assigning tasks. Like scavenging grids, the appeal is that existing resources are used more efficiently, so investment in new technology is minimal.

3. Dedicating resources: Resource sharing is not always practical because of

administrative, political, trust and bandwidth constraints. Instead, organizations can dedicate resources to grid computing rather than incorporating all existing systems in a grid structure.

Audit Challenges of Utility Computing

Data, Software and Hardware are held by 3rd party

Auditors do not have unrestricted access Need to rely on 3rd party’s auditor reports

Which probably will not address control over your company’s transactions directly

Asset ownership / security problems Should a company run into claims concerning

ownership of data (journalists reports, patents, etc.) Existence of records at a 3rd party site may cause

problems

Audit Challenges of Utility Computing

Audit Control over Transactions may be inadvertently weakened Because Utility software is not customized for the

audit client’s business, and End users may be more likely to make errors with

software that they don’t fully understand and control

“Service Organization” Audits Service Organizations must hire independent

external auditors (Dictated by SAS 70 “Service Organizations” in the US; Sec 5900

in CA, AGS in Oz and FIT 1&2/94 in UK) to express one of two types of opinions relevant to

adequacy of internal control (1) “relevant policies and procedures were in place at some

date” (2) item (1) plus “they are in fact operating effectively”

Obviously the auditor has to do more work if the opinion is of type (1) than of type (2)

But both are very weak requirements And place the burden on the auditor of the firm.

Service Audit Report Contents

Report of Independent Auditors Description of relevant Policies and Procedures

Operations (org chart) Control Environment Transaction flow (with flowcharts) Applications Program maintenance / change procedures Regulatory compliance

Control objectives set by Service Org Management Client control considerations

Ocean Manufacturing, Inc.The New-Client Acceptance Decision

Understand the types of information relevant to evaluating a prospective audit client

List some of the steps an auditor should take in deciding whether to accept a prospective client

Identify and evaluate factors important in the decision to accept or reject a pro spective client

Understand the process of making and justifying a recommendation regarding client acceptance

Case Study 5.2Significant Risk with Service Organization Application

Read pp. 61-64, the review of the Audit report of the service organization

Questions: (1) What transaction flows and assets are affected by

The flaws in the ‘old’ password system The flaws in the hierarchical security levels

(2) What is the expected financial risk (loss or misstatement of accounts) from each one of these flaws

Case Study 5.3A Qualified Opinion: ATM Network Service Organization

Read pp. 66-67 Questions:

(1) What should the internal auditors of your client conclude from this opinion

(a) that significant control weaknesses at the service organization would affect the internal control environment at their own (your client’s) firm

(b) that alternative test (i.e., extensive testing of internal ATM procedures was performed) substituted for the lack of a description of the firms control objectives and procedures

(2) What is the expected financial risk (loss or misstatement of accounts) from each one of these flaws

Case Study 5.4A Qualified Opinion: Credit Card Service Organization

Read pp. 67-71 Questions:

(1) What should the internal auditors of your client conclude from this opinion

(a) that significant control weaknesses at the service organization would affect the internal control environment at their own (your client’s) firm

(b) that alternative test (i.e., extensive testing of internal ATM procedures was performed) substituted for the lack of a description of the firms control objectives and procedures

(2) What is the expected financial risk (loss or misstatement of accounts) from each one of these flaws

Control Objectives

Read through Exhibit 5.1

How do you think management came up with this list?

How might you decide whether these ‘Control Objectives’ are adequate?

How to determine Appropriate ‘Control Objectives’ (Your Toolkit: Risk Assessment Matrix, Dataflow Diagrams and Systems Components Hierarchy

Your Toolkit: Computer Inventory, Risk Assessment Matrix, Dataflow Diagrams and Systems Components Hierarchy

Asset (Ex 2.1) Risk Assessment (Ex. 2.2 with improvements)

Primary OS Owner

Application

Asset Value ($000,000 to Owner)*

Transaction Flow Description

Total Annual Transaction Value Flow managed by Asset($000,000)*

Risk Description

Probability of Occurrence (# per Year)

Cost of single occurrence ($)

Expected Loss

Win XPReceiving Dock A/P 0.002

RM Received from Vendor 23 Theft 100 100 10000

Win XPReceiving Dock A/P 0.002

RM Received from Vendor 23

Obsolescence and spoilage 35 350 12250

Bu s in es s Ap p lic a t io nS y s tem s

T r an s ac tio n F lo w s

As s e t L o s s R is k s( I n te r n a l Au d its )

R ep o r t in g R is k s( E x te r n a l Au d it)

C o n tr o l P r o c es s R is k s( I n te r n a l & E x te r n a l

Au d its )

O p er a tin g S y s tem s( in c lu d in g D BM S , n e tw o r kan d o th e r s p ec ia l s y s tem s )

Har d w ar e P la tf o r m

Ph y s ica l a n d L o g ica lS e cu rity En v iro n m e n t

A u dit O bje ct iv e s

Alternatives to SAS 70 Type Audits

An increasing number of corporate functions are handled on the Internet

By small applications providers Or Web hosting companies

That cannot afford SAS 70 audit compliance

These problems are diminished by the use of 3rd party certification services

E.g., CyberTrust (from the merger of Ubizen / Betrusted and TruSecure in Nov 2004)

These services generally are much more effective at assuring security over Service Organization operations Than SAS 70 audits could ever hope to be

Cybertrust Large privately held security firm

Certifying web service providers 4,000 customers

Main role: provide clients (i.e., Service Operators) with intelligence, technology, and expertise to track threats, find security gaps, improve protection and

enhance procedures .

Areas of Focus » Identity management

» Threat management» Vulnerability management» Compliance management

Cybertrust Services secure access to mission-critical information assets

manage digital identities

detect and prevent security threats and vulnerabilities

improve security policies and infrastructures

predict, prioritize and help organizations better adapt to risks

assess security management needs

institute metrics, baselines and guidelines necessary to help quantify enterprise security productivity