El-Paso SOX TestingTraining- June 2007

El Paso CorporationIntroduction to

Sarbanes – Oxley Section 404Project

June, 2007

2

What You Will Learn from This Material

• Understand the purpose of the SOX 404 compliance project.

• Understand what the project team has accomplished and what is left to do.

• Understand the roles and responsibilities of the testing participants.

• Understand the detailed steps of completing management’s testing.

3

Success Factors

• Know and use available resources – training materials, templates, people.

• Understand your assigned business processes – “what can go wrong?”, “what is in place to prevent those failures?”

• DON’T BE AFRAID TO ASK QUESTIONS!!• Build a plan and stick to it!• It’s Management’s Risks, Management’s Controls,

Management’s Plan & Management’s Conclusions.• Call it like you see it!• Communicate, communicate, communicate.• Don’t go it alone!

4



– Sarbanes-Oxley Act Overview,

– Basics of internal control,

– COSO Framework,




5

Sarbanes–Oxley Act Overview

The Sarbanes – Oxley Act of 2002 (SOX) requires

company management to assess and report on the

company’s internal control.

The Public Company Accounting Oversight Board

(PCAOB) was established to set the standards that

companies must follow to comply with SOX.

6

Key Sections

Requires an annual report by management regarding internal controls and procedures for financial reporting, and an attestation as to the accuracy of that report by the company’s auditors. The annual report states management’s responsibility for establishing and maintaining an adequate internal control structure as well as management’s assessment of the effectiveness of that internal control structure.

Section 404

In general, the new certification requirements require companies to formalize control structures, improve controls and establish

monitoring programs to enable CEOs and CFOs to make their evaluations and report their conclusions.

Section 302Requires quarterly certification by the CEO and CFO of all companies filing periodic reports under section 13 (a) or 15 (d) of the Securities Exchange Act of 1934 regarding the completeness and accuracy of such reports as well as the nature and effectiveness of internal controls supporting the quality of information included in such reports.

Sarbanes-Oxley Act Overview

7

“What is internal control?”

“Internal control is broadly defined as a process, effected by an

entity’s board of directors, management and other personnel,

designed to provide reasonable assurance regarding the

achievement of objectives in the following categories:

• Effectiveness and efficiency of operations.

• Reliability of financial reporting.

• Compliance with applicable laws and regulations.”

(Source: Internal Control – Integrated Framework, COSO)

Sarbanes-Oxley Act Overview

8

What is internal control over financial reporting?

• Internal control over financial reporting-• Includes those policies and procedures that pertain to the ability to initiate, authorize,

record, process, and report financial data consistent with the assertions embodied in either annual or interim financial statements, or both, prepared in accordance with generally accepted accounting principles.

• Specifically includes (a) maintenance of records that in reasonable detail accurately and fairly reflect the transactions and dispositions of the assets of the entity, and (b) policies and procedures that provide reasonable assurance that

(1) transactions are recorded as necessary to permit preparation of financial statements in accordance with generally accepted accounting principles, and

(2) receipts and expenditures of the entity are being made only in accordance with authorizations of management and directors of the entity.

• Certain controls over financial reporting may be in information systems that are primarily designed to achieve objectives other than financial reporting objectives.

Basics ofinternal control

9

The SEC ruled that management’s evaluation of internal controls must be based on a “suitable framework” and that the COSO Internal Control – Integrated Framework met the criteria of “suitable.”

COSO, the Committee of Sponsoring Organizations, was formed in 1985 to improve the quality of financial reporting through business ethics, effective internal controls, and corporate governance. Based on these principles, they developed the COSO framework as a foundation for establishing internal control systems and determining their effectiveness.

COSO Framework

10

The Five Components

Control Activities Policies/procedures that ensure

management directives are carried out.

Range of activities including approvals, authorizations, verifications, recommendations, performance reviews, asset security and segregation of duties.

Monitoring Assessment of a control

system’s performance over time.

Combination of ongoing and separate evaluation.

Management and supervisory activities.

Internal audit activities.

Control Environment Sets tone of organization-

influencing control consciousness of its people.

Factors include integrity, ethical values, competence, authority, responsibility.

Foundation for all other components of control.

Information and Communication Pertinent information identified,

captured and communicated in a timely manner.

Access to internal and externally generated information.

Flow of information that allows for successful control actions from instructions on responsibilities to summary of findings for management action.

Risk Assessment Risk assessment is the

identification and analysis of relevant risks to achieving the entity’s objectives-forming the basis for determining control activities.

All five components must be in place for a control to be effective.

COSO Framework

11




– SOX Project Plan

– Project History

° Set Foundation

° Document Processes, Risks and Controls

° Conduct CFO Walkthroughs

° Develop Management’s Testing Approach

– Project Overview



12

SOX Project Plan

Insert Testing timeline here

13

Launched Teams at Each Business Unit

El Paso Corporation

Pipelines ProductionField

ServicesMerchant Energy Corporate

GTM• ANR

• TGP

• CIG

• EPNG

• SNG

Create new Org chart to include names of leaders

14

Sub-Cycles

Processes are classified by cycles to help ensure efficient testing. Processes leading to a common business objective or processes involving a common set of procedures or common employees, or are otherwise related to one another are grouped within the same cycle. This is a subjective process to some extent, but it helps divide the hundreds of in-scope processes into manageable groupings.

Revenue and A/R Expenditure Disbursements Payroll Treasury

Inventory Property Tax Financial Reporting

Information Technology

15

Critical & Significant Controls

In order to efficiently assess the control environment, management prioritized the controls to be evaluated. An overwhelming number of “key” controls were identified during the documentation initiative. Therefore, those controls were further segregated as critical, significant or non-key.

Controls

Key Controls CriticalControls

SignificantControls

The test program writers for 2007 reviewed all controls and re-evaluated the criticality for testing purposes. Some non-key controls were selected for testing while some critical and significant controls were moved to controls not being tested.

16

Objectives of Management’s Testing

• Provide evidence of the operating effectiveness of controls over financial reporting:

– Confirm controls operate as designed, and – Verify authorized and competent people execute controls.

• Identify exceptions in documentation or operation of controls.• Provide Management with sufficient evidence to conclude on the internal

control environment.

Finding and documenting what does work right is important.

Finding what is NOT working right is even more important because we have to fix those things.

17

• Testing of controls is a form of validating controls operation.

– Periodic testing of controls also evaluates the quality of self-assessment and monitoring processes.

• Testers use tests of controls to determine whether selected internal control policies and procedures were operating effectively during a period of time or as of a point in time.

• Testers must also assess whether the individuals responsible for executing the controls have the necessary authority as well as the appropriate competencies to do the job.

• Tests of controls should be performed at the process level (in addition to the entity level, although this level is not the focus of this phase of the project) -- tests at the process level include tests of:

– Manual processing controls ,

– Automated programmed controls.

Validation Methods – Tests of ControlsDevelop Management’s Testing Approach

18

“Where do I fit in?”

Management’s Responsibilities:• Accept responsibility for the effectiveness of the company’s internal control over financial reporting.• Evaluate the effectiveness of the company’s internal control over financial reporting using suitable

control criteria.• Support its evaluation with sufficient evidence, including documentation.• Present a written assessment about the effectiveness of the company’s internal control over

financial reporting as of the end of the company’s most recent fiscal year.

PwC Audit Responsibilities:• Express an opinion on the financial statements.• Express an opinion on internal control over financial reporting, which includes:

־ Evaluating management’s assessment of the effectiveness of internal control over financial reporting.

־ Testing the effectiveness of internal control over financial reporting directly.

Project Team Responsibilities:• Assist Management in meeting its responsibilities by performing Management’s Testing.

In order to comply with Sarbanes-Oxley Act, Section 404, Management must meets its obligations for PwC to give El Paso a clean opinion under its responsibilities outlined below.

19





– Testing Team Structure,

– Roles and Responsibilities,

– Project Team Structure.


20

El Paso ManagementRoles &Responsibilities

• Accepts responsibility for the effectiveness of the entity’s internal control over financial reporting.• Manages overall direction of the testing activities, making all critical decisions.• Mobilizes resources.• Reviews and approves testing plan.• Assumes sole responsibility for the sufficiency of the accumulation, gathering and summarization

processes of the testing activities.• Evaluates and concludes on testing results.• Presents a written assessment about the effectiveness of the company’s internal control over

financial reporting as of the end of the company’s 2007 fiscal year.

21

PMO

• Fosters clear communication and synchronizes activities among multiple testing teams.• Coordinates resources and skill sets.• Provides guidance to testing teams on standard methods and procedures, including documentation

standards.• Collaborates with Testing Team Managers on the sequencing of the testing schedule for each

Business Unit.• Coordinates testing of entity-level controls.• Measures and reports progress of testing teams.• Monitors processes requiring remediation to track completion of Remediation Action Plans and

coordinate re-testing.• Anticipates project efficiency issues; takes action to keep project on-time/on-budget.• Develops and communicates Management Reporting requirements for the project.• Performs Quality Assurance review of testing.• Coordinates management’s review of testing activities.• Coordinates communications with external auditor.• Coordinates review of testing with external auditor.• Manages and populates the Sarbox Portal.• Coordinates planning of refresh testing.• Supports The Self-Assessor team in building self-assessment program.

Roles &Responsibilities

22

Testing Team Manager

• Fosters clear communications and synchronizes activities within testing team.• Collaborates with the PMO on the sequencing of the testing schedule for each Business Unit.• Collaborates with the PMO on resourcing requirements.• Provides guidance to testers in every step of testing. • Coordinates testing activities with process owners.• Manages contact with process owners to minimize interruptions.• Understands process and relevant controls within cycle to manage testing. • Selects controls for testing.• Coordinates testing of entity-level or cross-cycle controls.• Reviews and approves the test plan and any subsequent changes.• Evaluates exceptions to document retention standards.• Evaluates results of tests. • Communicates internal control gaps to process owners.• Alerts PMO when processes require remediation and further testing. • Reviews Remediation Action Plans to ensure that plans adequately address identified gaps.• Tracks timing and progress of testing activities and routinely reports status to PMO.• Reviews testing efforts to ensure timely accomplishment of objectives.• Reviews quality of testing activities periodically and before submission of documentation to PMO.• Schedules PMO Reviews.• Plans refresh testing.


23

Testing Team

Lead Tester• Assists Testing Team Manager in overseeing day-to-day operations.• Also performs duties of tester.

IT Specialist• Provides guidance on testing process level IT controls, which are generally application controls.• Also performs duties of tester or Testing Team Manager, as assigned.

Tester• Understands specific processes and relevant controls to anticipate testing issues.• Writes the detailed instructions for performing the tests of controls.• Communicates testing requirements and document requests to process owner.• Coordinates with process owner to gather evidence required for test execution. Follows-up routinely.• Notifies Testing Team Manager of any changes to the testing plan.• Performs testing.• Routinely updates Testing Team Manager on timing and progress of testing activities.• Informs Testing Team Manager of any issues/concerns arising during the testing activities.• Validates testing results with process owner. • Communicates with process owner to understand the root cause of exceptions.• Employs standardized templates for documenting test plans and results.• Understands documentation standards & maintains neat, orderly workpapers.• Maintains testing documentation on El Paso shared drive.


24

Process OwnerRoles &Responsibilities

• Accommodates multiple requests for data and information related to testing activities.• Makes available key process personnel on a timely basis for inquiries, observations, etc.• Gathers and organizes all requested documentation to be used in testing activities.• Coordinates data requests directly with IT personnel, with assistance of tester.• Communicates any issues causing delays relating to personnel and documentation availability.• Confirms or resolves deficiencies identified during testing activities.• Prepares Remediation Action Plans for controls deemed to be operating ineffectively.• Corrects any processes and controls requiring remediation.

25

Project Team StructureThe project team is organized in a hybrid structure according to the Business Units and the Cycles that were established during earlier phases of the project.

Business Unit Focus: Revenue, Expenditure, Inventory & Property Cycles

Merchant Energy Production Pipelines (Revenue)

Pipelines (Expenditure, Inventory & Property)

Cycle Focus: Merchant, Production, Pipelines, Corporate

Tax Financial Reporting Information TechnologyPayroll, Treasury,

Control Environment, Corp Expenditure &

Corp Property

Field Services is not included in this chart. Their interaction with the overall project structure is dependent on the Gulf Terra merger & is still under consideration.

26






– Test Plan Creation

– Evidence Compilation

– Test Plan Execution

– Quality Assurance

27

“How do we test controls?”

The steps of completing detailed testing can be organized into four major activities, as illustrated below.

I. Test Plan Creation II. Evidence Compilation

III. Test Plan Execution IV. Quality Assurance

1. Conduct initial planning

2. Select controls

3. Draft test plan

4. Approve test plan

Test Step

5. Determine evidence accessibility

6. Select sample

7. Coordinate evidence collection

8. Manage testing support

Test Step

9. Execute test plan

10. Evaluate test results

11. Validate conclusions

12. Coordinate Remediation Action Plans

13. Report results of testing

14. Anticipate refresh testing

Test Step

15. Adhere to documentation standards

16. Review progress and quality of work

17. Schedule PMO reviews

Test Step

28

“Where do we record our work?”

Each process should have a Controls Test Matrix like the one below.

29

“How do we test controls?”

The following graphic demonstrates the relationship of the Controls Test Matrix to the testing steps. The Controls Test Matrix fields are beside the testing step to which they relate.

I. Test Plan CreationControls Test Matrix Field

------

------

A-P

Q

II. Evidence CompilationControls Test Matrix Field

R-T

U-V

------

------

III. Test Plan ExecutionControls Test Matrix Field

W-Y

Z

AA-AB, AE

AC-AD

------

AF

IV. Quality Assurance


2. Select controls

3. Draft test plan


Test Step


6. Select sample



Test Step







Test Step




Test Step Controls Test Matrix Field

------

------

------

30

Steps 1 – 4: Test Plan Creation

I. Test Plan Creation


2. Select controls

3. Draft test plan



------

------

A-P

Q

II. Evidence Compilation


31

What has been done?I. Test PlanCreation

• The teams have substantially completed the initial planning which included: understanding the processes, classifying the processes into “mini-cycles”, scheduling testing with process owners and setting roles and expectations of team members.What needs to be done: Start by gaining an overall understanding of the processes in which the controls to be tested reside as well as your specific role on your team. Assist with outstanding activities such as coordinating schedules with process owners.

• The teams have reviewed the classification of controls as critical and significant and have determined which controls they will be testing. Controls selected will cover all relevant financial statement assertions.

• The teams are currently drafting test plans for all controls selected for testing. The plans will be documented in the Controls Test Matrix (CTM) and will include relevant data from the RCM as well as details regarding the test of controls to be performed. Fields A-Q of the CTM will be completed at the end of I. Test Plan Creation.

• All test plans must be approved by the Test Team Manager before execution. This approval will be recorded in Field Q of the CTM. If at any point during the testing, the test plan changes, the Test Team Manager must re-approve the plan.

2. Select controls

1. Conduct initialplanning

3. Draft test plan


Ong

oing

32

3. Draft test plan

Who does this?Testing Team

What needs to be done?The test plan consists of the detailed instructions for verifying the operating effectiveness of controls. The plan will include information such as testing approaches, scopes and sample sizes.

The teams will use the Controls Test Matrix to document the test plan, execution, and results.

A Controls Test Matrix should exist for each process or subprocess (each Bold red or light red process on the PCF), and ALL critical and significant controls should be recorded on that Controls Test Matrix, even though only selected significant controls will be tested.

I. Test PlanCreation

33

3. Draft test plan (cont’d)

“What tools are available to assist in drafting the test plan?”The test team should leverage off the suggested test plan in the Risk Control Matrix when completing the test plan. However, the test team should constructively evaluate the suggested test plan to ensure that it conforms with the guidance provided here relating to the requirements set forth by management for performance of Management’s Testing.

In addition, the teams should review the materials gathered during the CFO Walkthroughs as evidence of control performance. These materials will provide additional information regarding the evidence that will be available for performing Management’s Testing.


34

3. Draft test plan - CTMI. Test PlanCreation

A. Header - The Header is the information gathered from the Risk Control Matrix: the Organization, Process, Process Owner, COSO Objective, Financial Reporting Element and Business Cycle.

B. Test Number - This is a unique, sequential number, one for each control test. Each process will start over with one.

C. Control Number - This is the number of the control being tested, as identified on the Risk Control Matrix and Process Maps. Control Description - This is the detailed description of the control being tested. It should agree word-for-word to the Control Description in the Sarbox Portal. No changes, additions, clarifications, etc. may be made to the wording of fields C/D.

C. Control Number/Description (editable) - This is the editable version of the control number and description in fields C and D.

D. Clarifying Remarks - This field should be used to provide additional detail regarding the control description or the process activities to which the control applies. This could include specific activities performed by specific individuals to accomplish the control’s overall objective; references to specific documents, reports or fields; more detailed linkage to process activities; etc. – any clarifying language needed to create a clear test. The Testing Team may need to talk to the process owner to obtain this clarifying information.

E. (C)ritical or (S)ignificant - This is the designation of the control as critical or significant, as indicated in the Sarbox Portal.

F. Control Type - This field will identify whether the control is “preventive” or “detective”.G. Control Method - This field will identify whether the control is “automated” or “manual”.H. Control Role - The control role refers to the function of the control within the cycle. The options are:

Data Capture, Data Transfer, Monitoring, Fraud, Non-routine or Other. A particular control may perform a combination of these roles and the teams may select more than one option.

C/D.2

*

*

*

*

**

* These fields are populated by downloading information from the Sarbox Portal.

C/D.*

35

3. Draft test plan – CTM (cont’d)

J. Assertion Type - The Assertion Type refers to the El Paso financial statement assertion that is achieved by the control. The El Paso assertions are: Authorization, Completeness & Accuracy, Evaluation of Balances, Access to Assets, Substantiation of Balances, Proper Classification, Presentation & Disclosure.


*

* These fields are populated by downloading information from the Sarbox Portal.

J. COSO Assertion Type – The COSO Assertion Type refers to the COSO financial statement assertion that is achieved by the control. The COSO assertions are: Existence, Occurrence, Completeness, Rights & Obligations, Valuation or Allocation, Presentation & Disclosure.

2

36

“Before I get lost. . .”

LETTER refers to a field on the Controls Test Matrix

Subsequent slides will be labeled with a LETTER (e.g., “A” or “K”). Each of these letters and these slides refer directly to a field that must be filled out on the Controls Test Matrix.

The fields are discussed within the specific test step in which they apply.

As we discuss these fields, take a look at the example Controls Test Matrix so that you know exactly where the field is.

NUMBER refers to the test step to which the field applies

37

Initial Data Capture Data Transfers Monitoring

Data capture controls help ensure that all information that should be in the financial reporting makes its way into the control environment in the first place.

Data transfer controls help ensure that data moves through the system completely and accurately and that reported results have integrity.

Monitoring controls (e.g., analytical review such as variation analysis and reasonableness tests performed by El Paso’s management) play a key role in ensuring that significant errors and omissions do not make their way into reported information.

I. Control Role2. Select controls

What goes in this field?The Control Role refers to the function of the control within the cycle. The options are: Data Capture, Data Transfer, Monitoring, Fraud, Non-routine or Other. A particular control may perform a combination of these roles and the teams may select more than one option.

What issues and concerns must be taken into account?“What do the various control roles mean?”

38

I. Control Role – Fraud2. Select controls

One additional consideration in choosing which significant controls should be tested is the role that a control plays in preventing or detecting fraud.

Fraud includes fraudulent financial reporting (e.g. earnings management), misappropriation of assets (e.g. payroll fraud), expenditure and liabilities for improper purposes (e.g. bribery), fraudulently obtained revenue and assets and/or cost and expenses avoided (e.g. tax fraud).

Many of the controls surrounding fraud are entity-level controls, as opposed to process-level controls. These include controls surrounding the code of conduct, ethics hotline, audit committee and human resources policies. However, some fraud controls may be embedded within the processes. For example, approval, authorizations, reconciliations and segregation of duties may all be relevant for preventing and detecting fraud.

“What if there are no controls in my area specifically related to fraud prevention and detection?”Processes that have a risk of fraud that could have a material effect on the financial statements should have fraud related controls and those controls should be evaluated. If a particular process does not have any of these type of controls identified, then the Testing Team Manager should contact the PMO to coordinate further documentation.

39

I. Control Role – Non-routine Transactions2. Select controls

“Why are non-routine transactions important?”The testing team may consider emphasizing testing of controls that involve the processing of non-routine transactions. Non-routine transactions may involve significant judgments and estimates (e.g., allowance for doubtful accounts), or they may involve management overrides of other controls (e.g., the booking of “topside” entries that undo or lessen the impact of other process-based entries).

Processing of Routine Transactions

Control 1 Control 2 Control 3 Control 4

Control 5 Control 6 Control 7 Control 8

$5,000,000

“TopsideEntry”

$(4,000,000) $1,000,000

All these controls may operate effectively 100% of the time. . .

. . .but one non-routine transaction. . .

. . .could undo all those controls.

NET EFFECT

40

J. (El Paso) Assertion Type

What goes in this field?The Assertion Type refers to the El Paso financial statement assertion that is achieved by the control. The El Paso assertions are: Authorization, Completeness & Accuracy, Evaluation of Balances, Access to Assets, Substantiation of Balances, Proper Classification, Presentation & Disclosure.

What issues and concerns must be taken into account?“Do all assertions apply to all processes?”All relevant assertions must be addressed within each cycle. Consider the following financial statement assertions which were used during the Documentation initiative.

Authorization - management has defined and communicated criteria for authorizing transactions.Completeness and Accuracy - all transactions have been recorded or considered in the proper period.Evaluation of Balances - all balances are recorded at appropriate amounts in accordance with standards.Access to Assets - physical safeguards restrict access to assets in accordance with management authorization.Substantiation of Balances - reports and database contents should be periodically substantiated.Proper Classification (All Transactions) - items in the financial statements are properly described and classified.Presentation and Disclosure - all items are fairly presented in conformity with generally accepted accounting principles.

If all relevant assertions are not addressed by critical controls, then additional controls must be designated as critical. In other words, if authorization is not covered by any of the critical controls within a particular cycle, then a significant or other control must be changed to be a critical control.


41

J2. COSO Assertion Type

What goes in this field?The COSO Assertion Type refers to the COSO financial statement assertion that is achieved by the control. The COSO assertions are: Existence, Occurrence, Completeness, Right & Obligations, Valuation or Allocation, Presentation & Disclosure.

What issues and concerns must be taken into account?“Why are there two set of assertions?”The PCOAB has indicated that companies should use the COSO framework for assessing internal controls over financial reporting. These standards were clarified after El Paso had started its documentation phase. During that documentation phase, El Paso chose an alternative set of assertions. Now, to ensure compliance with the PCAOB standards, El Paso is also recording the COSO assertion for controls that are being tested. Therefore, in addition to the El Paso assertions, the teams must also identify the relevant COSO assertions for each control included on the Controls Test Matrix.

42

J2. COSO Assertion Type (cont’d)

“How do El Paso’s assertions relate to the COSO assertions?”The following table presents the linkage between the COSO assertions (adopted by the PCAOB) and El Paso’s assertions. For illustration, consider a balance sheet, the assets of which include only one line item: Desks. . . $100. See the appendix for further guidance.

COSO Assertion(adopted by

PCAOB)Definition Example Related El Paso SOX

Assertions

Existence Assets, liabilities, and ownership interests exist as of a point in time.

Desks exist as of the reporting date. • Substantiation of Balances

• Access to Assets

Occurrence Recorded transactions represent economic events that actually occurred during the specified time period.

(If a purchase of desks had been made during the period, the cash flow statement might include “Purchases of desks. . . $25.” The purchase occurred.)

• Completeness and Accuracy

Completeness All transactions, events and circumstances that occurred during a specific period that should have been recognized in that period, have, in fact, been recorded and disclosed.

All the desks I own are represented here. • Completeness and Accuracy

Rights and Obligations

All assets on the balance sheet are property rights legitimately owned, and all liabilities are true obligations as of the date specified.

The desks are mine. I own them. • Authorization• Substantiation of

Balances

Valuation or Allocation

All transactions, events, and circumstances represented in the financial statements have been recorded at appropriate amounts according to relevant accounting principles.

The desks are truly worth $100. I paid $100 for them [assuming no depreciation].

• Completeness and Accuracy

• Evaluation of Balances• Substantiation of

Balances

Presentation and Disclosure

Items in the financial statements are adequately described, properly classified, and fairly presented.

What I am representing here is desks (as opposed to, say, chairs) and we can agree on what a generally accepted definition of chair is. The geography on the financial statements is correct.

• Presentation and Disclosure

• Proper Classification

43

K. Test Name3. Draft test plan

What goes in this field?This is a short description of the test being performed that should also distinguish the test from other tests. A more detailed test description will be documented later.

What issues and concerns must be taken into account?“Why do we need to create a “name” when we will be creating a detailed test description?”The Sarbox Portal (as currently configured) will use this name field to distinguish one test from another. Therefore, attempt to strike a balance between brevity and clarity. Since the Portal will eventually be used to list many tests, a user trying to differentiate multiple tests named, say, “Account Reconciliation” would have to look within each test individually until she finds the one she wants.

Examples:REP-MgrReviewOfInv-GLCodingMgrSignature;INQ-AP,Pol&Pro;OBS-AP_SegOfDuties;RETEST5-REP-MgrReviewOfInv-GLCodingMgrSignature

“What specific guidelines can we use to create test names?”1. The Test Name field in the Portal is limited to 50 characters, so the name may not be more than 50

characters long.2. The first three letters of the name should be abbreviations of the test type: Inquiry = INQ, inspection =

INS, observation = OBS and reperformance = REP (an exception will be for a retest which should begin with RETEST# then the abbreviation of the testing method.)

3. The rest of the name should be as descriptive as possible in revealing the nature of the test, including relevant distinguishing control and attribute features.

4. Use the list of abbreviations provided on the following slide (or others) wherever possible.

44

K. Test Name (cont’d)3. Draft test plan

45

L. Test Type

What goes in this field?This field documents whether the test is: Inquiry, Inspection, Observation or Reperformance. No other test types may be recorded here, and each test should be only one of these test types.A test could not be BOTH inquiry and reperformance. If a control is tested using both approaches, then the control would be associated with two tests - one inquiry test and one reperformance test.Example: Test – Discuss with warehouse personnel the criteria used when accepting goods for receipt into the warehouse. Observe warehouse personnel receiving goods, verifying goods match open purchase order and goods are not damaged. The discussion would be an inquiry test while the observation would be a separate observation test.

What issues and concerns must be taken into account?“What distinguishes inquiry, inspection, observation and reperformance?”The following exhibit details examples of each type of test:

3. Draft test plan

Shadowing

Viewing

Performance Metrics

Analytical Reviews

Inquiry Observation Inspection Reperformance

Scanning

Spot Checks

Document Reviews

Examinations

Witnessing

Interviews

Facilitated Sessions

Surveys

Confirmations

Representations

Reconciliations

Attributes Testing

46

L. Test Type (cont’d)

• Definition: “Asking the client about controls”

• Provides relevant information but not sufficient when used alone

– Does not provide adequate basis for management assessment

• Should include discussions on non-routine transactions and management override

3. Draft test plan

Inquiry Obser-vation

Inspec-tion

Re-perfor-mance

• Definition: “Seeing the control in action”• Useful when when physical controls are

present (example: blank checks are safeguarded)

• Risky to solely rely on observation as a testing technique

• Definition: “Reviewing relevant control documentation”

• Used when evidence exists of control operation (e.g. check marks, written explanations, etc.)

• Potential to conclude control is operating effectively when process owners are only “going through the motions”

• Should combine with limited inquiry

• Definition: “Redoing the operation of the control using selected transactions”

• More extensive than inquiry, observation, and examination

• Use when serious consequence of misstatement could occur if control is not operating effectively

• Potential to conclude control is operating effectively when process owners are only “going through the motions”

• Should combine with limited inquiry

47

L. Test Type (cont’d)3. Draft test plan

“How do I know which test type to choose?”Performing a combination of two or more of the four methods may be necessary to test each control. The conclusion regarding operating effectiveness is more reliable when corroborative evidence is obtained from a combination of testing methods.

When choosing a test type consider:

• Test Objective– What are you trying to determine in this test?– What level of assurance do you want from the test?

• Testing Methodology– How are you going to test this control? – What evidence is necessary to show that the control is operating

effectively?

• Sample– What data are you going to need to use in the test?– What data elements are you going to need to perform the test?– How is the data’s completeness ensured?– Is there any items that need to be excluded from the test?– How accessible is this data?– Where is this data located (e.g. system)?

48

“How do I know which test type to choose?” (cont’d)We want an efficient test plan but we do not want to compromise effectiveness for the sake of efficiency. Consider what evidence is available to test in order to choose the most effective combination of testing types.

3. Draft test plan L. Test Type (cont’d)

• Is there any documentation available?– Can the steps in the process to generate the documentation be reperformed? In this case, it

would make sense to combine inspection of documentation with reperformance.– Does the documentation indicate who performed each step? If so, it makes sense to perform

inquiry as well.• Does the control lend itself to observation?

– Manual controls are usually directly observable within the process. – Some system controls may not be directly observable but outputs may be obtained and

reviewed. – It is important to take into account the timing of the control execution when considering this

testing approach. Controls that are exercised infrequently may not lend themselves to observation.

• Who can you talk to about the operation of the control?– The control owner and/or others executing the control can provide valuable insight as to how

consistently and effectively the control is operating. This test type is most effective when combined with other types of tests to corroborate the information provided by the control owner and/or others executing the control.

• How can you reperform the execution of the control?– There are a number of controls that can be reperformed by the test team as part of the test

plan. We would suggest reperforming the controls within the test if it is feasible. • `

49


This attributes test is a REPERFORMANCE-type test, since it requires the tester to essentially reperform the type of analysis that the approving manager would have gone through to provide his approval.

The risk of this test is that the signed initials only imply the working of the control, since the tester cannot know whether the manager went through the analysis before signing – or whether the manager routinely ignores the control aspect of this procedure and signs everything that hits his desk without analyzing it.

Note: Although the tester is evaluating 4 attributes, this is one test.

The following discussion will help provide a context for evaluating the appropriateness of test types chosen for the testing. Controls testing to support Sarbanes-Oxley compliance will be somewhat different from the testing traditionally performed in external and internal audits.

Example: Control - Each operational manager reviews all invoices in his area for correct G/L coding, as well as accuracy and appropriateness of charges (i.e., it is an expense that correctly “belongs” to one of their projects). If all is correct, he manually signs and dates, indicating approval to pay. Incorrect or questionable invoices are investigated and resolved.

Test - Choose a sample of 30 invoices.

•Review for correct G/L coding.

•Review for appropriateness of business expenditure.

•Recompute charges to verify mathematical accuracy.

•Review for appropriate manager signature.

50


Sarbanes-Oxley Compliance Approach – REPERFORMANCE + INQUIRY

In order to mitigate the risk that a reperformance test is not sufficient by itself to gain comfort that the control is working the team should supplement testing with the following:

For a sample of invoices reviewed, inquire of the approving manager the following:

What analysis, specifically, do you perform on the invoices that you approve?

What procedures do you perform with respect to any invoices that you find to be questionable?

Please provide the most recent one or two examples of invoices that you questioned and walk me through the process by which you determined that it should be approved or rejected for payment.

Conclude as to the effectiveness of this control based both on the reperformance and the subsequent inquiries.

This approach provides a much more thorough understanding of the control’s operating effectiveness than reperformance alone. However, 30 individual follow-up inquiries would require a significantly greater level of resources to complete the testing than reperformance alone. The Testing Team must choose carefully the level of assurance needed and the means for getting there.

Note that the inquiry is a separate test from the reperformance.

51


“Why is an attributes test classified as a reperformance?”When performing an attributes test, like the one describe on slide earlier, the tester should be actually reperforming the steps that led to the particular attribute originally occurring.

Note that “inspecting” for the appropriate manager signature is only one step of reperforming this approval control. The objective is to reperform the analysis that the manager undertakes to determine whether approval to pay is appropriate.

When a single test such as this one involves two test types (reperforming the review, reperforming the computation, inspecting the signature) the more stringent test type should be chosen as the test type.

52

M. Test Description

What goes in this field?Document in detail the specific steps necessary to complete the control test from beginning to end. These steps should include adequate information so that a third-party could read the test procedures and understand the overall purpose of the test and could execute the test. Be clear and concise in writing the test procedures.

The test description should also indicate the proposed sampling unit, or item to be tested. The sampling unit constitutes one item in the population, such as a document, an entry, or a line item. Example: If the testing objective is to determine whether disbursements have been authorized and the prescribed control activity requires a duly authorized voucher before processing the disbursement, the sampling unit might be defined as the voucher. On the other hand, if one voucher pays several invoices and the prescribed control activity requires each invoice to be authorized individually, the line item on the voucher representing the invoice might be defined as the sampling unit.

If a significant control is not selected for testing, this is the area where the testing team manager should document the considerations leading to this conclusion.

3. Draft test plan

53

M. Test Description (cont’d)What issues and concerns must be taken into account?“Can I test the financial statement account for which transactions are being controlled?”No. The purpose of the SOX 404 testing is to verify internal controls. Tests should be designed to test the operation of those controls. Although the controls may be designed to provide comfort around account balances or disclosures, testing the affected account balance or disclosure does not provide the type of information needed to assess the operating effectiveness of the control.

Misstatements detected by the substantive procedures might alter the external auditor’s judgment about the effectiveness of controls. However, substantive tests of details do not provide the evidence needed to support an assertion on the effectiveness of internal control. The absence of misstatements detected by substantive procedures does not provide evidence that controls being tested are effective. Tests of controls in operation are required. (Source: PCAOB Release No. 2003-017; Paragraphs 143 & 144)

Example: Consider the controls related to accounts receivable.Test of Balances A test-of-balances approach used by an external auditor would include confirming specific receivables with third parties. This approach provides information about the balance, but it does not reveal much about the operation of the controls. Thus, the approach is not useful for controls evaluation.Test of Controls A test-of-controls approach would more likely involve an inquiry of the Accounts Receivable Manager regarding the specific criteria she uses to judge the collectibility of accounts during her review of the weekly A/R Aging report, along with an inspection of specific examples of accounts that she determined needed to be written off according to the specified criteria. The tester should strive to understand the manager’s process for dealing with both routine and non-routine events related to the collectibility of accounts.

3. Draft test plan

54

M. Test Description (cont’d)

“Are there any special considerations for monitoring controls?”When writing a test for a monitoring control, the tester should consider a review of the integrity of metrics, information and reports used during the monitoring process as well as understanding the actions taken by the process owner when any exceptions are identified, including identification of the root causes of the exceptions and ensuring appropriate process improvements or other necessary actions are taken to avoid the occurrence of future exceptions.

The test teams should remember to include inquiries regarding non-routine transactions and management overrides.

3. Draft test plan

55

N. Failure Conditions

What goes in this field?The test plan developer must make a precise statement of what constitutes a failure so that the individuals performing the testing procedures will have specific guidelines for identifying deviations.

What issues and concerns must be taken into account?“How do I know what a failure is?”A failure in tests of controls is a departure from adequate performance of the prescribed control activity. In other words, a failure is “what can go wrong” with respect to the operation of the control.

Assertions provide a good starting point for identifying potential failure conditions.Example: Suppose a prescribed control requires every disbursement to include an invoice, a voucher, a receiving report, and a purchase order, all stamped “paid.” If the existence of an invoice, voucher, receiving report and purchase order stamped “paid” are necessary to indicate adequate performance of the control, then a failure may be defined as “a disbursement not supported by an invoice, voucher receiving report or purchase order that have been stamped “paid.” All four documents must be present.

Note: Lack of sufficient documentation of control performance is a failure for all tests of controls.

3. Draft test plan

56

O. Assumptions3. Draft test plan

What goes in this field?The test plan developer will likely make certain assumptions about the data available to test. Assumptions may include but should not be limited to:1. Information/circumstances that need to exist prior to the control test beginning, and/or2. Information/circumstances that will provide additional assistance in executing the test plan.

What issues and concerns must be taken into account?“What is an example of an assumption?”Assumptions may be made relating to the availability of certain reports, for example. Since the teams will be using the documentation from the CFO Walkthroughs as a starting point for drafting the test plans, they should document any assumptions that are being made so that they may confirm those assumptions with the process owners.

57

P. Control Frequency3. Draft test plan

What goes in this field?This is a description of how often a control is performed.Examples: Daily, Monthly, QuarterlyThe purpose of this field is to aid in selecting an appropriate sample for testing.

What issues and concerns must be taken into account?“What do I include for controls that do not have any specific frequency?”Certain controls may not have a frequency.Example 1: Accounts Payable personnel maintain formal policies and procedures depicting the activities involved in the Accounts Payable process.

Example 2: Segregation of duties exist between the employees who enter invoices and employees who issue checks to vendors.

In these instances, place “Ongoing” in the field. No sampling is necessary. The controls testing will involve inspection of the policies, interviews with relevant individuals, etc. to ensure that the controls are operating as intended.

58

Steps 5 – 8: Evidence Compilation



6. Select sample




R-T

U-V

------

------


59


Who does this?Testing team and Testing Team Manager

What needs to be done?In order to execute the testing plan, the team must first select the items to be tested. This may include selecting people to interview and/or observe as well as documents to inspect and/or be used in reperformance. This will likely involve a discussion with the process owner to determine which people can answer questions and what documents are available. The team should also consult the CFO Walkthrough files to determine what documents are available.

At this point, the team will likely evaluate whether the test requires transactional or non-transactional support. Inquiries and observations are generally non-transactional in nature while inspections and reperformances are transactional. Transactional tests generally require the teams to obtain a population. A population consists of all of the items constituting the class of transactions subject to testing, as defined by the control.Example 1: If the tester tests a control designed to ensure that all shipments are billed, is the appropriate population shipped items or billed items? Answer: Shipped items.

Example 2: Consider the following control: “Invoices from alliance vendors are deemed approved when received by Accounts Payable because only approved purchasers are allowed to make purchases from alliance vendors.” In this instance the population is invoices from alliance vendors.

II. EvidenceCompilation

60

5. Determine evidence accessibility (cont’d)

Note: In some instances the process owner may direct the test team to IT personnel to assist with gathering population information. The inclusion of IT is encouraged, however, the team should not work directly with IT without the inclusion of the process owner. The process owner, who has knowledge of the process and the system fields, should coordinate all requests to IT to ensure that the appropriate information is being gathered. Ultimately, it is the process owner who is responsible for ensuring the achievement of the control objective. IT’s role, though important, is one of “helper,” not “owner” in achieving that objective, even if the control is completely automated. The testers should remain an active participant in this process.

“What if the information needed to execute the test is not available?”The testing team and Testing Team Manager need to determine why information is not available. If the issue is simply related to the way the test is worded and other testable evidence can be obtained from the process owner, the test team should rewrite the test as appropriate.

If, however, no alternative evidence is available for performing the test, the control can be considered to be not operating effectively. Based on the proposed PCAOB standards and PwC’s guidance, inadequate documentation is a deficiency.

“What do I do when process owner or IT does not provide data timely?”The teams should set deadlines when requesting information and they should work with the process owners to ensure those deadlines are met. If the process owners are not responding in a timely manner, the testers should notify the Testing Team Manager. If the issue persists, the Testing Team Manager should contact the PMO for assistance.


61

R. Population

What goes in this field?Clearly document the characteristics of the population, including all supporting information requested and the source from which the population was extracted. This could be a system file, binder, customer file, etc. Also, identify the employee (process owner) within the Business Unit who is responsible for maintaining/monitoring the information contained in the population. The Business Unit will be responsible for providing the testers with the information (sampled items) to test.

For non-transactional tests, such as inquiries and observations, the team should include information related to the “population” of people being interviewed or observed.

Example 1: From the previous example, the population characteristics are invoices from alliance vendors. The supporting information would be the invoice number, vendor number, vendor name, the invoice date, the payment date, amount, and G/L coding. The population source is the PeopleSoft A/P module. The responsible person is John Doe.

Example 2: Consider the following control: “Account reconciliations are performed monthly comparing the data transmitted from system A to system B. Any discrepancies are researched and resolved.” In this instance, the population consists of the reconciliations for the months within the testing period.

Example 3: 20 Revenue Accountants

5. Determine evidenceaccessibility

62

R. Population (cont’d)

What issues and concerns must be taken into account?“How do I know that I have the entire population?”It is not sufficient for a process owner to provide a file or list and say, “This is everything.” Testers must assume a posture of “professional skepticism” and obtain objective assurance that the population presented is both complete and accurate. If a process owner were trying to disguise an ineffectively operating control, the items he would be most likely to “leave out” of the population are the items we most likely want to see in our testing. It is the responsibility of testers to use sampling techniques that ensure they have a reasonable chance of choosing items indicating ineffective operation. Therefore, the tester must use professional skepticism to ensure the population is complete.

Note: One way to address this issue is to be involved with the process owners’ requests for data and having IT send population information directly to the tester. However, the process owner should be involved in discussions requesting the data, as indicated on slide 82.


63

S. Testing Period Start Date

What goes in this field?The testing period start date should be January 1, 2004, unless the control was not in operation at that time. In that case, the start date should be the date the control was placed into operation. For quarterly and yearly controls, the start date should be in 2003 to include Q4 ’03 for quarterly controls and the last time the control was performed for yearly controls.

What issues and concerns must be taken into account?How long should a control be in operation before testing?In general testing of controls for which gaps have been identified and remediation plans have been implemented should be delayed until the controls are known to be operating. Controls should be operating for a time period sufficient to measure via testing. If, for example, a control was remediated one week ago and that control has, therefore, been operating effectively for only one week, it is unlikely that it makes sense to test this control immediately. The control should be in operation for a “reasonable” period (i.e. at least long enough to select a sample according to the chart on slide 88.) For further guidance, see slide 106.


64

T. Testing Period End Date5. Determine evidenceaccessibility

What goes in this field?The testing period end date will be the date the population is requested.

What issues and concerns must be taken into account?“What if the testing period isn’t the entire year?” Tests of controls may be applied to transactions executed throughout the year or during the period from the beginning of the year to an interim date. When the testing period does not extend to the end of the year, “refresh” testing may need to be performed. See slide 120 for further information.

“Should we use a “clean” cut-off date?” Where appropriate the team may choose to use a “clean” cut-off dates, such as month-end. However, the team should strive to to use as long a period as possible to have as large a population as possible.

65

6. Select Sample


What needs to be done?In order to execute testing, the team must first select a sample to test. This involves determining which employees to select for interviews/observations as well as what documents to select for inspections/reperformances.

Before selecting the sample, however, the team must determine the sample size and selection method. The following slides provides additional information regarding sample sizes and methods.

The team should also consider using data mining techniques as a way of executing a thorough (100%) test when performing such a test would take about the same amount of time it takes to test a sample of items. The Testing Team Manager should consult the PMO for further guidance if your team identifies an area where data mining can be used.


66

U. Sample SizeWhat goes in this field?The team should enter the number of items that will be selected for performing the tests of controls.

What issues and concerns must be taken into account?“How many items should I test?”For manual controls, the minimum sample size should be determined based on the following table.

Critical Minimum Sample Size

Significant Minimum Sample

Size

Multiple-Times-Per-Day Control 30 5

Daily Control 20 3

Weekly Control 10 1

Monthly Control 3 1

Quarterly Control 3 1

Yearly Control 2 1

6. Select sample

These sample sizes take into account minimum requirements specified by PwC. The Testing Teamshould consider the following when determining the actual number of items to test:• Extent of manual oversight or involvement,• Complexity of control and• Planned reliance on control (critical versus significant).

Based on the results of the testing, additional items may need to be evaluated.(See Step 10. Evaluate Test Results.)

67

U. Sample Size (cont’d)

The sample sizes listed in the table on the previous slide are not absolute, though teams are discouraged from using smaller samples. Larger samples may be used for particularly significant controls or whenever a greater level of comfort is warranted.

“How do I test 3 quarterly and 2 yearly controls when we are starting testing in Q1?”Measuring the effectiveness of a once-a-year-only or quarterly control is of added risk because the external auditor will have to test the 2004 control directly, and it is unlikely there will be an opportunity to remediate the control if it fails. The team should initially test Q4’03 and Q1’04 for quarterly controls and’03 for yearly controls to identify any gaps. The teams must also test Q2’04 for quarterly controls and ’04 for yearly controls once those controls have been performed and are available for testing.

“If I have an automated control, do I have to test more than one?”Limiting the testing of automated controls to one test should only be done once IT general controls have been tested and found to be effective. The IT general controls are being tested simultaneously with the process controls (as a separate cycle); therefore, the teams should carefully consider limiting the sample sizes for automated controls. If the teams decide to assume the IT general controls are operating effectively and therefore conclude to limit testing of automated controls, those teams may have to do additional testing at a later date (prior to July 16) to ensure operating effectiveness of the process-level system controls.

Keep in mind that “testing one” means testing one of each condition as defined by the business rules, not just one transaction.

Example 1: The system requires invoices less than $500 to have one approval; over $500 have two. A tester must test two transactions: one less than $500 and one greater than $500.

6. Select sample

68

V. Sample Selection MethodWhat goes in this field?The team should document the process that was used for selecting the sample items.

What issues and concerns must be taken into account?“What selection methods are available?”

6. Select sample

Unrestricted random

numbers

Intervals

Stratifications

Haphazard

Selection Methods

• Each item in population has an equal chance of being selected

• Most common method• There is a uniform interval between each

item selected after a random start

• The population is segregated into two or more classes which are sampled separately

• Sample selected without any conscious bias, that is, without any special reason for including or omitting items from the sample; not careless

Explanation

• When items in population are numbered or are listed in a complete and accurate record

• Random sampling burdensome• No pattern in population that will bias

the sample• Items missing in population can be

identified• Considerable variation in population• More reliability from breaking the

population down into groups of comparable items

• No systematic way of selecting items

Population Characteristics

MORE Desirable

LESS Desirable

69

V. Sample Selection Method (cont’d)

“What role does materiality play in sampling in SOX compliance controls testing?”None. Materiality (i.e., of financial statement accounts) was indeed one factor considered in the Sarbanes-Oxley work several months ago during the selection of significant processes. But materiality plays no further role in controls testing.

Materiality of balances or transactions should NOT influence samples, unless the control explicitly states that the control applies ONLY to transactions of specified unit size or dollar value. If the control does not explicitly state such a condition, then that control applies to all transactions irrespective of their materiality or “significance” to particular financial statement accounts. In the absence of such a condition, sampling should NOT take into account transaction sizes or values (e.g., using stratified populations or dollar unit-type methods).

Example 1: A company has two bank accounts, one with a high level of activity and a high dollar balance, and one with a low level of activity and a low dollar balance. A control chosen for testing reads: Each company bank account is reconciled each month to its corresponding general ledger account. All differences between book and bank are investigated and resolved via the reconciliation. The general ledger cash account is promptly adjusted to reflect any unresolved differences. If the testing team were required to test only one reconciliation, it would NOT be appropriate to choose the higher-activity, higher-balance account on the basis of its higher activity and higher balance. The sampling method should strive to give equal weight to EACH instance of the control’s operation, in this case, EACH reconciliation. Each bank account should have an equal chance of being chosen for testing.

6. Select sample

70

V. Sample Selection Method (cont’d)“What role does materiality play in sampling in SOX compliance controls testing?” (cont’d)Example 2: A control governing payment processing that operates regardless of the dollar value of the payments or the underlying invoices should also be tested on a sample that is chosen without respect to payment or invoice values. In contrast, consider a control specifically written to address the value of expenditure transactions: All individual invoices totaling $50,000 or greater must be authorized for payment by the appropriate operational manager and at least one other Director-level employee in the department or project having budgetary authority for the expenditure. If such a control were chosen for testing, the testing team would, of course, request a population that would isolate expenditures of $50,000 or greater – an appropriate use of population stratification.

Example 3: A testing team plans to test control around inventory cycle counts. The control states that cycle counts should be performed monthly, with all inventory items being included at least once per quarter, with the exception of items individually valued at less than $25. The tester requests documentation of the last three cycle counts as well as a comprehensive listing of all inventory items individually valued at $25 or greater. Upon inspecting the documentation, the tester determines that cycle counts are routinely performed for only the top dollar value accounts. No cycle counts were performed on the other items in inventory during any of the last three months. The inventory accounting manager responds, “But why don’t you review the top-ten dollar value accounts like our external audit firm does when they perform cycle counts?” The tester responds, appropriately, “The external auditor is likely using their test to help evaluate the inventory balance in the general ledger. By contrast, we are testing the operational effectiveness of controls. Since the controls apply to each and every inventory item worth $25 or more, it would not be appropriate to exclude any items worth $25 or more from our review.”

It is important to remember that we are not conducting a financial statement audit when we perform tests of controls. If the testing team has any questions about how this affects the design or execution of their testing plans, they should consult the PMO.

6. Select sample

71

7. Coordinate evidence collectionWho does this?Testing Team

What needs to be done?The testing team needs to submit document requests and schedule inquiries and observations with the process owner. The team should clearly set deadlines for receiving requested documentation. The following form will facilitate the team’s interaction with the process owners.


This is an example format for a document request list. It is not a required workpaper, however, the volume of data requests will likely make the use of such a form necessary. It is recommended that the Testing Team Managers specify a suitable format for each of their teams, if this one does not meet their needs.

72

7. Coordinate evidence collectionThe Testing Team Manager should encourage coordination within the Test Team to limit the number of disturbances to process owners. When appropriate, multiple tests should use the same samples. The testers should consult previous data requests to see if the test could use earlier requested samples.

A team calendar is also recommended to document scheduled inquiries and observations to allow for coordination among testers. As noted in the planning section, the team may want to include process owner “black out” dates on their calendar.


73



What needs to be done?The team will need to maintain testing documentation in a neat and orderly manner to facilitate review by the Testing Team Manager and PMO. Documentation will include:

• High-level planning documentation by Business Unit - Cycle.

• Controls Test Matrix by subprocess.

• Requested documents list.

• Master calendar of inquiry and observation meetings.

• Workpaper files– Spreadsheets used for attributes tests– Memos documenting inquiries and observations– Copies of all documentary evidence obtained from process owners

A standard workpaper referencing scheme has been devised to aid in the organization and review of the workpapers.The teams should adhere rigorously to these documentation standards. Any deviations must be approved by the PMO.

Note: The only items that will be submitted to the PMO will be the Controls Test Matrix and the Workpaper files. The other items are suggested only for managing your work.


74

Standard Templates

These templates should be used for documenting detailed test results for inclusion in the Workpaper files.

8. Managetesting support

75

Steps 9 – 14: Test Plan Execution


III. Test Plan Execution







Testing Program Step Template Field

W-Y

Z

AA-AB, AE

AC-AD

------

AF


76

9. Execute test planWho does this?Testing Team

What needs to be done?The testing team should perform procedures described in the test description. The performance of the test procedures should be documented as described in 8. Manage testing support in accordance with the standards set forth in Step 15. Adhere to documentation standards.

“What if the control is not actually performed as described in the RCM?”The tester must review the control and the related risk to determine if changing the control description to reflect the actual performance of the control is acceptable or if changing the control would result in an unmitigated risk. If the control cannot be modified, then the control is deemed to be not operating effectively the team should consult slides 104 and 105 to determine how to proceed. If, however, the control as performed does mitigate the risk, then the test should be performed using the updated control description. Note, even changes in frequency (quarterly instead of monthly, for example) should be evaluated to determine if the control is mitigating the associated risk. Example: Consider the following control: “Account reconciliations are performed monthly comparing the data transmitted from system A to system B. Any discrepancies are researched and resolved.” If during the testing, the tester discovers that the reconciliations are only performed quarterly, the tester and Testing Team Manager should evaluate whether quarterly reconciliations are sufficient for mitigating the risk. The sample size and selected sample would be adjusted accordingly.

Any changes in the control description should be reviewed by the Testing Team Manager, as it may affect testing. A change control process is being developed for instances when the control description is incorrect and can be changed.

III. Test PlanExecution

77

W. Test Status9. Execute test plan

What goes in this field?Documentation status provides the status of the testing. The field should include one of the following: Complete, Not Started, Not Applicable, In Progress.

What issues and concerns must be taken into account?“Why do I need to mark the status?”Ultimately, documentation will be entered into the Sarbox Portal which includes this status field. However, in the interim, this field allows the Testing Team Manager to evaluate the progress of the Testers and to promote efficient review of each test.

78

X. Tester9. Execute test plan

What goes in this field?List the person who actually performs the controls testing. This person takes ownership of the testing and will assume responsibility for the timeliness and quality of the execution of the test and the accomplishment of the goal of determining whether a control is operating effectively.

What issues and concerns must be taken into account?“What if multiple people perform a test?”This field will be used to populate the Sarbox Portal which can only accommodate one name. Therefore, the team should appoint one person who is primarily responsible for each test and who can answer any questions related to the test.

79

Y. Workpaper Reference

What goes in this field?This field is used to record references for all workpapers that contain testwork related to the test.Example 1: X-CE-50Example 2: P-IT-1, P-IT-2, P-IT-18Example 3: M-AR-300 through M-AR-379

What issues and concerns must be taken into account?“What if multiple workpapers are used for one test?”There should be no workpapers related to a particular test that are not included in this field for that test. Completeness and accuracy are as important in this exercise as they are anywhere else in the testing, as reviewers, over-testers, and any others placing reliance on the testing will look to this field to determine where the testwork for any given test is documented.

Please see further guidance in Step 15. Adhere to Documentation Standards.


80

10. Evaluate test resultsWho does this?Tester, with thorough input from Testing Team Manager

What needs to be done?The goal of testing is to determine whether a control is operating effectively. In many cases, the answer will be clear. Example 1: In a test of five “attributes” in 30 sample items, there was not a single exception. In follow-up inquiries with the six employees who performed the control, it was clear that each was aware of his or her control responsibility; each indicated that he or she consistently performed all steps of the control; each was aware of exception-handling routines and provided at least one example of how he or she handled an exception; and each was able to provide the tester documentation supporting the handling of the exception. The tester would conclude that the control was operating effectively.

In other instances, the answer may not be as clear.

Example 2: In an inspection of two months’ variation analyses performed by the Director of the group, neither month’s package of supporting documents tied to the explanations indicated on the analysis. Upon inquiring about these apparent differences, the Director responded that the variations that she was questioning were resolved to her satisfaction, but that in neither month did she have time to include the appropriate supporting documents and actual variance explanations in the analysis package. The tester requested the evidence that the Director used to resolve her questions, discussed this evidence with her and understood how the evidence did indeed support the resolution in her analysis. It is likely the tester would conclude that the control is operating ineffectively. Lack of sufficient documentation is generally not going to produce an acceptable control outcome.


81

10. Evaluate test results (cont’d)“What do I do when a control fails?”The actions to take when a control fails depends on the importance of the control (critical versus significant), the frequency of the control operation and various other factors within those categories. The next two slides provide more detailed information. Using the initial sample sizes, one failure should lead the team to consult the guidance on these slides. In other words, the team DOES NOT need to finish testing the original sample.

After having considered alternative controls and expansion of testing, when the tester observes a “non-negligible deviation” when testing control performance and there is not an adequate explanation, it should generally be concluded that the control is “ineffective”.

• We generally should never agree a control is operating effectively when there is a “greater than insignificant” error rate.

• Our expectation is that the external auditor is likely to rarely, if ever, concur with a conclusion on effectiveness in situations where there are a significant number of errors.


82

10. Evaluate test results (cont’d)

Multiple times per day

Discuss failure with process owner to determine if additional failures are likely.

RemediateRetest using original

sample size based on frequency.

Critical Control


Daily

Weekly

Monthly

Quarterly

Yearly

Frequency

EndEffective

Ineffective

Additional failures likely

Expand sample* using statistical sampling:• 95% Confidence Level• 2% Upper Error Limit• 1% Expected Error Rate

Absolutely no failures anticipated

Ineffective**

EndEffective

“What do I do when a control fails?” (cont’d)*Results in sample size of 97.**May have 2 exceptions before test fails.

83


Multiple times per day

Discuss failure with process owner to determine if additional failures are likely.

Significant Control


Daily

Weekly

Monthly

Quarterly

Yearly

Frequency

Additional failures likely

Expand sample using statistical sampling:• 95% Confidence Level• 2% Upper Error Limit• 1% Expected Error Rate

Absolutely no failures anticipated

Ineffective

EndEffective

“What do I do when a control fails?” (cont’d)

Determine if an alternate

control mitigates the same risk(s)

Test alternate control using initial sample

sizes***

Ineffective

Remediate original control

End

Remediate both controls

Effective

Retest original control using original sample

size based on frequency.

Alternate control

available

Ineffective**

***The assessment of operating effectiveness of the original control will still be ineffective. However, the comments should reflect that the risk is mitigated by an alternate control which was deemed effective.

Alternate control NOT

available

Remediate original control

Effective

*Results in sample size of 97.**May have 2 exceptions before test fails.

84

Once remediation has been completed, Testing Teams will need to “retest” controls to ensure operating effectiveness.

“How long should we wait before retesting?”The controls should be operational for an adequate time period to enable management and the auditor to obtain sufficient evidence of the controls’ effective operation. Based on the suggested sample sizes (slide 68) we recommend the controls be in operation for the following time period before testing:

Note, however, that all testing must be completed by July 16, 2004. Therefore, a smaller sample size may need to be tested at this interim date to ensure the control is in fact operational as intended and a larger sample may be tested as part of the refresh testing to confirm operating effectiveness.

Minimum Time for Control Operation

Multiple-Times-Per-Day Control 1 Month

Daily Control 1 Month

Weekly Control 10 Weeks

Monthly Control 3 Months

Quarterly Control 2 Quarters

III. Test PlanExecution 10. Evaluate test results (cont’d)

85


“What if the results of the testing aren’t clear?”Much as we all would like pure black-and-white outcomes 100% of the time, there will be testing situations where the results do not produce black-and-white answers. Judgment is necessary. Example considerations include:

Do inquiries result in situations where people performing controls pay “lip service” to a control’s correct operation, but other evidence indicates they don’t follow through consistently?

Is the control one of a series? If it failed, how likely would it be that the failure would be detected and corrected by a process-level or monitoring “backup” control?

Are observation-based tests taking into account the likelihood that “watched” performers will perform controls correctly? What comfort do you have that the same performers behave similarly when they are not being observed?

The testing of the “other evidence” likely needs to be expanded. Results will need to fall within tolerable error limits for statistical samples.

Observation-based tests should be supplemented with additional testing.

The “backup” control should be tested and the results of that testing evaluated.

Consideration Possible Implication


When in doubt, the Test Team Manager should contact the PMO.

86

Z. Summary of Test Results

What goes in this field?Identify the test results for each attribute included in the control test. Document the number of exceptions identified through the controls testing in total and percentage.Example: Attribute 1 failed 0 times (0.0%); Attribute 2 failed 1 time (6.7%); Attribute 3 failed 0 times (0.0%). This indicates the control failed 1 time (6.7%).

What issues and concerns must be taken into account?“How can I record all the information about the test results in one field?”This is not the place for in-depth, detailed explanations of exception conditions or circumstances. This is a summary only. Detailed explanations should be documented in the supporting workpapers such as a memo, or a matrix that records the results of an attributes test, including explanations of any exceptions.

The testing team, with the assistance of the process owner, should document the resolution of each exception and whether an internal control deficiency exists. Once the tester has worked with the process owner to resolve all deficiencies possible, remaining deficiencies must be recorded on the Remediation Action Plan. (See Step 12. Coordinate Remediation Action Plan.)


87


Who does this?Testers.

What needs to be done?Testers should inform process owners of identified exceptions and set firm deadlines for addressing these items. Testers – and Testing Team Managers if necessary – should emphasize to process owners that addressing such items is a high priority because exception items generally indicate control gaps.

Before finally concluding that a tested item represents an exception, testers should notify process owners that such a conclusion is pending. This allows the process owner a “last chance” to seek and provide additional evidence that might result in a different conclusion.

Process owners should be given a reasonable amount of time to address these items, but test teams should also balance this need with the need to wrap up testing in a timely manner. It is generally not acceptable for process owners to ask for several days to address items, because this will result in a lengthy delay in completing testing.

“What form should the notification take?”Testing teams are encouraged to attach the standard cover page (see exhibit on the following slide) to the list of exception items of which the process owners are being notified.


88

11. Validate conclusions (cont’d)

This is an example of a cover page that could be attached to the list of exceptions provided to process owners. It is intentionally designed to direct the reader’s attention to the serious consequences that might result from unresolved exception items.

Testers are encouraged to discuss a proposed deadline with the process owners and fill in the blank during the discussion so as to ensure that the process owner acknowledges the deadline.

The use of this form is intended to streamline the notification to the process owners. Other than a quick follow-up e-mail (at the tester’s discretion), it is not expected that testers should have to contact process owners again to obtain the information. It is the process owners’ responsibility to provide the information once requested.


89

AA. Assessment of Operating Effectiveness

What goes in this field?Effective, Ineffective or Not Tested.

What issues and concerns must be taken into account?“Why would I use ‘not tested’?”“Not tested” should be used in the instances of significant controls that were not selected for testing.

“Is this the final assessment of operating effectiveness?”This is the team’s recommendation regarding the operating effectiveness. The final conclusion of operating effectiveness is the responsibility of management. Results from both the interim and refresh testing phases will be considered when finalizing the determination of operating effectiveness.If testing results differ between the interim testing and refresh testing phases, consideration should be given to the nature and timing of errors and whether a control was or should be remediated and the related impact on other controls.

“How do I record results for expanded tests, tests of alternate controls, or retests?”All results of testing must be maintained. Expanded tests should be recorded on the same line as the original test and the assessment should be based on the results of the expanded sample. For tests of alternate controls or retests, the original test should be marked ineffective and a separate line should be created on the Control Test Matrix for the test of the alternate control or retest with the additional assessment based on that new test. In the comments field for the original test, the Test Team should simply indicate the control was retested and deemed effective.Example: This control was remediated and deemed effective by RETEST1-REP-MgrReviewOfInv-GLCodingMgrSig.

11. ValidateConclusions

90

AB. Severity Level of DeficiencyWhat goes in this field?Deficiency, Significant Deficiency or Material Weakness.

What issues and concerns must be taken into account?“How do I choose the severity level?”The Test Team Manager should evaluate the significance of a deficiency in internal control over financialreporting initially by determining the following:

• The likelihood that a deficiency, or a combination of deficiencies, could result in a misstatement of an account balance or disclosure, and

• The magnitude of the potential misstatement resulting from the deficiency or deficiencies.

The tester then should conclude whether the ineffective control is likely to result in a deficiency,significant deficiency or material weakness.

• Deficiency – an internal control deficiency exists when the design or operation of a control does not allow management or employees, in the normal course of performing their assigned functions, to prevent or detect misstatements on a timely basis.

• Significant deficiency – a deficiency (or combination of deficiencies) that adversely affects the company’s ability to initiate, record, process or report external financial data reliably in accordance with GAAP.

• Material weakness – significant deficiency (or combination of significant deficiencies) that results in more than a remote likelihood that a material misstatement of the annual or interim financial statements will not be prevented or detected.

Any significant deficiency or material weakness should be reported to the PMO immediately.


91

AB. Severity Level of Deficiency“Is there any guidance on what should be included in the various levels?”Per guidance from PWC, deficiencies in the following areas are at least considered significant deficiencies:

• Controls over the selection and application of accounting policies that are in conformity with GAAP.• Antifraud programs and controls.• Controls over non-routine and nonsystematic transactions.• Controls over the period-end financial reporting process.

Note that PwC’s guidance continues to evolve as the PCAOB’s rules are released, clarified and interpreted.


92

AE. Comments

What goes in this field?Any additional information that would be useful to management in their overall evaluation of the test results for purposes of issuing a written assessment about the effectiveness of the company’s internal control over financial reporting.

What issues and concerns must be taken into account?“Is there anything that should be in the comments field?”When a team has done additional testing, the comments field is the place to note that subsequent testing. Since the original test must maintain the ineffective assessment, the comments field should indicate the control was retested and deemed effective.


93

12. Coordinate Remediation Action PlansWho does this?Testers, and, if necessary, Testing Team Managers.

What needs to be done?Once control gaps are identified in the testing and the results validated by the process owners, testing teams should assist process owners in preparing Remediation Action Plans to address the gaps. This may require meeting with the process owners to discuss potential resolutions.

When the decision tree leads to remediation, the team should consider the nature of the exception. Is the Nature of the exception due to:

• A poorly designed control (a design deficiency not detected during the earlier evaluation of design effectiveness?

• A properly designed control not operating as designed?• The person performing the control not possessing the necessary authority or qualifications to

perform the control effectively?Depending on the nature of the gap, it might require recharacterizing a control, i.e., changing it to state more specifically what steps are actually performed to mitigate the associated risk.

For all ineffective controls, an action plan should be developed to remediate the weakness as soon as practical. The remediation plan should allow sufficient time for validation by management AND the external auditor prior to year-end.

Once the Remediation Action Plan is prepared by the process owner, the Testing Team Manager should review the plan to ensure that it effectively addresses the gap and that the resulting control, if changed, is designed to appropriately mitigate the risk. Once the plan is deemed to sufficiently address the gap identified by testing, the Testing Team Manager should forward the plan to the PMO.


94


Internal Control Gap Additional Comments

5

Third party consultants (Mercer) prepare and coordinate with the Benefits department the assumptions for pension liability. The assumptions to be used in the Actuarial data are reviewed for reasonableness and approved by the El Paso Benefits Committee.

El Paso Benefits Committee doesn't currently review the assumptions for reasonableness. S

The Committee Chair will place review of assumptions on agenda for next committee meeting; include monthly. Linda Camarillo 1/15/2004

Responsible Party

Target Completion Date

Process Map

LinkageKey Control Description

(C) ritical or (S)ignificant

Control

Action to be Taken

The Testing Team should use the following form to document the action plan for resolving any gaps noted during the Management’s Testing. When completing the Remediation Action Plan, do not consider controls in isolation. Include discussion on processes, controls, roles and responsibilities, authority, reporting relationships, and training.


Tester ProcessOwner

95

AC. Internal Control Gaps

What goes in this field?This field is used to describe the specific gaps in the operating effectiveness of the control, including the root cause of the gap. In some cases the entire control may fail. In others, a portion of the control may fail.Example: A control might be the performance of analytical review procedures (e.g., variation analysis) as well as the complete and accurate documentation of that analytical review. Testing might discover that the analytical review is performed as stated, but the documentation is unacceptable. The gap should specify the “piece” of the control that fails, in this case the failure of the documentation step.

What issues and concerns must be taken into account?“Isn’t this already on the Remediation Action Plan?”The Remediation Action Plan is a separate document used to develop resolutions to internal control gaps. These gaps, which are discovered during the testing, must also be documented on the Controls Test Matrix.


96

AD. Overall Recommendation12. RemediationAction Plans

What goes in this field?This field contains the testing team’s recommendations regarding any gaps identified. Recommendations may include specific suggestions for remediation, changes to the control, or anything else that might benefit the control’s operation.

What issues and concerns must be taken into account?“Isn’t the process owner supposed to decide what action to take?”It is the process owner’s responsibility to prepare any necessary Remediation Action Plan steps related to the identified control gaps. The testing team should strive to provide meaningful assistance in this regard, but the recommendation field is not intended to place the burden of writing the Remediation Action Plan steps on the testing teams.

97


Who does this?Testing Team Manager.

What needs to be done?Testing teams should furnish the test results and associated Remediation Action Plans to the PMO. In addition, the Testing Team Manager should communicate the results of testing to the Business Unit’s respective CFO. The CFOs will want to know what is working well, but they will also be responsible for ensuring that whatever remediation efforts are needed are undertaken and completed. The CFOs will ultimately be responsible for resolving all gaps in the controls documentation, the design effectiveness and operation.

Remember, any significant deficiencies or material weaknesses should be reported to the PMO immediately.


98



What needs to be done?When the test covers an interim period, the tester must determine what additional evidence needs to be obtained for the remaining period until year end (“refresh” testing).

Considerations include:-The significance of the assertion involved.-The specific controls that were tested during the interim period.-Any changes in controls from the interim period.-The results of the tests of controls performed during the interim period.-The length of the remaining period.

Assuming that controls have not changed from the interim period, testing teams should provide their recommendation for updating testing at year end.


99

“What is refresh testing?”Refresh testing relates to situations where management has tested operating effectiveness at an interim date and must obtain additional evidence to support assertions as of the report date.

When testing is executed through an interim date, management must update testing through year-end. Testing controls at multiple points during the year helps to verify that evidence exists that a control is operating effectively throughout the year. However, it is necessary to perform sufficient testing through year-end to support the year-end assertion. Further, some controls may function only at year-end, thus it may only be feasible to test at year-end.

The Self-Assessor (TSA) will assist with performance of refresh testing. In addition, management will consider areas for additional review as suggested by the testing teams.

III. Test PlanExecution 14. Anticipate refresh testing (cont’d)

100

“What is TSA?”

The Self-Assessor (TSA) is a web-based tool that provides a central repository for key controls and action plans for an organization. It is a structured controls checklist that is accessed by El Paso’s process owners to monitor and rate control performance on a regular basis. TSA provides a detailed scoring and trending analysis of control performance on a specific control, control grouping, location, region, or other corporate-level aggregation.

The information related to processes and controls that is currently being populated in the Sarbox Portal will feed TSA. Process owners will then be asked to evaluate their areas, using specific questions and guidance provided by El Paso management. This tool enables El Paso to utilize the self-assessment validation method to assess the control environment at a point in time (i.e., as of year end).

The Suggested Refresh Testing provided by the Testing Teams will provide management with a basis for developing questions for process owners to answer in their assessment of their processes and controls.

III. Test PlanExecution 14. Anticipate refresh testing (cont’d)

101

AF. Suggested Refresh Testing12. RemediationAction Plans

What goes in this field?This field contains the testing team’s recommendations regarding updating testing at year end.

What issues and concerns must be taken into account?“What if the process changes before year-end?”The team should not speculate about changes to the processes and controls. The suggested testing approach should be written assuming the process will not change. If the team is aware of a pending change not related to a Remediation Action Plan, they should contact the PMO so documentation can be updated, if necessary. Otherwise the process owners are responsible for notifying the PMO of any changes in processes or controls.

102

Steps 15 – 17: Quality Assurance






Testing Program Step Template Field

------

------

------

103

15. Adhere to documentation standardsIV. QualityAssurance

Who does this?Testing teams.

What needs to be done?

ConsistencyAll documentation related to the testing should be prepared according to the documentation standards indicated here. These standards will provide consistency across the various testing teams.

Minimum level of workpapersAt a minimum the teams will have a Controls Test Matrix for each sub-process, supporting memos or attributes spreadsheets and the related source documents (e.g. invoices).

Clarity of presentationIt is of the utmost importance that whatever work is performed in the testing is accurately described. Vague descriptions of work performed are not helpful and could lead to an assumption that work was performed that was not actually performed.Example: It would be unacceptable to include in a memo (or, generally, anywhere else) the following sentence: “I spoke with employees in the accounting department about portions of the time reporting process.” Acceptable wording would be: “On March 4, 2004, I spoke with John Doe, Payroll Manager, and Jane Smith, Payroll Accountant, regarding the controls related to the approving of employee timesheets. Specifically, my inquiry focused on the following questions. . . .” Generally, if a reader can’t answer Who-What-Where-When-How-Why after reading the memo, note, or tic-mark explanation, the documentation needs to be clarified.

104

Memo

Memo

Questionnairewith responses

Inquiry Obser-vation

Memo

Recap

Tic marksdirectly on

sourcedocument

copies

Inspec-tion

Table (“matrix”)of attributes and

tic marks

Recap (e.g.,reasonableness

test)

Reperfor-mance

Examples of Documentation by Test Type15. Documentationstandards

Below are a few examples of documents that would typically be used as testing workpapers. This is not an exhaustive list, and testing teams should use whatever formats best support their testwork conclusions.

105

Memo Template15. Documentationstandards

This is a picture of the standard template that should be used for all memos created as workpapers.

106

Attributes Template15. Documentationstandards

This is a picture of the standard template that should be used for all tables of attributes created as workpapers.

107

Workpaper Referencing

Every workpaper must have a reference, which should be placed in the upper right hand corner of the page. The reference will be written in red in the form of: company code – cycle code – workpaper number. The table below summarizes the form of the referencing for each cycle in each business unit.Example: Workpaper X-PA-5 would be the fifth workpaper in the Corporate Payroll testing file.

15. Documentationstandards

Control Environment

Revenue and A/R

Expenditure and A/P Payroll Treasury Inventory Property Tax Financial

ReportingInformation Technology

CE AR AP PA TR IN PR TX FR IT

ANR A A-CE-__ A-AR-__ A-AP-__ A-PA-__ A-TR-__ A-IN-__ A-PR-__ A-TX-__ A-FR-__ A-IT-__

CIG C C-CE-__ C-AR-__ C-AP-__ C-PA-__ C-TR-__ C-IN-__ C-PR-__ C-TX-__ C-FR-__ C-IT-__

EPNG E E-CE-__ E-AR-__ E-AP-__ E-PA-__ E-TR-__ E-IN-__ E-PR-__ E-TX-__ E-FR-__ E-IT-__

TGP T T-CE-__ T-AR-__ T-AP-__ T-PA-__ T-TR-__ T-IN-__ T-PR-__ T-TX-__ T-FR-__ T-IT-__

SNG S S-CE-__ S-AR-__ S-AP-__ S-PA-__ S-TR-__ S-IN-__ S-PR-__ S-TX-__ S-FR-__ S-IT-__

Production P P-CE-__ P-AR-__ P-AP-__ P-PA-__ P-TR-__ P-IN-__ P-PR-__ P-TX-__ P-FR-__ P-IT-__

Corporate X X-CE-__ X-AR-__ X-AP-__ X-PA-__ X-TR-__ X-IN-__ X-PR-__ X-TX-__ X-FR-__ X-IT-__

Merchant M M-CE-__ M-AR-__ M-AP-__ M-PA-__ M-TR-__ M-IN-__ M-PR-__ M-TX-__ M-FR-__ M-IT-__

Field Services G G-CE-__ G-AR-__ G-AP-__ G-PA-__ G-TR-__ G-IN-__ G-PR-__ G-TX-__ G-FR-__ G-IT-__

108

Workpaper Referencing (cont’d)Hard copies of workpapers will physically reside in 3-ring binders and be organized in numerical order. There is no expectation that all numbers will be used (i.e., 1, 2, 3, 4, 5, 6, 7, . . .), only that any numbers that are used will be physically placed in numerical order within the files.Example: If EPNG’s Financial Reporting file contained a total of six workpapers, it would be organized like this: E-FR-1, E-FR-2, E-FR-10, E-FR-35, E-FR-36, E-FR-37. Or it might look like this: E-FR-1, E-FR-2, E-FR-3, E-FR-4, E-FR-5, E-FR-6. It would NOT be organized like this: E-FR-1, E-FR-4, E-FR-3, E-FR-6, E-FR-5, E-FR-2.

Workpapers containing multiple pages should be numbered sequentially using the following convention: 1/, 2/, 3/, etc.Example: A workpaper that is a two-page memo would have one workpaper reference (e.g., G-AP-15), but page one would be marked with 1/ and page two would be marked with 2/.

The specific workpaper reference __-__-1 should be used for the set of Control Test Matrices (one per subprocess) for each business unit and cycle, but there are no other prescribed numbers. The single workpaper reference should be used with separate page number as indicated below.

“Does every piece of paper in the workpaper files contain a reference?”Every workpaper must have a reference. However, if a stack of supporting evidentiary documents related to a sampled item can be stapled or clipped together easily, then a single workpaper reference may only be necessary for the stack. It may, however, be necessary to number the pages within the stack if frequent reference will be made to individual pages.

The key objective of the referencing is that there should be no unreferenced loose-leaf pages. A user of the file (e.g., a reviewer, an external-auditor over-tester, etc.) should not ever have to question to what particular test any given page in the workpapers applies.


109

Workpaper Referencing (cont’d)

“What should I consider in planning workpaper references?”Workpaper reference numbers should be considered carefully by the testing teams to promote the most efficient use and review of the workpapers. Specific considerations include:

• How many binders will eventually be used to house the workpapers for a given cycle?

• What types of tests will be used and what will the workpapers look like for each? Inquiry-type or observation-type tests may require brief or lengthy memos. Reperformance-type tests may require a summary checklist or matrix-type document and a large stack of copies of supporting evidentiary documents. Consider how many sampled items are being tested in determining how many binders will be needed and the best workpaper referencing scheme for organizing the paper in the binders.Example: If there will be several matrix-type documents summarizing detailed attribute testing, consider whether all the matrices will be grouped together and all the supporting documentation grouped together (Exhibit 15A) or whether it will be easier to include all the supporting documents directly behind each summary matrix document (Exhibit 15B).


110


Matrices

__-__-50 __-__-60 __-__-70

Matrix 3

Matrix 2

Matrix 1

__-__-500… __-__-600… __-__-700…

Supporting Documents

Support 1 Support 2 Support 3

Exhibit 15A


111


Matrix 3Matrix 2Matrix 1 Support 1

__-__-100 __-__-101… __-__-200 __-__-201… __-__-300

Support 3

__-__-301…

Support 2

Exhibit 15B


112


“Do I need to assign workpaper reference ranges?”Yes. If, for example, three different testers will be testing controls in three different processes within a cycle, it may be most efficient to assign three separate number ranges. Consider how many discrete “sets” of workpapers will comprise the overall set of workpapers for a given business unit cycle. It may be most efficient for the Testing team manager to assign workpaper numbers in ranges to the individual testers.Example: Consider a team of four testers, each having three controls to test, in which each test is expected to require significant numbers of supporting documents. The Testing Team Manager assigns numbers __-__-1000 to __-__-2999 to tester one, __-__-3000 to __-__-4999 to tester two, and __-__-5000 to __-__-6999 to tester three and instructs the testers to organize their testwork within those ranges.

The workpapers for each cycle will ultimately be housed in 3-ring binders.

“This all seems like an unnecessarily complicated way to go about the referencing.”Looking across the entire El Paso Sarbanes-Oxley Controls Testing phase, there may be hundreds of tests and potentially thousands of pages of source documents, memos, and other workpapers supporting each of those tests – each of which must be filed logically enough to enable a reviewer or external auditor to make their way through the whole thing – from risk, to control, to test, to testing support, to conclusion, to follow-up work related to the conclusion.


113

Other Documentation Standards

The files must contain all documents used to support the testing conclusions. Others will likely be using these files, so the objective in creating files should be to ensure that someone could pick up the file and follow the train of work from the test description itself – through the test workpapers and supporting documents – to the conclusion regarding the operating effectiveness of the control.

The files should not contain any pages that do not support the testwork. There should be no writing, scribbles, stray comments, pictures, doodles, or any other types of marks that do not support the testing. This includes any open ended comments.

Request that business unit personnel provide copies, where practical, of all requested documents, since it is likely that the copies will be included in our documentation files.

“Post-it”-style notes should not be used in the files. Use Workpaper Review Comments (“point sheets,” see slide 141) to indicate where additional work is needed in the file.

All workpapers should contain a:• Workpaper reference• Source (Examples: John Doe, El Paso Tax Department; Jane Banker, 7th National Bank)• Purpose (Example: This schedule summarizes the attributes testing for the following control tests. . . .)• Tester’s initials and date• Reviewer’s initials and date and squiggles or other identifying marks indicating specific items reviewed• Any explanatory notes necessary to understand relevant features of the testing. These may be written neatly wherever space permits on the paper or on a separate page so long as the user is referred to the note page.


114

Other Documentation Standards (cont’d)

C-AR-45Source: John Doe, Manager of . . .Purpose: This report is used to reconcile the . . . . DRT, 4/13/04

WRG, 5/30/04

BRW, 4/17/04

Note: According to David Doe, Accounting Supervisor, this reconciliation includes the expected activity for only half the month because. . . .

Source

Purpose

WorkpaperReference

Tester(s)’Initials and Date

Reviewer(s)’Initials and Date

ExplanatoryNote

Workpapers references and other elements of the documentation should be written in red pencil to aid efficient review.


Note: When a summary matrix is used (like those described on slide 105) this information may be included on the matrix and does not need to be repeated on each support document.

115

. . .

Memo Support –Other Source Docs

Memo Support –Reconciliation

Memo 2

Testing Support – Invoice 5 Testing Support – Invoice 4

Testing Support – Invoice 3 Testing Support – Invoice 2

Testing Support – Invoice 1

Attributes Testing Matrix

Memo 1

“So my workpaper files might look like. . .?”

This exhibit provides a graphical picture of what the structure of one testing binder – for one cycle – for one business unit – might look like. This is highly simplified. A typical set of workpapers is likely to be supported by many more memos, testing matrices, and other documents.


2/

Controls Test Matrix __-__-1 1/“Point Sheet”

(TEMPORARILYATTACHED TO

FILE UNTILALL REVIEWCOMMENTS

ARE CLEARED)

The Controls Test Matrix for each set of workpapers will be comprised of the full set of Excel spreadsheets (one for each process within the cycle), so it might number many pages when printed out and stored in the binder.

116

Documentation Retention

“What documents must be kept?”It will be necessary to retain copies of all documents reviewed as part of the testing activities. This will ensure that, in the event that questions are raised regarding the results of testing or if an external auditor desires to over-test our work, there will be no need to gather the original source documentation again (unless the external auditor specifically requires this).

Weigh practical alternatives, and use good judgment in deciding what to keep.Example: If a test requires inspection of many contracts, each of which numbers several hundred pages, it may not be necessary to copy and file each entire contract. Selected relevant pages would suffice, so long as what is included in the workpapers supports the testwork and conclusions.

“What if the process owner cannot/will not provide copies?”In certain instances, Business Unit managers may request that tested documents not be retained because they are of a confidential or sensitive nature. The Testing Manager must approve all instances in which a document used is not included in the workpapers. In these instances, it may be necessary to provide a detailed description of the excluded document in memo format to provide a sufficient “audit” trail.

“What about electronic files/queries?”Each Controls Test Matrix, and any supporting documentation, should be stored on the El Paso network and not on individual PCs. All supporting schedules created as part of the documentation should be saved to the network.

A “TEST” folder has been created by business unit for each subprocess. The teams should use these folders to store electronic files on the network.


117


Who does this?Testing Team Manager

What needs to be done?The testing team needs to plan the timing of the testwork to ensure that proper review is conducted. The Testing Team Managers are responsible for ensuring that testers are executing tests as documented, are exercising an appropriate level of professional judgment (including the required level of “professional skepticism”) during the execution, and are adhering with 100% consistency to the documentation standards.

“How do I conduct a review?”The reviews are expected to be thorough and in-depth. A proper review entails the following:• Revisit the related risk and control to provide a context for understanding the test.• Read the test to understand how the test accomplishes a proper evaluation of the related control.• Use the Workpaper Reference field to understand which workpapers record the underlying testwork.• The Workpaper Reference field must be complete and accurate.• Conduct a review of supporting documentation sufficient to understand whether the tester has executed the test accurately and completely and has completed testing or is making appropriate progress.• Read tester-prepared memos thoroughly for logic and grammar. Memos should be clear and concise. The body of memos must logically support any conclusions reached as a result of whatever testing activities are being recorded in the memo.

IV. QualityAssurance

118

16. Review progress and quality of work (cont’d)

“How do I conduct a review?” (cont’d)• Ensure that detailed reperformance-type tests are neatly organized. Each detail item tested must be documented in a way that leaves no doubt as to whether it constitutes an exception. If an item is, in fact, in doubt, the documentation must specify clearly the issue involved and what follow-up is being performed before it can be concluded upon.• Challenge the tester wherever documentation is not clear. Spot-check results and reperform testing activities to ensure that the tester has done exactly what is specified by the test. • Where follow-up with process owners is necessary to complete testing, satisfy yourself that the tester is making good progress in performing this follow-up. It may be necessary to be firm and insistent with process owners regarding follow-up issues – they’re busy, and they may need “encouragement” to provide your questions the necessary attention.• Provide concise guidance to testers as recurring documentation issues arise. Documentation standards are not optional, and it may be necessary to repeat them until they are learned and achieved consistently.• Ensure that all documentation standards are followed with 100% consistency.• Use Workpaper Review Comments (“Point Sheets”) to consolidate all review comments. See the example on the following slide.

Ultimately, testers are responsible for ensuring that the work they have done accurately measures the operating performance of the controls tested. Reviewers of their work ensure that testers understand and accept that responsibility as it manifests itself in the testwork.


119


This is an example of a “point sheet” that should be used to consolidate all review comments. All comments, or “points,” should be cleared by the tester. Until the reviewer clears the points, the point sheet may not be removed from the files. Points may be hand-written on any piece of paper, but they should be organized in the same way as the model point sheet shown here.


120


“What should the timing of the reviews be?”Timing of reviews is left to the discretion of the Testing Team Managers. However, it is recommended that their planning include target dates for performing reviews. It will likely prove impossible to conduct a proper review of the testwork all at one time at the completion of the testing. Sufficient time must be allowed for testers to clear all review comments, and this may require additional time spent with process owners to clear up outstanding questions and document requests.

At a minimum, it is recommended that reviewers:1. Spend time with each tester early in the testing execution to provide immediate, real-time guidance.2. Continue to be available at all times to answer questions and provide perspective on testing issues.3. Perform at least one interim review of the complete set of workpapers.4. Perform a final review of the complete set of workpapers.5. Be diligent in ensuring that all review comments are cleared.

The Testing Team Managers are encouraged, however, to structure their reviews and perform as many reviews as practical to ensure the highest quality and most efficient progression of work.


121


“Will others besides Testing Team Managers perform reviews?”It is anticipated that the PMO will also conduct reviews as an added quality assurance step. In addition, there may be representatives of the PMO “floating” from team to team to conduct an interim, targeted layer of reviews. These reviews are in addition to – not substitutes for – the Testing Team Manager’s reviews. The structure and nature of the additional reviews will evolve as the testing phase begins.


122


Who does this?Testing Team Manager

What needs to be done?The team needs to contact the PMO to allow for review of the work being performed. Because of the detail involved with testing and the related effort required to review the work, the PMO will need to review work periodically during the testing initiative. This review should be performed after the Testing Team Manager has performed an initial review of the testing, but should not be scheduled to occur solely at the completion of testing.


123

Recap of Today’s Material

Understand the purpose of the SOX 404 compliance project.

Understand what the project team has accomplished and what is left to do.

Understand the detailed steps of completing management’s testing.

Understand the roles and responsibilities of the testing participants.

124

Section 404 – Management Must Assess Internal Controls Annually Internal control report states management’s responsibility for establishing and maintaining adequate internal control structure and procedures for financial reporting.– Management must assess effectiveness of internal control

structure and procedures for financial reporting as of the end of the most recent fiscal year.

– Attestation by external auditor (Section 404 and 103).

Understand the purpose of the SOX 404 compliance project.


125

Understand what the project team has accomplished and what is left to do.


or

Gaps Identified Report

Finalize Documentation

Develop Testing Plans by Cycles (10) for 676 Business Processes

Project Management Knowledge Sharing Continuous Improvement Communication

Develop & Implement Solutions to Close Gaps; Modify Documentation; Monitor New Environment; Management Re-Tests to Ensure Gaps are Closed

Conduct Management Testing & Identify Control Gaps

External Auditor Testing & Re-Testing

OctNov Dec NovSepAugJulJunMayAprMarFebJan

Utilize Self Assessor to Monitor Control Environment & to Identify Future Changes

Remediate

Success

Remediate Success

Success

Milestones

CFO Walkthrough

s

PWC Walkthroughs

126

I. Test Plan CreationControls Test Matrix Field

------

------

A-P

Q

II. Evidence CompilationControls Test Matrix Field

R-T

U-V

------

------

III. Test Plan ExecutionControls Test Matrix Field

W-Y

Z

AA-AB, AE

AC-AD

------

AF



2. Select controls

3. Draft test plan


Test Step


6. Select sample



Test Step







Test Step





------

------

------


Understand the detailed steps of completing management’s testing.

127

Team Assignments

Understand the roles and responsibilities of the testing participants.


128

Success Factors

• Know and use available resources – training materials, templates, people

• Understand your assigned business processes – “what can go wrong?”, “what is in place to prevent those failures?”

• DON’T BE AFRAID TO ASK QUESTIONS!!• Build a plan and stick to it!• It’s Management’s Risks, Management’s Controls,

Management’s Plan and Management’s Conclusions.• Call it like you see it!• Communicate, communicate, communicate.• Don’t go it alone!

El-Paso SOX TestingTraining- June 2007

Documents

Transcript of El-Paso SOX TestingTraining- June 2007