Sampling Techniques That Work - Chapters Site - Home · PDF filePurpose of Sampling in...
Transcript of Sampling Techniques That Work - Chapters Site - Home · PDF filePurpose of Sampling in...
![Page 1: Sampling Techniques That Work - Chapters Site - Home · PDF filePurpose of Sampling in Governmental Auditing Application of an audit procedure to less than 100% of items within a class](https://reader033.fdocuments.us/reader033/viewer/2022052709/5a78d0c77f8b9a83238cdcb5/html5/thumbnails/1.jpg)
Sampling Techniques That Work for Public Sector Auditors 2013
IIA, San Diego March 14, 2013 Lois W. Sayrs, Ph.D.
![Page 2: Sampling Techniques That Work - Chapters Site - Home · PDF filePurpose of Sampling in Governmental Auditing Application of an audit procedure to less than 100% of items within a class](https://reader033.fdocuments.us/reader033/viewer/2022052709/5a78d0c77f8b9a83238cdcb5/html5/thumbnails/2.jpg)
Session Objectives
Understand basics of probability and non-probability sampling
Become aware of the range of samples that work well for public sector auditing
To know how, when and why to use various sampling techniques on audits
To review a sampling case study and draw implications to public sector audits
![Page 3: Sampling Techniques That Work - Chapters Site - Home · PDF filePurpose of Sampling in Governmental Auditing Application of an audit procedure to less than 100% of items within a class](https://reader033.fdocuments.us/reader033/viewer/2022052709/5a78d0c77f8b9a83238cdcb5/html5/thumbnails/3.jpg)
Purpose of Sampling in Governmental Auditing Application of an audit procedure to
less than 100% of items within a class or account balance for the purpose of evaluating some characteristics of that class or balance
![Page 4: Sampling Techniques That Work - Chapters Site - Home · PDF filePurpose of Sampling in Governmental Auditing Application of an audit procedure to less than 100% of items within a class](https://reader033.fdocuments.us/reader033/viewer/2022052709/5a78d0c77f8b9a83238cdcb5/html5/thumbnails/4.jpg)
Comparing Financial and Performance Sampling Techniques Financial
To provide assurance that the financial statements are not materially misstated
AICPA Audit Guide, Audit Sampling, 2012 • Performance materiality • Tolerable misstatment
Performance
To support conclusions of findings made as a result of the audit.
Risk of drawing false conclusion No direct guidance or clear audit standards
![Page 5: Sampling Techniques That Work - Chapters Site - Home · PDF filePurpose of Sampling in Governmental Auditing Application of an audit procedure to less than 100% of items within a class](https://reader033.fdocuments.us/reader033/viewer/2022052709/5a78d0c77f8b9a83238cdcb5/html5/thumbnails/5.jpg)
Performance Materiality
Performance materiality means the amount or amounts set by the auditor at less than
materiality for the financial statements as a whole to reduce to an appropriately low level the probability that the aggregate of uncorrected and undetected misstatements exceeds materiality for the financial statements as a whole.
If applicable, performance materiality also refers to the amount or amounts set by the auditor at less than the materiality level or levels for particular classes of transactions, account balances, or disclosures.
Performance materiality is similar to tolerable misstatement; however, tolerable misstatement is associated with sampling only. (AU 320.09)
![Page 6: Sampling Techniques That Work - Chapters Site - Home · PDF filePurpose of Sampling in Governmental Auditing Application of an audit procedure to less than 100% of items within a class](https://reader033.fdocuments.us/reader033/viewer/2022052709/5a78d0c77f8b9a83238cdcb5/html5/thumbnails/6.jpg)
Examples of Sampling from Public Sector Audits Validate data bases, reports Compliance tests Substantive tests on reasonableness
of balances Establish rates of occurrence in the
population Establish total dollars in the
population
![Page 7: Sampling Techniques That Work - Chapters Site - Home · PDF filePurpose of Sampling in Governmental Auditing Application of an audit procedure to less than 100% of items within a class](https://reader033.fdocuments.us/reader033/viewer/2022052709/5a78d0c77f8b9a83238cdcb5/html5/thumbnails/7.jpg)
Adapting Sampling Theory for Auditing Does not have to be statistical Can accept risk at fairly high levels Can accept likelihood of error Confidence levels adjusted for
variance and unknown populations Sample sizes are smaller Statements meet performance
materiality
![Page 8: Sampling Techniques That Work - Chapters Site - Home · PDF filePurpose of Sampling in Governmental Auditing Application of an audit procedure to less than 100% of items within a class](https://reader033.fdocuments.us/reader033/viewer/2022052709/5a78d0c77f8b9a83238cdcb5/html5/thumbnails/8.jpg)
Probability Theory/Sampling Distributions Why small samples work Example of sampling distribution
![Page 9: Sampling Techniques That Work - Chapters Site - Home · PDF filePurpose of Sampling in Governmental Auditing Application of an audit procedure to less than 100% of items within a class](https://reader033.fdocuments.us/reader033/viewer/2022052709/5a78d0c77f8b9a83238cdcb5/html5/thumbnails/9.jpg)
Sampling Techniques that work: 1. Ask what kind of sample do I need?
Probability: when generalization is required • Estimate total dollars in population • Estimate total errors
Non-probability: when generalization is not required
• Validation • Compliance
![Page 10: Sampling Techniques That Work - Chapters Site - Home · PDF filePurpose of Sampling in Governmental Auditing Application of an audit procedure to less than 100% of items within a class](https://reader033.fdocuments.us/reader033/viewer/2022052709/5a78d0c77f8b9a83238cdcb5/html5/thumbnails/10.jpg)
2. Know the unit of analysis
Sampling units-what are you sampling: item, dollar, event, individual, case, team, day, hour, complaint, etc.?
What is the question you are asking and what unit will best answer it?
![Page 11: Sampling Techniques That Work - Chapters Site - Home · PDF filePurpose of Sampling in Governmental Auditing Application of an audit procedure to less than 100% of items within a class](https://reader033.fdocuments.us/reader033/viewer/2022052709/5a78d0c77f8b9a83238cdcb5/html5/thumbnails/11.jpg)
3. Know the population
Estimate population if exact population is not available
Risk and confidence=prior knowledge Best practice is for the auditor to consider all special knowledge about the
items constituting the class prior to designing the sample AICPA 2.31. Estimate completeness 3 steps (external; internal-2 way)
![Page 12: Sampling Techniques That Work - Chapters Site - Home · PDF filePurpose of Sampling in Governmental Auditing Application of an audit procedure to less than 100% of items within a class](https://reader033.fdocuments.us/reader033/viewer/2022052709/5a78d0c77f8b9a83238cdcb5/html5/thumbnails/12.jpg)
4. Determine what type of sample will work best Representativeness Randomness Risk of making an error-drawing the wrong
conclusion Probability or non-probability
![Page 13: Sampling Techniques That Work - Chapters Site - Home · PDF filePurpose of Sampling in Governmental Auditing Application of an audit procedure to less than 100% of items within a class](https://reader033.fdocuments.us/reader033/viewer/2022052709/5a78d0c77f8b9a83238cdcb5/html5/thumbnails/13.jpg)
Probability Sampling
Types of Probability Samples A. Attribute sampling B. Acceptance sampling C. Discovery sampling D. Variable sampling E. Work sampling
• Random or predetermined • Extended cycle analysis • Fractioned professional estimates
![Page 14: Sampling Techniques That Work - Chapters Site - Home · PDF filePurpose of Sampling in Governmental Auditing Application of an audit procedure to less than 100% of items within a class](https://reader033.fdocuments.us/reader033/viewer/2022052709/5a78d0c77f8b9a83238cdcb5/html5/thumbnails/14.jpg)
A/B: Attributes/Acceptance Sampling Attributes and Acceptance Calculate maximum number of
acceptable errors based on probability tables.
Short-cut: “stop-n-go” Examples: files; data fields;
compliance tests; lots
![Page 15: Sampling Techniques That Work - Chapters Site - Home · PDF filePurpose of Sampling in Governmental Auditing Application of an audit procedure to less than 100% of items within a class](https://reader033.fdocuments.us/reader033/viewer/2022052709/5a78d0c77f8b9a83238cdcb5/html5/thumbnails/15.jpg)
C:Discovery Sampling
Discovery Sampling used to explore a population Small samples Big risk, low confidence Perfect for preliminary survey Poor for fraud test Must be added to during fieldwork
![Page 16: Sampling Techniques That Work - Chapters Site - Home · PDF filePurpose of Sampling in Governmental Auditing Application of an audit procedure to less than 100% of items within a class](https://reader033.fdocuments.us/reader033/viewer/2022052709/5a78d0c77f8b9a83238cdcb5/html5/thumbnails/16.jpg)
D:Variable Sampling
Variable Sampling Used for substantive audit tests Establishes valid estimates Examples: dollars lost to inefficiencies; cars
left unutilized; overpayments to school districts; wait time; consumer spending at the state fair
Requires big sample and well-conceived sampling strategy to minimize bias
Case review: Arizona MVD wait time
![Page 17: Sampling Techniques That Work - Chapters Site - Home · PDF filePurpose of Sampling in Governmental Auditing Application of an audit procedure to less than 100% of items within a class](https://reader033.fdocuments.us/reader033/viewer/2022052709/5a78d0c77f8b9a83238cdcb5/html5/thumbnails/17.jpg)
E:Work Sampling
Work Sampling Random evaluation of time periods to assess
compliance with work requirements or standards (efficiencies); assess delays
Predetermined evaluation of time periods to help set standards, e.g. peak times
Extended cycle analysis-sample all or part of the life cycle of a process through controlled reporting by professionals; uses time logs or ladders
Fractioned professional estimates: sampling various professionals on how they do a part or fraction of the process and relying on their estimates of how long it takes
Example: Arizona wait times for authorizing unemployment insurance via phone-combination method (random, predetermined, fractioned)
![Page 18: Sampling Techniques That Work - Chapters Site - Home · PDF filePurpose of Sampling in Governmental Auditing Application of an audit procedure to less than 100% of items within a class](https://reader033.fdocuments.us/reader033/viewer/2022052709/5a78d0c77f8b9a83238cdcb5/html5/thumbnails/18.jpg)
Common Errors in Attribute/Variable Sampling Sample too small Sample biased Generalize on the wrong basis
![Page 19: Sampling Techniques That Work - Chapters Site - Home · PDF filePurpose of Sampling in Governmental Auditing Application of an audit procedure to less than 100% of items within a class](https://reader033.fdocuments.us/reader033/viewer/2022052709/5a78d0c77f8b9a83238cdcb5/html5/thumbnails/19.jpg)
5. “Design on a dime”
Sample should optimize budget and ability to conclude so select carefully: A. Random B. Systematic C. Stratified D. Cluster E. Multi-stage F. PPS G. Combinations
![Page 20: Sampling Techniques That Work - Chapters Site - Home · PDF filePurpose of Sampling in Governmental Auditing Application of an audit procedure to less than 100% of items within a class](https://reader033.fdocuments.us/reader033/viewer/2022052709/5a78d0c77f8b9a83238cdcb5/html5/thumbnails/20.jpg)
Sampling Designs
A. Random-uses a random number generator, a random start
B. Systematic-uses a random start when data are ordered. Sample every “nth” item. Keep cycling through the data until you complete sample. Can be done with a computer for electronic data or a ruler for hard files.
![Page 21: Sampling Techniques That Work - Chapters Site - Home · PDF filePurpose of Sampling in Governmental Auditing Application of an audit procedure to less than 100% of items within a class](https://reader033.fdocuments.us/reader033/viewer/2022052709/5a78d0c77f8b9a83238cdcb5/html5/thumbnails/21.jpg)
Sampling Designs
C. Stratified-uses “cuts”, “layers” or “strata” to minimize variation (and risk) in the sample, e.g. school district size
D. Cluster-exploits data that naturally occurs in a clustered form such as school districts (district, school, classroom)
![Page 22: Sampling Techniques That Work - Chapters Site - Home · PDF filePurpose of Sampling in Governmental Auditing Application of an audit procedure to less than 100% of items within a class](https://reader033.fdocuments.us/reader033/viewer/2022052709/5a78d0c77f8b9a83238cdcb5/html5/thumbnails/22.jpg)
Sampling Designs
E. Multistage- samples at various stages in the process, e.g. contract bidding and contractor selection
F. PPS –probability proportionate to size(also known as dollar unit sampling)-break each transaction into single dollar units and sample over the dollar units to maximize likelihood of capturing large dollar transactions.
![Page 23: Sampling Techniques That Work - Chapters Site - Home · PDF filePurpose of Sampling in Governmental Auditing Application of an audit procedure to less than 100% of items within a class](https://reader033.fdocuments.us/reader033/viewer/2022052709/5a78d0c77f8b9a83238cdcb5/html5/thumbnails/23.jpg)
Sampling Designs
G. Combinations-effective joining of two or more sample types to minimize risk and variation, e.g. combine school district strata–sample over small medium and large school districts, by type (unified, elementary, union, high school) and clusters-by school, classroom and teacher within a district)
![Page 24: Sampling Techniques That Work - Chapters Site - Home · PDF filePurpose of Sampling in Governmental Auditing Application of an audit procedure to less than 100% of items within a class](https://reader033.fdocuments.us/reader033/viewer/2022052709/5a78d0c77f8b9a83238cdcb5/html5/thumbnails/24.jpg)
Sample Designs: Helpful Hints
When electronic data is not available:
bring a ruler When probability sampling is required:
plan on assigning enough audit budget to know your population
Let risk be your guide
![Page 25: Sampling Techniques That Work - Chapters Site - Home · PDF filePurpose of Sampling in Governmental Auditing Application of an audit procedure to less than 100% of items within a class](https://reader033.fdocuments.us/reader033/viewer/2022052709/5a78d0c77f8b9a83238cdcb5/html5/thumbnails/25.jpg)
6. Determine the sample size: key steps
Set risk and sample precision Set Confidence level Evaluate Variance Keep it simple
![Page 26: Sampling Techniques That Work - Chapters Site - Home · PDF filePurpose of Sampling in Governmental Auditing Application of an audit procedure to less than 100% of items within a class](https://reader033.fdocuments.us/reader033/viewer/2022052709/5a78d0c77f8b9a83238cdcb5/html5/thumbnails/26.jpg)
7. Consider Non-Probability Sampling
A. Risk-based B. Convenience C. Judgmental D. Snowball E. Quota
![Page 27: Sampling Techniques That Work - Chapters Site - Home · PDF filePurpose of Sampling in Governmental Auditing Application of an audit procedure to less than 100% of items within a class](https://reader033.fdocuments.us/reader033/viewer/2022052709/5a78d0c77f8b9a83238cdcb5/html5/thumbnails/27.jpg)
Non-Probability Samples
A. Risk-based/materiality based Uses highest risk items such as worst
offenders; most frequent users B. Convenience
Uses what is convenient such as closest behavioral health centers or schools
C. Snowball Uses one item to access another such as
using a school district employee to sample a third party
![Page 28: Sampling Techniques That Work - Chapters Site - Home · PDF filePurpose of Sampling in Governmental Auditing Application of an audit procedure to less than 100% of items within a class](https://reader033.fdocuments.us/reader033/viewer/2022052709/5a78d0c77f8b9a83238cdcb5/html5/thumbnails/28.jpg)
Non-Probability Samples continued D. Quota Uses a pre-specified amount and
stops such as open versus closed cases
E. Judgmental Uses auditor judgment to select
sample such as “Western States”; simply requires thought and adequate documentation.
![Page 29: Sampling Techniques That Work - Chapters Site - Home · PDF filePurpose of Sampling in Governmental Auditing Application of an audit procedure to less than 100% of items within a class](https://reader033.fdocuments.us/reader033/viewer/2022052709/5a78d0c77f8b9a83238cdcb5/html5/thumbnails/29.jpg)
8. Learn from experience Ask people who know Be creative Be confident
![Page 30: Sampling Techniques That Work - Chapters Site - Home · PDF filePurpose of Sampling in Governmental Auditing Application of an audit procedure to less than 100% of items within a class](https://reader033.fdocuments.us/reader033/viewer/2022052709/5a78d0c77f8b9a83238cdcb5/html5/thumbnails/30.jpg)
Capstone exercise
Small group exercise-911 calls response time
Develop at least two sampling strategies
Critique strategies Select one Share final sampling decision with
class
![Page 31: Sampling Techniques That Work - Chapters Site - Home · PDF filePurpose of Sampling in Governmental Auditing Application of an audit procedure to less than 100% of items within a class](https://reader033.fdocuments.us/reader033/viewer/2022052709/5a78d0c77f8b9a83238cdcb5/html5/thumbnails/31.jpg)
Finally…
Case Study Troubleshooting your own audit work
![Page 32: Sampling Techniques That Work - Chapters Site - Home · PDF filePurpose of Sampling in Governmental Auditing Application of an audit procedure to less than 100% of items within a class](https://reader033.fdocuments.us/reader033/viewer/2022052709/5a78d0c77f8b9a83238cdcb5/html5/thumbnails/32.jpg)
Thank you! Contact information: [email protected] Linked-in groups AGA ALGA IIA ICPA-International Center for
Performance Auditors NCSL NASACT