Changing Outlier Methodology for a Financial Survey
description
Transcript of Changing Outlier Methodology for a Financial Survey
![Page 2: Changing Outlier Methodology for a Financial Survey](https://reader033.fdocuments.us/reader033/viewer/2022051117/56815911550346895dc63d33/html5/thumbnails/2.jpg)
Outline
1. Motivation
2. Foreign Direct Investments (FDI) Survey
3. Outlier Detection & Treatment Methods
4. Analysis & Results
5. Recommendations
![Page 3: Changing Outlier Methodology for a Financial Survey](https://reader033.fdocuments.us/reader033/viewer/2022051117/56815911550346895dc63d33/html5/thumbnails/3.jpg)
Motivation
• Updated International Regulations• IMF’s Balance of Payments & International
Investment Position Manual• OECD’S Benchmark Definition of FDI
• New methodology for the FDI survey• New short questionnaire• Changes to stratification and sample design• Changes to Editing & Imputation methodology• Changes to Estimation & Outlier methodology
![Page 4: Changing Outlier Methodology for a Financial Survey](https://reader033.fdocuments.us/reader033/viewer/2022051117/56815911550346895dc63d33/html5/thumbnails/4.jpg)
Glossary
• Immediate Parent• Directly above affiliates in ownership chain
• Affiliate• Part owned by parent company (at least 10%)
![Page 5: Changing Outlier Methodology for a Financial Survey](https://reader033.fdocuments.us/reader033/viewer/2022051117/56815911550346895dc63d33/html5/thumbnails/5.jpg)
Foreign affiliates & UK parent companies
Foreign Direct Investments Survey (FDI)Collects information on the financial relationship between:
UK affiliates & foreign parent companies
UK Affiliate
Foreign Parent
1
Foreign Parent
2 Foreign Affiliate 2
Foreign Affiliate 1
UK Parent
The FDI survey is a key contributor to the UK’s balance of payments and international investments position
![Page 6: Changing Outlier Methodology for a Financial Survey](https://reader033.fdocuments.us/reader033/viewer/2022051117/56815911550346895dc63d33/html5/thumbnails/6.jpg)
FDI - Survey Details
Split into two surveys:• Inwards – UK affiliates, foreign parents• Outwards – UK parents, foreign affiliates
Both surveys collect quarterly & annual data
Estimate for three sectors: Oil, Finance, Other
Each sector contains a number of strata.• Stratified by size measures
![Page 7: Changing Outlier Methodology for a Financial Survey](https://reader033.fdocuments.us/reader033/viewer/2022051117/56815911550346895dc63d33/html5/thumbnails/7.jpg)
FDI - Survey Data
• Financial Survey – More than 35 questions• Most >= 0
• Four questions containing Positive & Negative values
• Large proportion of zeros• ‘Subsidiary Profit’ – 40% returned zeros in 2010 (Annual Inwards)
• Small sample size - ~2500 returned questionnaires (2010 Annual Inwards)
![Page 8: Changing Outlier Methodology for a Financial Survey](https://reader033.fdocuments.us/reader033/viewer/2022051117/56815911550346895dc63d33/html5/thumbnails/8.jpg)
Outlier Methods – What are Outliers?
Outliers - Extreme values, unlike the rest of the sample and with no special treatment could lead to over-estimates.
Representative Outliers:
A sample element with a value that has been correctly recorded and that cannot be regarded as unique.
-Chambers (1986)
![Page 9: Changing Outlier Methodology for a Financial Survey](https://reader033.fdocuments.us/reader033/viewer/2022051117/56815911550346895dc63d33/html5/thumbnails/9.jpg)
Outlier Methods – Why are Outliers important?
Estimated stratum totals (simplified):
Outliers give an inflated stratum mean, leading to over estimates.
N = population size, n = sample size
![Page 10: Changing Outlier Methodology for a Financial Survey](https://reader033.fdocuments.us/reader033/viewer/2022051117/56815911550346895dc63d33/html5/thumbnails/10.jpg)
Outlier Methods – Aims
Aims of this work:
• Compare 3 different outlier detection and treatment methods
• Test all 3 in a simulation study, based on real survey data
![Page 11: Changing Outlier Methodology for a Financial Survey](https://reader033.fdocuments.us/reader033/viewer/2022051117/56815911550346895dc63d33/html5/thumbnails/11.jpg)
Outlier Methods – Current Method (Trim)
Positive values only: Top 2% of values removed
Positive & Negative: Bottom 2% of values are also removed
Used to calculate mean
Before Trimming:Mean = 20.8
AfterTrimming:Mean = 5
![Page 12: Changing Outlier Methodology for a Financial Survey](https://reader033.fdocuments.us/reader033/viewer/2022051117/56815911550346895dc63d33/html5/thumbnails/12.jpg)
Outlier Methods –Distance from the Mean (Dist)
If this does not hold for y, then y is an outlier & removed.
Used to calculate mean
Before DIST:Mean = 20.8
After DIST:Mean = 5
![Page 13: Changing Outlier Methodology for a Financial Survey](https://reader033.fdocuments.us/reader033/viewer/2022051117/56815911550346895dc63d33/html5/thumbnails/13.jpg)
Outlier Methods – One-sided Winsorisation (Win)
Reduces large values which are considered outliers.
Example:Before WIN:Mean = 20.8
After WIN:Mean = 12.5
To determine & treat outliers, use the ‘L-value’ parameter (L), design weight ( ) and value mean ( )
y* replaces y when calculating the mean
![Page 14: Changing Outlier Methodology for a Financial Survey](https://reader033.fdocuments.us/reader033/viewer/2022051117/56815911550346895dc63d33/html5/thumbnails/14.jpg)
Winsorisation – Negative Values
Questions containing negatives:
One-sided Winsorisation will not work.
Solution: Create two new variables
![Page 15: Changing Outlier Methodology for a Financial Survey](https://reader033.fdocuments.us/reader033/viewer/2022051117/56815911550346895dc63d33/html5/thumbnails/15.jpg)
Analysis
• Take our returned sample data as the ‘population’• Sample and apply outlier detection methods• Calculate Bias Ratio & MSE over 10,000 independent
samples• Results created for the Finance & Other sectors
l = 1, 2, 3, ........, L Y = stratum pop total = stratum total (sample estimate)
![Page 16: Changing Outlier Methodology for a Financial Survey](https://reader033.fdocuments.us/reader033/viewer/2022051117/56815911550346895dc63d33/html5/thumbnails/16.jpg)
Results – MSE(Unquoted Equity Cap & Reserves)
Finance Sector: MSE against Iteration
MS
E
No. INDEPENDENT SAMPLES
![Page 17: Changing Outlier Methodology for a Financial Survey](https://reader033.fdocuments.us/reader033/viewer/2022051117/56815911550346895dc63d33/html5/thumbnails/17.jpg)
Results – MSE (Unquoted Equity Cap & Reserves)
Other Sector: MSE against Iteration
MS
E
No. INDEPENDENT SAMPLES
![Page 18: Changing Outlier Methodology for a Financial Survey](https://reader033.fdocuments.us/reader033/viewer/2022051117/56815911550346895dc63d33/html5/thumbnails/18.jpg)
Results – Bias, Variance & MSE
• Other sector – very large variance due to unrepresentative outlier – limitation of small population
![Page 19: Changing Outlier Methodology for a Financial Survey](https://reader033.fdocuments.us/reader033/viewer/2022051117/56815911550346895dc63d33/html5/thumbnails/19.jpg)
Results – MSE & Bias Ratio
• RR MSE – Scaled version of MSE.
• All 3 methods similar in RR MSE• All 3 methods under-predict at sector level
![Page 20: Changing Outlier Methodology for a Financial Survey](https://reader033.fdocuments.us/reader033/viewer/2022051117/56815911550346895dc63d33/html5/thumbnails/20.jpg)
Results – Outliers Detected
![Page 21: Changing Outlier Methodology for a Financial Survey](https://reader033.fdocuments.us/reader033/viewer/2022051117/56815911550346895dc63d33/html5/thumbnails/21.jpg)
Conclusions
• Compared to trimming (current method):• DIST- gives more stable results, but has larger
biases• Winsorisation – Higher Variance, but consistent
Bias Ratio. Best MSE
• Sampling caveats• Small population – hard to generalize to total population• Can cause problems with non-representative outliers• Changes to sampling rates require different L-values
![Page 22: Changing Outlier Methodology for a Financial Survey](https://reader033.fdocuments.us/reader033/viewer/2022051117/56815911550346895dc63d33/html5/thumbnails/22.jpg)
Recommendations
• Overall Winsorisation is the recommended method• Best in terms of RRMSE & gives good Bias ratios• Uses treated outliers, rather than removing them
(good for small amounts of data)• Due to be implemented in 2013
• Further work• Attempt simulation study with a pseudo-population• Apply simulation study to other surveys