Data Requirements for Valid Backtests June 2010 Marcus C. Bogue III, Ph.D. President, Charter Oak...
-
Upload
catherine-evans -
Category
Documents
-
view
214 -
download
0
Transcript of Data Requirements for Valid Backtests June 2010 Marcus C. Bogue III, Ph.D. President, Charter Oak...
Data Requirements for Valid Backtests
June 2010
Marcus C. Bogue III, Ph.D.
President, Charter Oak Investment Systems, Inc.
Data Requirements
• Data must be Accurate
• Data must reflect “What was Known at the Time”– Must answer the “What did you know and when
did you know it” question
Need Point in Time Data
Data Types and Reporting Characteristics
Market Data- Known immediately to all participants
Expectational Data- Known when published by an analyst
Fundamental Data- Known when reported by the company
FUNDAMENTAL DATA REPORTING CYCLE
• Preliminary Data– Press Release– Newswire– Etc.
• Finalized Data– Company Report
– Regulatory Filing
– Etc.
• Restated Data– Subsequent Company
Report
– Subsequent Regulatory Filing
RESTATEMENTS ALLOW INTER-PERIOD COMPARABILITY
Loews Corporation Total Revenue and Net Incomefor Fiscal Y06 - Y08 Initially Reported and Restated Prior Years(from Thomson Reuters Fundamentals Point in Time Database)
17911 17702
13844
18380
14320
2491 24912075
24891956
4319
13247
0
2000
4000
6000
8000
10000
12000
14000
16000
18000
20000
2/26/2007 Filing 2/27/2008 Filing 2/25/2009 Filing
Total Revenue Y06
Total Revenue Y07
Total Revenue Y08
Net Income Y06
Net Income Y07
Net Income Y08
Companies with Restated Compustat Net Income or Component Data
(based on July 29, 2005 Point in Time database)
0%
10%
20%
30%
40%
50%
60%
70%
80%
Mar
-87
Mar
-88
Mar
-89
Mar
-90
Mar
-91
Mar
-92
Mar
-93
Mar
-94
Mar
-95
Mar
-96
Mar
-97
Mar
-98
Mar
-99
Mar
-00
Mar
-01
Mar
-02
Mar
-03
Mar
-04
S&P 500
Next 1000 Largest by Market Capitalization*
Second 1000 Largest by Market Capitalization*
All Traded Companies**Excludes ADRs, REITs, Mutual Funds and Limited Partnerships
Restatements have yet to occur
% Interim Period Restatements U.S.(based on Thomson Reuters Fundamentals Point in Time Database)
0%
10%
20%
30%
40%
50%
60%
70%
Q1Y
98
Q1Y
99
Q1Y
00
Q1Y
01
Q1Y
02
Q1Y
03
Q1Y
04
Q1Y
05
Q1Y
06
Q1Y
07
Q1Y
08
Q1Y
09
INC Rstmt BAL Rstmt CAS Rstmt Any Rstmt
% Interim Period Restatements N. Europe(based on Thomson Reuters Fundamentals Point in Time Database)
0%
10%
20%
30%
40%
50%
60%
70%
Q1Y
98
Q1Y
99
Q1Y
00
Q1Y
01
Q1Y
02
Q1Y
03
Q1Y
04
Q1Y
05
Q1Y
06
Q1Y
07
Q1Y
08
Q1Y
09
INC Rstmt BAL Rstmt CAS Rstmt Any Rstmt
% Interim Period Restatements E. Asia (based on Thomson Reuters Fundamentals Point in Time Database)
0%
10%
20%
30%
40%
50%
60%
70%
Q1Y
00
Q1Y
01
Q1Y
02
Q1Y
03
Q1Y
04
Q1Y
05
Q1Y
06
Q1Y
07
Q1Y
08
Q1Y
09
INC Rstmt BAL Rstmt CAS Rstmt Any Rstmt
% Interim Period Restatements All Companies (based on Thomson Reuters Fundamentals Point in Time Database)
0%
10%
20%
30%
40%
50%
60%
70%
Q1Y
98
Q1Y
99
Q1Y
00
Q1Y
01
Q1Y
02
Q1Y
03
Q1Y
04
Q1Y
05
Q1Y
06
Q1Y
07
Q1Y
08
Q1Y
09
INC Rstmt BAL Rstmt CAS Rstmt Any Rstmt
% Annual Period Restatements U.S. (based on Thomson Reuters Fundamentals Point in Time Database)
0%
10%
20%
30%
40%
50%
60%
70%
Y98 Y99 Y00 Y01 Y02 Y03 Y04 Y05 Y06 Y07 Y08 Y09
Any Rstmt INC Rstmt BAL Rstmt CAS Rstmt
OVER / UNDER LAGGED(based on Thomson Reuters Fundamentals Point in Time Database)
LAG ASSUMPTIONS IMPACT RESULTS• Backtesting results can significantly differ by reporting lag assumptions
– Data is not Instantaneously available at the end of a reporting period
– No assumed reporting Lag is correct
– Reporting lag is a predictive variable
• Backtest when data was actually available to the market and with the values that were actually known
HIGH / LOW EARNINGS TO PRICE DECILES(based on Thomson Reuters Fundamentals Point in Time Database)
S&P 500 Monthly Rebalanced- 3 Month
0.1
1
10
10th E\P Decile - 3 mths lag 9th E\P Decile - 3 mths lag
2nd E\P Decile - 3 mths lag 1st E\P Decile - 3 mths lag
SP500 10th E\P Decile - PIT
9th E\P Decile - PIT 2nd E\P Decile - PIT
1st E\P Decile - PIT
S&P 500 Monthly Rebalanced- 4 Month
0.1
1
10
10th E\P Decile - 4 mths lag 9th E\P Decile - 4 mths lag
2nd E\P Decile - 4 mths lag 1st E\P Decile - 4 mths lag
SP500 10th E\P Decile - PIT
9th E\P Decile - PIT 2nd E\P Decile - PIT
1st E\P Decile - PIT
Point Data
• When Who Knew It?– The Market or Your Database
• Compustat and Worldscope Point in Time are based on When the Database Knew the Data
• Thomson Reuters Point in Time is based on When the Market Knew the Data
Compustat Average Pointdate from Period End Date
0
20
40
60
80
100
120
140
160
180
Q1Y
90
Q1Y
91
Q1Y
92
Q1Y
93
Q1Y
94
Q1Y
95
Q1Y
96
Q1Y
97
Q1Y
98
Q1Y
99
Q1Y
00
Q1Y
01
Q1Y
02
Q1Y
03
Q1Y
04
Q1Y
05
Q1Y
06
Q1Y
07
Q1Y
08
Q1Y
09
sp400 cavg sp500 cavg sp600 cavg
Reuters Average Pointdate from Period End Date
0
20
40
60
80
100
120
140
160
180
Q1Y
90
Q1Y
91
Q1Y
92
Q1Y
93
Q1Y
94
Q1Y
95
Q1Y
96
Q1Y
97
Q1Y
98
Q1Y
99
Q1Y
00
Q1Y
01
Q1Y
02
Q1Y
03
Q1Y
04
Q1Y
05
Q1Y
06
Q1Y
07
Q1Y
08
Q1Y
09
sp400 cavg sp500 cavg sp600 cavg
Comparison of Average Pointdate from Period End Date
0
20
40
60
80
100
120
140
160
180
Q1Y
90
Q1Y
91
Q1Y
92
Q1Y
93
Q1Y
94
Q1Y
95
Q1Y
96
Q1Y
97
Q1Y
98
Q1Y
99
Q1Y
00
Q1Y
01
Q1Y
02
Q1Y
03
Q1Y
04
Q1Y
05
Q1Y
06
Q1Y
07
Q1Y
08
Q1Y
09
Cmpst sp500 cavg Reuters sp500 cavg
Portfolio Value
Backtesting is an Art, Not a Science!