- 1. Empiricism :From Newton to Taleb; & its Relevance for
Quants! From Wikipedia: In thephilosophy of science
,empiricismemphasizes those aspects of scientific knowledge that
are closely related to experience, especially as formed through
deliberate experimental arrangements.It is a fundamental
requirement of thescientific methodthat
allhypothesesandtheoriesmust be tested againstobservationsof
thenatural world , rather than resting solely ona priori reasoning
,intuition , orrevelation . Hence, science is considered to be
methodologically empirical in nature. Allegiant Asset Management
Company is a SEC-registered investment advisor and a subsidiary of
National City Corporation.Wikipedia is a registered trademark of
the Wikipedia Foundation, Inc. Steven P. Greiner,
Ph.D.[email_address]
2. Newton & Einstein
- Thus far I have explained the phenomena of the heavens and of
our sea by the force of gravity, but I have not yet assigned a
cause to gravity.I have not as yet been able to deduce from
phenomena the reason for these properties of gravity.For whatever
is not deduced from the phenomena must be called a hypothesis; and
hypotheses whether metaphysical or physical, or based on occult
qualities or mechanical, have no place in experimental
philosophy
- Newton forgive me; you found the only way which in your age was
just about possible for a person with the highest powers of thought
and creativity.The concepts which you created are guiding our
thinking in physics even today, although we now know that they will
have to be replaced by others farther removed from the sphere of
immediate experience, for we know that science cannot grow out of
empiricism alone
3. Kelvin & Taleb
- When you can measure what you are speaking about, and express
it in numbers, you know something about it.But when you cannot
measure it, when you cannot express it in numbers, your knowledge
is of a meager and unsatisfactory kind: it may be the beginning of
knowledge, but you have scarcely, in your thoughts, advanced to the
stage of science
- William Thompson Lord Kelvin
- The relationship between the past and the future does not learn
from the relationship between the past and the past-previous to
it.When we think of tomorrow we just project it as another
yesterday.We fail to learn about the difference between our past
predictions and the subsequent outcomes
- From Black Swan; by Nassim Taleb
4. Empiricism : From Newton to Taleb;
- Hypothesis;Model;Theory;Law;
- Ahypothesisis a limited statement regarding cause and effect in
specific situations; it also refers to our state of knowledge
before experimental work has been performed and perhaps even before
new phenomena have been predicted.
- The wordmodelis reserved for situations when it is known that
the hypothesis has at least limited validity. Not necessarily a
precise and accurate depiction but it exists to provide a
mathematical representation of the hypothesis of interest allowing
comparison with experiment.
- Ascientific theory or lawrepresents an hypothesis, or a group
of related hypotheses, which has been upheld through repeated
experimental tests. Theories are often formulated in terms of a few
concepts and equations, which are identified with " laws of nature
, suggesting their universal applicability. Accepted scientific
theories and laws become part of our understanding of the world and
the basis for exploring less well-understood areas of
knowledge.Theories are not easily discardedbut have earned the
right to continue until the evidence is overwhelmingly in favor
against it.
- TheSCIENTIFIC METHODrequires that a hypothesis be ruled out or
modified if its predictions are clearly and repeatedly incompatible
with experimental tests.Further, no matter how elegant a theory is,
its predictions must agree with experimental results if we are to
believe that it is a valid description of the world.In experimental
science, " experiment is supreme " and experimental verification of
hypothetical predictions is an absolutely necessary (but not
necessarily sufficient) condition toward validation and
acceptance.
5. Application To Model Building for Use in Stock Selection and
Portfolio Construction
- Theorize (Einstein) : stock prices are a function of financial
statement data
- Collect Empirical Evidence (Newton) : form portfolios (stock
cohorts) based on financial statement data
- Obtain Numbers (Kelvin) : hold these portfolios for a period of
time and measure performance statistics on them
- Presume True Forward in Time (Taleb-NOT!) : thirsty bubbles
internet, credit crunches, LTCM, mortgages, October 1987, August
2007, sustain a competent level of paranoia..
- What happened?Whats wrong with this picture?Problem is with
understanding models...lets review..
6. Common Mistakes in ApplyingStatistics
- Interpreting aregressionR 2 as a measure of statistical
significance or using it to gauge whether one regression model is
significantly stronger than another.Its really rather poor at
that!
- Thinkingcorrelationis a general measure ofassociation .If two
factors areindependent , they aredefinitely un-correlated , but if
two factors areun-correlated , they arenot necessarily independent
.Covariance too.
-
- The correlation ofyandxin:y=x 2 is zero but they sure are
dependent!
- Ascertaining correlation, then implying causality.
- Not usingsingle value decompositionwhen doing regressions.In
practice, many similar variables are available and most factors in
a model are correlated.This can result in two such factors or
combinations of factors to fit the data equally well, yielding no
satisfied solution.
-
- Beware small (abs value) regression coefficients.
- Focusing on theaverage(i.e. return), and not being cognizant of
thespread or confidence intervaldue to errors or likelihood.
-
- A measurement without a quoted error is near meaningless.
-
- Accepting a model with an R 2of 0.05, because theaverage6 month
return of a cohort of 200 stocks out of 2,000 over 15 years is
3.12% higher than the benchmark!
7.
- Focusing on theaverage return , and not being cognizant of
thespread or confidence intervaldue to errors or likelihood.
8. Average Decile Model XS Returns Seemingly Looks Good! Source:
Allegiant proprietary information 9. Spread of Decile Model XS
Returns Okay Still! Source: Allegiant proprietary information 10.
Raw XS Return as function of Model Stock Rank argh.. Source:
Allegiant proprietary information 11.
- Focusing on theaverage return , and not being cognizant of
thespread or confidence intervaldue to errors or likelihood.
12. Source: Allegiant proprietary information 13. Source:
Allegiant proprietary information 14. Source: Allegiant proprietary
information 15. The 95% Confidence Interval about the Mean! Theres
some overlap, so its possible they have the same mean! Top Decile
Mod_1 (-0.74) Top Decile Mod_2 (5.92) Source: Allegiant proprietary
information 16. Source: Allegiant proprietary information 17.
Common Mistakes in ApplyingEmpiricism
- The most fundamental error is to mistake thehypothesisfor an
explanation of a phenomenon, without performing experimental tests.
Sometimes " common sense " and " logic " tempt us into believing
that no test is needed.
- Ignore or rule out data which do not support thehypothesis(
discounting pesky cases ). Anchoring often is an over-riding human
condition were unable to remove ourselves from ( i.e. buying value
stocks always works )
- The failure toestimate quantitativelysystematic errors. At the
minimum, we should be able to articulate their order of
magnitude.
-
- Error intrinsic to production of measurement. Because this type
of error has equal probability of producing a measurement higher or
lower numerically than the " true " value, it is called random
error.
-
- Non-random or systematic error, due to factors which bias the
result in one direction. No measurement, and therefore no
experiment, can be perfectly precise. At the same time, in science
we have standard ways of estimating and in some cases reducing
errors. Thus it is important to determine the accuracy of a
particular measurement and, when stating quantitative results, to
quote the measurement error.
-
- A measurement without a quoted error is near meaningless.
18. A Value Model Top (red) & Bottom (dots) Decile XS Return
Source: Allegiant proprietary information 19. Examine a Valuation
Component of the Model at Different Times.. Working Working Working
Source: Allegiant proprietary information ICs vary wildly through
time. 20. CONCLUSION :These data are representative of when factors
lose their efficaciousness at various points in history.One would
expect the slopes of return vs factor rank to fall off during the
bubble, 2003 and 2006 to the present,and indeed they do.The chart
below shows the R 2 s of regressions of the factors vs excess
return for each of the sixtime periods.Notice that the largest R 2
s are in the early years, after the bubble and in 2004 and 2005,
while the smallerrelationships hold for the problematic years
during the bubble, 2003 and 2006 to the present F1F2 F4F5F6F7F8F9
Source: Allegiant proprietary information 21. [ ] nICTC= IR Model
TC(i.e. optimization) [ ] Portfolio [ ] Bench + TC: Effectiveness
of Portfolio Construction Process IC: Effectiveness of Alpha Model
22. Implications of Empirically ConstructedModels on Portfolio
Construction
- One can have all top decile stocks in the portfolio and still
under-perform the benchmark ( why?, errors )
-
- due to volatility dynamics of models IC(-ve IC )
-
- due to TC being overly constrained by risk model or
constraints
-
- due to hard constraints not allowing feasible solution
-
- due to too strong a reliance on the calculated alpha
- The above can conspire to select the stocks which are
under-performers
- of the top decile, killing your IR
---------------------------------------------
- Remember, just because your optimizer gives you a solution, it
may not be a solution thats actually on the efficient Frontier (
most optimizers in practice will ALWAYS return a portfolio ).You
have to look to see if its a feasible solution and an efficient
portfolio!
23. Black Swan Mitigation, is it Possible??
-
- Usually We Employ the Following:
-
- More/Better Factors ( reduce each model to its simplest set and
no simpler )
-
- Tighter or Looser Tolerance to Bench in Optimization or
Portfolio Construction
24. Common Mistakes in ApplyingStatistics -II Out-of-Sample
Testing is Over-Rated
-
- Sample splittingmust occur, which results in aloss of
informationandlower predictive powerdue tosmaller sample sizesand
simultaneously,may fail to detect predictabilitythat
thein-sampletest would havefound .
-
- The claim thatout-of-sample testingisvoid of data miningis
factually bankrupt becauseout-of-sample testingis mostly used in
across validation construct .
-
-
- The researcher keeps iterating over different models with the
in-sample data, until they obtain a preferred out-of-sample
result.
-
- Often, there are periods of time during thein-samplewhere
performance of the model would even be worse than
theout-of-sampletime period .
-
- Lastly, out-of-sample testing usually leads to over/under
confidence.
-
-
- if a model out-performs above expectation during the
out-of-sample test period one typically would over-estimate its
capability and if it under-performs one rejects the model, or at
the very least, change it to get better performance in the
out-of-sample time period (i.e. data mining).
-
- Theexpert literatureon the subjectwould contendthere isno
evidencethat model distortions due todata miningare more prevalent
for in-sample tests than out-of-sample tests[below]and conclude
thatresultsofin-sample testsof predictability will typically be
morecrediblethan results ofout-of-sampletests.
-
- IN-SAMPLE OR OUT-OF-SAMPLE TESTS OF PREDICTABILITY: WHICH ONE
SHOULD WE USE? European Central Bank Working Paper Series No. 195;
by Atsushi Inoue & Lutz Kilian; November 2002
25. Black Swan Storms Cont
-
- One Cannot Predict Extreme Events .Best you can do is
familiarize yourself with what your model did during the last Black
Swan event and create disaster recovery strategies
-
- More factors only serves to model normal events more
closely
-
- Tighter/Looser Benchmark Constraints will move your portfolio
proportionally with the index during the next Black Swan Storm
-
- The past is not prologue, chaotic nature of markets means
onlytrendsarelikely calculable , accurate market trajectory isnt a
viable objective and should be avoided as a goal in modeling.
-
- Individual stock returns are not predicted at all by empirical
models, onlygroups of stocks most likely direction is.In a Black
Swan Storm, the fundamental data used to create your model,
disconnects from stocks for a time so that a model isuseless until
stocks return to reflect financial statement causes .True of risk
models too!
-
- This is the difference between stocks and companies that we
dont learn until years in the market.Stocks are not companies but
have other phenomena influencing their movements.Accounting data is
not comprehensive.At times OTHER DATA is dominant.
26. Hints on a Models Usefulness.
-
- Trust the models trends & direction, not its precise
levels
-
-
- when ranking 2,000+ stocks, a model has a hard time
discriminating between a stock ranked 42 and a stock ranked 63(
i.e. stock alphas of 2.5 +/-12 and 2.4 +/-12)
-
- Emphasize std-dev of return reduction rather than increasing
average decile return ( see next slide. )
-
- If you can utilize a plurality of models that are lowly
correlated, do so..
-
- Goods stocks and good companies can be mutually exclusive.Act
this way, dont just talk a talk (model regresses returns not
balance sheets)
-
- Understand a trading position can be just as profitable as an
investing situation, different model factors operate at different
frequencies
-
- Marrying bottoms up with top down is easy
-
-
- The stock alpha model selects the stocks, the weighting or
portfolio leaning can be driven by country, sector or industry
models
-
- Models do not model extreme Black Swan events.
-
-
- Try to keep risk management separated from alpha
predictionUNTILyou put the portfolio together
-
-
- Keep your expectations from models/technology low
27. Lowering the Std-Dev of Returns can be a More Fruitful
Research Objective then Raising the Average Return of a decile.
Source: Allegiant proprietary information Impact Of Lower Deviation
of Returns. Our Customers feel the Variance, Not the Mean .. 28.
Hints on a Models Usefulness.
-
- Maintain a healthy dose of mistrust of a model, rather than a
mistrusting of your quants...
-
- Avoid extrapolating, but include interpolating.....
-
- Not checking other methods of model construction, can lead to
blaming the methodology for results (good or bad)
-
- As above, but your fundamentally trained PMs will blame the
model constructor, not the methodology.CIOs please be aware
-
- Differentiating luck from skill need another full seminar to
discuss this topic, but its important!!!
-
-
- Keep your expectations from models/technology low
29. Common-Sense Additionsto Quant Strategies
-
-
- demand for energy.. Chin-Dia: buy Oil stocks, E&P,
commodities
-
-
- falling dollar, rising world economies: buy Large Cap Stocks
where the companies get a larger percentage of revenue in FX
-
- Maximize the serendipity around you
-
-
- buy more stocks than are necessary for diversification
alone
-
- Invest in preparedness not in prediction
-
-
- think about the worst case scenario, than develop plans for
what to do
-
-
- (SIPDE: scan, interpret, predict, decide, execute)
-
- Put yourself in situations where favorable consequences are
much larger than unfavorable ones
-
-
- if institutional buying is moving strongly into materials, dont
bet
30.
-
- The views expressed in this investment report represent the
opinions of the author and are not necessarily the views of
Allegiant Asset Management Company (Allegiant) or any affiliate.
They are not intended to predict or depict performance of any
investment. All information contained herein is for informational
purposes and should not be construed as investment advice. It does
not constitute an offer, solicitation or recommendation to purchase
any security. The information herein was obtained by various
sources; we do not guarantee its accuracy or completeness.Past
performance does not guarantee future results .These views are as
of the date of this publication and are subject to change based on
subsequent developments. Allegiant is SEC-registered investment
advisor and a subsidiary of National City Corporation.
Required Financial Homily.. Steven P. Greiner, Ph.D. Allegiant
Asset Mgmt Group One North Franklin; Ste 750 Chicago, IL 60606
(312) 384 8254 [email_address] Source: Allegiant proprietary
information