How "Informative" Is theInformation Ratio forEvaluating Mutual FundManagers?THOMAS BOSSERT, ROLAND FUSS, PHILIPP RINDLER,

AND CHRISTOPH SCHNEIDER

THOMAS BOSSERTis managing director(portfolio management) atUnion Investment Institu-tional GmbH in Frankfurt,Germany.

thomas.bossert @union-

mtestment.de

ROLAND FUSSis a professor of finance andholds the Union InvestmentChair of Asset Managementat European BusinessSchool (EBS), InternationalUniversity SchlossReichartshausen inOestrich-Winkcl. Germany.roland.fuess@ebs.edu

PHILIPP R I N D L E R

is a research assistant at theUnion Investment Chair ofAsset Management atEuropean Business School(EBS), hiternadonalUniversity SchlossReichartshausen inOesrrich-Winkel, Ciermany.ph¡lipp,rín(ller@ ebs.edu

C H R I S T O P H S C H N E I D E R

IS an analyst at MorganStanley, Investment BankingDivision in Frankñirt,Germany,christopli..schneider@ ebs.edu

Treynor and Black's [1973] Infor-mation Ratio (IR) is one of themost commonly used performancemeasures. It represents the ratio of

the excess portfolio return over a specifiedbenchmark, as well as excess return volatility.

Closely connected is the fundamental!aw of active portfolio management (Grinold[1989]), which relates a fund manager's skillsto the IR. This framework gives insights intohow to use the IR to construct active portfo-lios within predefined risk limits. In order forinvestors to apply the ratio to a specific port-folio choice problem, however, they needguidelines to identify superior funds.

Grinold and Kahn [2000] state that top-quartile managers have IRs of at least 0.5, whileexceptional managers achieve values above 1.0.These numbers are unqualified and shouldhold irrespective of asset class, country, or timeperiod. To the best of our knowledge, IR char-acteristics across difierent asset classes and coun-tries have not been extensively studied yet.Hence, this article addresses whether the IR isa useful and reliable performance ratio. Itfocuses particularly on empirically observablequartile ranges for various asset classes andcountries that investors can use as guidelinesto determine fund quality.

We use return data fi-om nearly 10,000mutual funds for the January 1998-December2008 period. The empirical results show thatstatic breakpoints can be widely niisleading

and that a focused asset class approach is nec-essary. Moreover, the quality and reliability ofthe IR depends on certain estimation choices.

First, benchmark choice strongly affectsthe ratio. Ideally, the benchmark should covera large proportion of the respective investmentuniverse. Second, data frequency should be asgreat as possible, because monthly data do notaccurately represent return volatility. Third,non-normally distributed fund returns cansubstantially affect the use of the IR. Finally,in order to separate lucky managers fromskilled ones, long-term track record can be animportant measure.

The remainder of this article is orga-nized as foUows, The next section, "The Infor-mation Ratio," discusses the IR and its rolewithin active portfolio management. Thesection after that, "Data Description," presentsour dataset and explains our choice of fundsand benchmarks. The empirical results are pre-sented next, in "Is the Information Ratio aReliable Performance Measure?", whichbegins by testing the IR for stability over timeand across different fund categories. We thendiscuss the robustness of the IR against theselection of different benchmarks and data fre-quencies. Finally, we examine the persistenceof IRs over time in order to separate lucky-managers from skilled ones. The final sectionprovides conclusions and su^esdons for futureresearch.

SPRING 2010 THEJOURNAL OF INVESTING 67

THE INFORMATION RATIO

Treynor [1965] defines two characteristics of a"good" performance measure. First, it should provide thesame value for the same performance, irrespective of marketconditions. Second, it needs to incorporate the preferencesand risk aversion of investors. Similarly, Hübner [2007]states that there are two factors that determine the qualityof a performance measure: stability and precision. A stablemeasure is robust under different asset pricing models, anddoes not vary over time in terms of its classification. Pre-cision means that it should be able to provide the "true"ranking of funds based on investor preferences.

Treynor and Black [1973] define the IR as:

IR = -^ER

o.(1)

where r is the portfolio return, r^ is the benchmark return,ER is the excess remrn, and O^^ is the volatility of the excessreturn. The rationale for the IR LS closely related to Jacobsand Levy's [1996] investor utility function. They explaintliat investors in active fiands are not risk-averse, but ratherregret-averse. Regret aversion means generally acceptingthe risk of a passive investment in this asset class, but—depending on the excess returns—regretting the decisionto invest in an active fund.

Similarly, Grinold and Kahn [2000] find that investorsselect among different opportunities based on their per-sonal preferences, \vhich, for actively managed funds,"point toward high residual return and low residual risk"(p. 5). Thus, by using the IR, investors can limit the funduniverse according to their personal risk preferences.

Grinold [1989] identifies two factors that lead tohigh IRs. The first is the manager's ability to correctlypredict residual returns in his invesmient universe. Referredto as the Information Coefficient (IC), it measures thecorrelation between actual and forecasted alpha. Thesecond, which describes the number of independentinvestment decisions made per year, is called breadth. Thefundamental law of active management illustrates the rela-tionship between IR, IC, and breadth, as follows:

IR = IC (2)

For purposes of this article, the crucial point aboutEquation (2) is that correct forecasting of residual returnsshould be a key skill of any active portfolio manager.

Depending on the number of independent bets a man-ager takes, different skill levels are required in order toachieve a "good" or "very good" IR (Wander [2003]).

As we noted earlier, Grinold and Kahn [2000] defineIR levels on the basis of cost-adjusted fund performance.A top-quartile portfolio manager would have an IR of 0.5,and an exceptional manager would achieve a 1.0 or above.Again, according to this study, this classification should holdfor all asset classes and rime horizons, with only slight devi-adons. Jacobs and Levy [1996] also found an IR of 0.5 orabove to be "very good," without restrictions to asset classes.

Goodwin [1998], on the other hand, analyzed theIR distribution for samples of funds with different invest-ment universes, and found significantly different resultsacross fund categories. We believe this approach is moreplausible than the findings of the two other studies. Thus,we expect to find different IR ranges in our empiricalanalysis when evaluating funds that invest in different assetclasses and countries.

DATA DESCRIPTION

Fund Data

Our initial sample includes all actively managed, open-end funds listed for sale in Germany, the U.K., and theUnited States by Reuters 3000 Xtra as of February 2009.Closed-end funds are excluded, because investors cannotfreely enter or exit them. REITs and hedge funds are alsoexcluded, because their particular characteristics deniandspecific performance measures that are not within the scopeof this article (see, for example, Ackermann et al. [1999] orBelow and Stansell [2003]). We focus separately on equity,fixed-income, and money-market funds because of theirdiffering risk-return characteristics. Furthermore, weexclude balanced funds, because we aim to analyze andcharacterize performance measures of distinct asset classes.To categorize the funds, we use the Lipper Global Classi-fication as it is used throughout Reuters 3000 Xtra.

In the equity class, we choose funds with a focus onthe major equity markets: Europe, Germany, the U.K.,and the United States. We also distinguish between large-and small-cap funds {although the limited number ofsmall-cap equity funds in Germany resulted in the elim-ination of this category).

In the fixed-income class, we choose corporateinvestment-grade bond flinds with a focus on the Britishpound, the euro, and the U.S. dollar, which are the threemajor currencies for corporate bond emissions according

68 H o * "iNRmMATIVE" Is THE INFOKMATION P-ATIO FOR EVALUATING MUTUAL FUNU MANAGERS? SPRING 2010

Co Reuters 3000 Xtra. Finally, we use these same threemajor currencies to select relevant funds in the moneymarket class. Our final iund sample comisted of 9,632 funds.'

Our time frame ranges from January 1, 1998through December 31, 2008. Our weekly return data,launch years, and base currencies for the funds comefrom Thomson Financial DataStream. We correct forerroneous data entries by excluding funds with extremeinformation ratios of above 20 or below —20. We alsoexclude funds with launch dates after January 1,2007.If a fund launched in the second half of a year, we setthe launch date to the next year in order to ensure a suf-ficient number of data points per calendar year for cal-culating test statistics. For funds quoted in a currencyother than the corresponding benchmark currency., weconvert the return data using the appropriate exchangerate. Additionally, we retrieve daily and monthly returndata for large-cap U.S. equity funds in our given time-frame in order to analyze the influence of data frequency.

However, because Reuten 3000 Xtra and ThomsonFinancial DataStream only list funds that are currentlyavailable on the market, the data are subject to survivor-ship bias. For us, this is especially relevant prior to 2007,because only the funds that survived are contained in ourdataset. We posit that the estimated performance measuresmay be biased upward. In the "Other Influences onPerformance Measures" section, we analyze the extent ofand possible corrections for survivorship bias.

Calculating the IR requires a market benchmark forcomparison. Fund managers normally define their bench-marks in a prospectus. However, in light of the largenumber of funds and corresponding benchmarks withinthe same fund category, it was not possible to use eachbenchmark to calculate the performance measures. Instead,we use a general benchmark for each fund category.

Initially, this may seem somewhat unfair. But webelieve it is more logical to judge each fund within a cer-tain investment universe against the same benchmark,although it does introduce a bias into the analysis. Fundmanagers that are actually managing their funds against thebenchmark tend to exhibit lower tracking errors, andtherefore higher IRs, than managers using another bench-mark. They bear the tracking errors versus their truebenchmark plus the tracking errors between the true andchosen benchmarks. Exhibit A2 in the Appendix providesan overview of the benchmarks assigned to the differentfund classes.

Descriptive Statistics

Exhibit 1 gives the descriptive statistics for each fundcategory as well as the average excess return over therespective benchmark. All numbers are annualized forbetter comparability.

In terms of risk/return relationships, we see that money-market and corporate bond funds behave as expected. But

E X H I B I T 1

Descriptive Statistics of Fund Returns

Fund Classification

Equity EuropeEquity GermanyEquity U.K.Equity U.S.Equity Small Cap EuropeEquity Small Cap U.K.Equity Small Cap U.S.Corporate Bonds EURCorporate Bonds GBPCorporate Bonds USDMoney Market EURMoney Market GBPMoney Market USD

Avg. Ann.Return

-0.72%0.18%1.97%

-2.57%1.5!%4.09%

-2.54%2.38%3.65%3.10%2.11%4.97%1.97%

Avg. Ann.Std. Dev.

17.73%23.42%15.30%18.23%19.25%14.02%21.68%

2.88%4.39%4.22%0.30%0.45%3.12%

Skewness

-0.539-0.418-0.722-1.092-0.986-1.223-1.151-0.666-0.572-0.551-3.5574.1931.379

ExcessKurtosis

2.8853.4843.1679.7342.8163.0209.1193.9142.6621.710

20.30026.24533.221

Avg. Ann.Excess Return

-1.7!%-0.60%0.68%

-3.22%2.50%2.27%

-6.45%-1.20%-1.12%-1.58%-0.25%0.72%

-0.93%

Note: Calculations are based on weekly data for thefanuaq' i 998—December 2008 period and are annualized.

SPRING 2010 THE JOURNAL OF INVESTING 6 9

the numbers for the equity segment are surprising- The poorperformance of equities is due mainly to the impact of the2008 financial crisis; Gains fixim 2003 to 2007 in the US.equity market were completely erased in 2008.

In terms of performance as measured by alpha, it isclear that, over the 11-year period, managers in almost allasset classes and fund categories were not able to beat thebenchmark on average after costs. Note also that themoney-market segment exhibits strong skewness and lep-tokurtosis. We will analyze the effects of non-normallydi.stributed returns on performance measures flirther in the"Other Influences on Performance Measures" section.

IS THE INFORMATION RATIO A RELIABLEPERFORMANCE MEASURE?

The Distribution of the Information Ratio

To analyze whether the distribution of [Rs is stableover time and across different fiand categories, we rankthe ratios for each year and asset class, and then dividethem into four quartiles. We use a Wilcoxon signed-ranktest and an optional student r-test to test the yearly valuesagainst the overall average for statistically significantdifferences. We present all results in annualized form forbetter readability and comparability by using arithmeticmean returns according to Goodwin's [ 1998] method 1 .'

Exhibit 2 presents the threshold values for the fourquartiles, which are averages over the 11-year horizon otthe dauset. Note that the IRs exhibit very different patterns

for each fund category, not just in terms of value but alsoin terms of range. A corporate bond fund with a positiveIR can usually be classified as "very good," while an EquityEurope Fund would only be average. Additionally, thevalue range for a "good" Equity Europe fund is far nar-rower than for a "good" Money Market EUR fund.

Nevertheless, the values and ranges within the a.ssetclasses seem similar. Further testing needs to be done toconfirm these results. But we find that general statementsabout the IR, such as those of Grinold and Kahn [2000](discussed earlier), are not applicable for all asset classes andyears because the threshold values vary considerably overtime. Exhibit 3 shows detailed information about how thethreshold values develop over time for the top quartiles.

But are these strong IR fluctuations statistically sig-nificant? To test for this difference, we calculate the median[R of the top half of all Equity US. funds for each of the11 years, i.e., the threshold value between the first 25%and the second 25% of the funds. We then test this valueeach year to see if it is statistically significantly differentfrom the average threshold value reported in Exhibit 2.

The results are outlined in Exhibit 4, with thethreshold values in the first data row and the z-statisticsin the second row. We again use the Wilcoxon signed-ranktest because the IRs are not normally distributed accordingto the Lilliefors test, and we assume they are dependenton each other (see Hollander and Wolfe [1973]).

The results in Exhibit 4 clearly show that thethreshold values are significantly different from the11-year average in every year. A look at the z-statistics

E X H I B I T 2Information Ratios of Different Fund Categories

Fund Classification

Equity EuropeEquity GermanyEquity U.K.Equity U.S.Equity Small Cap EuropeEquity Small Cap U.K.Equity Small Cap U.S.Corporate Bonds EURCorporate Bonds GBPCorporate Bonds USDMoney Market EURMoney Market GBPMoney Market USD

IR 1st 25%«Very Good"

>0.40>0.07>0.32>0.28>0.80>0.59>0.08

>-0.24>0.03>0.03>4.30>4.30>2.46

IR 2nd 25%«Good"

0.40 to 0.040.07 to-0.110.32 to-0.010.28 to -0.400.80 to 0.400.59 to 0.220.08 to -0.60

-0.24 to -0.760.03 to -0.460.03 to -0.584.30 to 1.364.30 to 0.312.46 to 0.39

IR 3rd 25%"Below Avg."

0.03 to -0.36-0.12 to 0.37-0.02 to -0.30-0.41 to-I.Ol0.29 to -0.090.21 to-0.12

-0.61 to-1.18-0.77 to-1.30-0.47 to -0.95-0.59 to-1.291.35 to-0.390.30 to-1.500.38 to-1.29

IR 4th 25%"Poor"

<-0.36<-0.37<-0.30<-1.01<-0.09<-0.12<-I.18<-1.30<-0.95<-1.29<-0.39<-1.50<-1.29

70 How ••INK>HM.\TIVE" IS THE RATK) FOR EVALUATING MUTUAL FUND MANAGERS? SPRING 2010

E X H I B I T 3

Information Ratio—Threshold Values for 1st Quartile Funds (very good)

Fund Classificatioii

Equity EuropeEquity GennanyEquity U.iCEquity U.S.

Equity Small Cap EuropeEquity Small Cap U.K.Equity Small Cap U.S.

Corporate Bonds EURCorporate Bonds GBPCorporate Bonds USD

Money Market EURMoney Market GBPMoney Market USD

1998

>0.23>0.02>-0.26>-0.39

>0.04>-0.92

>0.65

N/A>0.56>-0.37

N/A>0.33>I.8O

1999

>Î.3O>-0.16>0.79>0.36

>2.40

>2.50>1.50

N/A>-0.04>0.26

N/A>1.10>1.50

2000

>0.38>0.44>0.62>0.66

>0.60>0.67>-0.26

N/A>-0.64>0.46

N/A>1.20

>1.40

2001

>0.I5>0.2I>0.24

>0.51

>-0.21>0.19>-0.21

N/A>-0.50>-0.64

N/A>4.60>5.90

2002

>0.28>0.35>0.15>0.71

>0.50

>0.25>0.06

>0.08>-0.19>-0.28

>4.60>5.90>3.00

2003

>-0.12>0.I2

>0.65>0.36

>I.4O>1.30>0.44

>-0.30>-0.17>-0.01

>7.70>5.60>5.20

2004

>0.46>-0.14>0.44

>0.18

>1.70>1.40>-0.74

>-0.95>0.07>-0.26

>7.40>3.70>2.70

2005

>0.91>-0.02>0.31

>0.55

>1.60>0.47>-0.05

>-0.32>-0.15>-0.08

>7.80> 10.00

>0.98

2006

>0.81>0.08>0.58>-0.38

>1.60>1.50>-0.49

>0.26>-0.08>0.00

>4.20>7.90>2.10

2007

>-0.09>-0.22

>-0.23>0.44

>-0.28

>-0.68>0.49

>0.63>0.30>0.49

>0.51

>3.80>1.30

2008

>0.08>0.11>0.I8

>0.08

>-0.49>-0.18>-0.52

>-1.10>1.20>0.71

>0.04>3.20>1.20

E X H I B I T 4

Test Statistics for the Difference of Threshold Values of Equity U.S. Funds

1998-0.39*-16.5

19990.36*

-3.0

Wilcoxon

20000.66*

-15.4

20010.5 r

-12.5

Signed-RankTest on Differences in Mean2002

' 0.71*-19.8

2003

0.36*-9.0

2004

0.18*-3.2

2005

0.55*-20.7

2006 2007-0.38* 0.44*

-34.3 -15.4

20080.08*

-4.0

Avg.0.28

Note: ^Denotes values significantly different from average at the 5% significance level. AU test statistics for the Ulliefors test for normality are significant at the5% level. The test is a generalization of the Kolmogorov-Smirnov (KS) test, which requires specification of the population mean and mriatue. The Ulliefors testis capable of testing samples for normality thai hatv incompletely specified distribution characteristics (Ulliefors f1967}).

reveals that the values are statistically different from theiraverage. This is also highlighted by the spread in thresholdvalues, from -0.39 in 1998 to 0.71 in 2002. Thus, a fundevaluated on the yearly threshold value could be catego-rized as "below average " while simultaneously being ratedas "very good" on the overall average value.

To conclude, we believe IRs must be calculated aneweach year in order to be reliable. Because the relevantthresholds can only be calculated ex post, it is not possibleto use IRs when setting annual targets for fund managers.However, in the context of a multi-year planning process,long-term IRs might be applicable.

In the next step, we studied IRs across differentfund categories. Our focus was again on U.S. funds, as itseems more likely that we will find similar IRs whenlooking at several asset classes within one country than

across different countries. The procedure is exactly thesame as in the previous test on Equity U.S. funds; resultsare in Exhibit 5.

Similar to the results in Exhibit 4, Exhibit 5 showsthat all threshold values are significantly different fromtheir averages. This statement is valid for 1998 and for2008, so we consider it rather robust. IRs thus not onlychange over time, but also between different fund cate-gories. And we cannot confirm the general statementsabout fixed threshold values found in Grinold and Kahn[2000] or Jacobs and Levy [1996].

The results of this part of our empirical study aresimilar to the results of Goodwin [1998]. with the addi-tion that IRs also change over time. Exhibit 6 uses box-and-whiskers plots to graphically illustrate the differentdistributions of IRs over time for Equity U.S. funds.

SPRING 2010 THE JOURNAL OF INVESTING 7 1

E X H I B I T 5Test Statistics for the Difference of Threshold Values of U.S. Funds

Wilcoxon

z-score

Wilcoxon

z-score

Year Equity-0.39*

-17.53

0.08*

-10.10

Small Cap Equity0.65*

-6.39

-0.52*

-26.66

Fixed-Income-0.37*

-5.51

0.71*

-5.17

Money Market1.80*

-3.82

1.20*

-4.04

Average0.42

-

0.37

-

Note: *Denotes values significantly different from average at the 5% significance level. Similarly to the previous test, the IRs are not normally distributedaccording io the LilUefors test, and all test statistics are significant at the 5% level. Tlw second row gives ¡he z-scores of the Wilcoxon signed rank test.

E X H I B I T 6Box Plots of Equity U.S. Fund Information Ratios

Equity US Funds

T T T t i

1t t -

2ooe

The Art of Selecting a Benchmark

In fund management companies, benchmark selec-tion is usually the result of intense negotiations betweenthe fund manager and the investors, because it has a majorimpact on the fund's alpha. Depending on the style andcountry focus of a fund, one benchmark might be muchmore favorable to a flind manager than another (Goodwin[1998], Grinold and Kahn [2000]). Therefore, it is impor-tant to analyze the sensitivity of the IR toward the selectedbenchmark.

Lehmann and Modest [1987] show that benchmarkselection strongly influences the resulting alphas as wellas their volatility. Thus far, we have used the S&P 500throughout this article in connection with Equity U.S.

funds. But we will use two additional indices to comparethe resulting IRs, the equally weighted Dow Jones Indus-trial Average (DJIA) and the market-weighted Russell1000 Index. Exhibit 7 presents the threshold IRs for dif-ferent benchmarks using the same procedure as in theprevious section.

Note that the IRs based on the S&P 500 and theRussell 1000 are closely related, while the IRs based onthe DJIA behave differently and are far more volatile.It appears that the DJIA does not cover the Equity U.S.investment universe very well. This may be because thisindex is based on only 30 stocks.

We again use the Wilcoxon signed-rank test to testfor significance of the difFerence in threshold values.Exhibit 8 gives the results from the Russell 1000 and the

7 2 How "INFORMATIVE" IS THE 1NFORJ«IATION P ^ T I O FOR EVALUATING MUTUAL FUND MANAGERS? SPRING 2010

E X H I B I T 7The Effect of Benchmark Selection on the Information Ratio

- Dow Jones Industrial AverageS&P 600Russell 1000

2005 2006 2007 2006

E X H I B I T 8z-Statistlcs for Significant Difference of the Infonnation Ratios

z-Values for 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008D o w Jones -18.1* -9.6* -9.3* -22.0* -26.6* -26.4* -30.9* -32.9* -7.5* -8.6* -25.3*Russell 1000 -9.4* -4.2* -3.5* -14.2* -8.5* -17.1* -25.0* -12.7* -31.9* -21.1* -33.5*

Note: *Demtes values significantly different from average at the 5% significance level. Similarly to the previous test, the IRs are not normally distributedaccording to the LiUiefors test, and all test statistics are significant at the 5% level.

DJIA versus those from the S&P 500. All are significantlydifferent from those based on the S&P 500 (at a 5% sig-nificance level). These results are in line with Goodwin[1998], who also found that benchmark selection stronglyinfluences the resulting IRs.

The scatter plots in Exhibit 9 illustrate the rankingsbased on the three different IRs. We can see that the resultsare confirmed: There are noticeable differences betweenIRs based on the DJIA and those based on the S&P 500,while the changes in ranking between the Russell 1000and the S&P 500 are quite small. Selecting an appropriatebenchmark is therefore an important step during perfor-mance analyses in general.

However, we conclude that benchmark indices thatcapture a larger part of the investment univeree of a specific

fund category are superior to those based solely on a fewsecurities and industry sectors. Finally, the best way tojudge the real risk-adjusted value-added of a fund man-ager is to consider his actual benchmark, as well as thetrue benchmarks of the other managers within his peergroup. Or, as an alternative, we could use the peer group saverage as a general benchmark, which might lead to morestabiUty in the annual IR thresholds.

In the same sense that benchmark selection is socritical, investment restrictions are also quite important.The Transfer Coefficient (TC) measures the correlationof a manager's forecasts with the actual portfolio. A man-ager without constraints will end up with a TC of 1, whilethe constrained manager can only achieve a lower result.Furthermore, a typical long-oniy fund may achieve a TC

SPEUNG 2010 THE JOURNAL OF INVESTING 7 3

E X H I B I T 9Ranking Differences Caused by Different Benchmarks

200 doo eoo BOORank by Ir tornist ion Ral io (S&P 600)

200 400 600 eoo 1000Rank by mtoimalion ñatn (S&P 600)

between Ü.2 and (.).4, which would significantly impact theIR because it implies the manager is not able to fullytransfer his skills into actual investment decisions.

Assuming a (constrained) fund with a TC of 0.5, amanager must double his skill (IC), or quadruple hisbreadth, in order to achieve the same IR as an uncon-strained manager for an equal fund (Wander [2003]).

Does Data Frequency Matter?

Ané and Labidi [2íK)4] have shown that the returninterval has a significant impact on the return distribution.Monthly and quarterly returns come close to a normal dis-tribution, but weekly and especially daily data usuallyshow strong leptokurtosis. Furthermore, the annualizedstandard deviation varies with frequency.

Other research has also found that data frequencyinfluences correlations (Handa et al. [1989]). But we needto determine whether data frequency also impacts the IRand, in particular, the threshold values.

If we do fmd significant influence, we aim todescribe these differences and to provide guidance torselecting an appropriate return interval. Thus, we calcu-late annualized IRs for Equity U.S. funds using daily,weekly, and monthly fund returns. Using the rankingmethodology explained earlier, we create fund rankingsbased on three different IRs.

Results for the year 1999 are given in Exhibit 10.We see that the rankings of IRs based on daily and weeklydata do not differ significantly, but switching to monthlydata changes the ranking dramatically. We conclude that

monthly data are inappropriate for calculating reliableperformance measures. Monthly data also allow for only12 data points per year, which is insufficient to estimatethe standard deviation.

Other Influences on Performance Measures

Many studies have documented that returns are gen-erally non-normal. However, many popular performancemeasures are still based on mean-variance analysis. There-fore, as per Benson et al. [2008], we find that non-normalreturns lead to biased results.

Additionally, Kraus and Litzenberger [1976] foundthat positively skewed returns are actually favorable forinvestors. If we refer back to Exhibit 1, which presentsdescriptive statistics of fund returns for each category, itis striking that money-market funds in all currencies pro-duce strongly skewed and leptokurtotic returns. Com-paring the threshold values in Exhibit 2, the values for"top" money-market funds are uncharacteristically high.Thus, we conclude that common performance measuresare not applicable to these fiinds, probably because of theirspecial return distribution characteristics. Other perfor-mance measures, such as Keating and Shadwick's [2002]Omega Measure, could capture the deviation fromnormality.

Another important factor is the survivorship bias thatcan result because the most common data providers onlylist currendy available funds. This may lead to an upwardbias in the performance measure estimates. Brown et al.[1992] found that survivorship bias can be so strong that

74 How "INRIRMATIVE" IS IHE INR)RMATION RATIO FOR EVALUATING MUTU.^L FUND MANAGER.S? SPRING 2010

E X H I B I T 1 0Comparison of Rankings Based on Different Data Frequencies

200 400 600 600 1000 1200Rar* by Infomi^ion Haio (Ftequency: Vi&l

200 '100 SOG 800 !000REV* by trtomialKin R^io (Frequency: Weekly)

it can lead to the erroneous conclusion that mutual fundperformance is predictable. However, when we correct thesample for survivorship bias, this finding disappears.

In terms of quantifying the survivorship bias forEquity U.S. funds, Grinblatt and Titman [1989] found aO.iyo-0.3% bias per year. Brown et al. [1995] estimated thebias at between 0.2% and 0.8% per year, while Elton et al.[1996] posited an average bias of O.7r/o-O.77% annually.Although it is very likely that survivorship exists in ourdataset, we were unable to quantify its proportions. How-ever, because our study is set up similarly to the previouslycited studies, we believe our results would be subject toa similar order-of-magnitude of bias.

Costs and asynchronous pricing are two other fac-tors that influence our results. In order to estimate thereal risk-adjusted value-added of a fund manager, we needto compare his results net of fees with those of other fundmanagers. However, we also need another dataset to com-pute those figures. And fijnd investors may be more inter-ested in the final result than in whether the results weredue to skill or the cost structure of the product.

The asynchronous pricing introduces an upwardbias into the tracking error. For example, a fund that istracking its benchmark perfecdy, but whose NAV is notcalculated with the same security prices or foreignexchange rates as the benchmark, will inevitably exhibita tracking error that is different from zero. Again, quan-tifying this bias would require a dataset that draws heavilyon the internal valuation information of a fund manage-ment company, w hich is difficult to coine by.

Eliminating survivorship bias may lead to lower realaverage IRs, but costs and asynchronous pricing have anegative impact. Including these two factors will lead tosomewhat higher average IRs.

Performance Persistence: Outperformanceby Luck or by Skill?

Finally, we question whether a single ratio based onone year of data is sufficient to evaluate a manager's per-formance. It is important to determine whether achievinga high IR in any given year was attributable to luck oractual skill.

Here, the managers track record can be a usefU tool.The probability that good performance is attributable toskill increases when managers can position their tundsamong the top 25% for two or three years in a row (Bollenand Busse [2005]). However, care should be taken whentrying to predict fiiture fijnd returns based on past returns.

Horst and Verbeek [2000] show that some studiesclaiming to find performance persistence are likelyreporting spurious and biased results. Kahn and Rudd[1995] and Carhart [1997] also analyzed persistence ofequity mutual funds, and did not find a significant rela-tionship betw^een past and future performance.

We fmd similar results with our dataset. We catego-rize Equity U.S. funds launched in 1998 or before intoquartiles based on their 1998 IR. For each quartile, we cal-culate the average IRs for each year. The results are shownin Exhibit 11.

Si'KING 2010 THEJOURNAL OF INVESTING 7 5

E X H I B I T 1 1Performance Persistence of Equity U.S. Funds

4th Quartile

3rd Quartile

2ncl Quaiiile

1st Quartile

-2.61998 1999 2006 2007 2008

Note that the top-quartile funds of 1998 actuallyhave the lowest average IR afiier two years. The chart sug-gests a mean-reverting process and shows that, on average,good performance does not persist. Based on this fact, weconclude that lucky managers without skill are not likelyto remain among the best funds for multiple years in a row.

We therefore propose track record as a seconddimension to evaluate manager performance. First, wetrack the performance of fiinds from selected categoriesthat launched in 1998 or earlier and survived until 2008over the entire 11-year period. Then we calculate thenumber of years that each flmd ranked in the top 25%^within this period. Exhibit 12 presents our summarizedresults, which are as expected.

Note that, during the 11-year period, 95.5% of allEquity U.S. funds and 93.4% of all Equity Small Cap U.S.funds were in the top 25% at least once. Hence, if a fundsurvives for 11 years, it is likely to be in the top quartilein some years just by luck. However, it is important tonote that these results could be caused at least partly bychanges in fund management.•*

Taking the results from Exhibit 12 one step further,we calculate how many fiands remain in the top quartilefor two or three years in a row within a three-year periodbased on their IRs. According to Exhibit 13, such fundswould be considered extraordinary, since, on average, only2.76% of all funds have managed this achievement.We calculate the percentage of all fiinds that remain in

the top quartile for two or three years in a row for rollingthree-year periods, and the results are fairly stable andconsistent.

We interpret Exhibit 13 as follows. When lookingat the first row, for the 2008-2006 period, 0.93% of allEquity U.S. funds were in the top 25% of the funds in allthree years. For the 2008-2007 period, 2.33% of all EquityU.S. funds were in the top 25%, and for 2008, the numberwas 21.73%. These three values add to 25% with onlyminor rounding differences.

We perform the same calculation using the top 50%of funds. However, we conclude that the top 50% isachieved too easily, and is therefore not an appropriatemeasure.^ Based on these results, performance persistence(the track record of a manager) is another important factorin separating luck from skill in performance measure-ment. Investors should thus seek a fund manager with aconsistent series of performance measures, rather thanwhat could be unrelated episodes of good performanceover longer timeframes.

Agency Problems

The agency problem can be illustrated as follows.Consider a portfoho manager who makes just one activeinvestment decision per year, and whose correlationbetween forecasted and actual returns is 0.1. According tothe fundamental law of active management, this manager

7 6 How "INFORMATIVE" IS THE INFORMATION RAno FOR EVALUATING MUTUAL FUND MANAGERS? SPRING 2010

E X H I B I T 1 2Number of Top 25% Rankings Over Lifetime

Fund ClassificstionEquity U.S.Equity Small Cap U.

E X H I B I T 1 3

04.5%

S. 6.6%

112.8%11.0%

221.4%21.7%

Perfonnance Persistence of Equity U.S. Funds Over Time

Period

2008 to 2006

2007 to 2005

2006 to 2004

2005 to 2003

2004 to 2002

2003 to 2001

2002 to 2000

2001 to 1999

2000 to 1998

Mean

Top 25%

lYear

21.73%

20.41%

18.77%

16.76%

14.80%

19.76%

12.10%

11.11%

23.35%

17.64%

... Years i

2 Years

2.33%

2.88%

3.04%

4.07%

7.57%

2.33%

5.15%

13.17%

0.83%

4.60%

in a Row

3 Years

0.93%

1.72%

3.18%

4.15%

2.65%

2.87%

1.11%

0.72%

0.83%

2.76%

3 424.9% 17.7%24.4% 18.3%

Top 50%

lYear

25.73%

28.22%

26.48%

20.87%

18.20%

25.49%

13.22%

10.66%

34.09%

22.55%

5\1A%9.3%

... Years

2 Years

11.58%

7.81%

5.51%

9.32%

14.23%

6.51%

6.87%

29.66%

5.79%

10.81%

6 or more6.3%8.7%

in a Row

3 Years

12.67%

13.94%

17.99%

19.81%

17.54%

17.97%

29.87%

9.68%

10.12%

16.62%

\vill achieve an IR of 0.1. The empirical part of this articleshows that an IR of 0.1 for an Equity U.S. fund would inmost years be considered "good," and in some years even"very good," despite the fact that the manager may havedone very little. We believe the IR can potentially incen-tivize strategies that may he unfavorable to investors. Itseems that performance measures that use the trackingerror as a risk measure need a second dimension that cap-tures the active weights of the fund, such as the ActiveShare measure proposed by Cremers and Petajisto [2009].This measure is easy to calculate and can quantify theactive holdings of a mutual fund in relation to the corre-sponding benchmark.

CONCLUSION

Practical Implications

The aim of this article is to evaluate whether theIR is a useful and reliable measure of the performance of

mutual fund managers. Based on empirical evidence, wefind that the IR is in fact reliable and useful, but has certainlimitations. Overall, our analysis reveals that two dimen-sions are important to adequately judge manager perfor-mance in a given year: 1) the performance in that year,and 2) the track record of the fund over the previous threeyears. The former can be used to establish a ranking offunds that are then adjusted either upward or downwardby the latter.

In order to transform the IR into a grading system,we introduce a categorization of quartiles that definethresholds of fund qualities. IRs vary over time and alsoacross different fund categories, so it is necessary to cal-culate threshold values anew for every calendar year. Thismakes the IR a difficult choice when setting targets forportfolio managers, because they will not know how wellthey must perform until the end of the year.

We found that four factors influence the qualityof the IR: 1) benchmark selection, 2) data frequency.

SPRING 2010 THE JOURNAL OF INVESTING 7 7

E X H I B I T 14Framework for Performance Evaluation—Year 2008

-1.0 -0.5 0.0 0.5 1.0

Equity Euro

Equity Germa

Equity U.K.

orate Bonds GBP

Corporate Bonds USD

"below average" and "poor" "good" • "very good

1.5

3) non-normality of fund returns, and 4) any sur-vivorship bias inherent in the sample used to estiinatethe threshold values. Regarding the benchmark, werecommend selecting an index that captures a large partof the respective market. The data frequency should behigh—daily or weekly. Returns should also be testedfor normahty, as this influences the quality of the per-formance measures significantly. Finally, quantifying thesurvivorship bias within the IR is difficult and stillunclear. Thus, it is best left for iliture research. Note thatthe proposed framework is only valid for funds withsymmetric return profiles.

Exhibit 14 is an example of a pertorniance evalua-tion framework based on the IR that is calculated usingthe dataset of our empirical study. It is valid for funds ofthe selected categories in 2008 and can help estimate per-formance along the first dimension, the performance ofthe fund within a particular year. We make no differen-tiation between funds belonging to the third or fourthquartiles ("below average" or "poor" funds), because theirIRs are mostly negative and thereiore unreliable.

Further Research

While our results answer many of the research ques-tions, they also open up new issues. First, the returns arenot corrected for fees, so the performance is somewhat

biased. Performance should be (and in actualpractice is) measured using returns net offees. In fact, a significant part of the total feescannot be influenced by the portfolio man-agers, e.g., fund audit or custody fees.

Second, the sample is dominated byU.S. funds simply because of the dataproviders we used. Third, many funds aresubject to style drifts, which generally makereturns harder to compare (Chan et al.[20ü2]). Although we selected very broadfund categories, it would be interesting totest for biases caused by style drift.

Fourth, the sample is subject to sur-vivorship bias of up to 0.8% per year. Thisdistorts the performance measures calculatedon these returns (Brown et al. [1995]). Fifth,asynchronous pricing might result in trackingerror estimates with an upward bias, andtherefore to IRs that are lower than the realIRs.

In addition to our dataset, the analyses creates ideasfor additional research. For example, we would recom-mend comparing the results based on a generic bench-mark with results based on fund-specific benchmarks asdetermined by the portfolio manager. Alternatively, use ofthe peer group average as a benchmark might lead tomore stability in the wildly fluctuating [Rs.

Another suggestion is to analyze the IRs ot tundswith more specific style definitions, such as "U.S. valuestocks" or "European bank stocks." However, the numberof these funds is rather small, which may render the resultsinsignificant.

Finally, the effect of the Transfer Coefficient on amanager's active performance should be analyzed in moredetail. Fund managers face certain investment restrictionsthat prevent the allocation of funds to the best possibleportfolio. These restrictions will negatively affect the IR,although they are not influenced by the manager.According to Wander [2003], mutual tlinds can face TCsof 0.5 or even lower, and therefore managers would haveto double performance to obtain results comparable tounconstrained portfolio managers. Future research coulddevelop and empirically analyze ways to modify perfor-mance measures so that the impact of investment restric-tions is neutralized across funds.

78 H o * "INKIRMATIVE" Is THE INFORMATION RATIO FOR EVALUATING MUTUAL FUND MANAGERS? SPRING 2010

A P P E N D I X

E X H I B I T A l

Sample Size of the Fund Dataset Grouped by Fund Classification

Fund Classification

Equity EuropeEquity GermanyEquity U.K.Equity U.S.Equity Small Cap EuropeEquity Small Cap U.K.Equity Small Cap U.S.Corporate Bonds EURCorporate Bonds GBPCorporate Bonds USDMoney Market EURMoney Market GBPMoney Market USD

Source: Aggregation based on Reuters 3000 Xtra and

1998

12754

189970

3151

5290

50880

36202

Number of Funds in the

2000

21457

2671,341

6467

7750

86108

053

230

2002

36365

3702,117

9883

1,23749

12415816479

320

Tliotnson Financial DataStream.

2004

55370

5142,832

132109

1,65312916720322394

396

Dataset by Year

2005

68973

5703,203

152111

1,842151187211243

99410

2006

81380

6583,648

184127

2,057171211231283112433

2007/08

89584

6813,953

202132

2,184185222237300118439

E X H I B I T A 2Overview of Benchmark Indices

Fund Classification Benchmark NameEquity EuropeEquity GermanyEquity U.K.Equity U.S.Equity Small Cap EuropeEquity Small Cap U.K.Equity Small Cap U.S.Corporate Bonds EURCorporate Bonds GBPCorporate Bonds USDMoney Market EURMoney Market GBPMoney Market USD

MSCI EuropeDAXFTSE 100S&P 500MSCI EuropeFTSE All ShareS&P 600 Small CapiBoxx Liquid EUR CorporatesiBoxx Liquid GBP CorporatesMerrill Lynch Corporate MasterEUR Interbank 3M Offered RateGBP Interbank 3M Offered RateUSD Interbank 3M Offered Rate

DataStream Ticker"MSEROPDAXINDXFTSE100S&PCOMPMSEROPFTSEALLSHS&P600IIBELCALIB£CSALMLCORPMBBEUR3MBBGBP3MBBUSD3M

Source: Thomson Financial DataStream.

SPRING 2010 THE JOURNAL OF INVESTING 7 9

ENDNOTES

'See Exhibit AI in the Appendix for a complete overviewof the ftind types analyzed here.

""The reported results were not sensitive to the use ofmethods 2 to 4 and are omitted for brevity.

'Using the Information Ratio as the ranking criterion.^Due to limited data availability, it was not possible to

correct the sample for changes in fund management.^The results are available from the authors upon request.

REFERENCES

Ackermann, C , R. McEnally, and D. Ravenscraft. "The Per-formance of Hedge Funds: BJsk, Return and Incentives."Journii/of Finance, Vol. 54, No. 3 (1999), pp. 833-874.

Ané, T., and C. Labidi. "Return Interval, Dependence Struc-ture, and Multivariate Normality."_/oHmii/ of Economics andFinance, Vol. 28, No. 3 (2004), pp. 285-299.

Below, S.D., and S.R. Stansell. "Do the Individual Moments ofREIT Return Distributions Affect Institutional OwnershipPatterns?"yníirníj/ of Asset Management, Vol. 4, No. 2 (2003),pp. 77-95.

Benson, K., P. Gray, E. Kalotay, and J. Qiu. "Portfolio Con-struction and Performance Measurement When Returns areNon-Normal." w5íríi/ííi«yoMmíi/ of Management, Vol. 32. No. 3(2008), pp. 445-461.

Bollen, N.P.B., and J.A. Busse. "Short-Term Persistence inMutual Fund Performance." Review ofFinanríal Studies, Vol. 18.No. 2 (2005), pp. 569-597.

Brown., S.J., and WN. Goetzmann. "Performance Persistence."Journal of Finance. Vol. 5U. No. 2 (1995), pp. 679-698.

Brown, S.J., W.N. Goetzmann, R.G. Ibbotson, and S.A. Ross."Survivorship Bias in Performance Studies." Review ofFinatiaalStudies, Vol. 5. No. 4 (1992). pp. 553-580.

Brown, S.J., W.N. Goetzmann, and S.A. Ross. "Survival."_/owma/of Finance, Vol. 50, No. 3 (1995), pp. 853-873.

Carhart, M.M. "On Persistence in Mutual Fund Performance."Journal of Finance, Vol. 52, No. 1 (1997). pp. 57-82.

Chan, L.K.C., H.-L. Chen, and J. Lakonishok. "On MutualFund Investment Styles." Review of Finanaal Studies, Vol. 15,No. 5 (2002), pp. 1407-1437.

Cremers, M., and A. Petajisto. "How Active is Your Fund Man-ager? A New Measure That Predicts Performance." WorkingPaper, Yale School of Management, New Haven, 2009.

Elton,E.J.,M.J. Gruber, and C.R. Blake. "Survivorship Bias andMutual Fund Performance." Review of Financial Studies, Vol. 9,No.4 (1996), pp. 1097-1120.

Goodwin, T.H. "The Information Ratio." Financial AnalystsJournal, Vol. 54, No. 4 (1998), pp. 34-43.

Grinblatt, M., and S. Titman. "Mutual Fund Performance: AnAnalysis of Quarterly Portfolio Holdings."yí)Mmíí/ of Business,Vol. 62, No. 3 (1989), pp. 393-416.

Grinold, R.C. "The Fundamental Law of Active Management."Journal of Portfolio Management, Vol. 15, No. 3 (1989), pp. 30-37.

Grinold, R.C. , and R.N. Kahn, Active Portfolio Management:A Quantitative Approach for Providing Superior Returns and Con-trolling Risk, 2nd ed. New York: McGraw-Hill, 2000.

Handa, R,S.P. Kothari,and C. Wasley. "The Relation Betweenthe Return Interval and Betas: Implications for the SizeEffect."_/í>wmií/ of Financial Economics, Vol. 23, No. 1 (1989),pp. 79-100.

Hollander, M., and D.A. Wolfe. Nonparametric Statistical Methods.Hoboken, NJ:John Wiley 6¿ Sons, Inc., 1973.

Horst,J.T., and M. Verbeek. "Estimating Short-Run Persistencein Mutual Fund Performance." Raneu' ofEconomia and Statis-tics, Vol. 82, No. 4 (2000), pp. 646-655.

Hübner, G. "How Do Performance Measures Perform?"yiinrtta/of Portfolio Management, Vol. 33, No. 4 (2007), pp. 64-74.

Jacobs, B.I., and K.N. Levy. "Residual Risk: How Much is TooMuch?'''Journal of Portfolio Management, Vol. 21, No. 3 (1996).pp. 10-16.

Kahn, R.N., and A. Rudd. "Does Historical Performance Pre-dict Future Performance?" Financial Analysts Journal, Vol. 51,No. 6 (1995), pp. 43-52.

Keating, C , and W.F. Shadwick. "Omega: A Universal Perfor-mance Measure."yoHmii/ of Performance Measurement, Vol. 6, No. 3(2002), pp. 59-84.

Kraus, A., and R.H. Litzenberger. "Skewness Preference andthe Valuation of Risk Assets." Journal of Finance, Vol. 31, No. 4(1976), pp. 1085-1100.

80 How "IPJFORMATIVE"' IS THE INFORMATION RATIO FOR. EVALUATING McrruAL Fu^a) MANAGERS? SPRING 2010

Lehmann, B.N., and D.M. Modest. "Mutual Fund Perfor-mance Evaluation: A Comparison of Benchmarks and Bench-mark Comparisons." JoMmd/ of Finance, Vol. 42, No. 2 (1987),pp. 233-265.

LilHefors, H.W. "On the Kolmogorov-Smirnov Test forNormality with Mean and Variance Unknown."_/oHrííií/ ofthe American Statistical Association, Vol. 62, No. 318 (1967),pp. 399-402.

TreynorJ.L. "How to Rate Management of Investment Funds."Harvard Bttsiness Review, Vol. 43, No. 1 (1965), pp. 63-75.

. "Toward a Theory of Market Value of Risky Assets."Working Paper, 1961. Subsequently published in R.A.Kora-jczyk. Asset Pricing and Portfolio Performance: Models, Strategyand Performance Metrics. London: Risk Books, 1999.

Treynor.J.L., and F. Black. "How to Use Security Analysis toImprove Portfolio Selection." Jowrna/ of Business, Vol. 46, No. 1(1973), pp. 66-86.

Wander, B.H. "What it Takes to Beat a Benchmark."JtiMmij/ ofInvesting, Vol. 12, No. 3 (2003), pp. 37-42.

To order reprints of this article, please contact Dewey Palmieri atdpalmieri@iijournals.com or 2Í2-224-3675.

SPRING 2010 THE JOURNAL OF INVESTINÜ 8 1

©Euromoney Institutional Investor PLC. This material must be used for the customer's internal business use

only and a maximum of ten (10) hard copy print-outs may be made. No further copying or transmission of this

material is allowed without the express permission of Euromoney Institutional Investor PLC. Source: Journal of

Investing and http://www.iijournals.com/JOI/Default.asp