www.clapesuc.cl
Fifteen Years of Defined Contributions: Assessing the Chilean Pension Experience
Documento de Trabajo Nº 43
Fifteen Years of Defined Contributions: Assessing
the Chilean Pension Experience
Hans Schlechter
Pontificia Universidad Catolica de Chile
Santiago, CHILE
Bernardo K. Pagnoncelli
Universidad Adolfo Ibanez
Santiago, CHILE
Arturo Cifuentes
CLAPES UC
Santiago, CHILE
and
Columbia University
New York, USA
March 2018
Abstract
In 1980 Chile switched from a state-managed defined-benefit pension system to
a defined-contribution scheme based on individual capital accounts. The new
system was further refined in 2002 with the introduction of five investment funds,
with, allegedly, di↵erent risk-return profiles. The funds di↵er in their portfolio
composition which is driven by strict minimum and maximum limits (mostly
related to stocks and bonds), dictated by the regulator. We have examined the
performance of these funds over a fifteen-year period looking at their returns
and actual risk profiles, aided by three rank-order metrics. Unfortunately, our
results are unambiguously distressing: while the regulator succeeded in creating
five funds with clearly di↵erent risk profiles, their risk-adjusted returns as well
as their cumulative (absolute) returns are completely at odds with the desired
goal. In fact, during long stretches of time the funds exhibited a performance
that was exactly the opposite of what it was intended: an indictment on the
idea of controlling portfolio risk via asset allocation limits.
1 Introduction
The early social security system in Chile started in the 1920s and it was designed
to provide retirement benefits for the elderly, as well as other social benefits.
Under this system, di↵erent pension schemes were developed to attend the needs
of the di↵erent occupational groups in the country. By the 1970s, these schemes
had resulted in significant disparities in terms of the benefits received by each
of these groups. The system was based on a pay-as-you-go (PAYG) structure,
where active workers financed the pensions of the retirees. And pension obliga-
tions were met through withdrawals from the stock of accumulated savings, as
well as from the returns provided by those savings.
During the 1980s, the ine�ciencies associated with this arrangement, plus
doubts over its long-term financial feasibility, pushed the government to reform
the social security system. And in November 1980 a law introducing a new
defined contribution (DC) pension scheme based on individual capital accounts
managed by private institutions, was approved. The new system had two main
objectives. First, it established a clear link between the savings the worker had
accumulated during his active life and his pension. And second, it aimed at pro-
viding the future retiree a stable income with a high replacement rate1. Under
this arrangement, the workers’ monthly contributions are deposited in individual
(segregated) accounts and are managed by private institutions known in Chile
as AFPs (a Spanish acronym derived from their o�cial name, Administradoras
de Fondos de Pensiones). The AFPs are regulated by the Superintendencia de
Pensiones (SP), which dictates the guidelines that the AFPs must adhere to,
when investing the funds of the future retirees, also known as a�liates (afilia-
dos). Ideally, the savings deposited in these capital accounts, plus their accrued
earnings, should be su�cient to provide adequate long-term pension benefits to
the future retirees.1The replacement rate is the ratio obtained by dividing the retiree’s monthly income by a
representative value of his last years’ monthly earnings.
1
In August 2002 the pension system was further modified with the intro-
duction of five funds (multifunds, or multifondos in Spanish), known as A, B,
C, D and E. Fund A was supposed to be the riskiest, and Fund E the most
conservative. These funds were supposed to deliver long-term returns commen-
surate with their respective risk profiles. From the regulator’s perspective, the
rationale behind the multifund system was to o↵er the future retirees a rea-
sonable variety of risk-return investment options, in a setting simpler than the
full complexity of the stock and fixed income markets. The thought was that
younger workers could benefit from taking more risk, and therefore achieving
higher returns, while workers close to their retirement age should move grad-
ually to more conservative options, represented by Fund E. All in all, it was
hoped that by providing di↵erent investment options that could match di↵erent
investment horizons and risk preferences, the system could o↵er the a�liates a
chance to obtain a better pension.
Recently, critics of the current pension system have focused on the low
replacement rate attained by many retirees; the lack of competition (and high
concentration) in the AFP industry; the gap between the pensions received by
civilians and military personnel; the di↵erence in pensions between men and
women; and the fact that until 2008 most independent workers failed to make
regular contributions to their future pensions (since January 2012 it is manda-
tory for freelance workers to make monthly contributions to their retirement
accounts). Furthermore, the consequences of having inadequate pensions have
been one of the main topics of discussion in Chile as the country exhibits one
of the lowest replacement rates among the OECD members. In Chile, pensions
for men are equivalent to 40.1% of their pre-retirement earnings; in the case of
women, the corresponding figure is 36.3% (see [10]). In contrast, the equiva-
lent average values for OECD countries are 62.9% and 62.2% respectively, with
Turkey having the highest percentages (102.1% and 97.9%). The causes behind
the low replacement rate are complex, and involve a combination of factors
such as low contributions (the 10% minimum is likely to increase in the coming
2
months), and the fact that some workers stop contributing for long periods of
time, due to several reasons. Addressing those issues is fundamental to have a
healthy system, but the remedies fall into the realm of public policy and better
communication with the population, which are beyond the scope of this paper.
The regulator defines the risk-return profile of each of the funds via port-
folio constraints. They essentially deal with the percentage of the portfolio that
can be invested in equities and bonds. Of the five funds, Fund A is the one that
is allowed the highest percentage in equities (up to 80% of its holdings). This
percentage is reduced progressively as we move from Fund A to Fund E, where
it reaches a 5% value. By the same token, the percentage of the portfolio that
can be invested in fixed income securities increases from Fund A to Fund E. The
composition of each portfolio is also determined by several other constraints in
addition to the limits by asset class (that is, stocks, bonds, alternative invest-
ments). The SP also imposes limits on maximum exposures to issuers, as well
as asset managers (in case investments are made in mutual funds). In essence,
the restrictions imposed by the SP aim at o↵ering the a�liates the possibil-
ity of having access to di↵erent risk-return alternatives while maintaining an
adequate level of diversification within each option. The key point is that the
regulator has attempted to control the risk-return profile of the multifunds in-
directly, that is, via the asset allocation limits already mentioned. Our findings
show that such portfolio constraints fail to rank the multifunds according to the
desired order of decreasing returns, from Fund A to Fund E, when considering
periods of time greater than 5 years.
With this as background, our main goal in this paper is to o↵er a thorough
understanding of the true risk-return profile of the multifunds, making exten-
sive use of the data available in the period 2002-2017. More precisely, we want
to assess if the multifunds are performing the way the regulator intended, a
problem of paramount importance since the Chilean pension system has served
as a blueprint to modify the pension system of other countries, mostly in Latin
3
America and Eastern Europe. As described in [13], life-cycle funds have specific
characteristics that di↵erentiate them from other financial instruments and the
evidence in other countries indicate that a correct performance of these funds de-
pends on a well-designed system. For example, evidence from Turkey [4] shows
that having a passive investment system outperforms an active investment one,
or from Poland [6], which shows that the supervisory board structure has an im-
portant e↵ect on the funds performance. In the context of our research, the very
long investment horizon implies that the composition of an individual’s portfolio
has to change as retirement approaches, and in a DC system the regulator must
o↵er a su�cient number of alternatives that can accommodate those needs. Our
main finding is that the current system fails to do so, and the consequences for
the a�liates are severe.
In the following sections we evaluate the historic performance of the mul-
tifunds, we look at their true risk profile, and we propose some metrics to assess
whether their actual performance was consistent with the objectives spelled out
by the regulator. This is necessary to evaluate the suitability of the current
regulatory constraints. We conclude with some suggestions on how to modify
the current regulation so that risk-return profiles of the funds actually match
the desired objective.
2 Performance of the Funds
2.1 Returns
The pension of a worker is a function of both, the savings accumulated during his
active working life plus the profitability achieved by those savings. Thus, in this
section we focus on the returns achieved by the multifunds. For the purpose of
this study we have relied on the performance data of the multifunds as reported
in the regulator website (www.spensiones.cl). Specifically, the data gathered
4
start in October 2002 (the beginning of the multifund system) and end in July
2017. The data reflect the monthly returns of each fund (A, B, C, D and E),
measured based on inflation-adjusted Chilean pesos (a unit known as unidad
de fomento, or UF). Hence, these returns are actual (real) returns, not nominal
returns. Additionally, the returns used in this study are the monthly industry
averages for each fund. The Chilean AFPs exhibit strong herd behavior and
thus, working with the average industry returns (as opposed to the returns of
a particular AFP) makes more sense. The strong herd behavior has been the
result of an ill-designed performance benchmark that encourages the AFPs to
mimic each other’s portfolios in order to avoid the penalties associated with
deviations from the industry average. This topic has been treated in detail in
[3].
2.1.1 Average returns
With this in mind, we turn first to Figure 1, which shows the monthly returns
for each fund. We can observe that Fund A displays the highest volatility, while
Funds D and E display the lowest. It can also be seen that the riskier funds
fall and recover together, which suggest that their behavior is highly correlated.
Table 1 reports some basic statistics based on the funds monthly returns (178
data points for each fund). These statistics are consistent with the trends shown
in Figure 1, namely, Fund A exhibits the highest volatility (measured by the
standard deviation of returns), while D and E the lowest. Table 1 also indicates
that in terms of minimum and maximum monthly returns, as well as average
returns, the five funds are rank-ordered according to the intended risk profile,
namely, Fund A on top, and Fund E at the bottom.
These results might seem to indicate that the asset allocation constraints
designed by the regulator had indeed achieved their intended goal. However,
we will see that such conclusion is premature. The fact of the matter is that
the average monthly return is a poor proxy for long-term performance. More
5
20022003
20042005
20062007
20082009
20102011
20122013
20142015
2016
-20
-10
0
10R
etu
rns
in p
erce
nta
ge
(%)
Fund A
20022003
20042005
20062007
20082009
20102011
20122013
20142015
2016
-20
-10
0
10
Ret
urn
s
in p
erce
nta
ge
(%)
Fund B
20022003
20042005
20062007
20082009
20102011
20122013
20142015
2016
-20
-10
0
10
Ret
urn
s
in p
erce
nta
ge
(%)
Fund C
20022003
20042005
20062007
20082009
20102011
20122013
20142015
2016
-20
-10
0
10
Ret
urn
s
in p
erce
nta
ge
(%)
Fund D
20022003
20042005
20062007
20082009
20102011
20122013
20142015
2016
Period (year)
-20
-10
0
10
Ret
urn
s
in p
erce
nta
ge
(%)
Fund E
Figure 1: Monthly returns for the five funds (October 2002 - July 2017).
precisely, what is critical in terms of a pension, is really the cumulative return
over the relevant time-period, and not the average monthly returns and its
corresponding fluctuations (volatility). Thus, we now look at the cumulative
long-term return of the multifunds.
6
Table 1: Funds monthly returns, descriptive statistics, expressed in percentage
(%), October 2002 - July 2017.
Fund Mean Return St. Dev. Min Max
A 0.61 3.59 –21.28 9.50
B 0.50 2.59 –14.25 6.47
C 0.44 1.74 –8.00 4.25
D 0.38 1.10 –4.00 2.90
E 0.32 0.89 –2.81 3.40
2.1.2 Cumulative returns
Considering that a working person’s active life is roughly forty years, and that
we have five funds, it is reasonable to assume—as a first approximation—that
on average a typical worker would stay eight years on each fund, as he moves
sequentially from Fund A to Fund E. Thus, we consider the 8-year cumulative
return as an appropriate parameter to assess the multifunds’ performance. Fig-
ure 2 depicts for each of the five funds, the cumulative 8-year return, based on
di↵erent starting dates, beginning on October 2002, and ending on July 2009.
That is, we consider all the possible 8-year time-windows allowed by our data.
Therefore, each point in the graph represents the cumulative return a worker
would have obtained, had he entered that specific fund on the date indicated
on the horizontal axis, assuming he remains there for the entire 8-year period.
We observe that the funds returns for the period ranging from the end of
2002 until the end of 2003, and from the end of 2008 until the beginning of 2010,
are rank-ordered according to the sequence the regulator intended. That is, a
person who had entered the system at any time within those periods, would
have obtained higher returns if he had chosen Fund A over Fund B, or Fund
B over C, etc. In other words, the riskier the funds, the higher the cumulative
7
20022003
20042005
20062007
2008
Starting period (year)
0
10
20
30
40
50
60
70
80
90
100
110
120
Cu
mu
lati
ve
retu
rns
in p
erce
nta
ge
(%)
Fund A Fund B Fund C Fund D Fund E
Figure 2: Cumulative 8-year returns for the five funds, as a function of the
starting period (October 2002 - July 2009).
return. However, it is the period from the end of 2003 until the end of 2008
the one that draws our attention. During this period, there is a high variability
in terms of which fund enjoys the highest return. Moreover, throughout most
of the period, rank-ordering the funds in terms of their returns results in a
sequence which is exactly the opposite of what it was intended. For example,
a young person who had entered the system at the beginning of 2006 would
have received a higher return if he had chosen Fund E (the most conservative)
instead of Fund A (the one that is supposed to be the most adequate choice for a
young worker). As a matter of fact, there are many instances within this period
in which the cumulative return of Fund A was close to zero. It is impossible
to overlook the damage that a situation like this would have inflicted on those
workers. In summary, Figure 2 shows a very disturbing phenomenon—during a
significant period of time, the cumulative (long-term) returns of the multifunds
exhibited a pattern which was the exact opposite of what the regulator had in
8
mind. Longer (10- or 12-year) as well as shorter (5-year) time-windows result
also in graphs retaining the essential features identified in Figure 2, namely, an
inverted relationship between the risk and return of the multifunds that persists
for a significant length of time.
2.1.3 Sharpe ratio
Figure 3 is analogous to Figure 2, except that it shows the Sharpe ratio (SR)
instead of cumulative return, computed over 8-year time-windows. In this cal-
culation the risk-free rate was assumed to be zero, thus, the SR is actually the
average observed return divided by its corresponding standard deviation. The
20022003
20042005
20062007
2008
Starting period (year)
0
0.1
0.2
0.3
0.4
0.5
Sh
arp
e ra
tio
Fund A Fund B Fund C Fund D Fund E
Figure 3: Sharpe ratio (SR) for each of the five funds, considering 8-year time-
windows, as a function of the starting period (October 2002 - July 2009).
SR reflects the return adjusted by risk (i.e. normalized by units of risk). One
could argue that, in theory, after adjusting for risk, all five funds should exhibit
a similar performance. Clearly, this is not the case. It is clear from Figure 3 that
9
Fund E is always on top, while Fund A is always at the bottom. Considering
that the SR is actually obtained by dividing two numbers, it is di�cult—at least
initially—to attribute a low SR solely to poor return performance. We must
be mindful of the potential distortions caused by a denominator approaching
zero, a frequent case when dealing with low volatility portfolios such as funds
D and E. Therefore, although it is tempting to attribute to Fund E a superior
performance based on this metric, we must refrain, at this point, from deriving
sweeping conclusions based only on this metric. We will revisit this issue later.
2.1.4 Holding periods
Figure 4 shows the cumulative returns considering di↵erent holding periods,
expressed in months (along the horizontal axis), for each of the funds. The
holding periods—the time between entering and leaving a specific fund—range
between 1 to 96 months, and we consider all the possible starting months given
our data. Thus, each point on the graph represents the n-month cumulative
return achieved by a person who entered that specific fund at some time be-
tween October 2002 and July 2009. For each point in the x-axis we have 82
observations, corresponding to each of the possible starting months given our
data. For example, when considering 96-month holding periods (8 years), the
vertical points for each fund correspond to the curves in Figure 2. Finally, the
last frame (f) shows the average returns for all funds.
Several observations are in order. First, we notice that the cumulative
returns of Fund A are marked by a high dispersion with respect to the average.
This dispersion decreases as we move from Fund A to B, from B to C, and so
on. This situation appears to be in line with the regulator intention, namely,
volatility of returns should decrease from Fund A to Fund E.
However—once again—this conclusion is premature for frame (f) reveals
another worrying pattern. We note that the average cumulative returns of the
10
12 24 36 48 60 72 84 96-50
-30
-10
10
30
50
70
90
110
Cum
ula
tive
retu
rn i
n p
erce
nta
ge
(%)
Average
(a) Fund A
12 24 36 48 60 72 84 96-50
-30
-10
10
30
50
70
90
110
Cum
ula
tive
retu
rn i
n p
erce
nta
ge
(%)
Average
(b) Fund B
12 24 36 48 60 72 84 96-50
-30
-10
10
30
50
70
90
110
Cum
ula
tive
retu
rn i
n p
erce
nta
ge
(%)
Average
(c) Fund C
12 24 36 48 60 72 84 96-50
-30
-10
10
30
50
70
90
110C
um
ula
tive
retu
rn i
n p
erce
nta
ge
(%)
Average
(d) Fund D
12 24 36 48 60 72 84 96-50
-30
-10
10
30
50
70
90
110
Cum
ula
tive
retu
rn i
n p
erce
nta
ge
(%)
Average
(e) Fund E
12 24 36 48 60 72 84 96
Length of holding period (months)
0
10
20
30
40
50
60
Cum
ula
tive
retu
rn i
n p
erce
nta
ge
(%)
Fund A
Fund B
Fund C
Fund D
Fund E
(f) Average return for each fund
Figure 4: Cumulative returns for each of the five funds considering several
holding periods.
11
five funds, while easily distinguishable for short—and medium—term holding
periods, start to converge after five years. In fact, the longer the time horizon,
the more similar the returns of the five funds. This situation is quite revealing.
First, it indicates that it is misleading to evaluate the performance of the funds
without paying attention to long-term horizons. And second, it shows that when
matters the most—that is, for periods longer than five years—the funds behave
fairly similarly. This is problematic for the typical holding period of any given
fund is likely to be longer than five years. This convergence in performance
certainly undermines the rationale for selecting Funds A or B, as they appear
to o↵er only higher risk without adequate return compensation for taking this
additional risk. In fact, this revelation is consistent with what the SR hinted in
the previous section. Thus, we are now in a position to state that the riskier
funds o↵er poor risk-adjusted returns.
In summary, while the average returns paint a picture which appears to
be consistent with the regulator intentions, the cumulative returns and the risk-
adjusted returns (far more relevant metrics from the workers perspective) o↵er a
dramatically di↵erent view: the multifunds have exhibited during an important
part of their existence a performance which is almost the opposite of what it
was intended.
2.2 Risk Profile of the Funds
We now turn to the risk associated with the funds. For this purpose, we consider
two risk metrics commonly used in risk management and financial engineering,
the Value-at-Risk (VaR) and the Conditional Value-at-Risk (CVaR). The VaR
is the maximum loss that a portfolio can su↵er in a specific period of time,
estimated with a given (normally very high) level of confidence [9]. If the VaR
is estimated with a confidence 1 � ↵, it follows that the probability of having
losses exceeding the VaR is ↵. More formally, the VaR of a random variable
X with cumulative distribution function (cdf) F (·) and with a confidence level
12
↵ 2 [0, 1] is defined as
VaR↵[X] := min{t|F (t) � ↵} = min{t|P (X t) � ↵}.
The CVaR, which is an alternative risk metric to the VaR, considers only those
losses that exceed the VaR [12]. The CVaR of a random variable X with cdf
F (·) and with confidence level ↵ 2 [0, 1] is defined as
CVaR↵[X] :=1
1� ↵
Z 1
↵VaR� [X]d�.
In short, the CVaR (also known as expected shortfall) is the expected value
of the losses that exceed the VaR. When X is a discrete random variable with
support points z1 < z2 < . . . < zN and associated probabilities p1, . . . , pN , it
follows from [11] that
CVaR↵[X] :=1
1� ↵
" k↵X
k=1
pk � ↵
!zk↵ +
NX
k=k↵+1
pkzk
#,
where ↵ 2 (0, 1) and k↵ is such that
k↵X
k=1
pk � ↵ >k↵�1X
k=1
pk.
In this study we will focus on the VaR and the CVaR of the funds’ returns.
In this context, a fund is riskier if its VaR (or CVaR) is lower (more negative).
Relying solely on the VaR somehow limits the scope of the analysis, since the
VaR does not fully capture the tail end of the distribution associated with lower
returns. The CVaR, which focuses on the values exceeding the VaR, does.
Additionally, another shortcoming of the VaR is that it violates the so-called
subadditivity condition. The CVaR, a coherent risk metric [1], does not have
this limitation.
2.2.1 VaR
Figure 5 shows the 95%-confidence VaR, based on monthly returns, for the five
funds, considering 6-year time-windows (top frame) and 8-year time-windows
13
(bottom frame). The horizontal axis indicates the beginning of the period con-
sidered. For the 6-year time-windows the VaR of the funds ranges from –6.3%
to –0.9%, with Fund E consistently exhibiting a value fluctuating around –1%.
The variability of the VaR of the funds increases as we move from Fund E to
Fund A. We also notice that the rank order of the funds, which is in line with the
regulator expectations, is stable over time, although the numerical di↵erences
among the funds’ VaRs change widely. The 8-year VaR, shown in the bottom
panel of Figure 5, reveals the same patterns. However, the numerical di↵erences
among the VaR values tend to show greater consistency.
20022003
20042005
20062007
20082009
2010
Starting period (year)
(a) 6-year time window
-8
-7
-6
-5
-4
-3
-2
-1
0
VaR
in p
erce
nta
ge
(%)
20022003
20042005
20062007
2008
Starting period (year)
(b) 8-year time window
-8
-7
-6
-5
-4
-3
-2
-1
0
VaR
in
per
centa
ge
(%)
Fund A Fund B Fund C Fund D Fund E
Figure 5: VaR, based on monthly returns, for each fund, considering (a) a 6-year
time-window and (b) an 8-year time-window.
14
2.2.2 CVaR
Figure 6, which is similar to Figure 5, shows the corresponding 95%-CVaR
values, considering, again, 6-year (top frame) and 8-year time-windows (bottom
frame). The same pattern applies: we see a consistent rank order according
the regulator’s intentions, with some discrepancies in terms of the actual CVaR
values. These discrepancies are more significant when using the 6-year time-
window, which seems to suggest that over longer time-periods, the relationship
among the risk of each of the funds is more stable. It is also apparent—in
agreement with Figure 5—that starting in 2012, Funds D and E show an almost
identical performance in terms of risk.
20022003
20042005
20062007
20082009
2010
Starting period (year)
(a) 6-year time window
-14
-12
-10
-8
-6
-4
-2
0
CV
aR i
n p
erce
nta
ge
(%)
20022003
20042005
20062007
2008
Starting period (year)
(b) 8-year time window
-14
-12
-10
-8
-6
-4
-2
0
CV
aR i
n p
erce
nta
ge
(%)
Fund A Fund B Fund C Fund D Fund E
Figure 6: Monthly returns CVaR, considering (a) 6-year and (b) 8-year time-
windows.
15
In summary, we can conclude that in terms of risk, either by looking at
the absolute value of the relevant metrics, or the rank order they imply, the
funds behaved in the manner expected: the funds exhibited decreasing levels of
risk from Fund A to Fund E.
3 Rank Order Metrics
Having considered di↵erent criteria to evaluate returns and risk, we now examine
the rank order imply by these criteria. To this end, we focus on the two most
relevant parameters, the CVaR and the cumulative return, measured both using
di↵erent time windows from 6 to 12 years. Recall that the intention of the
regulator was twofold: (i) in terms of risk, the funds should exhibit decreasing
levels of risk, from Fund A to Fund E; and (ii) in terms of returns, the funds
should deliver decreasing returns when moving from Fund A to Fund E. We
focus on the CVaR (instead of the VaR) since as explained before the CVaR
is a more encompassing metric as it captures the behavior of the tail end of
the (return) distribution. And we select the cumulative return (instead of the
average return), since this is the factor that really dictates the magnitude of
the replacement rate. Therefore, an adequate performance of the system would
mean that the funds (A, B, C, D, E), judged by these two criteria, should be
rank-ordered as (1, 2, 3, 4, 5), if not always, at least most of the time.
3.1 Definition of the metrics
For this purpose, we consider three di↵erent rank order metrics. Each addresses
a di↵erent aspect of the departure from the correct (desired) rank order.
(i) Hamming distance. This metric assigns a value of 0 if a fund is in the
correct position and 1 otherwise (see [5]). Thus, the maximum possible
16
value is 5, corresponding to a situation in which all the funds are in the
“wrong” position. For example, the sequence (1, 2, 3, 4, 5) obviously
results in a value equal zero. However, the sequences (5, 3, 4, 2, 1) and
(3, 4, 1, 5, 2) are assigned a value of 5, while (1, 2, 3, 5, 4) would get a 2.
Let us note that the sequence (5, 4, 3, 2, 1), which in our case represents
the worst-case scenario, is assigned a value of 4, explained by the correct
position of Fund C. A shortcoming of this metric is that only focusses
on whether a fund is in the correct position, but not the distance to its
“correct” position.
(ii) Spearman footrule. This metric considers the absolute di↵erence be-
tween the position in which a fund is, and the position it should have, and
adds all five numerical values (see [2]). In short, it attempts to capture
the magnitude of the deviation from the correct rank order as well. For
example (3, 4, 1, 5, 2) results in a value equal to |3 � 1| + |4 � 2| + |1 �
3| + |5 � 4| + |2 � 5| = 10. If the funds are rank-ordered in exactly the
reverse sequence, i.e. (5, 4, 3, 2, 1), something we can describe as the
worst possible situation, the value is 12, which is indeed the maximum
possible value this metric can have.
(iii) Kendall Tau rank distance. This metric counts the number of pairwise
discrepancies between the correct rank order and the actual rank order (see
[7], [8]). Since we have five funds, the possible pairwise comparisons are
ten (1 with 2, 3, 4, and 5, and then 2 with 3, 4 and 5, and so on), and
hence the maximum possible value is 10. The following example clarifies
the calculation. Suppose the funds have been rank-ordered in the following
sequence: (3, 4, 1, 2, 5). In this case the Kendall Tau is 4 because the
pairs (3,1), (3,2), (4,1) and (4,2) represent pairwise disagreements with
respect to the original list (1, 2, 3, 4, 5).
17
3.2 Results
In all three metrics, higher values are associated with higher levels of discrepancy
in terms of the rank order. To facilitate the comparisons, we have normalized
all metrics by their maximum value. Thus, a value of 0 reflects a perfect rank
order, (1, 2, 3, 4, 5) in this case, whereas a value of 1 indicates the maximum
discrepancy with respect to the desired benchmark.
20022003
20042005
20062007
20082009
2010
Starting period (year)
(a) 6-year time window
0
0.2
0.4
0.6
0.8
1
Per
form
ance
met
ric
(norm
aliz
ed)
20022003
20042005
20062007
2008
Starting period (year)
(b) 8-year time window
0
0.2
0.4
0.6
0.8
1
Per
form
ance
met
ric
(norm
aliz
ed)
Hamming Spearman Kendall Tau
20022003
20042005
2006
Starting period (year)
(c) 10-year time window
0
0.2
0.4
0.6
0.8
1
Per
form
ance
met
ric
(norm
aliz
ed)
20022003
2004
Starting period (year)
(d) 12-year time window
0
0.2
0.4
0.6
0.8
1
Per
form
ance
met
ric
(norm
aliz
ed)
Figure 7: Rank order performance metrics based on CVaR, based on (a) 6-year,
(b) 8-year, (c) 10-year and (d) 12-year time-windows.
Figure 7 shows the normalized values of the three rank order metrics ap-
plied to the CVaR risk metric, while Figure 8 shows the values of the metrics
applied to the cumulative returns. We consider 6-, 8-, 10- and 12-year time-
windows. This is analogous to the situation described in Figures 2 and 3 (the
18
20022003
20042005
20062007
20082009
2010
Starting period (year)
(a) 6-year time window
0
0.2
0.4
0.6
0.8
1P
erfo
rman
ce m
etri
c (n
orm
aliz
ed)
20022003
20042005
20062007
2008
Starting period (year)
(b) 8-year time window
0
0.2
0.4
0.6
0.8
1
Per
form
ance
met
ric
(norm
aliz
ed)
Hamming Spearman Kendall Tau
20022003
20042005
2006
Starting period (year)
(c) 10-year time window
0
0.2
0.4
0.6
0.8
1
Per
form
ance
met
ric
(norm
aliz
ed)
20022003
2004
Starting period (year)
(d) 12-year time window
0
0.2
0.4
0.6
0.8
1
Per
form
ance
met
ric
(norm
aliz
ed)
Figure 8: Rank order performance metrics based on cumulative returns, based
on (a) 6-year, (b) 8-year, (c) 10-year and (d) 12-year time-windows.
dates along the horizontal axis mark the beginning of the time-window consid-
ered). Average values of the three di↵erent rank order metrics are summarized
in Tables 2 and 3 for the CVaR and cumulative returns, respectively, for the
various time-windows considered in the analysis.
Table 2: CVaR rank order metrics.
Length of time window
Distance 6y 8y 10y 12y
Hamming 0.127 0.048 0 0
Spearman 0.053 0.020 0 0
Kendall Tau 0.032 0.012 0 0
19
Table 3: Cumulative returns: rank order metrics.
Length of time window
Distance 6y 8y 10y 12y
Hamming 0.512 0.533 0.448 0.291
Spearman 0.578 0.576 0.461 0.181
Kendall Tau 0.547 0.542 0.417 0.120
In terms of absolute risk (CVaR, Figure 7), except for 34 out of 107 periods
at the end of the 6-year time-window (Figure 7(a)), and 10 out of 83 at the end
of the 8-year time-window (Figure 7(b)), the funds are rank-ordered correctly.
Table 2 shows a pattern which is consistent with Figure 7, namely, that in terms
of risk the funds are ordered, most of the time, in a satisfactory manner, as the
metrics are much closer to 0 than 1. Moreover, for 10-year periods or longer,
the order is perfect.
In terms of cumulative returns (Figure 8), all three metrics suggest a wor-
risome pattern. Let us focus on the 8-year time window (Figure 8(b)). We note
that in October 2003 the rank order starts to depart from the correct sequence
and gets increasingly distorted. Then, from July 2005 through October 2008,
the funds are consistently rank-ordered in a manner which departs significantly
from the desired sequence as two of the three metrics remain anchored at the
worst possible value, that is, 1. For this time-window, the funds are rank-ordered
in an undesirable manner in 60 out of 83 cases, and in 41 out of 83 cases the
funds are rank-ordered in the worst possible way: (5, 4, 3, 2, 1), in essence,
a rank order which is the exact opposite of what the regulator had in mind.
Table 3 presents a pattern consistent with what is shown graphically in Figure
8, that is, a rank order of the funds at odds with the regulators goal (metrics
values higher than, or close to, 0.5). Although for 12-year periods the metrics
improve (closer to 0 than 1), the significance of this improvement is arguably
20
very debatable as most people stay in a fund for a period much shorter than 12
years.
As a final observation, the fact that the funds are correctly rank-ordered
in terms of risk, but not in terms of cumulative returns, explains the pattern
exhibited by the SR (Figure 3). In short, this figure reveals that as we move
from Fund A to Fund E, the risk-adjusted returns improve, which—again—it is
exactly the opposite of what we should expect. In other words, investors in the
riskier funds received a poor compensation for taking more risk.
4 Conclusions
In 2002 the Chilean pension regulator introduced a multifund scheme: five funds
(A, B, C, D, E), which were supposed to o↵er the future retirees di↵erent risk-
return options according to their investment profiles and preferences. Specifi-
cally, the funds were designed to achieve increasing long-term returns commen-
surate with their risk profiles, with risk decreasing from Fund A to Fund E. The
ultimate goal of this multifund structure was to improve the pensions replace-
ment rate. Fifteen years after the implementation of this concept (almost twice
the average time a worker remains in each fund), it seems fitting to examine if
this idea has been successful.
Unfortunately, based on all the analyses already discussed, the verdict
is quite negative. Yes, the funds designed by the regulator—chiefly through a
number of upper and lower limits in terms of asset classes—have been successful
in the sense that the five funds are correctly rank-ordered in terms of risk.
However, their cumulative returns over long time periods have not been in line
with their risk profile. In fact, during significant stretches of time, the five
funds have been rank-ordered in terms of returns in a manner which is the
opposite of what it was intended (with Fund A exhibiting the lowest returns
21
and Fund E the highest). In essence, participants in Funds A and B took more
risk, but they did not receive returns that compensated them for this risk. The
consistency exhibited by the rank order of the funds based on their SRs (with
Fund A always at the bottom and Fund E on top), coupled with the strong
convergence of returns displayed after holding periods longer than five years,
add to a troubling picture. It leaves one wondering whether it would have been
better just to o↵er one fund. This situation has been particularly damming for
those young workers who entered the system around 2007 and most likely join
Fund A. For example, 100 UF invested in Fund A in November 2007 would have
meant to have only 102.12 UF in November 2015. In summary, the evidence
indicates that attempting to control the risk-return profile of the funds by means
of time-invariant asset allocation constraints has not worked in the way it was
intended. The risk-return profiles of the funds are at odds with the intended
goal.
The idea that attempting to define (and control) the risk-return profile
of a portfolio via concentration limits does not work should not be surprising.
This idea lacks both, a sound theoretical basis, and some credible empirical
evidence. If one wants to control risk, the obvious approach is to do so via some
of the commonly accepted risk metrics, applied to the entire portfolio under
consideration. The indirect approach taken by the SP, namely, manipulating the
asset concentration limits with the hope of obtaining some consistent risk-return
profiles, is somehow based on the notion that the key attributes of di↵erent asset
classes are time-invariant. Clearly, there is no evidence to support this claim.
In fact, the evidence points in the opposite direction. For example, return
correlation values between di↵erent asset classes are notoriously unstable and
change significantly as a function of time.
Finally, it must be clear that this study should not be taken as an in-
dictment of the Chilean pension system, and more broadly, an indictment of
the conceptual basis at the root of privately-managed DC schemes in general.
22
Neither is a call to return to a PAYG system or a push to promote the alleged
benefits of a state-managed pension fund system. This study is really a call to
re-examine the criteria currently employed by the regulator to control the risk
profile of the multifunds. At the very least, the idea of abandoning the practice
of attempting to control risk via asset allocation limits, in order to replace it
with sound portfolio-level risk metrics should be considered. Failure to do so
will continue to inflict permanent damage on the pensions of the future retirees.
References
[1] Philippe Artzner, Freddy Delbaen, Jean-Marc Eber, and David Heath. Co-
herent measures of risk. Mathematical finance, 9(3):203–228, 1999.
[2] Persi Diaconis and Ronald L. Graham. Spearman’s footrule as a measure of
disarray. Journal of the Royal Statistical Society. Series B (Methodological),
pages 262–268, 1977.
[3] Viviana Fernandez. Profitability of Chile’s defined-contribution-based pen-
sion system during the multifund era. Emerging Markets Finance and
Trade, 49(5):4–25, 2013.
[4] Umut Gokcen and Atakan Yalcın. The case against active pension funds:
Evidence from the Turkish private pension system. Emerging Markets Re-
view, 23:46–67, 2015.
[5] Richard W. Hamming. Error detecting and error correcting codes. Bell
Labs Technical Journal, 29(2):147–160, 1950.
[6] Krzysztof Jackowicz and Oskar Kowalewski. Crisis, internal governance
mechanisms and pension fund performance: Evidence from Poland. Emerg-
ing Markets Review, 13(4):493–515, 2012.
[7] Maurice G. Kendall. A new measure of rank correlation. Biometrika,
30(1/2):81–93, 1938.
23
[8] Maurice G. Kendall. Rank correlation methods. Hafner, New York, 2nd
ed., rev. and enl. edition, 1955.
[9] Thomas J. Linsmeier and Neil D. Pearson. Value at risk. Financial Analysts
Journal, 56(2):47–67, 2000.
[10] OECD. Pensions at a glance 2017. 2017.
[11] R. Tyrrell Rockafellar and Stanislav Uryasev. Conditional value-at-risk for
general loss distributions. Journal of banking & finance, 26(7):1443–1471,
2002.
[12] R. Tyrrell Rockafellar, Stanislav Uryasev, et al. Optimization of conditional
value-at-risk. Journal of risk, 2:21–42, 2000.
[13] Luis M. Viceira. Life-cycle funds, in Annamaria Lusardi, ed., Overcoming
the saving slump: How to increase the e↵ectiveness of financial education
and saving programs, 2008.
24
Top Related