Impact of research on monetary policy in transitional countries

Presented by

Viola Zentai, CPS

Background

• Monetary policymakers rely heavily on research in the Western countries

• In transitional countries, however, this cannot be taken for granted

• Strong link between monetary policy and research, on the other hand, is an important element in the process of EU integration

• Therefore, the main research problem is to investigate the links between research and policy in transitional countries

• We focus on the advanced transitional countries, Bulgaria, Croatia, Czech Republic, Hungary, Poland, Romania, Slovakia, and Slovenia

• The research aim is to empirically analyse which factors stimulate researchers to explicitly recommend policy modification, and which factors facilitate policymakers’ adoption of the suggested modifications

• The following are the main features of the monetary policy of these 8 countries:

The research problem• The aim is to extend and generalize the study of the

research-policy links beyond the case-study and descriptive approaches

• We take a rigorous approach based on cross-country and cross-temporal generalizations that simultaneously look into factors and influences contained in the research evidence and into the political and economic environment

• The research problem is thus to answer which research approaches, methods, and environmental factors affect the likelihood of implementation of a policy change or adjustment suggested in relevant research papers, studies, and analyses

• Subsequently, are the characteristics of research pieces such as use of particular methods, interpretation of the results or clarity of policy recommendations likely determinants of the likelihood that the policy option supported in the particular paper will be implemented

Methodology

• Our focus is on assessing the influence and impact of research on policy with an aim to understand the factors that affect the interaction between research and policy

• The two alternative approaches would be to either focus on individual pieces of research (or whole bodies of research) and ask the question whether this research had any impact on policymaking (the "forward-looking" approach), or to look at clear episodes of policy change and try to trace the factors that led to the policy change (the "backward-looking" approach) and in particular the impact of pieces or bodies of research.

• Contextual specifics of the monetary policy and the role of research in the monetary policymaking process dictate further narrowing of these two approaches

• The key characteristic of the monetary policy is the continuous use of research, which is commonly produced to large degree by the central banks' research departments for the specific purpose of monetary policymaking

• The central banks often commission and use external (e.g. academic or international) research and the policymaking is generally not based on single pieces of research, rather on a whole body of more or less relevant research reports, papers, and policy analyses

• Therefore, tracing any policy change to a single piece of research is not possible as for any policy change there will be more then one paper supporting it

• Therefore, the suitable methodological approach of linking changes in the monetary policy to research is to use statistical inference in a similar way as in the education and labour research

• The policy process is largely cognitive in nature, in a way, it is a learning process in which the policymakers are designing policy measures in response to their perception of the economic system and anticipation of the likely effect such measures will yield

• Thus, exposure to research information that summarizes and simplifies the state of the economic system and indicates possible effects of alternative policy actions is one of the factors affecting policymaking and its statistical significance can be tested in a similar fashion the exposure to education is tested in labour and income studies

• The education process is based on student's exposures to various pieces of study materials such as textbooks and lecture notes

• With the individuals as basic sample-cases, a typical income study will try to estimate the impact of education on earnings, controlling for various other income determinants (e.g. age, experience, race, etc.)

• Labour studies might estimate the effect of education on duration of unemployment or probability of getting certain type of jobs (e.g. management positions).

• Translating this approach to the investigation of the impact of research on policy, the proxy for linking individual pieces of research to policy implementation would be the exposure to the particular research piece, or the fact that the policymakers used, read, or otherwise "considered" particular pieces of research

• A similar logic can be applied to the investigation of the link between research and monetary policy, where the sample cases are individual research pieces and the impact is measures through the implementation indicator that records whether suggestions in each individual research piece was implemented

• The statistical sampling) nature of the problem is thus in the existence of a stochastic error term due to possible inefficiency of the exposure to research (i.e. though having particular research piece in hand, the policymakers cannot be assured by have it read carefully) and due to classical measurement error

• The likely measurement error might be due to inability to collect all relevant research pieces or by mistake include irrelevant ones, but for as long as the standard assumptions commonly made about the stochastic error can be justified, the approach will yield consistent results in large samples

• If we assume that policymakers are exposed to research information in a cumulative way (by considering multiple research pieces and cumulative research evidence reflected through various reports, studies, and analyses), then the main source of stochastic error in the research-policy impact models will be due to sampling of possible less relevant or exclusion of relevant pieces of research

• Given sufficiently large samples are available, the sufficient statistical assumption would be the randomness of erroneous inclusion/exclusion, i.e. lack of any systematic pattern in the sampling error

• A relevant issue in the monetary policy sector is whether the policy implication/suggestion (whether implicit or explicit) made in a particular research piece was implemented as a policy modification

• Furthermore, a distinction between policy change and policy adjustment is made in the specific context of monetary policy, where adjustment in the form of, e.g., interest rates increase within the same main policy framework, e.g., inflation rate targeting, is considered as policy adjustment, while a switch from, e.g., crawling peg to inflation targeting is considered a policy change

• Finally, the speed of policy change is recorded as the duration from the publication of the paper (or the time the paper was considered by the policymakers if the two dates do not coincide), from instantaneous (same year) to 2 years following the publication of the paper

The Model

• We model the implementation probability as a likelihood that a suggested policy modification gets implemented, and also the likelihood that researchers will make an explicit policy recommendation on the basis of various research characteristics and environmental factors. This model can be described in terms of two choices, of the decision makers, and of the researchers

• The important question is whether the likelihood of implementing the suggested policy actions can be, to some degree, attributed to the characteristics of the research papers; in other words, we wish to know whether the issue of who and how writes the papers might make the written suggestions more or less likely to be implemented in practice

• Which advice was followed, and which research-evidence was influential in the monetary sector's policy process are some of the central issues of relevance for understanding the research-policy linkages

• We formulate the research problem by starting from the individual papers written over a ten-year period and look whether their suggestions got implemented or not and then trying to distinguish between successful and unsuccessful papers on the grounds of how they were written, by whom and in which period (during political/economic crisis or at other times)

Data and descriptive statistics

• The methods of data collection and analysis are specifically designed to meet the objectives of standardization across countries, time, and socio-economic environment and of quantitative measurability and suitability for formal statistical analysis. The final objective is the policy relevance.

• Standardization necessarily dictates simplicity and robustness in the selection of variables, which need to be unambiguous in different spatial and temporal contexts. Moreover, the selected variables must be measurable through quantitative indicators, not necessarily with metric scale

• Quantitative measurability, however, is not in conflict with qualitative nature of many relevant variables, however, their qualitative categories must be clear and thus recordable by quantitative indicator variables

• We collected data on relevant research papers, monetary policy, and the economic and political environment for eight advanced transitional countries: Bulgaria, Croatia, Czech Republic, Hungary, Poland, Rumania, Slovakia, and Slovenia

• The basic observational units in our data set are individual research papers and we have content-analysed each paper on a number variables that characterize how the paper was written and structured

• Content analysis was used to extract the type of policy suggestion and to determine whether the paper merely implied a support for a particular policy option or whether it recommended it explicitly

• Such data structure overcomes the problems with the multilevel factors and allows statistical analysis at the micro-level without ignoring macro factors.

Descriptive statistics

Note: P = policy change, A = policy adjustment

t = same year, t + 1 = year later, t + 2 = two years later

Estimation results

• We estimate a structural equation model using weighted least squares with two equations

• The first equation measures the impact on the likelihood of policy modification (either change or adjustment)

• The second equation measures the likelihood that authors will make explicit policy recommendation in the paper

• Thus, we are jointly modelling two binary choice equations, the one for policymakers’ choice and the other for the authors’ choice

Estimation results

• We find that authors’ origin has significant impact indicating that foreign authors are more likely to make explicit policy recommendations

• Research characteristic that most notably appear to stimulate authors to make explicit recommendations is interpretation

• On the other hand, the use of data and econometric methods seem to make authors less prone to make explicit recommendations

• The existence of explicit policy recommendations, no doubt, has major impact on the likelihood of policy implementation

• Environmental factors have mostly marginal significance, with the exception of election year and new government

• Namely, the policymakers seem to significantly less likely to modify policy in the election year

• On the other hand, policy modification is significantly more likely when the new government is in the office

Conclusions & policy implications• We found a very strong link between explicit recommendations and

implementation, suggesting that papers that include explicit policy recommendations and not merely a suggestion favouring a particular policy action appear to also have higher chance of having their suggested policy modifications implemented, we modelled a possibility that factors affecting the likelihood of implementation also facilitate writing of clear and explicit policy recommendations

• Among the research characteristics, we find very strong and positive effect of clear interpretation of the results which apparently facilitates both implementation and formulation of clear and explicit recommendations

• Furthermore, the findings suggest that the use of econometric methods, graphics, and tables has no direct effect on the implementation probability, however these methods seems to adversely affect the formulation of explicit recommendations. This leads us to conclude that the use of more advanced research techniques does not necessarily increases the implementation likelihood, moreover, through the negative effect on the formulation of recommendations it might be affecting implementation likelihood negatively

• Therefore, it is apparent that policymakers in the transitional countries, at least over the last decade, tend to dislike quantitative data and methods, the more so as their complexity grows higher

• Overall, this paper extended the study of the research-policy links beyond the case-study and descriptive approaches to cross-country and cross-time generalisations that simultaneously look into factors and influences contained in the research evidence (policy papers, studies, etc.) and into the political and economic environment

• We aimed to answer which research approaches, methods, and other specific (environmental) factors affect the likelihood of implementation of a policy change or adjustment suggested in relevant research papers, studies, and analyses, and subsequently, are the characteristics of research pieces such as use of particular methods, interpretation of the results or clarity of policy recommendations likely determinants of the likelihood that the policy option a particular paper supports will be implemented