Edhec Position Paper on B-School Ranking

Business school rankings and business relevance, an overlooked dimension

July 2011

Stéphane GregoirDirector of the EDHEC Economics Research Centre, EDHEC Business SchoolResearch Director, EDHEC Business School

EDHEC BUSINESS SCHOOLECONOMICS RESEARCH CENTREEVALUATION OF PUBLIC POLICYAND REFORM OF THE STATE

393 promenade des Anglais06202 Nice Cedex 3Tel.: +33 (0)4 93 18 32 53Fax: +33 (0)4 93 18 78 40E-mail : [email protected]

2

Newspapers, academic or political institutions regularly promote their own assessment of the relative performances of a particular subgroup of higher education institutions conveyed through a one-dimensional ranking. This generates much comment from students, politicians and Directors of the ranked institutions. These rankings differ in the surveyed populations, their methodology and their objectives which are not always thoroughly presented. In the nineties, they were marketing operations to sell special issues of newspapers or simple lists compiled by academics to help parents and future students make their choice, they now have gained a political importance as a key piece of information used by funding agencies, regional and national politicians in their designing of new policies as well as heads of higher education institutions in their strategic choices. In such circumstances, their legitimacy has been questioned. They have come under close scrutiny and are today the subject of research works.

Academic rankings are relevant and informative for comparable institutions and foster competition between them. Nevertheless, they do not seem clearly related to teaching quality and do not deal with efficiency issues. Most of these rankings are conservative and have a backward-looking and self-fulfilling bias. They today are primarily a measure of reputation which is a slow cumulative process and reinforce it.

We focus here on Business School rankings and illustrate that within the informational content of the available rankings, the key dimension of the business relevance of the research activity and curricula has not been addressed. Nonetheless, the business relevance of the research and curricula is a

key element in a student’s choice. Research activity is necessary for professors to stay at the edge of their fields and improve their teaching, but students are more users and interested in practical implementations of new concepts or methodologies. Attending courses delivered by researchers in which new relevant concepts, new relevant methodologies or simply the current questions in the fields are introduced is a career accelerator for the student. Aware of the current debates in their fields, graduates can select appropriately new opportunities. Moreover, on the one hand, such a teaching allows for a swift dissemination of new practices which is socially beneficial and on the other hand, taking into account this dimension in rankings would introduce some forward-looking dimension.

We illustrate here the feasibility of such a business relevance oriented ranking based on citations in influential international business newspapers and magazine. Such a ranking brings additional information with respect to already existing rankings. To help users to make their choice or analyse the relative performances of business schools, we encourage the compilation of new rankings with explicit alternative goals and recommend the simultaneously use of several of them. A Business oriented citation ranking can be one of these.

Summary

The work presented herein is a detailed summary of academic research conducted by EDHEC. For more information, please contact Joanne Finlay of the EDHEC Research Department at [email protected] The ideas and opinions expressed in this paper are the sole responsibility of the author.

3

About the Author

Stéphane Gregoir is associate dean for research and head of the Economics Research Centre. His research relates principally to macroeconomics and econometrics, and more specifically to time-series methods. He has also worked on the theoretical analysis of anticipations formation or developed methods for dating the economic cycle. Winner of the Tjalling C. Koopmans Prize, Stéphane Gregoir has been editor, co-editor, and member of the editorial committee of various international academic journals. He has doctorate in applied mathematics (social sciences) from Université de Paris IX.

Table of Contents

4

Summary .................................................................................................................................................. 2

Introduction ............................................................................................................................................ 5

1. Rankings: Contents and Possible Uses ......................................................................................... 6

2. About the Business Relevance of Research .............................................................................. 13

Conclusion .......................................................................................................................................................16

Appendices .......................................................................................................................................................17

References.........................................................................................................................................................23

Position Papers and Publications of the EDHEC Economics Research Centre ......................24

5

Introduction

Rankings of higher education institutions are regularly disseminated in newspapers or on websites throughout the year. Each month, newspapers and academic or political institutions promote their own assessment of the relative performances of a particular subgroup of institutions conveyed through a single ranking. This generates much comment from students, public representatives and directors of the institutions ranked. The scope of these rankings can be international1 or national2. They differ in the populations surveyed, their methodology and their objectives, which are not always thoroughly presented. In the nineties, the rankings were marketing operations to sell special issues of newspapers or simple lists compiled by academics to help parents and future students. They have now gained political importance as key information used by funding agencies, regional and national public representatives in designing new policies, as well as heads of higher education institutions. Their legitimacy has thus been questioned. They have come under close scrutiny and are today the subject of research into their role and content. Academic rankings are relevant and informative for comparable institutions and foster competition between them. Nevertheless, they do not seem clearly related to teaching quality and do not deal with efficiency issues outputs with respect to inputs. In this regard, most of the rankings available today should be used cautiously by policymakers. Most of these rankings are conservative and have a backward-looking and self-fulfilling bias. Today, they tend to be a measure of reputation, which is a slow, cumulative process, and reinforce the reputation that already exists. We propose here to focus on business school rankings and illustrate that among the various informational content of the available rankings, the key dimension of the business relevance

of the research activity and curricula has not been addressed. Nonetheless, these elements are key ones for students’ choices. Research activity is necessary for professors to stay at the cutting edge of their fields and improve their teaching, but students are primarily users and interested in the practical implementations of new concepts or methodologies. Attending courses delivered by researchers in which new relevant concepts, new relevant methodologies or simply the current questions in the fields are introduced is a career accelerator for the student. When they are aware of the current debates in their fields, graduates can select new opportunities appropriately. Moreover, for one, such teaching allows also for swift dissemination of new practices, which is socially beneficial, and for another, taking this aspect into account in rankings would introduce a forward-looking dimension. We illustrate here the feasibility of such a business relevance-oriented ranking based on citations in international business newspapers and magazines and the fact that it brings additional information with respect to already existing rankings. To help users to make their choice, we encourage the compilation of new rankings with explicit alternative goals and recommend the simultaneous use of several of them. A business-oriented citation ranking could be one of these.

1 - For instance Times Higher Education Supplement, Forbes, Financial Times, The Economist, Shanghai Jiao Tong University Institute of Higher Education, Leiden University, Ecole des Mines-ParisTech, CHE – Centre for Higher Education Development (Germany) or the Sherpa Network (European project U-map, U-Multirank).2 - in France: Le Point, Challenge, L’Étudiant, Le Figaro…, in the U.S.A: U.S. News and World Report, Forbes,..., in the UK: The Guardian, The Times, Sunday Times,…

6

1.1 Target beneficiaries of rankingsFirst of all, it is important to stress that the information included in rankings is generally a subset of the information collected by national agencies in charge of accreditation (to grant degrees) or by non governmental agencies such as AACSB or EFMD which rely on in-situ audits. The large amount of information collected in these accreditation processes is used to compose very large classes of institutions (5-year accreditation, 3-year accreditation, no accreditation), which does not favour the story-telling sought by newspapers. It is included in their reports or their executive summaries, the reading of which may be too demanding for uninformed parents and students in comparison with a reader-friendly ranking, but a simple ranking device nevertheless cannot satisfy all the needs and may be misleading.

Indeed, there are various points of view associated with each stakeholder, from students to funding agencies, which are associated with different criteria of interest. These needs cannot be satisfied with a single one-dimensional ranking. Among the various beneficiaries and their respective interests, we can list (i) students and their parents who are mainly interested in the quality of the teaching and relevance of the curricula, conveyed by alumni employability and job offers, and in the whole cost (fees and living expenses) with respect to expected wages; (ii) professors, who are looking for institutions with bright students and high-level faculty or want to improve their bargaining power in wage negotiation with their dean when their professional success may have some impact on their business school’s ranking; (iii) deans in the ranked institutions, who want to promote their own institution; (iv) firms, who want to hire well-trained and efficient graduates; (v) regulatory

authorities, watchdogs, trustees, boards and funding agencies, who have to assess the efficient use of resources.

1.2 The growing influence of rankings National and international rankings have grown in importance over the last ten years. Their success may be due to their reader-friendliness and the fact that they filled a gap, but even if they are not able to convey the complexity and variety of situations related to their users’ various objectives, they play an important role and shape the environment in which all the stakeholders take their decisions. Rankings are binding and structuring. They focus on a set of elements that clearly partake in the institution’s performance: quality of the faculty, quality and selection process of the students, wages and the amount of job offers for those who graduate, etc. They allow for rough comparability and foster in a certain sense competition between institutions. For instance, the growing interest of North American or Asian business schools in Masters in Management degrees challenges European business schools that are used to delivering this kind of degree and favours quality improvement in their design and content. All in all, rankings basically allow for a description of relative improvements between competing institutions but through the lens of an easy-to-use synthetic measure of a large set of dimensions in the higher education process. The relevance of this particular lens can be questioned as well as the consequences of its use in the decision process of students, deans or funding agencies. Let us first describe the lens.

1.3 Informational content of rankingsWhile the general principles that govern

1. Rankings: Contents and Possible Uses

7


the compilation methods are generally known, the objectives of the measurement are rarely clearly stated and the detailed procedure is not known. Rankings try to capture various dimensions such as efficiency, reputation and influence. Therefore to understand the nature of the information conveyed by a particular ranking, users need to know the list of all the ingredients that are involved and their relative importance in order to make up their mind. This list is not always presented in detail, so assessing the information conveyed by a ranking cannot be done without a thorough and demanding analysis since all sorts of dimension are covered. From a pragmatic point of view, the selection of the measurements must rely on their appropriateness with respect to the objectives. They must be reliable and feasible. This means that the measurement must be representative of a well-defined situation for which good quality data is available and must allow for comparisons amongst countries and organisations. Without any indication on the exact objective associated with each measure, users must use their own acumen to try to determine why such a measure was taken into account.

Among the usual pieces of information collected are (i) ex-post assessments by students of teaching and faculty quality or by business school clients in the case of executive education, (ii) academic quality of faculty (percentage of PhD professors, international awards, highly cited researchers, number and quality of academic papers, citations, memberships of editorial boards), (iii) international openness (percentage of foreign-born faculty or students, number of languages taught, part of the programme taught abroad, number of joint degrees), (iv) services (career centre placement, professor/student ratio, budget share per student, library, database, data

processing, student mentoring), (v) return on investment (ROI) with assumptions on salaries waived, cost of debt, future salaries, etc., (vi) salary or change in salary of the graduate some years later, (vii) position and size of the graduate’s firm, change in comparison with the professional situation before entering the programme, (vii) percentage of employed some months after graduation, (viii) role and efficiency of the alumni network, (ix) positions of alumni: CEOs, members of boards of international firms.

We can split these categories into three main classes: inputs relating to the students, inputs relating to the educational process, and outcomes; to which we have to add items such as size, turnover, peer opinions, percentage of female members of faculty or the board, percentage of foreign-born members of the board, equipment, catering, sports facilities, private sector share of funding, national and specialised accreditations. This list of additional items illustrates the diversity of possible points of view and the fact that not only outcomes, but also inputs and environmental variables selected on the basis of some preconceptions are taken into account. Each item affects how the ranking is read. For instance, when size is taken into account (e.g., in the ranking compiled by Shanghai Jiao Tong University’s Institute of Higher Education), the influence of the national administrative organisation and the age of the institution are indirectly introduced into the ranking. Regarding the role played by preconceptions, the idea that the percentage of female members of faculty or the board (a percentage that changes as time goes by) is an objective description of a key input into the higher education process or the management of the institution is at the very least not scientifically grounded. It seems to be more related to political correctness. This

8


may be on average a reasonable objective in terms of representativeness of different approaches and sensitivities but the quality in a broad sense of a faculty body results in general from a delicate balance between big egos (of both sexes). A very low percentage figure of women or men is questionable, but in general, it cannot be linked to better fulfilment of objectives for a given institution. Moreover, a ratio close to 50% will be more easily attained by a very large faculty than by a small one, so a size criterion is implicitly introduced by this item.

Rankings cannot only be based on outcomes of the training process. At the very least, quantity and quality inputs and student’s incurred costs have to be taken into account if we want to compare institutions of different sizes and organisation and capture some efficiency dimensions. Comparing institutions with comparable quantitative inputs and size would certainly be easier and more straightforward but the populations might be small. Nevertheless, the role played by inputs is different from that of outputs. The way extensive measures related to inputs enter the ranking, naturally affects its interpretation. In a standard situation and when the focus is on efficiency, in the case of equal outcomes, the institution with the smallest consumption of inputs should have a better rank. If the one-dimensional synthetic measure increases linearly with some extensive measures of inputs, the ranking is not really appropriate for comparing institutions in efficiency terms but is trying to capture something else, which may sometimes be difficult to define. For instance, if the students’ average GMAT score is included in the general score (e.g., in The Economist MBA ranking), ceteris paribus, institutions with the brightest students (in GMAT score terms) have better ranks. On the one hand, this measurement conveys that the institution’s

reputation allows it to be selective. This is an indirect piece of information about reputation. On the other hand, this measure might be used to capture peer effects. In the economic literature, size of peer effects in the class-room on student achievements is still debated (inter alios Angrist and Lang (2004), Burke and Sass (2008), Graham (2008)). There might be an externality generated by the quality of the students that makes the school more appealing, but, first, the influence of peers depends on the classroom size and the quality of the professor and, second, to benefit from a peer effect, there must be heterogeneity in the classroom. The average GMAT of the bottom quartile or the interquartile distance would then be more appropriate to capture this phenomenon. Finally, this GMAT score could be an indirect indication that some schools have high academic standards, but it should be complemented by the dropout or success/failure rates, which are never taken into account. Similarly, in France or in the U.S.A., some magazines’ rankings use students’ achievements before entering the school (for instance, l’Etudiant in France and U.S. News & World Report in the U.S.). Introducing this kind of measurement in the ranking score again provides information about the reputation of the school but does not inform on its curricula and teaching. The result may be that heterogeneity amongst the graduates in top tier schools is not too large, which is valuable for employers, but does not provide information by itself on the relevance of their training. This reputation effect contributes to the value of a particular degree. This may increase its signalling value on the labour market, but does not necessarily lead to a beneficial level of investment in human capital.

As indicated above, academic research activities and quality of faculty may be other kinds of inputs. Some rankings do not take this dimension into account (The

9

Economist for instance disregards this point and limits it to the percentage of PhD professors, with a weight of 3.5%). Other institutions attribute substantial weight to this item in the global score (20% in the Financial Times MBA and EMBA rankings, 26% in l’Etudiant for instance). Their associated measurements are various. They mix the percentage of professors with a PhD, the number and quality of scientific publications, and sometimes more practitioner-oriented publications are taken into account. The rankings provided by The Economist and the Financial Times are substantially different.

If we first consider the 75 business schools that are ranked by both newspapers, their rank correlation (Kendall rank correlation statistic) is about 0.44, which means that they value the components of business school performance differently3. To complete this comparison, we can compare business schools’ rankings regarding their research activity measured by the number of publications in 24 frequently-cited academic journals available on the University of Texas at Dallas (UTD) website4. The rank correlation with this academic ranking is 0.24 for The Economist ranking (86 business schools are present in both


3 - A synthetic table of the criteria used by both rankings (and their relative weights) is presented in the Appendix.4 - The list of journals considered by the Financial Times is larger than that used by the University of Texas at Dallas.5 - FT Global MBA ranking 2011, FT research rank, The Economist’s MBA ranking 2010 and Academic ranking from the School of Management University of Texas at Dallas (2009-2011)

Business Schools FT The Economist UTD

London Business School 1 19 18

Pennsylvania, University of - Wharton School 1 8 1

Harvard Business School 3 4 2

INSEAD 4 23 11

Stanford Graduate School of Business 4 7 9

Hong Kong University of Science and Technology 6 52 14

Columbia Business School 7 12 13

IE Business School 8 22 218

IESE Business School - University of Navarra 9 5 197

Massachusetts Institute of Technology - Sloan School 9 13 16

Indian Institute of Management – Ahmedabad 11 85 257

Chicago, University of - Booth School of Business 12 1 4

Indian School of Business 13 Not ranked 124

IMD - International Institute for Management Development 14 6 218

New York University - Leonard N Stern School of Business 15 14 3

Yale School of Management 15 24 46

China Europe International Business School (CEIBS) 17 100 210

Dartmouth College - Tuck School of Business 18 2 45

HEC School of Management, Paris 18 9 53

Duke University - Fuqua School of Business 20 28 8

ESADE Business School 21 20 388

Northwestern University - Kellogg School of Management 21 16 6

National University of Singapore - The NUS Business School 23 84 38

University of Michigan - Stephen M. Ross School of Business 24 25 7

University of California at Berkeley - Haas School of Business. 25 3 22

University of Cambridge - Judge Business School 26 30 105

University of Oxford - Said Business School 27 71 93

SDA Bocconi School of Management 28 65 388

Manchester Business School 29 61 159

Table 15 : Comparison of FT and The Economist MBA rankings (from the FT population) and UTD research ranking

Source: Financial Times, The Economist, University of Texas at Dallas

rankings) and 0.20 for the Financial Times ranking (88 business schools are present in both rankings). Academic research is not considered by The Economist. Their ranking is based on career opportunities, educational experience, changes in salary and potential to network, but it presents comparable links with the research output captured by the UTD ranking as that of the Financial Times rankings, which take this dimension into account (weight of 20%). Notice that both rankings allot larger weights to changes in salary (cf. Table 5 in Appendix). A recent academic paper

illustrates a positive correlation between business school research and their students’ earnings after graduation (O’Brian et al. (2010)). This means that renowned business schools often have a research reputation aligned with their global reputation.

Both rankings are particularly different when considering the leading business schools in each ranking. In Tables 1 and 2, we present the ranking of the thirty leading MBA programmes according to the Financial Times with the ranks obtained in The Economist for the same programmes

10


Business School The Economist FT UTD

Chicago, University of - Booth School of Business 1 12 4

Dartmouth College - Tuck School of Business 2 18 45

University of California at Berkeley - Haas School of Business 3 25 22

Harvard Business School 4 3 2

IESE Business School - University of Navarra 5 9 197

IMD - International Institute for Management Development 6 14 218

Stanford Graduate School of Business 7 4 9

Pennsylvania, University of - Wharton School 8 1 1

HEC School of Management, Paris 9 18 53

York University - Schulich School of Business 10 49 62

University of Virginia - Darden Graduate School of Business 11 41 96

Columbia Business School 12 7 13

Massachusetts Institute of Technology - MIT Sloan School of Business 13 9 16

New York University - Leonard N Stern School of Business 14 15 3

Cranfield School of Management 15 34 n.a..

Northwestern University - Kellogg School of Management 16 21 6

Henley Business School 17 Not ranked n.a.

University of Southern California - Marshall School of Business 18 64 10

London Business School 19 1 18

ESADE Business School 20 21 388

Carnegie Mellon University - The Tepper School of Business 21 41 33

IE Business School 22 8 218

INSEAD 23 4 11

Yale School of Management 24 15 46

Michigan, University of - Stephen M. Ross School of Business 25 24 7

Mannheim Business School 26 Not ranked 146

Hult International Business School 27 61 n.a.

Duke University - Fuqua School of Business 28 20 8

Bath, University of - School of Management 29 Not ranked 197

Cambridge, University of - Judge Business School 30 26 105

Tableau 2: Comparison of FT and The Economist MBA rankings (from The Economist population) and UTD research ranking

Source: Financial Times, The Economist, University of Texas at Dallas

11


and, conversely, the thirty leading MBA programmes according to The Economist with the ranks obtained in the Financial Times. Their UTD rank is also introduced in these tables. As they differ in the populations they consider, both points of view can be illustrated. If we first consider the thirty leading business schools in each ranking, their rank correlation (Kendall rank correlation statistic) is about 0.29 when considering the Financial Times population and not significantly different from 0 when considering The Economist population. They similarly have a different rank correlation with the University of Texas at Dallas academic rank. When considering the thirty leading business schools selected by the Financial Times, the rank correlation with the UTD ranking is 0.24 and for the thirty leading business schools selected by The Economist, the rank correlation with the UTD ranking is not significantly different from 0.

These tables illustrate, first, that the selection of the population of business schools is a key element in a ranking. Second, different choices in the criteria and their relative weights may lead to similar correlation with a ranking with a unique objective. This emphasises the difficulty in stating exactly what kind of information is conveyed by these rankings. Third, as the Financial Times takes the research output into account (number of articles published in 45 academic and practitioner-oriented journals), we can compute the rank correlation between the Financial Times research rank and the UTD rank: it is greater than 0.9. Even if more practitioner-oriented journals are present in the FT list, the two rankings are close. This echoes AACSB’s assessment about the weak incentives given to faculty to produce practice-oriented research as schools usually take journal rankings into account when awarding tenure. We propose in the second section to

focus on this relevance for business of the industry-related activity of business school faculty (practice-oriented work, patents, industrial partnerships, members of boards of international firms, professional use of research outputs, number of registered cases). We close this first section with some comments about the limits and drawbacks of rankings.

1.4 Limitations and drawbacks of rankings First, the presentation of rankings may be misleading. Rankings seem to be the result of an accurate and rigorous approach, but in order to derive them it is necessary to render comparable very different quantitative measures that rely on a substantial set of assumptions. For instance, when comparing wages in different countries, not only purchasing power parity but also social security contributions levied on wages have to be taken into account. They vary largely depending the country and may have a non-negligible impact. Similarly, when producing a return on investment measure, the salaries waived during the period of attendance at training programmes as well as the implied extra cost of living must be estimated. Assumptions have to be discussed or presented in detail, but are not. They require extensive information and technical skills. On the other hand, information may be qualitative or quantitative and must be made commensurate which gives importance to convention and data manipulation. The coding of qualitative variables must be linearly related if a z-score transformation is used in practice to homogenise them. If this is not the case, different codings of the same qualitative information will lead to different contributions to the synthetic score. The natural consequence is that for subgroups of institutions, ranks are not significantly different (from a statistical point of view) and a fair presentation of


12

the ranking should provide the user with classes of comparable institutions. If this is not done, robustness checks should be provided. Moreover, most of the time, data is partly collected on a voluntary basis, which may imply some bias (particularly if the disseminated ranking can affect the career or salary of the person surveyed). Checking on the information provided by the institutions is costly and cannot be carried out on a regular basis. Finally, even if we consider that the set of weights that combine the various measures is relevant, the available information on the compilation method is not detailed enough and the complete dataset is not made public in order to allow the rankings to be reproduced.

Second, by the particular selection of general criteria, rankings favour general management programmes and general management research themes. Specific, high-quality programmes or research in a particular field outside the mainstream are naturally underrepresented. This favours large institutions with a long history and makes the choice to develop a niche strategy costly in terms of visibility. There are incentives for schools to adapt their offer to the criteria used in the rankings to improve their score. Rankings therefore tend to favour uniformity and limit the innovativeness of institutions, possibly at a collective cost.

Third, rankings are mainly backward-looking. They measure the current situation of former students and emphasise dimensions relating to the alumni network’s influence and past resources. The larger they are, the better the ranking will be. This conservative bias is reinforced by a fourth limitation. Rankings are self-fulfilling. They induce a set of incentives which reinforce their importance and can reduce institutions’ incentives towards innovativeness. By

disseminating non-significant differences between institutions, they induce a selection of applicants even when the ranking is not related to teaching and pedagogical dimensions (Bowman and Bastedo (2009), Espeland and Sauder (2009)). Their importance is also reinforced because, first, past rankings are used to reply to subjective appraisal questions by peers, students and employers in new surveys (Espeland and Sauder (2009), Bastedo and Bowman (2010)) and, second, because they are used by employers to set salary levels depending on the institution of origin, while salaries are frequently an input of the rankings. Similarly, as they are used by national offices for students going abroad, they contribute to reinforcing the quality of students in well-ranked institutions. All in all, rankings frequently reflect an institution’s reputation, which is the result of slow and progressive process, and are likely to contribute to its reinforcement.

Research activity is necessary for professors to stay at the leading edge of their fields and improve their teaching. Nevertheless, students are users of research and interested in practical implementations of new concepts or methodologies. Attending courses delivered by researchers in which new relevant concepts, new relevant methodologies or simply the current questions in the fields are introduced is a career accelerator for the student. Since they are aware of the current debates in their fields, graduates can grasp new opportunities more easily. Such teaching also allows for swift dissemination of new practices, which is socially beneficial. Moreover, taking such a dimension into account in a ranking would introduce a forward-looking dimension which is lacking today.

An AACSB report in 2007 stirred a lot of interest. Its objective was to question the place and value of research in business schools. Following extensive criticism (e.g. Jeffrey Pfeffer and Christina Fong (2002) or Warren G. Bennis and James O’Toole (2005) inter alios), the AACSB task force delivered several recommendations, among which the recommendation that schools should be required to demonstrate the value of their faculty’s research by listing its citations in journals as well as measuring “the impact of scholarship of all types” (“to inform teaching and learning, advance knowledge of theory, keep faculty aware and involved in issues of current interest, and improve aspects of management practice”) “on various audiences important to business schools.”

The AACSB report (2007) lists three main channels through which research in business schools can have an impact: the rigorous development of new concepts and methods in articles that are published in academic peer-reviewed journals and

monographs and debated in seminars, the promotion of these new methodologies and concepts in professional journals, business press or large audience books as well as professional events, and finally the illustration and pedagogical presentation of these new concepts and methodologies in cases and textbooks used in the business school curricula. More precisely, in their Table 1, page 14, the authors list various contributions to practice such as • articles in professional or trade journals or magazines, • publicly available technical reports for organisational projects, • chapters in professional or trade books, • significant contributions to trade journals or magazines authored by others, • significant presentations at trade meetings, etc. This list could be completed with indirect measures of the influence of faculty on industry such as the ones considered in some rankings: patents, industrial partnerships, memberships of boards of international firms, professional use of research outputs, number of registered cases. If patents are not relevant for business schools, industrial partnerships could be proxied by the percentage of resources coming from private firms when they can clearly be related to joint research projects (research chairs with the industry) and do not correspond to donations from alumni. Fiscal incentives are nonetheless different from one country to another and this may blur the signal. The number of alumni members of boards or in a CEO position is used in rankings such as the one provided by the “Ecole des Mines-Paristech” engineering school. This approach is nevertheless backward-looking and dependent on the economic institutions. On the one hand, CEOs are usually professionals who received their degree at least twenty years sooner and have several degrees from various institutions

2. About the Business Relevance of Research

13

whose relative influence on their career can be disentangled with difficulty. On the other hand, this also may be affected by a home bias. In a centralised country with an influential administration with close links to large companies, members of powerful administrative bodies who graduated from the same schools can occupy high-ranked positions.

Measuring the impact of research is not an easy task as time is needed to disseminate new ideas and convince busy people of their usefulness. We propose here to proxy

this impact by compiling the number of citations of research or professors in the influential business media as well as articles written by faculty published in these very media. The idea is that space in newspapers is expensive and, due to competition, journalists want to provide appropriate information to their readers. Five international business newspapers have been selected; some of them are involved in the compilation and dissemination of a business school ranking. We carried out a census of all the articles written by and quotations of


14

Business School FT rank (2010) Citation rank

London School of Economics and Political Science 26 1

City University: Cass 12 3

London Business School 2 3

University of Oxford: Saïd 11 4

Imperial College Business School 18 5

University of Cambridge: Judge 42 7

Cranfield School of Management 15 8

Insead 3 9

IMD 4 10

Lancaster University Management School 35 10

Nottingham University Business School 62 11

University of Strathclyde Business School 20 11

EDHEC Business School 25 12

Warwick Business School 21 14

TiasNimbas Business School, Tilburg University 24 16

Universität St.Gallen 16 16

University of Edinburgh Business School 62 19

Ashridge 34 20

SDA Bocconi 17 20

University College Dublin: Smurfit 30 20

Iese Business School 9 21

Manchester Business School 53 21

Henley Business School 58 22

Rotterdam School of Management, Erasmus University 6 22

Copenhagen Business School 23 24

Stockholm School of Economics 19 24

HEC Paris 1 25

University of Cologne, Faculty of Management 60 25

Esade Business School 8 26

Bradford University School of Management 33 29

Table 3: International Business School Citation ranking and Financial Times ranking

Compilation by EDHEC Business School from Dow Jones Factiva database and EBSCOHost Database.

professors from a list of business schools, published in 2010 in Business Week, The Economist, the Financial Times, the Wall Street Journal, the Wall Street Journal Asia. Two rankings were compiled: the first based on the number of citations, the second on the number of articles written by faculty. A linear combination of both rankings was then computed with equal weights. This survey was carried out for all the European business schools ranked in the Financial Times using the Dow Jones Factiva Database and EBSCOHost Database. Such a ranking can be compiled by theme (finance, economics, etc.) and extended to professional or trade journals or magazines. This first attempt aims at illustrating its feasibility and properties (reproducibility and informational content). Table 3 below presents the ranking obtained for the thirty leading European Business Schools.

The ranking we obtain is clearly different from the one produced by a publication that belongs to our set of international publications used to proxy the business relevance of the research carried out by faculty. This means that its informational content is clearly different. This can be illustrated with some figures such as rank correlations. The Kendall rank correlation of both rankings over the population of 75 business schools is 0.34. When we consider the Master in Management

ranking, 58 business schools are ranked and the correlation is not significantly different from 0. It amounts to 0.39 for the full-time MBA ranking. This new ranking illustrates a dimension of performance that was not taken into account up to now.We can as evoked above compile such a ranking for particular fields. In Table 4, we present with the same approach the ranking we obtain when limiting our interest to economics and finance. The citation ranking is modified slightly, illustrating the domains of expertise of the different schools.


15

Citation ranking Business Schools FT ranking

1 London School of Economics and Political Science 26

2 City University: Cass 12

2 London Business School 2

4 EDHEC Business School 25

5 University of Oxford: Saïd 11

6 Cranfield School of Management 15

7 Imperial College Business School 18

8 Insead 3

8 Lancaster University Management School 35

10 IMD 4

Table 4: Citation ranking (2010) in Economics and Finance

Compilation by EDHEC Business School from Dow Jones Factiva database and EBSCOHost Database.

Rankings of higher education institutions are becoming increasingly influential. Nevertheless, a unique ranking cannot satisfy all needs: from those of students to those of policymakers. They are relevant and informative for comparable institutions and foster competition between them. Nevertheless, they are not clearly related to teaching quality and do not deal with efficiency issues. Most of these rankings are conservative and have a backward-looking and self-fulfilling bias. Today, they are a measure of reputation (which is a slow cumulative process) and reinforce the reputation that already exists.

Introducing a forward-looking dimension into these rankings and fostering competition between business schools in relation to the relevance of their research and the contents of their curricula seems to be a socially beneficial objective. We illustrate here the feasibility of such a ranking based on citations in a selection of international business publications and magazines. Such a ranking seems to bring additional information, not present in the diverse existing rankings. To help users to make their choice, we encourage the compilation of new rankings with explicit alternative goals and recommend the simultaneous use of several of them. A business-oriented citation ranking could be one of these.

Conclusion

16

We consider five newspapers or magazine that are part of the ten most read business publications in the world, namely• The Financial Times• The Economist• The Wall Street Journal• The Wall Street Journal Asia• Business Week (Bloomberg)

To analyse their content and count the articles related to each Business School ranked by the Financial Times, we used the Factiva Dow Jones and the EBSCOHost databases to find all the citations and articles written by their professors.

Factiva Dow Jones is one of the leading providers of global business news and information with content from 31,000 sources, “from Factiva” is a business information and research tool owned by Dow Jones & Company.

EBSCOhost databases and discovery technologies are the most-used, premium online information resources for tens of thousands of institutions worldwide, representing millions of end-users. More precisely, the contents of the Financial Times, the Wall Street Journal, the Economist and the Wall Street Journal Asia were searched in Factiva Dow Jones database and that of Business Week in EBSCOHost database.

For each school ranked by the Financial Times, we searched for occurrences of its name in articles published by the five newspapers or magazine we selected. Every article in which one occurrence appears was read to define if it was a research citation, a research article or a corporate quotation.

Appendices

17

Indicator The Economist (Full MBA) FT (Full MBA)

Inputs relating to the Students Student achievement 8.8% 0%

Student diversity 5.8% 6%

Inputs relating to the Education process

Fulfilled goals (student assessment)

17.4% 3%

Academic research 3.5% 20%

Industrial relevance 0% 0%

International openness 4,4% 8%

Process 12.7% 2%

Outcomes Salary, employment 37.5% 54%

Alumni network 10% 0%

Other 0% 7%

Table 5: Criteria used in the compilation of the FT and The Economist rankings

Compilation by EDHEC Business School from Dow Jones Factiva database.

Appendices

18

Financial Times rankings of European Business Schools in 2010Information about these rankings can be found on:http://rankings.ft.com/businessschoolrankings/rankings

Source Financial Times

19

Appendices

By way of illustration of the international audience of these journals we provide the reader with some pieces of information. We first consider the international audience as given by Worldwide Professional Investment Community Study 2009/10 (PIC) which deals with the world’s most senior Professional Investors.

Overall, PIC provides insights of the composition of the professional investment community worldwide as well as measuring media consumption of this audience. The Erdos & Morgan Worldwide Professional Investment Community study has been conducted biennially since 1987.

The eleventh in the series, the survey targets a universe of individuals from the buy-side in companies with a minimum of US$100 million under management serving one of the following five functions: (Analyst - Chief Investment Officer - Director of Research - Head of Trading - Portfolio Manager)

Global Reach - Print Air

59%40%

35%

30%28%

34%34%

24%24%23%

22%16%

12%12%

The Wall Street Journal (WWW)The Economist

Financial Times

Bloomberg Markets

Institutional Investor

Business Week

Barron’s

Forbes

The New York Times

Fortune

Financial Analysts Journal

Pensions & Investments

Investo’s Business Daily

Time

The WSJ, The Economist, the FT hold the 3 first positions on the ranking. Business Week holds the 6th position.

Source: PIC 2009/10 [Universe 41,386 – Sample 3,956]

Appendices

20

Websites (visited yesterday)

33%29%

26%

16%16%

10%8%

7%6%

5%

4%4%

17%

4%

3%

wsj.com (WW)bloomberg.com

finance.yahoo.com

ft.com

nytimes.com

cnbc.com

google.com/finance

online.barrons.com

reuters.com

seekingalpha.com

morningstar.com

economist.com

CNNMoney.com

marketwatch.com

europe.wsj.com

wsj.com, Bloomberg.com (for business Week) and ft.com hold the 4 first positions on the ranking.

Source: PIC 2009/10 [Universe 41,386 – Sample 3,956]

We then turn to the European audience measured by a survey from Business Europe Elite entitled ”Europe media consumption”. The survey represents the 453,000 top executives in Europe, in charge of €3,000 billion corporate budgets.(http://www.ipsos.com/mediact/sites/ipsos.com.mediact/files/pdf/Ipsos_MediaCT-WP-ReachingTheBusinessElite.pdf)

BE: Europe - Print and Web coverage

23.2%16.7%

9.4%

5.5%5.5%

4.4%

8.4%9.1%

9.5%12%12%

Ft + ft.comEconomist + economist.com

Harvus Bus Rev + harvardbusinessonline.org

National Geographic + nationalgeographic.com

Bloomberg Markets Mag + bloomberg.com

BusWeek + busweek.com

Time + time.com

Newsweek + newsweek.com

WSJE + wsj.com

IHT + iht.com

Forbes + forbes.com

% coverage

Source: Business Elite Europe 2010 (Ipsos Mori) - AIR and dotcom monthly reach

Appendices

21

Reviews

The Financial Times

The Wall Street Journal

The Economist Business Week

Date of founding

1888 8 juillet 1889 September 1843 1929

Founders Harry Marks Charles Dow, Edward Jones and

Charles Bergstresser

James Wilson

Country United Kingdom United States United Kingdom United States

Topic Economy Economy and Finance

Economy and international affairs

Economy

Worldwide print circulation

381 658 2 000 000 1.473.939(July-December 2010 ABC)

923,457

Periodicity Daily Daily Weekly Weekly

Website www.ft.com www.wsj.com http://www.economist.com www.businessweek.com

Audience http://fttoolkit.co.uk/2011mediakit/survey_results.html

http://sales.marketwatch.com/newspaper/about/

audience

http://www.economistgroupmedia.

com/research/audience-profile/

Source: 1- http://www.courrierinternational.com/sources_overview 2 - http://www.btobonline.com/apps/pbcs.dll/article?AID=/20110502/MEDIAPOWER/305029990#seenit

Appendices

22

• Angrist J and K. Lang (2004), “Does School integration generate peer effects? Evidence from Boston’s Metco Program”, American Economic Review, vol.94 (5), 1613-1634.

• Bastedo, M. N., and Bowman, N. A. (2010), “The U.S. News and World Report college rankings: Modeling institutional effects on organizational reputation”, American Journal of Education, vol 116, 163-183.

• Bennis, W. G. and J. O’Toole (2005), “How Business Schools Lost Their Way”, Harvard Business Review, May.

• Bowman, N. A., and Bastedo, M. N. (2009), “Getting on the front page: Organizational reputation, status signals, and the impact of U.S. News and World Report on student decisions”, Research in Higher Education, 50, 415-436.

• Burke M. A. and T. R. Sass (2008) “Classroom Peer Effects and Student Achievement”, National Center for Analysis of Longitudinal data in Education Research, working paper n°18.

• Graham B. S. (2008), “Identifying Social Interactions Through Conditional Variance Restrictions”, Econometrica, 73 (3), 643-660.

• O'Brien J. P., P. L. Drnevich, T. R. Crook, and C. E. Armstrong (2010) “Does Business School Research Add Economic Value for Students?” Academy of Management, Learning and Education, 9(4), 638-650.

• Pfeffer, J. and Fong C.T. (2002) “The end of business schools? Less success than meets the eye”, Academy of Management Learning and Education, 1(1), 78-95.

References

23

Position Papers 2011• Courtioux, P., S. Gregoir. L’investissement public dans l’enseignement supérieur remet-il en cause l’équité fiscale ? (February 2011).

• Chéron, A. L’évolution de la formation professionnelle continue : une perspective internationale (January).

Position Papers 2010• Palomino, F. Peut-on rendre les stock options versées aux dirigeants plus efficaces ? (October).

• Courtioux, P., and S. Gregoir. Les propositions de l’EDHEC pour réformer l’enseignement supérieur : les contrats de formation supérieure (September).

• Amenc, N., Chéron, A., Gregoir. S., Martellini, L. Il faut préserver le Fonds de Réserve pour les Retraites (July).

• Chéron, A. Réformer la protection de l’emploi des seniors pour accompagner l’augmentation de l’âge de départ à la retraite : Que peut-on attendre d’une baisse du coût de licenciement d’un senior ? (May).

• Gregoir, S., M. Hutin, T.-P. Maury, and G. Prandi. Quels sont les rendements de l'immobilier en Ile-de-France ? (May).

• Chéron, A. Faut-il plus protéger les emplois à bas salaires ? (January).

• Courtioux, P. L’effet du système socio-fiscal sur les rendements privés de l’enseignement supérieur (January).

Position Papers 2009• Palomino, F. La parité homme-femme est elle soluble dans les concours ? (June).

• Chéron, A. Réformer l'indemnisation des chômeurs : plus de redistribution et moins d'assurance (June).

• Chéron, A. Quelle protection de l’emploi pour les seniors ? (January).

• Courtioux, P. Peut-on financer l’éducation du supérieur de manière plus équitable ? (January).

• Gregoir, S. L’incertitude liée à la contraction du marché immobilier pèse sur l’évolution des prix (January).

Position Papers 2008 • Gregoir, S. Les prêts étudiants peuvent-ils être un outil de progrès social ? (October).

• Chéron, A. Que peut-on attendre d'une augmentation de l'âge de départ en retraite ? (June).

• Chéron, A. De l'optimalité des allégements de charges sur les bas salaires (February).

• Chéron, A., and S. Gregoir. Mais où est passé le contrat unique à droits progressifs ? (February).

Position Papers and Publications of the EDHEC Economics Research Centre (2008-2011)

24

………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………

………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………

………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………

………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………

………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………

………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………

………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………

………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………

………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………

………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………

………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………

………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………

………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………

Notes

25

………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………

………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………

………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………

………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………

………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………

………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………

………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………

………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………

………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………

………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………

………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………

………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………

………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………

Notes

26

EDHEC BUSINESS SCHOOLECONOMICS RESEARCH CENTREEVALUATION OF PUBLIC POLICYAND REFORM OF THE STATE

393 promenade des Anglais06202 Nice Cedex 3Tel.: +33 (0)4 93 18 32 53Fax: +33 (0)4 93 18 78 40E-mail : [email protected]

The mission of EDHEC is to form students and managers to undertake projects and lead people in a multicultural context. EDHEC offers a wide range of educational programmes meant to meet all of the needs of businesses. Its wide range of degree-granting courses draws students from all over the world. Nearly 5,400 students and 5,500 professionals are enrolled in degree programmes or attending seminars and executive-education courses at EDHEC’s locations in Lille, Nice, Paris, London, and Singapore.

As part of its international strategy, EDHEC has put in place an innovative policy of doing research for business and set up six research centres. EDHEC, regularly ranked among the top European business schools, has AACSB, AMBA, and EQUIS accreditations.

For more information see: www.edhec.com

Since February 2006, EDHEC has had a research centre in economics devoted to the evaluation of public policy and government reform. The aim of the centre is to do innovative and applied research that will make EDHEC a source of academically recognised expertise on themes critical to the French economy.

Currently, the EDHEC Economics Research Centre has a team of ten full-time and affiliated researchers and professors working on themes centred along two axesone, the themes at the intersection of financial and economic issues, and the other linked to the French welfare state, in particular to the labour market and to education.

Copyright © 2011 EDHEC

Edhec Position Paper on B-School Ranking

Documents

Transcript of Edhec Position Paper on B-School Ranking