1.return on
Transcript of 1.return on
Return on investment (ROI) inGerman libraries
The Berlin School of Library and InformationScience and the University Library at the
Humboldt University, Berlin – a case study
Kathrin GrzeschikHumboldt Universitat zu Berlin/Berlin School for Library and Information
Science, Berlin, Germany
Abstract
Purpose – The purpose of this paper is to verify the proposition by the University of Illinois,Urbana-Champaign (UIUC), that their return on investment (ROI) formula developed for academiclibraries and based on hard facts is broad enough to be used throughout the world for ROI studies inacademic institutions/libraries. It further aims to verify that UIUC’s methodology is adaptable enoughto work in other academic environments as well.
Design/methodology/approach – The methodology developed by UIUC (an ROI formuladeveloped for academic libraries based on grant proposal applications and citations) has been“copied” and thereby adapted to enable it to be used in an academic environment in Europe/Germany.
Findings – The methodology developed by UIUC was adaptable enough to be used in a Germanacademic environment for calculating the ROI of a University library. However, the methodology wassometimes complicated and therefore simplified for this and possible further studies. Likewise, the ROIformula was very complex and this study found that it was possible to simplify it as well for furtheruse.
Research limitations/implications – There was difficulty in gathering all the informationnecessary for conducting such a study in Germany as grant proposals contain sensitive data thatpeople are unwilling to display. Further, it was noticeable that German statistics on funding wereunable to provide the necessary data without further enquiries, despite the German law that publicinstitutions are obliged to disclose funding information.
Originality/value – Previously no one else has tried to verify the methodology for an ROI studydeveloped by UIUC. This study gives evidence that UIUC was right in claiming that their ROI formuladeveloped for academic institutions/libraries may be used for any academic library in the world.Further, this study shows how the formula and the methodology may be adapted to fit individualacademic environments.
Keywords Return on investment, Libraries, Germany
Paper type Case study
IntroductionThe following paper describes the experiment to implement the return on investment(ROI) concept developed by the University of Illinois, Urbana-Champaign (UIUC) (seeKaufman, 2008a or Luther, 2008) for the field of LIS (Library and Information Science)at the University Library of the Humboldt Universitat zu Berlin (HU)[1] and the BerlinSchool of Library and Information Science (IBI) (www.ibi.hu-berlin.de/). It investigatesthe principal ideas and methodology developed by UIUC and applies those in a German
The current issue and full text archive of this journal is available at
www.emeraldinsight.com/0888-045X.htm
ROI in Germanlibraries
141
Received May 2010Revised June 2010
Accepted June 2010
The Bottom Line: Managing LibraryFinances
Vol. 23 No. 4, 2010pp. 141-201
q Emerald Group Publishing Limited0888-045X
DOI 10.1108/08880451011104009
academic environment. One of the features Illinois claimed for their study, was thechance for other institutions to use and copy the main steps.
Quantitative and qualitative measurements have been used in public, corporate andacademic libraries before, but this paper focuses on ROI as a quantitative method toevaluate a library’s monetary value. The research question is: “Can the methodologydeveloped by UIUC be applied to German universities?” It is not the aim of this study tocome up with a true ROI figure for the IBI and the HU, but the testing and evaluation ofthe UIUC method; nevertheless, all calculations aim to achieve as true of a possiblefigure as can be determined.
The institutions that are part of this study are both well established in highereducation in Germany, The IBI is one of the two European members of the iCaucus[2].Since the middle of the past decade, more and more grant proposals have been issuedby the IBI, which makes a study based on the ROI study of the University of Illinois,Urbana-Champaign feasible, as the core of the ROI study by UIUC are grant proposals.The university library at the Humboldt University, Berlin (UB) was founded in 1831,and currently has an inventory of 6.5 million media[3]. It sprawls over several sectionlibraries, but the collection for LIS is situated at the main library in the newly-builtJacob-und-Wilhelm-Grimm Zentrum in Berlin-Mitte.
Literature reviewThe “gain from investment” is not easily determined by libraries. The literature reviewwill demonstrate the lack and need of a continued investigation in ROI in academiclibraries in Germany, as well as the speciality of the concept developed by UIUC.
ROI is a financial metric performance measure to calculate what a certaininvestment is worth. Very simply put, ROI is how much one gets back for what one hasput into something. “To calculate ROI, the benefit (return) of an investment is dividedby the cost of the investment; the result is expressed as a percentage or a ratio”[4].A positive percentage or ratio indicates that more benefit than cost has been generated;a negative percentage or ratio indicates less benefit was generated. In order to calculateROI, one needs to be able to quantify how much money was invested into somethingand compare it with the loss or gain that is a result of handling the initial investment.ROI is a well-used performance measure in fields such as human resources, marketing,engineering, business studies, and training[5]. A problem libraries have to deal with isthat they are not capitalistic corporations, but exist to support their specific clientelewith information and technology. This is true for any library: the difficult part for aROI study in libraries is the assignment of monetary values for library services.
Among the uses of ROI in libraries, differences are discernible between the aimsaddressed in studies:
Quantitative measurements for a special library’s ROI include time saved by library users(. . .); the money users save by using the library instead of alternative sources; and revenuegenerated with the assistance of the library (Strouse, 2003, para. 9).
Commonly-used methods to filter monetary values for non-profit organisations are:. “assessing time costs (replacement value of a client’s time)”; and. “the contingent valuation method” (Poll and Boekhorst, 2007, p. 36).
BL23,4
142
The ROI study by UIUC focused on revenue generated with the assistance of thelibrary in question.
The first method is valuable when average salaries of users are known, as itcalculates time costs with the help of average salaries. These measures may thentranslate into actual dollar savings; nonetheless, the method is not feasible in anacademic surrounding, as most users of academic libraries are students who do notearn a regular salary.
The second method, contingent valuation (CV), is a technique that assigns economicvalue to non-market resources with the help of surveys. It has been around a while, butwas quite controversial until it gained a better reputation when scientists discussed themethod at a National Oceanic and Atmospheric Administration (NOAA) Panel in 1993.They found it meaningful when a set of rules was followed[6]:
[. . .] [B]y means of carefully designed surveys it is possible to elicit, in quantitative terms,how much people value a particular organisation or service (British Library, 2004, p. 4).
Other publications followed, for example Mitchell and Carson (1993) claimed toprovide:
[. . .] decision makers, policy analysts, and social scientists with a detailed discussion of a [. . .]technique for the valuation of goods not traded in private markets. [. . .] the technique drawsupon economic theory and the methods of surveys to elicit directly from consumers the thevalues they place upon public goods (Mitchell and Carson, 2000, p. xv).
These surveys ask participants how much they would be prepared to pay (WTP –willingness to pay), and how much they would be willing to accept for loss of quality oflife (WTA – willingness to accept). Since this panel, the method has been used incost-benefit analysis concerning the environment as Arrow et al. (1993) did, in realestate, and lately libraries have adopted this method as well.
As people (users and potential users) receive indirect (and direct) benefits fromlibraries, CV is a valuable method to use in the library environment:
Applied in a library/information context the methodology enables consideration to be givento the cost implications of having and of not having a library service. The direct economiccontribution of the library to their users is calculated including investment in terms of timeand travel (Missingham, 2005, p. 146).
Libraries employing CV as a method assign financial value to intangible products. Themost famous CV study performed in the area of library and information science is theBritish Library study from 2004[7]. The surveys used in the British Library CV studyasked questions such as “how much beneficiaries would be willing to pay for thelibrary’s continued existence” and “how much they would invest in terms of time andmoney to make use of the library” (British Library, 2004, p. 4). After evaluating thesurveys, the research organisations found that “[f]or every £1 of public funding theBritish Library receives annually, £4.40 is generated for the UK economy“ (BritishLibrary, 2004, p. 5).
It is difficult, however, to establish objective economic value based on surveys thatask people for estimations. Even the British Library (2004) indicates in its summarythat this method is seen as one of the most appropriate available, but it is still not anexact science. UIUC set their goals to find a more tangible ROI method that could besummarised in a formula adaptive enough to be used by other academic libraries and
ROI in Germanlibraries
143
institutions. Apart from that, the UIUC-study is the first ROI study based in anacademic setting. The British Library is a national library and their study was not setin such an environment; therefore, it is less adaptable to be used in an academic setting.
The interest in ROI and performance measurement in general has generated hugeinterest in the past decade. Missingham (2005) spoke about a third wave of ROI studies.Demonstrating that libraries are worth their expenditures is not a new topic forlibrarians. Librarians are no longer secure in their assumptions that the impact of alibrary will be recognised by the communities served, or the authorities that fund them.“As librarians we naturally tend to think our libraries and our services are invaluable.”(The Krafty Librarian, 2008, para. 1). But it is rather the opposite that is believed byacademic as well as public communities and their funding bodies.
One of the major reasons for the strengthened demand for value is the paradigmshift of media processed and stored in libraries. The shift has been induced by themassive changes in technology development in the past two decades. Libraries havebeen very adaptive, and included each new technology as it came along, but criticshave often voiced to librarians the familiar question: “Why do we need libraries wheneverything is on the internet now.” It is common knowledge among library staff thattechnology is altering the way information is created and thereby technology iscreating new ways of how information is distributed; nevertheless, this informationneeds to be prepared and made available the same way printed information has been.What has changed is the perception of the public and of funding bodies, “librariescannot take for granted, if they ever could, that they have a monopoly over theprovision of information” (Payne and Conyers, 2005, p. 1). This shift in thinking as wellas economic problems have lead to an accumulation of calls from funding bodies forproofs of value. Aabø (2009) conducted a meta-analysis on what she calls the“subgroup of library valuation that returns a return on investment or a cost-benefitratio”:
Public libraries [. . .] need to prove how the taxpayers’ money is used to benefit both theindividual citizens and the local communities. Academic libraries, school libraries, andspecial libraries in different businesses meet similar types of demands, being asked forperformance measurement, cost justifications, and return on investment from theadministration of their university, school, or enterprise. These demands have beenstrengthening due to increasing economic pressure (Aabø, 2009, p. 312).
Aabø emphasised that, owing to the financial crisis, the need for library valuationgrows stronger.
Libraries in the private as well as the public sector have a slightly longer tradition inproving their value with the help of ROI, compared to academic libraries. Althoughcollecting, analysing, and presenting quantitative and qualitative data has been part ofa library’s pursuits for a longer period of time, only in the past few years it has shiftedfrom “simple questionnaires to complex surveys, and from simple economiccost/benefit assessments to complex economic algorithms and forecasts” (Imholzand Weil Arns, 2008, p. 5). Only recently have German libraries realised thatperformance measurement is not only an internal task but an external one: “[. . .] dasBewusstsein fehlte, wofur Bibliotheken und Bibliothekar/innen eigentlich da warenund wie sie die Unterstutzung der Offentlichkeit [. . .], gewinnen und behalten konnen,[. . .] heute erst hat sich das geandert!” (Busch, 2004, p. 13)[8].
BL23,4
144
In the 1990s Holt et al. (1996) researched into frameworks for evaluating publicinvestment in public libraries. They emphasised that “economic impact occurs onlywhen a business’s or institution’s activities bring outside clients to the region, therebybringing new dollars into the region, or when an institution attracts fiscal support fromoutside the region for its activities”. (Holt et al., 1996, p. 3). They strengthened thenotion that libraries are among those institutions that are able to have economicimpact. A 2004 ROI study by the State Library and Archives of Florida showed “thatwhen Florida’s state and local governments invest in libraries, it enhances the qualityof life in communities and helps build a stronger state economy” (Griffith, 2004,Overview). Academic libraries supporting academic institutions that bring newdollars, or in this case Euro, into their institutional region, have the chance to havesimilar economic and qualitative impact as described above by Holt et al. and Griffith.
Public libraries have been especially creative in finding solutions to raise awarenessfor ROI regarding their library. Aabø found that “of the 38 studies [evaluated in hermeta-analysis], 32 are of public libraries” (Aabø, 2009, p. 311). Browsing the web pagesof public libraries showed many Anglo-American ones creating their own ROIcalculators. An example is the Library of Michigan ROI calculator[9] that enables taxpayers to calculate their own ROI when using the library[10]. Further discoveries werepublic libraries doing surveys, listing their studies’ results or uploading videos onYouTube[11].
ROI calculators by public libraries help customers to get an insight into their ROI ontax money spent when using a library; academic libraries need to focus on the ROI fortheir funding bodies. Poll and Payne (2006) demonstrated a variety of purposes andmethods used, again mostly by public libraries. They distinguished among othersbetween the “correlation of library use and academic or professional success [. . .]impact on information literacy, the importance of the local library on research [. . .]social impact [. . .] [and] the financial value of libraries” (Poll and Payne, 2006, p. 6).Two of their findings are particularly important for this study: first, they emphasisedthat measuring the financial value of libraries is the most interesting to fundinginstitutions. Despite the social importance of libraries and their impact on informationliteracy, what counts for funding bodies is the money made. There is evidence for thistheory even in Germany. German universities and their libraries are recognized moreand more for the monetary value they bring their communities. For example, theGerman Exzellenzinitiative[12] considers third-party funds in their benchmark ofparticipating universities; naturally, that makes German universities interested in themonetary surplus or deficit their libraries are producing. Second, Poll and Payne (2006)mentioned two studies that show that the impact of local libraries on research hassuccessfully been measured with the help of citations analysis. Both studies evaluated“what percentage of the material cited was (or could have been) retrieved via the locallibrary” (Poll and Payne, 2006, p. 7). Additionally, they conducted surveys to supporttheir assumptions. The study at UIUC started in a similar way, but went a step furtherand added real economic value to citations by calculating ROI with the help of citationanalysis in grand proposals.
There have been studies on performance measurement in Germany as well, but veryfew in the academic sector and even fewer considering ROI as a method. In 2003 asimilar tendency was seen in the Anglo-American academic field of LIS:
ROI in Germanlibraries
145
Precious little attention is currently given to development and collection of ROI data. Amongthe three primary types of special libraries (corporate, academic, government), corporatelibraries are most likely to study their value impact, and academic libraries are least likely(Strouse, 2003, para. 3).
Later international outcome-studies such as “Worth Their Weight” conducted byImholz and Weil Arns (2008) showed that opinions and trends, at least among thepublic library sector in continental America, have changed. This development giveshope for a new trend in Europe. Imholz and Weil Arns presented seventeenUS-American studies by public libraries punctuating current approaches and methodsused by contemporary scholars and researchers. Nevertheless, two years earlier Blanck(2006) surveying performance measurements in (mainly public) libraries in Germanywas correct to claim that “Im deutschen Bibliotheks- und Informationsbereich findethierzu bisher so gut wie keine Diskussion statt.” (Blanck, 2006, p.13)[13].
One exception in the growing field of evaluations of public library ROI studies (seeMissingham, 2005; Blanck, 2006; Imholz and Weil Arns, 2008; Aabø, 2009) is Blanck’spredecessor Fett (2004), who collected literature on performance measurement inacademic libraries. He devoted a short chapter to the topic of ROI and lists only theValue-Added Library Methodology (V þ LM) by Ruth MacEachern. This methodologyis based on CV and results from “eine Ermittlung von Schatten- oder Quasipreisendurch Erhebungen bei den Nutzern“ (Fett, 2004, p. 42)[14]; additionally, replacementcosts are calculated. As previously mentioned, the problem of CV and V þ LM are theoften very subjective monetary values assigned to library services that are then usedas hard facts.
Designing an ROI measurement based on hard facts and figures is a sensible taskfor the German field of academic libraries. UIUC’s study provides a new way tocalculate ROI for libraries based on hard facts instead of assumptions (see for examplethe British Library study). UIUC, however, had to gather certain figures with the helpof surveys. These minor approximations are amended in this study due to its smallerscale and the possibility to count and individually evaluate citations.
The making of the UIUC return on investment studyIn 2007, researchers of the Library at the University of Illinois, Urbana-Champaigntogether with Judy Luther, president of Informed Strategies at Elsevier and Dr CarolTenopir, Professor at the University of Tennesee at Knoxville, conducted the ROIstudy that is the basis for this study. The starting point of their study was the commonperception that academic libraries are in need of a tool that is easy to use for answeringthe demands for accountability from funding institutions. The research group aimed athaving numbers rather than figurative values for proving what their university librarygives back to their funding institutions, “[. . .] it is important to demonstrate thatinvestment in the library yields a ‘return’ that contributes to the strategic goals of theinstitution” (Kaufman, 2008a, p. 29). Their aim was to find a formula that wouldanswer the question “for every dollar invested by the University in the library, theUniversity received x dollars in return” (Kaufman, 2008a, p. 30). The ground-breakingapproach was that they did not plan to develop a model based on predictions as is thecase with the CV method, but they planned to base the formula on real figures. In orderto have real figures, they were in need to find a connection between the library and
BL23,4
146
strategic concerns of the university that could be expressed in quantifiable terms.Their goals were to:
[. . .] demonstrate that the library and its research collections contribute to [essential]income-generating activities [. . .], quantify the return on the University’s investment in itslibrary, highlight the library’s role in the extra-mural funding process on campus [and to]demonstrate correlation between the library and grant activities, rather than attempt to provecause and effect (Kaufman, 2008a, p. 30).
Roger Strouse’s (2003) approach for a ROI-study in corporate libraries was theapproach chosen by UIUC for the development of a generic model that is not predictiveand works in the academic environment. Strouse developed a formula that determinesvalue based on “revenue generated with the assistance of the library [. . .] [and] theimportance of information provided by the library that the user would not have foundor had access to without the library’s intermediation” (Strouse, 2003, para. 8).
His formula (adapted by Judith Luther (Kaufman, 2008a, Slide 7)) was to multiplythe percentage of respondents generating revenue with library’s support by thepercentage of instances when the library was used and thereby revenue generated.This was multiplied by the median revenue generated in US dollars. This would leavea certain amount of US dollars generated per library use (Kaufman, 2008b, p. 430). ThisCorporate Library Model by Roger Strouse (adapted by Judith Luther) is shown below:
xx% of respondents report generating revenue w= library’s support
£ xx% of instances when library was used £ $xx median revenue generated
¼ $xx average revenue generated per library use
The research team in Illinois decided that grant proposals are revenue generatingincomes in an academic setting that are created with the help of library resources. Inorder to confirm that assumption, Illinois distributed an online survey among 2,000members of their faculty. The survey elicited how users perceived the library and itsresources when constructing grant proposals. “Almost 75% of respondents stated thatmore than three-quarters of the citations they included in their grant applications wereaccessed through the library” (Kaufman, 2008a, p. 31). After evaluating the surveyresults, calculating the ROI for Illinois’ university library based on the use of citationsin successful grant proposals was confirmed as a reasonable method.
Calculating the ROI with the total library budget, the researchers determined a ROIof $4.38 for every dollar invested in the library, for the fiscal year 2006 (Kaufman,2008a). This result is based on surveys, i.e. evaluating “percentages of staff usingcitations in their grant proposals” and budgets provided i.e. “the actual size of grantsawarded ” by the finance department (see Table I).
The ROI was calculated by dividing the proportion of grant income in US dollarusing library materials by the total library budget (see Table I, E). The researchers atUIUC decided to use the total library budget instead of for example the materialsbudget or a subset of the budget (i.e. electronic resources), because staff and supportcosts are also part of acquiring materials that are used for citations. Nevertheless, toshow the difference, they calculated the ROI with the materials budget of UIUC libraryand it resulted in a ROI of approximately $12. It is important to note that:
ROI in Germanlibraries
147
[. . .] [f]aculty survey results factor into the equation in three separate places. In this way, themodel does not assume that all grant proposals use references, it does not assume that that allreferences come from the library, and it does not assume that citations are deemed critical toall grant proposals (Kaufman, 2008b, p. 433).
The differing amounts of citations per grant proposals in this study show, that UIUCwas right in assuming the above, but the model designed by UIUC is a feasible versionof including revenue generating actions supported by the library (citations in grantproposals (see Appendix 1)) into a ROI calculation for academic libraries.
Designing an ROI study for the Berlin School of Library and InformationScienceWhen initially planning to adopt the University of Illinois ROI study for a Germansetting, a study larger in scale and more similar to the Illinois ROI study was intended.It was planned to conduct the study for the Niedersachsische Staats- undUniversitatsbibliothek Gottingen[15] and its partner the Georg-August-UniversitatGottingen[16]. These institutions were possible candidates for a German ROI study,because they represent a major research university and library. They are major playersin receiving grants from the Deutsche Forschungsgemeinschaft (DFG)[17] as well asthe Bundesministerium fur Bildung und Forschung (BMBF)[18]. What the author didnot consider was the sensitive nature of data grant proposals contain; collecting thenecessary data was an impossible hurdle. In addition, the data that were available inthe end for running a ROI study comparable to the one developed by UIUC haveincompatible categories that cannot readily be disentangled in the time available. Theproblems and their possible consequences will be noted in the sections below.
No. tenure system faculty 2,045No. principal investigators 1,700 *Survey Q11-94% faculty use citations in
grant proposalsA ¼ % faculty using citations in grantproposals *
78.14 (1,700 £ 94%)/2,045
No. grant proposals 2,897 * *Survey Q12-94% proposals includecitations that are obtained via campusnetwork/Library Gateway
No. grant awards 1,456 * *Survey Q10-95% faculty statecitations important or essential in grantawards
B ¼ % proposals inc. citations obtainedthrough library
50.79 (1,456 £ 95%)/(2,897) £ 94%)
$ Average size grant 63,293C ¼ $ proportion of grant $ secured usinglibrary materials
25,369 (78.14% £ 50.79% £ $63,923)
No. grants (expended) in year 6,232D ¼ $ proportion of grant income usinglibrary materials
$158,099,608 ($25; 369 £ 6; 232)
$ Total library budget $36,102,613E ¼ University return in grant $ onlibrary
$4.38 ($158,099,608/$36,102,613)
Source: Kaufman (2008a)Table I.UIUC ROI calculations
BL23,4
148
To narrow down the scope and receive usable material, the author chose to examinethe Berlin School of Library and Information Science (IBI) and the Library andInformation Science section at the university library. The Berlin School for Library andInformation Science is the only LIS-research-orientated academic institution inGermany that actively participates in current state-of-the-art research. In past years,the IBI has actively participated in grant proposals applied for in cooperation withother major research universities, as well as applying on its own. Researching theactivities of other LIS-orientated academic institutions with the help of the DFG searchengine GEPRIS[19] showed no results. Neither of the major Universities of AppliedScience[20] had any projects listed in the field of LIS that were funded by the DFG inthe past five years.
By choosing a smaller institution than the University of Illinois Urbana-Champaignand its library for this study, the author had to adjust the longitudinal scope. In orderto have a substantial enough amount of grant proposals, it was extended to threeconsecutive years (2006-2009).
In total, 13 grant proposals applied for by the IBI and partners have been awardedwith grants in those three years. The total budget of these conglomerates wase3,190,502.72 (see Appendix 2 Grants Calculations). This budget is a compositenumber compiled with the help of budgets forecast in the grant proposals. Anapproximate number had to be established because official budget listings forthird-party funds vary a lot, and unfortunately the listings by the HU[21] are notsufficient for this study. The above figure has been composed of average wage costs(salary and fringe benefits) as well as travel costs, print and publishing costs, plusmiscellaneous costs listed in the grant proposals.
The average salary and fringe benefit costs were calculated on the basis of apersona[22] being just under 30 years of age and being paid on BAT IIa level 1[23].BAT IIa is a standard level for a scholarly member of staff working on a researchproject at the HU. The age plays an important part, as BAT scales income based onmaturity. For example, a 26-year-old person would earn a gross income of e2,293.30 permonth plus residence allowance and further allowances based on marital status andchildren. A person having the same conditions but being 30 years of age would earn agross income of e2,523.37 per month plus allowances simply because of being fouryears older. The basic idea was that people being older have had more experience andare therefore better qualified. This way of calculating easy changed from April 2010with a new collective wge agreement called TV-L (Tarfivertrag fur den offentlichenDienst der Lander)[24], but the average salary and fringe benefits will not changesignificantly because of those changes.
The salary and fringe benefits are not the amount such research project employeesreceive in their paychecks, but the amount of money the institution actually spends onthem. In Germany, institutions support their employees by paying proportions forcertain benefits such as health insurance, pension, unemployment insurance, andsometimes accident en route. These proportions are paid directly to the respectiveorganisation responsible for the benefit or the state. For the above-described persona( just under 30 years of age, level 1 BAT IIa) it amounts to e3,492 per month. Thisamount correlates with expenditures for one person-month. Person-month is a measureof work effort, for example if a project will take three months to finish with three people
ROI in Germanlibraries
149
working full time on it, the project requires 3*3 ¼ 9 person-month effort. That wouldimply e31,428 in salary and fringe benefits costs at the HU for such a project.
As mentioned above, UIUC conducted a survey. They found out that 94 percent ofproposals include citations that are obtained via a campus network or library gateway.This survey, however, is the weakest part of the UIUC ROI study. A response rate of 16percent is not seen as low, but a higher response rate would have been better, becauseUIUC based their percentage of proposals including citations obtained through thelibrary on the survey results. Certain factors determine the statistical confidence ofsurveys, these factors are for example the size of the population or the degree ofvariance in responses from the population. All the same, Bennekom (2002) states thereis no average response rate, and UIUC had Dr Bruce Kingma of Syracuse Universityconsulting their research methodology. Due to the smaller scale of this study, a surveywas not deemed necessary. It was possible to evaluate the citations individually toretrieve a more accurate proportion of grants in Euro secured by using librarymaterials; the number was acquired by calculating the (%) proportion of citationsobtainable through the library.
The starting-point was to find and investigate each citation in the successful grantproposal. All references made to a published or unpublished source, includingwebsites, were identified as citations and either found as footnote, link in the full-text,or traditionally in the bibliography. References in German grant proposals are usuallyadded in the section “Ausgangslage/Eigene Vorarbeit”[25], but the variety of proposalsfor the differing funding bodies showed that simply searching through this section wasnot enough; each proposal was searched from top to bottom. For the establishedreferences the focus was the availability either through the Online Public AccessCatalogue (OPAC) or the digital library accessible via Virtual Private Network (VPN).The OPAC of the UB is connected with the Kooperativer BibliotheksverbundBerlin-Brandenburg (KOBV)[26]. Loan sharing is one of the features KOBV (n.d.)facilitates among its participants, and any citations available via loan sharing weretherefore counted as belonging to the UB.
In the 13 grant proposals, 474 individual citations are discernible. Individual meansthat some proposals cited the same source twice, for example as a proper citation in thereferences list and as a link in the full-text; in this case, the citation was counted as oneonly. For 190 citations, its source could be traced back via the materials provided bythe UB. This means either the source being a physical or electronic item at the UB, orthe source being obtainable via loan sharing and thereby listed in the OPAC. 284citations were purely internet-related sources; quite a few of those were websitescreated by other projects led by HU members and/or institutions.
The number of citations per proposal varied a lot as well. Some projects had a verylow number of citations, others comparatively very high numbers (see Appendix 3Citations). The project LuKII (LOCKSS-und-KOPAL-Infrastruktur-und-Interoperabilitat)with the lowest amount of citations cites only seven sources, whereas the projectDistributed Open Access Reference Citation Services (DOARC) cites 103. Grandproposals applied for at the DFG are supposed to cite or add literature references, but“Der Antrag sollte nicht mehr als 20 Seiten umfassen und aus sich heraus, auch ohneLekture der zitierten oder beigefugten Literatur, verstandlich sein”[27]. Therefore theamount of citations seem to be individually determinable. Owing to the survey resultsgathered by UIUC, citations in grant proposals were assessed as valuable
BL23,4
150
measurement for revenue generated by the respective library. This assumption isbased on perceptions by the tenure system faculty. The question is if this is true forGerman grant proposals as well. The vastly differing amount of citations persuccessful grant proposal for the IBI suggests that citations might not carry such animportance for success or failure in Germany. Comparing the two studies in this matteris impossible, however, as UIUC did not count the individual citations and no surveywas conducted for this study. Hence for this study, UIUC’s assumption of value forcitations in grant proposals is taken as fact.
Some of the internet-based sources cited in the grant proposals link to projects bythe HU, those that are very closely linked to the IBI and which sources are found via theUB catalogue have been counted as belonging to the UB (i.e. DINI – Deutsche Initiativefur Netzwerkinformation e.V)[28]. Those that are linked to the HU by being a projectbut whose sources are not (yet) obtainable via the library are counted as internet-basedsources.
Some grant proposals cite internet-based sources more often than once; for exampleIUWIS lists press releases by “Borsenverein des Deutschen Buchhandels” in itsreferences list as well as in its footnotes. Sources listed twice were counted once only,but sources linking to differing locations on one website were counted twice. Thepercentage of citations obtainable via the university library were calculated accordingto above descriptions. This adds up to 40.08 percent citations that are obtainablethrough the UB (see Table II).
As briefly mentioned above, calculating the percentage of citations obtainable via thelibrary by counting and evaluating the citations single-handedly is a different way ofutilising the results as UIUC purported. With the help of data gathered with the surveyamong tenure system faculty, UIUC determined what percentage of faculty is usingcitations in grant proposals. Afterwards, they established the percentage of proposalsincluding citations obtained through the library, by multiplying the grants awarded withthe percentage of faculty stating that citations are important or essential in their grantsawards. This figure was then divided through the grant proposals made multiplied withthe percentage of proposals, including citations that are obtained via the library (seeTable I, UIUC ROI calculations). As said before, by counting and evaluating the citationsindividually, the established figures are more accurate and allow a more precisecalculation of the ROI. One has to keep in mind, however, that UIUC would have toevaluate the citations of 1,456 successful grant proposals for 2006 alone. Given thedifferences in scope, other institutions working with the UIUC method need to analyse ifcounting and evaluating costs more or less time and effort than executing a major survey.
The grant proposals evaluatedBetween 2006 and 2009, 13 grant proposals applied for by the IBI were successful andhad grants awarded by either DFG, BMBF, or the European Union (European
Total amount of citations 474Amount of citations obtained through UB 190Proportion of citations obtained through UB in allproposals (%) 40.08 (190/4.74)
Table II.Citations obtained
through UB (library) inproposals (%)
ROI in Germanlibraries
151
Commision/i2010)[29]. (for more details, see the paragraphs on each grant proposalbelow or Appendix 3 Citations). When looking at this appendix the comment “notapplicable” might catch someone’s eye, or the fact that there are more running numberslisted than citations in the total count. Not all footnotes were counted as citations. If thetext provided in the footnote was additional information regarding the proposal but nota link or reference, the information was not counted as citation.
As said before, the third-party funds statistics available at the HU website are notsufficient for the purpose of this study; the HU statistics do not segment third-partyfunds into individual years per institute for more than the last accounting year(currently 2008). The funds expended are split between the institutions receivinggrants, but the previous years are summarised as third-party funds expenditures forthe entire HU. This fashion of listing funds forced individual calculations for eachproposal to be made.
The budget calculations themselves proved to be challenging. For example, thetravel budget intended for DOARC was split up between the three participatinginstitutions. This information posed the question of how detailed the calculations weregoing to be. If only the travel costs allotted to the HU were calculated, the costs fortravel, publishing, and miscellaneous costs would minimize to e13,932.72 instead ofe23,132.72 in total. Initially this way seemed to be the right way to go, as this study isconcentrating on the IBI and the UB. Further considerations, however, indicatedsplitting up travel costs like that would imply splitting up the other costs as well. Thisin turn would demand a level of interrogation of applicant bodies that is not possible.In addition, it is not traceable which partner added which citation to the grant proposal.If the costs would be split up between the participating institutions, the citations wouldhave to be split up too. This approach is too highly structured and not feasible simplybecause of applicants not remembering their citations. Under US practice, it is quitepossible that grant money for one institution is also used to fund travel for people atother universities. Trying to tease out the amounts for other partners is not necessarilyimproving the comparability of data. Nevertheless, it is important to understand thatthis is a ROI study for the IBI and not for the other institutions, as the ROI is calculatedon the basis of citations obtainable via the UB and the materials budget for the UB.
The budgets of grant proposals applied for on a European level posed anotherproblem. Total budgets for European projects amount to millions; for example,EuropeanaConnect has a budget of e4,798,149 (without own contribution)[30] of whichthe HU receives only e191,986 (without own contribution). If this study were to use thetotal budgets allocated for European projects, as done for the DFG or BMBF fundedprojects, the final ROI figure would be distorted immensely and thus lead to seriousmisrepresentations. Additionally, the amount of e191,986 allocated to the IBI forEuropeanaConnect as well as the budgets of the other two European-based projects,correspond with the budgets calculated for the non-European projects. Although thisstudy does not aim at calculating a precise ROI number for the IBI and UB, but atreconstructing the UIUC study in a German academic setting, the figures should be asclose as possible to reality and provide an approximate result in the end. Hence, for allEuropean-based grant proposals only the budgets (without own contribution) allocatedto the IBI were used for calculating the total budget.
BL23,4
152
IUWISInfrastruktur Urheberrecht fur Wissenschaft und Bildung (IUWIS)[31] is a two-yearproject applied for by Professor Michael Seadle from the IBI and Professor RainerKuhlen from the University in Constance (Universitat Konstanz)[32]. It is based at theIBI and funded by DFG.
IUWIS has calculated to need 60 person-month on BAT IIa level or equivalent,creating a salary and fringe benefit budget of e209,520. Additionally e46,464 areneeded for four student assistants each working 40/h per month for both years. Costsfor printing, travel, as well as miscellaneous costs are summed up to e55,515; totallingthe grants awarded for IUWIS to e311,500.
The proposal has 28 citations for sources found at the UB and 36 purelyinternet-based ones, summing up to 64 citations in total. Some citations appear twice,once in the footnotes and once in the references; depending on their source they arecounted once or twice. For example, the Gottinger Erklarung[33] is listed as aninternet-based source, as well as a source obtainable through the library because thereferences in the grant proposal point to differing sources established by the GottingerErklarung.
LuKIILOCKSS-und-KOPAL-Infrastruktur-und-Interoperabilitat (LuKII) is one of the newestprojects currently pursued by the IBI. Its topic is the ongoing problem of long-termdigital archiving. Applicants are Professor Michael Seadle (IBI), Professor PeterSchirmbacher (IBI/CMS[34]) and Dr Elisabeth Niggemann (DNB).
LuKII was planned to run for two years and calculated to need 57 person-month onBAT IIa level or equivalent as well as one student assistant working 80/h month forboth years. The average salary and fringe benefits can be estimated at e222,276.Additional costs such as scholarly equipment, travel expenses, and publishing costsare calculated at e58,450; totalling the probable budget to e280,726.
Altogether, LuKII has only seven citations in total. Four citations are for sourcesobtainable through the UB and three are internet-based sources. This is speculative,but maybe because the websites for LOCKSS[35] and kopal[36] provide excellentinformation on each project and further references are not needed. In addition, LuKIIoperates on a new level of digital archiving and relevant sources may not be availableyet.
EuropeanaConnectEuropeanaConnect[37] is a project funded by the European Union working in closecontext with the major European library project Europeana[38]. The applicant at theIBI is Professor Stefan Gradmann; he is one of 31 pan-European applicants.
EuropeanaConnect asks for a total budget of e4,798,149 as contribution from theEuropean Commission. Of this large amount a mere e191,986 (4%) accrue to the IBI. Itis important to note that European projects are always funded only partly by theEuropean Commission, as much as up to half of the funding may come from within theparticipating institutions. In the case of EuropeanaConnect only 25 percent (e1,199,538)have to be contributed by the participating institutions. To incorporate thesedifferences for this study is unnecessary for the final ROI calculations; for eachEuropean-funded project the grants awarded without own contribution are used.
ROI in Germanlibraries
153
EuropeanaConnect has 26 citations; only one is from a source obtainable by the UB.All other citations are links to European websites. This grant proposal differs frommost of the others as it did not contain a separate reference list or high amount offootnotes, but links in the running text.
GalateasGeneralized Analysis for Logs for Automatic Translation and Episodic Analysis ofSearches (Galateas)[39] is a project with eight participants all based in Europe; theapplicant at the IBI is Professor Vivien Petras.
The total budget allotted for Galateas is e3,700,000 (without own contributions).The European contribution is e1,850,000 of which the IBI receives e62,356. This is thetotal budget for one research staff member and a student assistant working for twoyears, including travel costs.
Altogether, the grant proposal for Galateas holds 31 citations of which 19 sourcesare obtainable via the UB and twelve are internet-based ones. Among theinternet-based ones are two journals that have earlier issues available at the UB, butit was not clear if the needed issue is available as well; therefore, these were ascribed tothe internet-based citations as they were available online.
DOARCDistributed Open Access Reference Citation Services (DOARC)[40] is a project of theIBI, the Institute for Chemistry at the HU, the Carl von Ossietzky University inOldenburg, and the Institute for Science Networking in Oldenburg. The applicants areProfessor Peter Schirmbacher (IBI/CMS), Dr Frank Havemann (IBI), Dr WolfgangChristen (HU), Professor Volker Mellert (Carl von Ossietzky University Oldenburg),and Professor Eberhard R. Hilf (Institute for Science Networking).
Not surprisingly, considering the topic, DOARC is the grant proposal with mostcitations. DOARC has 103 citations of which 39 are obtainable via the UB databases orOPAC, 64 citations are internet-based ones. A couple of the internet-based citations linkto open access publications by the applicants, as for example Havemann (2004)“Eprints in der wissenschaftlichen Kommunikation.” One of the citations that werecounted as being obtainable via the UB is not explicitly allocable. DOARC’s citationNo. 51 (see Appendix 3 Citations is a publication by S. Katz in Science and PublicPolicy from 2006. The publication was not traceable at the HU for this specific year butfor the year 2000; nor did the authors’ website provide the publication from 2006. Asthe source from 2000 was at ACM and the double non-appearance of the laterpublication might indicate a typing mistake by the applicants, the citation was countedfor the UB.
The total third-party funds allocated for DOARC sum up to e183,692,72; e160,560are for staffing costs and e23,132.72 for travel, publishing, and miscellaneous costs.
EERQIEuropean Educational Research Quality Indicators (EERQI)[41] is a European researchproject in the 7t77h Framework Programme for Research in the Socio-economicSciences and Humanities Theme (SSH)[41]. The applicant at the IBI is Professor StefanGradman.
BL23,4
154
Altogether, nineteen European institutions are participating in this project, the totalamount of grants awarded are e1,494,624. The Berlin School for Library andInformation Science receives e136,720. As with the other European projects this projectis only partly funded by the European Commission, the total budget is e2,226,323.08and 67,13 per cent are funded. The amount left needs to be contributed by theparticipating institutions.
The grant proposal contains 43 citations; 20 refer to sources obtainable by the UBand 23 are purely internet-based. Like the grant proposal EuropeanaConnect, the grantproposal for EERQI contains many references as links in the full-text. No peculiaritieswere discernible for citations in the grant proposal for EERQI.
ForschungsdiversitatThe full title for this project is “Messung der Diversitat der Forschung”[42], applicantsare Dr Jochen Glaser (Freie Universitat Berlin)[43], Michael Heinz (IBI), and Dr FrankHavemann (IBI). This project aims at creating methods for measuring the diversity ofresearch in various special subject areas and their respective organisations.
The total budget for Forschungsdiversitat is e230,013. This is split up in e103,645for staff, consisting of e88,621 for 18 person-month on BAT IIa level or equivalent pluse17,424 for other staffing costs not further specified. e126,338 are allocated for travel,scholarly equipment, publishing, and other costs. The staff budget for this project wasnot calculated with the help of the persona developed for this study, but taken from thebudget listing provided by the grant proposal. In contrast to grant proposals appliedfor at DFG that list the person-month only, grant proposals applied for at BMBF haveto calculate the person-month and the costs connected. Eighteen person-monthmultiplied with the e3,492 used as median for salary and fringe benefits throughoutthis study sum up to e62,856. This is e25,765 less than the budget for staff costscalculated throughout this study. Forschungsdiversitat has therefore calculated theirbudget with an older person in mind. As this study does not aim to provide an exactROI result, these differences are not important. For a concrete ROI figure, furtherinformation such as what age the actual person working in a position has, as well asthe marital status and place of residence would be necessary; however, this kind ofinformation is deemed to be too private in Germany.
The citation count for Forschungsdiversitat is 48 in total; 41 are citations obtainablevia the UB. This is the grant proposal with the largest amount of citations obtainablevia the UB, the runner up is DOARC with 39 citations obtainable via the UB. Onlyseven citations are internet-based ones. None of the citations found were peculiar.
CARPETThe motivation for CARPET (Community for Academic Reviewing, Publishing andEditorial Technology)[44] was the ongoing process of virtualisation of workingprocesses in the academic area. The applicants are Professor Peter Schirmbacher(IBI/CMS), Dr Norbert Lossau (SUB Gottingen), and Dr Laurent Romary (Max PlanckDigital Library, Berlin)[45].
CARPET (n.d.) was planned for two years and calculated to need 72 person-monthor three members of research staff for two years. Additionally three student assistantseach working 80h/month for two years were estimated, the budget for staff costs ise321,120. Additional e27,820 for travel and publishing raised the total budget to
ROI in Germanlibraries
155
e348,940. This is the highest budget for a DFG-funded project the IBI is involved in, therunner-up is IUWIS with e1,441 less.
Twenty-six is the total count of citations made in the grant proposal for CARPET,with a very high amount of internet-based sources (20) and a small amount oflibrary-obtainable sources (five). Four internet-based resources (DRIVER, eSciDoc,Berlin Declaration, Budapest Open Access Initiative) point to projects the HU isinvolved in.
Meta-ImageArt history is a visual-orientated subject and the digitalisation of scholarly life hasopened new possibilities for research in art history. Applicants are Dr Martin Wanke(Leuphana Universitat Luneburg), Lisa Dieckmann (Universitat zu Koln)[46], andProfessor Peter Schirmbacher (IBI/CMS).
Meta-Image (n.d.) has a total budget of e271,520 composed of e209,520 for staff ande62,000 for travel, publications, and miscellaneous costs. There is no budget forscholarly equipment as these are goods on own account.
The citation count is relatively low, only 14 citations altogether. Nine are citationsfor publications obtainable via the UB and five are internet-based ones. One of thecitations obtainable via the library is not clearly allocable, because it is microfichebased at the UB until 1998 and then superseded by a link to the resources based inMarburg (Bildarchiv Foto Marburg)[47]. The resources provided on Marburg’swebsite, however, are freely accessible for everyone and do not have special accessrights. A test browsing the site with HU VPN and without it showed no differences inpresentation or access; therefore, the citation could have been listed as internet-basedsource as well. I decided to list it as UB source as the microfiche is on-site at the mainlibrary, the website is detectable via the OPAC, and the reference in the grant proposaldoes not refer to a certain year.
MUNIN-RSMUNIN-RS (Entwicklung und Implementierung einer Open-Source-Repository-Solutionfur vernetztes Arbeiten mit wissenschaftlichen Bild- und Multimediasammlungen/Modifiable Universal Image Network – Repository Solution) aims at developing andimplementing a kind of repository-based solution for picture and multimediacollections. The applicant is Prof. Schirmbacher (IBI/CMS).
The total budget estimated for MUNIN-RS is e276,880. Next to e214,080 for staff,the project calculates a very high amount of e53,000 for miscellaneous costs of which e
50,000 are intended for a service contract. A further e9,800 is budgeted for travel.A relatively low amount of citations (12) is split in five citations obtainable via the
UB and seven purely internet-based ones. One of the internet-based sources is awebsite by the HU (Medienportal)[48]; it is counted as an internet-based resourcebecause the Medienportal (n.d.) was not detectable via the OPAC or the digital library.
OANOpen Access Netzwerk (OAN)[49] is the oldest grant proposal of the 13 analysed forthis study. Applied for in 2006 the project was funded for two years by DFG; startingthe assembly of a network of certified open access repositories. The applicants are
BL23,4
156
Professor Peter Schirmbacher (IBI/CMS), Dr Judith Plumer (University of Osnabruck)[50],and Dr Norbert Lossau (SUB Gottingen).
Based on the calculations for the persona developed for this study, e202,464 arebudgeted for staff working 48 person-month on BAT IIa level as well as three studentassistants each working 40h/month for two years. In addition e38,700 are estimated forscholarly equipment, travel expenses, and publications. The total amount calculated ise241,164.
OAN is one of the grant proposals with a very high count of citations forinternet-based sources. Of the 39 counted 31 are internet-based and only eightobtainable via the UB. All but one of the eight are online sources belonging to the UB(i.e. DINI website or edoc-Server). All of these are detectable with the aid of the OPACand therefore counted as sources belonging to the UB. A high amount of internet-basedsources is not remarkable, considering the requirements related to open access that isthe key element of this grant proposal.
OAN2OA-Netzwerk 2 (OAN2)[51] is the direct continuation of the OAN project. The grantproposal for OAN2 was written in 2009 and continues where OAN terminated.Applicants are Professor Peter Schirmbacher (IBI/CMS), Dr Judith Plumer (Universityof Osnabruck) and Dr Norbert Lossau, (SUB Gottingen), in addition Professor StefanGradmann (IBI) has joined the team.
The total budget for OAN2 is e86,858 more than assessed for the previous projectOAN; the sum totals are e328,022. e286,272 are calculated for staffing; 72person-month on BAT IIa level or equivalent and 18 month for a student assistantworking 40/h a month. Further costs add up to e 41,750. OAN2 has the third highestbudget of all grant proposals.
Citations in the grant proposal for OAN2 add up to 17 citations only, of which 13 areinternet-based ones. Only four citations are obtainable via the UB; again, the citationsmirror the OA topic of the grant proposal. One of the internet-based sources links to theMathematics Subject Classification[52] which is available in print via loan sharing. Theprint version is from 2000, however, and the link refers to an up-to-date version, thecitation was counted as internet-based resource and not as belonging to the UB.
DocupediaDocupedia is a project by Professor Peter Schirmbacher (IBI/CMS), Professor WilfriedNippel (Institute for History at the Humboldt University)[53], and Professor MartinZabrow (Zentrum fur Zeithistorische Forschung Potsdam)[54]. It aims at creating asubject-specific organised, dynamically-growing repository containing encyclopaedictexts about contemporary history.
This project was planned to run for three years and funding was applied for twoyears. The applicants calculated for 60 person-month on BAT IIa level or equivalent aswell as two student assistants each working 80/h per month for both years. Theaverage salary and fringe benefit costs can be estimated at e255,984. Additional costsare calculated at e35,000 totalling the approximate budget to e290,984.
The proposal has 45 citations; only seven citations can be found with the help of theUB and 38 citations are purely internet-based ones. Of the latter nine, links to websitescreated by projects the IBI, CMS or HU are involved. Quite a few links direct to the
ROI in Germanlibraries
157
same page but at varying contents. For example arxiv.org[55] is counted twice becauseone link in the grant proposal leads to the main page and the second link to theinformation “How to replace an article in arxiv”[56]. Double references like those arecounted as two separate citations, because they do indicate two different destinations.
Determining the ROIUsually determining the ROI is a simple mathematical, monetary calculation (seeequation below)[57], but as described before, UIUC had to conduct a survey andevaluate certain data to fit into this kind of basic mathematical formula:
ROI ¼ðGain from investment 2 Cost of InvestmentÞ
Cost of Investment
The term “gain from investment” sounds innocent, but as discussed in the literaturereview of this study is rather difficult for libraries, because no direct monetary gainfrom investment is discernible.
For both studies, investment in libraries is understood as the budget allocated to alibrary. For UIUC the total library budget is $36,102,613. As visible in Table I, UIUCdid not as indicated in the formula above, subtract the “cost of investment” from the“gain from investment” before dividing it. They simply divided the “total librarybudget” from the “gain from investment.” In the UIUC study the gain from investmentis the proportion of grant income using library materials in US dollars. Thisassumption is the basis for their study. Probably UIUC decided to change the basisformula accordingly, because a library budget is not a real cost of investment for acertain product. Without the library budget a library would not exist. In contrast toalmost 100 per cent of a library budget being provided by the federal state inGermany[58],59 library budgets or higher education in general are not highlystate-subsidized in the USA:
Public support for higher education in the USA has declined over the past quarter century.Nearly half of Illinois’ budget came from state funding in 1980; by 2008, that figure haddeclined to less than 17% (Kaufman, 2008a, p. 29).
Nevertheless, the budget allocated is always the core of existence for a library andagain for both studies the basis for the ROI calculation.
In their study, UIUC calculated the ROI with the total library budget and only forcomparison reasons with the materials budget. “If the materials budget, rather than thetotal budget, had been used, the ROI would have been approximately $12.” (Kaufman,2008a, p. 32). For this study, the materials budget assigned to the Library andInformation Science section at the UB is used. This is done because this study focuseson the IBI and not on the HU in general. For comparison, a second calculation is donewith the complete materials budget of the UB. The materials budget of the UB isusually a combined amount of federal state money and earmarked capital such asthird-party funding, funds for educational books, and special grants for the acquisitionof electronic resources as well as funds spent on specific research. The latter is notincluded in the materials budget used for comparison, because the budget for the LISsection comes from the part of the materials budget that is state-funded. This secondcalculation makes sense as the citations in the grant proposals by the IBI do not remain
BL23,4
158
entirely in the LIS section. Some additional sources from fields such as law, economics,and ICT (Information and Communication Technology) are used as well.
In 2009 the materials budget for the UB was e4,700,000. Approximately e2,200,000was earmarked capital as explained above, the rest (approximately e2,500,000) wasdistributed among the various subjects. The UB has a defined model for the dispersionof budgets among all subjects available at the HU. In 2009 the budget for the LISsection was e30,600 (1.21 per cent of the federal state funding). The ROI is calculatedwith both values, e30,600 and e2,500,000.
Doing the mathBetween 2006 and 2009, 13 grant proposals applied for by applicants related to the IBIhave been successfully accomplished or begun. The total amount of grands awardedfor these grant proposals sums up to approximately e3,190,502.72 (in three years),approximately e1,063,500.90 per year. The average size of grants the IBI received isapproximately e245,423.29. Owing to the decisions made for calculating the budgetand evaluating the citations, the figures are all approximate figures.
Illinois calculated their proportion of grants secured using library materials in USdollar, by using their average size grant (see Table III, lines 7 and 8 of the ROIcalculation by UIUC below).
In a second step, UIUC multiplied the result of the above calculation with thenumber of grants expended in the year 2006. This provided UIUC with the size of thegrant income using library materials (see Table IV, lines 9 and 10 of the ROIcalculation by UIUC).
For the German study, the total amount of Euro secured with the help of third-partyfunds is available for calculating the proportion of grant income using librarymaterials; hence no average size grant is needed. The two steps above were combinedinto one. The proportion of grants awarded in Euro using the library is then calculatedby multiplying the percentage of citations obtainable through the UB with the totalamount of third-party funds awarded for 2006-2009 (see Table IV ROI calculation forthe IBI and the UB).
Table V ROI calculations for the IBI and the UB shows the following results:. 13 grant proposals contain 474 citations of which 190 are obtainable through the
library;. more than 40 percent of the grants awarded to the IBI came from citations in
those 13 grant proposals that are obtainable through the UB;
$
Average size grant 63,923Proportion of grant $ secured using library materials 25,369 (78.14% * 50.79% * $ 63,923)/100/100
Table III.Lines 7 and 8 of the ROI
calculation by UIUC
No. grants (expended) in year 6,232$ proportion of grant income using library materials 158,099,608 (25,369 * 6,232)
Table IV.Lines 9 and 10 of the ROI
calculation by UIUC
ROI in Germanlibraries
159
. for 2006-2009 the total grant income at the IBI is approximately e3,190,502.72;
. the average proportion of the grant income generated by using the libraryresources is approximately e1,278,753.49;
. dividing the proportion of the grant income generated by using the libraryresources through the approximate amount of the materials budget expended forthe subject area LIS in three years, calculates a ROI of e13,93 for every Euroinvested in the subject area LIS between 2006 and 2009; and
. dividing the proportion of the grant income generated by using the libraryresources through the approximate total materials budget expended in threeyears, calculates a ROI of e0,17 for every Euro invested in the materials budgetof the UB between 2006 and 2009.
The results show that implementing the UIUC concept and formula in a different, inthis case German academic setting, is possible. Certain amendments had to be made,but no major problems occurred.
Similarities and differencesIn summary, it is clear that the essentials of the study created by UIUC have stayed thesame, but that components were adapted for the study to work in a German academicsetting that is considerably smaller in scope than UIUC’s facilities. For one, UIUC hadto deal with a very large amount of grant proposals in just one year. This study had toadapt UIUC handling of large amounts of proposals in order to be able to work with a
No. grant proposals 13 Total no. of successful grant proposals atthe IBI between 2006-2009
No. citations 474 Total count of citationsNo. citations obtainable through the UB 1901st calculation% citations obtainable through the UB 40.08% For all grant proposals
e total size of grants awarded 3,190,502.72In total (see Appendix 2. GrantsCalculations Final)
2nd calculatione proportion of grant income using the UB 1,278,753.49 (40,08% * 3,190,502.72)/100No. of materials budget expended for LISin 2009 e30,600Approx. no. of materials budget expendedfor LIS in 2006-2009 e91,800 (e30,600 * 3 years)3rd calculationUniversity return in e grant on LIS 13,93 (1,278,753.49/91,800)
Approx. no. of total materials budget forUB in 2009 2,500,000Approx. no. of total materials budget forUB in 2006-2009 7,500,000 (e2,500,000 * 3 years)4th calculationUniversity return in e grant on librarywith the help of the IBI (materials budget)
e0,17 (1,278,753.49/7,500,000)Table V.ROI calculations for theIBI and the UB
BL23,4
160
comparably very small amount of 13 grant proposals. The huge difference in scopeindicated the possibility and maybe necessity to change the analysis and evaluation ofthe citations. As mentioned previously, the smaller scale of this study made afull-blown survey unnecessary. It was possible to evaluate the citations individuallyand thereby to retrieve a more accurate proportion of grants in Euro secure-able byusing library materials. Using citations in grant proposals as basis for incomegenerated by the library has stayed the same.
Further, UIUC had to establish that applicant bodies do actually use their ownlibrary and its resources for citations made in grant proposals. For this they had toconduct a large scale survey examining the tenure system faculty perceptions on therole the library plays in their research and grant-seeking activities (Kaufman, 2008a).On the one hand, the study for the IBI and the UB could rely on UIUC’s findings that“75% of respondents stated that over three-quarters of the citations they included intheir grant applications were accessed through the library” (Kaufman, 2008a, p. 31). Onthe other hand, individually evaluating each citation gave a clear result which citationsand how many citations were obtainable through the library (UB), not indicatinghowever if the applicant bodies really used the UB.
Regarding the latter, when asking faculty at the IBI for the necessary documentsand grant proposals to conduct this study, a question intended to affirm UIUC’sproposition was included. The recipients were asked to rate on a spectrum rangingfrom one to five how often they use materials that could be made available by theuniversity library; including databases, eBooks, electronic journals, and theedoc-Server. Rating with a one would imply 0 per cent of one’s materials aredetectable with the help of the UB, two would imply 25 per cent are obtainable, threewould imply 50 per cent, four would imply 75 per cent and five would imply 100 percent are obtainable through the UB[59]. The spectrum was kept very elementary tosimplify answering the question. It was done in the hope of receiving a greater amountof replies. Even so only two answers came back; one voted a one or maximum two,stating that the grant proposals involved are usually focusing on applied researchrather than subject-specific fundamentals, thereby indicating that those grantproposals are not citing many resources obtainable through a library. Interestinglyenough, the grant proposals by this person had low amounts of citations. The secondanswer voted a four but was very unsure about it, simply because the person felt it wasimpossible to prove it. The grant proposals this person was involved in containedconsiderably more citations than the grant proposals by the previous person.
As it was possible to analyse each citation individually, receiving answers from allapplicants was not that important for this study, but it would have been interesting tofind out how much perception (rate) and reality (analysed citations) there is. If moreanswers were received, it would have been possible as well to verify that UIUC wasright in their assumption to use the statements by their tenure system faculty (i.e. 78.14per cent of faculty using citations in grant proposals) as basis for their ROIcalculations. All the same, this is not the research question of this study, but could bean interesting follow-up study.
The final calculations differ at two places. On the one hand, UIUC calculates theproportion (%) of grant income using library materials with the help of an average sizegrant that is multiplied with the number of grants expended in the year in question.For the IBI/UB study, calculating the average size grant was not necessary. As part of
ROI in Germanlibraries
161
the evaluation of the thirteen grant proposals, the total amount of grants awarded wasestimated and used for receiving the result (proportion of grant income using librarymaterials).
In the final calculation by UIUC, the ROI was calculated with the total librarybudget of UIUC’s university library. The ROI calculated as part of this study, wascalculated with the materials budget allocated for the LIS subject area. Only forcomparability was it calculated with the total materials budget as well. The lattercalculation is not important for this study, because the grant proposals that wereevaluated are all based on the IBI and the subject area Library and InformationScience.
Table VI lists all similarities and differences at a glance.Essentially, UIUC’s formula for calculating a ROI for an academic library has been
maintained, but certain amendments due to scale and information available were made.
ConclusionAlthough a different academic setting, the study for the IBI is very small in scale andfocused on one subject area only. A further study for all subject areas at the HumboldtUniversity, Berlin would provide a clearer picture of the strength and weaknessesadapting the formula devised by UIUC into a different environment. The author,however, does not believe that any major problems would appear when conducting thestudy on a larger scale in any German academic setting. Going along as this study did,and individually analysing hundreds of citations in grant proposals might be too muchof an effort on a larger scale. This depends on the amount of people involved, however,and one could always resort to the way UIUC estimated their amount of proposalsincluding citations. When conducting such a survey, a limitation of people who wouldreceive the survey is not necessary in Germany; professors, faculty, and members ofresearch staff should be allowed to answer if they take part in grant proposalapplications.
Interestingly, another question came up that, in the author’s opinion, could beanswered with the help of UIUC’s formula. The author believes the difference in grant
Description ofprocess UIUC IBI/UB
Initial position/scope
Study conducted for a major universityand its library
Study conducted for a subject area andits institution at a university
Amount ofproposals
Large (2,879 proposals/1,456 successfulones)
Small (13 successful proposals)
Analysis With the help of estimations/based on alarge-scale survey (50,79% proposalsinc. citations obtained through library)
Individual evaluation for each citation(474 citations in total, 190 obtainablethrough the library ¼ 40,08%)
Calculation 1 Calculates $ of grant income usinglibrary materials with an average sizegrant multiplied with grants (expended)in 2006
Calculates the proportion of grantincome using library materials with thetotal amount of grants awarded between2006-2009
Calculation 2 Calculates ROI with the total librarybudget
Calculates ROI with the materialsbudget for LIS (plus total materialsbudget/total library budget)
Table VI.Similarities anddifferences
BL23,4
162
proposal application behaviour between subject areas could be explored. One wouldexpect application-orientated subjects such as ICT, physics, chemistry, and economicsto have large amounts of grant proposals with a high ROI, as they are understood asthe subjects third-parties are interested in. It is true that technically- andeconomically-orientated universities have higher third-party funding than others.Some of the application-orientated grant proposals for the IBI had low amounts ofcitations, however, and would not have a very high ROI in regards to the library. Aquick glance at the amount of grant proposals at the HU[60] in subject areas such asAfrican studies/literature (ten research projects), English and American Studies (26research projects), or German literature (43 research projects) show a surprisingamount of research studies; it occurred that it would be interesting to evaluate theirROI in regards to the library. The final ROI figures would not indicate which subject isbetter or worth more in creating a ROI for the university, but the size of results wouldperhaps illuminate differences in research behaviour and eventually help a universitylibrary to decide which collections for which subject area to extend.
UIUC has planned applying their ROI formula to multiple institutions as well asexpanding the longitudinal scope. It will be interesting to see their follow-up studiesthat might show differences in institutional characters or in the countries as well.
The only problem that can be seen with the study by UIUC is the fact that they basethe proportion of grant proposals containing citations obtainable through the libraryon a survey. Kingma validated their model, but the survey is UIUC’s weakest link, andany study going along the same lines needs to make sure that their survey isimpervious.
This study validates the original research question “Can the methodologydeveloped by UIUC be applied to German universities?” It shows that UIUC developeda ROI model and formula that is modifiable enough to fit into different academicsettings. Their formula had to be adapted, but it was easy to do so. As said before,university libraries implementing the formula have to take extra special care whenevaluating the citations either with the help of a survey, or by counting themindividually. If striving for a concrete ROI figure, the evaluation of the citations is thefocal point.
Hopefully, ROI studies as this one by UIUC will help to value academic libraries’impact on research and revenue generating activities.
Notes
1. www.hu-berlin.de/ (accessed April 25, 2010).
2. “Deans from a number of leading iSchools have joined together to leverage the power ofleading iSchools in building awareness of, support for and involvement with the iFieldamong key constituencies, principally the media, business community, those who fundresearch, student prospects, and users of information.“ www.ischools.org/site/about/(accessed February 15, 2010).
3. www.ub.hu-berlin.de/ueber-uns/profil (accessed February 15, 2010).
4. www.investopedia.com/terms/r/returnoninvestment.asp (accessed March 27, 2010).
5. See publications in Social Enterprise Journal (2005), Strategic HR Review (2009), Journal ofConsumer Marketing (1999), International Journal of Hospitality Management (1983),Handbook of Business Strategy (2004), Journal of Accounting and Public Policy (1996).
ROI in Germanlibraries
163
6. www.google.de/url?sa ¼ t&source ¼ web&ct ¼ res& cd ¼ 1&ved ¼ 0CBAQFjAA&url ¼http%3A%2F%2Fciteseerx.ist.psu.edu%2Fviewdoc%2Fdownload%3Fdoi%3D10.1.1.129.2114%26rep%3Drep1%26type%3Dpdf&ei ¼ zmSFS-eCNKTqmgP-8cGtAg&usg ¼AFQjCNE3g7V508WEU8gUbIigWvmf-No_CQ&sig2 ¼ Sb7IC6GJzU9hotNT-kHoUQ(accessed February 21, 2010).
7. www.bl.uk/pdf/measuring.pdf (accessed February 21, 2010).
8. This quote translates as: “The awareness for reasons why libraries and librarians exist wasmissing as well as the comprehension how to gain and keep support from their beneficiaries.Only now this has changed.”
9. http://mel.org/files/calculatorcode.php (accessed March 23, 2010).
10. The Library of Michigan Return on Investment Calculator as well as the North SuburbanLibrary System (NSLS) calculator allows its users to enter the amount of books, magazines,audio books, movies, etc. borrowed per month and calculate the ROI in the case of NSLS withthe help of average list and retail prices of those items. Michigan in return does not specifywhere the numbers come from and let you speculate if they used retail prices as well orconsidered other costs such as library budget used for human resources needed for recordsmanagement as well.
11. www.youtube.com/watch?v ¼ TgqoM5ZNu3Q (accessed March 23, 2010) The video showsa selfmade advertisement for the North Suburban Library System’s ROI calculator.
12. www.bmbf.de/de/1321.php (accessed February 21, 2010). The German Exzellenzinitiativefunded by the BMBF (Bundesministerium fur Bildung und Forschung) claims to supportcutting-edge research at outstanding universities, but the first two rounds (2006, 2007) showa strong bias towards richer German states. Only a fraction of the nominated winninguniversities are not in the two richest German states Baden-Wurrtemberg and Bavaria.
13. Freely translated this quote says: “The German field of Library and Information Science(LIS) has not shown much inclination to deal with this.”
14. Freely translated this quote says: “Virtual prices are estimated through surveys conductedwith users.”
15. www.sub.uni-goettingen.de/ (accessed February 18, 2010).
16. www.uni-goettingen.de/ (accessed February 18, 2010).
17. www.dfg.de/index.jsp (accessed March 23, 2010).
18. www.bmbf.de/ (accessed March 23, 2010).
19. http://gepris.dfg.de/gepris/OCTOPUS/ (accessed March 23, 2010).
20. The five Universities of Applied Science searched for, were: Hochschule fur AngewandteWissenschaften Hamburg, Fachhochschule Potsdam, Hochschule Darmstadt, Hochschulefur Technik, Wirtschaft und Kultur Leipzig and Hochschule der Medien Stuttgart.
21. http://forschung.hu-berlin.de/statistik/ (accessed March 16, 2010).
22. Personae are fictive people devised to represent a certain clientele (in Marketing) or users(Usability studies). The persona devised for this study is between 26 and 28 years old, single,has a postgraduate degree and starts the first job as research assistant.
23. For calculations of the BAT (Bundesangestelltentarif) see http://de.wikipedia.org/wiki/Bundesangestelltentarifvertrag or http://oeffentlicher-dienst.info/tv-l/berlin/ (accessedMarch 26, 2010). People employed in the public service in Germany are paid based on acomplicated system considering previous knowledge, age, marital status, place of residenceand other. As this is very complex only average costs may be calculated.
BL23,4
164
24. For more information on TV-L see http://de.wikipedia.org/wiki/Tarifvertrag_f%C3%BCr_den_%C3%B6ffentlichen_Dienst_der_L%C3%A4nder (accessed March 26, 2010) Againthere are no sources available in English.
25. Translates as: Initial position/own preparatory work.
26. www.kobv.de/ (accessed April 16, 2010). Kooperativer Bibliotheksverbund BerlinBrandenburg’ is the cooperative network of all academic, public and many corporatelibraries in Berlin and Brandenburg supporting each other by creating a wholesome stock ofmaterials and media as well as devising new services for their clientele.
27. www.dfg.de/forschungsfoerderung/formulare/download/1_02.pdf (accessed January 12,2010). Freely translated this quote says: “The grant proposal should not exceed 20 pagesand should be coherent on its own; reading citations and additional literature should not benecessary for comprehension”.
28. www.dini.de/ (accessed April 10, 2010).
29. http://ec.europa.eu/information_society/eeurope/i2010/index_en.htm (accessed March 30,2010).
30. Most of the time European projects receive only 50 per cent of the estimated budget forfunding. The other half needs to be contributed by the participating institutions (owncontributions). Hence is it important to indicate when talking about European budgets if thenumbers are including or excluding own contributions. As far as I understood thecalculations, own contributions are not always monetary, but can be set against equipmentsuch as computers, costs for Internet connection and such.
31. www.iuwis.net/ (accessed April 13, 2010)
32. www.uni-konstanz.de/ (accessed April 23, 2010).
33. www.urheberrechtsbuendnis.de/index.html.de (accessed April 14, 2010).
34. CMS are the facilities responsible for all computer and technical support at the HU situatedin South-East Berlin in Adlershof. The director of the CMS is at the same time a professor atthe IBI.
35. http://lockss.stanford.edu/lockss/Home (accessed April 23, 2010).
36. http://kopal.langzeitarchivierung.de/index_koLibRI.php.de (accessed April 23, 2010).
37. www.europeanaconnect.eu/ (accessed April 5, 2010).
38. http://europeana.eu/portal/ (accessed April 5, 2010).
39. http://cordis.europa.eu/fp7/ict/language-technologies/project-galateas_en.html (accessedApril 20, 2010).
40. http://doarc.projects.isn-oldenburg.de/ (accessed April 5, 2010).
41. www.eerqi.eu/ (accessed April 3, 2010).
42. Translates as: “Measuring the diversity of research”
43. See: www.fu-berlin.de/ (accessed April 15, 2010).
44. www.carpet-project.net/ (accessed April 20, 2010).
45. www.mpg.de/instituteProjekteEinrichtungen/weitereEinrichtungen/mpdl/index.html
46. www.uni-koeln.de/
47. www.fotomarburg.de/ (accessed April 12, 2010).
48. http://medienportal.hu-berlin.de/ (accessed April 15, 2010).
ROI in Germanlibraries
165
49. www.dini.de/projekte/oa-netzwerk/ (accessed April 15, 2010).
50. www.uni-osnabrueck.de/ (accessed April 25, 2010).
51. www.dini.de/projekte/oa-netzwerk/ (accessed April 15, 2010).
52. www.ams.org/mathscinet/msc/msc2010.html (accessed April 16, 2010).
53. www.geschichte.hu-berlin.de/ (accessed April 25, 2010).
54. www.zzf-pdm.de/ (accessed April 2, 2010).
55. http://arxiv.org/ (accessed April 2, 2010).
56. http://arxiv.org/help/replace (accessed April 2, 2010).
57. Source: http://www.investopedia.com/terms/r/returnoninvestment.asp (accessed April 16,2010)
58. www.bibliotheksportal.de/hauptmenue/bibliotheken/bibliotheken-in-deutschland/bibliothekslandschaft/unterhaltstraeger/ (accessed April 16, 2010) “Wegen dergrundgesetzlich verbrieften Kulturhoheit sind in Deutschland die Lander die wichtigstenTrager wissenschaftlicher Bibliotheken.” Translates as: “Due to the chartered independencein educational and cultural matters in Germany, federal states are the fundamental fundingbodies for academic libraries.”
59. The original question in German: Wie haufig benutzen Sie fur Ihre Antrage Materialien, dieauch von der Universitatsbibliothek in irgendeiner Form (Datenbanken, eBooks, Bucher,Zeitschriften, elektronische Zeitschriften, etc.) zur Verfugung gestellt werden? Bitte beziehenSie Materialien vom edoc-Server auch mit ein. Um die Antwort zu vereinfachen schatzen SieIhre Benutzung bitte auf einer Scala von 1 – 5, eins bedeutet 0 percent, 2 ¼ 25 percent,3 ¼ 50 percent, 4 ¼ 75 percent und 5 ¼ 100 percent. Eins bedeutet dass Ihre Zitierungennicht mit Hilfe der UB gefunden werden konnen und auch nicht mit Hilfe der UB daraufzugegriffen hatte werden konnen. 100 percent bedeutet dass Ihre Zitierungen alle mit Hilfeder UB gefunden und zuganglich gewesen waren.
60. http://forschung.hu-berlin.de/fdb/ (accessed April 20, 2010). The Forschungsdatenbank(database for research projects) at the HU lists all current and older projects accomplished byinstitutions at the HU. It is possible to search by institution (subject area), project leader,keyword and funding body.
References
Aabø, S. (2009), “Libraries and return on investment (ROI): a meta-analysis”, New Library World,Vol. 110 Nos 7/8, pp. 311-24.
Arrow, K., Solow, R., Portney, P.R., Leamer, E.E., Radner, R. and Schuman, H. (1993), “Report ofthe NOAA panel on contingent valuation”, available at: www.google.de/url?sa¼t& source¼web&ct¼res&cd¼1&ved¼0CBAQFjAA&url¼http%3A%2F%2Fciteseerx.ist.psu.edu%2Fviewdoc%2Fdownload%3Fdoi%3D10.1.1.129.2114%26rep%3Drep1%26type%3Dpdf&ei¼zmSFS-eCNKTqmgP-8cGtAg&usg¼AFQjCNE3g7V508WEU8gUbIigWvmf-No_CQ&sig2¼Sb7IC6GJzU9hotNT-kHoUQ (accessed February 21, 2010)
Bennekom, van F.C. (2002), Customer Surveying. A Guidebook for Service Managers, CustomerService Press.
Blanck, S. (2006), Neues fur Bibliotheken – Neues in Bibliotheken. Wert und Wirkung vonBibliotheken, BIT Verlag, Wiesbaden.
British Library (2004), “Measuring our value”, available at: www.bl.uk/pdf/measuring.pdf(accessed February 21, 2010).
BL23,4
166
Busch, R. (2004), Wie viele Bibliotheken brauchen wir?, Bock þ Herrchen, Bad Honnef.
CARPET (n.d.), “carpet-project.net: CARPET – Community for Academic Reviewing, Publishingand Editorial Technology: Startseite”, available at: www.carpet-project.net/ (accessedApril 20, 2010).
Fett, O. (2004), “Impact – Outcome – Benefit. Ein Literaturbericht zur Wirkungsmessung furHochschulbibliotheken”, in Umlauf, K. (Ed.), Berliner Handreichungen zurBibliothekswissenschaft, Vol. 142, pp. 1-71.
Griffith, J-M. (2004), “Taxpayer return on investment in Florida public libraries: Summary ReportSeptember 2004”, available at: http://dlis.dos.state.fl.us/bld/roi/pdfs/ROISummaryReport.pdf (accessed March 23, 2010).
Havemann, F. (2004), “Eprints in der wissenschaftlichen Kommunikation”, available at: www.ib.hu-berlin.de/, fhavem/E-prints.pdf (accessed April 5, 2010).
Holt, G.E., Elliott, D. and Dussold, C. (1996), “A framework for evaluating public investment inurban libraries”, The Bottom Line: Managing Library Finances, Vol. 9 No. 4, pp. 4-13.
Imholz, S. and Weil Arns, J. (2008), “Worth their weight: an assessment of the evolving field oflibrary evaluation”, Public Library Quaterly, Vol. 26 Nos 3/4, pp. 31-48.
Kaufman, P.T. (2008a), “The library as strategic investment: results of the University of IllinoisReturn on Investment Study”, Online Information 2008 Proceedings, pp. 29-36.
Kaufman, P.T. (2008b), “The library as strategic investment: results of the University of IllinoisReturn on Investment Study”, Library Quarterly, Vol. 18, pp. 424-37.
KOBV (n.d.), “KOBV: Startseite”, available at: www.kobv.de/ (accessed April 16, 2010).
(The) Krafty Librarian (2008), available at: http://kraftylibrarian.com/2008/02/how-much-are-you-worth.html (accessed April 6, 2010).
Luther, J. (2008), “University investment in the library: what’s the return? A case study at theUniversity of Illinois at Urbana-Champaign”, White paper 1-18, 2010), available at: http://libraryconnect.elsevier.com/whitepapers/lcwp0101.pdf (accessed January 2, 2010).
Medienportal (n.d.), “Medienportal – JadisNet”, available at: http://medienportal.hu-berlin.de/(accessed April 15, 2010).
Meta-Image (n.d.), available at: www.leuphana.de/meta-image/ (accessed April 10, 2010).
Missingham, R. (2005), “Libraries and economic value: a review of recent studies”, PerformanceMeasurement and Metrics, Vol. 6 No. 3, pp. 142-58.
Mitchell, R.C. and Carson, R.T. (1993), Using Surveys to Value Public Goods: The ContingentValuation Method, Johns Hopkins University Press, Baltimore, MD.
Mitchell, R.C. and Carson, R.T. (2000), Using Surveys to Value Public Goods: The ContingentValuation Method, Johns Hopkins University Press, Baltimore, MD.
Oder, N. (2009), “Study at UIUC suggests $4,38 in grant income for each library dollar”, LibraryJournal, 22 January, available at: www.libraryjournal.com/article/CA6631202.html(accessed March 30, 2010).
Payne, P. and Conyers, A. (2005), “Measuring the impact of higher education libraries: theLIRG/SCONUL Impact Implementation Initiative”, Measuring the Impact of HigherEducation Libraries: The LIRG/SCONUL Impact Implementation Initiative, Library andInformation Research, Vol. 29 No. 91, pp. 1-9.
Poll, R. and Boekhorst, P. (2007), Measuring Quality: Performance Measurement in Libraries,K.G. Saur, Munchen.
Poll, R. and Payne, P. (2006), “Impact measures for libraries and information services”, availableat: http://eprints.rclis.org/9161/1/payne_poll_final_web.pdf (accessed February 27, 2010).
ROI in Germanlibraries
167
Strouse, R. (2003), “Demonstrating value and return on investment: the ongoing imperative –assessing your library’s value statement”, Information Outlook, March, available at: http://findarticles.com/p/articles/mi_m0FWE/is_3_7/ai_99011610/?tag¼content;col1 (accessedFebruary 25, 2010).
Further reading
About the iSchools (n.d.), available at: www.ischools.org/site/about/ (accessed February 15,2010).
Aktionsbuendnis (n.d.), “Aktionsbuendnis – Urheberrecht fur Bildung und Wissenschaft”,available at: www.urheberrechtsbuendnis.de/index.html.de (accessed April 14, 2010).
arXiv.org (n.d.), “arXiv.org e-Print archive”, available at: http://arxiv.org/ (accessed April 2,2010).
arXiv.org help (n.d.), “arXiv.org help – to replace an article”, available at: http://arxiv.org/help/replace (accessed April 2, 2010).
Ausgewahlte Statistiken – Humboldt Universitat zu Berlin (n.d.), available at: http://forschung.hu-berlin.de/statistik/ (accessed March 16, 2010).
Frdgliotheksportal (n.d.), “Frdgliotheksportal: Unterhaltstrager”, available at: www.frdgliotheksportal.de/hauptmenue/frdgliotheken/frdgliotheken-in-deutschland/frdgliothekslandschaft/unterhaltstraeger/ (accessed April 16, 2010).
BMBF (n.d.a), “BMBF: Exzellenzinitiative”, available at: www.bmbf.de/de/1321.php (accessedFebruary 21, 2010).
BMBF (n.d.b), “BMBF: Bundesministerium fur Bildung und Forschung”, available at: www.bmbf.de/ (accessed March 23, 2010).
Bundesangestelltentarifvertrag (n.d.), “Bundesangestelltentarifvertrag – Wikipedia”, availableat: http://de.wikipedia.org/wiki/Bundesangestelltentarifvertrag (accessed March 26, 2010).
CORDIS (n.d.), “CORDIS: ICT: programme: language technologies”, available at: http://cordis.europa.eu/fp7/ict/language-technologies/project-galateas_en.html (accessed April 20, 2010).
Dawson, S.M. (1983), “A synthesis of alternative techniques for calculating the return oninvestment”, International Journal of Hospitality Management, Vol. 2, pp. 135-9.
DFG (n.d.a), “DFG – Deutsche Forschungsgemeinschaft – Formulare und Merkblatter”, availableat: www.dfg.de/foerderung/formulare_merkblaetter/index.jsp (accessed January 12, 2010).
DFG (n.d.b), “DFG – Deutsche Forschungsgemeinschaft”, available at: www.dfg.de/index.jsp(accessed March 23, 2010).
DFG (n.d.c), “DFG – GEPRIS”, available at: http://gepris.dfg.de/gepris/OCTOPUS/;jsessionid¼54F0262DC449F16AAE3FDF3184DCB689 (accessed March 23, 2010).
DINI (n.d.a), “DINI – Deutsche Initiative fur Netzwerkinformation e.V.: OA-Netzwerk”, availableat: www.dini.de/projekte/oa-netzwerk/ (accessed April 5, 2010).
DINI (n.d.b), ”DINI – Deutsche Initiative fur Netzwerkinformation e.V.: Startseite”,available at:www.dini.de/ (accessed April 10, 2010).
DNB (n.d.), “DNB, Deutsche Nationalfrdgliothek – home”, available at: www.d-nb.de/ (accessedApril 26, 2010).
DOARC (n.d.), “DOARC – Prototyp”, available at: http://doarc.projects.isn-oldenburg.de/(accessed April 5, 2010).
EERQI (n.d.), “About EERQI”, available at: www.eerqi.eu/ (accessed April 3, 2010).
BL23,4
168
Europeana (n.d.), “Europeana – homepage”, available at: http://europeana.eu/portal/ (accessedApril 5, 2010).
EuropeanaConnect (n.d.), available at: www.europeanaconnect.eu/ (accessed April 5, 2010).
Flockhart, A. (2005), “Raising the profile of social enterprises: the use of social return oninvestment (SROI) and investment ready tools (IRT) to bridge the financial credibilitygap”, Social Enterprise Journal, Vol. 1 No. 1, pp. 29-42.
Forschungsdatenbank der HU (n.d.), .),”Forschungsdatenbank der HU – Humboldt Universitatzu Berlin”, available at: http://forschung.hu-berlin.de/fdb/ (accessed April 20, 2010).
Freie Universitat Berlin (n.d.), “Freie Universitat Berlin: Startseite”, available at: www.fu-berlin.de/ (accessed April 15, 2010).
Georg-August-Universitat Gottingen (n.d.), available at: www.uni-goettingen.de/ (accessedFebruary 18, 2010).
Gordon, L.A. and Iyengar, R.J. (1996), “Return on investment and corporate capital expenditures:empirical evidence”, Journal of Accounting and Public Policy, Vol. 15, pp. 305-25.
Home – LOCKSS (n.d.), available at: http://lockss.stanford.edu/lockss/Home (accessed April 23,2010).
Homepage – Humboldt Universitat zu Berlin (n.d.), available at: www.hu-berlin.de/ (accessedApril 25, 2010).
Humboldt Universitat zu Berlin (2009), Berlin School of Library and Information Science,Humboldt Universitat zu Berlin, Berlin.
i2010/Europa (n.d.), available at: http://ec.europa.eu/information_society/eeurope/i2010/index_en.htm (accessed March 30, 2010).
Institut fur Frdgliotheks (n.d.), “Institut fur Frdgliotheks- und Informationswissenschaft – IBI”,available at: www.ibi.hu-berlin.de/ (accessed April 25, 2010).
Institut fur Geschichtswissenschaften HU-Berlin (n.d.), “Institut fur GeschichtswissenschaftenHU-Berlin – Startseite”, available at: www.geschichte.hu-berlin.de/ (accessed April 25,2010).
IUWIS (n.d.), available at: www.iuwis.net/ (accessed April 13, 2010).
kopal (n.d.), “kopal – Kooperativer Aufbau eines Langzeitarchivs digitaler Informationen –koLibRI: kopal library for Retrieval and Ingest”, available at: http://kopal.langzeitarchivierung.de/index_koLibRI.php.de (accessed April 23, 2010).
Library of Michigan (n.d.) Vol. 2, “Library of Michigan return on investment calculator”,available at: http://mel.org/files/calculatorcode.php (accessed March 23, 2010).
Max Planck Gesellschaft (n.d.), “Max Planck Gesellschaft – Max Planck Digital Library(MPDL)”, available at: www.mpg.de/instituteProjekteEinrichtungen/weitereEinrichtungen/mpdl/index.html (accessed April 24, 2010).
MSC2010 (n.d.), “MSC2010 database”, available at: www.ams.org/mathscinet/msc/msc2010.html(accessed April 16, 2010).
Niedersachsische Staats- und Universitatsfrdgliothek Gottingen (n.d.), available at: www.sub.uni-goettingen.de/ (accessed February 18, 2010).
Offentlicher-Dienst.Info (n.d.), “Offentlicher-Dienst.Info – TV-L – Berlin”, available at: http://oeffentlicher-dienst.info/tv-l/berlin/ (accessed March 26, 2010).
Phillipps-Universitat Marburg (n.d.), “Phillipps-Universitat Marburg – DeutschesDokumentationszentrum fur Kunstgeschichte – Bildarchiv Foto Marburg: Willkommen– Bild”, available at: www.fotomarburg.de/ (accessed April 12, 2010).
ROI in Germanlibraries
169
Profil – Universitatsfrdgliothek der HU Berlin (n.d.), available at: www.ub.hu-berlin.de/ueber-uns/profil (accessed February 15, 2010).
Return on Investment (n.d.), “Return on investment (ROI)”, available at: www.investopedia.com/terms/r/returnoninvestment.asp (accessed April 16, 2010).
Seymann, M. and Kleinhanzi, B. (2004), “Return on directors: maximizing return on investment inthe board room”, Handbook of Business Strategy, Vol. 5 No. 1, pp. 23-8.
Tarifvertrag (n.d.), “Tarifvertrag fur den offentlichen Dienst der Lander – Wikipedia”, availableat: http://de.wikipedia.org/wiki/Tarifvertrag_f%C3%BCr_den_%C3%B6ffentlichen_Dienst_der_L%C3%A4nder (accessed March 26, 2010).
Universitat Konstanz (n.d.), available at: www.uni-konstanz.de/ (accessed April 23, 2010).
Universitat Osnabruck (n.d.), available at: www.uni-osnabrueck.de/ (accessed April 25, 2010).
Universitat zu Koln (n.d.), available at: www.uni-koeln.de/ (accessed April 25, 2010).
van Bennekom, F.C. (2002), Customer Surveying. A Guidebook for Service Managers, CustomerService Press, Bolton, MA.
Yapp, M. (2009), “Measuring the ROI of talent management”, Strategic HR Review, Vol. 8 No. 4,pp. 5-10.
You Tube (n.d.), “You Tube – libraries return on investment (ROI)”, available at: www.youtube.com/watch?v¼TgqoM5ZNu3Q (accessed March 23, 2010).
Zajas, J.J. (1999), “Measuring brand communication: return on investment”, Journal of ConsumerMarketing, Vol. 16 No. 6, pp. 616-29.
Zentrum fur Zeithistorische Forschung Potsdam (n.d.), available at: www.zzf-pdm.de/ (accessedApril 2, 2010).
Appendix 1. Grant proposals. Applicant body: Christen, W., Havemann, F., Hilf, E.R., Mellert, V. and Schirmbacher, P.
(2008), Erschließung von Zitationen in verteilten Open-Access Repositorien (DistributedOpen Access Reference Citation Services) Acronym: DOARC. Antrag auf Forderung durchdie DFG im Forderungsbereich Wissenschaftliche Literaturversorgungs- undInformationssysteme (LIS), pp. 1-26.
. Applicant body: Dieckmann, L., Schirmbacher, P. and Warnke, M. (2008), Meta-Image –Forschungsumgebung fur den Bilddiskurs in der Kunstgeschichte. Antrag auf Forderungdurch die DFG im Forderungsbereich Wissenschaftliche Literaturversorgungs- undInformationssysteme (LIS), pp. 1-23.
. Applicant body: Glaser, J., Havemann, F. and Heinz, M. (2008), Messung der Diversitat derForschung. Antrag auf Forderung durch das BMBF, pp. 1-15.
. Applicant body: Gradmann, S., Lossau, J., Plumer, J. and Schirmbacher, P. (2009),Weiterentwicklung und Betrieb des Netzwerks zertifizierter Open-Access-Repositorien furDeutschland. (Fortsetzungsantrag) Kennwort: OA-Netzwerk 2. Antrag auf Forderungdurch die DFG im Forderungsbereich Wissenschaftliche Literaturversorgungs- undInformationssysteme (LIS), pp. 1-42.
. Applicant body: Kuhlen, R. and Seadle, M. (2009), Aufbau einer Informationsinfrastrukturzum Urheberrecht fur Bildung und Wissenschaft. Urheberrechtliches Wissen fur Bildungund Wissenschaft (urhWiss). Antrag auf Forderung durch die DFG im ForderungsbereichWissenschaftliche Literaturversorgungs- und Informationssysteme (LIS), pp. 1-25.
. Applicant body: Lossau, N., Plumer, J. and Schirmbacher, P. (2006), Aufbau einesNetzwerks zertifizierter Open-Access Repositories. Kenntwort: OA-Netzwerk. Antrag auf
BL23,4
170
Forderung durch die DFG im Forderungsbereich WissenschaftlicheLiteraturversorgungs- und Informationssysteme (LIS), pp. 1-37.
. Applicant body: Lossau, N., Romary, L. and Schirmbacher, P. (2007), Aufbau einerInformationsplattform fur das Publizieren auf der Basis generischerPublikationswerkzeuge. Kennwort: CARPET. Antrag auf Forderung durch die DFG imForderungsbereich Wissenschaftliche Literaturversorgungs- und Informationssysteme(LIS), pp. 1-34.
. Applicant body: Niggemann, E., Seadle, M. and Schirmbacher, P. (2009),LOCKSS-und-KOPAL Infrastruktur-und-Interoperabilitat. Antrag auf Forderung durchdie DFG im Forderungsbereich Wissenschaftliche Literaturversorgungs- undInformationssysteme (LIS), pp. 1-34.
. Applicant body: Nippel, W., Sabrow, M. and Schirmbacher, P. (2007),Docupedia-Zeitgeschichte – thematische Netzwerke und informelles Publizieren mit demSoftware-Framework MediaWiki. Antrag auf Gewahrung einer Sachbeihilfe durch dieDFG, in Bezugnahme auf das DFG-Schwerpunktprogramm: Informations-Infrastrukturenfur netzbasierte Forschungskooperation und digitale Publikationen und ElektronischePublikationen im Literatur- und Informationsangebot wissenschaftlicher Bibliotheken,pp. 1-48.
. Applicant body: Schirmbacher, P. (2008), Entwicklung und Implementierung einerOpen-Source-Repository Solution fur vernetztes Arbeiten mit wissenschaftlichen Bild-und Multimediasammlungen. Kennwort: MUNIN-RS. Antrag auf Forderung durch dieDFG im Forderungsbereich Aufbau und Vernetzung von Repositorien, pp. 1-21.
. eContentplus. EuropeanaConnect (2008), Description of Work, pp. 1-140.
. European Educational Research Quality Indicators, EERQI (2008), Grant agreementNo. 217549, Annex I, Description of Work, pp. 1-96.
. GALATEAS Generalized Analysis of Logs for Automatic Translation and EpisodicAnalysis of Searches (2009), Proposal Part B, pp. 1-88.
(Further Appendices overleaf)
ROI in Germanlibraries
171
Appendix 2. Grant calculations final
Con
sist
ing
of
Mon
eyca
lcu
late
dE
Sta
ffco
sts
ES
chol
arly
equ
ipm
ent
ET
rav
elex
pen
ses
EP
ub
lish
ing
EO
ther
cost
sE
Com
men
ts
IUW
IS34
7,49
9.00
255,
984.
000.
006,
990.
0038
,700
.00
9,82
5.00
Lu
KII
280,
726.
0022
2,27
6.00
10,0
00.0
011
,100
.00
1,35
0.00
36,0
00.0
0E
uro
pea
naC
onn
ect
191,
986.
00£
££
££
EU
pro
ject
Gal
atea
s62
,356
.00
££
££
£E
Up
roje
ctD
OA
RC
183,
692.
0016
0,56
0.00
7,48
2.72
13,8
00.0
01,
850.
000.
00E
ER
QI
136,
720.
00£
££
££
EU
pro
ject
For
sch
un
gsd
iver
sita
t23
0,01
3.00
103,
645.
0032
,996
.00
33,5
00.0
06,
910.
0052
,962
.00
Car
pet
348,
940.
0032
1,12
0.00
0.00
23,2
220.
004,
600.
000.
00M
eta-
Imag
e27
1,52
0.00
209,
520.
000.
0027
,000
.00
5,00
0.00
30.0
00.0
0M
UN
IN-R
S27
6,88
0.00
214,
080.
000.
009,
800.
000.
0053
,000
.00
OA
N24
1,16
4.00
202,
464.
0012
,900
.00
23,1
00.0
02,
700.
000.
00O
AN
232
8,02
2.00
186,
272.
002,
000.
0031
,550
.00
8,20
0.00
0.00
Doc
up
edia
290,
984.
0025
5,98
4.00
5,50
0.00
16,0
00.0
01,
200.
0012
,300
.00
Tot
alam
oun
t3,
190,
502.
00
Table AI.
BL23,4
172
Appendix 3. Citations
IDW
ISR
efer
ence
sP
aid
4In
tern
etR
esou
rce
Loa
nsh
arin
g@ U
BIn
tern
et-
bas
edC
omm
ents
1.H
elm
hol
tz-G
emei
nsc
haf
t1
2.L
stin
gN
otap
pli
cab
le3.
Got
tin
ger
Erk
laru
ng
14.
Ste
llu
ng
nah
me
der
deu
tsch
enA
kad
emie
nd
erW
isse
nsc
haf
ten
15.
IDW
Pre
ssem
itte
ilu
ng
16.
All
ian
zS
chw
erp
un
kti
nit
iati
ve
UR
Ln
otfo
un
d7.
Hel
mhol
tzN
ewsl
ette
r20
071
8.M
axP
lan
ckIn
stit
ut
Pre
ssem
itte
ilu
ng
19.
Sie
ber
/Hoe
ren
Urh
eber
rech
tfu
rB
ild
un
gu
nd
Wis
sen
sch
aft
110
.R
efer
ente
nen
twu
rfzw
eite
sG
eset
z/R
egel
un
gd
esU
rheb
erre
chts
2004
111
.R
ech
tsp
olit
tisc
hes
Pos
itio
nsp
apie
rd
esD
tsch
.B
ibli
oth
eksv
erb
un
des
112
.U
Kin
itia
tiv
e1
13.
Dts
ch.
Ph
ysi
kal
isch
eG
esel
lsch
aft
e.V
114
.V
GW
ort
115
.D
euts
cher
Hoc
hsc
hu
lver
ban
dK
urz
info
rmat
ion
(2/0
8)1
16.
Un
iS
tutt
gar
t/R
ech
tsfr
agen
117
.V
ario
us
inte
rnet
reso
urc
es,
i.e.
Lin
kto
Bor
sen
ver
ein
des
dts
ch.
Bu
chh
and
elor
law
-blo
g.d
e1
18.
DIN
ID
oku
men
te1
19.
Dri
tte
Ste
llu
ng
nah
me
zum
Ers
ten
Kor
b(D
INI)
120
.D
INI-
Net
zwer
k1
21.
Op
enac
cess
-ger
man
y.d
e1
22.
Max
Pla
nck
Pu
bli
kat
ion
en1
23.
Ver
sch
ied
ene
un
iver
sita
reju
rist
isch
eE
inri
chtu
ng
en:
Mu
nst
er,
Han
nov
er,
Saa
rbru
cken
,K
arls
ruh
e1
24.
Deu
tsch
esF
orsc
hu
ng
snet
z1
UR
Ln
otfo
un
d25
.In
stit
ut
fur
Urh
eber
un
dM
edie
nre
cht,
Mu
nch
en1
26.
UB
Un
iH
eid
elb
erg
/Urh
eber
rech
tin
der
Info
rmat
ion
sges
ells
chaf
t1
27.
Bib
liot
hek
sver
ban
dW
ebsi
te1
UR
Ln
otfo
un
d28
.S
chu
len
ans
Net
ze.
V.
1
(con
tinued
)
Table AII.
ROI in Germanlibraries
173
IDW
ISR
efer
ence
sP
aid
4In
tern
etR
esou
rce
Loa
nsh
arin
g@ U
BIn
tern
et-
bas
edC
omm
ents
29.
Bu
nd
esm
inis
teri
um
der
Just
iz1
UR
Ln
otco
mp
lete
30.
iRig
hts
Urh
eber
rech
tin
der
dig
ital
enW
elt
131
.In
stit
ute
for
Sci
ence
Net
wor
kin
gO
lden
bu
rg1
32a.
Info
rmat
ion
onp
ub
lica
tion
sb
yK
uh
len
all
list
edin
the
bib
liog
rap
hy
Not
app
lica
ble
32b
.P
olte
rman
,A
.(2
002)
Gu
tzu
Wis
sen
133
.K
uh
len
,R
.(2
008)
Erf
olg
reic
hes
Sch
eite
rn1
34.
An
hor
un
gd
esB
un
des
tag
/Urh
eber
rech
t1
35.
Info
rmat
ion
Not
app
lica
ble
36.
Info
rmat
ion
Not
app
lica
ble
37.
Info
rmat
ion
Not
app
lica
ble
1.U
rheb
erre
chts
bu
nd
nis
,G
otti
ng
erE
rkla
run
g1
2.B
enk
ler,
Y.
(200
6),
Th
ew
ealt
hof
net
wor
ks.
How
soci
alp
rod
uct
ion
tran
sfor
ms
mar
ket
san
dfr
eed
om1
3,P
ress
ein
form
atio
nd
esB
orse
nv
erei
nd
esd
euts
ch.B
uch
han
del
s20
061
4.S
tell
un
gn
ahm
ed
esB
un
des
rate
s.E
ntw
urf
ein
esZ
wei
ten
Ges
etze
s1
5.C
aste
lls,
M.(
1996
)T
he
rise
ofth
en
etw
ork
soci
ety
.Th
ein
form
atio
nag
e:E
con
omy
,so
ciet
yan
dcu
ltu
re1
6.D
FG
Em
pfe
hlu
ng
end
esA
uss
chu
sses
fur
wis
s.B
ibli
oth
eken
“Ak
tuel
leA
nfo
rder
un
gen
der
wis
s.In
form
atio
nsv
erso
rgu
ng
”1
7.D
FG
“Pu
bli
kat
ion
sstr
ateg
ien
imW
and
el”.
Erg
ebn
isse
ein
erU
mfr
age
18.
DF
G-P
osit
ion
spap
ier:
“Ele
ktr
onis
ches
Pu
bli
zier
en”.
Em
pfe
hlu
ng
end
esU
nte
rau
ssch
uss
es1
9.E
uro
pai
sch
eK
omm
issi
on:
Mit
teil
un
gd
erK
omm
issi
onan
das
EU
Par
lam
ent
110
.C
oun
cil
Con
clu
sion
son
scie
nti
fic
info
rmat
ion
inth
ed
igit
alag
eN
ov.
2007
111
.A
bsc
hlu
ßb
eric
ht
des
Eu
rop
ean
Res
earc
hA
dv
isor
yB
oard
112
.F
ourn
ier,
J.(2
007)
OA
ind
erd
tsch
.F
orsc
hu
ng
sgem
ein
sch
aft
113
.G
oun
alak
is,
G.
(200
7)E
inn
euer
Mor
gen
fur
den
Wis
sen
sch
afts
par
agra
fen
1
(con
tinued
)
Table AII.
BL23,4
174
IDW
ISR
efer
ence
sP
aid
4In
tern
etR
esou
rce
Loa
nsh
arin
g@ U
BIn
tern
et-
bas
edC
omm
ents
14.
Die
Gru
nen
:U
rheb
erre
cht
imd
igit
alen
Zei
talt
er1
15.
Han
sen
,G
.(2
005)
Zu
gan
gzu
wis
s.In
form
atio
n–
alte
rnat
ive
urh
eber
rech
tlic
he
An
satz
eG
RU
R1
16.
Han
sen
,G
.(2
006)
Ind
ieZ
uk
un
ftp
ub
lizi
eren
(Ste
mp
fhu
ber
)1
17.
Hil
ty,R
.M.(
2006
)D
asU
rheb
erre
cht
un
dd
erW
isse
nsc
haf
tler
GR
UR
118
.H
ilty
,R
.M.
(200
7)S
un
den
boc
kU
rheb
erre
cht?
119
.H
oere
n,
T.
(200
7)U
rheb
erre
cht
ind
erW
isse
nsg
esel
lsch
aft
In:
Au
sP
olit
iku
nd
Wis
sen
sch
aft
120
.H
oere
n,
Tan
dK
och
erJ.
K.
(200
5)D
erW
isse
nsc
haf
tler
als
Au
tor
121
.H
oere
n,
T.
and
Kal
lber
g,
N.
(200
6)D
eram
erik
anis
che
TE
AC
HA
ct1
22.
Hu
gen
hol
tz,P
.B.a
nd
Ok
edij
i,R
.L.(
2008
)Con
ceiv
ing
anin
tern
atio
nal
inst
rum
ent
onli
mit
atio
ns
and
exce
pti
ons
toco
py
rig
ht
123
.Ja
szi,
P.
(200
5)P
ub
lic
inte
rest
exce
pti
ons
inco
py
rig
ht
124
.K
uh
len
,R
.(2
008)
Got
terd
amm
eru
ng
125
.K
uh
len
,R
.(2
006)
Op
enIn
nov
atio
n:
Tei
lei
ns
126
.L
itm
an,
J.(2
001)
Dig
ital
Cop
yri
gh
t:P
rote
ctin
gin
tell
ectu
alp
rop
erty
onth
ein
tern
et1
27.
Lu
tter
bec
k,
B.
and
Geh
rin
g,
R.
(200
3)K
riti
kau
sd
erS
ich
tei
nes
Hoc
hsc
hu
lleh
rers
un
dei
nes
Wis
sen
sch
aftl
ers,
den
Ges
etze
ntw
urf
der
Bu
nd
esre
gie
run
gzu
mU
rhG
un
dd
ieU
nte
rric
htb
esti
mm
un
gfu
rU
nte
rric
ht
un
dF
orsc
hu
ng
bet
reff
end
.1
28.
Nen
twic
h,M
.(20
03)C
yb
ersc
ien
ce.R
esea
rch
inth
eag
eof
the
inte
rnet
129
.P
flu
ger
,T.
(200
7)O
pen
Acc
ess,
Ch
ance
nu
nd
Her
ausf
ord
eru
ng
en–
ein
Han
db
uch
130
.R
oyal
Aca
dem
y(2
003)
Kee
pin
gS
cien
ceO
pen
131
.S
pin
dle
r,G
.(2
006)
Rec
htl
ich
eR
ahm
enb
edin
gu
ng
env
onO
A-P
ub
lik
atio
nen
132
.S
utt
orp
,A.(
2005
)Die
offe
ntl
ich
eZ
ug
ang
lich
mac
hu
ng
fur
Un
terr
ich
tu
nd
For
sch
un
g1
012
1636
Alt
oget
her
64
Table AII.
ROI in Germanlibraries
175
Lu
KII
Ref
eren
ces
Pai
d4
Inte
rnet
Res
ourc
eL
oan
shar
ing
@ UB
Inte
rnet
-b
ased
Com
men
ts
1.O
pen
DO
AR
1L
ink
avai
lab
lev
iaO
PA
C2.
Sta
tist
ic@
Sta
tCou
nte
r1
3.A
CM
Por
tal/C
onfe
rence
Pro
ceed
ings
2008
AC
M/I
EE
E-C
Sjo
int
con
fere
nce
14.
Sea
dle
/Gre
ifen
eder
“In
Arc
hiv
ing
we
tru
st”
1E
mer
ald
5.O
pen
DO
AR
1L
ink
avai
lab
lev
iaO
PA
C6.
Her
itri
x1
7.In
stal
lin
gL
OC
KS
S1
20
23
Alt
oget
her
7
Table AIII.
BL23,4
176
Eu
rop
ean
aC
onn
ect
Ref
eren
ces
Pai
d4
Inte
rnet
Res
ourc
eL
oan
shar
ing
@U
BIn
tern
et-
bas
edC
omm
ents
1.In
form
atio
nN
otap
pli
cab
le2.
Info
rmat
ion
Not
app
lica
ble
3.L
ink
zuE
uro
pea
nal
ocal
.se
Pro
jek
tbes
rkiv
enin
g05
081
Des
crip
tion
ofp
roje
ct4.
Wik
iped
ia1
5.In
form
atio
non
Mil
esto
nes
Not
app
lica
ble
1.L
ink
inte
xt/
Cre
ativ
eC
omm
ons
Liz
ense
n1
2.L
ink
inte
xt/
OA
I-O
RE
13.
Lin
kin
tex
t/E
uro
pea
nC
omm
issi
onJo
int
Res
earc
hC
entr
e1
4.L
ink
inte
xt/
Eu
roW
ord
Net
Fin
alR
esu
lts
15.
Lin
kin
tex
t/X
DX
FD
icti
onar
y1
6.L
ink
inte
xt/
On
lin
ed
icti
onar
y–
Loo
kw
ayu
p1
7.L
ink
inte
xt/
On
lin
etr
ansl
ater
18.
Lin
kin
tex
t/O
nli
ne
tran
slat
or1
9.L
ink
inte
xt/
Tin
yU
RL
for
goo
gle
map
slo
cati
on1
10.
Lin
kin
tex
t/E
uro
pea
nC
omm
issi
onIN
SIR
EG
eop
orta
l1
11.
Lin
kin
tex
t/W
ebW
idg
etfo
rV
isu
aliz
ing
Tem
por
alD
ata
112
.A
sab
ove
w/
dif
fere
nt
dep
icti
on1
13.
Lin
kin
tex
t/D
ISM
AR
Cau
dio
stor
age
dat
abas
e1
14.
Lin
kin
tex
t/D
BP
edia
115
.L
ink
inte
xt/
Tex
tGri
dV
ern
etzt
eF
orsc
hu
ng
sum
geb
un
gin
den
eHu
man
itie
s1
16.
Lin
kin
tex
t/C
yb
erIn
fras
tru
ctu
reO
urC
ult
ura
lCom
mon
wea
lth
1N
otap
pli
cab
le/L
ink
bro
ken
17.
Lin
kin
tex
t/B
ibn
etL
ux
emb
ourg
118
.L
ink
inte
xt/
Nat
ion
alb
ibli
oth
ekL
ux
emb
ourg
119
.L
ink
inte
xt/
Ru
ssia
nL
ibra
ryH
erit
age
onli
ne
120
.L
ink
inte
xt/
Fra
nk
van
Har
mel
enW
ebsi
te1
21.
Lin
kin
tex
t/L
arg
eK
now
led
ge
Col
lid
er1
22.
Lin
kin
tex
t/A
nto
ine
Isaa
cW
ebsi
te1
23.
Lin
kin
tex
t/S
UB
Got
tin
gen
124
.L
ink
inte
xt/
Un
iver
sity
ofO
xfo
rd1
00
125
Alt
oget
her
26
Table AIV.
ROI in Germanlibraries
177
Gal
atea
sR
efer
ence
sP
aid
4In
tern
etR
esou
rce
Loa
nsh
arin
g@ U
BIn
tern
et-
bas
edC
omm
ents
1.In
form
atio
nN
otap
pli
cab
le2.
Info
rmat
ion
Not
app
lica
ble
3.In
form
atio
nN
otap
pli
cab
le4.
Info
rmat
ion
Not
app
lica
ble
5.F
ueg
i,D
.an
dJe
nn
ing
s,M
.(2
004)
,“I
nte
rnat
ion
alli
bra
ryst
atis
tics
”1
1.P
roce
edin
gsof
the
7th
IWP
T(R
oux
,C
han
od,
Sal
ah)
2001
1L
iste
dat
Un
iT
rier
,ac
cess
ible
via
Cit
eSee
r2.
ICS
DP
roce
edin
gs,
Ber
nad
iet
al.,
Mu
ltil
ing
ual
Acc
ess
toL
ibr.
Cat
alog
ues
l1
3.T
he
Annals
ofA
pplie
dS
tati
stic
s,B
lei
and
Laf
fert
y
1
KO
BV
/ZD
Bfr
eeac
cess
afte
r36
mon
ths
via
Eu
clid
bu
tn
osu
bsc
rip
tion
via
UB
4.C
LE
F20
08,
Bos
caan
dD
ini
15.
ICS
DP
roce
edin
gs,
Bos
caan
dD
ini
(200
9),
Th
eR
ole
ofL
ogs
inIm
pro
vin
g...
16.
Pro
ceed
ings
ofth
eW
orks
hop
onS
tati
stic
alM
ach
ine
Tra
nsl
ati
on,
Cal
liso
n-B
urc
h,
For
dy
ce,
Koe
hn
,et
al.
17.
Ch
urc
h,
K.
and
Hov
y,
C.
(199
3),
“Goo
dap
pli
cati
ons
for...“
,M
ach
.T
ranls
.8
18.
Cu
cerz
an,
S.
and
Yar
owsk
y,
D.
(199
9),P
roce
edin
gsJo
int
SIG
DA
TC
onfe
rence
19.
Cu
rton
i,P
.an
dD
ini,
L.
(200
6),
CL
EF
2006
Wor
kin
gN
otes
110
.In
form
ati
onP
roce
ssin
gand
Manage
men
t,D
un
lav
y,
D.M
.an
dO
’Lea
ryD
.P.
(200
7)1
11.
Com
munic
ati
ons
ofth
eA
CM
,Etz
ion
i,O
.et
al.
(200
8),O
pen
Inf.
Ex
trac
tion
from
the
Web
112
.G
uil
ian
o,C
.P
roce
edin
gsof
the
13th
Con
fere
nce
onC
ompu
tati
onalN
at.
Lang.
Lea
rnin
g(2
009)
(CoN
LL
)1
13.
Pro
ceed
ings
ofth
eN
ati
onalA
cadem
yof
Sci
ence
,Gri
ffith
s,T
.an
dS
tey
ver
s,M
.(2
004)
1Jo
urn
al
(con
tinued
)
Table AV.
BL23,4
178
Gal
atea
sR
efer
ence
sP
aid
4In
tern
etR
esou
rce
Loa
nsh
arin
g@ U
BIn
tern
et-
bas
edC
omm
ents
14.
Gri
ffith
s,T
.,T
ech
nic
alR
epor
t1
Via
Cit
eSee
r15
.H
ein
rich
,G
.,T
ech
nic
alR
epor
t1
Via
Cit
euL
ike
16.
Hof
man
n,
T.,
Pro
ceed
ings
ofU
AI
(199
9)1
Sou
rce
has
mov
edfr
omU
AI
web
site
17.
Lib
rary
and
Info
rmati
onS
cien
ceR
esea
rch
(200
6),
Jan
sen
,B
.J.
1N
atio
nal
lice
nce
18.
Ste
ve,
J.et
al.
(200
0),In
t.Jo
urn
alon
Dig
.L
ibra
ries
119
.P
roce
edin
gsof
ICA
SS
P,
Kn
igh
t,K
.an
dM
arcu
,D
.(2
005)
120
.K
oeh
n,
P.
and
Mon
z,C
.(2
006)
,P
roce
edin
gsof
NA
AC
L1
AC
M21
.K
oeh
n,
P.
and
Mon
z,C
.(2
005)
,P
roce
edin
gsof
AC
LW
orks
hop
122
.K
oeh
n,
P.
and
Hie
u,
H.
(200
7),
Con
fere
nce
onE
mp
iric
alM
eth
ods...
(EM
NL
P)
1V
iaK
OB
V23
.K
oeh
n,
P.et
al.
(200
7)A
nn
ual
Mee
tin
gof
the
Ass
ocia
tion
for
Com
pu
tati
onal
Lin
gu
isti
cs(A
CL
)1
24.
Lop
ez,
A.
(200
8),
AC
MC
omp
uti
ng
Su
rvey
s1
25.
Mik
hee
v,
A.et
al.
(199
9),P
roce
edin
gsof
EA
CL
126
.O
ard
,D
.W.
and
Och
,F
.J.
(200
3),P
roce
edin
gsM
TS
um
mit
IX1
On
ly19
89v
iaK
OB
V27
.P
opes
cuet
al.
(200
7),
Wor
ksh
opon
Sem
anti
cE
val
uat
ion
s1
28.
Qia
ng
Pu
and
Gu
o-W
eiY
ang
(200
6),A
dv
ance
sin
Neu
ralW
ork
s1
29.
Lin
gvis
ticæ
Inve
stig
atio
nes
(200
7),S
teib
erg
er,R
.an
dP
oliq
uen
,B.,
Ban
d30
1Y
es,
bu
tn
otN
o.30
/200
730
.W
ei,
X.
and
Cro
ft,
W.B
.,P
roce
edin
gsof
AC
MS
IGIR
1 126
112
Alt
oget
her
31
Table AV.
ROI in Germanlibraries
179
DO
AR
CR
efer
ence
sP
aid
4In
tern
etR
esou
rce
Loa
nsh
arin
g@ U
BIn
tern
et-
bas
edC
omm
ents
1.O
AI-
PM
H1
2.S
cien
ceC
itat
ion
Ind
ex1
3.W
ebof
Sci
ence
14.
Info
rmat
ion
Not
app
lica
ble
5.H
ilse
,H
.-W.
and
Kot
he,
J.(2
006)
,Im
plem
enti
ng
Per
sist
ent
Iden
tifier
s1
6.L
arew
nce
,S
.et
al.
(199
9),IE
EE
Com
pute
r,32
17.
Cit
eSee
r1
8.B
rod
y,T
.(20
03),
Con
fere
nce
onW
orld
wid
eC
oher
ent
Wor
kfo
rce
19.
Bro
dy
,T
.et
al.,
“Cit
atio
nIm
pac
tof
OA
Art
icle
sv
sA
rtic
les
avai
lab
le...”
(on
goi
ng
)1
10.
Goo
gle
Sch
olar
111
.E
doc
-Ser
ver
/Din
i-S
chri
ften
112
.M
cVei
gh
,M
.E.
(200
4),
“Op
enA
cces
sin
the
ISI...
”1
13.
Har
dy
,R
.et
al.
(200
5),
“OA
Cit
atio
ns
Info
rmat
ion
/ep
rin
ts”
1JI
SC
and
Un
iver
sity
ofS
outh
amp
ton
14.
Ku
rtz,
M.G
.et
al.,
“Th
eef
fect
ofu
sean
dac
cess
onci
tati
ons”
115
.T
ong
,J.
(200
6),
Cit
atio
nS
tyle
Gu
ide
for
Inte
rnet
and
Ele
ctro
nic
Sou
rces
116
.B
ran
des
,J.
(200
6),G
uid
eto
Cit
ati
on:
apa
styl
e1
Lin
kb
rok
en17
.v
anH
illa
rd,
E.,
Fu
llT
ext
Mag
azin
eA
rtic
lefr
omD
atab
ase
118
.M
ayr,
P.
(200
6),
“Res
earc
hE
val
uat
ion
,C
onst
ruct
ing
exp
erim
enta
lin
dic
ator
s...”
119
.G
un
dle
r,J.
(200
5),
“In
die
Zu
ku
nft
pu
bli
zier
en–
Her
ausf
ord
eru
ng
enan
das
Pu
bli
zier
en...”
120
.IF
Q1
21.
AC
MP
orta
l1
22.
DB
LP
Com
pu
ter
Sci
ence
Lib
rary
123
.A
rXiv
134
.S
PIR
ES
HE
P1
25.
Kri
chel
,T
.,R
ePE
cR
epor
tsin
Eco
nom
ics
1
(con
tinued
)
Table AVI.
BL23,4
180
DO
AR
CR
efer
ence
sP
aid
4In
tern
etR
esou
rce
Loa
nsh
arin
g@ U
BIn
tern
et-
bas
edC
omm
ents
26.
Ed
oc-S
erv
erD
isse
rtat
ion
s-F
orm
atv
orla
ge
127
.B
ibst
er1
28.
Haa
se,
P.
(200
4),
“Bib
ster
–A
Sem
anti
c...,
”in
Pro
ceed
ings
ofth
eIn
t.S
emanti
cW
ebC
onfe
rence
129
.H
aase
,P
.et
al.,
inP
roce
edin
gsof
the
Int.
Sem
anti
cW
ebC
onfe
rence
130
.B
ibst
er,
Lin
ux
-Dev
elop
erF
assu
ng
131
.R
ich
ard
Cam
eron
/Cit
eUL
ike
Not
app
lica
ble
32.
VIK
EF
133
.K
apla
n,
A.,
San
dor
,A
.an
dR
ond
eu,
G.
(200
6),
“Dis
cou
rse
and
cita
tion
...”
134
.E
ER
QI
135
.H
yp
erjo
urn
al1
36.
DB
IS-D
aten
ban
kIn
fosy
stem
137
.C
atal
ogu
eB
n-O
pal
ep
lus
138
.S
PIN
and
My
SC
itat
ion
139
.E
BS
CO
140
.B
ibso
nom
y1
41.
Info
rmat
ion
Not
app
lica
ble
42.
Th
eD
OI
Sy
stem
143
.O
pen
UR
L1
44.
Pow
ell,
A.,
Dis
trib
ute
dS
yst
ems:
Op
enU
RL
dem
onst
rato
r1
45.
Hir
sch
,J.
E.,
Pro
ceed
ings
ofth
eN
ati
onalA
cadem
yof
Sci
ence
s,10
21
46.
Mar
shak
ova,
(197
3),I.V
.N
ach
no-
Tek
hnic
hes
kaya
Info
rmats
iya
Ser
iya
21
47.
Sm
all,
H.
(199
4),Jo
urn
alof
Am
eric
an
Soc
iety
for
Info
rmati
onS
cien
ce,
241
48.
Kes
sler
,M
.M.
(196
3),A
mer
ican
Doc
um
enta
tion
1N
atio
nal
lice
nce
49.
Sm
all,
H.
(199
4),S
cien
tom
etri
cs1
(con
tinued
)
Table AVI.
ROI in Germanlibraries
181
DO
AR
CR
efer
ence
sP
aid
4In
tern
etR
esou
rce
Loa
nsh
arin
g@ U
BIn
tern
et-
bas
edC
omm
ents
50.
Ch
en,
C.
and
Car
r,L
.(1
999)
,“V
isu
aliz
ing
the
Ev
olu
tion
ofa
Su
bje
ctD
omai
n:
AC
ase
Stu
dy
“1
51.
Kat
z,S
.(2
006)
,S
cien
ceand
Publ
icP
olic
y1
Via
AC
Mv
ersi
onfr
om20
00fo
un
d,
ver
sion
from
2006
nei
ther
atA
CM
nor
atau
thor
’sw
ebsi
te52
.S
oler
,J.
(200
6),
“AR
atio
nal
Ind
icat
orof
Sci
enti
fic
Cre
ativ
ity
”1
53.
Ber
ner
s-L
ee,T
.et
al.,
Sci
enti
fic
Am
eric
an
,“T
he
Sem
anti
cW
eb”
154
.B
ern
ers-
Lee
,T.e
tal.
(200
5),“
Jou
rnal
pu
bli
shin
gan
dau
thor
self
-ar
chiv
ing
:P
eace
ful...
”1
55.
Bol
len
,J.e
tal.
(200
5),I
nfo
rmati
onP
roce
ssin
gand
Manage
men
t1
56.
Bro
dy
,T
.et
al.
(200
6),Jo
urn
alof
the
Am
eric
an
Ass
ocia
tion
for
Info
rmati
onS
cien
ceand
Tec
hnol
ogy
157
.G
arfi
eld
,E
.(1
955)
,“C
itat
ion
Ind
exes
for
Sci
ence
:A
New
Dim
ensi
on...”
158
.G
arfi
eld
,E
.(1
962-
1973
),“C
itat
ion
Fre
qu
ency
asa
Mea
sure
ofR
esea
rch
Act
ivit
yan
dP
erfo
rman
ce”
159
.H
arn
ad,
S.
(200
1),
“Res
each
acce
ss,
imp
act
and
asse
ssm
ent”
160
.H
arn
ad,S
.etal.
(200
3),“
Man
dat
edon
lin
eR
AE
CV
sli
nk
edto
...
Ari
adn
e”1
61.
Kle
inb
erg
,M
.J.,
“Hu
bs,
Au
thor
itie
s,an
dC
omm
un
itie
s”,
AC
M1
62.
Ku
rtz,
M.J
.(2
004)
,“T
he
Eff
ect
ofU
sean
dA
cces
son
Cit
atio
ns”
163
.L
and
auer
,T
.K.
(199
8),
“In
trod
uct
ion
toL
aten
tS
eman
tic
An
aly
sis”
1N
otav
aila
ble
for
HU
64.
New
man
,M
.E.J
.,P
roce
edin
gsof
the
Nati
onalA
cadem
yof
Sci
ence
s1
65.
Pag
e,L
.etal.,
“Th
eP
ageR
ank
Cit
atio
nR
ank
ing
:Bri
ng
ing
Ord
erto
the
Web
”1
66.
Sm
ith
,A
.;E
yse
nck
,M
.(2
002)
,“T
he
corr
elat
ion
bet
wee
nR
AE
rati
ng
san
dci
tati
onco
un
tsin
Psy
chol
ogy
”1
67.
Moe
d,
H.F
.(2
005)
,“C
itat
ion
An
aly
sis
inR
esea
rch
Ev
alu
atio
n”
1
(con
tinued
)
Table AVI.
BL23,4
182
DO
AR
CR
efer
ence
sP
aid
4In
tern
etR
esou
rce
Loa
nsh
arin
g@ U
BIn
tern
et-
bas
edC
omm
ents
68.
“Th
eef
fect
ofop
enac
cess
and
dow
nlo
ads
(‘hit
s’)
onci
tati
onim
pac
t:a
bib
liog
rap
hy
ofst
ud
ies”
169
.P
ub
lik
atio
nsa
rch
ivd
esIn
stit
uts
fur
Ph
ysi
k,
Old
enb
urg
170
.P
ub
lik
atio
nen
des
Inst
itu
tefo
rS
cien
ceN
etw
ork
ing
171
.D
ien
ste-
En
twic
klu
ng
end
esIn
stit
ute
for
Sci
ence
Net
wor
kin
g1
72.
Zim
mer
man
n,K
.et
al.
(200
1),D
isse
rtat
ion
enO
nli
ne,
“Wor
kfl
owu
nd
Nac
hw
eis
fur
Dis
sert
atio
nen
”1
73.
Dis
sert
atio
nO
nli
ne
174
.P
hy
sNet
Ph
ysi
csN
etw
ork
Wor
ldw
ide
175
.M
yM
eta
Mak
er1
76/
Op
enA
rch
ives
Dis
trib
ute
dO
AD
177
.D
C-C
hec
ker
178
.L
iLi
–L
ink
list
end
erP
hy
sik
:eL
earn
ing
Mat
eria
lien
179
.D
ocu
men
ten
Ret
riev
alS
yst
emD
oRe
des
GS
IG
roß
fors
chu
ng
sin
stit
uts
,D
arm
stad
t:1
80.
En
gin
eE
rzeu
ger
Inn
en-N
utz
erIn
nen
-Gu
tach
terI
nn
en-I
nte
rak
tion
bei
pro
fess
ion
elle
mN
ach
wei
sv
erte
ilte
rm
ult
imed
iale
reL
earn
ing
-Ob
jek
te1
81.
Hil
f,E
.R.
and
Mim
kes
,J.
(200
2),
“Zu
ein
emv
erlu
stfr
eien
Pu
bli
zier
enu
nd
Arc
hiv
iere
n–
Mat
hem
atis
che
Au
ssag
enin
Ph
ysi
ku
nd
Ch
emie
”1
82.
Hil
f,E
.R.
(200
5),
Kan
nm
anT
eXb
eib
rin
gen
,P
hy
sik
zuv
erst
ehen
183
.H
ilf,
E.R
.et
al.
(200
8),
Mat
hem
atic
alK
now
led
ge
Man
agem
ent
184
.K
rich
el,
T.,
Aca
dem
icC
ontr
ibu
tor
Info
rmat
ion
Sy
stem
185
.H
ilf,
E.R
.an
dK
rich
el,
T.
(200
2),
42n
dS
tree
tP
aper
:“S
chol
arly
Com
mu
nic
atio
nis
auth
or-d
riv
enra
ther
than
read
er-d
riv
en”
186
.E
uro
scie
nce
Con
fere
nce
,M
un
ich
2006
187
.S
had
bol
t,N
.etal.
(200
6),“
Op
enA
cces
s:K
eyS
trat
egic
,Tec
hn
ical
and
Eco
nom
icA
spec
ts”
188
.H
arn
ad,
S.et
al.
(200
4)S
eria
lsR
evie
w,
301
(con
tinued
)
Table AVI.
ROI in Germanlibraries
183
DO
AR
CR
efer
ence
sP
aid
4In
tern
etR
esou
rce
Loa
nsh
arin
g@ U
BIn
tern
et-
bas
edC
omm
ents
89.
Hav
eman
n,
F.,
Sch
rift
env
erze
ich
nis
190
.H
avem
ann
,F.(
2002
),“B
ibli
omet
risc
he
Dat
enfu
rd
ieD
ebat
teu
md
enW
and
eld
erU
niv
ersi
tat”
191
.H
avem
ann
,F
.(2
003)
,“B
ibli
omet
ric
ind
icat
ors
and
thei
ru
sefo
rre
sear
chev
alu
atio
n–
anan
aly
sis
ofh
igh
lyp
rod
uct
ive
bio
med
ial
team
s”1
92.
Hav
eman
n,
F.
(200
1),
“Col
lab
orat
ion
beh
avio
ur
ofB
erli
nli
fesc
ien
cere
sear
cher
sin
the
last
two
dec
ades
...”
,S
cien
tom
etri
cs1
93.
Hav
eman
n,F
.(20
01),
“Bib
liom
etri
sch
eA
nal
yse
von
Koo
per
atio
nu
nd
Pro
du
kti
vit
atb
iom
ediz
inis
cher...”
194
.H
avem
ann
,F
.,H
ein
z,M
.an
dK
rets
chm
er,
H.
(200
6),Jo
urn
alof
Bio
med
icalD
isco
very
and
Col
labo
rati
on1
95.
Sch
mid
t,M
.et
al.
(200
6),
Inte
rnat
ion
alW
ork
shop
onW
ebom
etri
cs,
Info
rmet
rics
and
Sci
ento
met
rics
and
Sev
enth
CO
LL
NE
TM
eeti
ng
196
.B
eren
dt,
B.
and
Hav
eman
n,
F.
(200
7),
“Bes
chle
un
igu
ng
der
Wis
sen
sch
afts
kom
mu
nik
atio
nd
urc
hO
pen
Acc
ess...”
197
.H
avem
ann
,F
.(2
004)
,“E
pri
nts
ind
erw
isse
nsc
haf
tlic
hen
Kom
mu
nik
atio
n”
198
.H
avem
ann
,F
.(2
004)
,“G
row
thd
yn
amic
sof
Ger
man
un
iver
sity
enro
lmen
tsan
dof
scie
nti
fic
dis
cip
lin
esin
the
19th
cen
tury
”1
99.
Hav
eman
n,
F.et
al.
(200
5),Jo
urn
alof
the
Am
eric
an
Soc
iety
for
Info
rmati
onS
cien
ceand
Tec
hnol
ogy
110
0.L
ian
g,
L.et
al.
(200
6),
“Str
uct
ura
lsi
mil
arit
ies
bet
wee
nsc
ien
ceg
row
thd
yn
amic
sin
Ch
ina...”
110
1.C
hri
sten
,W
.(2
001)
,C
lust
erS
cien
ceN
etC
SN
.1
102.
DO
AR
C1
103.
Info
rmat
ion
Not
app
lica
ble
104.
Ph
ysN
et1
105.
Info
rmat
ion
Not
app
lica
ble
(con
tinued
)
Table AVI.
BL23,4
184
DO
AR
CR
efer
ence
sP
aid
4In
tern
etR
esou
rce
Loa
nsh
arin
g@ U
BIn
tern
et-
bas
edC
omm
ents
106.
Th
eA
CIS
Aca
dem
icC
ontr
ibu
tor
Info
rmat
ion
Sy
stem
Con
sort
ium
110
7.In
form
atio
nN
otap
pli
cab
le10
8.P
ub
lik
atio
nsl
iste
des
Inst
itu
tsfu
rP
hy
sik
,O
lden
bu
rg1
109.
Ph
ysi
k,
Old
enb
urg
111
0.In
form
atio
nN
otap
pli
cab
le11
1.In
form
atio
nN
otap
pli
cab
le22
413
64A
ltog
eth
er10
3
Table AVI.
ROI in Germanlibraries
185
EE
RQ
IR
efer
ence
sP
aid
4In
tern
etR
esou
rce
Loa
nsh
arin
g@ U
BIn
tern
et-
bas
edC
omm
ents
1.L
ee,
K.P
.et
al.
(200
2),
JAM
A1
2.B
utl
er,
L(2
006)
,“B
ibli
omet
rics
and
Res
earc
hP
erfo
rman
ceF
ram
ewor
kIn
dic
ator
sF
oru
m/U
niv
ersi
tyof
New
Sou
thW
ales
”1
Lin
kb
rok
en3.
CE
RIF
1L
ink
bro
ken
4.B
oor,
R.M
.(1
982)
,A
mer
ican
Psy
chol
ogis
t1
5.M
oed
,H
.F.
(200
5),C
itati
onA
naly
sis
inR
esea
rch
Eva
luati
on,
Sp
rin
ger
16.
Hic
ks,
D.
(200
4),H
andbo
okof
Quanti
tati
veS
cien
ceand
Tec
hnol
ogy
Res
earc
h1
7.P
hil
ipp
e,J.
and
Dev
illa
rd,
J.(2
005)
,“I
mp
lem
enti
ng
rele
van
td
isci
pli
nar
yev
alu
atio
ns
inth
eso
cial
scie
nce
s”1
8.E
uro
pea
nC
omm
issi
on:
Stu
dy
onth
eec
onom
ican
dte
chn
ical
evol
uti
onof
the
scie
nti
fic
pu
bli
cati
onm
ark
ets
inE
uro
pe
19.
Info
rmat
ion
Not
app
lica
ble
10.
Moe
d,
H.F
.(2
005)
,C
itati
onA
naly
sis
inR
esea
rch
Eva
luati
on,
Sp
rin
ger
111
.M
oed
,H
.F.
(200
5),Jo
urn
alof
the
Am
eric
an
Soc
iety
for
Info
rmati
onS
cien
ceand
Tec
hnol
ogy
112
.B
rod
y,
T(2
004)
,“C
itat
ion
An
aly
sis
inth
eO
pen
Acc
ess
Wor
ld”
113
.P
ark
,H.W
.an
dT
hel
wel
l,M
.(20
03),
“Hy
per
lin
kA
nal
yse
sof
the
Wor
ldW
ide
Web
:A
Rev
iew
”1
14.
Bro
dy
,T
.et
al.
(200
6),Jo
urn
alof
the
Am
eric
an
Ass
ocia
tion
for
Info
rmati
onS
cien
ceand
Tec
hnol
ogy
(JA
SIS
T),
57(8
)1
15.
Car
r,L
.et
al.
(200
6),
“Ex
ten
din
gjo
urn
al-b
ased
rese
arch
imp
act
asse
ssm
ent
tob
ook
-bas
edd
isci
pli
nes
”1
16.
van
Lee
uw
en,
Th
.N.et
al.
(200
1),
“Lan
gu
age
bia
ses
inth
eco
ver
age
ofth
eS
cien
ceC
itat
ion
Ind
exan
dit
sco
nse
qu
ence
sfo
rin
tern
atio
nal...”
Sci
ento
met
rics
117
.P
errs
on,
O.et
al.
(200
4),
“In
flat
ion
ary
bib
liom
etri
cv
alu
es,
the
role
ofsc
ien
tifi
cco
llab
orat
ion...”
,S
cien
tom
etri
cs1
(con
tinued
)
Table AVII.
BL23,4
186
EE
RQ
IR
efer
ence
sP
aid
4In
tern
etR
esou
rce
Loa
nsh
arin
g@ U
BIn
tern
et-
bas
edC
omm
ents
18.
Per
rson
,O
.et
al.
(200
4),
“Han
db
ook
ofq
uan
tita
tiv
esc
ien
cean
dte
chn
olog
yre
sear
ch”
119
.S
chn
eid
er,
J.(2
006)
,“C
once
pt
Sy
mb
ols
Rev
isit
ed...”
,S
cien
tom
etri
cs1
20.
CN
RS
/Fra
nz.
Web
site
121
.In
form
atio
n1
22.
Lin
kin
tex
t/E
ER
Aw
ebsi
te1
23.
Lin
kin
tex
t/P
hy
snet
124
.L
ink
inte
xt/
Mar
ine
Res
earc
hIn
st.
125
.L
ink
inte
xt/
Pro
jek
tU
ni
Kar
lsru
he
126
.L
ink
inte
xt/
CN
RS
127
.L
ink
inte
xt/
Ph
ysi
kM
ult
imed
iaP
roje
kt
Old
enb
urg
128
.L
ink
inte
xt/
Met
ager
129
.L
ink
inte
xt/
Hit
Cou
nte
ru
nd
Lif
eS
tati
stik
en1
30.
Lin
kin
tex
t/R
esea
rch
Por
tal.n
et1
Lin
kb
rok
en31
.L
ink
inte
xt/
Lei
bn
izU
ni
Han
nov
er1
32.
Lin
kin
tex
t/B
otte
,A
.(2
004)
,“A
chie
vem
ent
orp
erfo
rman
ce:
obse
rvat
ion
ofp
rod
uct
ivit
y...”
133
.L
ink
inte
xt/
Els
evie
r(L
earn
ing
and
Inst
ruct
ion,
Journ
al)
1S
ourc
eb
yE
lsev
ier
bu
tn
otp
art
ofth
eH
Ud
eal
34.
Lin
kin
tex
t/E
lsev
ier
(Educa
tion
alR
esea
rch,
Journ
al)
1S
ourc
eb
yE
lsev
ier
bu
tn
otp
art
ofth
eH
Ud
eal
35.
Ast
rom
,F.a
nd
Pet
ters
son
,L.(
2006
),L
ibra
ries
and
Culture
,41/
21
36.
Per
rson
,O
.an
dA
stro
m,
L.
(200
5),B
iblio
met
ric
Not
es,
7/2
137
.A
stro
m,
F.
(200
2),
“Em
erg
ing
Fra
mew
ork
san
dM
eth
ods”
,C
oLIS
41
38.
Vis
ser,
M.S
.an
dM
oed
,H
.F.
(200
4),
“Mea
suri
ng
the
imp
act
ofn
on-I
SI
sou
rce
item
s”1
(con
tinued
)
Table AVII.
ROI in Germanlibraries
187
EE
RQ
IR
efer
ence
sP
aid
4In
tern
etR
esou
rce
Loa
nsh
arin
g@ U
BIn
tern
et-
bas
edC
omm
ents
39.
Vis
ser,
M.S
.an
dM
oed
,H
.F.
(200
5),
“Dev
elop
ing
bib
liom
etri
cin
dic
ator
sor
rese
arch
per
form
ance
inco
mp
ute
rsc
ien
ce”
140
.S
mey
ers,
P.
and
Dep
aep
e,M
.(2
003)
,“B
eyon
dE
mp
iric
ism
:O
nC
rite
ria
for
Ed
uca
tion
alR
esea
rch
”1
41.
Sch
nei
der
,J.
etal.
(200
7)P
roce
edin
gs
ofIS
SI
2007
142
.S
chn
eid
er,
J.an
dB
orlu
nd
,P.
(200
7)M
atri
xco
mp
aris
on/J
AS
IST
143
.S
chn
eid
er,
J.et
al.
(200
6)“B
ibli
omet
ri:
En
bib
liot
eks-
ogin
form
atio
nsv
iden
skab
elig
kom
pet
ence
”1
44.
Sch
nei
der
,J.(
2006
),“C
once
pt
sym
bol
sre
vis
ited
”,S
cien
tom
etri
cs1 13
34
23A
ltog
eth
er43
Table AVII.
BL23,4
188
For
sch
un
gsd
iver
sita
tR
efer
ence
sP
aid
4In
tern
etR
esou
rce
Loa
nsh
arin
g@ U
BIn
tern
et-
bas
edC
omm
ents
1.In
form
atio
nN
otap
pli
cab
le2.
Info
rmat
ion
Not
app
lica
ble
3.P
oin
ter
toa
refe
ren
ceN
otap
pli
cab
le4.
Poi
nte
rto
refe
ren
ces
Not
app
lica
ble
1.A
dam
s,J.
and
Sm
ith
,D
.(2
003)
,F
undin
gR
esea
rch
Div
ersi
ty1
2.A
lter
,O
.et
al.
(200
0),P
roce
edin
gsof
the
Nati
onalA
cadem
yof
Sci
ence
s1
3.B
ord
ons,
M.e
tal.
(200
4),H
andbo
okof
Quanti
tati
veS
cien
ceand
Tec
hnol
ogy
Res
earc
h1
4.C
allo
n,
M.
(199
5),
“Fou
rM
odel
sfo
rth
eD
yn
amic
sof
Sci
ence
”1
5.C
allo
n,
M.et
al.
(198
3),S
ocia
lS
cien
ceIn
form
ati
on,
221
6.D
eerw
este
r,S
.et
al.
“In
dex
ing
by
late
nt
sem
anti
can
aly
sis”
,JA
SIS
T1
7.E
gg
he,
L.
and
Rou
ssea
u,
R.
(199
0),
“In
trod
uct
ion
toIn
form
etri
cs”,
Quanti
tati
veM
ethod
sin
Lib
rary
and
Info
rmati
onS
cien
ce1
8.E
van
s,J.
(200
8),
“Ele
ctro
nic
Pu
bli
cati
onan
dth
eN
arro
win
gof
Sci
ence
and
Sch
olar
ship
”1
9.G
lase
r,J.
(200
6),
“Wis
sen
sch
aftl
ich
eP
rod
uk
tion
sgem
ein
sch
afte
n:
Die
sozi
ale
Ord
nu
ng
der
For
sch
un
g”
110
.G
lase
r,J.
etal.
(200
8),
“Ev
alu
atio
nsb
asie
rte
For
sch
un
gsfi
nan
zier
un
gu
nd
ihre
Fol
gen
/Wis
sen
fur
En
tsch
eid
un
gsp
roze
sse”
111
.G
lase
r,J.
and
Lau
del
,G.(
2007
),“E
val
uat
ion
wit
hou
tE
val
uat
ors:
Th
eim
pac
tof
fun
din
gfo
rmu
lae
onA
ust
rali
anU
niv
ersi
tyR
esea
rch
”1
12.
Gla
ser,
J.et
al.
(200
2),
“Im
pac
tE
val
uat
ion
-bas
edF
un
din
gon
the...”
113
.G
rup
p,H
.(19
90),
“Th
eco
nce
pt
ofen
trop
yin
scie
nto
met
rics
and
inn
ovat
ion
rese
arch
”1
(con
tinued
)
Table AVIII.
ROI in Germanlibraries
189
For
sch
un
gsd
iver
sita
tR
efer
ence
sP
aid
4In
tern
etR
esou
rce
Loa
nsh
arin
g@ U
BIn
tern
et-
bas
edC
omm
ents
14.
Har
ley
,S
.an
dL
ee,
F.S
.(1
997)
,“R
esea
rch
sele
ctiv
ity
,m
anag
eria
lism
,an
dth
eac
adem
icla
bor
pro
cess
”1
15.
Hav
eman
n,F
.et
al.
(200
7),“
Mea
suri
ng
Div
ersi
tyof
Res
earc
hin
Bib
liog
rap
hic
-Cou
pli
ng
Net
wor
ks”
116
.H
ein
z,M
.et
al.
(200
9),“
Sel
bst
org
anis
atio
nin
Wis
sen
sch
aft
un
dT
ech
nik
.Ja
hrb
uch
Wis
sen
sch
afts
fors
chu
ng
2008
”1
17.
HR
K(2
007)
,D
ieZ
uk
un
ftd
erk
lein
enF
ach
er:
Pot
enzi
ale
–H
erau
sfor
der
un
gen
–P
ersp
ekti
ven
.B
onn
118
.Ja
nss
ens,
F.et
al.
(200
7),
“AH
yb
rid
Map
pin
gof
Info
rmat
ion
Sci
ence
”1
19.
Jan
ssen
s,F
.et
al.
(200
6),S
cien
tom
etri
cs1
20.
Join
tst
atem
ent
(200
3),
Th
eH
igh
erE
du
cati
onW
hit
eP
aper
and
rese
arch
fun
din
gse
lect
ivit
y.
18th
Jun
e20
031
21.
Kes
sler
,M.M
.(19
63),
“Bib
liog
rap
hic
cou
pli
ng
bet
wee
nsc
ien
tifi
cp
aper
s”,A
mer
ican
Doc
um
enta
tion
122
.K
nor
r-C
etin
a,R
.(1
982)
,“S
cien
tifi
cC
omm
un
itie
sor
Tra
nse
pis
tem
icA
ren
asof
Res
earc
h?...”
,S
ocia
lS
tudie
sof
Sci
ence
,12
123
.K
nor
r-C
etin
a,R
.(1
999)
,“E
pis
tem
icC
ult
ure
s:H
owth
eS
cien
ces
Mak
eK
now
led
ge”
124
.L
and
auer
,T.e
tal.
(200
7),H
andbo
okof
Late
ntS
emanti
cA
naly
sis
125
.L
atou
r,B
.(1
988)
,“A
Rel
ativ
isti
cA
ccou
nt
ofE
inst
ein
’sR
elat
ivit
y/S
ocia
lS
tud
ies
ofS
cien
ce”,
181
26.
Mag
urr
an,
A.
(200
4),M
easu
ring
Bio
logi
calD
iver
sity
127
.M
ann
,G.S
.et
al.
(200
6),J
CD
L’0
6:P
roce
edin
gsof
the
6th
AC
M/
IEE
E-C
Sjo
int
confe
rence
onD
igit
alL
ibra
ries
128
.M
arsh
akov
a,I.
V.
(197
3),
“Sis
tem
asv
yaz
eym
ezh
du
dok
um
enta
mi,
pos
troy
enn
aya
na
osn
ove
ssy
lok
(po
uk
azat
ely
u‘S
cien
ceC
itat
ion
Ind
ex’)”
1
(con
tinued
)
Table AVIII.
BL23,4
190
For
sch
un
gsd
iver
sita
tR
efer
ence
sP
aid
4In
tern
etR
esou
rce
Loa
nsh
arin
g@ U
BIn
tern
et-
bas
edC
omm
ents
29.
Mit
esse
r,O
.(20
08),
“Lat
ente
sem
anti
sch
eA
nal
yse
zur
Mes
sun
gd
erD
iver
sita
tv
onF
orsc
hu
ng
sgeb
iete
n–
Met
hod
end
isk
uss
ion
un
dA
nw
end
un
gsb
eisp
iel”
130
.M
ites
ser,
O.et
al.
(200
8),
“Mea
suri
ng
Div
ersi
tyof
Res
earc
hb
yE
xtr
acti
ng
Lat
ent...
”,P
roce
edin
gsof
WIS
2008
131
.M
olas
-Gal
lart
,J.a
nd
Sal
ter,
A.(
2002
),“D
iver
sity
and
Ex
cell
ence
:C
onsi
der
atio
ns
onR
esea
rch
Pol
icy
”,IP
TS
Rep
ort
132
.N
oyon
s,E
.C.
and
Van
Ran
,A
.F.J
.(1
998)
,“M
onit
orin
gsc
ien
tifi
cd
evel
opm
ents
from
a...”
,Jo
urn
alof
the
Am
eric
an
Soc
iety
for
Info
rmati
onS
cien
ce1
33.
Per
rson
,O
.et
al.
(200
4)“I
nfl
atio
nar
yb
ibli
omet
ric
val
ues
:th
ero
le...”
,S
cien
tom
etri
cs1
34.
Por
ter,
A.
and
Ch
ub
in,
D.
(198
5),
“An
ind
icat
orof
cros
s-d
isci
pli
nar
yre
sear
ch”,
Sci
ento
met
rics
135
.R
afol
s,I.
and
Mey
er,
M.
(200
7),
”Div
ersi
tym
easu
res
and
net
wor
k...”
,P
roce
edin
gsof
ISS
I2007
136
.R
icot
ta,
C.
(200
4),
“AR
ecip
efo
rU
nco
nv
enti
onal
Ev
enn
ess
Mea
sure
s”,A
cta
Bio
theo
reti
ca,
521
37.
Ric
otta
,C
.an
dS
zeid
l,L
.(2
006)
,“T
owar
ds
au
nif
yin
gap
pro
ach
tod
iver
sity
mea
sure
s...”
,T
heo
reti
calP
opula
tion
Bio
logy
,70
138
.R
ouss
eau
,R
.an
dH
eck
e,P
.V.
(199
9),
“Mea
suri
ng
bio
div
ersi
ty”,
Act
aB
ioth
eori
tica
139
.R
ouss
eau
,R
.et
al.
(199
9),
“Th
ere
lati
onsh
ipb
etw
een
div
ersi
typ
rofi
les”
,...
Envi
ronm
enta
land
Eco
logi
calS
tati
stic
s1
40.
San
z-M
enen
dez
,L
.et
al.
(200
1),
“In
terd
isci
pli
nar
ity
asa
mu
ltid
imen
sion
al...”
,R
esea
rch
Eva
luati
on,
101
41.
Sch
mid
t,M
.(2
006)
,E
ntr
opie
ein
esF
orsc
hu
ng
sgeb
iete
s.T
heo
reti
sch
eE
rarb
eitu
ng
un
dD
urc
hfu
hru
ng
ein
erK
ozit
atio
ns-
Clu
ster
anal
yse
imP
roje
kt
“Tes
tin
gth
eH
omog
enis
atio
nT
hes
is–
Mea
suri
ng
the
Div
ersi
tyof
Res
earc
h”
1
(con
tinued
)
Table AVIII.
ROI in Germanlibraries
191
For
sch
un
gsd
iver
sita
tR
efer
ence
sP
aid
4In
tern
etR
esou
rce
Loa
nsh
arin
g@ U
BIn
tern
et-
bas
edC
omm
ents
42.
Sch
mid
t,M
.et
al.
(200
6),
“Am
eth
odol
ogic
alS
tud
yfo
rM
easu
rin
gth
e...”
,In
tern
ati
onalW
orks
hop
onW
ebom
etri
cs,
Info
rmet
rics
...
143
.S
imp
son
,E
.(1
949)
,“M
easu
rem
ent
ofd
iver
sity
”,N
atu
re,
163
144
.S
mal
l,H
.(1
973)
,“C
o-C
itat
ion
sin
the
Sci
enti
fic...”
,Jo
urn
alof
the
Am
eric
an
Soc
iety
for
Info
rm.S
cien
ce1
45.
Sta
r,S
.(1
989)
,“T
he
stru
ctu
reof
ill-
stru
ctu
red
solu
tion
s...”
,D
istr
ibute
dA
rtifi
cialIn
telli
gence
146
.S
tirl
ing
,A.(
2007
),“A
gen
eral
fram
ewor
kfo
r...”
,Jou
rnalo
fth
eR
oyalS
ocie
tyIn
terf
ace
47.
van
Lee
uw
en,
Tan
dT
ijss
en,
T.
(200
0),
“In
terd
isci
pli
nar
yd
yn
amic
sof
mod
ern...”
,R
esea
rch
Eva
luati
on1
48.
Wh
itle
y,
R.
(200
7),
“Ev
alu
atio
nw
ith
out
Ev
alu
ator
s...”
,T
he
Changi
ng
Gov
ernance
ofth
eS
cien
ces
149
.W
olg
ar,S
.(19
79),
“Th
eId
enti
fica
tion
and
Defi
nit
ion
ofS
cien
tifi
cC
olle
ctiv
es”,
Per
spec
tive
son
...
Not
fou
nd
any
wh
ere/
not
app
lica
ble
50.
Zit
t,M
.an
dB
asse
cou
lard
,E
.(2
006)
,“D
elin
eati
ng
com
ple
xsc
ien
tifi
cfi
eld
s...”
,In
form
ati
onP
roce
ssin
gand
Manage
men
t1 17
1014
7A
ltog
eth
er48
Table AVIII.
BL23,4
192
Car
pet
Ref
eren
ces
Pai
d4
Inte
rnet
Res
ourc
eL
oan
shar
ing
@U
BIn
tern
et-b
ased
Com
men
ts
1.B
ud
apes
tO
AIn
itia
tiv
e1
2.B
erli
ner
Dec
lara
tion
13.
Pu
bli
cK
now
led
ge
Pro
ject
14.
Hyp
erJo
urn
al
15.
Dp
ub
s/D
igit
alP
ub
lish
ing
Sy
stem
16.
Dri
ver
17.
Igit
ur/
Pu
bli
shin
gan
dA
rch
ivin
gS
erv
ice
Un
iver
sity
Lib
rary
Utr
ech
t1
8.A
ePIC
/Ad
van
ced
e-p
ub
lish
ing
Infr
astr
uct
ure
s1
9.D
igit
alP
eer
Pu
bli
shin
gN
RW
110
.R
evu
es.
Org
/Fre
nch
elec
tr.
pu
bli
shin
gsi
te1
11.
ND
LT
D/e
-pu
bli
shin
gw
ebsi
te1
12.
Liv
ing
Rev
iew
s(2
1,22
and
24)
113
.R
ead
Wri
teW
eb/
114
.D
INI
115
.ed
oc-S
erv
er1
16.
SC
OP
E/e
doc
-Ser
ver
117
.D
igit
ale
Dis
sert
atio
nen
On
lin
e1
18.
Lan
gze
itar
chiv
ieru
ng
DE
/Nes
tor
119
.T
EX
Doc
um
ent
Cen
tre
120
.M
edia
con
omy
/For
sch
un
gsv
erb
un
dIn
tern
etok
onom
ied
erG
eorg
-Au
gu
st-U
niv
ersi
tat
Got
tin
gen
123
.H
erm
es/s
eman
tic
XM
Let
c.se
lf-p
ub
lish
ing
tool
125
.M
axP
lan
cked
oc1
26.
ww
w.
esci
doc
.de
127
.In
form
atio
nN
otap
pli
cab
le28
.S
oftw
are
Rev
iew
san
dR
atin
gs
129
.V
ario
us
Lin
ks
such
as:o
hlo
h,i
tera
tin
gan
djo
t.co
m(n
owg
oog
le)
130
.W
arn
er,
S.
(200
4),O
verl
ay
Journ
als
1@
E-L
IS0
05
20A
ltog
eth
er25
Table AIX.
ROI in Germanlibraries
193
Met
a-Im
age
Ref
eren
ces
Pai
d4
Inte
rnet
Res
ourc
eL
oan
shar
ing
@ UB
Inte
rnet
-b
ased
Com
men
ts
1.W
arb
urg
,A
.(2
003)
,“G
esam
mel
teS
chri
ften
/Der
Bil
der
atla
sM
NE
MO
SY
NE
”1
@U
Bon
ly1.
ed.
2.W
arb
urg
,A.(
1992
),“A
usg
ewah
lte
Sch
rift
enu
nd
Wu
rdig
un
gen
”1
3.F
lick
r1
4.d
igil
ib(i
nfo
rmat
ion
)1
5.H
yp
erIm
age
(in
form
atio
n)
(&9.
)1
6.W
arn
ke,
M.
(199
8),
“Ein
ed
igit
ale
Ord
nu
ng
der
Din
ge/
Ein
Hy
per
med
iale
sB
ild
-Tex
t-A
rch
ivzu
...”
17.
War
nk
e,M
.(20
03)D
aten
un
dM
etad
aten
–O
nli
ner
esso
urc
enf.
d.
Bil
dw
isse
nsc
haf
t/Z
eite
nb
lick
e1
8.W
arn
ke,
M.
(200
7),
Hy
per
Imag
e–
Imag
eO
rien
ted
e-S
cien
ceN
etw
ork
s1
Inte
rnet
Res
ourc
e10
Info
rmat
ion
onH
yp
erIm
age
atU
ni
Lu
neb
urg
Not
app
lica
ble
11.
Info
rmat
ion
Not
app
lica
ble
12.
Pro
met
heu
s-B
ild
arch
iv1
13.
Bil
dar
chiv
Ph
oto
Mar
bu
rg1
Mik
rofi
che
bis
1998
/lat
erL
ink
14.
Cen
sus/
An
tiq
ue
wor
ks
and
arch
itec
ture
pic
ofth
eR
enai
ssan
ce(d
atab
ase
orw
ebsi
te)
115
.L
uh
man
n,
N.
(199
2),
“Kom
mu
nik
atio
nM
itZ
ette
lkas
ten
:E
inE
rfah
run
gsb
eric
ht”
116
.s.
o.1
17.
Icon
clas
s1
02
75
Alt
oget
her
14
Table AX.
BL23,4
194
MU
NIN
-RS
Ref
eren
ces
Pai
d4
Inte
rnet
Res
ourc
eL
oan
shar
ing
@U
BIn
tern
et-b
ased
Com
men
ts
1.In
form
atio
nN
otap
pli
cab
le2.
HU
OA
-Erk
laru
ng
13.
Haf
fner
,D
.(2
006)
,D
igit
ale
Bib
lioth
eken
und
Bild
arc
hiv
e/R
undbr
ief
Fot
ogra
fie,
131
4.W
ink
elm
ann
Inst
itu
tfu
rk
lass
isch
eA
rch
aolo
gie
15.
Wes
sel,
Aan
dH
och
,A
.(2
007)
,“C
ave
pla
nth
opp
ers
onH
awai
i...
”,E
volu
tion
inA
ctio
n/A
lsdas
Leb
enla
ufe
nle
rnte
16.
Bu
rrow
s,M
.et
al.
(200
7),
“Ju
mp
ing
beh
avio
ur
ina
gon
dw
anan
reli
ctin
sect
”,T
he
Journ
alof
Exp
erim
enta
lB
iolo
gy1
7.U
MA
CW
orld
wid
eD
atab
ase
ofU
niv
ersi
tyM
use
um
san
dC
olle
ctio
ns/
Aca
dem
icH
erit
age
and
Un
iver
siti
es/m
ore
pro
ject
s...
18.
dl-
foru
m.d
e/d
euts
ch/p
roje
kte
1L
ink
bro
ken
9.C
ISC
Stu
dy
oncy
ber
met
rics
(Sp
anis
h)
1L
ink
bro
ken
10.
Sch
irm
bac
her
,P
.(2
007)
,“O
pen
Acc
ess
–In
form
atio
ns-
un
dR
epos
itor
y-N
etzw
erk
inD
euts
chla
nd
/Bei
trag
zur
Arb
eits
gru
pp
eE
lek
tron
isch
esP
ub
lizi
eren
”1
11.
Med
ien
por
tal
der
HU
112
.V
ollm
er,
A.
(200
5),
“Mn
eme:
Das
dig
ital
eG
edac
htn
is”
113
.L
ink
inT
ext/
Sta
nd
ard
ized
Hy
per
Ad
apta
ble
Met
adat
aE
dit
or1
Not
app
lica
ble
01
47
Alt
oget
her
12
Table AXI.
ROI in Germanlibraries
195
OA
NR
efer
ence
sP
aid
4In
tern
etR
esou
rce
Loa
nsh
arin
g@ U
BIn
tern
et-
bas
edC
omm
ents
1.D
INI
Web
site
/Rep
osit
orie
s1
2.O
AIs
ter
dat
abas
e1
3.S
cop
us
14.
por
tal.i
si.k
now
led
ge
15.
Goo
gle
Sch
olar
16.
Dir
ecto
ryof
OA
Jou
rnal
s/D
OA
J1
7.D
irec
tory
ofO
AR
epos
itor
ies/
DO
AR
18.
Reg
istr
yof
OA
Rep
osit
orie
s/R
OA
R1
9.R
egis
try
ofO
pen
Acc
ess
Rep
osit
ory
Mat
eria
lA
rch
ivin
gP
olic
ies/
RO
AR
MA
P1
10.
Info
rmat
ion
Not
app
lica
ble
11.
Net
wor
ked
Dig
.L
ibr.
ofT
hes
isan
dD
isse
rtat
ion
112
.D
INI
Web
site
/Pol
icie
s1
13.
edoc
-Ser
ver
HU
/DIN
IP
aper
114
.B
erli
nD
ecla
rati
on1
15.
DF
GW
ebsi
te1
16.
DIN
IW
ork
shop
2005
117
.D
INI
Wor
ksh
op20
05/P
rog
ram
me
118
.S
chol
z,F
.an
dD
obra
tz,S
.(20
06),
“In
stit
uti
onal
Rep
osit
orie
san
dE
nh
ance
dan
dA
lter
nat
ive
Met
rics
ofP
ub
lica
tion
Imp
act
(...
)Dir
ecto
ryof
OA
Journ
als
119
.S
her
pa/
Rom
eo1
10.
Web
site
Pro
ject
for
OA
-Pol
icie
s1
21.
ww
w.
alas
tor.
di.o
ua.
gr/
DR
IVE
R1
Lin
kb
rok
en22
.w
ww
.G
ap-p
orta
l.de
1N
oac
cess
toth
isw
iki
23.
OA
ver
sion
ofg
apw
ork
s1
24.
Web
site
OA
Ger
man
y1
25.
Info
rmat
ion
Not
app
lica
ble
26.
Info
rmat
ion
Not
app
lica
ble
(con
tinued
)
Table AXII.
BL23,4
196
OA
NR
efer
ence
sP
aid
4In
tern
etR
esou
rce
Loa
nsh
arin
g@ U
BIn
tern
et-
bas
edC
omm
ents
27.
Pu
bli
shin
gp
latt
form
for
geo
scie
nce
128
.L
ink
toU
niv
ersi
tyof
Col
ogn
e1
29.
Lin
kto
Ges
is/i
nfo
rmat
ion
for
soci
alsc
ien
ce1
30.
Lin
kto
aw
ebsi
teon
his
tory
131
.L
ink
tow
ebsi
teon
pro
gra
mm
ing
/ww
w.
typ
o3.n
et1
32.
Info
rmat
ion
Not
app
lica
ble
33.
Info
rmat
ion
gu
idin
gto
DIN
IW
ebsi
te1
34.
Info
rmat
ion
gu
idin
gto
edoc
-Ser
ver
135
.A
sab
ove
bu
td
iffe
ren
tlo
cati
on1
36.
As
abov
eb
ut
dif
fere
nt
loca
tion
137
.In
form
atio
nN
otap
pli
cab
le38
.In
form
atio
nN
otap
pli
cab
le39
.L
ink
tow
ebsi
teon
stan
dar
ds
140
.M
ath
and
Ind
ust
rya
Por
tal
141
.L
ink
toU
niv
ersi
tyof
Got
tin
gen
142
.D
spac
e1
43.
Info
rmat
ion
Not
app
lica
ble
44.
Lin
kto
DN
B/I
nfo
rmat
ion
onD
DC
(Dew
eyD
ecim
alC
lass
ifica
tion
)1
45.
As
abov
eb
ut
inE
ng
lish
146
.In
form
atio
nN
otap
pli
cab
le47
.P
roP
rin
t/P
rin
ton
Dem
and
Ser
vic
e1
48.
Info
rmat
ion
Not
app
lica
ble
00
831
Alt
oget
her
39
Table AXII.
ROI in Germanlibraries
197
OA
N2
Ref
eren
ces
Pai
d4
Inte
rnet
Res
ourc
eL
oan
shar
ing
@ UB
Inte
rnet
-b
ased
Com
men
ts
1.oa
net
.cm
s.h
u-b
erli
n.d
e(&
5.)
1L
ink
bro
ken
2.L
iste
deu
tsch
erD
oku
men
ten
serv
er/D
INI
13.
Info
rmat
ion
Not
app
lica
ble
4.O
pen
DO
AR
16.
Info
rmat
ion
Not
app
lica
ble
7.R
epos
itor
ies
Su
pp
ort
Pro
ject
(JIS
CR
epos
itor
yN
et)
18.
Dew
yD
ezim
alK
lass
ifik
atio
nW
ebsi
te1
9.A
PS
Jou
rnal
s/P
hy
sics
and
Ast
ron
omy
Cla
ssifi
cati
onS
chem
ew(P
AC
S)
110
.M
ath
emat
ics
Su
bje
ctC
lass
ifica
tion
1(O
ldon
esav
aila
ble
inp
rin
tv
ialo
ansh
arin
g)
11.
Hen
zin
ger
,M
.(2
006)
,“F
ind
ing
Nea
rD
up
lica
teW
ebP
ages
:A
Lar
ge
Sca
leE
val
uat
ion
ofA
lgor
ith
ms”
112
.S
tam
ou,
S.et
al.
(200
6),
“Cla
ssif
yin
gW
ebD
ata
inD
irec
tory
Str
uct
ure
s”,C
ompu
ter
Sci
ence
113
.A
rXiv
.org
114
.C
hen
g,P
.C.e
tal.
(200
8),“
Dom
ain
-Sp
ecifi
cO
nto
log
yM
app
ing
by
Cor
pu
s-B
ased
Sem
anti
cS
imil
arit
y”,
Pro
ceed
ings
ofN
SF
CM
MI
Engi
nee
ring
Res
earc
hand
Innov
ati
onC
onfe
rence
115
.C
CS
(Th
eA
CM
Com
pu
tin
gC
lass
ifica
tion
Sy
stem
)1
16.
ZD
M(I
nte
rnat
ion
alR
evie
ws
onM
ath
emat
ical
Ed
uca
tion
)1
17.
ICD
(In
tern
atio
nal
Sta
tist
isch
eK
lass
ifik
atio
nd
erK
ran
kh
eite
nu
nd
ver
wan
dte
rG
esu
nd
hei
tsp
rob
lem
e)1
18.
OP
S(O
per
atio
nen
-u
nd
Pro
zed
ure
nsc
hlu
ssel
Inte
rnat
ion
ale
Kla
ssifi
kat
ion
end
erP
roze
du
ren
ind
.Med
izin
ein
sch
ließ
lich
...)
119
.O
bje
ctR
euse
and
Ex
chan
ge
120
.E
uro
pea
na
10
04
13A
ltog
eth
er17
Table AXIII.
BL23,4
198
Doc
up
edia
Ref
eren
ces
Pai
d4
Inte
rnet
Res
ourc
eL
oan
shar
ing
@U
BIn
tern
et-b
ased
Com
men
ts
1.P
ub
lik
atio
nss
trat
egie
nim
Wan
del
(200
5),
DF
G1
2.S
eeab
ove
13.
Lor
enz,
M.
(200
6),
“Wik
iped
ia.
Zu
mV
erh
altn
isv
on#
Str
uk
tur
un
dW
irk
un
gsm
ach
tei
nes
hei
mli
chen
Lei
tmed
ium
s”,
Wer
ksta
ttG
esch
ichte
,43
14.
Lin
k/W
ikim
atri
x1
5.O
’Rei
lly
,T
.(2
005)
,“W
hat
isW
eb2.
0?”
16.
Gra
ham
,P
.,W
eb2.
01
7.H
elle
r,L
.(20
06),
“Wis
sen
sch
aftl
ich
esP
ub
lizi
eren
mit
Wik
is...“
,O
pen
Sou
rce
Jahrb
uch
18.
Info
rmat
ion
Not
app
lica
ble
9.M
eyer
,B
.(2
006)
,“D
efen
sean
dIl
lust
rati
onof
Wik
iped
ia”
110
.R
osen
zwei
g,R
.(20
06),
“Can
His
tory
be
Op
enS
ourc
e?W
ikip
edia
and
the
Fu
ture
ofth
eP
ast”
,T
he
Journ
alof
Am
eric
an...
111
.L
ink
toW
ikip
edia
“Th
eori
efin
du
ng
”1
12.
Info
rmat
ion
Not
app
lica
ble
13.
Info
rmat
ion
Not
app
lica
ble
14.
Lin
kto
wik
iped
ia“w
ikiv
ersi
ty”
115
.L
ink
toS
chol
arp
edia
116
.L
ink
toH
eise
.de
“Cit
izen
diu
mso
llb
esse
reW
ikip
edia
wer
den
”(2
006)
117
.L
ink
toa
pro
ject
by
BM
BF
/WIK
ING
ER
118
.C
lio
On
lin
eS
urv
ey1
19.
Info
rmat
ion
Not
app
lica
ble
20.
Sam
ida,
S.
(200
6),
“Wis
sen
sch
afts
kom
mu
nik
atio
nim
Inte
rnet
.N
eue
Med
ien
ind
erA
rch
aolo
gie
”1
21.
Lin
kto
Jak
obN
iels
ons
Ale
rtb
ox(2
006)
122
.L
ink
toD
INI
123
.L
ink
toar
xiv
.org
124
.O
pen
Acc
ess
pol
itic
sat
the
Geo
rg-A
ug
ust
Un
iver
sity
Got
tin
gen
1
(con
tinued
)
Table AXIV.
ROI in Germanlibraries
199
Doc
up
edia
Ref
eren
ces
Pai
d4
Inte
rnet
Res
ourc
eL
oan
shar
ing
@U
BIn
tern
et-b
ased
Com
men
ts
25.
Lin
kto
edoc
/DiM
L1
26.
Lin
kto
edoc
/up
load
ing
anar
ticl
ew
ith
met
adat
a1
27.
Lin
kto
arx
iv.o
rg/H
owto
rep
lace
anar
ticl
e1
28.
Lin
kto
Cli
oO
nli
ne/
Gen
eral
Ter
ms
and
Con
dit
ion
sA
ct1
29.
Lin
kto
Wik
iped
ia/G
NU
Fre
eD
ocu
men
tati
onL
icen
ce1
30L
ink
tocr
eati
ve
com
mon
s1
31.
Lin
kto
Dig
ital
Pee
rP
ub
lish
ing
,N
RW
132
.L
ink
toC
lio
On
lin
e/G
uid
es1
33.
Lin
kto
Wik
iboo
ks
134
.L
ink
toed
oc/H
isto
risc
hes
For
um
135
.L
ink
toC
ase
Wik
i1
36.
Lin
kto
Wik
iMed
ia/S
hib
bol
eth
Au
then
tifi
cati
on1
Lin
kb
rok
en37
.L
ink
toW
ikip
edia
/Pag
eb
yP
age
Acc
ess
1L
ink
bro
ken
38.
Lin
kto
Wik
imed
ia/I
nd
exof
tru
nk
139
.L
ink
toO
DF
Ad
d-I
n1
40.
Lin
kto
Mic
roso
ftD
owlo
adC
entr
e1
41.
Lin
kto
O’R
eill
yx
ml.c
om1
42.
Lin
kto
htm
l2w
ikip
edia
143
.L
ink
toC
PA
N1
44.
Lin
kto
Wik
imed
ia/W
ord
mac
ros
1L
ink
bro
ken
45.
Lin
kto
AR
T-D
ok1
46.
Lin
kto
DIN
I/D
INI
Zer
tifi
kat
147
.S
chm
idt,
B.
(200
6)G
esch
afts
mod
elle
des
Op
enA
cces
s-P
ub
lizi
eren
s:W
elch
eP
ersp
ekti
ven
...
148
.N
eum
ann
,J.
(200
6),
“Au
fd
emW
egzu
ein
emO
pen
-Acc
ess-
Ges
chaf
tsm
odel
l”,O
pen
Sou
rce
Jahrb
uch
149
.L
ink
toC
lio
On
lin
e/B
ekan
nth
eit
un
dA
usb
auv
onH
sozu
Ku
lt1
50.
Info
rmat
ion
Not
app
lica
ble
13
338
Alt
oget
her
45
Table AXIV.
BL23,4
200
About the authorKathrin Grzeschik has recently graduated with a Master of Arts from the Berlin School forLibrary and Information Science and is now a member of research staff coordinating a project onlong-term preservation. Kathrin Grzeschik can be contacted at: [email protected]
Internet-based 284@ UB 82Loan sharing 40Paid for Internet source 67190 citations count for UB 190Total count 474
Table AXV.Summary
ROI in Germanlibraries
201
To purchase reprints of this article please e-mail: [email protected] visit our web site for further details: www.emeraldinsight.com/reprints