Facts and Frameworks: An Approach to Studying the Users of ...

American Archivist / Vol. 49 / Fall 1986 393

Facts and Frameworks: AnApproach to Studying the Usersof Archives

PAUL CONWAY

Abstract: The continuing reluctance of the archival profession to develop a betterunderstanding of users seems less a problem of will than a problem of method. Theframework presented here is a first attempt to structure a comprehensive program ofuser studies. Built on definitions of users, information needs, and use, the frameworkcombines the basic elements of information that should be recorded, analyzed, andshared among archivists with a scheme to gather this information. The author illustrateshow parts of the framework can be implemented as an ongoing program through theuse of a reference log and suggests applications of the framework at the personal,repository, and professional levels.

About the author: Paul Conway has been an archivist at the Gerald R. Ford Library since 1977.He has an M.A. in history and archives administration from the University of Michigan and cur-rently serves on SAA 's Task Force on Institutional Evaluation. This article is a revised version of apaper presented at the 49th annual meeting of the Society of American Archivists, 31 October1985, Austin, Texas. It was written as a product of his participation in the 1985 ResearchFellowship Program for Study of Modern Archives administered by the Bentley HistoricalLibrary, University of Michigan, and funded by the Andrew W. Mellon Foundation and theEarhart Foundation of Ann Arbor.

394 American Archivist / Fall 1986

Science is built up with facts as ahouse is with stones. But a collec-tion of facts is no more a sciencethan a heap of stones is a house.

Henri PoincareScience and Hypothesis

FRAMEWORKS ARE THE STRUCTURES thatorganize the "heap of stones" that is theworld around us.1 Varied in form andcontent, frameworks are simplificationsof reality—ways of reducing complexitiesto a set of meaningful, manageable ideas.Their great use is to summarize old factsand lead to new ones. Central to the bestframeworks is the possibility of actionderived from practical and useful rules.As such they are interim steps on the wayto a developed theory. Archivists can useanalytical frameworks to understandcomplex issues in systematic ways and toshare the knowledge gained in the pro-cess.2

The framework for studying the usersof archives presented here is a first at-tempt to structure a comprehensive,profession-wide program of user studies.Built on definitions of users, informationneeds, and use, the framework describesthe basic elements of information thatshould be recorded, analyzed, and sharedamong archivists to assess programs andservices. In addition, the framework il-lustrates a scheme to gather informationon groups of users over time that takesadvantage of accepted reference practices

already in place in many repositories. Theframework should be widely usefulbecause it is not rooted in specific institu-tional procedures. If tested and applied,the framework has the potential to helparchivists compare and assess the resultsof individual studies.

In recent years archivists have de-scribed why we need a more systematicapproach to understanding users. ElsieFreeman, Mary Jo Pugh, William Joyce,Bruce Dearstyne, and William Maherespecially have been in the forefront urg-ing the archival profession to develop agreater balance between archivalmaterials and those who use them.3 Theyargue that recognizing and responding tothe information needs of users is centralto the wider use of historical informationin contemporary problem solving, centralto the proper documentation of society,and central to the viability of a professionfaced with rapid technological change."Use of archival records is the ultimatepurpose of identification and administra-tion," declares the final report of theSociety of American Archivists' TaskForce on Goals and Priorities. Archivistsare beginning to consider high quality re-search on users an essential means towardthis goal.4

Archivists are less sure about how todesign useful user studies, especially whoand what should be studied, when andwhere user studies should be conducted,

1 Henri Poincare, The Foundations of Science: Science and Hypothesis, The Value of Science, Scienceand Method, trans. George Bruce Halsted (Lancaster, Pa.: Science Press, 1946), 127.

2Charles Nordmann, "Henri Poincare: His Scientific Work; His Philosophy," Annual Report of theBoard of Regents of The Smithsonian Institution (Washington, D.C.: U.S. Government Printing Office,1912), 741-63.

'Elsie T. Freeman, "In the Eye of the Beholder: Archives Administration from the Users Point of View,"American Archivist 47 (Spring 1984): 111-23; Mary Jo Pugh, "The Illusion of Omniscience: Subject Accessand the Reference Archivist," American Archivist 47 (Winter 1982): 33-40; William L. Joyce, "Archivesand Research Use," American Archivist 47 (Winter 1984): 124-33; Bruce W. Dearstyne, "The Impact ofResearch in Archives" (Paper presented at the annual meeting of the Society of American Archivists,Washington, D.C., September 1984); William J. Maher, "The Use of User Studies," Midwestern Archivist11, no. 1 (1986): 15-26.

'Planning for the Archival Profession: A Report of the SAA Task Force on Goals and Priorities(Chicago: Society of American Archivists, 1986), 22. See also Karen Benedict, "Invitation to a Bonfire,"American Archivist 47 (Winter 1984): 48; Maynard J. Brichford, Archives and Manuscripts: Appraisal andAccessioning (Chicago: Society of American Archivists, 1977): 9; Nancy Sahli, "National InformationSystems and Strategies for Research Use," Midwestern Archivist 9, no. 1 (1984): 10-12.

Users of Archives 395

and how to gather informationsystematically. Freeman, Pugh, andothers have suggested that archivistsmake better use of orientation and exitinterviews to query researchers.5 But ar-chivists have yet to develop a comprehen-sive approach that links the basic objec-tives of a user study program and a prac-tical way for gathering and recordingvalid, reliable information from users. Inshort, archivists' continuing reluctance todevelop a better understanding of usersseems not so much to be a problem ofwill as a problem of method.

Archivists are not alone in their con-cern that user studies be useful analyticaltools. For initial guidance on building acomprehensive framework, they mightturn to the perceptive literature describ-ing and criticizing the many studies oflibrary patrons. A thirty-year tradition ofresearch on national, regional, state, andlocal levels and in public, university, andspecial libraries has produced a backlogof over one thousand studies.6 Theirtopics run the gamut from in-house useof library materials and circulation pat-terns, to characteristics of users and non-users, to the assessment of programs andservices.7 In general they have tended todescribe programs in particular libraries,rather than take a multi-institutional ap-proach. When they have focused on usersin broader terms, library user studiesoften have described behavior of in-dividuals rather than of groups.8 Finally,library user studies have not shown howinformation gathered in the study process

can be applied to designing and assessingprograms for users beyond the clientelebeing studied.9 Individually, library userstudies are not very useful for archivists.

By considering them as a group, how-ever, several perceptive critics of libraryuser studies in the United States and theUnited Kingdom—including GeoffreyFord, John Brittain, Colin Mick, andothers—have identified patterns of find-ings. Some of the concepts underlyingthese patterns may be useful for ar-chivists because, removed from their par-ticular library setting and stripped oftheir specific library procedures, userstudies have identified some of the com-ponents of the process of informationtransfer. This fundamental form of com-munication is the point of departure forbuilding user oriented services in bothlibraries and archives.10 In an archives,information transfer occurs in many dif-ferent ways, but most typically when a re-searcher with a specific information needinteracts with archivists and finding aidsand in the process acquires archival infor-mation of use in meeting some part of theneed. For archivists, the three importantparts of this equation are users, informa-tion need, and use.

Users, in the most elementary sense,are people who seek information in ar-chival materials. They may be researcherswho visit repositories or who use itemsobtained from archives. Archivists areusers when they extract information froma body of files and organize it in a findingaid, reference letter, or exhibition. More

'Freeman, "Eye of the Beholder," 119; Pugh, "Illusion of Omniscience," 39; Carl M. Brauer, "Research-er Evaluation of Reference Services," American Archivist 43 (Winter 1980): 79.

'Susan Crawford, "Information Needs and Uses," Annual Review of Information Science and Technolo-gy 13 (1978): 61-82.

'Ronald Powell, "The Utilization of User Studies in the Development of Performance Measures" (Un-published paper, University of Michigan School of Library Science, 1984), 2-13.

•Colin Mick, Georg N. Lindsey, and Daniel Callahan, "Toward Usable User Studies," Journal of theAmerican Society for Information Science 31 (1980): 347.

'Geoffrey Ford, ed., User Studies: An Introductory Guide and Select Bibliography (Sheffield, U.K.:Centre for Research on User Studies, 1977), 70.

1 "Ford, User Studies, 7-17; Mick, "Toward Usable User Studies," 347-56; John M. Brittain, Informationand Its Users (Bath, U.K.: Bath University Press, 1970).


fundamentally, users are also people whomay never visit an archives but utilize ar-chival information indirectly, for exam-ple, as partners or clients in a law firm,students in a classroom, viewers of adocumentary film, or editors of a news-paper. Users of archives are, therefore,all beneficiaries of historical information.

l y j i j ,, \ By this definition, it is unlikely therey rrJ | are many non-users of archives. Critics of

tf"< [library user studies have described ways}^/ ** ""' to think about grouping users into mean-

ingful categories. Nearly all agree that re-searchers rarely act in isolation, butrather as members of networks with avariety of informal and formal ways ofsharing information. Library studieshave also shown that the functionalnature of the groups to which the userbelongs has a much greater impact thanindividual personal or professionalcharacteristics on how that personperceives a need for information and goesabout satisfying i t ." In other words, thefact that a lawyer visited an archives onbehalf of a client is more useful to knowthan either the specific name of the lawfirm or the lawyer's race, sex, or age.

\\ Q[) Archivists should regularly question re-

i I ^ . searchers who visit archives. The primary(\@ \ purpose of this questioning, however, is

to identify the most immediate groups ofbeneficiaries of archival information andbegin to understand the process of infor-mation transfer within and beyond thearchives.

An information need is simply a ques-tion for which archival information mayprovide all or part of the answer. Theconcept is far easier to define than to ex-plain. Geoffrey Ford has suggested thatthe purposes for which individuals seek

*/'

particular forms of information arelargely determined by their roles along awork-leisure continuum. Colin Mick hasfurther refined this concept to distinguishbetween applicational needs (e.g., aspecific document) and more abstractnutritional needs that increase generalknowledge or competence.12 An impor-tant goal of a broad user study programis to contribute to a better understandingof the factors determining informationneeds that can be influenced by archivalprograms and services. Archivists need toknow how individuals define particularinformation needs and why they seektheir answers in archives.

Use of archival materials is comprisedof two distinct activities. Use occurs in aphysical sense in reference rooms whenresearchers scan collections, series,folders, or individual items in search ofinformation relevant to their needs. Thisform of use is most frequently document-ed in annual or quarterly administrativereports. For example, X number of re-searchers made Y number of daily visitsand in doing so consulted Z number ofcollections, series, or items. Archivistsneed to evaluate systematically such usenot simply for basic reporting purposesbut also to assess the impact of physicaluse on archival materials and to evaluatealternatives to physical use.

A second kind of use is more difficultto explain but as important to document—usefulness, or the use made of archivalinformation to benefit individuals,groups, or society as a whole.'3 The im-pact of use beyond the repository is notdocumented through the ubiquitous listsof important research visitors and theirimportant projects, but through a careful

"William J. Paisley, "Information Needs and Uses," Annual Review of Information Science andTechnology 3 (1968): 1-31; Ford, User Studies, 18-37.

12Ford, User Studies, 38-44; Colin Mick, "Human Factors in Information Work," Proceedings of theAmerican Society for Information Science 17 (1980): 21-23.

nDearstyne, "The Impact of Research in Archives," 2.


OBJECTIVEST l

5'c3

Frame

work

for

CO

Q.< ;

1e Users

o

Arch

i

ID(A

.

CD~CjCo

3

3'

Iit ion

Value

o o

CD

TellO

thei

CO•o

valuable in.

o3a

tio.

Cu

3

CO"

CD3

CD

ootact ^

oc• o

•oth•er sourc

CDCO

• de

cisi on-makinq

• com

.

3unitynetw

ork

.CQ

3

cu

Iiatio

3

fun

d,

§Cu3Q.

|

3

|3"

o

a|

• c/fai7cin pa

ttern

s• fo

/a/

•Do(e/?if/a/ dems

3a

Metw

oi

•

1ose in term

CO

O

CD3

COhip

3 "

—1itendled U

se

03

co<9ignific

03

Int

o

• increa

asn

pas

Infori

3ation in S

c

Q

CD'• <

lifica

n

!-§•

COCD

Impac

t of Use

Role

o_

storica

Integrity

CQree toR

ui

CDCO

da tab

cuCOCDCO

• valueand use of

rrTilephone

•verbalsources

.

1e spen

tw

ithfile

Co

micro

forms

• techi

1and /nto

r

3c»

1

CDddress1

n sourcescolle

ctions

5

• valueand use of

CD•tfl

U3

3

awe

mg:

a)

Q.

COjrvic

1./CDCO

uency

0—h

CCOCD

Use

• linka

CQ

CO

Jth inform

CDtioi

nenm

tificEitior

•xnowledge

of

0Id-

—

CDnsity £

Q .

T l

CD

Aite

rnative to P

rsica

lF

orm

Co

5"depend

CD3OCD

3e spenttalki

Bu.

Quality

>nticip

ate

d S

ervi

0CD

t

1

wa

ds

aye

as

03 -3 '

CQ

•basici'applied

m

3 -0 recomm

ended

1 • levelsof service

•stag

eof definec1 pro

!

CDCQCO

s>arch1 • appro.aches to sea.

1

CO

"c5'0

it, scop

CD

•

uadx

a§'0

CD

.

Co

irch ordt

1 • styles

of research

• perc

; of use

S-'finitio

n

3 "

CDrms

0/

TJCDenearcher

lech

an

DC

S

ZCUtu

re 0

CO7T

TJrepara

tion

0

C/3CDCUrch S

ti

CU

CDCQ

CD'CO

CU

Q .

Ex pecS

atisfactio

n

1Iiindexes

tatio

ns an

Q .

Acce

COCO

CUid Non-

cEg

(allusers/aIw

ays)

cers at selec

i

inws)

(sai

ICD

118

tin

I

random sam

pli

•X-

!al groups)

Registratio

3

Orienta

tio

3

F0II1

0

c•a

Survey

m•aCDim

en

•-*to

CO

Stag

e

ro

CO

&CD

U

Stag

e 4

JSige 5

mHXOOO

OO


analysis of the dissemination of historicalinformation within a wide variety of writ-ten and oral contexts.^ By identifyingsystematically both the physical use of ar-chival materials and the impact of ar-chival information beyond a singlerepository, archivists can better evaluateand plan archival programs and moreclearly realize the value of the servicesthey provide.

Elements of the Framework

Based on these working definitions ofusers, information needs, and use, theframework presented in Figure 1 depictswhat archivists could learn from a com-prehensive program of user studies andhow they could build such a program. Asa whole the framework is an alternativeto the common view of reference serviceas one segment in a linear progression ofprocesses for handling archival materials.From this perspective appraisal, acquisi-tion, processing, description, reference,and outreach activities each flow oneafter the other and set the bounds foreach successive process. Archivalmanuals breaking the sequence into itscomponent parts reflect the conventionalapproach, as do archival managementdecisions that establish functionalspecializations.'4 At best, information onthe needs of users is factored in if andwhen it is available.

The framework's structure presupposesthat service to users is the foundation ofarchival programs, regardless of the ad-ministrative structure of an institutionand regardless of how archivists chooseto organize their work.15 It envisions areference program as both a direct servicefor researchers and a central evaluative

mechanism for the repository. Informa-tion gathered in user studies providesessential raw data to help administratorsevaluate reference programs, descriptivepractices, outreach activities designed toincrease use, processing priorities, and amyriad of other programs traditionallyconsidered beyond the domain of thereference room. Information from userscombined with the substantive knowledgethat archivists acquire from archivalmaterials are the two keys to understand-ing fully these programs.

The framework's structure also em-phasizes that archival program objectivescannot be evaluated solely by researcherswho visit. Past researchers, potentialusers, and even the broad extra-institutional community served by an ar-chives must sometimes be queried to gaina full understanding of such complexissues. While the framework identifiesthe basic elements of an ongoing userstudy program, it is not a script fordirecting the program itself or referenceservice activities in general. Instead itenhances the traditional, passivecustodial role of archivists by helpingthem become active gatherers and con-sumers of information about the servicesand programs they provide.

The core of the framework describesthree complex objectives summarized bythe words Quality, Integrity, and Value.Expressed as simple questions, Quality is"How good are the services?", Integrity is"How good is the protection of archivalinformation?", and Value is "What gooddo the services do?" Quality and Valuemay be considered more meaningfulsubstitutions for the information sciencejargon "effectiveness" and "benefit."16

14Kenneth W. Duckett, Modern Manuscripts: A Practical Manual for Their Management, Care and Use(Nashville: American Association for State and Local History, 1975). See also SAA's Basic Manual SeriesI, published in 1977.

"Frank G. Burke, "Archival Cooperation," American Archivist 46 (Summer 1983): 295."See R. H. Orr, "Measuring the Goodness of Library Services: A General Framework for Considering

Quantitative Measures," Journal of Documentation 29 (1973): 315-32.


Integrity is derived from the essentialresponsibility of archivists to preserve theintegrity of archival materials.

The choice of terms with a certain fuz-ziness of meaning is intentional forseveral reasons. First, there are nospecific archival terms equivalent to thethree objectives. The intuitive nature ofthe words indicates that they mean bothsomething simple and something com-plex. Indeed, the notion that an impor-tant problem may have layers of under-standing is fundamental to implementingthe framework. Each objective consistsof five gradations, and informationgathered in each stage clarifies the ques-tions to be asked in each successive stage.In addition, such non-technical wordshave meaning simply because all of theconcepts are not totally measurable. Theframework assumes that quantitativemeasures and qualitative assessmentstogether provide a complete descriptionof the issue. It recognizes that the subjec-tive judgments of archivists are validassessment tools if checked against objec-tive measures wherever possible.'7

In the framework, the Quality of ar-chival programs and services is assessedin terms of how well archivists under-stand the information needs of their usersand how well the programs and servicesare able to meet those needs they are in-tended to serve. At the most basic stage,Quality involves understanding how re-searchers define their task in terms of thesubject, format, and scope of informa-tion needed. At more complex levels,Quality involves understanding researchcapabilities and expectations for service(stage 2); research strategies andproblem-solving methods (stage 3); andthe nature and degree of satisfaction withthe research process (stage 4). Ultimately,

one objective of archival programs andservices is to recognize how users actuallyor potentially approach archival informa-tion, including why archives are not used,and to take every possible measure toenhance access to useful information(stage 5).

In the framework, the Integrity of ar-chival programs and services is assessedin terms of how well archivists balancetheir responsibilities to enhance use whilepreserving the information in archivalmaterials. At the most basic stage, In-tegrity involves identifying researchers asa first step toward limiting theft andabuse of materials.18 At more complexlevels, Integrity expands to include howresearchers become aware of available in-formation (stage 2); assessments of theimpact of physical use on the preserva-tion of materials (stage 3); andassessments of the value of alternatives tophysical use, including microforms andcomputerized data bases (stage 4).Ultimately another objective of archivalprograms and services is to understandand take action to assure that the infor-mation in archival materials is preservedand made available regardless of the for-mat in which it is located (stage 5).

In the framework, the Value of ar-chival services and programs is assessedin terms of the effects of use on in-dividuals, groups, and society as a whole.At the most basic stage, Value simply in-volves understanding the extent to whichindividual researchers are a part ofgroups with similar interests and ac-tivities. At more complex levels, Qualityexpands to include understanding the in-tended uses of archival information(stage 2); the relationship of archival in-formation to other sources of informa-tion (stage 3); and the impact of use

17Ibid., 317-18.l8Nancy Lankford, "Ethics and the Reference Archivist," Midwestern Archivist 7, no. 1 (1983): 7-13.


beyond the repository (stage 4). Ultimate-ly, another objective of archival pro-grams and services is to understand andincrease the role of historical informationin contemporary society.

Beyond identifying and describingthree primary objectives of archival pro-grams and services, the framework in-cludes a scheme for gathering assessmentinformation from users. The methodolo-gy combines two assumptions. First, ar-chivists can gather information within acontinuum of standard reference ser-vices, but at the most complex levels ofunderstanding, rigorous surveys and evencontrolled experiments are needed. Sec-ond, archivists can benefit by under-standing and using basic survey researchtechniques to limit the total populationstudied while increasing the reliabilityand validity of the findings.

Sue Holbert has described researcherregistration, orientation interviews, andexit interviews as standard elements ofthe reference continuum.19 Robert Tiss-ing and Carl M. Brauer, among others,have suggested how archivists can makespecial use of orientation and exit inter-views to gather information.20 Thevaluable opportunities of exit interviewsoften are lost through well-meaning pro-crastination on the part of reference ar-chivists or unceremonious departures byresearchers. As a partial solution, ar-chivists could banish the term "exit inter-view" from their vocabularies andsubstitute instead "follow-updiscussions." Less formal and more fluidthan interviews, follow-up conversationsmay begin anytime after a researcher hassettled into the research room.

In the framework, five increasinglymore sophisticated research methodsparallel the stages of the three objectives—Quality, Integrity, Value. Just as infor-mation from each stage of an objectiveserves as a base for the succeeding stage,the research methods at each stage helpdefine or narrow the sample populationof users studied at succeeding stages. Tosample users effectively archivists mustbuild a base of information about theuniverse of actual researchers.21 Stage 1,Registration, may be the best place to ac-complish this task. Registration formscan help ensure essential security andgather information on research problemsand the circumstances of use. Archivalrepositories should always require everyresearcher to complete a registrationform.

Stage 2, Orientation, and Stage 3,Follow Up, allow archivists to select aspecific time period during which tostudy researchers or to further limit thepopulation by selecting a specific groupof researchers for special emphasis. Inboth stages, the reference room serves asthe principal location of user studies.Stage 4, Survey, and Stage 5, Ex-periments, recognize that the most com-plex aspects of a research problem are notnecessarily best understood throughroutine repository procedures. Instead,sophisticated surveys of randomly sam-pled populations and controlled ex-periments with groups selected by a varie-ty of criteria may be more appropriate.The last two methodological stages neednot be confined to researchers who ac-tually visit archives, but may tap abroader base of users. At all stages of the

1 'Sue E. Holbert, Archives and Manuscripts: Reference and Access (Chicago: Society of American Ar-chivists, 1977), 12-13.

20Robert W. Tissing, "The Orientation Interview in Archival Research," American Archivist 47 (Spring1984): 173-78; Brauer, "Researcher Evaluation of Reference Services," 79; Freeman, "Eye of theBeholder," 119.

2 'The best introduction to survey research is Charles Backstrom and Gerald Hursh-Cesar, SurveyResearch, 2nd ed. (New York: Wiley and Sons, 1981). Also useful is Maurice B. Line, Library Surveys(London: C. Bingley, 1982).


framework, archivists should choose thepopulations and method most ap-propriate to the problem being studied.

The framework is an integrated unitand presents a comprehensive approachto user studies. Reading an objectivefrom left to right gives a sense of howeach of the three objectives becomesmore complexly defined in succeedingstages. Reading one of the stages fromtop to bottom indicates the most impor-tant elements of the three objectives thatshould be recorded at that stage. Theelements in each cell of the frameworkare those aspects of an ongoing userstudy program that should be sharedamong archivists and repositories todevelop a profession-wide understandingof Quality, Integrity, and Value.

Implementing the Framework

Translating the framework into anongoing research program at a repositorydoes not necessarily involve massivequestionnaires and sophisticated com-puter analysis. The "Reference Log" inFigure 2 is one example of how stages 1,2, and 3 of the framework could be madeoperational for daily use. It is called a logbecause it tracks and records researchersat different points along the referenceservice continuum. By linking activitiesand assessments across time, archivistscan better understand the process of seek-ing and finding useful information. Assuch, the log would enhance, but notreplace a sign-in register recording dailyvisits to the research room. The referencelog in the example combines the two mostbasic survey research approaches—obser-vation and questioning—but distributesthe task of completion about equally be-tween reference archivists and the re-searchers.

The sample reference log is designed asa two-sided form with four distinct parts.Except for the section labeled "SearchReport," the questions on the log havebeen field-tested for accuracy andusefulness.22 Nevertheless, the specificwording of the questions and responsecategories may be tailored to the patronsof specific repositories. Given the log'slevel of inquiry and analysis, the underly-ing elements of a question are more im-portant than the specific wording of thetext or response categories.

Information requested within theblocked area of the log is based largely onthe elements in stage 1 of the framework.As a registration form, this section asksresearchers to identify themselves, waivethe confidentiality of their research ac-tivity, and agree to basic procedures. Allthese elements are standard features ofmany registration forms.23 The most im-portant parts of this section are the open-ended question on the nature of theresearch project and the multiple-choicequestion, "What work brought you to thearchives?" This form presents the primeopportunity for a researcher and the ar-chivist to record an understanding of thesubject, scope, time frame, and functionof the research question. In combinationwith information on institutional affilia-tion, these questions form the basis forunderstanding a researcher's group iden-tity and approach to the archival record.

The log's second section, "Orienta-tion," is a brief questionnaire for re-searchers to complete during or im-mediately following the orientation inter-view. The questions tap the elementsfrom stage 2 of the framework. Includedare inquiries on the intended use of infor-mation located, how a researcher foundout about holdings or services, the nature

"Paul Conway, "Research in Presidential Libraries: A User Study," Midwestern Archivist 11, no. 1(1986): 35-36. Ina Berzins, et. al., The Methodologies Report on Research and Public Service ComponentProgram Evaluation Study, Volume III, report prepared for the Public Archives of Canada by Currie,Coopers & Lybrand, Ottawa, Canada, January 1985.

"The Archival Forms Manual (Chicago: Society of American Archivists, 1982).


Figure 2

Reference LogInformation in this box is required to use the research room facilities. The principal purpose of

this form is to identify and record individuals who use materials at the archives, to help us Ides-

more detailed studies of reeearch use.

APPLICANT'S NAME (Last, First. Middle Initial)

PERMANENT ADDRESS (Street. City. State, ZIP)

DESCRIPTION OF RESEARCH PROJECT (Include subject,names, type of material needed)

MAY WE ADVISE OTHER INDIVIDUALS OF THE SUBJECTOF YOUR RESEARCH? I—1 r—1

L J YES 1 1 NO

I HAVE READ "REGULATIONS FOR THE PUBLIC USE OFRECORDS" AND I WILL COMPLY WITH THE RULES.Applicant's Signature Date

PERMANENT PHONE NO.

INSTITUTION

dates, Important

OCCUPATION (please be aespecific as possible)

WHAT WORE BROUGHT YOUTO THE ARCHIVES?

DDD

D

DDDD

MAY WE ADVISE OTHER INDIVIDUALS WHICHITEMS WERE SERVED TO YOU? 1 1 1 1

[_J YES |_) NO

MAY WE CONTACT YOU BY MAIL OR TELEPHONE AS PARTOF A FUTURE USER STUDY? •——.

I YES 1 1 NO

Tour answers to questions In this section of the form will help the referencer w i A n + A + i archivist orient you to using the archives' holdings. Together with answersU r i c f l X a C l O n from other researchers, the information you provide will enable archivists to

assess the overall use of the archives.

What is the purpose of your current research projectthat involves using the archives' holdings orservices (Circle all that apply)

1. Academic requirements

2. Genealogy

3. Publication (book, article)

4. Background information for newspaper, magazinearticle, advertising

5. Exhibition

6. Film, radio, television program

7. Government research

8. Professional research (for Individual, group.

9. Personal interest/hobby

10. Other

Some researchers prefer to rely on their backgroundpreparation or the finding aid system in the researchroom. Others feel most comfortable if referencearchivists guide their searches of the holdings.

Please mark the scale below to show your personalpreference for doing archival research.

1 U |3 \* |Srely on archivist and rely onfinding aids finding aids archivist

Before your first visit on this project, did youwrite or telephone to get Information onholdings or services?

Q YES Q NO Q DON'T DOW

Excluding writing or telephoning the archivesdirectly, which of the following sources did youmost rely on to Identify the holdings or servicesoT"use In your research. (Circle the beet choice)

1. References, citations In published works

2. Published guides to archives, primarysources, bibliographies

3. Teacher, professor, colleagues

4. Archivist/librarian at other institutions

5. Information from historical, professionalor genealogical organisations

6. Television, radio, newspaper

7. Presentation by archives staff

S. Visit to museum exhibition

9. General knowledge, assumptions

10. Other

If you have done archival research in the last fiveyears, please write the urns of the archives inwhich von save most recently worked.


Figure 2 (continued)

C p - r - u The reference archivist should not* the first tss collection consulted by the researcher,J c a l ( * " the source of recosawndatloo, and whether the researcher located information of use la theReDOft research project. Data for this section Is obtained from pacing slips, photocopy request

^ forms, observation, and If necessary, by questioning researcher.

RECOMfEHDATION COLLECTION NAHE

aDaaaDD••

aa•a•aa••

aDaa•aaaD

D a DARCHIVIST USER FINDING AID

SEARCH

aD•a•a•Daa

POSITIVE

RESULTS

DDDaaaaaa•

NEGATIVE

Follow Up The Information you provide la this section will assist archivists to understandhow archival research Is carried out, and how archival information nay be used.

Please provide an approximate breakdown of the totaltine you spent at the archives during this researchproject across each of the activities listed below.(Hake sure the total equals 100%)

Orienting yourself to the archives'services and facilities _ %

Searching; through finding aids andcollections to locate documents

Actually readIng/viewing/studyingdocuments

Discussing research project witharchivists or other researchers

What portion of your research project will be basedon archival materials located at tbls archives orother archives?

1. I hope to use primarily archival sourcesof information.

2. I bops to use archival sources and othersources about equally.

3. I hope to use other sources of informationprimarily.

4. I don't know yet.

TOTAL 100%

If you expect to share tbe information you find atthe archives, please describe below In what waysthe information will be used. Please use tblsopportunity to name the title of a proposed publi-cation, describe the group that may benefit fromyour archival research, or describe tbe results ofyour research ID more detail.

On the line below, please write tbe name of thecollection in which you located tbe most usefulinformation.

• | no useful1 information

1 — 1 located


of the service anticipated, and the ex-istence of prior archival research ex-perience. If a sampling strategy isnecessary to limit the population studied,this section might only be completed by aselected group of researchers—for exam-ple, undergraduate students—or during aselected time period—for example, thefirst week of every month. The informa-tion in this section could also help sub-divide the full researcher population forfurther, more detailed questioning.

The reverse side of the log contains thethird and fourth sections, labeled "SearchReport" and "Follow Up." Together thesesections apply the elements from stage 3of the framework. The "Search Report,"an untested proposal at this point, isdesigned as a way to learn about the com-plex problem of research strategies andthe process of locating useful informa-tion. Like the "Orientation" section, it isnot necessarily intended for all research-ers at all times, but for selected groups ofresearchers sampled during specificpoints in a year. Alternatively, this sec-tion may serve simply to illustrate thevaried ways collections are used bygroups of researchers.

Reference archivists administer thesearch report by observing researcherbehavior, examining evidence such as callslips and photocopy requests, and notingthe names of the first ten collections con-sulted. Archivists would then check a boxfor each collection noted to indicatewhether the reference archivist, research-er, or in-house finding aids were the prin-cipal source of the recommendation thatthe collection may contain useful infor-mation. Finally, the observer-archivistwould indicate whether the researcherlocated useful information. While thislast information may be obtainedthrough observation, some direct ques-tioning may be necessary.

The fourth section of the reference log,"Follow Up," draws on additional

elements from stage 3. Included are ques-tions on how a researcher spent time, theresearcher's expected use of archival andnon-archival sources of information, andthe most significant collections used. Afinal open-ended question enables theresearcher to comment in more detail onthe results of work or how the obtainedinformation will be used. As with the"Orientation" section, the follow-upquestions in the log represent only thevery basic pieces of information thatshould be obtained and recorded fromthe groups of researchers chosen forstudy. The queries should be considered apoint of departure for further discussionsor a way of clarifying issues that could beaddressed more completely in stage 4 and5 projects.

A reference log is not the only way toimplement the research strategy ex-plained in the framework's first threestages. Archivists could design separatesurvey instruments to collect informationfor each stage. A microcomputer at thereference desk, equipped with data basemanagement software, could substitutequite well for a whole range of surveyquestionnaires. It bears emphasizing thatthe fundamental goal in implementingthe first three stages of the framework isto record information gathered so that itcan be used to link groups of researcherswith their evaluations of services and pro-grams.

User surveys and special studies im-plementing stages 4 and 5 should be builton information gathered in the earlierstages. They are opportunities to movebeyond researchers in reference rooms, totap the behavior and attitudes of pastusers, potential users, and the communityof beneficiaries. Surveys and specialstudies are also more amenable to multi-institutional approaches. Examples ofresearch topics of this nature might in-clude factors that contribute to usersatisfaction, the impacts of frustration


and limited access on scholarly research,citation patterns in a given subject area,and the value and usefulness of non-paper records. As archival research ques-tions become more complexly defined, sotoo should research methods becomemore rigorous and sophisticated.

Applying the Framework

Putting to practical use informationgathered in a user study program beginswith an analysis of the data. This doesnot require computing equipment. Areference log will yield a wealth of infor-mation from hand tabulation. Simplyshuffling logs into piles with specificquestions in mind can be enlightening.Take for instance the question, "Thisyear, what portion of their time did gene-alogists spend in the research roombecoming familiar with the repository'sholdings and services?" Dividing the logscompleted by genealogists into threegroups according to the range ofresponses to the appropriate follow-upquery will produce a simple answer. Acloser inspection of other parts of thecompleted logs in each group will help ar-chivists understand why some genealo-gists begin research more quickly thanothers. At this level of analysis, archivistsequipped with microcomputers merelywill find the data analysis more efficientand convenient than hand tabulation.

Both archivist and researcher shouldfind an ongoing user study programfounded on a systematic framework avaluable learning process. Structuredcontacts actively demonstrate to re-searchers the concern archivists have forsuccessful research efforts without add-ing substantially to the researchers'burdens. If widely implemented, re-searchers may even come to expect op-portunities for orientation questionnairesand follow-up discussions. For thereference archivist, structured contacts

transform a role that sometimes begins toresemble that of a traffic cop into a cen-tral evaluative mechanism for the entirerepository. Routinely recording useful in-formation on researchers is much lessburdensome than recording routine pro-duction statistics—especially if the infor-mation gathered is an integral part of thedesign and evaluation of all archival pro-grams.

Integrating the results of user studiesinto the administrative structure of arepository begins with more creativereporting to resource allocators, senioradministrators, and user communities.Head counts and daily researcher visitscan be augmented by reports on the widevariety of questions addressed througharchival research. Reports based on vividdata from users of archival services maybe far more powerful tools than thosebased on dry statistics. Archives adminis-tration from the users' point of view re-quires that archival programs be defend-ed from the users' point of view.

Beyond its value in reporting, an on-going user study program can have a cen-tral role in the design and modification ofarchival programs only if archivists trustthe guidance users provide. Data col-lected from users in an ongoing program,even at a very basic level, may be substan-tially more reliable, and hence moreuseful, than even sophisticated one-timesurveys because of the program's capacityto reflect change. Evaluating the successof programs for users, such as anautomated catalog or an outreach pro-gram designed to increase physical use, isfar easier if user evaluations providebenchmarks over time. In short, withoutdirect and continuous user evaluations,archivists can only suppose that their in-formation needs are being met on aregular basis. For public institutions witha mandate to serve the research public,suppositions can only be stretched so farat budget time.


A general framework for understand-ing users can serve as an agenda for talk-ing about the archival profession's pro-gress toward the goal of user responsiveprograms and services. But at some pointthe talk must take a back seat tocooperative study projects. Perhaps theclassroom is an appropriate place tobegin building a research tradition. ElsieFreeman has suggested that archivaleducation should include training insurvey research techniques.24 Archivaleducators might consider transformingthe practicum portion of a two-course se-quence into a laboratory for teams ofstudents to design, carry out, andevaluate archival research projects, in-cluding user studies.

Knowledge from user studies will havevalue beyond single repositories only ifwidely shared. Mechanisms for a coop-erative effort need to be developed andsupported. At the very least, an archivalinformation clearinghouse, such as theone proposed by NAGARA, should beprepared to actively gather, evaluate, anddisseminate the results of user studies.25

A proactive clearinghouse can serve thesame broad functions for archives thatthe perceptive critics of user studiesserved for libraries—to discover patternsin isolated studies, encourage further re-search, and develop strategies for in-tegrating research findings into standardsof practice.

Mathematician and philosopher ofscience Henri Poincare was not the firstscientist to explain in clear terms the fun-damental need for structures to orderrandom observations. But his pragmaticapproach to scientific investigation seemsparticularly appropriate to a professiontrying to find larger meaning in the

details of history. Poincare saw no di-chotomy between theory and practicebecause any theory worth the name couldonly be judged in practical terms. Indeed,Poincare saw in the problems the fledg-ling library profession was confronting indeveloping descriptive practices a propermetaphor for the problems of math-ematical physics. Theories organizethought as a means of organizing action.

Archivists too are universalists ofsorts. They are proud of their broadunderstanding of the process of historicalresearch and their attention to the detailsof past experience. Given the profession'spractical nature, archivists may neverdevelop a scientific theory that meetsFrank Burke's definition of universal andimmutable laws.26 But to avoid buildinga profession on a set of immutableplatitudes, archivists' practice can andshould be based on solid conceptualstructures that transcend the limits oflocal precedent. Analytical frameworksencourage systematic problem-solving,promote cooperation, and allow ar-chivists to collect the informationnecessary to develop standards of prac-tice benefiting the profession as a whole.

The framework for studying users andthe reference log are offered as basictools in such a process. Archivists shouldbegin with the basics, coordinate researchdesigns, collect data, share results widely,and revise their approaches based onthese results. The specifics of the pro-posed system need to be tested and re-fined. Archivists can acquire the skillsand insight necessary to design an ongo-ing evaluation program that includesusers by studying library surveys andmastering sound survey researchmethods. But ultimately archivists must

"Freeman, "Eye of the Beholder," 122."Planning for the Archival Profession, 31."Frank G. Burke, "The Future Course of Archival Theory in the United States," American Archivist 44

(Winter 1981): 40.


design their own methods and definetheir own goals, beginning with a com-mitment to develop a truly user-responsive archival profession. Makingthe reference room rather than the

loading dock the hub of archival activityrequires facts about users—recordedfacts, shared facts, but most of all factsorganized for clear objectives.

;') I

for people who knowall about Records StoragePaige Boxes are for professionals,experienced people who have learnedall about the equipment available forhandling, transporting, and storage ofrecords, microfilm, data processingmaterial, and computer printouts.Those people know that Paige filesare durable equipment at lowestpossible cost. Available Acid Free.Prices at wholesale level. No SalesPeople. No Distributors. No Stores.Write for Brochure, Prices, CaseHistories.

THE PAIGE COMPANY432 Park Avenue SouthNew York, N.Y. 10016 n OR 9-6626

Facts and Frameworks: An Approach to Studying the Users of ...

Documents

Transcript of Facts and Frameworks: An Approach to Studying the Users of ...