SOCIAL SCIENCE RESEARCH COUNCIL

VOLUME 32 • NUMBERS 3/4 • DECEMBER 1978 605 THIRD AVENUE • NEW YORK, N.Y. 10016

Herbert A. Simon Awarded 1978 Nobel Memorial Prize in Economics: Council Board Member,

1958-71 HERBERT A. SIMON, who was awarded the 1978

Alfred Nobel Memorial Prize in Economic Science in Oslo on December 10, has had a long association with the Council, and the announcement on October 16 of his selection was particularly gratifying to Council members. A political scientist who serves as Richard King Mellon university professor of computer sci­ence and psychology at Carnegie-Mellon University, Mr. Simon has exemplified in his distinguished career the interdisciplinary purposes and program of the Council.

In making the award, the Royal Swedish Academy of Sciences cited Mr. Simon's "pioneering research into the decision-making process within economic organizations," adding that "modern business eco­nomics and administrative research are largely based on Simon's ideas."

A member-at-large of the Council's board of di­rectors from 1958 until 1971, Mr. Simon was board chairman from 1961-65. He was a member of the Committee on Problems and Policy from 1962 through 1968; here his multidisciplinary interests and his rigorous standards for research made him an in­valuable member. He was a member of the Commit­tee on Business Enterprise Research from 1953 to 1957 and of the Committee on the Simulation of Cognitive Processes from 1957-62; he was chairman from 1959-62. In the summer of 1958, Mr. Simon and Allen Newell codirected a now-historic four-week

Contents of this issue-see page 4 J

41

summer trammg institute in computer simulation techniques. Held at the offices of the Rand Corpora­tion, with funds provided by the Ford Foundation, this workshop created a nucleus of trained faculty members on some dozen campuses. Much of the sub­sequent research and publication in this field can be traced back to initial training received at this institute.

The Council has benefited enormously from Her­bert Simon's wise counsel and his interest both in the simulation of decision-making processes and in the application of this knowledge to administrative and business organizations. It is a pleasure to acknowl­edge this debt on the occasion of his receiving the 1978 Nobel Prize.

Indian Law as an Indigenous Conceptual System When the Joint Committee on South Asia was reconsti­

tuted in 1976, it took as its major mandate the analysis of South Asian conceptual systems with a particular concern for their value in developing culturally sensitive analytical tools. The underlying notion was that many of the concepts of Western social science fit badly in non-Western societies, and efforts should be made to examine them in their own terms.

In the September 1977 Items, Stanley J. Heginbotham, Congressional Research Service, who serves as chairman of the Joint Committee on South Asia, noted that the "commit­tee'sfocus on South Asian perceptions of the world reflects a concern that social scientists, in formulating and utilizing analytic categories, logical linkages, and models, keep in mind that the people of the region often think in very differ­ent categories, use very different logical linkages, and as­sume distinctive models of human behavior."

The committee is seeking to examine a number of different indigenous systems: notions of political authority, folklore, concepts of the person, the concept of karma, responses to. risk and uncertainty, and the legal system. The following reflections of Mr. Galanter on Indian law give an indication of the kinds of issues being addressed.

WHEN I first became interested in Indian law more than 20 years ago, I encountered some quizzical reactions on the ground that since most of Indian law is palpably Western in origin, studying it could not reveal much about India. Nearly ten years ago, I wrote a paper expressing my feeling of wonder that independent India had not seen any serious attempts to dislodge the legal system imposed on it by the British (Galanter 1972). Both intellectually and in­stitutionally, there seemed an acceptance of the mid-20th century system as something fully Indian. There is nothing, for example, like the conflicted but very genuine desire to get rid of the English language as a medium of public life. This 1972 paper tries to ex­plain this lack of interest in dismantling the legal system; the findings boil down to a widespread feel­ing among professionals, the urban elite, and some villagers that "this is our system." 42

by Marc Galanter*

This situation points to the question of what we mean by an indigenous conceptual system. Is it one that the indigenes feel is theirs? Or do we have to look to some elite among the indigenes? If so, which elite do we look to? That is very much the question here, because in India there has been a displacement of an older elite, who were the carriers of an earlier indig­enous tradition. This does not imply that the earlier legal complex was an exact counterpart of the law complex that developed during British rule, although each combined authoritative learning, governmental force, and local dispute institutions. 1 The strength of the modern law complex, I have argued, is that it introduced a new and powerful set of ties among these elements (Galanter 1968).

The older legal tradition fit with other components of social life in ways recognizable to elites and to lay people alike-and to observers from afar. It elabo­rated such familiar notions as dharma and it was em­bodied in persons of religious and political eminence. What replaced it was an institutional-intellectual complex that does not fit neatly with the other institu­tions. In the new dispensation, there are discon­tinuities. But at the same time, law as an institution is more differentiated from everything else, so that its inconsistency with the internal content of other in­stitutions (e.g., family, temples, castes) doesn't matter as much (cf. Mayhew 1971).

Perhaps India has moved from a situation in which all the major intellectual-institutional complexes in the civilization were more or less compatible, based

* Marc Galanter. professor of law and South Asian studies at the University of Wisconsin. is a member of the Committee on Law and Social Science and the Joint Committee on South Asia of the Council and the American Council of Learned Societies. This paper was prepared for the October 1978 meeting of the Joint Committee on South Asia.

I Rocher (1972) suggests that the British made a fundamental error in equating dharmasastra (Hindu systematic learning about duty) and its expositors with law and lawyers.

VOLUME 32, NUMBERS 3/4

(in theory at least) on a common fund of ideas and worked out by a common fund of techniques. Now the institutional-intellectual complexes are bigger, more internally differentiated, more differentiated from each other, and more independent of one an­other. Thus, a complex like "law" can exist with its own internal socialization of key actors, its own tech­nical vocabulary, its own styles of thought, its own prestige hierarchy, its own system of social support, and its own system of concepts and meanings. It forms a large and partly autonomous subculture of its own. Is this the case with other institutional­intellectual complexes as well (e.g., government, education, art, journalism)?

When a society undergoes this kind of transforma­tion, it seems to me that "indigenous" takes on a new meaning. In the old dispensation, one could spot a foreign element because of its lack of fit with others. But in the new world, institutions don't flow into and reflect each other in the same way. It is a world of discontinuities and abrupt shifts between different parts of one's life experience. The indigenes are not always recognizable or accessible to one another. There may be great barriers between one indigenous phenomenon and another.

The law may loom as intrusive, alien, and external to many within Indian society, while others may find it comfortable, known, wholly familiar, and expres­sive of their values and outlook. Clearly, "indigenous" cannot mean just that which is not of foreign origin. How far back do you go? There have been many grafts on the tree. The Indians have taken British law and have reworked it, imposed unique stylistic fea­tures on it (Galanter 1967, 1968). The affinities with British or American law (there was a second wave of borrowing of American constitutionalism) are strik­ing; the kinship is apparent, yet the Indian style is quite distinctive.

As George Gadbois (1977) has pointed out, there must have been some special predilection among In­dians underlying this extraordinary receptiveness. This eager and unfaltering embrace of Western law simply did not take place in Southeast Asia or in Japan (and I think this is true for Africa as well).

The attachment to Anglo-Indian law (and its American-style constitutional overlay) runs deep. Two major new pieces of evidence suggest the tenac­ity of these forms in late 20th century India. First, there is the resistance to alterations in these institu­tional forms manifested in connection with the 1975-77 Emergency regime. Second, there is the dis­integration of post-Independence efforts to revive nyaya panchayats (village tribunals). DECEMBER 1978

During the 1975-77 Emergency, there was a deci­sive break with the liberal constitutionalist vision of the legal order that was shared by the Bar, officials, the political elite, and a large part of the populace since before Independence-an order in which inde­pendent courts vindicated private rights and curtailed official arbitrariness. The major contours of the Emergency regime are familiar by now: the elimina­tion of civil liberties, particularly by control of the press and by a reduction in the autonomy of or­ganized groups such as labor unions and the Bar; the elevation of the executive to decisive supremacy over Parliament; and especially a reduction of the power of the .iudiciary to decide public issues.

After a few gestures in the direction of repudiating the constitutional design, the government of Mrs. Gandhi pursued a policy of removing the courts from politically sensitive areas, making them more subser­viant politically but retaining, thus truncated, the rule of law. The new constitutional amendments elimi­nated the judiciary's power to pass on the propriety of constitutional amendments, severely curtailed the power of higher courts to adjudge statutes uncon­stitutional, and eliminated the jurisdiction of the or­dinary courts over such politically sensitive areas as government employment, revenue, land reforms, labor disputes, and elections. Roughly, all of the polit­ically volatile cases were withdrawn from the regular courts and put into tribunals that would, presumably, have been more politically responsive. The super­visory power of the regular courts over these tripu­nals was curtailed or eliminated. But with these signifi­cant amputations-and subject to some attempts to influence the judges by arbitrary transfers, etc.-the regular courts were left to function as before. The pattern that emerged was reminiscent of what the

CONTENTS OF THIS ISSUE

41 Herbert A. Simon Awarded 1978 Nobel Memorial Prize in Economics

42 Indian Law as an Indigenous Conceptual System­Marc Galanter

47 G~vernment Audits of Social Experiments-Ronald P. Abeles

51 Personnel -New directors and officers -Staff appointments

52 Recent Council Publications

43

Spanish sociologist Jose Toharia (1975, pages 494-5) suggests is the classical authoritarian response to law. Referring to the final years of the Franco regime, he wrote:

The situation of the ordinary courts under the present Spanish regime [is characterized] ... on the one hand by the consider­able degree of independence allowed to the judges (with a consequent lack of political indoctrination and the exercise among them of a certain ideological diversity) and on the other by the sharply curtailed and relatively unimportant sphere of action afforded them.

The administration of justice is thus controlled by avoiding the political mobilization of individuals and institutions and by "the systematic reduction of the area of competence of ordinary tribunals."

In spite of these drastic incur~ions into the legal order, there were during the Emergency remarkable continuities. Most prominently, there was an adher­ence to superficial legal proprieties and an unwilling­ness to depart from the major institutional forms of Anglo-Indian law. Although there were a few official mutterings about people's courts, these did not amount to more than a scare. There seems to have been no serious consideration of replacing or trans­forming the familiar brand of Anglo-Indian courts either with "people's courts" or with indigenous pan­chayats. Indeed, the government undertook new commitments to support a massive legal aid program, presumably to increase access to and use of the legal system.2 If there were outcroppings of a genuine de­sire to make legal institutions responsive to popular needs, there was little sense that these institutions would require alteration or transformation in order to accomplish this.

Although both technocratic and populist themes were invoked during the Emergency to constrain the autonomy oflegal institutions, the new legal measures were in very much the same style that had previously animated courts, lawyers, and legislators. The Emergency measures invo~ved no attempt to reshape legal forms, but only sought to truncate and contain them. Law remained formalist rather than in­strumentalist, a matter of individual entitlements con­ferred from the top rather than one of facilitating group mobilization, a matter for professionals rather than for lay participants.

2 While curtailing judicial power, the infamous 42nd Amend­ment to the Indian Constitution added a new Directive Principle that the State shall "provide free legal aid . . . to ensure that opportunities for securing justice are not denied to any citizen." The justice to be secured is, by implication, that dispensed by the then current variety of laws, COUTts, and lawyers.

44

As soon as the Emergency ended there was an immediate concern to restore legal institutions to their former condition. Is this restoration only a tem­porary reversal of a long run trend toward the extru­sion of a foreign element? In that light, one might see Mrs. Gandhi as a precursor of a movement to develop truly Indian political and legal forms. But what is remarkable is that even during the Emergency there was no significant challenge to the basic form and style of legal institutions. There was griping from the government side about the courts interfering with what were thought to be issues that should be left to Parliament and the Executive. But the basic shapes and style of courts as institutions and of the legal learning that they cultivate and dispense was never questioned. Both Mrs. Gandhi and her opponents shared a basic acceptance of India's legal institutions.

The second body of data is that concerning the disintegration of the nyaya panchayats (village tribu­nals) promoted by the government after Indepen­dence. In a recent assessment of this program (in press), Upendra Baxi and I found that the nyaya pan­chayats established in the 1950s and early 1960s are for the most part moribund. Where these tribunals have not been abolished by state governments dis­mayed at their workings, they have low case loads and enjoy little public or official regard. Although lack of government support undoubtedly made an impor­tant contribution to their demise, it is clear that they never attracted significant support from the villagers in whose name they were established. It is not clear whether they withered away because they lacked the qualities of the traditional indigenous tribunals or because they displayed them all too well. Most likely, it was because they represented an unappetizing combination of the formality of official law with the political malleability of village tribunals. Nevertheless, there now appears to be a new wave of interest in these institutions.

Recent proposals to promote nyaya panchayats tend to visualize them as agents for the efficacious delivery of official justice rather than as means to revive indig­enous justice. Thus, in a report (Government of India 1973) that views itself as a radical critique of Indian legal arrangements,Justice Krishna Iyer speaks glow­ingly of nyaya panchayats, citing Lenin in their favor, and commends them as a promising counterpart to "the system of justices of peace in the United King­dom and People's Courts in the Soviet Union .... " These examples he reads as establishing that "laymen may well be entrusted with dispensing legal justice provided certain safeguards are written into the

VOLUME 32, NUMBERS 3/4

scheme."3 It is clear that the "legal justice" to be dis­pensed is the law of the land and not that of the villagers or their spiritual advisors. Nyaya panchayats are commended as inexpensive, accessible, expedi­tious, and suitable to preside over conciliatory hear­ings. The Report recommends that they be staffed by superannuated judges and retired advocates. Pan­chayat justice is seen as part of a larger scheme of legal aid and as providing public access to the courts.

Charged with proposing concrete measures to se­cure access to justice for the poor, the Bhagwati Re­port (Government of India 1978) elaborates this dis­cussion of panchayats. After the obligatory nod to Mohandas Gandhi, the Report observes (in chapter 6) that

... there can be little doubt that our legal and judicial system is not adequate to meet the needs of the new society which is emerging in our country. It is not effective to provide a solution to the new problems which are coming up and presenting a challenge to contemporary society. It is not sufficiently respon­sive to the new norms and values which are replacing the old and it does not reflect properly and adequately the new ap­proach which characterizes the true purpose and function of law.

Thus, the legal system needs to be changed to become a more effective instrument for delivering justice to the poor and disadvantaged and to do this it is neces­sary to consider ways of making justice cheap, ex­peditious, and accessible. But again, this is justice of the current variety.

The Report continues with an eloquent char­acterization of the "file-fed ... dehumanized" and procrastinating character of the existing system. It comes out for a system of "law and justice at the panchayat level with a conciliatory methodology." An idealized account of pre-British panchayat justice and lengthy citation of Gandhi are juxtaposed with Lenin's commendation of popular participation and a former law minister's equation of panchayat justice with people's courts.

Nyaya panchayats are endorsed on the ground that they would remove many of the defects of the British system of administration of justice since they would be manned by people with knowledge of local cus­toms and habits, attitudes and values, familiar with the ways of living and thought of the parties before them. "[T]he poor would feel that the authority which is administering justice to them is their own and not part of an alien system which they neither understand nor trust." Yet the proposed panchayats do not depart from current notions of law. The panchayat is to be

3 For an illuminating critique, see Baxi (1975).

DECEMBER 1978

comprised of a chairman, the panchayat judge, "hav­ing knowledge of law," and two respected lay mem­bers who would be drawn from a select panel for each case by the panchayat judge. The lay members would be given rudimentary legal training. The tribunal would proceed informally, would emphasize concilia­tion; there would be no professional lawyers, etc. Legal questions would be decided by the judge and his decisions could be reviewed by the District Judge. What seems to be proposed is an informal, concilia­tory, nonadversarial, small claims court with some lay participation.

These reports indicate that even those Indian jurists who are most critical of the present system don't visualize an "indigenous" alternative to it-at least if we take "indigenous" to mean a return to earlier forms. I think it is fair to say that at the level of discourse about policy, there is an almost total obliter­ation of indigenous elements.4 However, if one looks at the social organization of modern Indian law, I think it is possible to identify elements that seem to be the extension or projection of traditional patterns.

For example, when I recently attempted to trace in detail the way a body of legal doctrine developed in contemporary India, I was surprised to see how equivocal it was and how that equivocation was so solidly institutionalized. Doctrines of authority and precedent and the hierarchic organization of courts led me to expect a body of fixed hierarchically­established doctrine. But the pressures of case load, turnover of judges, the system of sitting in small benches, the limitations of legal research, and other factors combined to transform the law into something very different. What purported to be a pyramidal hierarchy establishing fixed doctrine turned out to be a loose collegium presiding over an open-textured body of learning within which conflicting tendencies could be accommodated and elaborated.s

This suggests that the pattern of continuities and discontinuities is more complex than we might infer from the overt and dramatic displacement of pre­British law. What was displaced was, after all, not a seamless monolith but a complex comprising a dis­tinctive body of elevated legal learning, linked indi-

4 The indigenous content of family law, the last redoubt of traditional law in the modern system, has been eroded by a series of post-Independence reforms. A brief account may be found in Galanter (1978). J .D.M. Derrett (1978), the leading student of these matters, contends that the traditional mold of Hindu mar­riage law was shattered by 1976 legislation.

S The example is elaborated in my forthcoming book, "Compet­ing Equalities: The Indian Experience with Compensatory Dis­crimination. "

45

rectly and unevenly to governmental practice and to the self-regulatory activities of villages, guilds, castes, and other groupings. Hindu legal learning con­templated broad delegation of regulatory functions to these groups, an expectation that was if anything ex­ceeded in practice.s "Law" in the indigenous style included not only regulation that was local and com­prehensible, but elements that were more external and opaque to the locals. The imposition of British law introduced both new legal learning and new techniques for impressing this learning on the various lesser regulatory systems. Much of the subsequent history of law in India may be seen as the interplay between the nationalization of legal activity and the continuing drive for self-regulation in many sectors of social life: local interests and understandings found new channels of expression and the new na­tional institutions learned to accommodate diverse sorts of local regulatory activity.

The tension between authoritative higher law and local)aw-ways was not introduced with British law, but was a constituent part of the earlier indigenous system.7 Perhaps modes of combining local and "ex­ternal" elements have survived the drastic changes of content. Perhaps new and distinctively Indian ways of accommodating these tensions have arisen. Or perhaps the Indian experience exhibits universal fea­tures of the accommodation of diverse normative or­derings in the legal systems of complex heteroge­neous societies. Certainly neither the presence of a multiplicity of normative orders nor the gap between local law-ways and the most authoritative legal doc­trine are phenomena unique to India. But the sociol­ogy of law has had great difficulty in addressing these phenomena-in part because it has shadowed a tradi­tion of normative learning which insistently obscured the visibility and centrality of these features by brand­ing them as marginal, transient, or pathological (Gal­anter 1974). Hindu law, on the other hand, openly embraced normative diversity; legal learning was at­tuned to a multiplicity of legitimate group norms. It may well be that the older Indian learning has some­thing to tell us about the patterns that prevail in India today. For example, indigenous notions of collegial authority or of deference to local decision makers may help us to understand patterns of regulation in contemporary India. And by extension they may help

6 The actual situation was rendered complicated beyond elab­oration here by the presence of Islamic as well as Hindu legal traditions.

7 Kidder (1978) attempts to separate the foreignness and the "external" character of British law in India.

46

us to develop concepts for exploring the way in which the official, national legal systems in modern societies accommodate the variety of normative orderings which, in spite of the law's pretensions to hierarchic control, coexist with it. The relation of official law to the other normative orders may, I surmise, provide the key to a deeper understanding of law in modern societies (Galanter 1971). By learning from India's very different way of visualizing them, we can hope to illuminate these relations.

References

Baxi. U pendra. "Legal Assistance to the Poor: A Critique of the Expert Committee Report." Economic and Political Weekly, 10(27): 1005-13, 1975.

Baxi, Upendra and Marc Galanter. "PanchayatJustice: An Indian Experiment in Access." In Mauro Cappelletti and Bryant Garth, editors. Access to Justice: Emerging Perspectives and Issues. Leiden and Boston/Milan: Sijthoff/Giuffre (in press).

Derrett. J .D.M. The Death of a Marriage Law: Epitaphfor the Rishis. New Delhi: Vikas, 1978.

Gadbois, George H. Jr. "The Emergency: Ms. Gandhi. the Judiciary and the Legal Culture." Paper presented to the 16th Annual Meeting of the Southeast Conference of the Associa­tion for Asian Studies, 1977.

Galanter. Marc. "The Uses of Law in Indian Studies." In Lan­guages and Areas: Studies Presented to George V. Bobrinskoy. Chicago: University of Chicago Press. 1967. pages 37-44.

--. "The Displacement of Traditional Law in Modern India." Journal of Social Issues, 24:65-91. 1968. --. "Hinduism, Secularism, and the Indian Judiciary." Philos­

ophy East and West, 21:467-87, 1971. --. "The Aborted Restoration of 'Indigenous' Law in India."

Comparative Studies in Society and History, 14:53-70, 1972. --. "The Future of Law and Social Sciences Research." North

Carolina Law Review, 52: 1060-68. 1974. --. "Remarks on Family Law and Social Change in India." In

David C. Buxbaum, editor. Chinese Family Law in Historical and Comparative Perspective. Seattle: University of Washington Press, 1978. pages 492-97.

Government of India. Ministry of Law, Justice and Company Affairs, Expert Committee on Legal Aid. ProcessualJustice to the People, 1973. [Krishna Iyer Report].

Government of India, Ministry of Law, Justice and Company Affairs. Report on NationalJuridicare: EquaIJustice-SociaIJustice, 1978. [Bhagwati Report].

Kidder. Robert. "Western Law in India: External Law and Local Response." In Harry M. Johnson, editor. Social System and Legal Process. San Francisco: Jossey-Bass. 1978, pages 155-80.

Mayhew, Leon. "Stability and Change in Legal Systems." In Ber­nard Barber and Alex Inkeles, editors. Stability and Social Change. Boston: Little, Brown. 1971, pages 187-210.

Rocher. Ludo. "Schools of Hindu Law." In J. Ensink and P. Gaeffke, editors. India Maior: Congratulatory Volume Presented to J. Gonda. Leiden: Brill, 1972.

Toharia, Jose. "Judicial Independence in an Authoritarian Re­gime: The Case of Contemporary Spain." Law & Society Review, 9:475-96, 1975.

VOLUME 32, NUMBERS 3/4

Government Audits of Social Experiments

IN recent years both social scientists and officials of the United States General Accounting Office (GAO) have expressed concern about the problems involved in reviewing the data and the methods of social sci­ence field research. As part of its general charge from the Congress to review the expenditure of appropri­ated funds, the GAO has the responsibility for audit­ing and evaluating large-scale, policy-related pro­grams of federally-funded social research. The scope of GAO reviews may go well beyond that of the fiscal and management audits traditionally undertaken by governmental audit agencies. The reviews are carried out not only to guard against possible fiscal improp­rieties; they are also intended to assess the method­ological adequacy of the research. The GAO, accord­ingly, may evaluate the appropriateness of the re­search design, the manner in which the design is implemented, the adequacy of data processing pro­cedures, and the appropriateness of the data analysis. The GAO audits are thus potentially very broad. In assuming this responsibility, the GAO has been aware of the difficulties inherent in any evaluation effort and has recently published a review of the issues involved entitled Federal Program Evaluation: Status and Issues (USGAO 1978).

GAO audits of research may include reinterviews with respondents. This potential for reinterviews is a complex issue for those who are concerned with the effects of government audits upon social research. Two major problems with this procedure have been identified. First, with experimental designs generally, uncontrolled outside events that intrude upon an ex­periment may endanger its validity. Governmental audits, especially when they entail direct contact with participants in a study, may cause respondents either to drop out of the experiment or to answer questions with less candor. Second, government audits may vio­late the researcher's prior pledges of confidentiality to participants. This concern seems especially war­ranted when the experimental program is concerned with sensitive issues such as welfare payments or drug use. The potential invasion of the privacy of respon­dents raises the issue of whether respondents should be informed in advance that they might be inter­viewed again by the GAO. Thus, reinterviews may affect the confidentiality of information and the valid­ity of the results obtained from field research. Ac­cordingly, both the GAO and the Council recognized the need to examine the impact of such audits and to

DECEMBER 1978

by Ronald P. Abeles*

develop methods for mitigating or avoiding their in­trusive effects.

Under a contract from the GAO, the Council's Committee on Evaluation Research! carried out a survey of what is known about the effects of audits on the conduct of field research in the social sciences, as well as on the quality of the data obtained. It also identified and evaluated alternative methods by which the GAO might meet its legislated respon­sibilities in ways less potentially disruptive to canons of ethical conduct, to experimental designs, and to research results. The committee saw its review as an opportunity to offer suggestions to the research community about practices that might make the evaluation of research projects easier and more use­ful. The present article summarizes the committee's report and its major recommendations to the GAO.2

The impact of reinterviews

Given the GAO's special interest in the effect of reinterviews (i.e., the subsequent interviewing of par­ticipants in social experiments by GAO auditors), staff members of the GAO and the Council attempted to identify as many audits containing reinterviews as possible. After reviewing recent GAO audits and con-

* Ronald P. Abeles served as Council staff for the Committee on Evaluation Research during the preparation of the report upon which this article is based. He is currently a research scien­tist at the American Institutes for Research (Palo Alto, Califor­nia).

1 The chairman of the Committee on Evaluation Research is Robert F. Boruch, professor of psychology at Northwestern Uni­versity. He served as staff for the Council's former Committee on Experimentation as a Method for Planning and Evaluating Social Intervention (1971-75), and is a coauthor (with Henry W. Rie­cken and others) of that committee's major product, Social Ex­perimentation (1974), and a coeditor (with Henry W. Riecken) of a supplementary volume, Experimental Testing of Public Policy (1975) . In addition to Mr. Boruch, the members of the Committee on Evaluation Research are Howard E. Freeman, University of California, Los Angeles; Edward M. Gramlich, University of Michigan; Richard Light, Harvard University; Peter H. Rossi, University of Massachusetts; Joseph Steinberg, Survey Design, Inc. (Silver Spring, Maryland); and Harold W. Watts, Columbia University.

2 Copies of the committee's report may be obtained without charge from the GAO by requesting Audits and Social Experiments, PAD-79-I, October 1978. Requests should be directed to: U.S. General Accounting Office, Distribution Section, Room 1518,441 "Gn Street, N.W. , Washington, D.C. 20548.

47

tacting state audit agencies, the staffs found only one instance in which participants in a social research project had been interviewed first by a research team and then by the GAO. This single example, the Ex­perimental Housing Allowance Program (EHAP), does not lend itself well to analysis or interpretation of the potential impact of reinterviews because the reinterviews took place after the researchers had completed their data collection, when they could no longer have had an impact upon the experimental results. Accordingly, it became necessary to seek indi­rect evidence.

Some indirect evidence of the potential impact of audits was available from studies of the influence of confidentiality pledges on respondents' willingness to cooperate in research.3 In these case studies, experi­ments, and surveys, willingness is typically measured (obversely) either by refusals to respond (i.e., refusals to answer any questions at all or refusals to answer particular questions) or refusals to respond truth­fully. The evidence is only indirectly relevant to GAO audits, since none of the cases, surveys, or experi­ments involved GAO reinterviews.

This evidence is pertinent in the sense that it relates to the issue of the willingness of participants to re­spond. This issue is particularly critical in situations in which there is a conflict between the goal of keeping research records confidential and an administrative interest in using the records. "Administrative inter­est" implies that an individual's record, which con­tains both identification information and substantive information collected by the researcher, may be used to make administrative decisions about the individual. For example, data collected by researchers might be appropriated and used by an investigatory agency to decide whether welfare payments should or should not be made to particular individuals. In this case, the appropriation of individually-identified research rec­ords for nonresearch purposes both runs counter to the researcher's assurances to the respondents and may have harmful consequences for the experiment.

The committee noted that there has been an insuf­ficient number of audits by the GAO-or by other government agencies-involving the reinterviewing of participants to reach a firm judgment on the im­pact of reinterviews. The available indirect evidence suggests that there is a clear risk of disrupting re­search, but the evidence is ambiguous as to the mag­nitude of the risk and the consequences of a disrup­tion.

3 A review of this literature may be found in Boruch and Cecil (1979, in press). See especially Singer (1978).

48

Alternatives to reinterviews

For the auditor with an interest in establishing the quality of interview data, reinterviews with a sample of respondents may appear to be a natural course of action. However, for a number of reasons the com­mittee believes that the benefit of such reinterviews will be mattginal at best. First, a well-designed project will include a parallel study of the validity and relia­bility of its measures. For example, the veracity of responses to factual questions may be compared to already existing archival data. In such cases, reinter­views as validity or reliability checks add little, if any­thing, to comparisons made with existing data.

Second, any major study will generate information whose quality can be assayed-up to a point-without reinterviews. The researcher's procedure for editing records, for making internal checks on the consis­tency of information provided by respondents, and for making comparisons of the statistical data obtained with similar data from prior studies can be reviewed by auditors without reinterviews.

Third, reinterviews by outside auditors can yield ambiguous results. Ord~nary measurement error, for example, will typically produce differences be­tween data yielded by the interview and by the rein­terview. Differences in these results may also reflect actual changes in the views of the respondents that occur during the time between the two interviews. Respondents, for example, will have had time to think about a topic or issue, even a factual matter such as income, and may change their judgment about it or their interpretation of the question by the time the reinterview is conducted. Discrepancies may also ap­pear as the result of differences in the interview pro­cedures and the skills employed by the researchers a~d .auditors. Finally, reinterviews may produce different results merely because they are carried out under different auspices than the original interviews.

Fourth, reinterviews by outside auditors raise a host of problems associated with the researcher­respondent relationship. More of the time of the re­spondent is required. Respondents must be told that they may be reinterviewed, which may affect partici­pation rates. Any loss of goodwill among respondents that stems from reinterviews may lead to a public controversy and this may make others less willing to participate in similar studies in the future.

Since reinterviews by auditors may produce am­biguous or irrelevant data, and since audits may dis­rupt social experiments, it is clear that alternatives need to be considered. The committee offered four such alternatives for the GAO's consideration.

VOLUME 32, NUMBERS :3/4

Review of research procedures. Obviously, poor re­search data will be generated by poor research prac­tices. It is also clear that for the large majority of projects an examination of the research procedures is a sufficient basis for determining whether data will be poor. Further, a critical examination of the research methods and procedures generally involves no seri­ous privacy problems and can be undertaken without serious risk of disrupting research in the field. For these reasons the committee recommended that the GAO take every opportunity to examine research procedures before reinterviews are considered.

Parallel sampling by auditors. When a main objective of the audit agency is to establish the accuracy of the sampling and of the responses, a reasonable strategy is to obtain an independent sample of the same target population used in the original survey. This strategy provides a legitimate statistical basis for judging the quality of the initial survey. In particular, it will help to determine the extent to which a sample, described as having been selected by specified methods by the researcher, is similar in characteristics to a sample selected by the GAO from the same population and using the same selection methods.

Surrogate auditors. Direct reinterviews of respon­dents in an ongoing social experiment can disrupt the research if the reinterviews are conducted by people who are viewed with suspicion by the respondents. In particular, respondents who are able to distinguish between auditors and researchers may be less cooperative with the former, perhaps on invasion­of-privacy grounds. It may be possible to cope with this problem by using a surrogate interviewing agency to reinterview a subsample of the original respon­dents. The results of the surrogate's survey would be provided by aggregate statistical measures, so that no identification of individual responses would be possi­ble, "including screening the statistical results to pre­vent the deductive disclosure of identified responses.

Subsample reinterviews by auditors and sample augmenta­tion. Direct reinterviews may on occasion be essential to accomplish audit agency goals. For example, the GAO could choose to verify that interviews had in­deed been conducted by a data collection agency and that certain responses were given in order to check the integrity of the original interviewers. Parallel sampling is normally insufficient for accomplishing these goals, and the use of a surrogate audit agency may be unacceptable to the GAO.

An obvious approach to minimizing disruption of the ongoing research is to minimize the number of individuals who must be reinterviewed. The GAO, for instance, might select a probability sample from DECEMBER 1978

the existing experimental sample. Members of the subsample would then be reinterviewed by the GAO to elicit the information of interest to them.

The subsample involved in the auditor's reinter­views may subsequently be of little or no use to the researchers. That is, reinterviews may provoke indi­viduals to drop out of the study or to respond differ­ently in the future. If these conditions prevail, the research agency can remove the GAO subsample from its sample and analyze the remaining probability sample with results properly weighted to reflect the reduced sample size. This has implications for re­search planning. If audit reinterviews are anticipated, and if this may have negative effects, then the sample size must be increased beyond that normally required by the research design in order to accommodate sub­sequent attrition as a result of the reinterviews.

One major risk of this subsampling strategy is that the reinterview process may have an adverse effect on the participation rates of members of the original sample who are not reinterviewed. If the reinterview­ing process takes on the character of an administra­tive investigation, and if members of the sample learn of the reinterviews of the subsample through the press or through informal networks, their participa­tion rate is likely to decline.

Responsibilities of sponsors and researchers

Whether the GAO will need to reinterview respon­dents, and what roles the GAO could most effectively take in overseeing social experiments, depends partly on the activities and responsibilities of sponsors and researchers. Consequently, the committee considered several ways in which the sponsors of social experi­ments and the researchers who evaluate them may contribute to an experiment's success or failure as well as to the utility of an evaluation of the experi­ment.

Developing and implementing "Requests for Proposals." Almost without exception, social experiments and their evaluations are now undertaken in response to "Requests for Proposals" (RFPs) issued by sponsoring agencies within the government. There are three as­pects of RFPs that should be considered in regard to their impact on the substantive and methodological quality of social research.

First, many RFPs inadequately describe the re­search problem. In some cases, the description is so inadequate that the prospective researchers must create their own list of topics to be researched. Further, many RFPs fail to describe fully the policy considerations that led to the need for the social ex-

49

periment or program evaluation. And they often fail to specify the policy alternatives that are practical and politically feasible in the event of different types of findings resulting from the study. Without such in­formation, it is impossible to develop appropriate study designs.

Second, the time available for the preparation of good proposals based upon the RFPs can be insuffi­cient. That is, the length of time between the advertis­ing of the RFPs and the submission deadline for the proposals is sometimes so short that it is unlikely that any organization could develop a high quality study design.

Finally, the quality of the award process can be very uneven. The judgment processes range from peer­review procedures (similar to those at NIH) to in­house reviews, often by administrators who do not have sufficient training to make competent decisions. In addition, the awards are often made hurriedly, e.g., a sizeable proportion of RFPs end up being processed in the last months of the federal fiscal year.

Monitoring projects. The project monitor is generally the only person who has the opportunity to make sure the work is done in reasonable conformity to the design, to modify the design and funding because of unexpected contingencies, and to assure the proper conduct of the project in terms of the protection of human subjects.

Ideally, monitors should be persons with consider­able research experience and management ability. However, monitors vary greatly in research experi­ence, in interest in their job, and in availability to the project. This makes the use of advisory boards and consultants crucial, and both should be regarded as sharing responsibility for monitoring projects. Further, project monitors are often hindered by fed­eral and state regulations that prevent them from being flexible, even when they and the researchers see the advantages of modifications in design, budget, or time schedule. Because of these problems, moni­toring often becomes a pro forma activity, and social control opportunities as well as consultative inputs are foregone.

Dissemination. The sponsors of research need to provide maximum opportunities for the dissemina­tion of the results of their studies. At the present time, for example, final reports from projects are often difficult to obtain.

The dissemination of information, including re­ports, is important for two reasons. First, if studies are

50

worth doing and worth supporting, then their find­ings should be made known to the widest possible audience. This is true whether or not the projects are "successful." Second, dissemination serves as another control mechanism for improving the quality of re­search. If researchers and sponsors realize that their work will be subject to public and peer scrutiny and that this scrutiny may influence their personal careers, their performance is likely to improve.

Quality control of data. Another important responsi­bility of both sponsors and researchers is to insure that high quality data are collected and processed. However, there is no set of standards for quality that applies equally well to data collection activities for all social experiments and all social program evaluations. Although any set of standards would have to be tai­lored to particular settings, the committee did iden­tify general guidelines for reasonable practice which can be used to help assure the quality of the resultant data. The committee outlined these for the GAO primarily because the researcher's quality control procedures are an important factor to be weighed by an auditor in deciding upon the scope of an audit. In particular, the better the quality control procedures used by the researcher, the less the need for auditors to reinterv!ew respondents. 4

The activities and report of the committee provide an example of how representatives of the scientific and governmental communities can cooperatively review problems raised by the use of social science research by govet:nment. It is readily apparent that govern­mental agencies depend heavily upon social science research for evaluations of their programs and for the development of new programs. It is also apparent that the norms, values, and goals of science are often at variance with those of government. These dif­ferences cannot be ignored, and only through con­tinuous interchanges and inquiries such as that re­ported in this article can high quality social science research meet the needs of government and society.

4 Basic considerations in assaying the quality of research data are described in Riecken et al. (1974), Nunnally & Durham (1975), and many contemporary texts. Various bibliographies and pro­ceedings of specialized meetings on quality control of data are also available (National Center for Health Services Research 1977a, b; U.S. Bureau of the Census 1974; Dalenius & Lyberg 1966; Kulley 1974; Ferber 1966; Sinaiko & Broedling 1976; Sudman & Bradburn 1974).

VOLUME 32, NUMBERS 3/4

References Boruch, R.F., and J.S. Cecil. Assuring the Confidentiality of Social

Research Data. Philadelphia: University of Pennsylvania Press, 1979 (in press).

Education, and Welfare, Public Health Service, Health Re­sources Administration, 1977b(DHEW Document No. (HRA) 78-3204).

_, and Henry W. Riecken. Experimental Testing of Public Pol­icy: The Proceedings of the 1974 Social Science Research Council Conference on Social Experiments. Boulder, Colorado: Westview Press, 1975.

Dalenius, T. and L. Lyberg. Bibliography on Nonsampling Problems.

Nunnally,J.C., and R.L. Durham. "Validity, Reliability and Spe­cial Problems in Measurement in Evaluation Research." In E.L. Struening and M. Guttentag, editors. Handbook of Evaluation Research, Vol. 1, Beverly Hills, California: Sage Publications, 1975, pp. 289-354.

Institute of Statistics, University of Stockholm, Sweden, 1975. Ferber, R. The Reliability of Consumer Reports of Financial Assets and

Debts. Urbana: University of Illinois, 1966.

Riecken, H.W., et al. Social Experimentation: A Methodfor Planning and Evaluating Social Programs. New York: Academic Press, 1974.

Kulley, A.M. The Reliability and Validity of Survey Measurement of Health Related Variables: A Research Bibliography. Publication 8, Health Services Research and Training Program, Department of Sociology, Purdue University, West Lafayette, Indiana, 1974.

Sinaiko, H.W. and L.A. Broedling, editors. Perspectives on Attitude Assessment: Surveys and Their Alternatives. Champaign, Illinois: Pendleton, 1976.

National Center for Health Services Research. Advances in Health Survey Research Methods. Research Proceedings Series. Washing­ton, D.C.: U.S. Department of Health, Education, and Welfare, Public Health Service, Human Resources Administration, 1977a (DHEW Publication No. (HRA) 77-3154).

Singer, E. "Informed Consent: Consequences for Response Rate and Response Quality in Social Surveys." American Sociological Review, 43: 144-162, April 1978.

Sudman, S., and N .M. Bradburn. Response Effects in Surveys: A Review and Synthesis. Chicago: Aldine, 1974.

U.S. Bureau of the Census. Indexes to Survey Methodology Literature. Technical Paper #34. Washington, D.C.: U.S. Department of Commerce, Social and Economic Statistics Division, 1974. National Center for Health Services Research. Experiments in

Interviewing Techniques: Field Experiments in Health Reporting! 1971-1977. Washington, D.C.: U.S. Department of Health,

U.S. General Accounting Office, Federal Program Evaluation:

New directors and officers The Council's board of directors, at its

meeting on June 17, 1978, elected five directors to serve three-year terms. Philip Jackson, psychology, University of Chicago, was reelected a director-at-large. Newly-elected directors-at-large are Jane B. Lancaster, anthropology, University of Oklahoma, and Murray L. Schwartz, law, University of California, Los Angeles. Also elected were two new directors from professional associations: Rosedith Sit­greaves Bowker, Stanford University (American Statistical Association), and Robert A. LeVine, Harvard University (American Anthropological Association).

The board also elected the Council's officers for 1978-79: Otto N. Larsen, sociology, University of Washington, chairman; Samuel C. Patterson, political science, University of Iowa, vice chair­man; Mr. Levine, secretary; and Mrs. Bowker, treasurer.

DECEMBER 1978

Status and Issues. PAD-78-83, October 1978.

Personnel Staff appointments

George Reid Andrews joined the Council staff on September 15, 1978. as staff associate for the Joint Committee on Latin American Studies. Mr. Andrews re­ceived a B.A. in history from Dartmouth College in 1972, an M.A. in history from the University of Wisconsin in 1974, and a Ph.D. in history from Wisconsin in 1978. He comes to the Council from the Pan American Society of New England (Boston), where he held the position of director of the Society'S Shattuck Library from 1977 to 1978. His major interests are in the fields of comparative and social history.

Lonnie R. Sherrod joined the Council as a staff associate on September 6, 1978; he will staff the Committee on Life­Course Perspectives on Middle and Old Age, the Committee on Work and Per­sonality in the Middle Years, the Commit­tee on Biosocial Science (biological foun­dations of parenting and offspring de­velopment), and the Committee on

Biological Bases of Social Behavior. Mr. Sherrod received a B.A. in psychology and zoology from Duke University in 1972, an M.S. in biology from the Univer­sity of Rochester in 1974, and a Ph.D. in psychology from Yale University in 1978. At Yale, he worked with William Kessen in the Studies of Infancy program and with Jerome L. Singer at the Family Tele­vision Research and Consultation Center.

Anne F. Thurston joined the Council on August 7, 1978 as staff associate for the Joint Committee on Contemporary China. Miss Thurston received a B.A. in government from Tufts University in 1965, an M.A. in political science from the University of California at Berkeley in 1966, and a Ph.D. in political science from Berkeley in 1975. She comes to the Coun­cil on leave from Fordham University, where she has been assistant professor of political science since 1974. Her special interest is in rural local politics and eco­nomic development in China.

51

52 VOLUME 32, NUMBERS 3/4

Recent Council Publications

a-pan in the Muromachi Age, edited by John Whitney Hall and Toyoda Takeshi, with the assistance of Kanai Madoka and Richard Staubitz. Papers from a confer­ence sponsored by the Joint Committee on Japanese Studies. Berkeley and Los Angeles: University of California Press, 1977. xv + 376 pages.

/ d h' . . th Mo ern C lOese Literature 10 e May Fourth Era, edited by Merle Goldman. Papers from a conference sponsored by the Joint Committee on Contemporary China, August 1974. Harvard University Press, 1977.464 pages + xiii. $15.00.

./ Modes of Perceiving and Processing In­formation, edited by Herbert L. Pick, Jr. and Elliot Saltzman. Papers based upon two workshops held during the spring of 1974 and 1975 and sponsored by the Committee on Cognitive Research. Hills­dale, New Jersey: Lawrence Erlbaum Associates, 1978. Distribllted by the Hals­ted Press Division of John Wiley and Sons.

Political Development in Eastern Europe, edited by Jan F. Triska and Paul M. Cocks. Product of a conference spon­sored by the Joint Committee on Eastern Europe of the American Council of 'Learned Societies and the Social Science Research Council, held at Stanford Uni­versity, December 4-5, 1975, New York: Praeger Publishers, 1977.374 pp. + xxiv. Hardbound, $30.00; paperback, $7.95.

.J Rural Small-scale Industry in the People's Republic of China, report of the American rural small-scale industry dele­gatio!). chaired by Dwight H. Perkins, Harvard University, whose travel was a part of the exchange program of the Committee on Scholarly Communication with the People's Republic of China and the Chinese Scientific and Technical Association. The Committee is jointly sponsored by the Council, the American Council of Learned Societies, and the Na­tional Academy of Sciences. Berkeley: University of California Press, 1977. 296 pages.

L:king to Children: Language Input and Acquisition, edited by Catherine E. Snow and Charles A. Ferguson. Papers from a conference sponsored by the Committee on Sociolinguistics, Septem­ber 6-8, 1974 at the American Academy of Arts and Sciences, Boston, Massachu­setts. Cambridge, London, New York, and Melbourne: Cambridge University Press, 1977. 369 pages, including an an­notated bibliography.

Toward a Metric of Science: The Advent of Science Indicators, edited by Yehuda Elkana, Joshua Lederberg, Robert K. Merton, Arnold Thackray, and Harriet Zuckerman. Product of a conference sponsored by the Social Science Research Council and the Center for Advanced Study in the Behavioral Sciences. New York, Chichester, Brisbane, and Toronto: Wiley-Interscience, 1978. xiv + 354 pages. $19.95.

SOCIAL SCIENCE RESEARCH COUNCIL 605 THIRD AVENUE, NEW YORK, N.Y. 10016

Incorporated in the State of Illinois, December 27, 1924, for the purpose of advancing research in the social sciences

Directors, 1978-79: IRMA ADELMAN, ROSEDITH SITGREAVES BOWKER, ROBERT EISNER,jACOBj. FELDMAN, CLIFFORD GEERTZ, PETER R. GOULD, PHILIP W.

JACKSON, FRANKLIN W. KNIGHT, GERALD H. KRAMER, JANE B. LANCASTER, OTTO N. LARSEN, ROBERT A. LEVINE, CORA BAGLEY MARRETT, PAUL H.

MUSSEN, SAMUEL C. PATTERSON, MURRAY L. SCHWARTZ, ELEANOR BERNERT SHELDON, STEPHAN A. THERNSTROM

Officers and Staff ELEANOR BERNERT SHELDON, President; DAVID L. SILLS, Executive Associate; GEORGE REID ANDREWS, RONALD AQUA, ROBERT A. GATES,

MARTHA A. GEPHART, ROBERTA BALSTAD MILLER, ROWLAND L. MITCHELL,jR., ROBERT PARKE,JAMES L. PETERSON, PETER B. READ, LONNIE R. SHERROD,

DAVID L. SZANTON, ANNE F. THURSTON; MARTHA W. FORMAN, Assistant Treasurer; NANCY L. CARMICHAEL, Librarian

SOCIAL SCIENCE RESEARCH COUNCIL 605 Third Avenue, New York, N.Y. 10016

ADDRESS CORRECTION REQUESTED