Is the Census Good Enought (Keyfitz N., 1984)

7/27/2019 Is the Census Good Enought (Keyfitz N., 1984)

1/8

Association for Public Policy Analysis and Management

Is the Census Good Enough?Author(s): Nathan KeyfitzSource: Journal of Policy Analysis and Management, Vol. 3, No. 4 (Summer, 1984), pp. 597-603Published by: Wiley on behalf of Association for Public Policy Analysis and Management

Stable URL: http://www.jstor.org/stable/3324547 .Accessed: 30/07/2013 00:48

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at .http://www.jstor.org/page/info/about/policies/terms.jsp

.JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of

content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms

of scholarship. For more information about JSTOR, please contact [email protected].

.

Wiley,Association for Public Policy Analysis and Management,John Wiley & Sons are collaborating with

JSTOR to digitize, preserve and extend access toJournal of Policy Analysis and Management.

http://www.jstor.org
http://www.jstor.org/action/showPublisher?publisherCode=blackhttp://www.jstor.org/action/showPublisher?publisherCode=appamhttp://www.jstor.org/stable/3324547?origin=JSTOR-pdfhttp://www.jstor.org/page/info/about/policies/terms.jsphttp://www.jstor.org/page/info/about/policies/terms.jsphttp://www.jstor.org/stable/3324547?origin=JSTOR-pdfhttp://www.jstor.org/action/showPublisher?publisherCode=appamhttp://www.jstor.org/action/showPublisher?publisherCode=black


2/8

Insights

ISTHECENSUSGOOD ENOUGH?NathanKeyfitz

RandallR. BovbjergJamesW. VaupelEditors

Any large collection of data contains errors, and the 1980 census,being a count of the large and restless population of the UnitedStates, contains many errors. The count is formally as of midnightbetween March 31 and April 1, but persons who died after thatdate and before enumeration can easily be missed. Lodgers inrooming houses that are in violation of zoning codes, men whoseknown presence in a household would result in cessation of welfarechecks, persons who entered the country illegally-these groupsare not going to be counted if they can possibly avoid it. On theother hand, people with two homes may be counted twice. Theintensity of the 1980 census publicity made double counting aspecial hazard; when one persistently calls on people to do theirduty as citizens because it will help their municipality, some willdo more than their duty.Yet with all its errors the census is easily good enough to tell usthat between 1970 and 1980 there was a reversal of the rural ex-odus in the United States; that the move to the Sunbelt is con-tinuing; that Spanish-speaking people are on the way to catchingup with blacks as the major minority group of the United States.Such facts and thousands like them are given with all needed ac-curacy by the census as it was taken in 1980 and for many decadesearlier, in a tradition of careful enumeration that goes back to the18th century. The census-taking process includes measurement ofits own errors,and these are well under control for statistical uses.In the free and mobile population of the United States countingto the last million persons, let alone the last person, is out of thequestion. And even if the count could be exact the definitions arearbitrary. The census counts college students at the place wherethey are studying, rather than at their parents' homes, but sec-ondary school students are counted at their parents' homes.Americans traveling abroad are counted at their homes in theUnited States; foreigners touring in the United States are omitted.It is easy to find countries where such matters are handled differ-Journal of Policy Analysis and Management, Vol. 3, No. 4, 597-620 (1984)? 1984 by the Association for Public Policy Analysis and ManagementPublished by John Wiley & Sons, Inc. CCC0276-8739/84/030597-24$04.00

This content downloaded from 186.18.32.91 on Tue, 30 Jul 2013 00:48:28 AMAll use subject to JSTOR Terms and Conditions
http://www.jstor.org/page/info/about/policies/terms.jsphttp://www.jstor.org/page/info/about/policies/terms.jsphttp://www.jstor.org/page/info/about/policies/terms.jsp


3/8

Is the Census GoodEnough?

ently, and everywhere there is a penumbra around the count con-stituted by the arbitrarydefinitions.Though all this is perfectly harmless for statistical purposes, itraises problems for the apportionment of seats in the House ofRepresentatives and for the new uses for census figurescreated byCongressin the last two decades. The allocation of large amountsof funds by formulas that include population requires exactness.Using such formulas, Congress has directed the distribution oflarge quantities of funds among lower-level jurisdictions-atpresent 39,000 of them. High local taxes cause people to flee; butno flight within the United States from the taxing power of UncleSam is possible. The Bureau of the Census apparently did acareful job in 1980, and its measures of both sampling and non-sampling error show that these are sufficiently small for most ap-plications, but not for a Treasuryofficial who has to transmit thepayment pursuant to the formula embodied in a statute. For ac-counting purposes nothing less than absolute accuracy-to thelast person-will serve.It has not been sufficiently noticed that for this new use of thecensus the entire apparatus of measured error and the concept ofsufficient accuracy becomes inapplicable. An indeterminacy thatis negligible in relation to the study of migration or of educationbecomes a stumbling block when the Treasury s requiredto writeout a check. Though city treasurers may well compromise on adiscrepancyof 1person,or 1000persons, it has to be insisted uponthat the accounting and the statistical approaches are distinct inlogic as well as in practice. No willingness to compromise canescape the fundamental difference in principle between ac-counting and the probability calculations of statistics.This point is emerging fromsuits now in the courts whose objectis to require the United States Bureau of the Census to adjust itsfigures for purposes of allocation. The case that is being watchedparticularly closely is that of New YorkCity and New York Statejointly against the Bureauof the Census;if the plaintiffs win thereare many cases in other courts that will come to life. No one cansay that the judicial process is precipitate; New York began itslegal onslaught more than three years ago; witnesses have nowbeen heard and a decision is expected by the summer of 1984.The plaintiffs contend that New YorkCity's population is shortby 9%, or that 1 person in 11 was omitted. One would havethought that so gross an omission would be readily visible, andyet the kind of evidence that it has been possible to secure hasbeen uncertain and in places self-contradictory.Theplaintiffs, whose witnesses include outstanding statisticiansand demographers,have assimilated the problem to that body ofstatistical literature that has made such enormous advances notonly in theory, but in application in a score of fields, from agri-culture to industrial quality control. Thus, the distinguished JohnTukey argued for the plaintiffs that since an average of two num-bers, each subject to random error, is better than either one ofthem by itself, the census government should take the sample

598



4/8

Is the CensusGoodEnough? 599

checks that were carried out in Apriland Augustof the census yearand average their results with the decennial census data, which isequivalent to using them to correct the decennial data.It is certainly true that the average of two estimates has anexpected departure from the true figure that is less than the av-erage departure of either one separately-if the two estimates areboth unbiased and of equal legitimacy. But this condition of sym-metry is not applicable here; the decennial census is indeed anestimate, but one made household by household across the lengthand breadth of the country. It emerges from a 200-year tradition;it corresponds to the expectation of the public; in lengthy congres-sional debates on revenue sharing and other grants no mention ofa modification of the census is to be found. The proposed correc-tion, on the other hand, is a resurvey matched name by name tothe census. Its matching error is larger than the supposed errorofthe census.Asymmetry in the two figures, either of legitimacy or of accu-racy, makes the averaging argument questionable.An example of the inadequacy of standard statistical theory isthat an error of one person in the census count could deprive astate of a representative. It makes people uncomfortable to hearthis, and my statement to this effect has even been disputed byprofessionals, but is is an easily demonstrated mathematical fact:There has to be some point at which one person omitted from thecensus will deprive a state of a representative. The proof is simple,by the method of assuming the opposite and obtaining a contra-diction. Suppose that there is no point at which a difference ofone person will lose a representative for a state. Then if thenumber of representatives is a function f(P) of the population P,we have for all values of P the equation f(P) = f(P - 1). This beingtrue for all values of P, the equalities f(P - 1) = f(P - 2), f(P -2) = f(P - 3), ... , all hold, so all values of the function f(P) areequal to one another. In short, f(P) is not a function of populationat all. If there is no point at which a difference of one personmakes a difference of one representative, then the number of rep-resentatives is not a function of population. But we know that thenumber of representatives in a state is a function of population.The eminent witnesses for the plaintiff, with the probabilitymodel in mind, say in effect, "Let us do this thing right." Citingstatistical principles that have worked in industrial quality con-trol, in agricultural trials, and in many other fields, they proposeselecting a proper sample of individuals from various lists outsidethe census that are known to be relatively complete, including thecarefully enumerated monthly CurrentPopulation Survey, searchthe census to see if each individual has been enumerated or not,and so obtain a clear statement of completeness for each state;then we could add to the census count the number by which eachstate is short. Theoretical statistics assures us that we can makethe error of such an adjustment as small as we want, simply byincreasing the size of the sample that is searched. With a large



5/8

Is the Census GoodEnough?

enough sample we can have whatever accuracy is required. Infact that work has already been done by the Bureauof the Census,using impressively large samples.Yet sound principles of probability sampling bump againstmatching uncertainty. A typical outcome is that out of 1000names derived from noncensus lists 900 are identified beyonddoubt in the census, 20 are proven omissions, and 80 are indeter-minate. For each of those 80 names, some name is found in thecensus that might be the same person, but the address is differentand the age is not the same, or there is a failure to match on othercharacteristics. If the two are not the same personthen the censusis short by one.In such a search to test the census those 80 unmatched personsare a serious stumbling block. If none of them is in the censusthen the shortfall is 100 persons or 10%. If all of them are in thecensus the shortfall is 2%. The most that the procedurehas provedis that between 2 and 10%of the population has been missed, andthat is not very helpful.One could split the uncertainty and say that half of the 80 per-sons are really in the census but could not be identified forcertain,and the other half were genuinely missed, so the census is exactly40 + 20 = 60 persons or 6%short. That is the kind of exactitudethat the plaintiffs in the New York case are urging on the court.Such an argument has a superficial resemblance to the commontheory that errors balance each other out, but it is a caricature ofstatistics. Those unidentified persons are no random sample, andno convincing probability mechanism can be contrived that wouldshow balancing out.Thus, the side in the trial that is statistically most pure, thatsays in effect, "Let us do the thing right," ends up by producingan exact figure, but it does so by arbitrarily taking a midpoint ofa range of undeterminacy. I sympathize with its position, becauseI too have been frustratedby the obstinacy of the matching processon more than one occasion. Yet it is ironic that an argumentbeginning with a very attractive and detailed exposition of refinedstatistical principles ends up by splitting the difference likesomeone trying to sell a used car.We are fortunately protected against acceptance of such argu-ment, if not by common sense and skepticism, then by the factthan an equally convincing argumenton other data produces verydifferent results.For the United States as a whole we know fairly well, usingdemographic methods developed by Ansley Coale, the degree ofcompleteness of enumeration for whites and for blacks. Blackshave always been less completely enumerated than whites. If thenational undercoverage ratios are applied to the states separatelyfor blacks and for whites, this provides an estimate called "syn-thetic" for each state and smaller jurisdictions. On this estimateWyoming, having few blacks, looks as though it was well enumer-ated. Yet in the matching study Wyoming appeared among theleast completely enumerated states.

600



6/8

Is the Census Good Enough? 601

When methods that are individually flawed give similar resultswe are encouraged to overlook the flaws and take it that any oneof them is satisfactory. When methods that are individuallyflawed give results that are far apart, then we have to admit thatthe sum of the knowledge obtained from them is small; they havecancelled one another out. That is why the U.S. Bureau of theCensus, even though it has invested large amounts of time andmoney in tests, is disinclined to use any results so far obtained foradjustment.But let us overlook the disagreements among methods of ad-justment and suppose that all of them give some improvement onthe census. We cannot adjust without choosing, so suppose wetake matching studies as the best of the methods and proceed toapply it. No one contends that it will improve on the census fig-ures for all states and other jurisdictions; the most anyone assertsis that it makes an improvement on the average. Suppose that itimproves 60% of jurisdictions and takes 40% farther from thetruth. Is that acceptable?I do not believe that it is, but have no way of proving that eitherit should or should not be accepted. Everything depends on whatloss function one has, which is to say, how seriously one regardsan error made in a traditional form of census as against an errormade in pencil-and-paper adjustments after the census has beentaken. If an error made in the traditional census is less offensivethan an equal error gratuitously superimposed, then just makingan improvement on the average is not good enough. One mightwant assurance that 90% of the areas are better for the adjust-ment; one might even insist on 99%.In fact, we would still have trouble if there was even 1% ofjurisdictions whose count as a fraction of the United States wasincorrectly adjusted downward. I mean trouble in logic, as wellas in practical politics. A town has its own count of 3019 people,say; the census counted 3019 people; how does one explain to itthat super-sophisticated methods of calculation give it only 2973people?Ofcourse, most areas would not know their numbers so exactly.To adjust an area incorrectly downward as a proportion of thewhole, and then to hide behind the fact that it would have a hardtime proving this against you, is not the kind of conduct one rec-ommends to an official agency. Onepercent of 39,000 areas is 390,and all of them could potentially sue on the ground that the Bu-reau of the Census made a gratuitous adjustment that deprivedthem of funds or representation. How far do we want to encouragethis protest, however hard it may be to prove?In 1970 the census was estimated to be short by about 5 millionpersons. It might well have been possible to adjust so that in 80%of the 39,000 jurisdictions the adjusted value would have beencloser to the truth, and in 20% farther. In 1980 the census wasmore complete, and that makes adjustment more difficult. Nowit might be possible to improve 60% of the jurisdictions as re-ported in 1980, at the same time making 40% of them worse.



7/8

602 Is the Census Good Enough?

There is no way of knowing these percentages, which I offer onlyas hypothetical, and even if they were known we would be igno-rant of which jurisdictions were better estimated and which werepoorly estimated after adjustment.There is a clear division of labor here: It is up to the statisticianto estimate in what fraction of jurisdictions a proposed methodwill improve on the census; it is up to the policymakeror the courtto decide whether that fraction justifies the adjustment.What comes out of all this is that the side that argues for sta-tistical purity turns out to be the one that ends by taking an as-sumptional shortcut for which the justification is thin or nonex-istent. It would be good if standard statistical theory could bebrought to the problem of census completeness, but not at theprice of making gross assumptions that to common sense seemwholly implausible. And not at the price of disregardingthe entireset of public expectations that start with the Constitution, whichspeaks of the number of persons enumerated, and continues in anunbroken tradition to our own day.The inapplicability of the probability model in this use forces anew approach. We need exactitude, which is not provided in sta-tistics. For such exactitude we must look in a different direction-the establishment of a convention that would be agreed to or im-posed on all the stakeholders. Thepossible conventions are (1) theunadjusted census numbers; (2) the census as adjusted for color,age, and sex, on a synthetic method with ratios for color, age, andsex, on a synthetic method with ratios obtained from the countryas a whole; also in this category would be a matching study withobjective rules agreed on in advance; (3) the census numbers ad-justed by all relevant variables and the work carried out by areferee in whom the stakeholders would have confidence. If thereferee is given the right to choose variables the process losesuniqueness. Thus, the third possibility would be the most accu-rate on the average, but would require unquestioning acceptanceof the referee'sjudgment.The first of these-the census numbers as they emerge fromreasonably careful, and certainly expensive, routine procedure-has been the convention of the past, and was apparently whatCongresshad in mind when it passed the legislation. The secondis easily carried out, is perfectly objective, would make an im-provement on the average, and has only the drawback that somelocalities would be made worse by it.That is the kind of consideration that throws us back for 1980to a convention that we agreeon the census as enumerated. What-ever may have been true earlier, the census as taken in 1980 wasof sufficient accuracythat no one has so far found an unambiguousway of improving it. Improvement on the average is not goodenough when the method assigns wins and losses, in many in-stances incorrectly.This way of making the census exact by having it constitute aconvention on which all parties would agree does not mean thatany way of taking the census is good enough. A census demon-strably in gross error,readily improvable, is deprived of the legit-



8/8

Encouraging omesticity: CloserLook 603

imacy that is so important for the present purpose. It could beargued that the 1970 census was indeed improvable by adjust-ment. The 1980 figure has a measured undercount of a small frac-tion of 1%. It is short by the number of the illegals, but thatnumber is so elusive that there is no way of adjusting for it.The acceptance of a convention by the stakeholders is mucheasier before the outcome of the census count is known. Knowl-edge of who would gain and who would lose by a convention agree-ment is impossible. That is the case for deciding the rules aheadof the actual taking of the census, and the Bureau of the Censuswent some distance in informing people in advance how it wasgoing to take the census, though it did not decide in advance onadjustment.Ansley Coale points out that whether the runner is safe on baseor out is decided by the umpire on the spot, and no photographicreplay alters that decision. I would like to follow up on that met-aphor. What if the umpire repeatedly favors one team, or justmakes too many mistakes in the opinion of the public and theplayers? The remedy is to seek a new umpire for subsequentgames, not to adjust his decisions one by one. We need a censusbureau good enough that its announced results are trustworthy asthey stand. No one has suggested that some other agency couldimprove on the U.S. Bureau of the Census. As long as it continuesunquestionably superior to any alternative, we must accept itsresults. The quality control is exercised on the institution, not onthe individual numbers it puts out.

SummaryThe new use that Congress has established for the census-allo-cation of revenue sharing and other grants-requires exactitude,and does not fit with the probability models that have been soeffective in other fields. Probability methods can only provideexact numbers by a series of gratuitous assumptions. As long asthe census count is reasonably close to the actuality as revealedby the checks now made, its acceptance as a simple conventionwould seem to be indicated. We do need an external quality con-trol, of course, but that control should be on the institution, noton its individual results.NATHANKEYFITZ s the head of the PopulationProgramat IIASA.

ENCOURAGINGn his recent contribution to Insights, Neil Gilbert1argues that anDOMESTICITY:ncrease in full-time homemakers would lead to better child-A CLOSEROOKrearing, thus improving the health, behavior, and values of thenation's youth. Gilbert proposes to encourage parents to stayKarlHauskerhome by awarding "social credits" to full-time homemakers. The

Encouraging omesticity: CloserLook 603

imacy that is so important for the present purpose. It could beargued that the 1970 census was indeed improvable by adjust-ment. The 1980 figure has a measured undercount of a small frac-tion of 1%. It is short by the number of the illegals, but thatnumber is so elusive that there is no way of adjusting for it.The acceptance of a convention by the stakeholders is mucheasier before the outcome of the census count is known. Knowl-edge of who would gain and who would lose by a convention agree-ment is impossible. That is the case for deciding the rules aheadof the actual taking of the census, and the Bureau of the Censuswent some distance in informing people in advance how it wasgoing to take the census, though it did not decide in advance onadjustment.Ansley Coale points out that whether the runner is safe on baseor out is decided by the umpire on the spot, and no photographicreplay alters that decision. I would like to follow up on that met-aphor. What if the umpire repeatedly favors one team, or justmakes too many mistakes in the opinion of the public and theplayers? The remedy is to seek a new umpire for subsequentgames, not to adjust his decisions one by one. We need a censusbureau good enough that its announced results are trustworthy asthey stand. No one has suggested that some other agency couldimprove on the U.S. Bureau of the Census. As long as it continuesunquestionably superior to any alternative, we must accept itsresults. The quality control is exercised on the institution, not onthe individual numbers it puts out.

SummaryThe new use that Congress has established for the census-allo-cation of revenue sharing and other grants-requires exactitude,and does not fit with the probability models that have been soeffective in other fields. Probability methods can only provideexact numbers by a series of gratuitous assumptions. As long asthe census count is reasonably close to the actuality as revealedby the checks now made, its acceptance as a simple conventionwould seem to be indicated. We do need an external quality con-trol, of course, but that control should be on the institution, noton its individual results.NATHANKEYFITZ s the head of the PopulationProgramat IIASA.

ENCOURAGINGn his recent contribution to Insights, Neil Gilbert1argues that anDOMESTICITY:ncrease in full-time homemakers would lead to better child-A CLOSEROOKrearing, thus improving the health, behavior, and values of thenation's youth. Gilbert proposes to encourage parents to stayKarlHauskerhome by awarding "social credits" to full-time homemakers. The

This content downloaded from 186.18.32.91 on Tue, 30 Jul 2013 00:48:28 AM
http://www.jstor.org/page/info/about/policies/terms.jsphttp://www.jstor.org/page/info/about/policies/terms.jsp

Is the Census Good Enought (Keyfitz N., 1984)

Documents

Transcript of Is the Census Good Enought (Keyfitz N., 1984)