Anonymity and Sensitivity

Fritz Scheuren, SpeakerNORC University of Chicago

February 2013

Scope of USA Migration Studies

• Emigration from USA to World• Immigration from World to USA

Direct Indirect

Outline of Talk

• Problem• Science of Approach• Engineering of Its Implementation• Results So Far!• Next Steps?

Immigrants Sans Paper

• Physical Border Protection Worldwide?•Mostly “Thought” to be South to North?• Only An Indirect Estimate Available

Legal Immigrants First?

• Over a Million Come Annually, still Largest Legal Flow Of Any Country• About 150 Million Made citizens since 1940

Undocumented May Be Much Smaller?

• Flow is believed to average Less than Legal • Stock Estimated, perhaps, at 12 million• Flow is Business Cycle Driven, but usually less

Physical Border Protection

• Extremely long Borders with Canada and Mexico• Mexico Border Not Well Guarded • Drug Trafficking a Major Problem for USA/Mexico

Mainly South to North?

• Mexican Border Crossers but This Model is too Simple• Student and Visitor Visa Overstays • Still Often Tied to US Jobs

Who Comes Legally Now?

• Traditional Americas and Western Europeans but Less legally than earlier• Eastern Europeans Still in Large Numbers• Koreans and Chinese up, Africa coming up

Science Approach

• Two Card Method (Handout)• Two Independent Replicates• Difference Estimate of Number of Undocumented

Two Card Example(Green Card)

First Card

• Two options Given --• First option: US Citizenship or Undocumented• Second Option: Green Card or Other

Two Card Example(Other than Green)

Second Card

• Two Options Given –• First Option: US Citizenship• Second Option: Green Card or Other• Both Cards: Safe Answer

Difference Undocumented

• Estimate Unbiased, if Question Understood and Answered Honestly• But Variance Twice as Large as Direct Question• However Bias Eliminated

Engineering Solution

• Human Factors Design Study• To test Factor Importance• Protecting Anonymity to Gain Unbiased Answers

A Credible Balancing Bar?

Balance of Trust?• Key is Creating High Anonymity to Overcome PERCEIVED level of risk.• Given the Sensitivity of the Subject• Need Data so we tested.

Sensitivity Loss

• The Two Card method has a Variance Cost and is a challenge in moving Univariate analysis to a Multivariate form• The Newness remains for Anonymity form

Anonymity Loss

• Credibility of Promise is a real concern?• So Move to a Promise that CONSTRUCTS Established Trust? • Key Improvement!

Results So Far• About 350 tests of the Two-Card method and its variations have been undertaken.• The tests show that the results are robust to minor wording changes

More Results So Far

• The Results are not robust to being wordy. • Potential respondent are not paid, probably a mistake. • If version was lengthy, more break-offs likely.

Still More Highlights

• Location changes, also are not robust. • Intercepts near the Mexican Border (near San Diego) were seen as more risky, so stopped.

Other Changes?

• If scaled up, more characteristics and more intercept locations are definitely needed.• Respondent based area probabilities needed to adjust for self selection.

Interviewer Training

• The extensive paradata calculated needs to be turned into a model for training.• Intercept Interviews are changing NORC Team. • How Measure/Control?

Intercept Scalability

• How replicable are the Interceptors we chose?• How replicable and scalable is this process?•What controls are there and how scientific are we?

Selected Respondent Comments

• [Interview] was easy, because none of the questions are hard and explained it very well.• [Glad you] understand the immigrants and their education level.

More Respondent Comments

• He was hesitant at first and I asked him why. • He said because people [might identify him].I asked him how I was going to find out. I told him [many] are similar].

Next Steps Coming?

•Write up Results of this small NSF grant?• Expand to another Undocumented group: Asians, Russian Jews?• Take Sting off Hispanics

Portable to Europe?Your Question!

• In a register based society is this needed?• Some evidence says Yes but little leakage?• More references needed? Short list follows?

Many, Many Thanks!

• Contact Details Fritz Scheuren +1-202-320-3446 scheuren@aol.com

References• Cochran, William G. Sampling Techniques.

New York: Wiley, 1977. Print.• Dyson, Freeman J. "Is Science Mostly

Driven by Ideas or by Tools?" Science 338.6113 (2012): 1426-427. Print. • Galison, Peter. Image and Logic: A Material

Culture of Microphysics. Chicago: University of Chicago, 1997.

References

• GAO. A "Grouped Answers" Approach to Surveying Foreign-Born Respondents. Rep. no. GAO-06-775. N.p.: GAO, 2006. Print.• Kuhn, Thomas S. The Structure of

Scientific Revolution. Chicago: University of Chicago, 1996. Print.

References• Lehoux, Daryn, and Jay Foster. "History

and Philosophy of Science: A Revolution of Its Own." Science 338.6109 (2012): 885-86. Print.

• Pasic, Peter.” What the Romans Really Knew,” Science 338:273-274.

• Reynolds, Gregg. Immigrant Experience Survey Interim Report. , 2013.