An Introduction to An Introduction to Multiple Systems Estimation Multiple Systems Estimation

for Estimating a for Estimating a Count of Adverse EventsCount of Adverse Events

Jana AsherCarnegie Mellon University

October 16, 2002

OutlineOutline• Background• Overview of capture-recapture • Capture-recapture model assumptions • Multiple systems estimation

• log-linear models• Rasch models

• Example• Ethnic Albanian deaths in Kosovo, March

– June 1999

BackgroundBackground• Used for estimating a population count.

• The size of a wildlife population.• The number of WWW pages.• The number of people in the USA.• The number of human rights violations (civilian

deaths) in Guatemala and Kosovo.• Capture-recapture = dual systems estimation.• Multiple capture-recapture = multiple systems

estimation.

Overview of Overview of Capture-RecaptureCapture-Recapture

Capture 1 Capture 2Overlap

Overview of Overview of Capture-RecaptureCapture-Recapture

N TotalOutIn

TotalOutIn

xxxxxxxxx

01

10001

11011

1 List

2 List

11

11ˆxxx N

Capture-Recapture Capture-Recapture AssumptionsAssumptions

• Independence of lists• Homogeneity of capture probabilities• Error-free matching across lists• No in- or out-migration• No duplicates within a list• Lists are random samples

Multiple Systems Estimation Multiple Systems Estimation for Three Listsfor Three Lists

00010

00000010

01100110

xxxxxxxxx

OutIn

OutIn

1 List

2 ListOut3 List

10111

10001011

11101111

xxxxxxxxx

OutIn

OutIn

1 List

2 ListIn3 List

• Three lists allow for modeling of dependency and/or heterogeneity.

Multiple Systems Estimation Multiple Systems Estimation for Three Listsfor Three Lists

• Three lists allow for modeling of dependency and/or heterogeneity.

• Model for dependency:

where

)(23)(13)(12)(3)(2)(1log jkikijkji uuuuuuu )(mijk

)E( m ijkijk x

Multiple Systems Estimation Multiple Systems Estimation for Three Listsfor Three Lists

• Three lists allow for modeling of dependency and/or heterogeneity.

• Full quasi-symmetry (Rasch) model for heterogeneity:

where

23

1

332211 )log 321

j

jkkk kkkk ( )(

}){ ),( ( cell in count a observing 10Pr 321321 , kkkk jkkk

Multiple Systems Estimation Multiple Systems Estimation for Three Listsfor Three Lists

• Three lists allow for modeling of dependency and/or heterogeneity.

• Full quasi-symmetry (Rasch) model for heterogeneity:

23

1

332211 )log 321

j

jkkk kkkk ( )(

Rasch model enables projection to missing cell via moment constraints (inequality restrictions).

Multiple Systems Estimation Multiple Systems Estimation for More than Three Listsfor More than Three Lists

• Same modeling techniques, more parameters.

• More high-quality lists available means less assumptions are required.

Example: KosovoExample: Kosovo• Analysis required for the trial of former

Yugoslav President Slobodan Milosevic for war crimes allegedly committed in Kosovo.

• Question of interest: Did a systematic campaign by Yugoslav forces lead to Kosovar Albanian deaths and expel Kosovar Albanians from their homes?

Example: KosovoExample: Kosovo• Migration data from two sources; analyzed

using standard demographic techniques.• Ethnic Albanian death data from four

sources; estimates of number of deaths derived via multiple systems estimation.

Kosovo: Data SourcesKosovo: Data Sources• The American Bar Association Central and East

European Law Initiative: 1,674 interviews; 5,089 incidents.

• Exhumations by international teams on behalf of the International Criminal Tribunal for the Former Yugoslavia: 1,767 exhumations.

• Human Rights Watch: 337 interviews; 1,717 incidents.• The Organization for Security and Cooperation in

Europe: 1,837 interviews; one or more incidents each interview.

Kosovo: Data MatchingKosovo: Data Matching

• Duplicates within each list removed.• 6 matches performed; one for each pair of

lists.• Human coders used match-facilitation software.• Each list pair matched 2-4 times by different

coders. • Number of individual deaths (killings where

the victim can be named): 4,400.

Kosovo: Data MatchingKosovo: Data Matching

ABA Yes Yes No NoEXH Yes No Yes No

HRW OSCEYes Yes 27 32 42 123Yes No 18 31 106 306No Yes 181 217 228 936No No 177 845 1,131 ???

Total: 4,400

Kosovo: Death Count EstimatesKosovo: Death Count Estimates

• Estimate of overall number of deaths created from a log-linear model of the four-way cross-classification table: 10,356 (9,002, 12,122).

• Two-day time period estimates of number of deaths created from log-linear models of three-way cross-classification tables; four such cross-classification tables per time period.

Kosovo: AnalysisKosovo: Analysis

Kosovo: AnalysisKosovo: Analysis

• Regression analysis performed using KLA and NATO activity data as independent variables and death/migration estimates as dependent variables.

• The analysis supports the conclusion that a systematic campaign of Yugoslav forces was responsible for ethnic Albanian migrations and deaths in Kosovo between March and June of 1999.

Overall ConclusionsOverall Conclusions

• Where several high-quality pre-existing incomplete lists of adverse events exist, multiple systems estimation is a viable technique for estimating a total count of adverse events.

• Relatively sophisticated technical expertise is required to use this estimation technique well.

Further ReadingFurther Reading

• Ball, P., Betts, W., Scheuren, F., Dudukovich, J., and Asher, J. (2002). Killings and Refugee Flow in Kosovo March - June 1999: A Report to the International Criminal Tribunal for the Former Yugoslavia. American Association for the Advancement of Science, Washington, DC.• Contains a good reference list.• Available on my website: http://www.stat.cmu.edu/

~asher/PAPERS2002/polkilkos_020109.pdf