ICCS 2009 4 th NRC Meeting, February 15 th - 18 th 2010, Madrid 1 Sample Participation and Sampling...

ICCS 2009 4th NRC Meeting, February 15th - 18th 2010, Madrid1

Sample Participation and Sampling Weights

ICCS 2009 4th NRC Meeting, February 15th - 18th 2010, Madrid

Overview

Progress reportParticipation in ICCSWeights– What are weights?– Why use weights?– What weights are there in ICCS?– How were ICCS weights calculated?


Progress report

38 educational systems participated in ICCSSampling weights were calculated for all of them– For the student study and the teacher study– For six additional grades

Variables for variance estimation were preparedParticipation rates were calculated


Participation categories

Three categories for sampling participation were defined in ICCS for students and for teachers– Category 1: acceptable participation rate without

the use of replacement schools– Category 2: acceptable participation rate when

replacement schools are included– Category 3: participation rate not acceptable even

when replacement schools are included

Each country was grouped into a category during sampling adjudicationEach country was informed about the decision


Participation categories - students

Category 1: 28 countriesCategory 2: 8 countriesCategory 3: 2 countries


Participation categories - teachers

Category 1: 22 countriesCategory 2: 5 countriesCategory 3: 9 countriesTwo countries could not be reported


Weights. They are everywhere.

Weights are in your dataWeights are in the tables of the reports– All means and percentages in the reports

will be calculated with weighted data


What are weights?

Weights are values that are assigned to every sampling unitThe weight of a sampled unit indicates the population that is represented by this sampled unit

2200


Why do we need weights?

Weights allow drawing conclusions about the population based on information from the sampleWeights allow unbiased estimates of population parametersUn-weighted data only allow conclusions about the sampled units


Population sample

Example: in the population, 20% of the students are in private schools, 80% are in public schoolsThe NRC decides to over-sample students from private schoolsIn the sample, 50% of the students are from private schools, 50% are from public schools


Sample estimate

To estimate the correct proportion of students in the population, we must assign different weights to the students in the sample


What weights are there in ICCS?

Student weights– TOTWGTS

Teacher weights– TOTWGTT

School weights– TOTWGTC


Why do ICCS weights differ?

ICCS student weights differ, because the following five elements differ: Selection probabilities– 1) ... of the sampled schools– 2) ... of the classes within the schools

Non-participation– 3) ... of sampled schools– 4) ... of sampled classes within the schools– 5) ... of students within the classes

Each element corresponds to a weight factor


1) School Base Weight

WGTFAC1The school base weight is the inverse of the selection probability of the school– Low selection probability large weight – High selection probability small weight

5

1


1) School Base Weight: PPS

Schools were sampled with probability proportional to size (PPS)Large schools had larger selection probabilitiesTherefore, they now have smaller school base weights

Example: the large school is three times largerThe small school has a three times larger school base weight

w w x 3


2) Class Base Weight

WGTFAC2SThe class base weight is the inverse of the selection probability of the classDifferent selection probabilities because of the different number of classes in a school

6

2


3) School Weight Adjustment

WGTADJ1Adjusts for schools that did not participateExample: Schools from two different explicit strata – one blue school does not participate

5/4


4) Class Weight Adjustment

WGTADJ2SAdjusts for classes that did not participateIn most countries, only one class was selected per schoolAlmost always, this factor equals 1


5) Student Weight Adjustment

WGTADJ3SAdjusts for students that did not participateExample: Students from two different classes – one blue student does not participate

5/4


The final student weight

We now can compute the TOTAL STUDENT WEIGHT

TOTWGTS = x School Base Weightx Class Base Weightx School Adjustment Factorx Class Adjustment Factorx Student Adjustment Factor


Teacher weights

The TOTAL TEACHER WEIGHT is computed similarlyTOTWGTT =

x School Base Weightx Teacher Base Weightx School Adjustment Factorx Teacher Adjustment Factorx Teacher Multiplicity Factor

– adjusts for the fact that some teachers teach in more than one school


More about weights:

IDB Training on Thursday– how weights are used for data analysis – some examples of what can go wrong if

weights are not used for data analysis

Please join!


Country sessions

If you have any questions about participation rates or weights...Please sign up!


Thank you for your attention!

ICCS 2009 4 th NRC Meeting, February 15 th - 18 th 2010, Madrid 1 Sample Participation and Sampling...

Documents

Transcript of ICCS 2009 4 th NRC Meeting, February 15 th - 18 th 2010, Madrid 1 Sample Participation and Sampling...