Data Management Seminar, 9-12th July 2007, Hamburg Data Entry Overview.

Data Management Seminar, 9-12th July 2007, Hamburg

Data Entry OverviewData Entry Overview

Data Management Seminar, 9-12th July 2007, Hamburg2

Table of Content

1. Organizing the Data Entry Process

2. Data Manager’s Responsibilities

3. Necessary Material for Data Entry

4. Control Receipts of Survey Material

5. Organize Material for Data Entry

6. Organize Material for Marking

7. The Process of (Reliability) Marking

8. Outlook: Data Entry with WinDEM


Data Documentation

Data

Organizing the Data Entry Process

Instruments/ Forms

InternationalCodebooks

WinW3S File

Data Entry Rules

...

...

DEUnit

Data Manager

3


Data Manager’s Responsibilities

Data EntrySoftware

Source ofInformation

Hardware

Software

Adaptation ofCodebooks

Verification

Documentation

Instruments

TrackingForms

Marking Material

Organize:

Training in: Organize: Perform:

Send Out

MaterialsData Entry

StaffEquipment Data Files

Data Manager

4


From ICCS International Study Center

http://iccs.acer.edu.au/International codebooks (20. August 2007)

- Questionnaire codebooks - Achievement and reliability codebooks

Survey Operations Procedures (Unit 3 FT)

Windows Within-School Sampling Software (WinW3S)

Windows Data Entry Manager (WinDEM) program

IEA Survey System

Necessary Material for Data Entry

5


Important documents for data entry– Survey Operations Procedures (Unit 3 FT)

Copy relevant parts of documents for key punchers

Copies of survey tracking forms should be available for key punchers

Prepare Key Punchers for Data Entry

6


Planning for Required Resources

Staff for marking and data entry depends on…– How much material?

– How much time?

– How much experience?

– How much money?

– How many persons available?

Computer requirements– IBM-compatible (Pentium processor or higher)

– OS: Windows 2000/NT 4.0/XP

– Office 97 or higher

Ask countries that participated in previous IEA studies for their

experiences!

7


Amount of MaterialsMarking: Marker

6 Achievement Booklets =

~ 700Student Achievement Test Booklets

~ 350Student Achievement Test Booklets for Reliability Marking

~ 25 School Questionnaires

~ 250 Teacher Questionnaires

~ 700 Student Questionnaires

~ 700 Regional Module Questionnaires

~ 700 Student Achievement Test Booklets

~ 350 Reliability Marking Sheets

Data Entry: Key Punchers

8


Rece

ivin

g M

ate

rial

InstrumentPreparation

Test

Adm

inis

trati

on

Codebook Adaptation

Rece

ivin

g M

ate

rial

Data

Su

bm

issi

on

Sampling and Preparing

Test AdministrationMarking

Data Entryand Verification

9


Rece

ivin

g M

ate

rial

10


Control Receipts of Test Material

Record return date on ICCS FT School Sampling Form

Contact schools that have not returned testing material or where the material is incomplete!

11


Control Receipts of Test MaterialCheck that all survey tracking forms have been returned and completed correctly

...

...Check participation statusparticipation status on tracking forms!

Contact schools that have not returned tracking forms or returned incomplete!

12


Control Receipts of Test MaterialCheck that all booklets/ questionnaires have been returned

Verify identification codesidentification codes on instruments!

Contact schools that have not returned all testing instruments!

13

European ModuleStudent Questionnaire

Teacher QuestionnaireSchool Questionnaire

Student Test Booklet


OrganizeMaterial

for Data Entry

Rece

ivin

g M

ate

rial

14


Separate questionnaires and tracking forms from booklets!

Sort and forward…

Organize Material for Data Entry and Marking

Forms + Questionnaires to

Data Entry Staff

Booklets to

Markers

...

...

15


Forms + Questionnaires to

Data Entry Staff

...

Sorting Forms and Questionnaires

16


8 0 0 1 0 1 0 1

8 0 0 1

8 0 0 1 0 1

8 0 0 1 0 1 8 0 0 1

0 1 0 1

0 1

8 0 0 1 0 1

8 0 0 1

Hierarchical ID SystemIDSCHOOL: 4 digits school

IDCLASS: 6 digits school + class

IDSTUD: 8 digits school + class + student

IDTEACH: 6 digits school + teacher

Unique identifier in ICG file

Unique identifier in xSG/xSE/xSL, xSA, xSR file

Unique identifier in ITG file

8 0 0 1

17


Arrange Instruments for Data Entry

School Questionnaires

Sort by IDSCHOOL

...

18


Teacher Questionnaires

Sort by IDTEACH

...


19


...

IDSCHOOL 8001

IDCLASS 800101

...

IDSCHOOL 8001

IDCLASS 800102

...

IDSCHOOL 8002

IDCLASS 800201

...

IDSCHOOL 8002

IDCLASS 800202

...

Sort by IDSTUD

Student Questionnaires


20


...

IDSCHOOL 8001

IDCLASS 800101

...

IDSCHOOL 8001

IDCLASS 800102

...

IDSCHOOL 8002

IDCLASS 800201

...

IDSCHOOL 8002

IDCLASS 800202

...

Sort by IDSTUD

Regional Module Student Questionnaires


21


OrganizeMaterial

for Data Entry

OrganizeMaterial

for Marking

Rece

ivin

g M

ate

rial

22


Sorting Booklets for MarkingTwo Teams for Marking: A and B

Booklets to

Markers

Marking Team A

Marking Team B

23


...

1,2,...,5,6

Steps for Booklet Arrangement

Step 1: Sort by Booklet Type

24


Arrange Test Booklets for Marking


Booklet

1 R

Booklet

1 R

Booklet

1 R

Booklet

1 R

Booklet

2

Booklet

2

Booklet

2

Booklet

2 R

~120 ~120 ~120

~ 700

~120

Booklet

1

Booklet

1

Booklet

1

Booklet

1 R

Booklet

2

Booklet

2

Booklet

2

Booklet

2 R

...

...

...

25



Step 2: Divide into Two Sets (only Booklets 1-3) ...

Set A (IDSCHOOL is an odd number),

Set B (IDSCHOOL is an even number)

...


26

Note: From now on the example for Reliability Marking will focus on

booklet 1 only!

1,2,...,6



Step 2: Divide into Two Sets (only Booklets 1-3)

Booklet

1

Booklet

1

Booklet

1

Booklet

1 R

~60

~60

Booklet

1 R

Set A

Booklet

1 R

Set A

Booklet

1 R

Set A

Booklet

1 R

Set A

Booklet

1

Set B

Booklet

1

Set B

Booklet

1

Set B

Booklet

1 R

Set B

~120

Booklet

1

Booklet

1

Booklet

1

Booklet

1 R

Marking Team A

Marking Team B

50%

50%

27



Step 3: Divide into Batches

Step 2: Divide into Two Sets

Step 1: Sort by Booklet Type ...

1,2,...,6

...

Set A, Set B

...

Batch 1, Batch 2, ...

28




60

Booklet

1R

Set A

Booklet

1

Set A

Booklet

1R

Set A

Booklet

1 R

Set A~ 15

Booklet

1R

Set A

Booklet

1 R

Set A

Marking Team A

~ 15

Booklet

1R

Set A

Booklet

1 R

Set A

~ 15

Booklet

1R

Set A

Booklet

1 R

Set A

~ 15

Booklet

1R

Set A

Booklet

1 R

Set A

~ 15

Booklet

1R

Set B

Booklet

1 R

Set B

~ 15

Booklet

1R

Set B

Booklet

1 R

Set B

60

Booklet

1R

Set B

Booklet

1

Set B

Booklet

1R

Set B

Booklet

1 R

Set B

Marking Team B

~ 15

Booklet

1R

Set B

Booklet

1 R

Set B

~ 15

Booklet

1R

Set B

Booklet

1 R

Set B

29






1,2,...,6

...

Set A, Set B

...


30



Test

Adm

inis

trati

on

Codebook Adaptation

Rece

ivin

g M

ate

rial

Data

Su

bm

issi

on


MarkingSampling

and PreparingTest Administration

31


Marking


Test

Adm

inis

trati

on

Codebook Adaptation

Rece

ivin

g M

ate

rial

MarkingSampling


32

Data

Su

bm

issi

on



ReliabilityMarking

MainMarking

Marking

33


Step 5: Reliability Marking



Step 4: Main Marking (Booklets 4-6)

Items

Marking

Sheet

Step 6: Return “R” Booklets

Step 7: Main Marking (Booklets 1-3)



1,2,...,6

...

Set A, Set B

...


34


Main MarkingStep 4: Main Marking (Booklets 4-6)

Booklet

4

101

102

201

202

35

Booklet

5

Booklet

6

Booklet

4

Booklet

4

Booklet

5

Booklet

5

Booklet

6

Booklet

6

~ 120 ~ 120 ~ 120

Note: Booklets 4-6 can be marked by any marker!

But: Single items should not all be marked by the same marker.


Reliability MarkingStep 5: Reliability Marking

Marking Team A

Booklet

1R

Set A

Items

Marking

Sheet

Marking Team B

~ 8

201

Booklet

1R

Set A

Items

Marking

Sheet

~ 8

202

Booklet

1R

Set B

Booklet

1R

Set B

Items

Marking

Sheet

Items

Marking

Sheet

~ 8

~ 8

101

102

36


Documenting Reliability Marker

101

Marking Team A

37

101

Markes ONLY on Constructed Response Items Marking Sheet

even number

Booklet

1R

Set B


Return „R“ Booklets for Main Marking

Step 6: Return “R” Booklets

Booklet

1R

Set A

~ 8

Marking Team A

Marking Team B

~ 8

Booklet

1

Set A

38

~ 8

Booklet

1

Set A

~ 8

Booklet

1R

Set A

Booklet

1R

Set A

~ 8

Booklet

1R

Set A

~ 8Booklet

1R

Set B

~ 8

Booklet

1R

Set B

~ 8

Booklet

1R

Set B

~ 8

Booklet

1R

Set B

~ 8

Booklet

1R

Set B

~ 8

Booklet

1R

Set B

~ 8


Main MarkingStep 7: Main Marking (Booklets 1-3)

~ 8

~ 8

~ 8

~ 8

Booklet

1R

Set A

Booklet

1R

Set A

Booklet

1R

Set B

Booklet

1R

Set B

Marking Team A

Marking Team B

201

202

101

102

39


Documenting Main Marker

Marking Team B

202

Booklet

1R

Set B

201

40

even number

101



Test

Adm

inis

trati

on

Codebook Adaptation

Rece

ivin

g M

ate

rial

MarkingSampling


41

Data

Su

bm

issi

on






Test

Adm

inis

trati

on

Codebook Adaptation

Rece

ivin

g M

ate

rial

MarkingSampling


42

Data

Su

bm

issi

on


CompleteStudent

Information

EnterQuestionnaire Data

EnterAchievement and

Reliability Data


43


WinDEM – New Data FileCreate New Data file (For the First time) From a codebook

44


WinDEM – New Data File

Extension *.sdb

Codebook (*.sdb)

Data File (*.dbf)

45


WinDEM – New Data File New Data file File READY for: Data Entry

46

Data Management Seminar, 9-12th July 2007, Hamburg Data Entry Overview.

Documents

Transcript of Data Management Seminar, 9-12th July 2007, Hamburg Data Entry Overview.