School Health Policies and Programs Study (SHPPS) Data ...School Health Policies and Programs Study...
35
Macro International Inc. 7315 Wisconsin Avenue, Suite 400W, Bethesda, MD 20814 (301) 657-3070 | www.orcmacro.com School Health Policies and Programs Study (SHPPS) Data Collection Management System Ben Harper Tonja Kyle
Transcript of School Health Policies and Programs Study (SHPPS) Data ...School Health Policies and Programs Study...
Macro International Inc.
7315 Wisconsin Avenue, Suite 400W, Bethesda, MD 20814
(301) 657-3070 | www.orcmacro.com
School Health Policies and Programs Study
(SHPPS) Data Collection Management System
Ben Harper Tonja Kyle
(SHPPS) Data Collection Management System
The data collection management system was used to support a collection with:
• 18,200 respondents
• 68 data collectors in the field
• 1,100 site visits
• 50 States
Planning efficiency was a top priority
Each data collector needed to visit:
• 2 schools each week
• 2 days at each school
• 9+ data collections per school
(SHPPS) Data Collection Management System
Will Be Discussing:
• The data collection
• The system designed to support it
• Features developed to insure efficiency
• Lessons learned
Study Overview
(SHPPS) Data Collection Management System
Funded by the Centers for Disease Control and Prevention (CDC), Division of Adolescent and School Health (DASH)
Also funds/supports other surveillance studies:
• Youth Risk Behavior Survey (YRBS)
• State-level YRBS studies
(SHPPS) Data Collection Management System
Assessed eight components of school health programs:
• Health education
• Physical education
• Health services
• Mental health and social services
• School policy and environment
• Food service
• Faculty and staff health promotion
• Family and community involvement
(SHPPS) Data Collection Management System
History
• First conducted in 1994, the study is conducted every 6 years
• Interviews conducted at the elementary, middle/junior and senior high school levels
• District- and state-level interviews conducted using Computer Assisted Telephone Interviewing (CATI)
• School- and classroom-level data collected via Computer Assisted Personal Interviewing (CAPI)
• SHPPS 2006 targeted all 50 States and DC, 819 districts and 1,420 schools
(SHPPS) Data Collection Management System
Study Challenges
• Recruit and track contacts and respondents
from 50 States, 819 districts and 1,420 schools
times 7
• Generate tailored letters with individualized cc:
lists and inserts for contacts and respondents
• Manage participation and respondent status
• Schedule 1,420 schools for data collection to
occur in a 12-week window
(SHPPS) Data Collection Management System
Study Challenges (continued)
• Locate schools in the sample that are clustered together for a more timely and cost effective approach to data collection
• Randomly select courses and teachers from within each sampled school
• Manage 68 data collectors and their CAPI workstations
• Manage uploading of completed CAPI records
• Resolve data errors
The Collection Management System
Logical layout
QuickSAT
CAPI Tool
Macro
CATI System
Respondents
Contact Permission
Collection Contacts
State
Respondents
Contact Permission
Collection Contacts
District
Availability
Class Respondents
School Respondents
Contact Permission
Collection Contacts
School
Contact Management Letter Generation Reporting
Data Collector Assignment
Trip Planning
School Scheduling Nearest Neighbor
Calendar
SHPPS Collection Management System
The SHPPS Management System
The Collection Management System…
As Told by The Interface Designer
SHPPS: Designing for the User You Know
Recruiters were well represented during the
design process.
Their workflow was well documented and
understood.
The Contacts database designed for them
performed well, and the design has been
carried forward into newer systems.
School Information
Collection Contacts
Participation (State)
Cc: Lists
State/District/School Respondents
SHPPS: Designing for the User You Don’t Know
Letters requesting participation in the data collection were sent to states, districts, schools, and participants.
Thousands of letters.
The letters merged information from the contact database for both their content and their cc: listings.
However, the workflow for letter generation was not entirely known at design time, and we never had the opportunity to speak to the users who would be performing that task.
Letter Generation
SHPPS: Taking Advantage of the technology
With data collectors traveling across the country it was important to align their schedules with the geography.
Schools that were ready for visits were placed on the calendar.
Groups of nearby schools could be put into a trip which could be assigned to a data collector.
What do you do when an isolated school needs to be visited? Look for other schools in the same county? The same State?
Finding Nearby Schools
SHPPS: Feature Rich Calendar
The Calendar interface combined the needs of the
school, the wishes of the data collector, and the
limitations of linear time into one interface.
The task it supported was very complex, so the
interface provided some tools to reduce
complexity.
With training the calendar was effective for its
intended task, and supported workflow that kept
the data collection moving forward.
Calendar
Adding Schools to the Calendar
Visit Schedule
SHPPS: “Problem Children”
If you think a design is trouble when you design it,
you’re probably right.
The system supported random sampling of Health
and PE instructors, classes and sections.
By answering a number of questions in a set order,
recruiters could be sure that the sample selected
was not biased.
Errors on the other hand, were not accounted for.
Classroom Sampling (1/3)
Classroom Sampling (2/3)
Classroom Sampling (3/3)
SHPPS: Monitoring Data Submission Integrity
Data collectors in the field submitted
records nightly, allowing for potential
errors.
A submission monitoring system flagged
potential trouble before data was added to
the database.
Data Error Resolution
Macro International Inc.
7315 Wisconsin Avenue, Suite 400W, Bethesda, MD 20814
(301) 657-3070 | www.orcmacro.com
(SHPPS) Data Collection Management System
Ben Harper - [email protected]
Tonja Kyle - [email protected]
Questions?
Presentation Supported by…
Macro International Establishment Survey Research Center
• A virtual clearinghouse for research on procedures and practices in conducting
surveys of establishments
• Based in Macro International’s Bethesda offices but drawing on expertise from a
number of University and Corporate entities
• Web-portal to provide links to:
existing research
conferences
collections of original research and white papers
Other data sources
www.ESRConline.com (Coming Summer 2007)
