1

The National Tuberculosis Surveillance The National Tuberculosis Surveillance System Training Program to Ensure System Training Program to Ensure

Accuracy of Tuberculosis DataAccuracy of Tuberculosis Data2010 USPHS Public Health Conference2010 USPHS Public Health Conference

Lilia Manangan, Elvin Magee, Cheryl Tryon

Division of Tuberculosis EliminationCenters for Disease Control and Prevention (CDC)

Presentation OverviewPresentation Overview

• National Tuberculosis Surveillance System National Tuberculosis Surveillance System Training ProgramTraining Program

• Process for Developing, Implementing, and Process for Developing, Implementing, and Evaluating the Training ProgramEvaluating the Training Program

• Systematic Approach to Health EducationSystematic Approach to Health Education• Lessons LearnedLessons Learned• Next StepsNext Steps

National Tuberculosis National Tuberculosis Surveillance SystemSurveillance System

• Report of Verified Case of TB (RVCT) - Report of Verified Case of TB (RVCT) - standardized data collection form for TB standardized data collection form for TB disease surveillance disease surveillance

• Web-based reportingWeb-based reporting

• Data used to create annual TB Data used to create annual TB surveillance report, monitor TB trends, surveillance report, monitor TB trends, and program planning and program planning

Revised Report of Verified Case of Revised Report of Verified Case of Tuberculosis (RVCT)Tuberculosis (RVCT)

44

RVCTRVCT

Page 1 (Items 1 – 16)Page 1 (Items 1 – 16) Page 2 (Items 17 – 25)Page 2 (Items 17 – 25) Page 3 Page 3 (Items 26 – 37)(Items 26 – 37)

Initial Drug Initial Drug Susceptibility Report Susceptibility Report

(Follow-Up Report – 1) (Follow-Up Report – 1) (Items 38 – 40)(Items 38 – 40)

Case Completion Report Case Completion Report

(Follow-Up Report - 2) (Follow-Up Report - 2) (Items 41 – 49) (Items 41 – 49)

55

19931993 20082008

RVCT revision work group conducted compre-hensive review of RVCT

RVCT implement-ed

20012001 20092009

Last major RVCT revision

Continuedcollabor-ation with experts on specific variables

20052005 20072007

Requested comments from TB partners

ReceivedCDC and OMB clearance

RVCT Revision TimelineRVCT Revision Timeline

Developed software

Developed RVCT training materials

ConductedRVCT trainings

66

RVCT Training ProgramRVCT Training Program• Purpose Purpose

– Train health care workers to accurately complete Train health care workers to accurately complete the revised RVCTthe revised RVCT

• Target audienceTarget audienceHealth care workers from state and local healthHealth care workers from state and local health

departments, territories, and Pacific Islands who departments, territories, and Pacific Islands who • Collect data from patients Collect data from patients • Complete RVCT form Complete RVCT form • Enter data from RVCT into data systemEnter data from RVCT into data system• Monitor accuracy of TB program data Monitor accuracy of TB program data

collectioncollection• Analyze data from RVCTAnalyze data from RVCT

77

Create a Training PlanCreate a Training Plan• Provide a detailed description of the projectProvide a detailed description of the project• Share with team membersShare with team members• Revise throughout projectRevise throughout project

Training Plan

• Background• Goal of the Project• Target Audience• Objectives• Project Description• Related Projects

(if appropriate)• Authorization• Primary Team Members

(roles and responsibilities)

• Additional Team Members (reviewers, partners and stakeholders)

• Development Schedule• Training Schedule• Travel Needs• Budget• Evaluation Plan• Clearance (if necessary)

88

Systematic Approach toSystematic Approach toHealth EducationHealth Education

Conduct Needs Assessment

2

Develop & Pretest

Concepts, Messages, & Materials

1

3 Implement the Program

Assess Effectiveness

& Make Refinements

4

99

Why Use the Systematic Process?Why Use the Systematic Process?

Effective and successful health education

materials and trainings

Following these

essential steps

leads to

1010

Goal of Any Training ProgramGoal of Any Training Program

Increase knowledgeIncrease

knowledge

Improveskills

Improveskills

ImproveattitudesImproveattitudes

Change behavior so that performance will improveChange behavior so that performance will improve

Our goal: To increase accuracy Our goal: To increase accuracy of TB surveillance dataof TB surveillance data

Our goal: To increase accuracy Our goal: To increase accuracy of TB surveillance dataof TB surveillance data

1111

Systematic Approach toSystematic Approach toHealth EducationHealth Education

Conduct Needs Assessment

2

Develop & Pretest

Concepts, Messages, & Materials

1

3 Implement the Program

Assess Effectiveness

& Make Refinements

4

1212

Conduct a Needs AssessmentConduct a Needs Assessment

The way things “are”

The way things “ought to be”

A needs assessment is aA needs assessment is a systematic process for systematic process for obtaining and analyzingobtaining and analyzing

information to determine information to determine the gaps betweenthe gaps between

Alwaysdo this

1313

Needs AssessmentNeeds Assessment

• Identified target audience(s) characteristicsIdentified target audience(s) characteristics– Job positionsJob positions

– Knowledge level about the contentKnowledge level about the content

– Skill level needed to perform the tasksSkill level needed to perform the tasks

• Determined what was needed to perform the Determined what was needed to perform the jobjob– Content Content (e.g., information, facts, statistics)(e.g., information, facts, statistics)

– Tasks Tasks (skills such as complete registers/forms, (skills such as complete registers/forms, analyze data) analyze data)

– Resources Resources (e.g., materials, supplies, equipment, (e.g., materials, supplies, equipment, space)space)

1414

Information GatheringInformation Gathering

• Reviewed surveillance dataReviewed surveillance data

• Conducted expert reviewsConducted expert reviews

• Conducted focus groupsConducted focus groups

• Reviewed surveillance Reviewed surveillance training methodstraining methods

• Determined budgetDetermined budget

• Developed justification for Developed justification for trainingtraining

1515

Systematic Approach toSystematic Approach toHealth EducationHealth Education

Conduct Needs Assessment

2

Develop & Pretest

Concepts, Messages, & Materials

1

3 Implement the Program

Assess Effectiveness

& Make Refinements

4

1616

Process for Developing MaterialsProcess for Developing Materials

• Chose Project TeamChose Project Team

• Selected Type(s) of TrainingSelected Type(s) of Training

• Selected Training MethodsSelected Training Methods

• Selected MediaSelected Media

• Determined ResourcesDetermined Resources

• Developed ContentDeveloped Content

• Designed and Produced MaterialsDesigned and Produced Materials

• Pretested MaterialsPretested Materials

Selection of Types of RVCT Selection of Types of RVCT Trainings and Training ActivitiesTrainings and Training Activities

• Based on adult learning principlesBased on adult learning principles

• Best methods for gaining the skills Best methods for gaining the skills

• Needed flexible types of trainings to Needed flexible types of trainings to meet various situationsmeet various situations

1717

1818

3 Basic Learning Styles3 Basic Learning Styles

Learn through

• Watching

• Observing

• Reading

Learn through

• Listening

• Speaking

Learn through

• Moving

• Doing

• Practicing

• Touching

Visual Auditory Tactile

1919

What Adults RememberWhat Adults RememberRead

Hear

See

Hear & See

Say

Say & Do

% of What Adults Remember

Moreparticipation

equalsmore

remembered

For people to learn something, they sometimes have to hear it 7 times.

2020

Types of RVCT TrainingTypes of RVCT Training

RVCT Self-Study ModulesRVCT Self-Study Modules Participants worked at their own Participants worked at their own pace using the print-based modulespace using the print-based modules

Training activitiesTraining activities• Read instructions on how to Read instructions on how to complete each item complete each item• Worked through study questionsWorked through study questions and case studies and case studies

2121

Types of RVCT Training (cont.)Types of RVCT Training (cont.)Facilitated Training SessionsFacilitated Training Sessions

Facilitators led training sessionsFacilitators led training sessions• Conducted with large and small Conducted with large and small groupsgroups• Introduced the content and led Introduced the content and led discussions discussions (not a lecture-based training) (not a lecture-based training)

Training activitiesTraining activities• Worked through the Self-Study Worked through the Self-Study Modules Modules• Discussed content, study Discussed content, study questions, and case studies questions, and case studies

2222

Types of Training Activities (cont.)Types of Training Activities (cont.)Training-of-TrainersTraining-of-Trainers

Training activitiesTraining activities• Facilitators used interactive lectures Facilitators used interactive lectures and exercises on training basics and exercises on training basics• Participants Participants

• Observed training demonstrationsObserved training demonstrations• Practiced teaching the coursePracticed teaching the course• Received feedback Received feedback

Teachback Methodology was Teachback Methodology was used to teach participants how used to teach participants how to teach the facilitated training to teach the facilitated training session to others session to others

2323

Conducted 5 Curriculum Pretests Conducted 5 Curriculum Pretests with Target Audiencewith Target Audience

• Analyzed resultsAnalyzed results– Reviewed comments from participantsReviewed comments from participants

• Remained objective and open to suggestionsRemained objective and open to suggestions• Incorporated appropriate ideasIncorporated appropriate ideas

– Conducted item analysis on all questionsConducted item analysis on all questions• Determined how each question was answered Determined how each question was answered

(by entire group and specific individuals)(by entire group and specific individuals)• Determined which items had problems and Determined which items had problems and

what was causing the problemswhat was causing the problems

• Revised materials as necessaryRevised materials as necessary

This is important

2424

Course ManualsCourse Manuals

RVCT RVCT Self-Study Modules Self-Study Modules

Participant Participant ManualManual

RVCT RVCT Self-Study Modules Self-Study Modules

Facilitator Facilitator ManualManual

2525

Materials on CD ROM Materials on CD ROM or download from FTP siteor download from FTP site

• RVCT Self-Study Modules – Participant ManualRVCT Self-Study Modules – Participant Manual• RVCT Self-Study Modules – Facilitator Manual and RVCT Self-Study Modules – Facilitator Manual and

other training materialsother training materials• RVCT Instruction Manual (instructions only)RVCT Instruction Manual (instructions only)• RVCT Self-Study Modules Exercises (exercises only)RVCT Self-Study Modules Exercises (exercises only)• RVCT Materials DescriptionRVCT Materials Description

ftp://ftp.cdc.gov/pub/Software/TIMS/2009 RVCT Documentation/RVCT Training Materials/

2626

Systematic Approach toSystematic Approach toHealth EducationHealth Education

Conduct Needs Assessment

2

Develop & Pretest

Concepts, Messages, & Materials

1

3 Implement the Program

Assess Effectiveness

& Make Refinements

4

2727

ImplementationImplementation• Collaborated with health departmentsCollaborated with health departments

• Trained volunteer trainersTrained volunteer trainers

• Developed a distribution and marketing plan Developed a distribution and marketing plan for the materialsfor the materials– Reviewed a sample production copy before mass Reviewed a sample production copy before mass

copies were duplicatedcopies were duplicated

• LogisticsLogistics– Determined number of trainings and course Determined number of trainings and course

materials neededmaterials needed

– Monitored travel and budgetMonitored travel and budget

2828

States/Jurisdictions that Have States/Jurisdictions that Have Received RVCT TrainingReceived RVCT Training

Received training

No training

D.C.

• American Samoa• Federated States of

Micronesia• Guam• Northern Mariana Islands

• Palau• Puerto Rico• Republic of Marshall

Islands• Virgin Islands

2929

Number of Health Care Workers Number of Health Care Workers Trained on the RVCT as of 9/30/2009Trained on the RVCT as of 9/30/2009

Training Type # Trained

5 Field Tests5 Field Tests 6464

11 Trainings11 Trainings 233233

Total TrainedTotal Trained 297297

Training Type # Trainers Trained

Training-of-Training-of-TrainersTrainers

8282

3030

Systematic Approach toSystematic Approach toHealth EducationHealth Education

Conduct Needs Assessment

2

Develop & Pretest

Concepts, Messages, & Materials

1

3 Implement the Program

Assess Effectiveness

& Make Refinements

4

3131

Types of EvaluationsTypes of Evaluations

Type When is it conducted

What is measured

Formative During pretests

• Reactions• Learning and performance

Process During implementation

• Procedures• Tasks• What is working

Outcome After implementation

• Immediate or short-term effects• Learning and performance

Impact After implementation

• Long-range results of the training program

• impact on job performance and change in program indicators

3232

Participant Comments about the Participant Comments about the RVCT Training CoursesRVCT Training Courses

• I learned a lot. Very informative.I learned a lot. Very informative.I will now be able to apply the I will now be able to apply the newest revisions to the RVCT.newest revisions to the RVCT.

• It gives me complete information It gives me complete information on how to fill out the RVCT form on how to fill out the RVCT form accurately and concise accurately and concise information to report on all TB information to report on all TB cases in geographic, treatments, adverse effects on cases in geographic, treatments, adverse effects on any medications, and the drug susceptibility and any medications, and the drug susceptibility and

completion of therapy.completion of therapy.

3333

Participant Comments about the Participant Comments about the RVCT Training Courses (cont.)RVCT Training Courses (cont.)

• The course was good because The course was good because it has interactive sessions and it has interactive sessions and all participants had input andall participants had input andcomments. It is very good to comments. It is very good to see the revised/new variables.see the revised/new variables.

• The Instructors are motivating The Instructors are motivating as trainers and ask questions if as trainers and ask questions if we need to clarify all that is discussed.we need to clarify all that is discussed.

• There were some items that I thought I was filling in There were some items that I thought I was filling in correctly, but now I know that I need to change some correctly, but now I know that I need to change some things.things.

Lessons LearnedLessons Learned• Input from the field staff (reviews, field tests, Input from the field staff (reviews, field tests,

trainings) trainings) – Enhanced our training materialsEnhanced our training materials– Motivated people to collect more accurate dataMotivated people to collect more accurate data

• Established strong and valuable Established strong and valuable relationships with field staffrelationships with field staff– Communication opened-up with groups that have Communication opened-up with groups that have

been distant in the pastbeen distant in the past– Encouraged more collaboration and questions to Encouraged more collaboration and questions to

submit accurate datasubmit accurate data

Lessons Learned (cont.)Lessons Learned (cont.)• Use of case studies help apply contentUse of case studies help apply content

– Participants said it was “fun”Participants said it was “fun”

• Use of visuals (e.g., tables, diagrams) to Use of visuals (e.g., tables, diagrams) to make content easy to grasp and refer tomake content easy to grasp and refer to

• Designed manual to be Designed manual to be – User -friendly, easy to locate things for referenceUser -friendly, easy to locate things for reference– Flexible for use in various types of trainings (self-Flexible for use in various types of trainings (self-

study, facilitator-led training sessions)study, facilitator-led training sessions)

Lessons Learned (cont.)Lessons Learned (cont.)• Best as facilitator-led training with Best as facilitator-led training with

discussionsdiscussions• Not appropriate for computer-based courseNot appropriate for computer-based course• Not appropriate for webinar (good for update Not appropriate for webinar (good for update

or reviewing a small number of items)or reviewing a small number of items)

3737

Next StepsNext StepsProvide updates at

national and regional meetings

Evaluate training effectiveness and data accuracy

Develop and implement new training course on quality assurance

Conduct additional RVCT trainings

3838

Thank you