March 2011 Hcahps Introduction Training Slides Session I 2 28 2011

Introduction to HCAHPS Survey Training

March 2011

2


March 2011

Welcome!In the HCAHPS training sessions, we will:

• Explain purpose and use of HCAHPS survey

• Provide instruction on managing the survey

• Discuss modes of survey administration

• Instruct on sampling, data preparation, data submission and public reporting

3


March 2011

Session I

4


March 2011

Background and Development of the

HCAHPS Survey

5


March 2011

Overview of Presentation

Background & Development of HCAHPS

Composition of the Survey

Roles and Responsibilities

How to Join HCAHPS in 2011

6


March 2011

The Name of the Survey• Formal name: CAHPS® Hospital Survey

• Also known as Hospital CAHPS® or

HCAHPSPronounced “H-caps”

CAHPS® is a registered trademark of the Agency for Healthcare Research and Quality, a U.S. Government agency.

7


March 2011

The Method of HCAHPS• Ask patients (survey)

• Collect in standardized, consistent manner

• Analyze and adjust data

• Publicly report hospital results

• Use to improve hospital quality of care

8


March 2011

Upcoming for HCAHPS~April 21, 2011 13th HCAHPS public reporting; July ’09 -

June ‘10 discharges; ~3,800 hospitals

April 13 Submission deadline for 4th quarter 2010

April 26 – May 26 Preview Period for July public reporting

July 1 Release 3.3 effective for discharges

~ July 21 14th public reporting of HCAHPS results

~ October 20 15th public reporting of HCAHPS results

~ January 2012 16th public reporting of HCAHPS results

9


March 2011

Objectives of HCAHPS

• Standardized survey to allow meaningful comparisons across hospitals for public reporting

• Increased hospital accountability and incentives for quality improvement

• Enhanced public accountability

10


March 2011

HCAHPS and the HQA• Implementation through national

Hospital Quality Alliance (HQA)

– Public-private partnership on hospital quality reporting

– Members include: AHA, FAH, AAMC, JCAHO, AMA, ANA, AARP, AFL-CIO, AHRQ, & CMS

• Currently reporting heart attack, heart failure, pneumonia care, surgical infection, mortality, children’s asthma, and readmission rates

11


March 2011

CAHPS Family of SurveysConsumer Assessment of Healthcare P roviders & Systems:

- HCAHPS- Home Health CAHPS- Medicare Health Plan CAHPS- Prescription Drug Plan CAHPS- Clinician & Group CAHPS- ESRD CAHPS- Nursing Home CAHPS- Dental CAHPS

12


March 2011

HCAHPS 101• Short-term, acute care hospitals

– “General Hospitals” (AHA)• Excludes pediatric, psychiatric and specialty hospitals

• Eligible patients– Adult– Medical, surgical or maternity care– Overnight stay, or longer– Alive at discharge – Excludes hospice discharge, prisoner, foreign address, “no-publicity” patients,

& patients discharged to nursing homes and skilled nursing facilities• HCAHPS encompasses - 85% of inpatients

• Survey after discharge – Four modes of survey administration – Standardized data collection, submission, analysis and public reporting– Self-administer, use a survey vendor or administer for other hospitals

13


March 2011

Survey Mode

Second quarter 2010 hospitals (3,892):

• Mail: 2,621 hospitals; 67%

• Telephone: 1,241 hospitals; 32%

• Mixed: 9 hospitals; 0.2%

• IVR: 21 hospitals; 1%

14


March 2011

Participation in HCAHPSThird quarter 2010:

• 41 Approved survey vendors

• 88 Self-administering hospitals

• 2 Multi-site hospitals

15


March 2011

Serving Multiple Stakeholders

• Multiple stakeholders and viewpoints

• Accommodated to the extent possible– While adhering to goals of HCAHPS

• Evolving scope of HCAHPS– Part of pay-for-reporting for IPPS hospitals– New languages added for mail mode– Proposed for inclusion in CMS Hospital Value-Based

Purchasing (VBP) pay-for-performance program

16


March 2011

HCAHPS and Hospital VBP: PROPOSED

• Baseline Period: Patients discharged July 1, 2009 through March 31, 2010

• Performance Period: Patients discharged July 1, 2011 through March 31, 2012

• Program begins in Fiscal Year 2013 – Patients discharged October 1, 2012 and

forward

17


March 2011

Composition of SurveyHCAHPS contains 27 items:

• Items 1-22: Core of HCAHPS– Put first; do not alter; keep together

• 18 substantive questions • 4 “screener” items

• Items 23-27: Demographic (“About You”)– Place later; keep together; do not alter

18


March 2011

HCAHPS Six CompositesWhat patients/consumers want to know:1. Communication with nurses2. Communication with doctors 3. Responsiveness of hospital staff 4. Pain management5. Communication about medicines6. Discharge information

19


March 2011

HCAHPS Individual ItemsWhat patients/consumers want to know:

1. Cleanliness of hospital environment

2. Quietness of hospital environment

20


March 2011

HCAHPS Global Items

1. ‘Overall rating of hospital’– 0 to 10 scale

2. ‘Recommend this hospital’– Four point scale

21


March 2011

HCAHPS Core Items (1-22)Core HCAHPS items form module:

• Can be followed by hospital-specific items in an integrated format

— or —

• Entire HCAHPS can be used as stand-alone questionnaire

22


March 2011

HCAHPS Public Reporting

• Only official HCAHPS items are to CMS submitted and publicly reported

• Ten hospital-level measures that summarize responses to HCAHPS items– All patient data is de-identified– On Hospital Compare Web site, updated

quarterly

• Only non-IPPS hospitals may suppress

23


March 2011

Public Reporting PeriodsReporting is based on 12 months of discharges

HCAHPS PUBLIC REPORTING: April 2011

• QUARTERS INCLUDED: 3Q09, 4Q09, 1Q10, 2Q10

• PREVIEW PERIOD: January – February 2011

• PUBLIC REPORTING: April 21, 2011

• NEW Public Reporting dates: January, April, July, & Oct.

24


March 2011

HCAHPS On-Line Web sitewww.hcahpsonline.org

Information available:• State and national summary table

• HCAHPS “Top-box” and “Bottom-box” percentiles for each HCAHPS measure

7

• Patient-level Pearson correlations of HCAHPS measures

• HCAHPS Hospital Characteristics Comparison Charts

• Bibliography of published HCAHPS research conducted by the HCAHPS Project Team

http://www.hcahpsonline.org/�





25


March 2011

Roles and ResponsibilitiesHospitals using a survey vendor

• The Vendor’s role in data collection and submission:– Receive or develop sample frame of eligible discharges– Draw sample of discharges and administer survey– Submit HCAHPS data in standard format via My QualityNet– Monitor submission reports– Comply w ith oversight process– Conduct ongoing quality assurance activities– Monitor HCAHPS Web site for updates

26


March 2011

Roles and Responsibilities (cont’d)Hospitals using a survey vendor

• The Hospital’s role in data collection and submission:

– Submit entire discharge list to survey vendor, or develop sample frame of eligible discharges

– In a timely manner– Monitor submission and feedback reports– Comply w ith oversight process– Monitor HCAHPS Web site for updates

27


March 2011

Roles and Responsibilities (cont’d)Self-administering Hospitals

• Develop sampling frame of eligible discharges• Draw sample of discharges and administer survey• Submit HCAHPS data in standard format via

My QualityNet • Monitor submission and feedback reports• Comply w ith oversight process• Conduct ongoing quality assurance activities• Monitor HCAHPS Web site for updates

28


March 2011

Roles and ResponsibilitiesAll Hospitals

• Comply with all HCAHPS survey protocols– HCAHPS Quality Assurance Guidelines V6.0

• Provide patient discharge list in timely manner– Allow sampling and surveying within contact window

• Provide all administrative data in timely manner

• Do not influence patients about HCAHPS survey– Communication with patients– Concurrent surveys

29


March 2011

Roles and Responsibilities (cont’d)

Government: Support, Report & Oversee• Provide training and technical assistance

• Accumulate data

• Calculate and publicly report results

• Analyze results

• Oversee all survey processes

• Issue HCAHPS Bulletins, as needed

30


March 2011

Advertising Guidelines• The Hospital Compare Web site is the

official source of HCAHPS results

• CMS does not endorse hospitals or survey vendors

• Hospital Compare is designed to provide objective information to help consumers make informed decisions about health care providers

31


March 2011

Steps to Join HCAHPS in 20111. Submit an HCAHPS Participation Form

• For self-administering hospitals, hospitals administering survey for multiple sites and survey vendors

• Form available online, March 4, 2011

2. Do an HCAHPS Dry Run• Voluntary, but strongly suggested• Last month in calendar quarter • Contact HCAHPS Project Team for details

− [email protected]

3. Collect and submit HCAHPS survey data on a continuous basis

mailto:[email protected]�

32


March 2011

More information on HCAHPS• Registration, applications, background information,

reports, and HCAHPS Executive Insight :www.hcahpsonline.org

• Submitting HCAHPS data:www.qualitynet.org

• Publicly reported HCAHPS results:www.hospitalcompare.hhs.gov


http://www.qualitynet.org/�

http://www.hospitalcompare.hhs.gov/�

33


March 2011

Questions?

34


March 2011

Participation and Program Requirements

35


March 2011

Participation Overview• HCAHPS Web site and Technical Support• Rules of Participation

– Step 1: Introduction to HCAHPS Survey Training– Step 2: Program Participation Form and Teleconference– Step 3: My QualityNet Registration– Step 4: Data Collection– Step 5: Participate in Oversight Activities– Step 6: Public Reporting– Step 7: Update Training

• Minimum Requirements

36


March 2011

HCAHPS Web site and Technical Support

www.hcahpsonline.org• Official web site for content, announcements, HCAHPS

Bulletins, updates, reminders• Monitor weekly for “What’s New”• Quick links to Current News, Background, Participation,

etc.


37


March 2011

HCAHPS Web site Home Page• Quick links:

Current News | HCAHPS and IPPS Payment Provisions | Background | About the Survey | Participation | For More Information | To Provide Comments or Questions| Internet CitationCurrent News

• Medicare Proposes New Hospital Value Based Purchasing Program• Patient-mix Coefficients for March 2011 HCAHPS Results Have

Been Posted• Hospital Compare Has Been Refreshed• December 2010 Update...HCAHPS Executive Insight Click on

Gold Navigation Button on the left side of this screen • New Tables on Summary Analyses Page• 2011 and 2012 HCAHPS Data Submission Deadlines

http://www.hcahpsonline.com/whatsnew.aspx�






http://www.hcahpsonline.com/default.aspx�

http://www.hcahpsonline.com/executive_insight/default.aspx�


http://www.hcahpsonline.com/facts.aspx�

http://www.hcahpsonline.com/modeadjustment.aspx�

http://www.hcahpsonline.com/forms/exception_discrepancy.aspx�

http://www.hcahpsonline.com/app_vendor.aspx�

http://www.hcahpsonline.com/qaguidelines.aspx�

http://www.hcahpsonline.com/trainingmaterials.aspx�

http://www.hcahpsonline.com/techspecs.aspx�

http://www.hcahpsonline.com/surveyinstrument.aspx�

http://www.hcahpsonline.com/contactslinks.aspx�

http://www.hcahpsonline.com/SummaryAnalyses.aspx�

38


March 2011

HCAHPS Technical Support• Email: [email protected]− Contact information− Hospital name− Hospital 6 digit CMS Certification Number (CCN)

• Telephone: 1-888-884-4007− Contact information− Hospital name− Hospital 6 digit CMS Certification Number (CCN)


39


March 2011

Step 1: Introduction to HCAHPS Survey Training

• Who is required to attend?– Survey Vendors– Hospitals conducting HCAHPS for multiple sites– Hospitals self-administering HCAHPS– Subcontractors (or HCAHPS Update training)– New project managers

• Who is recommended to attend?– New staff assigned to work on HCAHPS administration– Hospitals contracting with a survey vendor or another

hospital for survey administration – Quality Improvement Organizations (QIOs)

40


March 2011

Step 2: Program Participation Form and Teleconference

• Available online at www.hcahpsonline.org– Participation Forms available March 4, 2011 through

May 31, 2011

• Who needs to submit a Participation Form?– Hospitals self-administering HCAHPS– Hospitals conducting HCAHPS for multiple sites– Survey vendors (administering on behalf of hospitals)– Not required for hospitals contracting with survey

vendor


41


March 2011

Step 2: Program Participation Form and Teleconference (cont’d)• Participation Form must be completed in its

entirety– Additional explanations must be provided if applicable– Staff assigned as key HCAHPS project staff must be

identified

• Submit Participation Form– Agreement to comply with all HCAHPS Protocols

42


March 2011

Step 2: Program Participation Form and Teleconference (cont’d)

• Teleconference– Key staff must be available to participate in a

teleconference to discuss relevant survey experience, organizational survey capability and capacity, as part of the Participation Form review process

43


March 2011

Step 3: My QualityNet Exchange • Contact:

– State QIO (hospitals)– HCAHPS Information and Technical Support

(survey vendors)

• If already registered with QualityNet, register specifically for HCAHPS and obtain necessary roles– Contact QualityNet Help Desk for questions on how to

complete the forms• [email protected]

44


March 2011

Step 4: Data Collection• Hospitals/Survey vendors will:

– Adhere to the HCAHPS Quality Assurance Guidelines V6.0 (QAG V6.0)

– Submit an Exceptions Request Form for approval for requesting variations to HCAHPS protocols

– Review the compliance and the accuracy of their data collection processes

– Alert HCAHPS Project Team to any discrepancies occurring during survey administration and submit a Discrepancy Report online via the HCAHPS Web site

– Submit data by HCAHPS data submission deadline

45


March 2011

Step 4: Data Collection (cont’d)

• Dry run – Participation in a dry run is voluntary

• Strongly suggested• Last month in calendar quarter• Contact the HCAHPS Project Team to provide

notification of participation in a dry run– [email protected]

• Dry run data are not publicly reported


46


March 2011

Step 5: Participate in Oversight Activities

• Submit HCAHPS Quality Assurance Plan

• Submit additional information as requested

• Comply with on-site visit requests

• Comply with conference call requests

• Implement corrective action(s), as necessary

47


March 2011

Step 6: Public Reporting• HCAHPS results will be publicly reported on a

quarterly basis on Hospital Compare Web site

• The appropriate pledges must be signed and on file– Contact your state QIO for more details

48


March 2011

Step 7: Future Update Trainings• As scheduled by CMS

• Details to be posted on www.hcahpsonline.org

• Required for all approved survey vendors, hospitals conducting survey for multiple sites, self-administering hospitals, and subcontractors

• Recommended for hospitals using a survey vendor

• Recommended for QIOs

49


March 2011

Minimum Requirements

1. Relevant survey experience– Demonstrated recent experience in

fielding surveys using requested mode(s) of administration• Survey experience• Number of years in business• Number of years conducting surveys

50


March 2011

Minimum Requirements (cont’d)

2. Organizational survey capacity– Capability and capacity to handle a required

volume of surveys and conduct surveys in specified time frame• Personnel (no volunteers are permitted)• System resources• Survey administration• Data submission (cannot be subcontracted)• Technical assistance/customer support

51


March 2011

Minimum Requirements (cont’d)

3. Quality control procedures– In-house training for staff and subcontractors

involved in survey operations– Quality control activities

• Documentation of these activities• Discussion of these activities

52


March 2011

Questions?

53


March 2011

Sampling Protocol

54


March 2011

Overview

• Steps of Sampling Process• Methods of Sampling• Quality Control for Sampling• Sampling Facts

55


March 2011

Steps of Sampling Process

A. Population (All Patient Discharges)B. Identify Eligible PatientsC. Remove ExclusionsD. De-Duplication ProcessE. HCAHPS Sample FrameF. Draw Sample

See Quality Assurance Guidelines V6.0, HCAHPS Sampling Protocol Illustration

56


March 2011

Step A: Population(All Patient Discharges)

57


March 2011

Step A: Population (cont’d)

• Patients of all payer types are eligible for sampling

• Hospitals contracting with survey vendors are strongly encouraged to provide entire patient discharge lists (excluding no-publicity patients and patients excluded because of state regulations) to their survey vendor

58


March 2011

All Eligible Patients

•18 years or older at the time of admission

•Admission includes at least one overnight stay in hospital

•Non-psychiatric MS-DRG/principal diagnosis at discharge

•Alive at the time of discharge

Ineligible Patients• Record count of ineligible patients

Step B: Identify Eligible Patients

59


March 2011

Step B: Identify Eligible Patients (cont’d)

• Adult Inpatients – 18 years or older• Hospital Admission – minimum one

overnight stay, or longer• Non-Psychiatric MS-DRG code/principal

diagnosis at discharge• Alive at discharge

60


March 2011

Determination of Service Lines• Use the principal discharge MS-DRG code to…

– Identify the eligible patients – Classify into the Service Line as either:

• Medical • Surgical• Maternity Care

• Hospital without Surgical or Maternity Care departments may participate


61


March 2011

• V.28 MS-DRG codes effective October 1, 2010– To classify into Medical and Surgical service lines

• The Federal Register Notice – most recent August 16, 2010 (updated approximately twice per year)

– To classify into Maternity Care service line• Use MS-DRG codes 765 – 768, 774, 775

• Current Service Line-MS-DRG Crosswalk Table– Quality Assurance Guidelines V6.0


62


March 2011

• Accepted methodologies for determination of service line (Exceptions Request not required)1. V.28, V. 27, V.26, or V.25 MS-DRG codes2. V.24 CMS-DRG codes3. Mix of V.28, V.27, V.26, V.25, V.24 codes based on payer

source4. ICD-9 codes5. Hospital unit6. New York State DRGs


Hospitals/Survey vendors must submit an Exceptions Request Form online for approval to use other means

63


March 2011


• Include patients unless there is positive evidence that a patient is ineligible– Missing or incomplete MS-DRG, address

and/or telephone number does not exclude patient from being sampled

64


March 2011


• Hospitals with zero eligible HCAHPS patient discharges (“zero cases”) – Submit monthly or quarterly, an HCAHPS Header Record (Survey

Month Data) online via My QualityNet

• Hospitals with five or fewer eligible HCAHPS patient discharges for that given month– May choose not to survey those patients for that given month– If patients are not surveyed, an HCAHPS Header Record (Survey

Month Data) will still need to be submitted online via My QualityNet

65


March 2011

Step C: Remove Exclusions


Ineligible Patients

Exclusions•“No-Publicity” patients•Court/Law enforcement patients (i.e., prisoners)

•Patients with a foreign home address

•Patients discharged to hospice care

•Patients who are excluded because of state regulations

•Patients discharged to nursing homes and skilled nursing facilities

66


March 2011

Step C: Remove Exclusions (cont’d)

• “No Publicity” patients• Court/Law enforcement (i.e., prisoners)• Foreign home address

– Note: U.S. territories – Virgin Islands, Puerto Rico, Guam, American Samoa, and Northern Mariana Islands are not considered foreign addresses and therefore are not excluded

• Discharged to hospice care• Excluded as a result of state regulation• Discharged to nursing homes and skilled nursing facilities

(beginning with July 1, 2011 and forward patient discharges)

67


March 2011

Step C: Remove Exclusions(cont’d)

• Record count of patients by each exclusion category

• Hospitals/Survey vendors must retain documentation that verifies all exclusions

68


March 2011

Step D: De-Duplication Process


Ineligible Patients

Exclusions

De-Duplication•Household•Multiple discharges

69


March 2011

Step D: De-Duplication Process(cont’d)

• De-Duplication by Household– Sample only one patient per household in a

given calendar month• De-duplicate by address and/or telephone number

from medical records and patient unique IDs within each month

• Do not de-duplicate by address and/or telephone number for health care facilities (unless residents are family members)

70


March 2011

• De-Duplication by Multiple Discharges– Sample patient only once in a given calendar

month– Patients are eligible to be included in the

sample in consecutive months


71


March 2011

• De-Duplication by Multiple Discharges (cont’d)– For continuous daily sampling, use only the

first discharge date• Each daily list should be compared to previous

discharge lists in the month to exclude additional discharges for a particular patient


72


March 2011


• De-Duplication by Multiple Discharges (cont’d)– For weekly sampling, use the first discharge encountered in the

sample frame• Discharges encountered in subsequent weeks would be excluded

from the sample frame• In the event a patient is listed with two discharges in the same

week (provided the patient had not been included in the sample frame in an earlier week within the same month), then include only the last discharge date during the week in the sample frame

• Each weekly discharge list must be compared to the previous discharge lists received in the month in order to exclude additional discharges for a particular patient

73


March 2011


• De-Duplication by Multiple Discharges (cont’d)– For end of the month sampling, de-duplicate

across all discharges in the month and use only the last discharge date

74


March 2011

Step E: HCAHPS Sample Frame

All Eligible Patients from which Sample

is Drawn

Ineligible Patients

ExclusionsDe-Duplication

75


March 2011

Step E: HCAHPS Sample Frame (cont’d)

• Option 1: Survey vendor generates sample frame (Strongly recommended)– Contracted hospital submits their entire

patient discharge list, excluding no-publicity patients and patients excluded because of state regulations

– Survey vendor applies Eligible Population criteria and removes Exclusions and generates the sample frame before sampling

76


March 2011

Step E: HCAHPS Sample Frame(cont’d)

• Option 2: Hospital generates sample frame – File contains all patients that meet Eligible

Population criteria– Hospital provides all required data file elements

• Total count of ineligible patients• Total count of patients by each exclusions category

– Survey vendor validates the integrity of the sample frame before sampling

77


March 2011


• Include all patients: – Who meet eligible population criteria – Discharged between first and last days of

month• Include patients even if:

– Missing or incomplete address/telephone number

– Missing eligibility criteria

78


March 2011


• Do not include patients in the Sample Frame whose discharge dates are beyond the 42-day initial contact period– if this is known before the sample is drawn

• Include these patients towards the count in the Eligible Discharge field

• Must file a Discrepancy Report to account for patient information received beyond the 42-day initial contact protocol

79


March 2011


• Example of sample frame layout (Appendix K)– Strongly recommend that hospitals/survey vendors collect all of

the elements from this layout– File content (i.e., All Patient Discharges or HCAHPS Sample

Frame)– Total number of ineligibles– Total number of exclusions and number in each exclusions

category– Total number of patient discharges

• Must maintain sample frame for a minimum of three years

80


March 2011

Step F: Draw SampleEligible Patients

Not Selected in Sample

SampledPatients

Ineligible Patients

ExclusionsDe-Duplication

81


March 2011

Step F: Draw Sample (cont’d)

• Target: Obtain at least 300 completed HCAHPS surveys over the 12-month public reporting period– Small hospitals

• If cannot obtain 300 completed surveys, sample all eligible discharges

82


March 2011


• Why 300?– Target for the statistical precision of the

ratings, which is based on a reliability criterion – 300 completes ensures that the reliability for

the publicly reported measures will be 0.80 or higher

– All hospitals must calculate sample size based on target of at least 300 completes no matter the number of discharges

83


March 2011


• Draw a random sample of eligible discharges on a monthly basis– Sampling may be continuous or at the end of the

month• Continuous – every two days, each week, every two weeks,

etc.– Same sampling ratio or timeframe maintained

• End of month – one sample is drawn following last day of the month

– Create sample frame in a timely manner in order to initiate contact for all sampled patients within 42 days of discharge

– Sample frame must include eligible discharges from the entire month

84


March 2011


• Draw sample for each unique CMS Certification Number (CCN)

• Hospitals that share CCN– 300 completes for CCN– All hospitals under one CCN must participate– Use same survey vendor– Use same mode of administration– Use same sample type and frequency

85


March 2011


Sample Size Calculation• Estimate the proportion of patients expected

to complete the survey:I = proportion of discharged patients who are ineligibleR = expected response rate among eligible patientsP = the proportion of discharged patients who

actually respond to the surveyP = (1 - I) x R

86


March 2011


• How many discharges are needed to produce 300 completes?C = number of completed surveys needed (300)N12 = Number of discharges to be sampled over 12

month periodN1 = Number of discharges sampled each month

N12 = C/PN1 = N12/12

87


March 2011


Example: Sample Size CalculationAssumptions: • ≈17% of discharged patients will be ineligible for

the survey– Source: National Hospital Discharge Survey

• ≈40% of eligible patients will respond to the survey– Source: CMS Three State Pilot

• Ineligible rates and response rates should be adjusted based on each hospital’s experience

88


March 2011


Example: Sample Size Calculation1. Estimate the proportion of patients

expected to complete the survey:P = (1 - I) x R

= (1 - 0.17) x 0.40= 0.33

89


March 2011


Per 12-monthN12 = C/P

= 300/.33= 909

Per monthN1 = N12/12

=909/12=75

Example: Sample Size Calculation2. Determine how many discharges are

needed to produce 300 completes?

90


March 2011

• Should estimate I and R from hospital’s own data

• Should adjust the target in subsequent quarters– Sampling rates should be consistent among

the months in a given quarter


91


March 2011

Step F: Draw Sample (cont’d)• If More than 300 Completed Surveys:

– Do not stop surveying when a total of 300 is reached

– Continue to survey every patient in the sample– Surveying must continue even if hospital’s

predetermined target (quota) has been met– Full protocol for each mode of administration

must be completed– Submit the entire sample

92


March 2011


• If Less Than 300 Completed Surveys:– Attempt to obtain as many as possible– Survey all eligible discharges– All hospital results will be publicly reported on

Hospital Compare Web site– The lower precision of scores based on less

than 100 and less than 50 completed surveys will be noted in public reporting

93


March 2011

Methods of Sampling

• Option 1: Simple Random Sample (SRS)– HCAHPS preferred sampling method– Group of patients randomly selected from a

larger group– Census sample of all eligible patients is

considered a simple random sample

94


March 2011

Methods of Sampling• SRS Example 1: Continuous daily sampling

throughout the month– Based on sampling 2 out of every 5 (this is equal to a 40%

sampling rate) eligible discharges is utilized for drawing a sample for the hospital

• Day 1: – 12 eligible patients are randomly

sorted, then numbered 1 through 12

– 4 patients would be selected with 2 patients left over to begin the next day’s count.

[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12]

• Day 2: – 8 eligible patients are randomly

sorted, then numbered 1 through 8– Add the two patients left over from

Day 1 – 4 patients would be selected, with 0

patients left over to begin the next day’s count

[11, 12, 1, 2, 3, 4, 5, 6, 7, 8]

95


March 2011

Methods of Sampling

• SRS Example 2: Census sampling– Hospital chooses to sample all eligible discharges

• Each patient has an equal chance (100%) of being included in the sample and the patients are not stratified in any manner

– Hospital has 80 eligible discharges for a given month• Each of the 80 eligible patients is included in the

hospital’s HCAHPS sample

96


March 2011

Methods of Sampling• SRS Example 3: End of month sampling

– Sampling for hospital is conducted only once for a given month at the end of the month

– Hospital has 150 eligible discharges for a given month and uses a 50% sampling rate • Randomly sort all 150 eligible patients prior to sampling• Select 50% of the 150 eligible discharges for a monthly

sample size of 75 patients• Since the eligible discharge list is already randomly

sorted, the first 75 patients may be selected to form the monthly random sample

97


March 2011

Methods of Sampling (cont’d)

• Option 2: Proportionate Stratified Random Sample (PSRS)– Patient discharge population divided into strata

• Due to continuous sampling (by day or by week)• Divided by hospital unit, or floor, etc.• Multiple hospitals share the same CCN and the random

sample is drawn separately from each hospital before each hospital’s data are combined

– Same sampling ratio applied to each stratum– Exceptions Request Form not required

98


March 2011


Stratum Week Eligible Discharges Sampling Rate Sampled

Patients1 1 20 0.20 20 * 0.20 = 42 2 25 0.20 25 * 0.20 = 53 3 30 0.20 30 * 0.20 = 64 4 15 0.20 15 * 0.20 = 35 5 10 0.20 10 * 0.20 = 2

• PSRS Example 1: Weeks—Strata are defined as weeks within a month– Sample is pulled each week, creating 5 strata: Wk1, Wk2, Wk3, Wk4, Wk5– Even though the number of eligible discharges differs across the five

weeks, the same proportion (or percentage) of “sampled” discharges is used each week

– 20% of eligible discharges are randomly pulled from each stratum– Results in different number sampled from each week, but each eligible

discharge had an equal chance of being chosen

99


March 2011

Methods of Sampling (cont’d)• PSRS Example 2: Hospital Units—Strata are defined as

units within a hospital– Sample is pulled from three units, creating 3 strata: Unit 1, Unit 2, and

Unit 3– Even though the number of eligible discharges is different in each of the

three units, the same sampling ratio is used for each unit– 30% of eligible discharges are randomly pulled from each stratum– Results in different number sampled from each unit, but each eligible

discharge had an equal chance of being chosen

Stratum Unit Eligible Discharges Sampling Rate Sampled Patients

1 1 150 0.30 150 * 0.30 = 45

2 2 50 0.30 50 * 0.30 = 15

3 3 400 0.30 400 * 0.30 = 120

100


March 2011

Methods of Sampling (cont’d)• PSRS Example 3:

Combinations of Location and Time Period– Sample is pulled each week from

2 locations, creating 10 (2x5) strata

– 50% of the eligible discharges are randomly pulled from each hospital location per week

– The number of sampled patients differs in the two locations and among the five weeks

– Every eligible discharge had an equal chance of being selected for sampling, regardless location or week

Stratum Week Location Eligible Discharges

Sampling Rate Sampled Patients

1 1 East 100 0.50 100 * 0.50 = 50

2 1 West 60 0.50 60 * 0.50 = 30

3 2 East 110 0.50 110 * 0.50 = 55

4 2 West 72 0.50 72 * 0.50 = 36

5 3 East 130 0.50 130 * 0.50 = 65

6 3 West 54 0.50 54 * 0.50 = 27

7 4 East 96 0.50 96 * 0.50 = 48

8 4 West 64 0.50 64 * 0.50 = 32

9 5 East 106 0.50 106 * 0.50 = 53

10 5 West 70 0.50 70 * 0.50 = 35

101


March 2011


• Option 3: Disproportionate Stratified Random Sample (DSRS)– Patient discharge population divided into strata– Dissimilar sampling ratio applied to each stratum– Sample a minimum of 10 eligible discharges in every

stratum in every month– Additional information collected to weight data– Exceptions Request Form required

102


March 2011


Stratum Unit Eligible Discharges Sampling Rate Sampled

Patients1 1 20 0.50 20 * 0.50 = 10

2 2 40 0.25 40 * 0.25 = 10

3 3 100 0.10 100 * 0.10 = 10

• DSRS Example 1: Hospital Units—Strata are defined as units within a hospital– A sample is pulled for three units in each month, creating three strata: Unit

1, Unit 2, and Unit 3– Even though the number of eligible discharges is different in each of the

three units, the same number of eligible discharges from each unit is selected

– Ten eligible discharges are randomly pulled from each unit– The number of eligible discharges selected for the sample does not result

in the same proportion of discharges across the three units

103


March 2011


Stratum Week Eligible Discharges

Sampling Rate

Sampled Patients

1 1 100 0.10 100 * 0.10 = 102 2 108 0.50 108 * 0.50 = 543 3 102 0.50 102 * 0.50 = 514 4 110 0.10 110 * 0.10 = 115 5 30 0.10 30 * 0.10 = 3

• DSRS Example 2: Weeks—Strata are defined as weekly time periods– A sample is pulled in each week of the month at a lower rate at

the beginning and end of the month– Sampling rates used are: 10%, 50%, 50%, 10%, and 10% for

Week 1, Week 2, Week 3, Week 4, and Week 5, respectively

104


March 2011

Methods of Sampling (cont’d)• DSRS Example 3: All

Combinations of Hospital Unit and Time Period– A sample is pulled once per

week (Week 1, Week 2, Week 3, Week 4, and Week 5) from each of three hospital units (Unit 1, Unit 2, and Unit 3)

– Since there are 5 weeks within the time period (month) and 3 units, this sampling scenario utilizes 15 strata (5 x 3)

– Sample ratios are: 25% of eligible discharges from Unit 1, 10% from Unit 2, and 50% from Unit 3 across all 5 weeks

Stratum Week Unit Eligible Discharges

Sampling Rate

Sampled Patients

1 1 1 40 0.25 40 * 0.25 = 10

2 1 2 60 0.10 60 * 0.10 = 6

3 1 3 18 0.50 18 * 0.50 = 9

4 2 1 52 0.25 52 * 0.25 = 13

5 2 2 50 0.10 50 * 0.10 = 5

6 2 3 12 0.50 12 * 0.50 = 6

7 3 1 44 0.25 44 * 0.25 = 11

8 3 2 60 0.10 60 * 0.10 = 6

9 3 3 14 0.50 14 * 0.50 = 7

10 4 1 60 0.25 60 * 0.25 = 15

11 4 2 70 0.10 70 * 0.10 = 7

12 4 3 16 0.50 16 * 0.50 = 8

13 5 1 28 0.25 28 * 0.25 = 7

14 5 2 20 0.10 20 * 0.10 = 2

15 5 3 12 0.50 12 * 0.50 = 6

105


March 2011

Population, Sample Frame and Sample

Hospital Population (All Patient Discharges) = 1 + 2 + 3 + 4 + 5

HCAHPS Sample Frame: generated by hospital/survey vendor (entire Eligible Population) = 1 + 2

Sample: randomly selected = 1

Population (All Patient Discharges)

Sample Drawn

2

1

3

45

106


March 2011

Quality Control for Sampling

• Receipt of patient discharge list – Secure file transfer– Within 42-day initial contact period

• Application of eligibility and exclusion criteria • Method used to determine HCAHPS service line• Update patient discharge information

107


March 2011

Key Sampling Facts

• Same sampling type must be maintained throughout the quarter

• Sample must include discharges from each month in the 12-month reporting period

• HCAHPS sample drawn first if multiple surveys administered

• Do not stop sampling/surveying if 300 completed surveys are attained

108


March 2011

Questions?

109


March 2011

BREAK

110


March 2011

Survey Administration

111


March 2011

Overview• Survey Management• Survey Instrument and Materials• Options for Survey Integration• Supplemental Questions• Modes of Survey Administration

112


March 2011

Survey Management• Establish survey management process to

administer survey– System resources – Customer support lines– Personnel training– Monitoring and quality oversight– Safeguarding patient confidentiality and privacy– Data security– Data retention

113


March 2011

Survey Management (cont’d)

• System resources– Adequate physical plant resources available to

handle survey volume– Survey system to track sampled patients

through the data collection protocol• Store the sample frame• Track key events• Assign random, unique, de-identified IDs and

match to outcome for each sampled patient

114


March 2011

• Establish customer support telephone lines– Survey vendor lines must be toll-free– Telephone staffed live during business hours– Voice mail is acceptable “after hours,” but

must be regularly monitored and replied to within one business day

– Voice mail recording must specify that the caller can leave a message about the survey

– Database or tracking log of calls maintained


115


March 2011


• Establish customer support telephone lines– Recommendations for support line operations

• Staffed live 9 AM to 8 PM Monday thru Friday • Sufficient capacity – 90% answered live• Voice mailbox for nights and weekends• Messages returned within one business day• Established return call standard of two business days

for questions that cannot be answered at the time of the call

• Provide optional support via the Internet

116


March 2011


• Personnel training– Project staff and subcontractors– Customer support personnel– Mail data entry personnel– Telephone interviewers and IVR operators

117


March 2011

Survey Management (cont’d)• Monitoring and quality oversight

– Ongoing monitoring of staff and subcontractors and the survey administration process

– Performance evaluations and feedback– System to evaluate patterns of errors– Detection and correction of performance

problems– Documentation of QA activities

118


March 2011

Survey Management (cont’d)• Safeguarding patient data

– Follow HIPAA guidelines– Restrict access to confidential data– Obtain confidentiality agreements from staff

and subcontractors who have access to confidential information• Agreements must mention HCAHPS or surveys

– Establish protocols for identifying security breaches and instituting corrective actions

119


March 2011


• Patient Confidentiality and Data Security– Establish protocols for secure patient

discharge file transfer from hospitals• Emailing of PHI via unsecure email is prohibited

– Recommend that hospital’s HIPAA privacy officer confirm that hospital’s transmission methods for patient discharge files are in compliance with HIPAA regulations

120


March 2011


• Confidentiality and privacy assurancesto patients– HCAHPS survey question responses are

confidential and private and reported in an aggregate format to CMS

– Hospital supplemental questions may voluntarily ask for patient name

121


March 2011

Survey Management (cont’d)• Physical and electronic data security

guidelines– Returned mail surveys stored in secure and

environmentally controlled location– All HCAHPS-related files, including patient

discharge files, must be retained for a minimum of three years

– Firewalls and other mechanisms for preventing unauthorized system access

– Access levels and security passwords to safeguard sensitive data

122


March 2011


• Physical and electronic data security guidelines – Electronic data files must be easily retrievable

regardless of whether they have been archived– Backup procedures in place to safeguard system

data– Frequent saves to media to minimize data losses– Electronic data backup files must be tested

quarterly

123


March 2011

Survey Instrument and Materials

• Survey instrument content– Core Survey questions 1-22– “About You” questions 23-27

• Survey materials availability—questionnaires, cover letters and OMB language– English language materials (Appendix A)– Spanish language materials (Appendix B)– Chinese language materials (Appendix C)– Russian language materials (Appendix D)– Vietnamese language materials (Appendix E)

• Survey materials availability—scripts– English telephone script (Appendix F)– Spanish telephone script (Appendix G)– English IVR script (Appendix H)

124


March 2011

Patient Pre-notification Guidelines

• Cannot show the HCAHPS survey or cover letter to patients prior to discharge from the hospital

• Cannot mail any pre-notification letters or postcards after discharge informing patient about the HCAHPS survey

125


March 2011

Options for Integration of Hospital Surveys

1. Integrated hospital and HCAHPS survey using one consistent format and transitions

– HCAHPS Items 1-22 (Core questions) are first questions

– HCAHPS Items 23-27 (“About You” questions) 2. Two separate mailings – one with the HCAHPS

survey and another with the hospital-specific survey

126


March 2011

Supplemental Questions

• May add a reasonable number of supplemental questions to the survey after the Core survey items (1-22)– Must ask the “About You” questions (23-27) following the Core

survey items but placement in the survey is at the discretion of the hospital/survey vendor

• Use appropriate phrasing to transition from the HCAHPS survey to the supplemental items– Example: “Now we would like to gather some additional detail

on topics we have asked you about before. These items use a somewhat different way of asking for your response since they are getting at a little different way of thinking about topics.”

127


March 2011

Supplemental Questions (cont’d)

• Avoid the following types of supplemental questions- Numerous, lengthy and complex questions- Questions with potential impact on responses to HCAHPS

questions- Sensitive medical or personal topics which may cause a

person to terminate the survey- Questions that may jeopardize a patient’s confidentiality

such as SSN- Questions that ask the patient to explain why he or she

chose their specific response

128


March 2011

Modes of Survey Administration

• Mail Only• Telephone Only• Mixed (Mail with Telephone Follow-up)• Active Interactive Voice Response (IVR)

129


March 2011

Modes of Administration Overview

• Data collection begins within 48 hours to 6 weeks (42 days) after discharge from hospital

• No proxy respondents• No communication to patients that is intended to

influence survey results• No incentives of any kind• If a patient is found to be ineligible, discontinue

survey administration for that patient

130


March 2011

Modes of Administration Overview (cont’d)

• No changes are permitted to the order of the HCAHPS questions or answer categories for the Core or “About You” questions

• The “About You” questions must remain as one block of questions, regardless of whether they follow the Core or hospital/survey vendor supplemental questions

• Final data files submitted to CMS via My QualityNet by the data submission deadline

131


March 2011

Modes of Administration Overview (cont’d)

• Copyright language – HCAHPS questions cannot be copyrighted– In the event an organization copyrights their survey

materials, then the following language must be included:• “The Core HCAHPS questions (Questions 1-22) and ‘About You’

HCAHPS questions (Questions 23-27) are works of the U.S. Government. These HCAHPS questions are in the public domain and therefore are NOT subject to U.S. copyright laws.”

132


March 2011

Mail Only Mode• Protocol

- Send first questionnaire with initial cover letter to sampled patient(s) between 48 hours and 6 weeks (42 days) after discharge

- Send second questionnaire with follow-up cover letter to non-respondent(s) approximately 21 days after the first questionnaire mailing

- Complete data collection within 42 days after the first questionnaire mailing

- Submit data to CMS via My QualityNet by the data submission deadline

133


March 2011

Mail Only Mode (cont’d)

• Cover letter specifications– Name and address of sampled patient included

• “To Whom It May Concern” is not acceptable salutation– OMB language included– Letter is not attached to the survey– Customization is acceptable; cannot add content that

would introduce bias– Printed on hospital or survey vendor letterhead– Signed by hospital administrator or survey vendor

project director

134


March 2011

Mail Only Mode (cont’d)• Cover letter specifications

– Language indicating the purpose of the unique patient identifier must be printed either on the cover letter or after the survey instructions on the questionnaire (or on both)• “You may notice a number on the survey. This

number is ONLY used to let us know if you returned your survey so we don’t have to send you reminders.”

135


March 2011

Mail Only Mode (cont’d)• Cover letter language requirements

– Purpose of survey• “Questions 1-22 in the enclosed survey are part of a national

initiative sponsored by the United States Department of Health and Human Services to measure the quality of care in hospitals.”

– Participation is voluntary– Hospital name and discharge date of patient– Patient’s health benefits will not be affected by

participation in the survey– Customer support number– If applicable add--“Survey responses will be shared

with hospitals for quality control purposes.”

136


March 2011

Mail Only Mode (cont’d)• Cover letter requirements

– OMB Paperwork Reduction Act language: “According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless it displays a valid OMB control number. The valid OMB control number for this information collection is 0938-0981. The time required to complete this information collected is estimated to average 7 minutes per response for questions 1-22 on the survey, including the time to review instructions, search existing data resources, gather the data needed, and complete and review the information collection. If you have any comments concerning the accuracy of the time estimate(s) or suggestions for improving this form, please write to: Centers for Medicare & Medicaid Services, 7500 Security Boulevard, C1-25-05, Baltimore, MD 21244-1850.”

137


March 2011

Mail Only Mode (cont’d)• Cover letter options

– English, Spanish, Chinese, Russian, and Vietnamese versions of cover letters

– Language directing the patient how to request the mail survey in Spanish, Chinese, Russian, and Vietnamese

– Repetition of any instructions that appear on the questionnaire

– Return address of hospital/survey vendor

138


March 2011

Mail Only Mode (cont’d)• Questionnaire guidelines and formatting

requirements- Question and answer category wording is not

changed nor is the order of Core HCAHPS questions or answer categories (items 1-22)

- “About You” questions follow the Core HCAHPS questions and remain as one block

- Question and answer categories remain together in the same columns and on the same pages

- Randomly generated unique identifiers for patient tracking purposes are placed on the first or last pages of the survey and may appear on all pages

139


March 2011


requirements- All instructions on the top of the survey are copied

verbatim- The patient’s name is not printed on the survey- Name and return address of hospital/survey vendor

must be printed on last page of questionnaire- The OMB control number must appear on the front

page of the survey or on the cover letter. It is OMB # 0938-0981

140


March 2011


requirements– Question and response options must be listed vertically

• Response options listed horizontally or in a combined vertical and horizontal format are not allowed

• No matrix formats for question and answer categories

– Wording that is underlined or bolded in the HCAHPS questionnaire must be underlined or bolded in the hospital or survey vendor questionnaire

– Arrows || that show skip patterns in the HCAHPS questions or response options must be included in hospital or survey vendor questionnaire

141


March 2011


options– Small coding numbers next to response

choices– Patient discharge date – Place for patients to voluntarily fill in their

name/telephone number placed after the Core HCAHPS questions (1-22)

142


March 2011

Mail Only Mode (cont’d)• Questionnaire guidelines and formatting options

– Hospital logos may be included on the questionnaire; other images and tag lines are not permitted

– Title of questionnaire “HCAHPS Survey” may be eliminated– Phrase “Use only blue or black ink” may be used– Name of contracted hospital may be printed in transition

phrases before Q1 and Q21– Phrase “There are only a few remaining items left” before

the “About You” questions may be eliminated

143


March 2011


• Questionnaire guidelines and formatting -suggestions– Minimum font size 10 point– Readable font such as Arial– Margins are wide (at least 3/4 inch) and

survey has white space to enhance its readability

– Question formatting in two columns

144


March 2011


• Mail Out - Requirements– Guidelines for mailings

• Addresses acquired from hospital record• Addresses updated using commercial software• Mailings sent to patients by name

– Mailing content• Survey mailings include

– Cover letter– Questionnaire– Self-addressed, stamped business reply envelope– First class postage or indicia, suggested

145


March 2011


• Timing of Mailings– Survey mail out timing begins with first mailing– Acceptable to send an additional mailing within the

42-day survey administration period due to address correction or patient request• Timing does not restart if another “first mailing” is sent

out • Must not send any mailings after the 42-day survey

administration period has concluded

146


March 2011


• Mail Receipt—Blank Questionnaire– If first survey mailing is returned with all missing

responses (i.e., no questions are answered), send a second survey to the patient, if the data collection time period has not expired

• If second survey mailing is returned with all missing responses, then code the final Survey Status as “7—Non-response: Refusal”

• If the second mailing is not returned then code the Final Survey Status as “8—Non-response: Non-response after maximum attempts”

147


March 2011

Mail Only Mode (cont’d)• Data receipt and entry

– Key entry or scanning allowed for data capture• Key-entered data is entered a second time

by different staff and any discrepancies between the two entries are identified; any discrepancies should be reconciled

• Programs verify that record is unique and has not been returned already

• Programs identify invalid or out-of-range responses

148


March 2011


• Data receipt and entry– Survey receipt is recorded in a timely

manner– Surveys are date stamped– Ambiguous situations follow HCAHPS

decision rules

149


March 2011


• Data receipt and entry – Final survey status codes

• Calculate lag time for all codes• Maintain a crosswalk of interim disposition codes

to HCAHPS Final Survey Status codes• Capture the mail wave attempt in which the final

disposition of the survey is determined

150


March 2011

Mail Only Mode (cont’d)• Data retention/storage guidelines

– Paper questionnaires that are key-entered must be stored in a secure and environmentally controlled location for a minimum of three years

– Optically scanned questionnaire images must be retained in a secure manner for a minimum of three years and are easily retrievable

151


March 2011

Mail Only Mode (cont’d)• Quality control guidelines

– Hospitals/Survey vendors must: • Update address information

–National Change of Address (NCOA)–USPS CASS Certified Zip+4 software–Other commercial software/search engines

• Check quality of printed materials• Check survey packet contents

152


March 2011

Mail Only Mode (cont’d) • Quality control guidelines

– Hospitals/Survey vendors must:• Check a sample of mailings for inclusion of all

sampled patients• Check for timeliness of manual or automated date

stamping• Provide ongoing oversight of staff and

subcontractors• Check for accuracy of mailing contents

153


March 2011

Mail Only Mode (cont’d)• Quality control guidelines

– Hospitals/Survey vendors must:• Conduct seeded (embedded) mailings to

designated hospital or survey vendor HCAHPS project staff on a quarterly basis to check for:

– Timeliness of delivery– Accuracy of address – Accuracy of mailing contents

• Document results of all oversight activities

154


March 2011

Telephone Only Mode• Protocol

– Initiate systematic telephone contact with sampledpatient(s) between 48 hours and 6 weeks (42 days) after discharge

– Complete telephone sequence within 42 days of initiation so that a total of 5 telephone calls are attempted• at different times of day• on different days of the week• in more than one week (eight days or more)• and between AM and PM respondent time

– Submit data to CMS via My QualityNet by the data submission deadline

155


March 2011

Telephone Only Mode (cont’d)

• Scheduling calls– If it is determined that the patient may be

available in the latter part of the 42-day data collection time period, then hospitals/survey vendors MUST use the entire data collection time period to schedule telephone calls with that patient

156


March 2011

Telephone Only Mode (cont’d)• Telephone script

– Standardized telephone script provided forHCAHPS portion of survey

• Question and answer category wording may not be changed nor the order of questions and answer categories

• “About You” questions 23-27 must be placed anywhere after the Core survey questions 1-22 and remain together as one block of questions

• Supplemental questions may be added after the Core survey questions 1-22

• Transitional phrases must be added before supplemental questions

157


March 2011

Telephone Only Mode (cont’d)• Interviewing systems

– Electronic telephone interviewing, including CATI orother alternative systems (required of survey vendors and of hospitals conducting surveys for multiple sites)• Programmed with standardized HCAHPS telephone script

– Manual data collection (allowed only for hospitals self-administering surveys)• Follow standardized HCAHPS telephone script using paper

questionnaires

158


March 2011

Telephone Only Mode (cont’d)• Interviewing systems

– Monitoring and recording of telephone calls• Follow state regulations (see QAG, page 75)

– Caller ID• May be programmed to display “on behalf of [HOSPITAL

NAME]” with permission and compliance of hospital’s HIPAA/Privacy officer

– Dropped calls• Calls that are inadvertently dropped must be re-contacted

immediately

– Default response option • DO NOT pre-program a specific response as a default option

159


March 2011


• Obtaining telephone numbers– Main source of telephone numbers is

hospital discharge records– Update missing or incorrect telephone

numbers using • commercial software • internet directories• directory assistance • other tested methods

160


March 2011


• Data receipt and data entry– Electronic data collection, CATI

• Linked electronically to survey management system– Manual data collection of paper questionnaires

• Key entry• Scanning

161


March 2011


• Data receipt and entry – Final survey status codes

• Calculate lag time for all codes• Maintain a crosswalk of interim disposition codes

to HCAHPS Final Survey Status codes• Capture the telephone attempt in which the final

disposition of the survey is determined

162


March 2011


• Data retention and storage guidelines– Data collected through electronic telephone

interviewing systems must be maintained in a secure manner for a minimum of three years

– Paper questionnaires collected manually and then key-entered must be stored in a secure and environmentally controlled location for a minimum of three years

– Optically scanned paper questionnaire images must be retained in a secure manner for a minimum of three years

163


March 2011


• Quality control guidelines– Formal interviewer training to ensure

standardized, non-directive interviews– Telephone monitoring and oversight of staff

and subcontractors• At least 10% of HCAHPS attempts and interviews

must be monitored• All interviewers conducting HCAHPS surveys must

be monitored

164


March 2011

Mixed Mode• Protocol—Mail followed by telephone

Send questionnaire with cover letter to sampled patient(s) between 48 hours and 6 weeks (42 days) after discharge

Initiate systematic telephone contact for all non-respondent(s) approximately 21 days after mailing the questionnaire

Complete telephone sequence within 42 days of initiation so that a total of 5 telephone calls are attempted • at different times of day • on different days of the week• in more than one week (eight days or more)• and between 9 AM and 9 PM respondent time

165


March 2011

Mixed Mode (cont’d)• Protocol

– Mixed mode survey administration• Follow guidelines for Mail Only mode

– Use one questionnaire mailing instead of two• Follow guidelines for Telephone Only mode


166


March 2011

Mixed Mode (cont’d)• Hospitals/Survey vendors must keep track of the

mode in which each survey was completed (i.e., mail or telephone):1. For completed surveys retain documentation in

survey management system that the patient completed the survey in the mail phase ortelephone phase of the Mixed mode of survey administration, then

2. Assign the appropriate “Survey Completion Mode” in the administrative record for this patient

167


March 2011

Active Interactive Voice Response (IVR) Mode

• Protocol– Initiate systematic IVR contact to sampled patient(s)

between 48 hours and 6 weeks (42 days) after discharge

– Complete IVR sequence within 42 days after initiation so that a total of 5 telephone calls are attempted • at different times of day • on different days of the week• in more than one week (eight days or more)• and between 9 AM and 9 PM respondent time


168


March 2011

Active IVR Mode (cont’d)

• IVR interviewing systems– Programmed with standardized HCAHPS IVR script– Capable of recording and storing patient answers– Capable of touch tone key pad response– Opt out option available for patients who do not want

to continue with IVR (other interviewing option available)

169


March 2011


• Live operator – Introduces patient to the survey and IVR system– Obtains patient consent to participate – Transitions patient to IVR– Available to answer questions/FAQs– Available to triage patients to another electronic

system (CATI) or to conduct the interview themselves for reluctant respondents

170


March 2011


• Follow Telephone Only mode guidelines– Data collection – Data receipt and retention– Quality control guidelines

• Staff/subcontractor training• Monitoring and oversight• Documentation

171


March 2011

Active IVR Mode (cont’d)• Hospitals/Survey vendors must keep track of

the mode in which each survey was completed (i.e., IVR or telephone):1. For completed surveys retain documentation in the

survey management system that the patient completed the survey in the IVR mode orTelephone mode of the IVR mode of survey administration, then

2. Assign the appropriate “Survey Completion Mode” in the administrative record for this patient

172


March 2011

Questions?

173


March 2011

Telephone and Active IVRTraining

174


March 2011

Overview• Telephone and Active IVR Script and Programs• Survey Introduction• Guidelines for Reaching Respondents• Identifying Possible Ineligible Respondents• Definition of a Telephone/Active IVR Attempt• Interviewing Guidelines and Conventions

– System Conventions– Avoiding Refusals – Probing for Complete Answers

• Customer Service FAQs

175


March 2011

Telephone & Active IVR Script and Programs

• Question and answer category wording must not be changed

.

• No changes are permitted to the order of the HCAHPS questions

• The five “About You” HCAHPS questions must remain together

• No changes are permitted to the order of the “About You” HCAHPS questions, even if they are placed before or after any supplemental questions

176


March 2011

Telephone & Active IVR Script and Programs (cont’d)

• All underlined content must be emphasized

• Only one language (English or Spanish) may appear on the electronic telephone interviewing system screen

• Supplemental questions allowed for hospital-specific items

• Skip patterns should be programmed into the telephone and IVR systems

177


March 2011

Telephone & Active IVR Script and Programs (cont’d)

• Conventions provide instructions for programmers and interviewers

• Every question should have a “Missing/Don’t Know” option programmed

• Do not program a specific response category as the default option

• See Appendices F, G & H in the Quality AssuranceGuidelines V6.0 for scripts

178


March 2011

Survey Introduction• Critical to gaining cooperation• Provides survey purpose • Confirms respondent eligibility• Informs respondent that survey will take

about seven minutes or [VENDOR SPECIFY]• Any changes to the survey introduction

require an approved Exceptions Request before implementation

179


March 2011

Survey Introduction (cont’d)

• Introduction script provided• Speak professionally and with confidence• After gaining agreement to participate,

interviewers should move swiftly into first question without rushing

• Maintain pace and avoid long pauses

180


March 2011

Introducing Active IVR• Live operator connects patient to active IVR

system after:– gaining participation through initial telephone contact– confirming patient eligibility

• Patient will hear electronic message confirming successful connection to active IVR system

• Required to use live operators to conduct the HCAHPS interview when a patient does not wish to continue with the IVR interview

181


March 2011

Guidelines for Reaching Respondents

• Five attempts to reach patient• Do not leave messages on answering machines

since this could violate a patient’s privacy• Maximize the probability of reaching the patient

by attempting contacts– at various times of the day– on different days of the week– and in more than one week

182


March 2011

Guidelines for Reaching Respondents (cont’d)

• If the patient is away temporarily, he or she is contacted upon return

• If the patient does not speak the language the survey is being administered in, thank the patient for his or her time and terminate the interview

• If the patient is temporarily ill, re-contact the patient to see if there has been a recovery before the end of data collection

• Attempt to correct wrong telephone numbers• If the call is inadvertently dropped and the interview is

interrupted, the patient should be re-contacted immediately to complete the remainder of the survey

183


March 2011

Identifying Possible Ineligible Respondents

• INEL1: Were you ever at this hospital?– <1> YES [GO TO INEL2] – <2> NO [GO TO INEL_END]

• INEL2: Were you a patient at this hospital in the last year?– <1> YES [GO TO INEL3] – <2> NO [GO TO INEL_END]

• INEL3: When was this?– IF ANY PERIOD WAS WITHIN TWO WEEKS OF [DISCHARGE

DATE], GO TO Q1_INTRO; OTHERWISE, GO TO INEL_END. • INEL_END: Thank you for your time. It looks like we made

a mistake. Have a good (day/evening).

184


March 2011

Definition of a Telephone/Active IVR Attempt

• Telephone rings six times with no answer

• An answering machine or voicemail is reached

• Busy signal—interviewer gets a busy signal on each of 3 consecutive attempts (counts as one attempt)

• Interviewer or operator reaches the household and is told that the patient is not available to come to the telephone

• Patient asks the interviewer or operator to call back at a more convenient time

185


March 2011

Interviewing Guidelines and Conventions

• Interviewer/Operator tone:– Speak in an upbeat and courteous tone– Establish rapport– Maintain professional and neutral relationship– Do not provide personal information or opinions– Do not try to influence patients’ responses to

questions in a certain way• See Appendix I in Quality Assurance Guidelines

V6.0

186


March 2011

Interviewing Guidelines and Conventions (cont’d)

• Question asking:– Questions, transitions and response choices

are read exactly as worded on script– Do not provide extra information or lengthy

explanations to respondent questions– Never skip questions– End the survey by thanking the respondent

for his or her time

187


March 2011


• System conventions– Text that appears in lower case letters must be read out loud– Text in UPPER CASE letters must not be read out loud– Text that is underlined must be emphasized– Characters in < > must not be read out loud– [Square brackets] are used to show programming

instructions that must not actually appear on the computerized interviewing screens

– Skip patterns should be programmed into the electronic telephone/IVR interviewing system

188


March 2011


• Avoiding refusals– Be prepared to convert a refusal into a

completed survey– Emphasize importance of participation– Never argue with or antagonize a patient– Remember! First moments of the interview

are most critical for gaining participation

189


March 2011


• Probing for complete data– When respondent fails to provide adequate

answer– Never interpret answers for respondents– Code “Missing/Don’t Know” when

respondent cannot/does not provide complete answer after probing

190


March 2011


• Types of probes:– Repeat question and answer categories – Interviewer says:

• “Take a minute to think about it”• “So would you say…”• “Which would you say is closer to the

answer?”

191


March 2011


Example of response probe: Question 21

We want to know your overall rating of your stay at [FACILITY NAME]. This is the stay that ended around [DISCHARGE DATE]. Please do not include any other hospital stays in your answer.

Using any number from 0 to 10, where 0 is the worst hospital possible and 10 is the best hospital possible, what number would you use to rate this hospital during your stay?

192


March 2011


• Patient 1 Answers – “The hospital is fine.”

• Probe for Patient 1 – “Please pick a number from 0

to 10, where 0 is the worst hospital possible and 10 is the best hospital possible. What number would you say is closest to your answer?”

• Patient 2 Answers – “I would give the hospital a

rating of 7.5”

• Probe for Patient 2 – “We’re asking you to choose

one response. What number would you use to rate this hospital, a 7 or 8?”

Example of response probe: Question 21 (cont’d)

193


March 2011



In general, how would you rate your overall health? Would you say that it is…

<1> Excellent,<2> Very good,<3> Good,<4> Fair, or<5> Poor?<M> MISSING/DK

194


March 2011


• Patient 1 Answers – “My health is okay.”

• Probe for Patient 1 – “We’re asking you to

choose one response. Would you say your overall health is…” [Repeat all answer categories]

• Patient 2 Answers – “My health is great.”

• Probe for Patient 2 – “Would you then rate

your overall health as Excellent , Very good or Good?”


195


March 2011



What is the highest grade or level of school that you have completed? Please listen to all six response choices before you answer. Did you…

<1> Complete the 8th grade or less,<2> Complete some high school, but did not graduate,<3> Graduate from high school or earn a GED,<4> Complete some college or earn a 2-year degree,<5> Graduate from a 4-year college, or<6> Complete more than 4-year college degree?<M> MISSING/DK

196


March 2011


• Patient 1 Answers – “I graduated from school.”

• Probe for Patient 1– “We’re asking you about the

highest grade or level of school that you completed. Would you say you completed…” [Repeat all answer categories]

• Patient 2 Answers – “I graduated from college.”

• Probe for Patient 2 – “We’re asking you about the

highest grade or level of school that you completed. So would you say completed some college or earned a 2-year degree, graduated from a 4-year college, or completed more than 4-year college degree?”


197


March 2011


Q26: When I read the following list, please tell me if the category describes your race. I am required to read all five categories. You may choose one or more.

Q26A Are you White?<1> YES/WHITE<0> NO/NOT WHITE<M> MISSING/DK

Q26B Are you Black or African- American?<1> YES/BLACK OR AFRICAN- AMERICAN<0> NO/NOT BLACK OR AFRICAN- AMERICAN <M> MISSING/DK

Q26C Are you Asian?<1> YES/ASIAN<0> NO/NOT ASIAN<M> MISSING/DK

Q26D Are you Native Hawaiian or other Pacific Islander?<1> YES/NATIVE HAWAIIAN OR OTHER PACIFIC ISLANDER<0> NO/NOT NATIVE HAWAIIAN OR OTHER PACIFIC ISLANDER<M> MISSING/DK

Q26E Are you American Indian or Alaska Native?<1> YES/AMERICAN INDIAN OR ALASKA NATIVE<0> NO/NOT AMERICAN INDIAN OR ALASKA NATIVE<M> MISSING/DK

198


March 2011


• Q26 (cont’d)– Question 26 is broken into parts A – E– Do not stop reading the list when you get a “Yes”

answer– Enter all of the race categories that the patient has

selected• If the patient responds “Yes” to a race category, enter “1.’ • If the patient responds “No” to a race category, enter “0.” • If the patient does not provide a response to any race

categories or skips the question, enter “M – Missing/Don’t Know.”

199


March 2011

Customer Service FAQs• Interviewers/Operators should be knowledgeable

about the survey and its goals, and be prepared to answer questions

• FAQs provide answers to:– General questions about the survey– Concerns about participating in the survey – Questions about completing/returning the survey

• See Appendix J in Quality Assurance Guidelines V6.0

200


March 2011

Exceptions Requests and Discrepancy Reports

201


March 2011

Purpose• Exceptions Request

– Request alternative methodologies

• Discrepancy Report– Notification of variation from HCAHPS

protocols during survey administration

202


March 2011

Exceptions Request• Allowable exceptions to standardized protocols

– Disproportionate stratified random sampling– Service line determination– Other exceptions

• Exception request proposal must include how the proposed exception will maintain the integrity of data collection

• Exceptions not allowed for alternative modes of survey administration

203


March 2011

Exceptions Request (cont’d)• Request for exceptions

– Submit Exceptions Request Form(s) online• Justification for exception• Submit Exceptions Request Form through

www.hcahpsonline.org– Exceptions Request must be submitted and

approved prior to implementing– Exceptions must be submitted by survey

vendors on behalf of their contracted hospitals

204


March 2011

Exceptions Request (cont’d)• Appeals process for unapproved exception

– Written notification with explanation provided by HCAHPS Project Team

– Hospital/Survey vendor has five business days to appeal an unapproved exception

– Use Exceptions Request Form

205


March 2011

Discrepancy Report• Notification of inadvertent and temporary survey

administration discrepancies– Examples: missing eligible discharges from a

particular date or computer programming issues that caused an otherwise eligible discharge to be excluded from the sample frame

• Discrepancy Reports must be submitted by survey vendors on behalf of their contracted hospitals

206


March 2011

Discrepancy Report (cont’d)• Complete and submit report immediately upon discovery

of issue– Provide sufficient detail

• Hospital CCN• How issue was discovered• Average monthly eligible count• Number of eligible discharges affected• Average monthly sample size• Number of sampled patients affected• Corrective action plan• Other details and information, including time period affected

207


March 2011

Discrepancy Report (cont’d)• Review Process

– Assessment of actual or potential impact on publicly reported results, therefore there may be a delay before results of review are communicated

– Reviews may result in assignment of footnotes to publicly reported results

– Additional information may be requested– Notification of review outcome

208


March 2011

Questions?

209


March 2011

Contact UsHCAHPS Information and Technical Support• Web site: www.hcahpsonline.org• Email: [email protected]• Telephone: 1-888-884-4007

March 2011 Hcahps Introduction Training Slides Session I 2 28 2011

Business

Transcript of March 2011 Hcahps Introduction Training Slides Session I 2 28 2011