Introduction to the In-Center Hemodialysis CAHPS Survey · • The webinar provider will mute your...

Introduction to the In-Center Hemodialysis CAHPS Survey

Webinar Training Session

February 10, 2016

Introduction to the ICH CAHPS Survey

• Welcome and Introductions

• Overview of the Training Session

• Training Session Logistics

2

Introduction to the ICH CAHPS Survey (cont’d)

Training Topics

• Overview of the ICH CAHPS Survey, including background and purpose and an overview of the development of the ICH CAHPS Survey

• Roles and responsibilities of ICH facilities, ICH CAHPS Survey Vendors, and the ICH CAHPS Coordination Team

• ICH CAHPS Survey participation requirements

• The ICH CAHPS Survey patient sample and sampling selection and distribution processes

3


Training Topics

• Survey administration protocols and specifications for each approved data collection mode

• Confidentiality and data security

• Data processing and coding

• The ICH CAHPS website

• Data file preparation, data submission, and reports

4


Training Topics

• Quality Control Procedures

• Oversight Activities

• Exceptions Request Form and DiscrepancyNotification Report

5

Training Session Logistics and Reminders

Training Session Logistics• We expect to take short breaks during this session.• If you leave the session at any time, please do not

disconnect from either the web or telephone connections.

• If you get disconnected and have problems accessing the session, call the webinar provider at:

1-877-812-4501

6

Training Session Logistics and Reminders (cont’d)

• We will stop for questions after each major topic is presented.

• The webinar host will explain at the end of the first presentation what you will need to do to ask a question.

• We will address as many questions as we can during the Q&A periods, but the answer to some questions may be disseminated via e-mail or posted on the ICH CAHPS website after this session.

7


• The webinar provider will mute your telephone during the presentation and unmute it so you can ask questions during the Q&A sessions.

• The “Chat” feature will also be enabled during the Q&A sessions so that you can ask questions.

• Each new vendor’s designated ICH CAHPS Survey Administrator must complete the Training Certification Form, which will be available on the ICH CAHPS website beginning at 8:00 AM EST on February 11, 2016.

8


• Some of the information covered in this training session is not included in the ICH CAHPS Survey Administration and Specifications Manual, so please be attentive throughout this training session.

9

Background and Development of the ICH CAHPS Survey

10

Background and Development of the ICH CAHPS Survey Overview

• Overview of the background, purpose, and goals of the ICH CAHPS Survey

• Brief review of the ICH CAHPS Survey Questionnaire and related reporting composites

• Roles and responsibilities (ICH facilities, survey vendors, Coordination Team)

• Sources of information about the ICH CAHPS Survey

11

The Quality Initiative

• November 2001, the Department of Health and Human Services announced the Quality Initiative.

• Dialysis facility measures publicly reported on Dialysis Facility Compare at http://www.medicare.gov.

• Facilities have been conducting ICH CAHPS Surveys since 2012.

• Attestations were required in CROWNWeb for payment years (PYs) 2014 and 2015 for ICH CAHPS.

12

Overview of CAHPS Surveys

What is CAHPS?

• Consumer Assessment of Healthcare Providers and Systems®.

• Family of surveys designed to collect data from patients and other consumers about their experiences with the care they receive from their health care providers.

• Developed by the Agency for Healthcare Research and Quality (AHRQ).

• CAHPS is a registered trademark of AHRQ, a U.S. government agency.

13

Overview of CAHPS Surveys (cont’d)

• CAHPS development methods include the following:

• public call for measures,

• literature reviews,

• focus groups with patients,

• cognitive interviews,

• stakeholder input,

• public response to Federal Register notices, and

• field tests.

14

Overview of CAHPS Surveys (cont’d)

Goals of CAHPS Surveys• Conduct the survey and collect the data in a

standardized manner.• Analyze and adjust data for patient mix.• Publicly report survey results.• Survey results are used:

• by consumers to choose a health care provider,• by providers to improve the quality of care they

deliver to their patients, • to monitor performance of health care providers, and• for payment to providers based on quality of services.

15

The In-Center Hemodialysis CAHPS Survey Development Timeline

• August 2003, call for measures issued.

• In 2004, several rounds of cognitive testing conducted.

• In 2005, a field test was conducted to evaluate the draft ICH CAHPS Survey instrument.

• In 2006, the pilot survey was approved by the U.S. Office of Management and Budget (OMB).

• In 2007, the survey was approved by the National Quality Forum.

• In public domain for use by ICH facilities since 2007.

16

The In-Center Hemodialysis CAHPS Survey Development Timeline

(cont’d) • November 2011, final rule for the ESRD Quality Incentive

Program published for PY2013 and PY2014.

• Introduced ICH CAHPS as reporting measure for PY2014 (attestation that ICH CAHPS Survey was done in CY2012).

• September 2013, OMB approval for national implementation of ICH CAHPS.

• November 2013, Final Rule published for ESRD Quality Incentive Program published for PY2015 and PY2016.

• Winter 2014, ICH CAHPS mode experiment was conducted.

• Fall 2014, national implementation of ICH CAHPS began.

• November 2014, Final Rule published for ESRD Quality Incentive Program published for PY2017 and PY2018.

17

The 2016In-Center Hemodialysis CAHPS

Survey • For PY2018, all ICH facilities must administer the ICH CAHPS

Survey in CY2016 if they served 30 or more survey-eligible patients in CY2015.

• Sampling and data collection will be conducted on a semiannual basis (Spring and Fall Surveys).

• Surveys will be conducted by independent approved survey vendors working under contracts with ICH facilities.

• Three modes of data collection are allowed—mail only, telephone only, and mail with telephone follow-up of nonrespondents(mixed mode).

• Standardized protocols for data collection must be followed.

18

The In-Center Hemodialysis CAHPS Survey Questionnaire

• Contains 44 core questions plus demographic items and self-reported health status items.

• Is available in English, Spanish, traditional and simplified Chinese, and Samoan.

• Includes questions about care, communication, and interactions with facility staff, including doctors, plus questions about treatment options.

• Asks patients to rate the care received from the ICH facility and to rate the center’s staff and the center.

19

National ImplementationData Submission Deadline

ICH CAHPS data submission deadline will always be on:• The last Wednesday in July for Spring Surveys • The last Wednesday in January for Fall Surveys

For PY2018, the deadlines for submission of ICH CAHPS data are:

• 2016 Spring Survey: July 27, 2016• 2016 Fall Survey: January 25, 2017

20

Reporting Measures and Composites

• Results based on combined data from the 2014 ICH CAHPS Survey and the 2015 Spring Survey will be posted on the ICH CAHPS website in February 2016.

• CMS is planning to begin publicly reporting survey results on the Dialysis Facility Compare (DFC) website at http://www.Medicare.gov.

• The first results reported on the DFC will be based on data from the semiannual surveys conducted in calendar year 2015.

• Results will be “refreshed” on the DFC on a periodic basis.

21

Reporting Measures and Composites

• Individual measures, composite scores, and global ratings will be reported by facility CCN.

• Composite Score• Nephrologists’ Communication• Quality of Dialysis Center Care and Operations• Providing Information to Patients

• Global Items• Rating of the nephrologist• Rating of the dialysis center staff• Rating of the dialysis facility

22

Roles and Responsibilities: Survey Vendors

• Follow protocols in the Survey Administration and Specifications Manual.

• Adhere to Minimum Business Requirements.

• Submit the Vendor Registration and Vendor Application.

• Attend entire Introductory training and any Update trainings.• Required training attendance: Anyone with a significant role in

administering the ICH CAHPS Survey:• vendor’s project and data managers,• subcontractors with a significant role, or • anyone handling personally identifiable information (PII).

• The Survey Administrator must complete and pass the Training Certification Form.

23

Roles and Responsibilities:Survey Vendors (cont’d)

• Enter into a formal contract with each ICH facility client.

• Verify that each client has authorized your organization to submit data on its behalf.

• Complete a Data Use Agreement with CMS.

• Download sample files from the ICH CAHPS website.

• Administer survey and oversee quality of work.

• Prepare and submit data on behalf of ICH facilities.

• Review all data submission reports for ICH facilities.

• Submit a Quality Assurance Plan.

24

Roles and Responsibilities:ICH Facilities

• Contract with a CMS-approved ICH CAHPS Survey vendor to conduct the survey.

• Register on the ICH CAHPS website. Complete an online form which authorizes a facility’s contracted survey vendor to download, collect, and submit data on its behalf.

• Monitor survey vendor data submissions by reviewing reports.

• Preview public reporting results.

• Monitor the ICH CAHPS website for announcements and other news about the surveys.

25

Roles and Responsibilities: ICH CAHPS Coordination Team

• Train survey vendors.

• Approve survey vendors.

• Translate survey and related materials into additional languages.

• Maintain ICH CAHPS website.

• Disseminate news and information about the survey in a timely manner.

• Select sample of patients from registered ICH facilities and distribute via the website to survey vendors.

26

Roles and Responsibilities: ICH CAHPS Survey Coordination Team

(cont’d)

• Provide technical assistance to ICH facilities and vendors.

• Ensure integrity of data collection by conducting oversight and quality assurance of survey vendors.

• Receive and conduct final processing of data submitted by survey vendors.

• Calculate and adjust results for mode and patient-mix effects.

• Analyze and publicly report ICH CAHPS Survey results.

27

Sources of Information About ICH CAHPS Survey

• Several sources available:• ICH CAHPS Survey Administration and Specifications

Manual• The CMS ICH CAHPS website (https://ichcahps.org)

• Announcements• Registration/Application• Forms

• Send queries to [email protected]• Call technical support at 1-866-245-8083

28

Questions?

29

ICH CAHPS SurveyParticipation Requirements

30

Participation Requirements Overview

• Survey Vendor Participation Requirements

• Survey Vendor Minimum Business Requirements

• Communications With Patients About the ICH CAHPS Survey

• Survey Vendor Analysis of ICH CAHPS Survey Data

• Communications and Technical Support

31

Survey Vendor Participation Requirements

• Register on the ICH CAHPS website.

• Complete and submit a Vendor Application.

• Participate in this training session and any subsequent update trainings.

• Vendor’s survey administrator must attend the entire session and complete the training certification following the session.

• Vendor’s data managers or other staff with significant roles must attend.

• Vendor’s subcontractors with significant roles must attend.

32

Survey Vendor Participation Requirements (cont’d)

• Agree to provide additional information about the vendor organization that might be needed to grant conditional approval as an ICH CAHPS Survey vendor.

• Submit a Data Use Agreement (DUA) to CMS.

• Administer the survey according to the procedures and specifications described in the ICH CAHPS Survey Administration and Specifications Manual.

• Develop and submit a Quality Assurance Plan and update it as required.

33


• Implement quality control procedures on all aspects of the ICH CAHPS Survey.

• Verify that all subcontractors have required capabilities and staff to perform work for which they will be responsible.

• Review and monitor the work performed by subcontractors and verify that they have well-documented quality control procedures.

• Participate in all oversight activities.

34


• Process survey data and submit ICH CAHPS Survey data files to the Data Center.

• Monitor data submission reports and correct and resubmit data files as needed.

• Adhere to the ICH CAHPS Survey Vendor Minimum Business Requirements.

• Approval will be discontinued if the vendor does not administer the survey within 2 years after approval is granted.

35

Survey Vendor Business Requirements

• Survey vendors must have the following relevant organizational experience:

• minimum of 3 years in business,

• minimum of 2 years conducting surveys using requested mode, and

• minimum of 2 years conducting surveys of individuals.

• If staff have relevant experience while employed by another organization, that experience does not count toward the 2-year minimum.

36

Survey Vendor Business Requirements (cont’d)

• Vendors must have experience conducting surveys of individuals responding about their own experiences.

• Definition of “Survey of Individuals”:

• The collection of data from individuals selected by statistical sampling methods and the data collected are used for statistical purposes.

37


• Examples of data collection activities that do not satisfy the requirement of valid “survey of individuals” experience:• polling questions administered to trainees or participants of

training sessions or educational courses, seminars, or workshops;

• focus groups, cognitive interviews, or any other qualitative data collection activities;

• surveys of fewer than 600 individuals;• surveys that did not involve statistical sampling methods;• Internet or web-based surveys; and• Interactive Voice Recognition surveys.

38


• Survey vendors must have the survey capability and capacity to collect and process all survey-related data following standardized procedures regarding:

• personnel,

• facilities and systems,

• survey administration, and

• data processing and file submission.

39


• Survey vendors must adhere to quality assurance guidelines and documentation, including:

• incorporating well-documented quality control procedures for training, mailing/printing/receipt, coding, and scanning/keying;

• participating in conference calls and site visits; and

• providing documentation as requested for site visits and conference calls.

40


Organizations that will not be eligible to administer the ICH CAHPS Survey:

• organizations or divisions within organizations that own or operate or provide ICH services, even if the division is run as a separate entity to the ICH facility;

• organizations that provide telehealth—that is, the monitoring of hemodialysis patients’ health via telephone, or teleprompting services for ICH facilities; and

• organizations that provide staff to ICH facilities who provide care to ICH patients.

41

Survey Vendor Approval Process

• (This week) Prospective vendors must attend Webinar training.

• Prospective Vendors must complete and submit the Vendor Application to the Coordination Team.• Assuming that all business requirements are met and

training certification is accepted, prospective vendors will receive “conditional approval” status.

• Prospective vendors must submit a QAP to the Coordination Team within 6 weeks after first data submission.• Assuming that the QAP meets requirements, vendor will

receive final approval status.

42

Communications With ICH Patients About the ICH CAHPS Survey

• ICH patients are an especially vulnerable patient population who:

• depend on hemodialysis care for their survival;

• might be reluctant to participate in the survey for fear of retribution; and

• might not provide survey responses that accurately reflect their experience at their ICH facility.

• ICH facility staff may tell their patients that they might be asked to respond to a patient experience of care survey.

43

Communications With ICH Patients About the ICH CAHPS Survey (cont’d)

ICH Facilities and Survey Vendors MUST NOT:

• help the patient answer survey questions;

• attempt to influence patients’ answers to survey questions;

• tell patients that the facility hopes or expects its patients to give them the best or highest rating or will respond in a certain way to survey questions; or

• offer incentives for participating (or not) in the survey.

44


In addition, ICH facilities must not:

• Ask patients questions that are the same or similar to those included in the ICH CAHPS Survey questionnaire 4 weeks prior to and during the data collection period.

• Give a copy of the ICH CAHPS Survey questionnaire or cover letters to their patients.

• Include words or phrases verbatim from the ICH CAHPS Survey questionnaire in marketing or promotional materials.

45


• Include any messages or materials promoting the facility or the services it provides in survey materials.

• Use the ICH CAHPS Survey to identify or ask about other patients who might need hemodialysis care.

• Ask their patients if they would like to be included in the survey.

46

Survey Vendor Analysis of ICH CAHPS Survey Data

Survey vendors cannot:

• Provide patient-level datasets to facilities.

• Share a sample patient’s survey responses with an ICH facility, even if the patient gives the survey vendor consent to share his or her responses.

47

Survey Vendor Analysis of ICH CAHPS Survey Data (cont’d)

• CMS-calculated results for the ICH-CAHPS Survey are the official survey results.

• Survey vendors can analyze ICH CAHPS Survey data to provide facilities with results they can use for quality improvement purposes.• Must report as blank any cells or results that are based

on survey responses from 10 or fewer sample patients. • When there are blank cells in a table, the vendor must

not report row and column totals so that the cell value cannot be derived.

48

Questions?

49

Sample Selection and Distribution

50

Sampling Selection and Distribution Overview

• Survey Schedule

• Patient Survey Eligibility

• Sample Selection

• Sample Identification Number

• Sample Distribution Process

• Sample File Variables

51

Survey Schedule

• The sampling window for each semiannual survey will be 3 months.

• 2016 Spring Survey sampling window is October 1, 2015, through December 31, 2015.

• 2016 Fall Survey is April 1, 2016, through June 30, 2016.

• The sample selected for each participating ICH facility in each survey period will consist of patients who received hemodialysis during the sampling window who meet survey eligibility criteria.

52

2016 ICH CAHPS Spring Survey Schedule

2016 Spring Survey Activity Date

CT Upload Sample Files April 1, 2016

Mail Prenotification Letters April 22, 2016

Mail First Questionnaire Package/Begin Telephone Interviews

May 6, 2016

Mail Second Questionnaire/Begin Telephone Follow-Up for Mixed Mode

June 3, 2016

End Data Collection July 15, 2016

Data Submission Deadline 11:59 PM ET on July 27, 2016

53

Patient Survey Eligibility Criteria

To be included in the ICH CAHPS Survey sample, patients must:

• be 18 years old or older by the end of the sampling window,

• be alive as of last day of sampling window,

• have received ICH care from their current facility on an outpatient basis for 3 consecutive months or longer,

• not be receiving hospice care,

• be currently not residing in an institution (residential nursing home, jail/prison).

54

Patient Eligibility (cont’d)

To the extent possible, CMS will remove ineligible patients from the sampling frame before the sample is selected. These include patients who:

• are under age 18 by the end of the sampling window,• have not received ICH at the sample ICH facility for 3

months or longer, • currently receive dialysis care at home (not in an

in-center hemodialysis facility) or receive peritoneal dialysis as indicated in the CROWNWeb file, and

• were deceased before the sampling window closed.

55


Some ineligible patients will be identified by the survey vendor after data collection begins. These include patients who:

• are no longer receiving ICH care from the sample facility,

• have not received dialysis care from the sample facility for 3 months or longer,

• are physically or mentally incapable of participating in the survey,

• are deceased (died after the sampling window closed), or

• receive home or peritoneal dialysis.

56


• Ineligible patients identified during the data collection period also include those:• who do not speak one of the approved languages offered

by the survey vendor;• who are institutionalized (live in a nursing home, prison, or

jail); and• who currently receive hospice care.

• The questions in the About You Section of the questionnaire will be administered to some ineligible patients.

57

Sample Selection

• Sample Source: Patient-level data from CMS’s Consolidated Renal Operations in a Web-Enabled Network (CROWNWeb)

• Sample selected at CCN level

• Target minimum of 200 completed ICH CAHPS Surveys per ICH facility over a 12-month period

58

Sample Selection (cont’d)

Sampling will differ depending on the number of ICH patients served by each ICH facility:

• 200 or fewer patients: Census of all ICH patients

• 201 or more patients: Simple random sample of patients

59

Survey Response Rates

Expected response rates based on other CAHPS Surveys

Data Collection Mode Expected Response Rate

Mail Only 35%

Phone Only 45%

Mixed Mode 50%

60

Sample Identification (SID) Number

• Unique Sample Identification (SID) number assigned by the Coordination Team.

• Survey vendors must not change the SID number assigned to a sample patient.

• An SID number will never be reassigned once it has been used. A new SID will be assigned to patients each time they are sampled for the survey.

• Vendors must submit de-identified data files to the Data Center. It is critical that submitted survey data contain the original SID number assigned to each sample patient.

61

Sample Distribution

Sample files will not be released until the vendor has executed a Data Use Agreement (DUA) with CMS.

• A copy of CMS’s DUA is included in Appendix B in the ICH CAHPS Survey Administration and Specifications Manual.

• The DUA must be renewed each calendar year. The deadline for renewing the DUA with CMS will be in an e-mail from CMS.

62

Sample Distribution (cont’d)

Sample files will not be released to a survey vendor until the ICH facility has authorized the survey vendor to collect and submit ICH CAHPS Survey data on its behalf.

• ICH facilities that did not authorize a vendor in 2015 and those that are changing to a different vendor must authorize their contracted vendor by:

• 2016 Spring Survey: February 28, 2016• 2016 Fall Survey: August 31, 2016

• Survey vendors must review their Vendor Authorization Report to make sure each of their facility clients has completed the online Vendor Authorization.

63


• Check the Vendor Authorization Report weekly in the weeks leading up to the vendor authorization deadline.

• When reviewing the Vendor Authorization Report,• make sure that the CCN for each facility client appears

on the report; and

• if samples have been provided in previous survey periods for a facility under two different CCNs, make sure that both of the CCNs appear on the Vendor Authorization Report; and

• contact the Coordination Team as soon as possible if a CCN for one or more of your facility clients is not on the report.

64


• When reviewing the Vendor Authorization Report, check it for the following (cont’d):• An End Date of Fall 2015 does NOT appear on the

report for each CCN listed. If it does, contact the facility involved and ask the facility to update the online vendor authorization form before the deadline.

• You have a written contract to administer the ICH CAHPS Survey for every CCN that appears on the Vendor Authorization Report.

65


• Survey vendors will download a file containing the sample via the ICH CAHPS website.

• Survey vendors will be notified that sample files are available for download via announcements posted on the ICH CAHPS website and e-mail.

• Vendors are required to download sample file within 2 business days after files are made available.

• Survey vendors must attest that they are taking control of the sample file for each of their ICH facility clients.

• Survey vendors cannot download the ICH CAHPS Survey sample file until attestation has been completed.

66

Sample File Format

• A Microsoft Excel file containing contact information for each sample patient.

• If the vendor is authorized to submit data on behalf of multiple ICH facilities, information for sample patients from all facilities will be included in one Excel file.

67

Variables on Sample Files

Facility-Level Information Patient-Level InformationCCN SID No.Name First NameStreet Address 1 Middle NameStreet Address 2 Last NameCity Street Address 1State Street Address 2Zip Code CityESRD Network No. State

Zip CodeTelephone No.Date of BirthAge Gender

68

Questions?

69

Survey Administration

70

Survey Administration Overview

• Survey Instrument and Materials

• Supplemental Questions

• Survey Management Systems

• Modes of Survey Administration

71

The ICH CAHPS Survey Questionnaire and Materials

• Survey Content:

• Core Questions (Qs. 1–44)

• “About You” Questions (Qs. 45–62)

• Survey Materials Available:

• English (Appendix C)

• Spanish (Appendix D)

• Traditional and Simplified Chinese (Appendices E and F)

• Samoan (Appendix G)

• Supplemental Questions (Appendix I)

72

Survey Instrument and Materials (cont'd)

• The ICH CAHPS Questionnaire and related survey materials are currently available in five languages:• English and Spanish (for both mail and phone survey

administration); and• Traditional Chinese, simplified Chinese, and Samoan (for

mail only survey administration)

• Survey vendors must use the CMS-approved translations of the survey materials.

• CMS will consider requests from vendors and facilities for translation of survey materials into additional languages.

• Vendors may ask for all patients’ language preferences from ICH facilities.

73

Supplemental Questions

• ICH facilities may add questions to the ICH CAHPS Survey mail questionnaire or telephone interview:

• ICH CAHPS supplemental questions,

or

• facility-specific questions.

• ICH CAHPS supplemental questions and facility-specific questions must be placed after the core ICH CAHPS Survey questions (Qs. 1–44). They may be placed either before or after the ICH CAHPS Survey “About You” questions.

• Do not include responses to supplemental questions in ICH data files that will be uploaded to the Data Center.

74

Supplemental Questions (cont’d)

ICH CAHPS Supplemental Questions

• ICH facilities may choose to use some or all of the 21 ICH CAHPS supplemental questions.

• ICH CAHPS supplemental questions do not need further approval before use.

• Vendors must use CMS-approved translations of the ICH CAHPS supplemental questions.

• ICH CAHPS supplemental questions are in Appendix I and under the “Survey and Protocols” menu at https://ichcahps.org/.

75


Facility-Specific Questions• All facility-specific questions must be approved by CMS

prior to use. • If questions were approved for use during a previous

survey period, no further action is needed, unless the question wording or response options changed.

• Deadline for submitting new facility-specific supplemental questions for the 2016 Spring Survey was January 31, 2016. Deadline for Fall Survey is July 31, 2016.

• ICH facilities or their vendors are responsible for translating all facility-specific questions added to the questionnaire.

76


Facility-Specific Questions (cont’d)

• Facility-specific questions cannot:

• repeat any ICH CAHPS core items,

• be used for marketing or promoting ICH facility services,

• ask for identification of people who might need in-center hemodialysis, or

• ask sample patients why they responded a certain way to a core CAHPS question.

77


Facility-Specific Questions (cont’d)

• We recommend that facilities/vendors avoid sensitive questions or lengthy additions, because these will likely reduce response.

• Consider adding some transitional phrasing to help focus the respondent on the facility-specific questions.

78

Survey Management Systems: All Modes

• Survey management systems allow the vendor to track the status of sampled cases through all phases of data collection.

• Cases are assigned and tracked using the unique SID number assigned by the Coordination Team.

• All cases must be assigned a final ICH CAHPS Survey disposition code at the end of the data collection period.

• Vendors can use their own pending or internal codes to track cases before they are finalized. However, vendors must be able to map pending codes back to the correct final disposition code and to the correct SID number.

79

Survey Management Systems: All Modes (cont’d)

• Survey management system must ensure that the appropriate cases: • are included in a second mailing (for mail-only

administration), • receive the required number of call attempts (for

phone-only and mixed-mode administration), and• are rolled over to telephone follow-up (for mixed-mode

administration).

• The survey management system and computer-assisted telephone interview (CATI) or data entry systems must be synchronized so that the current status of a case is readily accessible.

80

Overview: All Survey Modes

• Prenotification letter must be mailed 3 weeks (21 days) after downloading the sample file.

• If the prenotification is not mailed 21 days after receiving the sample file, vendor must file a Discrepancy Notification Report (DNR).

• Data collection must begin 15 calendar days after the prenotification mailing.

• Data collection must end 12 weeks (84 days) from prenotification mailing.

81

Overview: All Survey Modes (cont’d)

• No changes are permitted to the ICH CAHPS Survey core questions or to the About You questions.

• Data collection must not stop for a given ICH facility even if the targeted number of completes is reached.• Every sample patient must be given an opportunity to

participate in the survey. • Data collection efforts must continue on each case until

the survey is completed or otherwise finalized (the patient is ineligible, refused to participate, or could not be reached after maximum attempts were made on the case).

82


• Proxy respondents are NOT permitted.

• No incentives may be offered.

• ICH facilities may not influence sample patients in any way (avoid sending sample patients materials or do anything that could compromise the vendor’s ability to implement the survey protocols).

• Final data files must be submitted to the Data Center via the ICH CAHPS Survey website.

83


Prenotification Letter• Required for all survey administration modes.• Provides information about the survey purpose and alerts

sample patients that they will be contacted.• Contains the CMS logo and signed by CMS.• Personalized for each survey vendor, but not personalized for

sample patients.• The text of the prenotification letter cannot be changed.• Vendor-specific PDF file with the prenotification letters will be

provided to survey vendors.

84


Prenotification Letter (cont’d)• Survey vendors will print and mail letters. • Vendors must verify mailing addresses using commercial

address update services, such as National Change of Address (NCOA). • Vendors are permitted to ask for updated addresses from ICH

facilities.

• Prenotification envelope requirements:• The CMS logo, along with the vendor’s return address, which will

replace the previous requirement of printing “Centers for Medicare & Medicaid Services” on the prenotification envelope;

• “Address Service Requested” or “Return Service Requested,” or “Change Service Requested”; and

• Sample patient’s name and address.

85

Mail-Only Administration Procedures

86

Mail-Only Administration Procedures

Data Collection Schedule

• Prenotification letter 3 weeks (21 days) after receiving the sample file.

• Initial questionnaire with cover letter sent 15 calendar days after the prenotification letter.

• Second questionnaire with follow-up cover letter sent 4 weeks (28 days) after initial questionnaire mailing.

• Data collection ends 12 weeks (84 days) after prenotificationletter mailing.

• Submit data to Data Center by semiannual submission deadline (submit as early as possible prior to the deadline).

87

Mail-Only: Materials Production

Cover Letters Overview• Every questionnaire package must contain a cover

letter.• Explains the purpose of the survey, how results will be

used, how to participate, and contains vendor’s toll-free number.

• Must be signed by someone within the survey vendor organization.

• Vendors can use example cover letters or develop their own.

88

Mail-Only:Materials Production (cont’d)

Cover Letters—Requirements • Must be personalized with the name and address of

the sample patient.• Must be printed using font size equal to or larger than

Times New Roman or Arial 11 point font.• Must contain the sample patient’s SID number.• Must be separate from the questionnaire so that no

personally identifying information appears on the questionnaire.

• Cannot offer sample patients the option to complete the survey by phone if the mode is mail only.

89


Cover Letters—Requirements (cont’d)

• Cover letters must:

• include wording describing the purpose of the survey;

• state that information the sample patient provides is protected by the Privacy Act;

• include wording stating that if help is needed, the sample patient should ask a friend or family member for help and not the ICH facility personnel; and

• include wording stating that participation is voluntary and will not affect dialysis care or Medicare benefits.

90

Mail-Only: Materials Production (cont’d)

Cover Letters—Requirements (cont’d)

• Survey vendor’s name or logo must appear.

• Name of ICH facility and vendor’s name must be inserted into letter text, where indicated.

• The facility’s logo may appear at the top of cover letter on right-hand side; however, it must not be printed such that it shows through a window envelope.

• Toll-free customer support telephone number (goes to the vendor) must be provided.

91


Cover Letters—Recommendations

• If offering the questionnaire in any of the approved translations, add a sentence instructing the sample patients how to request the questionnaire in their preferred language.

• Try to format the cover letter so that it is only one page.

• If offering two languages, consider printing English on one side and the second language on the other.

92


Questionnaire—Requirements • Only CMS-approved translations of the ICH CAHPS Survey

questionnaire and the ICH CAHPS supplemental questions are allowed.

• Core items (Qs. 1–44) must be placed first in the questionnaire.• “About You” questions (Qs. 45–62) must be administered

together. • Questions and response options may not be split across pages.• Font size must be no smaller than 11 point.• No matrix formatting of the questions is allowed.• Vendors must be consistent throughout the questionnaire in

formatting response options either vertically or horizontally.• See guidelines for incorporating supplemental questions in the

Survey Administration and Specifications Manual.

93


Questionnaire—Requirements (cont’d)• The SID number must appear on at least the first page of the

questionnaire.

• The ICH facility name must be printed where indicated in the questionnaire.

• The OMB number must be printed on the questionnaire cover page or, if no cover page, on the first page of the questionnaire.

• The OMB disclosure notice must be printed on either the questionnaire or the cover letter.

• If the disclosure notice (with the OMB number included) is printed on the questionnaire, the OMB number must also appear separately from the OMB disclosure notice on the first page of the questionnaire.

• The vendor name and return address must be printed on the last page of the survey.

94


Questionnaire—Recommendations • Maximize the use of white space.

• Use a standard and easy-to-read font, like Arial or Times New Roman.

• Use a font size of 12 or larger.

• Use a two-column format, so there are two columns of questions per page.

• If keying is used, may include small numbers next to the response option boxes.

• If administering in different languages, consider putting questionnaires in both languages in envelope.

95

Mail-Only: Mailing Requirements

Mail Survey Envelopes

• Vendors are responsible for supplying:

• outgoing prenotification letter envelopes,

• outgoing questionnaire package envelopes, and

• business-reply envelopes for returning questionnaires.

96

Mail-Only: Mailing Requirements

Mailing Requirements• Mailings must follow the prescribed schedule.

• Vendors must verify mailing addresses using commercial address update services, such as NCOA. Vendors are permitted to obtain address information from ICH facilities for all patients served during the sampling window.

• Each questionnaire package must contain a personalized cover letter separate from the questionnaire, a questionnaire, and a postage-paid return envelope.

• A vendor or ICH facility cannot include promotional information or materials in the mail survey package.

• Vendors must send a package to all cases, even those without complete addresses.

• Vendors must attempt to reach homeless patients.

97

Mail-Only: Mailing Recommendations

Mailing Recommendations

• Vendors should attempt to obtain a new or updated address for any mail returned as undeliverable in time to use it in the second mailing.

• Prenotification letters and questionnaires should be sent using first-class postage or indicia, to ensure timely delivery and maximize response rates.

• Vendors should “seed” each mailing.

• Survey vendors have the option of including the CMS logo on the questionnaire envelope. The Coordination Team provides a copy of the CMS logo to survey vendors.

98

Mail-Only: Data Receipt and Scanning/Keying

Requirements

• All returned questionnaires must be marked with a date of receipt that will be entered into the data file for that case.

• Questionnaires must be logged into the tracking system in a timely manner.

• If two questionnaires are received from the same respondent, keep the one that has the more complete data.

• Questionnaires must be reviewed prior to scanning, including comments and notes.

• A final ICH CAHPS Survey disposition code must be assigned to each case.

99


Requirements (cont’d)

Scanning Requirements

• The scanning program must not permit scanning duplicate questionnaires.

• The scanning program must not permit out-of-range or invalid responses.

• A sample of questionnaires (minimum 10%) must be rescanned and compared with the original as a quality control measure.

100



Key Entry Requirements

• The key entry process must not permit keying of duplicate questionnaires.

• The key entry program must not permit out-of-range or invalid responses.

• All questionnaires must be 100% rekeyed by a different keyer for quality control purposes.

• A supervisor must resolve any discrepancies.

101



Scanning and Key Entry Requirements

• If the response marked falls between two answer choices, select the answer choice closest to the mark.

• If two responses are marked for same question, select the answer that appears darker.

• If a response is missing, leave the response blank and code as missing.

• Open-ended question responses are not to be included in the submitted data file.

102

Mail-Only: Staff Training

• All support staff must be trained on ICH CAHPS Survey protocols.

• Relevant sections of the manual must be made available, depending on staff roles.

• Staff must be trained on: • use of relevant equipment (scanning, case

management systems, data entry programs); and • decision rules and coding guidelines for returned

questionnaires (data receipt and data entry staff).

• Training must include proper handling and storage of paper and electronic data.

103

Mail-Only: Staff Training (cont’d)

• Toll-free customer support staff must be trained on:• responses to commonly asked questions,• how to respond to questions when they do not know

the answer, and• rights of survey respondents.

• Patients may call in with complaints about their ICH facility or ICH care. Staff should provide the patient with the name and telephone number of the ESRD Network that serves the state in which the patient lives.

104

Questions?

105

Telephone-Only Administration Procedures

106

Telephone-Only: Data Collection Schedule


• Prenotification letter must be mailed 3 weeks (21 days) after receiving the sample file.

• Begin telephone contact 15 calendar days after the prenotification letter is mailed.

• End data collection 12 weeks (84 days) after prenotification letter is mailed.

• Submit data to the Data Center by semiannual submission deadline (submit as early as possible prior to the deadline).

107

Telephone-Only: Telephone Interviewing Systems

Telephone Interviewing Systems Requirements

• To administer the ICH CAHPS Survey using a telephone-only data collection mode, vendors must use a CATI system.

• Paper-and-pencil administration is not permitted for telephone surveys.

108

Telephone-Only: Telephone Interviewing Systems

(cont'd)

Telephone Interviewing Systems Requirements (cont’d)

• Vendors must also have a survey management system to ensure that sample patients are called at different times of the day and across different days of the week.

• Predictive or auto-dialers are permitted as long as:

• they are compliant with FTC and FCC regulations, and

• respondents can easily interact with a live interviewer.

• FCC regulations prohibit auto-dialing of cell phone numbers. Therefore, cell phone numbers need to be identified in advance to allow the vendor to treat cell phone numbers in a way that complies with FCC regulations. It is the vendors’ responsibility to familiarize themselves with all applicable state and federal laws and abide by those accordingly in regard to calling cell phone numbers.

109

Telephone-Only: Telephone Interview Script

Telephone Interview Script• Vendors must use standardized telephone scripts for

the ICH CAHPS Survey. • The scripts include the introductory screens and the

survey questions. • Questions 1–44 are the “Core” ICH CAHPS questions.

Questions 45–59 are the “About You” questions.• Questions 60–62 in the mail survey are not included in

the telephone script.• No changes in wording are allowed in the ICH CAHPS

Survey questions or response options.

110

Telephone-Only: Telephone Interview Script (cont’d)

Telephone Interview Script

• The Core ICH CAHPS Survey questions must be administered first and in the order in which they appear.

• The “About You” questions may be placed either before or after any supplemental questions that are added to the questionnaire.

• Facilities/vendors may add their own facility-specific questions or the ICH CAHPS supplemental questions, following the guidance about adding supplemental questions.

• Only CMS-approved translations are permitted for the ICH CAHPS Survey (including the ICH CAHPS supplemental questions).

111

Telephone-Only: Contacting Guidelines

Requirements

• Vendors must attempt to contact every sample patient.

• Vendors will make a maximum of 10 telephone contact attempts for each sample patient, unless the sample patient refuses or the vendor learns that the sample patient is ineligible to participate in the survey.

• The 10 contact attempts must be made on different days of the week and at different times of the day spread over the 12 weeks of data collection.

112

Telephone-Only: Contacting Guidelines (cont’d)

• A telephone contact attempt is defined as one of the actions below:• The telephone rings six times with no answer or an

answering machine is reached.• The interviewer reaches someone other than the

sample patient and is told that the sample patient is not available to take the call.

• The interviewer reaches the sample patient and is asked to schedule a call-back at a later date.

• The interviewer gets a busy signal on two consecutive phone call attempts, spaced at least 20 minutes apart.

113


• Vendors may make more than one telephone attempt in one 7-day period but cannot make all 10 attempts in one 7-day period.

• Work on a case may be continued after 10 attempts if the tenth call attempt results in a scheduled appointment with the sample patient, as long as the appointment is within the data collection period.

• Vendors must continue to work every case in the sample until the maximum number of attempts has been made for each case.

114


Fast Busy Signals• Fast busy signals should be redialed immediately. • If the interviewer again receives a fast busy signal, the

interviewer should re-call the telephone number on a different day of the week and at a different time of the day than the initial calls.

• If the third call attempt again results in a fast busy signal, the vendor should apply the appropriate final disposition code to the case.

115


Out of Service Message• If the interviewer receives a recorded message

indicating the telephone number is “temporarily out of service,” the interviewer should redial the telephone number within 5 days after the initial call was made.

• If the second call attempt results in the same recorded message, the interview should call the telephone number a third time, 5 days after the second call attempt was made.

• If the third call attempt results in the same recorded message, the vendor should apply the appropriate final disposition code to the case.

116


Hang-Ups• If a sample patient hangs up immediately before or

while the interviewer is reading the introductory script, the case should be called again at a later point in time. That is, on a different day of the week and at a different time of the day.

• If the sample patient hangs up after the introductory script has been read to him or her, the interviewer should code the case as a refusal. That is, the vendor should not make any additional calls to that sample patient.

117


• Phone calls must be made at different times of the day and on different days of the week throughout the data collection period.

• Interviewers may not leave voicemail messages on answering machines or with a person who answers the phone.

• Vendors may tell a person who answers the phone that they are calling about a “study about health care” only.

• Vendors must be able to provide the Coordination Team with a call log indicating the date and time calls were made to each sample patient.

• If the vendor finds that a sample patient is ineligible for the ICH CAHPS Survey, the vendor must stop further contact attempts with that sample patient.

118


• If the respondent does not complete the interview on the first attempt, the vendor should make additional attempts to complete it.

• If a respondent does not feel up to participating in the telephone interview because of his or her medical treatment, make an appointment to conduct the interview at a better time.

• The vendor must be able to offer the interview to each sample patient in both English and Spanish, if Spanish is offered.

• Telephone survey data collection must end 12 weeks after the prenotification letter was mailed.

119

Telephone-Only: Contacting Difficult-to-Reach Sample

Patients

• Survey vendors must verify telephone numbers using a commercial address/telephone database service or directory assistance.

• Vendors may request updated telephone numbers from ICH facilities for all patients served during the sampling window.

• We strongly recommend that survey vendors attempt to identify a new or updated telephone number for any sample patient whose telephone number is no longer in service or who has moved.

120

Telephone-Only: Contacting Difficult-to-Reach Sample

Patients (cont’d)• If the telephone number is incorrect, the interviewer

may ask the person who answers the phone for a new telephone number for the sample patient.

• Interviewers must attempt to recontact the sample patient before the data collection period ends if the sample patient is temporarily ill, on vacation, or unavailable during initial contact.

• Vendors must attempt to contact homeless patients, if a telephone number is included on the sample file.

• Please note that survey vendors are responsible for screening any newly identified telephone numbers to identify if they are cell phone numbers as needed to maintain compliance with FCC guidelines.

121

Telephone-Only: Staff Training

• Telephone interviewer and customer support staff training must include teaching interviewers:

• how to establish rapport with the respondent,

• the content and purpose of the interview so that they can effectively communicate this information to the sample patient,

• to read the questions as they are worded,

• to speak only from script and to not provide additional information, and

• to maintain a professional manner and adhere to quality control standards.

122

Telephone-Only: Staff Training (cont’d)

• Training must also include teaching interviewers:• how to use effective neutral probing techniques, • to use the Frequently Asked Questions (FAQs) so that

they can answer questions in a standardized manner (see Appendix J),

• how to be sensitive to the patient or family member, and

• how to handle distressed respondents.

• Patients may have complaints about their ICH facility or ICH care. Staff should provide the patient with the name and telephone number of the ESRD Network that serves the patient’s state of residence.

123


Train Interviewers on Interviewing Techniques

• Do not read words that appear in ALL CAPITAL LETTERS to the respondent. This includes both questions and response categories (e.g., “DON’T KNOW,” “REFUSED”).

• Emphasize all bolded words in the question text.

• Ask the questions exactly as they are written.

• Ask questions in the exact order in which they are presented.

• Do not skip any questions.

124


Train Interviewers on Interviewing Techniques

• Do not change the order of the response options.

• Do not suggest answers to the respondent.

• Do not help the respondent answer the questions.

• If the answer to a question indicates that the respondent did not understand the intent of the question, repeat the question.

125


Train Interviewers on Probing Techniques

• Probe to obtain a more complete or more specific answer from a respondent.

• Repeat the question and answer choices if the respondent does not seem to understand.

• When probing, never suggest answers or lead the respondent.

• Encourage the respondent to give his or her best guess if he or she gives a “don’t know” response.

• Code an incomplete answer as “missing/don’t know” if after probing, the respondent cannot give a response.

126


Train Interviewers on Avoiding Bias• Interviewers should:

• remain neutral at all times during the interview;

• read all statements, questions, and responses exactly as they are written;

• use neutral probes that do not suggest answers (What do you mean?” “How do you mean?” “Tell me what you have in mind.” “Tell me more about....”);

• not provide their own personal opinions or answers in an effort to “help” respondents; and

• not use verbal cues, such as a cough or a yawn to influence the respondent’s answers.

127


Train Interviewers on Avoiding Refusals

• The first and most critical step in avoiding refusals is to establish a rapport with reluctant sample patients.

• Interviewers should:

• treat respondents the way they would like to be treated;

• always use an effective/positive/friendly tone and maintain a professional outlook; and

• listen as an ally, not an adversary, and not debate or argue with the respondent.

128


• Vendors must conduct an interviewer certification process, either oral, written, or both.

• The certification process should assess interviewers’ knowledge and comfort administering the survey and ability to answer respondent questions.

• Documentation of training and certification of all telephone interviewers and customer support staff and outcomes is subject to review during oversight visits.

129

Questions?

130

Mixed-Mode Administration Procedures

131

Mixed Mode: Data Collection Schedule


• Prenotification letters must be mailed 3 weeks (21 days) after receiving the sample file.

• Initial questionnaire with cover letter must be sent 15 calendar days after the prenotification letter is mailed.

• After 4 weeks, assign nonrespondents to the questionnaire mailing for telephone follow-up.

• Data collection must end 12 weeks (84 days) after the prenotification letter was mailed.

• Submit data to Data Center by the semiannual submission deadline (submit as early as possible prior to the deadline).

132

Mixed Mode: Survey Administration Guidelines

• Follow all guidelines for mail survey administration, but send only ONE questionnaire mailing instead of two.

• Follow all guidelines for telephone survey administration for the telephone follow-up portion of the mixed-mode implementation.

• The mixed-mode design cannot offer the telephone interview in Chinese or Samoan.

• For mixed-mode use, mail questionnaires should be mailed in the appropriate language. If Chinese or Samoan, telephone follow-up with nonrespondents should be attempted in English.

• If Chinese and Samoan sample patients cannot respond to the phone interview in English, the case should be given a final disposition code of language barrier.

133

Questions?

134

Confidentiality and Data Security

135

Confidentiality and Data Security Overview

• Assurances of Confidentiality

• Safeguarding Patient Data

• Physical and Electronic Data Security

• Important Reminders About PII and PHI

136

Data Confidentiality, Security, and Storage: All Modes

Assurances of Confidentiality

• The information provided is protected by the Federal Privacy Act of 1974.

• All ICH CAHPS project staff must sign affidavits of confidentiality.

• Survey responses will never be reported with a respondent’s name or other identifying information.

• Survey responses will be reported in aggregate and no ICH facility will see any respondent’s individual answers.

• Respondents can skip or refuse to answer any question.

• Participation in the study will not affect care or services respondent currently receives or expects to receive in the future.

137

Data Confidentiality, Security, and Storage: All Modes (cont’d)

Safeguarding Patient Data

• Patient data must be safeguarded.

• Follow HIPAA guidelines.

• Confidential data must be kept secure; limit access to authorized project staff only.

• All staff and subcontractors who might have access to confidential data should sign a confidentiality agreement.

• Establish procedures for handling data security breaches.

• No personally identifying information can be submitted to the Data Center—all files submitted to the Data Center must contain de-identified data only.

138


Physical and Electronic Data Security

• Electronic security measures may include:

• firewalls,

• Restricted-access levels, or

• password-protected access.

• Physical security measures may include:

• locked file cabinets, or

• locked or restricted-access rooms.

139


Physical and Electronic Data Storage• Paper copies of questionnaires must be stored in a

secure location and kept for 3 years unless electronic images of the questionnaires are being kept.

• Electronic images of paper questionnaires or keyed data, including telephone interview data, should be retained for 3 years, also in a secure location.

• Do not remove data files or questionnaires from the office environment.

• Data stored electronically should be backed up frequently to minimize data loss.

140

Data Security and Sample File Transmission

Important Reminders About Protecting PII and PHI Data

• Vendors are responsible for protecting all personally identifiable information (PII) and protected health information (PHI) of all in-center hemodialysis patients.

• PII and PHI should never be sent via e-mail without adequate security protection.

141

Data Security and Sample File Transmission (cont’d)

• To ensure confidentiality and maintain data security, sample files should be at a minimum encrypted and password-protected.

• Whether using File Transfer Protocol (FTP), Secure File Transfer Protocol (SFTP), or website protocols (HTTP or HTTPS), vendors must ensure that they transmit PHI and PII as securely as possible to any authorized subcontractors.

• PII or PHI should not be transmitted between vendors and their ICH facility clients for any reason.

142

Questions?

143

ICH CAHPS SurveyData Processing and Coding

144

Data Processing and Coding Overview

• Survey Disposition Codes

• Decision Rules and Coding Guidelines for Mail and Telephone Surveys

• Definition of a Completed Survey

• Computing the Response Rate

145

Survey Disposition Codes

• The ICH CAHPS Survey requires that vendors assign a final survey disposition code to each sampled case on XML data files submitted to the Data Center.

• Vendors are free to use their own internal interim or pending disposition codes to track the status of work on a case before it is finalized.

• Survey vendors must assign a final disposition code to each case to indicate the final result of work on case.

146

ICH CAHPS Survey Final Disposition Codes (cont’d)

Code Description110 Completed Mail Survey120 Completed Telephone Interview130 Completed Mail Questionnaire – Survey Eligibility Unknown140 Ineligible: Not Currently Receiving Dialysis150 Ineligible: Deceased160 Ineligible: Does Not Meet Eligibility Criteria 170 Ineligible: Language Barrier180 Ineligible: Mentally or Physically Incapacitated190 Ineligible: No Longer Receiving Care at Sample ICH Facility199 Survey Complete by Proxy Respondent210 Break-Off220 Refusal230 Bad Address/Undeliverable Mail240 Wrong, Disconnected, or No Telephone Number250 No Response After Maximum Attempts

147

Final Survey Disposition Codes (cont’d)

Code 110 and 120—Completed Survey (Patient is Eligible for the Survey)

• Code 110—Completed Mail Questionnaire

• Mail survey was completed; meets the completeness criteria

• Code 120—Completed Phone Interview• Telephone interview was completed; meets

completeness criteria

148


Code 130—Completed mail questionnaire, survey eligibility unknown

Assign Code 130 if:

• the answer to Q1 is “At home” or “not currently receiving dialysis” or the answer to Q2 is “Less than 3 months” or “No longer receives hemodialysis at this facility” or Q1 or Q2 are blank or Q1 and Q2 both indicate ineligibility,

AND

• the sample patient marked an answer to any one of the questions from Qs. 3–44.

149


Code 130—Completed mail questionnaire, survey eligibility unknown (cont’d)

• Mixed-mode mail cases that responded by mail must never be assigned for telephone follow-up, including cases for which Code 130 was assigned.

• Survey vendors must include survey response data from cases assigned Code 130 on the XML data file submitted to the Data Center.

150


• Before a final completed survey final disposition code can be assigned, survey vendors must determine whether:

• the patient is eligible for the survey based on the responses to Qs. 1 and 2;

and

• the completed questionnaire meets the completeness criteria.

151


Disposition Codes 140, 160, and 190—Ineligibility Determined

• Code 140—Ineligible: Not Currently Receiving Dialysis

• The sample patient’s answer to Q1 is “not currently receiving dialysis” and he or she does not answer any of the questions in Qs. 2–44.

• Code 160—Ineligible: Does not meet eligibility criteria

• The sample patient’s answer to Q1 is “At home” and he or she does not answer any of the questions in Qs. 2–44.

• The sample patient’s answer to Q2 is “Less than 3 months” and he or she does not answer any of the questions in Qs. 3–44.

152


Disposition Codes 140, 160, and 190—Ineligibility Determined (cont’d)

• Code 190—Ineligible: No Longer Receiving Care at Sample Facility

• The sample patient’s answer to Q2 is “No longer receives care at this facility” and he or she does not answer any of the questions in Qs. 3–44.

• Include survey response data from cases assigned Code 140, Code 160, and Code 190 on XML files submitted to the Data Center.

• Please note that if Q1 is blank, Q2 is option 5, and Qs 3–44 are blank, the case should be coded a 190.

153


Disposition Code 160• Code 160 will also be used to indicate that a sample

patient is ineligible to participate in the survey because the sample patient:• is under age 18,• is receiving hospice care, or• resides in a nursing home or other skilled nursing facility

or other long-term facility such as a jail or prison.

Note: Patients who reside in assisted living facilities are eligible to participate in the survey.

154


Other Disposition Codes to Denote Patients Ineligible for the Survey

• Code 150—Sample patient is deceased

• Code 170—Ineligible: Language barrier

• Assign if the sample patient does not speak one of the approved languages offered.

155


Other Disposition Codes to Denote Patients Ineligible for the Survey (cont’d)

• Code 180—Ineligible: Mentally or physically incapacitated

• Includes mental and physical impairments.

• Includes visually impaired for mail-only mode.

• Includes hearing impaired with no TTY service for telephone-only mode.

• Code 199—Survey Completed by Proxy Respondent (Mail surveys)

• The response to Q63 is “Answered the questions for me.”

156


Nonresponse (Disposition Codes 210, 220, 230, 240, and 250)

• 210—Break-off • Assign when fewer than 50% of core questions applicable to all

sample patients are answered (i.e., case does not meet completeness criteria).

• Do not assign Code 210 if the patient is ineligible because he or she receives dialysis at home, is not currently receiving dialysis, has not received dialysis care from sample facility for at least 3 months, or no longer receives dialysis care at the sample facility.

• 220—Refusal• Sample member indicates in writing or verbally that he or she

does not wish to participate.

157


• 230—Bad Address/Undeliverable Mail

• Use for mail-only mode.

• The address is not viable: The Coordination Team does not provide an address, vendor has attempted and failed to obtain a new address, the questionnaire is returned as “undeliverable, no forwarding address,” “addressee unknown,” or other similar Post Office return notices.

• Note that vendors are only permitted to ask ICH facilities to provide updated addresses and telephone numbers for all patients treated within the sampling window.

158


• 240—Wrong, Disconnected, or No Telephone Number

• For telephone-only or mixed mode.

• Assign this code if:

• the Coordination Team did not provide a telephone number or the number provided is disconnected, not working, or no longer belongs to the sample patient;

and

• the vendor has attempted but cannot obtain a new telephone number for the sample patient.

159


• 250—No Response After Maximum Attempts

• Assign to:

• Mail-Only Cases if address is viable but there is no response to the mail survey.

• Mail-Only Cases if a completed mail survey is received after the data collection period has ended.

• Telephone-Only Cases if a telephone number is viable but maximum number of call attempts (10) does not result in a completed interview or other final disposition code.

• Mixed-Mode Cases if address and telephone number are viable but questionnaire mailing and maximum call attempts (10) do not result in a completed interview or other final disposition code.

160

Vendor Review of Respondent Notes Included With Returned

Questionnaires

• Survey vendors must review all questionnaires returned for respondent notes and review the notes.

• Comments and notes written in the questionnaire or on separate paper included with the questionnaire may indicate whether the respondent is eligible to participate in the survey.

• Survey vendors must assign the applicable final disposition code if the note indicates that the sample patient is ineligible to participate in the survey.

161

Decision and Coding Rules for Multiple Responses

• Only one answer choice is accepted for most questions.

• If two or more answer choices are marked for a single-answer question:

• select the one that appears darkest OR,

• leave the response blank and code as “Missing” if it is not possible to determine the respondent’s answer.

162

Decision and Coding Rules for Multiple Responses (cont’d)

• If a response mark falls between two answer choices:• select the answer choice that is closest to the marked

response; OR• if the marked response is not clearly closer to one

answer choice, code as “Missing.”

163

Decision and Coding Rules for Multiple Responses (cont’d)

• Questions 59 and 62 are the only questions in the ICH CAHPS Survey questionnaire for which multiple responses are allowed.

• These questions have an instruction that asks the sample patient to check all answer choices that are applicable to him or her.

• For these questions, scan or key all answer choices that are marked. For all answer responses that are not chosen by the sample patient, the vendor should code as Not Applicable (Code X).

• If no answer choices are chosen, then all answer responses should be coded as Missing (Code M).

164

Decision and Coding Rules for Screening and Follow-up Questions

• Screening questions: Q1, Q2, Q20, Q23, Q37, Q41, Q42, and Q60.

• Enter the response provided by the respondent regardless of whether the response agrees with the screener question.

• If the screener question is left blank, code it as “Missing.”

165


(cont’d)

166


(cont’d)

• Q21, Q24, Q38, Qs. 42–44, Q57a*, Q58a*, Q59a*, Q59b*, Q61, and Q62 are follow-up questions.

• If the follow-up question is blank because the respondent correctly followed the skip instruction beside the response option marked in the preceding screening question, assign the “not applicable” code (Code X) to the response.

• If the follow-up question should have been answered but was left blank, assign Code M to indicate that the response is missing.

(*Included in telephone script only)

167


(cont’d)• For follow-up questions, enter the response provided by

the respondent regardless of whether the response agrees with the screener question.

168


(cont’d)• If the follow-up question is correctly left blank because

the answer marked for the preceding screener question was “No,” code the question as “Not Applicable.”

169


(cont’d)

• If the follow-up question is incorrectly left blank because the respondent skipped it, enter “Missing.”

170

Decision Rules for Coding Open-Ended Survey Items

• Questions 57, 61, and 62 have both preprinted and open-ended response options.

• Vendors are not required to scan or key any open-ended responses, but can do so if they prefer.

• Vendors must not submit open-ended responses to the Data Center. If scanned or keyed, the data must be removed before the XML file is submitted to the Data Center.

• Vendors may share responses to open-ended responses if answered by more than 10 sample patients at the facility AND no answers allow for identification of the respondent.

171

Decision and Coding Rules for Handling Returned Mail Surveys

Mail-Only Mode:• If the first questionnaire is returned blank, send

second questionnaire to sample patient as long as it is mailed before the end of the data collection period.

• If the first questionnaire is returned blank or never returned and the second questionnaire is returned blank, assign Code 220—refusal.

• If the first questionnaire is returned blank or never returned and the second questionnaire is never returned, assign Code 250—no response after maximum attempts.

172

Decision Rules for Coding and Handling Returned Mail Surveys

(cont’d)

Mail-Only Mode:• If a sample patient returns two completed questionnaires,

• use the one that has the more complete data;OR

• if both questionnaires have the same number of questions answered, use the questionnaire received first.

• Vendors must not use any surveys returned after the data collection period ends.

• Questionnaires received after the data collection period ends must be assigned Code 250.

173

Decision Rules for Coding and Handling Returned Mail Surveys

(cont’d)

Mixed Mode

• All mixed-mode cases that are not finalized as a result of the first questionnaire mailing must be assigned for telephone follow-up.

• Cases that are returned blank or as undeliverable mail should be assigned for telephone follow-up.

• If a sample patient returns a completed mail survey and participates in the telephone interview, use data from the telephone interview.

174

Definition of a Completed Survey

• A survey is considered complete if at least 50% of the core ICH CAHPS questions applicable to all sample patients are answered.

• The core ICH CAHPS questions that are applicable to all sample patients are:

• Q1–Q20,

• Q22–Q23,

• Q25–Q37, and

• Q39–Q41

• A list of the core ICH CAHPS Questions is included in Chapter IX of the ICH CAHPS Survey Administration and Specifications Manual.

175

Definition of a Completed Survey (cont’d)

• Step 1: Sum the number of core ICH CAHPS questions applicable to all sample patients that the respondent answered.

• As per ICH CAHPS protocols, “Don’t Know” and “Refuse” must be recoded to missing.

• Do not include a Missing response in the count of questions that the respondent answered.

• Step 2: Divide the total number of core questions answered by 38, which is the total number of core ICH CAHPS questions applicable to all sample patients, and then multiply by 100 to determine the percentage.

176


• Step 3: If the percentage is ≥ 50%, assign a final disposition code to indicate a “Completed Survey” (either Code 110 or 120, as appropriate).

• If the percentage is < 50%, assign the final disposition Code “210—Break-off.”

177


Determining Completeness (Example)

• A mail survey is returned to the vendor from the first questionnaire mailing. The respondent answered the following questions: Q1, Q2, Q3, Q4, Q5, Q6, Q7, Q8, Q9, Q10, Q11, Q12, Q13, Q14, Q15, Q16, Q17, Q18, Q19, Q20, Q22, Q23, Q25, and Q26. The other core questions that were applicable to all sample patients were left blank.

• Step 1: Count the number of questions answered: 24.

• Step 2: 24/38 * 100 = 63.2%.

• Step 3: Percentage is equal to or greater than 50%, so we assign disposition Code 110 to the case.

178

Computing a Response Rate

• Vendors are not required to compute a response rate.

• There are no penalties for low response rates.

• Final disposition codes used to calculate response rates are shown in parentheses.

# Surveys Completed (Codes 110 + 120)# Patients Sampled − # Ineligible Sample patients

(Codes 130–199)

179

Computing a Response Rate (cont’d)

Computing a Response Rate (Example) • An ICH CAHPS Survey vendor administers both mail

and phone surveys.• 109 patients were sampled • 68 completed surveys: 40 assigned Code 110 + 28

assigned Code 120 • 6 ineligible patients: 2 code 150s + 1 code 180 + 2

code 190s + 1 code 199

Completed Surveys (68)Patients Sampled (109) − Ineligible Patients (6) = 103

RR: 68/103 = 66.0%

180

Questions?

181

The ICH CAHPS

Website

182

The ICH CAHPS Website Overview

• Website Purpose and Functions

• Overview of the Website

• Access to Private Side—Secured Links

• Survey Vendor Access

• Facility User Access

• Keeping the Website Secure

183

ICH CAHPS Website Purpose

• The ICH CAHPS Survey website serves four purposes:• provides a central location to post announcements

about the ICH CAHPS Survey, • serves as a portal through which most project activities

will be conducted,• provides a secure way for vendors to download sample

files, and• is the location of the data submission tool for

submitting data.

• ICH CAHPS Survey website is located at https://ichcahps.org.

184

Website Functions

• The ICH CAHPS Survey website• provides general information about the survey,

including training information;• contains the survey questionnaire for mail and

telephone and related survey materials;• includes the ICH CAHPS Survey Administration and

Specifications Manual;• is used to distribute sample files to each survey vendor;• contains data file specifications and templates for data

submission; and• contains a model Quality Assurance Plan (QAP).

185

Website Functions (cont’d)

• Contains online forms, including:

• Training Registration Form,

• Vendor Registration and Application,

• Facility User and CCN Registration,

• Vendor Authorization Form,

• Exceptions Request Form,

• Discrepancy Notification Report, and

• Facility Closing Attestation Form.

• Contains the data submission tool and data submission reports for facilities and vendors.

186

Website Structure

• Public and private portals or sections.

• Public side contains links available to the public and does not require login credentials.

• Private side requires login credentials for access to all private side links and activities.

187

Website Home Page

188

Website LayoutPrivate Links

189

Survey Vendor Access to Private Links

Potential vendors must designate a staff member as the ICH CAHPS Survey Administrator.

Survey Administrators must:

• designate another staff member as a backup Survey Administrator, and

• serve as the main contact with the Coordination Team.

190

Survey Vendor Access to Private Links (cont’d)

Step 1: Complete the Vendor Registration Form.

• A confirmation e-mail will be sent to the Survey Administrator once the Registration Form has been completed.

Step 2: After completing the Registration Form, survey vendor is routed to the Survey Administrator’s personal dashboard on the website.

Step 3: The Survey Administrator can then complete and submit the Vendor Application to become a conditionally approved ICH CAHPS Survey vendor.

• A link to the Vendor Application is included on the dashboard.

191

Survey Vendor Access to Private Links: Survey Vendor

Dashboard

192

Facility Closing Attestation Form

• ICH CAHPS Survey vendors must submit the online Facility Closing Attestation Form if they learn that one of their ICH facilities has closed or will be closing.

• If the ICH facility closes or will be closing after the Coordination Team has provided a sample for that facility but before the data collection period begins, the survey vendor authorized to collect data on behalf of that facility must complete and submit the online Facility Closing Attestation Form as soon as possible after learning that the facility has closed or will be closing.

193

Facility Closing Attestation Form (cont’d)

• Vendors can access the Facility Closing Attestation form by logging into the ICH CAHPS website and choosing “Facility Closing Attestation” form which is under the For Vendorsmenu at the top of the website’s home page.

• Once a Facility Closing Attestation is submitted by a survey vendor, an e-mail will be sent to the facility’s ICH CAHPS Survey Administrator to notify the facility that the vendor has submitted the Facility Closing Form on the facility’s behalf.

• If an ICH facility closes after data collection activities have begun, the vendor must submit an XML file for the closed facility. The data file must contain survey data collected and a final disposition code must be assigned to each sample patient.

194

Survey Vendor Consent Form

• From the dashboard, the Survey Administrator can download and print the Consent Form.

• Completing the Consent Form indicates that the individual accepts the responsibilities of the ICH CAHPS Survey Administrator role.

• Purpose of the Consent Form

• Helps to validate the identification of the individual completing the form.

• Helps ensure that the individual completing the form is with the organization he or she claims to represent.

195

Survey Vendor Consent Form (cont’d)

• The vendor’s ICH CAHPS Survey Administrator mustcomplete the hardcopy version.

• The Consent Form must be signed and notarized.

• Mail the original completed hardcopy Consent Form to the Coordination Team.

196

ICH Facility User Access to Private Links

• ICH facilities must designate a staff member to serve as their Survey Administrator.

• The facility’s designated Survey Administrator must:

• complete the online Facility User Registration Form,

• complete and submit the ICH CAHPS Facility Consent Form, and

• designate another staff member as a backup Survey Administrator.

197

ICH Facility User Access to Private Links (cont’d)

• The ICH Facility Survey Administrator must (cont’d):

• if necessary, add non-administrator users,

• maintain/update user information and access, and

• serve as the main point of contact with the ICH CAHPS Coordination Team.

198

Facility User Access to Private Links (cont’d)

• A confirmation e-mail will be sent to the Survey Administrator once the Facility User Registration Form has been completed.

• The Survey Administrator will be directed to the custom dashboard.

• The dashboard will contain links to the major features available to ICH facilities.

199

Facility User Access to Private Links: Facility Dashboard

200

Facility User Access to Private Links (cont’d)

• Backup ICH CAHPS Survey Administrator

• It is important to designate someone within the organization as the backup Survey Administrator in case the primary Survey Administrator is not available.

• The backup Survey Administrator will have all the same permissions as the primary ICH facility Survey Administrator.

201

Facility Consent Form

• The Survey Administrator acknowledges that he or she accepts the roles and responsibilities of the Survey Administrator for the facility on the Consent Form.

• A printable/downloadable version of the ICH CAHPS Survey Facility Consent Form will be available on the website once the facility registration form has been completed.

• The facility must complete and submit a notarizedFacility Consent Form with original signatures.

• Mail the completed Facility Consent Form to the Coordination Team.

202

Website Security

• Facility and survey vendor staff should protect the security of the website.• Do not share or allow any staff to use the login

credentials of another staff member.• The designated ICH CAHPS Survey Administrator is

responsible for limiting user access to the appropriate access level to ensure security.

• The ICH CAHPS Survey Administrator or backup Administrator should remove or close access for any staff no longer working on the survey.

203

Questions?

204

ICH CAHPS SurveyFile Preparation and Data

Submission

205

File Preparation and Data Submission Overview

• Survey Vendor Authorization

• File Specifications

• Data Preparation and Submission Procedures

• Reports

206

Survey Vendor Authorization

• For the ICH CAHPS Surveys, vendors must be approved to submit any data.

• A list of approved survey vendors is currently available on the ICH CAHPS website.

• Facilities must authorize their survey vendor to submit data on their behalf by the deadline announced by the Coordination Team prior to each survey period.

• Facility users will log into the secure side of the ICH CAHPS website to authorize their survey vendor.

• Note: Facilities not wishing to change an existing survey vendor do not need to reauthorize the vendor for new survey periods.

207

Survey Vendor Authorization (cont’d)

• Select “Authorize a Vendor” under the For Facilities menu.

• Four options:• Authorize a vendor for the first time.• Change the start or end date for your current

vendor.• Change/switch to a different approved survey

vendor.• View current vendor authorizations for all facilities.

208


• Steps to Authorize a Vendor for the First Time:

• Select “Select a vendor for the first time” from the “Select Action” drop-down list.

• Select your vendor from the drop-down list.

• Select appropriate “Beginning Survey Period” from the drop-down list.

• Select the ICH facility(ies) to which this vendor authorization applies by checking the box next to each facility CCN/name.

• Click the “Submit” button.

209

Survey Vendor Authorization (cont’d):First-time Authorizations

210


• Steps to Change Survey Vendors:

• Select “Switch to a different vendor” from the “Select Action” drop down list.

• Select your new vendor from the drop down list.

• Select appropriate “Beginning Survey Period” from the drop-down list.

• Select the ICH facility(ies) to which this vendor authorization applies by checking the box next to each facility CCN/name.

• Click the “Submit” button.

211


• ICH facilities must authorize their survey vendor by the vendor authorization deadline.

• Each survey period will have a deadline for vendor authorization.

Survey Period Vendor Authorization DeadlineCY2016 Spring Survey February 28, 2016

CY2016 Fall Survey August 31, 2016

212


• The Coordination Team will not distribute an ICH facility’s sample to a survey vendor until the facility has authorized that survey vendor.

• Once the vendor authorization deadline passes, ICH facilities cannot change the survey vendor until after the current ICH CAHPS Survey data submission period ends.

213

Survey Vendor Authorization – Facility Non-Participation Form (cont’d)

• ICH CAHPS Facility Non-Participation Form

• To be completed in January/February each year if an ICH facility chooses not to administer the ICH CAHPS Survey.

• If not administering the ICH CAHPS Survey, existing vendor authorizations do not need to be updated.

• Steps to Complete Facility Non-Participation Form

• Enter CCN(s)

• Click check box to attest to Non-Participation Status

• The Coordination Team will not provide a sample for a CCN that submitted a Facility Non-Participation Form.

214


• Survey vendors can log into the ICH CAHPS website to view an updated Vendor Authorization Report.

• The report will show the name and CCN number of each facility that has authorized the survey vendor to download sample files and submit data on its behalf.

• Survey vendors should check their Vendor Authorization Reports regularly.

• Survey vendors should contact and urge their facility clients to complete the vendor authorization as soon as possible once a formal contract is in place.

215


Reminders• Sample files will not be available for download until the

facility has authorized the survey vendor.• Vendor authorizations must be completed before the

deadline for a given survey period.• Facilities should not select an End Date on the

Authorization Form unless they know that they will not be using this vendor for future survey periods.

• The online Facility Non-Participation Form must be completed by ICH facilities that choose not to administer the ICH CAHPS Survey for a survey period. The deadline for 2016 is February 28, 2016.

216

XML File Specifications

• Survey vendors will submit data using XML (Extensible Markup Language) files.

• Survey vendors can download and review the required XML template from the ICH CAHPS website under the Data Submission menu.

• Survey vendors will submit one XML file per ICH facility for each survey period.

217

XML File Specifications (cont’d)

• XML files consist of three sections:

• Header Record

• Patient Administrative Data Record

• Patient Response Record

218

XML Header Record

• All XML files MUST contain the header record.

• The header record MUST contain the following data variables:• Facility Name• Facility ID• Survey Period• Survey Year• Survey Mode• Number of Patients Sampled• Data Collection Period Start Date• Data Collection Period End Date

219

XML Patient Administrative Data Record

• There MUST be a Patient Administrative Data Record for every patient in the sample.

• Data variables in the Patient Administrative Data Record include:• Facility ID• Survey Period• Survey Year• Sample ID• Final Survey Status• Date Completed• Survey Language• Survey Mode

• If no survey was completed, enter 88888888 for Date Completed and X for Not applicable for Survey Language and Survey Mode.

220

XML Patient Response Record

• The Patient Response Record contains the recorded responses to each question in the questionnaire for an individual patient.

• All data variables in the Patient Response Record are required (missing/don’t know option is provided).

221

XML Patient Response Record (cont’d)

• There must be a Patient Response Record for every sample case assigned one of the following final disposition codes:

• 110 (completed mail questionnaire),

• 120 (completed telephone interview),

• 130 (completed mail questionnaire, survey eligibility unknown),

• 199 (survey completed by proxy respondent), or

• 210 (break-off).

• Most cases assigned to these codes will also have a Patient Response Record (unless coded from a call or note):

• 140 (ineligible: no longer receives dialysis)

• 160 (ineligible: does not meet eligibility criteria),

• 190 (ineligible: no longer receiving care at sample ICH facility).

222

When Not to Include Patient Response Data on XML

• Each XML file must contain a Header Record and a Patient Administrative Record for each sample patient at the facility.

• There are two situations when patient survey response data should not be included on the XML file:

• The survey vendor learns via a telephone call from a knowledgeable person or as a note on a completed questionnaire that a sample patient is deceased and someone else completed the survey (assign Code 150).

• The completed mail survey was received after the data collection period has ended.

223

Data Submission Deadlines

• Data files for all ICH facilities MUST be submitted before the submission deadline for that survey period.

• Submit your data as early as possible. Do NOT wait until the data submission deadline to begin uploading files.

• Survey vendors will submit one file per ICH facility for each survey period.

• If vendor has collected data for a facility that has closed, the vendor must submit the survey data collected for that facility to the Data Center.

224

Data Submission Deadlines (cont’d)

Data Submission Deadline for the CY2016 ICH CAHPS Spring Survey

11:59 PM Eastern Time

July 27, 2016

• ICH CAHPS Spring Survey Data Submission Deadlines:

• 11:59 PM ET on the last Wednesday of every July

• ICH CAHPS Fall Survey Data Submission Deadlines:

• 11:59 PM ET on the last Wednesday of every January

225

Data Submission Procedures

• Log into the ICH CAHPS Survey website.

• Proceed to the Data Submission Tool.

• Survey vendors can upload:• single XML file,• multiple XML files, or • multiple XML files in one ZIP file.

• Submit file using the Upload button.

• The system will run validation checks on each file as it is being uploaded and provide results via the Data Submission Report.

226

Data Submission Procedures (cont’d)

227

XML Data File Validation

• XML files will be validated immediately upon upload. The validation program will:

• check for valid vendor authorization,• check for required sections,• check for required data variables,• check for duplicate sample IDs, and• check existing data variables against valid ranges.

228

Data Submission—Key Points

• Do NOT alter the XML template, this will create data upload errors.

• Make sure you have downloaded and use the most current version of the Data Submission Validation Tool.

• Make sure you have the most current version of XML Schema Definition, or XSD, files (required for XML validation).

• Clearly name your XML files; include the facility’s CCN and the survey period in the file name; don’t use spaces.• Example: ICHCAHPS_201601_2016spring.xml

229

Data Submission—Key Points (cont’d)

• Be sure to check the Data Submission Reports.

• A file is not accepted until after it passes all validation checks.

• Survey vendors should submit at least 2 days before the data submission deadline to ensure that all files are accepted.

• If a survey vendor resubmits an XML data file, the system will overwrite the data from the file that was previously submitted.• If a file is resubmitted, the survey vendor must check

the data submission report to make sure that the file that was resubmitted was accepted.

230

Data Submission Process

231

Data Submission: Reasons Files May Be Rejected

Most common reasons that files may be rejected: • Values for some variables are out of range.• Data for some variables are left blank (a value must

be entered for every variable, even if it is Missing).• The survey vendor changed the XML name space. • The survey vendor resubmitted a file for a survey

period but the resubmitted file did not pass all validation checks, therefore it was not accepted.

• The survey vendor attempts to submit the file after the data submission deadline.

232

Understanding Data SubmissionReports: Overview

• Reports for Survey Vendors

• Data Submission Summary Report

• Survey Vendor Authorization Report

• Reports for Facilities

• Data Submission Summary Report

233

Reports for Survey Vendors:Data Submission Summary Report

• Available immediately after file upload process is complete.

• Provides transmission details on each file submitted.

• Confirmation of record count for files successfully uploaded.

234


(cont’d)

Error messages for files that fail validation checks:• XML file not properly formatted—rejects file immediately.• Missing values for required variables—report will list each

variable with missing values.• Invalid values—report will list any value that is outside of the

allowed range. • Duplicate records—report will list any sample ID that is

duplicated for given facility.• Missing response record values.• Patient response record does not pass the completeness

test.

235


(cont’d)

236

Reports for Survey Vendors: Survey Vendor Authorization Report

• Allows vendors to see which facilities have authorized them to upload data.

• Vendors are responsible for verifying that any facility with which they have contracted has authorized them.

• Report can be filtered by survey period.

237

Reports for Survey Vendors: Survey Vendor Authorization Report

(cont’d)

• REMINDER: The Coordination Team will not release sample files to a survey vendor if the facility has not completed the online Authorize a Vendor Form.

• The deadline for authorizing a vendor for CY2016 Spring ICH CAHPS Survey is February 28, 2016.

238

Report for ICH Facilities:Data Submission Summary Report

239

Questions?

240

Quality Control

241

Quality Control Overview

• Sample File Download Quality Control Guidelines

• Mail-Only Quality Control Guidelines

• Telephone-Only Quality Control Guidelines

• Mixed-Mode Quality Control Guidelines

• XML File Quality Control Guidelines

242

Sample File Download Quality Control Guidelines

REQUIRED Sample File Download Quality Control Procedures:

• Use the appropriate electronic equipment and software to securely download the sample file from the ICH CAHPS website.

• Ensure controlled access to the data (e.g., password protections, firewalls, data encryption software, personnel access limitation procedures, and virus and spyware protection).

243

Sample File Download Quality Control Guidelines (cont’d)

REQUIRED Sample File Download Quality Control Procedures (cont’d):

• Verify that the file contains a sample for all of your ICH facility clients. Report any discrepancies immediately to the ICH CAHPS Coordination Team.

• Verify that the number of patients for whom sample information is provided matches the number of patients indicated as having been sampled.

244


REQUIRED Sample File Download Quality Control Procedures (cont’d):

• If you received a sample file for a facility that you will not be collecting data for because of nonpayment issues, please alert the Coordination Team immediately.

• If you received a sample file for a facility that you have learned is closed or will be closing before data collection begins, delete all of the personally identifying information provided on the sample file for the closed facility, as per Item 6 of the vendor’s DUA with CMS.

• The vendor should also submit the online Facility Closing Attestation Form.

245


Recommended Sample File Download Quality Control Procedures:

• Store the sample files in an encrypted format at all times when not in use. Only decrypt the sample files when access to the patient information is required.

• Download the sample files as soon as they are available and before the download deadline date to ensure sufficient time to address any technical issues that may arise with the download process.

246

Mail-Only Quality Control Guidelines

REQUIRED Mail-Only Survey Administration Quality Control Measures:

• Verify mailing addresses using commercial address update services, such as NCOA.

• Maintain documentation that all staff involved with the mail survey implementation were properly trained on survey specifications and protocols.

• Check a minimum of 10% of all printed materials (questionnaires, prenotification letters, cover letters) to ensure print quality.

• Check a minimum of 10% of all outgoing questionnaire packages to ensure that package contents are correct and that the SID number on the questionnaire matches the SID number on the cover letter.

247

Mail-Only Quality Control Guidelines (cont’d)

REQUIRED Mail-Only Survey Administration Quality Control Measures (cont’d):

• Check a sample of cases to make sure the name and address on the cover letter match the name and address on the envelope.

• Make sure the number of questionnaire packages mailed matches the number of sampled cases.

248


Recommended Mail-Only Survey Administration Quality Control Measures

• “Seed” each mailing by including names of designated staff members to assess timeliness and completeness of questionnaire packages.

249


REQUIRED Mail-Only Data Processing and Submission Quality Control Measures:

• A sample of completed questionnaires (10% minimum) must be rescanned and compared with the original.

• All keyed questionnaires must be 100% reentered by a different staff member.

• A minimum of 5% of completed surveys must be reviewed to ensure that the coding rules were followed correctly.

250


Recommended Mail-Only Data Processing and Submission Quality Control Measures:

• Develop a way to measure error rates of both data receipt staff (recognizing marginal notes and passing these on to someone for review), data entry or scanning operators, and coders.

• Vendors should work with their staff to minimize error rates.

251

Telephone-Only Quality Control Guidelines

REQUIRED Telephone-Only Survey Administration Quality Control Measures:

• Verify telephone numbers using commercial telephone update services.

• Survey vendors must prepare and retain written documentation that all telephone interviewing and customer support staff have been properly trained.

• Copies of interviewer certification exam scores must be retained, as should documentation of any retraining required―these will be subject to review during oversight visits.

252

Telephone-Only Quality Control Guidelines (cont’d)

REQUIRED Telephone-Only Survey Administration Quality Control Measures (cont’d):

• Beginning shortly after the start of data collection, vendors must silently monitor a minimum of 10% of all interviews.

• Survey vendors are responsible for identifying and adhering to federal and state laws and regulations in states in which they will be administering the ICH CAHPS Survey.

• Survey vendors must establish and communicate clear telephone interviewing quality control guidelines for their staff to follow.

253


Recommended Telephone-Only Survey Administration Quality Control Measures:

• In addition to audio monitoring, an electronic system should be used to simultaneously observe the interviewer while conducting telephone interviews.

• Supervisors should provide performance feedback to interviewers as soon as possible after the monitoring session has been completed.

254


Recommended Telephone-Only Survey Administration Quality Control Measures (cont’d):

• Interviewers should be given the opportunity to correct deficiencies identified. Interviewers consistently receiving poor monitoring feedback should be removed from the project.

• Survey vendors should conduct regular Quality Circle meetings with telephone interviewing and customer support staff.

255

Telephone Survey Quality Control Guidelines (cont’d)

REQUIRED Telephone-Only Survey Data Processing and Submission Quality Control Measures:

• Survey vendors must compare the survey responses for a sample of cases included on the XML data files directly with the survey responses that were entered on the CATI system file, to ensure that the responses match.

256


Recommended Telephone-Only Data Processing and Submission Quality Control Measures:

• Survey vendors should generate and review frequencies of cases at the various pending and final disposition codes.

257

Mixed-Mode Quality Control Guidelines

REQUIRED and Recommended Mixed-Mode Quality Control Measures:

• All mail-only and telephone-only required and recommended quality control measures detailed on the preceding slides must be followed by all survey vendors administering the mixed mode.

258

XML File Quality Control Guidelines

REQUIRED XML Quality Control Measures• Use the XML Schema Validation tool to conduct initial

quality control on the XML file. • The schema validation tool is available on the ICH CAHPS

website under the “Data Submission” menu.

• Make sure information is included on the XML file for every sample patient included on the sample file provided by the Coordination Team.

• Make sure that the SID numbers included in the Patient Administrative Record on the XML file match the same set of SID numbers that were included on the sample file.

259

XML File Quality Control Guidelines (cont’d)

REQUIRED XML Quality Control Measures (cont’d):

• Make sure patient survey response data are matched to the correct patient.

• Make sure the appropriate final code (either a completed interview code or a partial data/break-off code) is assigned based on the results of the completeness criteria check.

• Compare the variables included in the Patient Survey Response section on the XML file to the hardcopy questionnaire (for mail surveys) or to the CATI file (for surveys completed by telephone) on a sample of cases.

260


REQUIRED XML Quality Control Measures (cont’d):

• Make sure questions that are appropriately skipped are correctly coded “X” for “Not Applicable,” rather than “M” for missing.

261


Recommended XML Quality Control Measures:

• Conduct a final check of the final disposition codes assigned to all sampled cases on the Patient Administrative Record.

262

Questions?

263

In-Center Hemodialysis CAHPS

Survey

Oversight Activities

264

Overview of In-Center Hemodialysis CAHPS Survey Oversight Activities

• Purpose and Description of Oversight Activities

• Quality Assurance Plan (QAP) Requirements

• Review of Submitted Data

• Site Visits and Corrective Action Plans

• Timing of Oversight Activities

265

Purpose of Oversight Activities

Purpose of Oversight Activities:

• Ensure compliance with ICH CAHPS Survey Standardized Protocols and Procedures.

• Ensure that survey data are complete, valid, and timely.

Description of Oversight Activities:

• Quality Assurance Plan (QAP) content requirements, submission requirements, and Coordination Team review of submitted QAPs

• Review of submitted data

• Site visits

266

Quality Assurance Plan Content and Requirements

• Vendors must submit a QAP, which describes how they will implement, comply with, and provide oversight of ICH CAHPS Survey data collection and processing activities.

• The QAP must provide all required information noted in the Model QAP, including:

• Organization Background and Staff Experience;

• Work Plan;

• Survey Implementation Plan;

• Data Security, Confidentiality, and Privacy Plan; and

• Questionnaire and Materials Attachments.

267

Quality Assurance Plan Content and Requirements (cont’d)

• The Model QAP is included in the ICH CAHPS Survey Administration and Specifications Manual and posted on the ICH CAHPS website.

The QAP must: • provide sufficient information to understand the

specific procedures you are using to implement the ICH CAHPS Survey, and

• include a copy of the formatted mail questionnaire and cover letters or a full set of the screen shots from the vendor’s electronic telephone interview system.

268

Quality Assurance Plan Submission Requirements

• Vendors must submit a QAP within 6 weeks of the vendor’s first submission of ICH CAHPS Survey data.

• Vendors must submit an update to the QAP annually (on or before March 9 each year) and whenever there is a change in key staff, a process, or vendor capabilities.

• All changes to previously approved QAP versions must be shown in track changes mode.

• Submit QAP via upload to the ICH CAHPS Survey website, at https://ichcahps.org.

269

Coordination Team Review of QAPs

• All QAPs will be reviewed by multiple ICH CAHPS Coordination Team members who independently evaluate each QAP and then meet to reach a consensus.

• A QAP may be accepted as is—that is, no revisions required—or the Coordination Team may request revisions, clarifications, or a complete rewrite of the plan.

• Vendors will be notified of the review outcome via e-mail.

• An accepted QAP is necessary for final approval as an ICH CAHPS vendor.

270

Review of Submitted Data

• The Coordination Team will review the quality of data submitted by each survey vendor and check for:• out-of-range and missing data,• unusual data patterns, or• unusual response rate patterns.

• The Coordination Team will notify the vendor of any ongoing data issues.

271

Site Visits

• Ensure compliance with ICH CAHPS Survey protocols.

• All approved vendors will receive site visits.

• The Coordination Team will review and observe facilities, systems, procedures, and documentation used to conduct the ICH CAHPS Survey.

• Subcontractors with significant roles will be subject to the same review as survey vendors.

• All information gathered during site visits and all vendor communications will be treated as confidential by all Coordination Team members.

272

Site Visit Procedures

Site visits will include the following activities:

• meetings with key project staff;

• physical walkthrough of vendor’s facilities/systems; and

• observation of applicable survey administration procedures, including activities occurring in real time, such as:• questionnaire production/assembly/mailing, • questionnaire receipt/coding, and• telephone interviewing/monitoring.

273

Site Visit Procedures (cont’d)

Site visits will also include a review of the following activities, procedures, and documents:

• process to download and store sample patient files;

• data processing activities, including how final disposition codes are assigned using the original SIDs;

• activities and procedures related to data file preparation and data file submission;

• documentation associated with survey activities; and

• data security and storage procedures.

274

Site Visit Procedures (cont’d)

• The Coordination Team will provide the vendor a Site Visit Report that includes all required vendor follow-up activities:• request for additional clarification or documentation, or• request a change in system or process.

• A timeline will be specified for when change must be implemented.

• The Coordination Team will follow up with the vendor via conference call/e-mail.

• The vendor may be subject to additional site visits, conference calls, or increased oversight.

275

Corrective Action Plans

• If a vendor fails to demonstrate adherence to ICH CAHPS Survey protocols and guidelines, the Coordination Team may implement a formal corrective action plan.

• A corrective action plan will provide vendors with specific tasks and task deadlines to show compliance with ICH CAHPS protocols and guidelines.

• Failure to comply with the corrective action plan may result in the vendor’s approval status being revoked or the affected ICH facility’s data not being publicly reported.

276

CY2016 Oversight and Other Activities

ICH CAHPS Survey Activity CY2016Submit and Review Vendor QAPs

Review Submitted Data

Conduct Vendor Site Visits

Technical Assistance (phone and e-mail)

Hold Protocol Review Conference Calls

Distribute ICH CAHPS Announcements

Hold Refresher Training (If Needed)

277

Questions?

278

Exceptions Request Formand

Discrepancy Notification Report

279

Exceptions and Discrepancy Process Overview

• Overview

• Exceptions Request Form

• Discrepancy Notification Report

280

Exceptions and Discrepancy Process Overview

• Vendors must complete and submit an Exceptions Request Form, on behalf of their ICH facility, for any planned deviations from the standard protocol.

• Vendors must complete and submit a Discrepancy Notification Report, on behalf of their ICH facility, to notify the Coordination Team of any unplanned deviation from standard protocols.

• Can be the result of actions by either the ICH facility or the vendor.

281

Exceptions Request Process

• Survey vendors must submit an Exceptions Request Form for any process or strategy not described in the ICH CAHPS Survey Administration and Specifications Manual.

• Any time there is a planned deviation, vendors must submit an Exceptions Request Form, on behalf of their ICH facility. Vendors must notify their ICH client facilities of the request.

• Must be submitted before planned deviation.

• The Exceptions Request Form can be accessed and submitted online at https://ichcahps.org/.

282

Exceptions Request Example

• Examples of exceptions requests would be:

• To use a commercial address/telephone database service or directory that differs from those listed in the project manual.

• To slightly reformat the mail survey.

• Examples of exceptions that will not be granted:

• A request to use any modes of survey administration other than those currently approved for the ICH CAHPS Survey.

• An ICH facility wanting to oversample its patients. CMS might allow oversampling in future survey periods.

283

Exceptions Request Review Process

• The Coordination Team will review all exceptions requests, evaluating the methodological strengths and weaknesses of the proposed approach.

• Depending on the type of exceptions request, the Coordination Team may request a conference call or site visit to review the vendor’s implementation procedures.

• The exception may be requested for multiple ICH facilities for which the vendor is collecting data.

284

Exceptions Request Review Process (cont’d)

• Vendors have 5 business days to appeal a denial of a request for an exception.

• To submit an appeal, the vendor must check “Appeal of Exception Denial” in box 1a of the Exceptions Request Form and update the form.

• The Coordination Team will review the appeal and return a final decision to the vendor within 10 business days.

285

Discrepancy Notification Report Process

• The Discrepancy Notification Report (DNR) is used to notify the Coordination Team of an unplanned deviationfrom standard protocols that will require some form of corrective action by the vendor.

• Example of scenarios requiring a DNR:• The vendor is unable to initiate data collection within 21

days after downloading the sample file.• A variable was incorrectly coded on the XML file and the

problem was not discovered until after the data submission deadline.

286

Discrepancy Notification Report Process (cont’d)

• The affected ICH facility must be informed of the DNR submission. The vendor must send a copy of the DNR that was submitted to the affected ICH facility.

• The DNR can be accessed and submitted online at https://ichcahps.org/.

• The vendor must notify the Coordination Team within 24 hours of discovery of the discrepancy by submitting a DNR.

287

Discrepancy Notification Report Process (cont’d)

The following information must be included on the online DNR form:

• The affected survey period (Spring or Fall Survey and the survey year)

• The reason the discrepancy occurred

• The affected CCN(s)

• Number of patients affected (if known or applicable)

• If a late start, the data collection start/end dates

• A detailed description of the discrepancy (for each CCN)

• Remediation plan and actions taken to try to avoid the situation in the future, including a proposed timeline for correcting the issue that caused the discrepancy

288

Discrepancy Report Review Process

• The Coordination Team will review the DNR to determine the following:

• What is the impact, if any, on publicly reported data?

• Is any additional information needed to document or correct the discrepancy?

• The Coordination Team will inform the vendor of the actions the vendor must take to correct the discrepancy.

289

Questions?

290

The ICH CAHPS Mode Experiment

291

The ICH CAHPS Survey Mode Experiment

• Purpose

• The Mode Experiment Sample

• Data Collection Results

• Analyses

• Results and Next Steps

292

Purpose of Mode Experiment

• The ICH CAHPS Survey mode experiment was conducted from January through April 2014.

• The purpose of the mode experiment was to

• determine if patients’ rating of their hemodialysis care varied based on the data collection mode (mode effects); and

• determine which if any patient characteristics affect patients’ rating of their hemodialysis care.

293

Purpose of Mode Experiment (cont’d)

• Adjustments to survey results may be needed to account for sources of bias that are outside the dialysis facility’s control.

• Potential sources of bias include data collection mode (mail-only, telephone-only, or mixed mode), patient mix factors, and response propensity.

• Mode experiment results will be used to statistically adjust ICH CAHPS Survey results that will be publicly reported.

294

Mode Experiment Sample

• The sampling window for the mode experiment was from July 1, 2013, through September 30, 2013.

• Data from CROWNWeb were used to construct the sampling frame.

• Patient survey eligibility was the same as that in the national implementation of the survey.

• A total of 11,893 eligible patients were sampled.

• Sampled patients were randomly assigned to one of the three data collection modes.

295

Mode Experiment Data Collection Results

• A total of 3,557 surveys were completed, including 1,355 for the mail-only mode, 994 for the telephone-only mode, and 1,208 for the mixed mode.

• The overall survey response rate was 33.6%.

• Response rates for mail-only and telephone-only modes were similar at 30.4% and 30.7%, respectively.

• Response rate for the mixed mode was 41.8%.

296

Mode Experiment Analyses

• Mode experiment data analyses were conducted in two stages.

• The first stage included the mode and patient mix analyses.

• The second stage included the nonresponse analyses and an assessment of whether the nonresponse analysis contributed explanatory power to the survey results beyond the explanatory power accounted for by the mode and patient mix analyses.

297

Mode Experiment Analyses (cont’d)

The analysis file included two types of variables:

• Dependent variables represented the outcome measures calculated from the ICH CAHPS Survey data.

• Dependent variables included three variables calculated from individual survey items and three variables calculated from multiple survey items.

• The independent variables represented mode and patient mix factors that may affect the dependent variables, and included 21 variables.

• Patient mix included health status, disease, and disability status, demographics, and dialysis treatment variables.

298

Mode Experiment Analyses (cont’d)

• Analysis methods for the mode and patient mix analysis included descriptive statistics, correlation analysis, and multivariate regression analysis.

• The multivariate regression models assessed the degree to which the dependent variables were affected by the mode and patient mix characteristics represented by the independent variables.

• Six linear, ordinary least squares multivariate regression models were estimated, including one model for each of the six dependent variables.

299

Mode Experiment Results

• The mode and patient mix analyses found 14 independent variables that were statistically significant in at least one of six multiple regression models.

• These 14 independent variables were identified for use as patient mix adjusters for the ICH CAHPS Survey results that will be publicly reported.

300

Mode Experiment Results (cont’d)

The 14 variables identified for use as patient mix adjusters are:

Mode of survey administration Difficulty dressing or bathingOverall health AgeOverall mental health SexHeart disease EducationDeaf or serious difficulty hearing Does the patient speak a language

other than English at homeBlind or serious difficulty seeing Did someone help the patient

complete this surveyDifficulty concentrating, remembering, or making decisions

Total years on dialysis

301


• The results of the nonresponse analysis logistic regression were that three of the independent variables were statistically significant predictors of nonresponse: age, diabetes as the cause of ESRD, and total years on dialysis.

• The correlation analyses of the nonresponse weights for survey respondents with the residuals from the mode and patient mix regression models found none of the correlation coefficients to be statistically significant.

302


• Mode and patient mix are sufficient for adjusting the ICH CAHPS outcome measures for public reporting.

• Nonresponse adjustments are not needed because the nonresponse analyses did not add any explanatory power beyond that provided by the mode and patient mix analyses.

• Survey data will be adjusted using the 14 independent variables included in the final mode and patient mix regression models.

303


Next Steps

• The Coordination Team will use data from the 2014 ICH CAHPS Survey to repeat some of the analysis conducted on mode experiment data.

• The purpose is to determine if the results will be the same using a larger sample size and if results remain constant over time.

304

The ICH CAHPS Survey

Questions?

305

Technical Assistance

For technical assistance on ICH CAHPS

By e-mail: [email protected]

By telephone: (866) 245-8083

By FAX transmission: (919) 541-7400

Please also check the ICH CAHPS website on a regular basis for news and updates about the ICH CAHPS Survey.

306

Thank You!

• As a final reminder, all designated ICH CAHPS Survey Administrators must log onto the ICH CAHPS website and complete the Training Certification.

• The Training Certification will be available on the website beginning tomorrow at 8:00 AM Eastern Time.

• Please complete the Training Session Evaluation Form before disconnecting from this Webinar session.

• Thank you for your participation in the ICH CAHPS Webinar training session.

307

Introduction to the In-Center Hemodialysis CAHPS Survey · • The webinar provider will mute your...

Documents

Transcript of Introduction to the In-Center Hemodialysis CAHPS Survey · • The webinar provider will mute your...