AAPOR 2016 - Robyn Rapoport - Cheaters vs. Good Samaritans

17
Cheaters vs. Good Samaritans: Predicting Who Will Cash Prepaid Incentive Checks AAPOR 2016 Presented by Robyn Rapoport, SSRS Vice President, Health Care, Public Policy & International Research COAUTHORS: Michelle Doty, The Commonwealth Fund Elizabeth Hamel, The Kaiser Family Foundation

Transcript of AAPOR 2016 - Robyn Rapoport - Cheaters vs. Good Samaritans

Cheaters vs. Good Samaritans:Predicting Who Will Cash Prepaid

Incentive Checks

AAPOR 2016

Presented by

Robyn Rapoport, SSRSVice President, Health Care, Public Policy & International Research

C O A U T H O R S :

Michelle Doty, The Commonwealth Fund

Elizabeth Hamel, The Kaiser Family Foundation

2

Outline

• Pre-incentives and Surveys of Physicians – Previous Research

• Background: Two Surveys with Primary Care Physicians

• Results

• Conclusions and Future Research

© ssrs | all rights reserved

3

Background

• Use of pre-incentives considered a “best practice” in physician research

− Associated with meaningfully higher response rates vs. promised incentives

− “Cheaters” are those who cash the pre-incentive but do not complete the

survey

− “Good Samaritans” are those who complete the survey but do not cash the

pre-incentive

• Understanding who is likely to cash check among those who complete and do

not complete survey is critical in determining the cost of conducting research

© ssrs | all rights reserved

4

US PCP Survey & International Health Policy (IHP) Survey

US PCP Survey IHP PCP Survey

Survey Mode

Mail recruit to self-administered paper

(12 pages – 49 questions) or

online survey

Mail recruit to self-administered paper

(8 pages – 51 questions) or

online survey

Population

US Primary Care Physicians,

stratified to oversample PCPs

working in low-income areas, etc.

US Primary Care Physicians,

nationally representative

(10 other countries also surveyed in this

research).

Sample Source SK&A SK&A

Weeks in Field 12 weeks – 2 wave design 14 weeks – 2 wave design

Sample Size 1,624 842

Pre-incentive $50 $25

Response Rate 34% 31%

© ssrs | all rights reserved

5

Procedures

US PCP Survey IHP PCP Survey

Initial MailingInvitation letter with web link,

paper survey, $50 incentive check

Invitation letter with web link,

list of past IHP publications

Second MailingCover letter with web link;

paper survey, $25 incentive check

Reminder Letter (s) Reminder letter with paper survey

Two reminder letters with paper survey

(Wave 1)

One reminder letter with paper survey

(Wave 2)

Postcard Reminders Two post card reminders Two post card reminders

Email Reminders Two email reminders Five email reminders (Wave 1)

Three email reminders (Wave 2)

Phone Reminders One phone reminder

© ssrs | all rights reserved

Results

7© ssrs | all rights reserved

Pre-Incentive Cashing Among All Survey Contacts

For both studies, approximately one third of the sample cashed the pre-incentive check;

a somewhat larger portion of the US PCP sample cashed

Cashed37%

Not Cashed63%

US PCP Study

Cashed33%

Not Cashed67%

IHP PCP Study

8© ssrs | all rights reserved

Pre-Incentive Cashing by Survey Mode Among All Survey Contacts

Fewer than one in ten cashed (‘cheated’) among those who didn’t complete;

the vast majority of those who completed also cashed their pre-incentive check

8%

85% 80%86%

92%

15% 20%14%

NotCompleted

TotalCompleted

WebComplete

Mail Complete

IHP PCP Study

Cashed Not Cashed

8%

91% 90%83%

93%

92%

9% 10%17%

7%

NotCompleted/Terminated

TotalCompleted

TotalComplete/Terminated

WebComplete/Terminated

MailComplete/Terminated

US PCP Study

Cashed Not Cashed

9© ssrs | all rights reserved

Completed Survey/Terminated Among All Survey Contacts

– Practice Characteristics

Highest completion rates among solo practitioners, pediatricians and FPs/GPs

28%

39%

27%

35%

36%

27%

33%

26%

32%

33%

35%

Multi (D)

PED (C)

IM (B)

FP/GP (A)

PED (C)

IM (B)

FP/GP (A)

9 + (D)

4 to 8 (C)

2 to 3 (B)

Solo (A)

Pra

ctice S

pecia

lty

Ph

ysic

ian

Sp

ecia

lty

Pra

ctice S

ize

IHP PCP Study

BCD

32%

40%

34%

38%

39%

31%

37%

30%

36%

35%

40%

Multi (D)

PED (C)

IM (B)

FP/GP (A)

PED (C)

IM (B)

FP/GP (A)

9 + (D)

4 to 8 (C)

2 to 3 (B)

Solo (A)

Pra

ctice S

pecia

lty

Ph

ysic

ian

Sp

ecia

lty

Pra

ctice S

ize

US PCP Study

BCD

B

B

BD

BD

B

B

BD

BD

ABCD denotes significant differences at the 95% confidence interval

10

US PCP Survey IHP PCP Survey

© ssrs | all rights reserved

Completed Survey/Terminated Among All Survey Contacts

– Email Availability

Somewhat more likely to complete if have email address in sample

34%

36%

Completed/Terminated

30%

32%

Completed

11

9%

5% 4% 3%4% 5% 6% 5%

Solo 2 to 3 4 to 8 9 or More

IHP PCP Study

Cheater Good Samaritan

7%5% 4% 4%3% 3% 4% 3%

Solo 2 to 3 4 to 8 9 or More

US PCP Study

Cheater Good Samaritan

© ssrs | all rights reserved

Cheater & Good Samaritan Profiles by Practice Size

Most cheaters among solo practitioners; similar proportions of Good Samaritans across

practice size groupings

A B C D E F G H

A B C D E F G H

CEG

CEG

ABCDEFGH denotes significant differences at the 95% confidence interval

12

6% 5% 4%4%2% 3%

FP/GP Internal Medicine Pediatrics

US PCP Study

Cheater Good Samaritan

6% 6%4%

6%3%

5%

FP/GP Internal Medicine Pediatrics

IHP PCP Study

Cheater Good Samaritan

© ssrs | all rights reserved

Cheater & Good Samaritan Profiles by Physician Specialty

Somewhat fewer Cheaters among Pediatricians vs. other specialties; ratio of Cheaters to

Good Samaritans is highest among Internists

ABCDEF denotes significant differences at the 95% confidence interval

A B C D E F

A B C D E F

D

DD

E

13

6% 5% 4% 5%4%2% 3% 4%

FP/GP Internal Medicine Pediatrics Multi-Specialty

US PCP Study

Cheater Good Samaritan

6%7%

5% 5%7%

2%5% 5%

FP/GP Internal Medicine Pediatrics Multi-Specialty

IHP PCP Study

Cheater Good Samaritan

© ssrs | all rights reserved

Cheater & Good Samaritan Profiles by Practice Specialty

Similar pattern – Ratio of Cheaters to Good Samaritans is highest in IM Practices

ABCDEFGH denotes significant differences at the 95% confidence interval

A B C D E F G H

A B C D E F G H

D

DD

D

D

14

4%6%

4%2%

Email Available No Email

US PCP Study

Cheater Good Samaritan

D

4%7%

5% 5%

Email Available No Email

IHP PCP Study

Cheater Good Samaritan

© ssrs | all rights reserved

Cheater & Good Samaritan Profiles by Email Availability

More cheaters among PCPs without an email address in both studies; more Good Samaritans

among PCPs with email addresses in US PCP study

ABCD denotes significant differences at the 95% confidence interval

A

A

A B C D

A B C D

15

70%

30%

Good Samaritans

77%

23%

Completed & Cashed

73%

27%

Completed/Terminated & Cashed

49%

51%

Good Samaritans

© ssrs | all rights reserved

Survey Mode Among PCPs Who Complete the Survey

– Cashed Check vs. Good Samaritans

Good Samaritans are more likely to complete the survey online; those who

cash pre-incentive check are more likely to complete the mail survey

Mail Complete or

Terminate

Web Complete

or Terminate

Mail Complete

Web Complete

Mail Complete or

Terminate

Web Complete

or Terminate

Mail Complete

Web Complete

US PCP Study

US PCP Study

16

Conclusions & Future Research

• PCPs who are sole practitioners are more likely to complete and also more likely

to cheat vs. PCPs in larger practices

• Cost/interview for Internists is higher than other PCPs - least likely to complete,

less likely to be a good Samaritan, more likely to cheat

Similar pattern for PCPs in Internal Medicine practices

• Best options for ‘saving’ money – make it easier to complete survey online

• Longer mail survey (more pages) may induce more to complete web survey

© ssrs | all rights reserved

contact uswith any questions

@RobynRapoport | 484-840-4354 | [email protected]

Robyn RapoportVice President, Health Care, Public Policy & International Research