Study Retention as Bias Reduction in a

Hard-to-Reach Population

Bruce Western∗

Anthony A. Braga

David M. Hureau

Catherine Sirois

August 2015

∗Department of Sociology, 33 Kirkland Street, Cambridge MA 02138. E-mail: western@wjh.harvard.edu. This research was supported by grant5R21HD073761-02 from NIH/NICHD, SES-1259013 from the National ScienceFoundation, a grant from the Russell Sage Foundation, and the Radcliffe Institutefor Advanced Study at Harvard University. We gratefully acknowledge the sig-nificant assistance of the Massachusetts Department of Correction who providedaccess to correctional facilities and advice and collaboration throughout the re-search. The data for this paper are from the Boston Reentry Study, a researchproject conducted by Bruce Western, Anthony Braga and Rhiana Kohl.

Abstract

Collecting data from hard-to-reach populations is a key challenge for re-search on poverty and other forms of extreme disadvantage. Even amongthe disadvantaged, men and women recently released from prison experi-ence relatively severe hardship that is closely associated with diminishedstudy participation. With data from the Boston Reentry Study (BRS), wedocument the extreme marginality of released prisoners and the challengesfor study retention and analysis. The BRS design and data yield threefindings. First, released prisoners show high levels of “contact insecurity,”correlated with social insecurity, in which residential addresses and con-tact information change frequently. Second, strategies for data collectionare available to sustain very high rates of study participation. Third, sur-vey nonresponse in highly marginal populations is strongly nonignorable,closely related to social and economic vulnerability. The BRS response rateof 94 percent over a one-year follow-up period allows analysis of hypothet-ically high nonresponse rates comparable to those in earlier studies. Highrates of nonresponse attenuate regression estimates in analyses of hous-ing insecurity, drug use, and re-incarceration. The analysis underlines theutility of small specialized data collections for highly disadvantaged popu-lations.

Extreme social and economic disadvantage poses a fundamental method-

ological challenge for sociological research. Deeply disadvantaged pop-

ulations are under-enumerated and at high risk of survey nonresponse

(Hunter, de la Puente, and Salo 2003; de la Puente 2004; Martinez-Ebers

1997). Thus studies of homelessness, severe poverty, and drug addiction,

for example, have developed special methods for data collection to include

those living outside of conventional households and to maximize study re-

tention (Desmond et al. 1995, Cottler et al. 1996, Faugier and Sargeant

1997).

Under current conditions of historically high incarceration rates, peo-

ple moving through America’s prisons and jails have become an impor-

tant population for research and policy (Travis et al. 2014). The formerly-

incarcerated often present a range of co-occurring disadvantages and vul-

nerabilities that are of key sociological interest, but that also make them

difficult to contact or unwilling to participate in research studies. As a

result, follow-up studies of the formerly-incarcerated have suffered from

high rates of attrition that are likely correlated with outcomes of key inter-

est. Becky Pettit (2012) called the incarcerated population “invisible men”

because of their under-enumeration in social survey estimates of socioeco-

nomic well-being. Because of their deep marginality in social and economic

life, this invisibility in social science data collection also extends to those

who have moved from incarceration to communities.

Recent longitudinal studies of the formerly-incarcerated suffered from

30 to 60 percent attrition over a one to two year follow-up period (e.g.,

Nelson et al. 1999, Visher et al. 2004). In contrast, the Boston Reentry

Study (BRS) followed Massachusetts state prisoners through the first year

of prison release and achieved a retention rate of 91 percent. This paper

1

uses data from the BRS to explore three main challenges for longitudinal

data collection among the formerly-incarcerated.

First, we document extreme social insecurity in the BRS sample that

is closely associated with a high level of contact insecurity—the frequent

changing of residential addresses and phone numbers that makes nonre-

sponse more likely. Second, we describe a variety of strategies for data

collection that yield a high rate of study participation. Third, we show that

the risk of survey nonresponse is associated with deep social disadvantage

including histories of addiction and mental illness. In this setting, non-

response contributes to a loss of statistical precision in data analysis, and

nonresponse is nonignorable, biasing estimates of risk and vulnerability in

the population. Exploiting the high rate of study participation in the BRS,

we explore the sensitivity of sample quantities to survey nonresponse pro-

viding evidence of selection bias in analyses of housing insecurity, drug use,

and re-incarceration.

Our analysis focuses on the formerly incarcerated but the methods and

findings reported here are generally relevant to the study of other hard-to-

reach populations that are unstably housed, difficult to contact, or other-

wise weakly attached to conventional households. Although large house-

hold surveys have been central to the study of poverty and related topics,

our results suggest the utility of specialized, and often small-scale, data

collections and methods for highly marginal populations that fall outside

of the scope of conventional survey methods. We discuss the implications

of these designs for research on extreme poverty and other severe disad-

vantage below.

2

1 RELEASED PRISONERS AS A HARD-TO-REACH POPULATION

Hard-to-reach populations present a variety of challenges for data collec-

tion. Marginalized groups, such as the homeless, drug users, or sex work-

ers, are significantly under-represented in conventional household surveys.

Researchers have developed special methods for recruitment and analysis

involving, for example, snowball and respondent-driven sampling (Faugier

and Sargeant 1997; Salganick and Heckathorn 2004). Even where vulner-

able populations can be sampled, attrition and other survey nonresponse

pose serious threats in follow-up studies (Farrington et al. 1990; Hough

et al. 1996). Researchers report low rates of follow-up among drug users

(Desmond et al. 1995; Cottler et al. 1996; Messiah et al. 2003), low-income

minority parents (Martinez-Ebers 1997; Teitler et al. 2003), women at risk

of HIV infection (Brown-Peterside et al. 2001), the homeless mentally ill

(Hough et al. 1996), and youth engaged in crime and delinquency (Farring-

ton et al. 1990; Cotter et al. 2002). In research on the homeless mentally

ill, for instance, study recruitment rates are often quite high—from 80 to

90 percent of targeted individuals—but retention rates can be significantly

lower, from 30 to 80 percent over six months to two years of follow up

(Hough et al. 1996).

Among vulnerable populations, prison releasees are unusually disad-

vantaged often contending with multiple risks and adversities. Besides

their involvement in serious crime, people who serve time in prison suf-

fer from high rates of mental illness and drug addiction. After prison re-

lease, the formerly incarcerated are often homeless or insecurely housed

(Metraux, Roman, and Cho 2007; Travis 2005) and they are more likely to

reside in group quarters—in shelters or other transitional housing—rather

3

than in conventional households (Sirois 2015). Those with outstanding

warrants may be on the run, evading both researchers and police. Employ-

ment is also unstable and frequently undocumented, and social programs

are under-used. The formerly-incarcerated are thus weakly connected to

mainstream social institutions and as a result they are often inaccessible to

standard data collections using surveys or administrative records (Kornfeld

and Bloom 1999; Harding et al. 2014).

High rates of nonresponse reduce the precision of estimates and may be

a source of bias. With the small sample sizes typical of the specialized data

collections for hard-to-reach populations, even low rates of nonresponse

yield a large increase in sampling variance. Where survey nonresponse is

related to the values of variables of key interest, nonresponse is called non-

ignorable and can be a source of selection bias (Little and Rubin 1987, 15).

Say drug users are difficult to contact for interviews and substantive inter-

est focuses on drug use after incarceration for respondents with a history of

addiction. More formally, for respondent i at wave t , drug use measured by

yi t can be written as a function of xi , a dummy variable indicating drug ad-

diction that is measured at baseline and is completely observed. Addiction

may also be a risk factor for nonresponse.

The regression of interest is,

yi t =β0 +β1xi +ei t . (1)

where ei t is a random error. To see the effects of nonresponse we write a

response indicator, Ri t , that equals 1 if a survey is completed and 0 in the

case of nonresponse. A sample selection model writes

Ri t = 1 if ηi t ≥ 0

0 if ηi t < 0

4

where ηi t describes the respondent’s propensity to be observed,

ηi t = δ0 +δ1xi +δ2zi +ui t , (2)

zi is another completely-observed covariate predicting nonresponse, and

ui t is a random error. The dependence of nonresponse on the level of

drug use yi t is usually modeled by allowing the errors ei t and ui t to be

correlated. Note that equation (1) cannot be fit directly because yi t is un-

observed when Ri t = 0. Analyzing just the observed data, the conditional

expectation of the dependent variable depends on covariates and an ad-

justment factor reflecting the predicted probability of being observed,

E(yi t |xi , zi ,Ri t = 1) =β0 +β1xi +h(δ0 +δ1xi +δ2zi ), (3)

where h is a function whose form depends on assumptions about the bi-

variate distribution of ei t and ui t . The adjustment factor, h, in equation (3)

is common to models for sample selection bias (e.g., Fitzgerald et al. 1998;

Heckman 1979).

In the naive regression of the observed values of drug use on addiction—

a regression of yi t on xi —bias results from the correlation between the co-

variate, xi , and the residual which includes the omitted term, h. Below,

we assess the magnitude of selection bias in a (near) completely observed

data set reporting estimates that exclude observations that are at high risk

of nonresponse. The key intuition is that if drug users with a history of

addiction are difficult to contact for interviews, the true level of drug use

among drug addicts will be understated by the observed data and the re-

gression coefficients will be under-estimated. The attenuation of regres-

sion relationships is a common product of nonrandom sample selection on

dependent variables, though different patterns of nonresponse might also

cause regression coefficients to be over-estimated.

5

In the study of marginal populations, risks and adversities such as drug

use, criminal involvement or housing insecurity may all contribute to at-

trition from panel surveys and are also outcomes of key interest. In these

cases, nonignorable nonresponse biases sample statistics but can be diffi-

cult to diagnose in the absence of additional information such as a refresh-

ment sample or population statistics for comparison (Hirano et al. 2001).

In sum, besides the widely acknowledged problem of undercount in highly

disadvantaged populations, risk of nonresponse is likely confounded with

quantities of key interest and is a source of bias in data analysis.

2 ATTRITION AND RETENTION IN STUDIES OF RELEASED PRISONERS

Table 1 lists nine studies since 1999 that have collected data from samples

of newly-released prison and jail inmates. Most of these so-called reentry

studies have sampled prison releasees and follow-up periods have generally

extended from a month to two years. Most studies sampled from cities, al-

though the large-scale Serious and Violent Offender Reentry Initiative con-

ducted interviews in 14 states. Sample sizes in reentry studies are typically

relatively small, in the hundreds. The challenges of study participation are

reflected in the retention rates at exit interviews. In the earliest study, the

Vera Institute’s First Month Out, only around half of a sample of prison and

jail inmates released in New York City were re-interviewed after 30 days.

Most reentry studies record retention rates between 50 and 70 percent.

The Michigan Reentry Study provides an important exception, retaining 86

percent of a small sample of parolees over a two-year follow-up period.

The final study in Table 1, the Boston Reentry Study, forms the focus

of this paper. Following a sample of Massachusetts state prisoners over a

6

Table 1. Retention rates of longitudinal studies of released prisoners.

Sample Follow-Up Retained atYear(s) Site Size (months) Exit (%)

1. First Month Out 1999 New York 88 1 56.02. RH Maryland 2001–03 Baltimore 324 6 32.13. RH Illinois 2002–03 Chicago 400 16 49.54. RH Ohio 2002–03 Cleveland 424 12 69.05. RH Texas 2004–05 Houston 676 8–10 55.96. SVORI 2004–07 14 states 2391 15 68.57. Michigan Reentry 2007–09 SE Michigan 22 24 86.48. Smartphone Project 2012–13 Newark 152 3 70.09. Boston Reentry Study 2012–14 Boston 122 12 91.0

Note: RH=Returning Home; SVORI=Serious and Violent Offender Reentry Ini-tiative. References are as follows: (1) Nelson et al. (1999), (2) Visher etal. (2004), (3) Kachnowski (2005), (4) Visher and Courtney (2007), (5) LaVi-gne et al. (2009), (6) Lattimore and Steffey (2009), (7) Harding et al. (2014), (8)Sugie (2014). First Month Out interviewed prison and jail releasees. ReturningHome and the Michigan Reentry Study interviewed former prisoners. SVORI in-terviewed former adult prisoners and juveniles. The Newark Smartphone Projectinterviewed parolees.

7

Table 2. Study participation across five waves of the Boston Reentry Study.

MedianDays from IQR of

Number of Response Respondents Release to Days toInterviews Rate (%) Attritted (N) Interview Interview

Baseline 122 100.0 − −8 111 week 117 95.9 0 7 32 month 113 92.6 2 64 86 month 113 92.6 4 186 1712 month 111 91.0 5 373 29Total (N) 576 94.4 11 − −

Note: Survey nonresponse includes all those who could not be contacted or sched-uled for a follow-up interview plus those unreachable through incarceration orhospitalization as a percentage of those eligible to be interviewed. Attrition ina given wave is defined as missing the current interview and all subsequent in-terviews. The two-month interview count includes one respondent who was ad-ministered a re-incarceration interview in prison. The six-month interview countincludes six respondents who were given re-incarceration interviews in prison.

year after prison release, the BRS achieved a retention rate of 91 percent.

The study consisted of a baseline interview one week before prison release,

a follow-up interview one week after release, then further interviews at

two months, six months, and twelve months after release. With a baseline

sample size of N = 122, and 5 scheduled interviews for each respondent,

the initial design included 610 interviews. A total of 576 interviews were

completed yielding an overall response rate of 94.4 percent over a year of

follow-up.

Table 2 provides more detail about the pattern of nonresponse in the

BRS. Nonresponse increased over time as 5 out of 122 respondents were

missed at the first follow-up survey and 11 respondents were missed at the

12-month exit interview. Only a minority of nonrespondents were drop-

outs in the sense of being permanently lost to follow up. Although the

8

study sustained a high response rate, data on the days to interview sug-

gest the increasing difficulty of completing interviews. The median days

to interview shows that half the sample were interviewed on the sched-

uled follow-up day or earlier. Latecomers, at the 75th percentile of the

days-to-interview distribution, got progressively later even though nonre-

sponse did not markedly increase after the two month follow-up. The full

distribution of the days to interview for each survey wave is shown in Fig-

ure 1. By the 12-month interview, a quarter of the sample were interviewed

at least a month later, and 10 percent of the sample were interviewed at

least ten weeks late. Groves and Couper (1998) distinguish between non-

respondents and “reluctant respondents” who are either difficult to contact

or hesitant to consent to an interview. We can think of the latecomers, in

the final quartile of the time-to-interview distribution, as reluctant respon-

dents who are at risk of nonresponse without additional efforts at study

retention. Latecomers and nonrespondents consist mostly of those who

were who were just intermittently missing or late for 1 or 2 waves (85 per-

cent) and a smaller number who were persistently late or missing in at least

3 of the 4 waves (15 percent).

3 STUDY RETENTION IN A COHORT OF PRISON RELEASEES

The BRS was designed to collect data intensively in the early stages of

prison release. Respondents were recruited to the study with an informa-

tion sheet distributed by prison staff to prisoners who were approaching

release and planned to live in Boston. Respondents who were interested to

participate were scheduled for an interview about a week before release.

There appears to be little nonrandom selection on observable characteris-

9

0 100 200 300 400 500 600

0.00

0.05

0.10

0.15

0.20

0.25

Days from Prison Release

Den

sity

1 week

2 month

6 month

12 month

Figure 1. Density plot showing time to interview from prison release. Interviewswere scheduled for one week, two months, 6 months and 12 months after prisonrelease, Boston Reentry Study (N = 122).

10

tics into the study sample. The offense profile, criminal history, level of risk

and demographic characteristics of the BRS sample are similar to those for

the Boston-area reentry population as a whole. After release, rates of rear-

rest and reincarceration in the BRS sample are also similar to those for the

whole Boston reentry population (Western et al. 2015). Study recruitment

varies less across respondent characteristics than across correctional facili-

ties where implementation of the research protocol varied significantly. In

some facilities, prison staff showed a keen interest in the research and were

active in their efforts to distribute information about the study. The BRS

respondents represent a one-quarter sample returning to the Boston area

from Massachusetts state prisons.

Following recruitment, the research design consisted of five face-to-face

interviews over a one-year follow up period. The baseline interview was

conducted in prison by a university researcher with a staff member from

the research unit at the Department of Correction (DOC). At the baseline

interview, contact information was obtained for the first follow-up inter-

view two weeks later in the community. All follow-up interviews were

conducted face-to-face by university research staff who worked in pairs.

If a respondent was re-incarcerated, follow-up face-to-face interviews were

conducted inside correctional facilities.

3.1 Contact Insecurity and Social Insecurity

How did the BRS sustain such a high response rate? Characteristics of the

BRS sample, as in other surveys of released prisoners, place them at high

risk of interview nonresponse (Table 3). The sample is entirely urban, liv-

ing in and around Boston, mostly male, African American and Hispanic

with very low levels of schooling. In addition, a large proportion of the

11

Table 3. Means and percentage distribution for selected characteristics of BostonReentry Study respondents (N = 122).

DemographicsAverage age (years) 36.5Female 12.3Black 50.1Hispanic 18.9HS dropout 59.8

Baseline characteristicsSubstance abuse history 54.1Mental illness history 44.3Average time in prison (months) 32.4Unemployed before arrest 41.0Unstable housing before arrest 19.7

Post-release characteristicsNo probation/parole after release 38.5Incarcerated after release 22.1Unstable housing after release 62.0

respondents have histories of mental illness or heavy drug use, and had

previously lived in unstable housing, either dividing time between multi-

ple residences, living on the streets, or in shelters or other group quarters.

In contrast to most prison reentry studies, BRS respondents were not re-

cruited from probation or parole agencies and over a third of the sample

was released to the street with a completed sentence and no post-release

supervision. After release, nearly a quarter of the sample returned to in-

carceration in the one-year follow-up period and nearly two-thirds lived

in some kind of unstable housing. In short, like most newly-released pris-

oners, the BRS respondents were socially unstable and often difficult to

contact for follow-up interviews.

A direct sense of the difficulties of follow up is given in Table 4. As we

detail below, respondents’ cell phones were the main means of contact for

check-ins and scheduling interviews. At each wave of the survey, 5 to 7 per-

12

Table 4. Change in phone contact, residence, or criminal justice status at 1 week,2 months, 6 months, and 12 months after prison release, Boston Reentry Study(N=122).

Release to 1 week to 2 months 6 months to1 week 2 months 6 months 12 months

No phone (excl. incarcerated) 5.0 6.7 4.5 17.3Changed phone 56.8 44.8 42.4 52.3Unstable/unknown residence 39.3 38.5 44.3 53.3Changed residence 40.2 34.7 49.1 57.9New charge/arraignment 0.0 5.7 9.8 27.9Entered prison or jail 0.8 0.8 7.4 13.9

Note: Unstable residence includes staying in multiple residences, treatment pro-grams, transitional housing, shelters, correctional facilities, and homeless or on thestreet. Data on new charges and prison or jail stays is drawn from administrativerecords and thus is complete for the whole sample.

cent of respondents did not have working phones and 40 to 50 percent of

respondents changed phones between survey waves. In addition to being

hard to reach by phone, 40 to 60 percent of respondents frequently changed

residence and around half were living in unknown or unstable residences.

Criminal justice involvement may also contribute to nonresponse for re-

spondents with outstanding warrants or who are detained while awaiting

trial. The level of criminal justice involvement increased over time. In the

final six months of the follow-up period, over a quarter of the BRS sample

were arrested on new charges and 14 percent were re-incarcerated. When

contact information—chiefly phone numbers and residential addresses—

is intermittently unavailable or changes frequently, respondents exhibit a

high level of contact insecurity that makes survey nonresponse more likely.

13

3.2 Strategies for Study Retention

In this challenging setting for study retention, a variety of strategies were

adopted to reduce nonresponse. In a context of extreme contact insecu-

rity, we collected an extensive array of supplementary contact information,

maintained a high level of contact between interviews, and tried to increase

the willingness of respondents to participate. Four specific features of the

BRS research design were designed to maintain study retention:

1. Interview incentives. BRS respondents were paid $50 for each in-

terview. Incentives were previously found to increase participation among

low-income respondents and parolees (Martin et al. 2001; Harding et al. 2014).

Following Harding and his colleagues (2014), BRS respondents were paid

in cash at the conclusion of an interview (Harding et al. ). In the case of

re-incarceration, research staff deposited $50 into the respondent’s prison

commissary account. Besides the cash incentives, interviewers were trained

to present a strongly nonjudgmental posture in interactions with respon-

dents. Respondents reported that this approach contrasted with official

interactions with criminal justice authorities, and that the attitude of inter-

viewers provided an additional incentive for participation.

2. Phone check-ins and letters. Research staff also conducted regular

phone check-ins with study respondents. Between the baseline, 1-week,

2-month, and 6-month interviews, interviewers phoned respondents every

few weeks. Between the 6-month and the 12 month waves, interviewers

checked in by phone each month. Phone check-ins were used to update the

respondents’ residential information and to maintain contact and rapport.

Phone contact involved both conversations and texting. Where phone con-

tact was lost, we emailed or wrote letters to the respondents, often using

address information previously provided by family members and friends.

14

A strongly nonjudgmental approach in interviews combined with frequent

phone check-ins were intended to build rapport and trust between inter-

viewers and respondents.

3. Proxy interviews and secondary contacts. Proxy interviews were con-

ducted with close family and friends who could also help us locate hard-to-

find respondents. In the Fragile Families Study of Child Well-Being, mother-

father couples were interviewed and proxy responses by mothers compen-

sated for a high rate of nonresponse among formerly-incarcerated fathers

(Lopoo and Western 2005). We elaborated this approach by obtaining at

the baseline interview a list of secondary contacts to be used to help lo-

cate respondents after prison release. The contact list was updated at each

subsequent interview. Because we expected proxy respondents to provide

useful information about the respondent’s family contacts and well-being,

we also aimed to conduct at least one substantive interview with a family

member or close friend for each focal respondent. We completed 81 proxy

interviews.

4. Justice agencies and community contacts. When conventional reten-

tion strategies were exhausted, a variety of justice agencies and community

partners helped re-establish and maintain contact with the study subjects.

The DOC provided weekly updates on the criminal justice status of all re-

spondents and monitored whether respondents returned to state custody.

For subjects under criminal justice supervision in the community, the Mas-

sachusetts probation and parole agencies—and in a few cases, the Boston

Police Department—helped locate subjects for interviews. Parolees have

been the focus of earlier reentry studies and below we study whether pa-

role/probation status is associated with study retention. For those who

were not on probation or parole supervision, we tried to reestablish contact

15

through a variety of street and community workers operating in inner-city

neighborhoods where study respondents resided.

In addition to the four main strategies for study retention, survey records

were also linked to administrative records on arrests, charges, sentencing,

and incarceration. Although not a strategy for study retention, record link-

age yields a completely-observed record of criminal justice involvement for

all respondents for the entire follow-up period. Besides providing a com-

plete criminal justice history for the sample, administrative records help

us to explore nonrandom survey nonresponse. Differences in arrest or in-

carceration rates between respondents and nonrespondents may suggest

nonrandom differences for survey variables for, say, self-reported crime,

drug use, or illegal income.

3.3 Day-to-Day Operations

In the day-to-day operation of the reentry study, the retention strategies

were applied continuously and concurrently, so much of the work of the

data collection revolved around maximizing study participation. Interview

incentives were particularly important for study participation immediately

after prison release. At the first meeting in the community, respondents

received a total of $100, $50 for the baseline interview in prison and $50

for the one-week interview. One hundred dollars was a significant payment

for people with little income, particularly for one fifth of the sample who

were released from prison with less than $100. The last question for each

post-release survey asked respondents why they were participating in the

study. A quarter of the sample cited the interview payment as a strong

incentive at the one-week interview. The interview incentive appeared to

be less crucial to study retention over time. By the 12-month interview,

16

about 10 percent of respondents indicated they were participating for the

money.

Interviews were conducted by a pair of researchers and one member of

the pair was charged with maintaining consistent contact with the respon-

dent for the duration of the study. Consistent contact—reinforced by a non-

judgmental approach to interviews and regular phone check-ins—fostered

trust and rapport between researchers and study participants. Reflecting

the quality of the interviewer-respondent relationship, around one-third of

the sample reported they were participating in the research because they

were committed to completing the study or because they liked the inter-

viewer. An additional 20 percent said it was helpful for them to be able

to share their experiences after incarceration. At each post-release inter-

view, nearly half of the sample reported that they hoped their participation

would ultimately help others or have an impact on criminal justice policy.

For most respondents, phone calls were our primary mode of contact.

While the majority of respondents provided the number of a family home or

program at the baseline interview, half of the sample obtained a cell phone

within seven days of release. Respondents often informed researchers of

their new number when we called to schedule the one-week interview. Af-

ter the first week out, 23 out of 122 respondents maintained the same

phone number throughout the year out of prison. These 23 respondents

with a high level of contact security completed all post-release interviews

and were less likely to be scheduled late than others in the sample.

More commonly, phone status was unstable. Respondents changed their

phone number eight times on average over the course of the study. In some

cases, people’s primary phones would not work if phone bills went unpaid

or if residential status changed, such as when they left a program or re-

17

turned to prison or jail. Others obtained new phones when they started

working or changed their phone numbers every few months. Some re-

ported that regularly changing phones was a habit picked up when they

were involved in illegal activity and feared their phones were being tracked.

In addition, a number of phones went out of use when they were lost or

stolen, particularly in the chaotic environments of homeless shelters and

transitional housing. When respondents were unreachable by phone, proxy

contacts were helpful for maintaining contact. Several mothers with stable

home or cell phone numbers would provide us with updated contact infor-

mation for their sons and daughters every few months when their phones

went out of service.

In a small number of cases, phone check-ins and secondary contacts

were insufficient to maintain study participation, and criminal justice agen-

cies and community contacts also helped to follow-up with study respon-

dents. Criminal justice queries often took the form of a request to proba-

tion, parole, or the police for a phone number or address for hardest-to-

reach respondents. Respondents had earlier consented to such contact and

the queries were designed to be unobtrusive both for the agencies and the

respondents.

Community contacts were also invaluable for reaching the most elusive

respondents. In particular, streetworkers (or gang outreach workers) were

especially helpful for reestablishing contact with younger respondents who

were associated with gangs or other street groups. Because gangs are a

source of connection to social networks and neighborhoods (Papachristos

et al. 2013), and because Boston’s streetworkers are assigned to neigh-

borhoods and gangs, streetworkers were well positioned to assist in con-

necting with individuals with gang and neighborhood ties. In some cases,

18

streetworkers were able to provide a direct link—through a working phone

number or current address—for a study respondent that had gone missing.

More often, however, streetworkers “knew someone who knew someone,”

and connected the research team to a third party (say an ex-girlfriend,

neighborhood acquaintance, or a former youth worker) that could arrange

contact with a missing respondent. When subjects dropped out, commu-

nity contacts also offered explanations for nonresponse. For example, we

learned that one subject had developed extreme paranoia about being “set

up” following a shooting in his area in between survey waves and was reluc-

tant to meet or talk with all but a small group of trusted friends. Mistrustful

and occupied with the challenges of survival, participation in the study was

not something he could arrange.

Consistent with the nonignorable risk of nonresponse, contact insecu-

rity was often linked to insecurity in daily life. In these cases, different

strategies for study retention were useful at different points throughout

follow-up. Omar, a Puerto Rican man in his mid-forties, changed his phone

number 17 times over the course of the study. Omar had been a regular

heroin user for much of his life. At 16 years old, he suffered brain damage

after a car accident and reported it had been difficult to function without

drugs since that time. His physical and mental health problems compro-

mised his ability to maintain a steady phone number and to remember

dates of medical appointments and study interviews. Still, Omar was eager

to participate and on days when he was clear-headed and had a working

cell phone, he would sometimes call to check in.

For the first three months after incarceration, Omar resided at several

addresses and was sometimes homeless on the streets. He soon rekindled

a relationship with a former girlfriend, Jessie, who was just released from

19

prison herself. Jessie had been a heavy drug user but was in better health

and assisted Omar with attending appointments, including interviews. She

stayed with him on the streets and moved with Omar into a friend’s apart-

ment four months after his release. When Omar’s cell phone was out of

service, we would often call Jessie, and she would hand her phone to him

so that we could check in.

Omar was sentenced to probation upon release but never reported to

his probation officer, he told us, due to his drug use. Just before his six-

month interview, he received a warrant for not reporting. Though Omar

mentioned his trouble with probation during interviews and phone calls,

we received further information from our contacts at the Department of

Correction. Omar’s warrant was dropped on the condition that he com-

plete a residential drug treatment program. However, when we called the

program where he was assigned, a staff member reported he was no longer

there. We had trouble reaching Omar to schedule the 12-month interview,

perhaps because his legal status was compromised and he was no longer

living with Jessie.

When Omar and Jessie were both out of contact over the course of

the year, we would call Omar’s sister Lena, an older woman in her late

fifties. Lena was 16 years old when the family moved from Puerto Rico to

Boston and was more comfortable communicating with Spanish-speaking

members of the research staff. For nearly 15 years, she had maintained a

public housing unit in Boston’s inner city. Lena was the most stable person

in Omar’s life, and he used her residence as his mailing address. Lena

would get updated contact information from him every so often when he

stopped by her place for meals. After we learned that Omar had left his

treatment program, we contacted Lena who gave us his new cell phone

20

number. Two days later, we conducted Omar’s 12-month interview. Though

the interview took place two and a half months late, it provided valuable

information about Omar’s housing insecurity, drug use, and probation late

in the year after incarceration.

Between the 6- and 12-month interviews, we completed five phone

check-ins with Omar, failed to reach him on 18 call attempts, spoke to

his sister or girlfriend five times, and received information on his legal and

residential status from local criminal justice agencies. His case points to

the necessity of a range of redundant strategies to retain vulnerable study

participants and the high risk of nonresponse for people who are struggling

with housing insecurity, drug relapse, mental illness, and criminal involve-

ment.

Efforts at sample retention are summarized in Table 5. At each wave of

the survey, the distribution of retention effort is given for subjects who are

interviewed on time, interviewed late (at the 75th percentile of the days-

to-interview distribution or later), and who missed an interview. Success-

ful contacts include the average number of successfully completed phone

calls, text, or email exchanges. Respondents who completed late or who

missed interviews received a much higher number of unsuccessful contact

efforts. By the final six months of the survey, late respondents and non-

respondents were receiving two to three times more unsuccessful contact

efforts than successful. The proxies of these hard-to-observe respondents

also received many more calls than the proxies of respondents who were in-

terviewed on time. Finally, justice system queries, although relatively few

in number, were almost entirely focused on the late and missing respon-

dents. In sum, the late and missing respondents received disproportionate

retention effort. Late respondents received twice as many phone call con-

21

Table 5. Average level of retention effort between survey waves for on-time inter-views, late interviews, and missed interviews.

JusticeInterview Successful Failed Calls to Letters SystemStatus Contacts Contacts Proxies Sent Query N

Baseline to One WeekOn time completion 1.48 .63 .68 .03 .00 90Late completion 2.00 1.30 1.11 .19 .04 27Missed 1.00 1.40 .60 .40 .60 5

One Week to Two MonthsOn time completion 3.31 .47 .15 .00 .00 85Late completion 2.65 1.38 .27 .15 .04 28Missed 2.50 7.50 1.83 .83 .33 9

Two Months to Six MonthsOn time completion 3.55 1.28 .17 .00 .04 83Late completion 2.80 1.63 .27 .10 .27 30Missed .75 6.25 1.75 1.50 1.00 9

Six Months to Twelve MonthOn time completion 3.77 3.10 .45 .05 .15 86Late completion 2.92 7.24 2.04 .76 .80 25Missed 1.80 8.60 1.20 .40 .60 11

Total (N) 1,331 890 250 53 58 122

22

tacts than those who were interviewed on time, and the most intensive

retention efforts—through secondary contacts, letters, and justice system

queries—were focused on the nonrespondents.

3.4 A Model of Reluctance and Nonresponse

We can link respondent characteristics to nonresponse across the whole

sample with a multinomial logit regression that takes late and missing in-

terviews as response categories. Completely observed covariate character-

istics can be taken from the baseline interview and from linked administra-

tive data on arrests and incarceration. Such a model allows us to test two

hypotheses. First, including lagged dummy variables for late and missed in-

terviews indicate respondents with an enduring propensity to nonresponse.

Lagged measures also provide estimates of the relationship between late-

ness and nonresponse indicating whether late respondents are at high risk

of nonresponse. Second, we can also assess the association of lateness and

nonresponse to measures of extreme disadvantage that are closely related

to contact insecurity. These markers of social insecurity include histories of

mental illness, addiction, and marginal housing, all measured at baseline.

Contact insecurity is also closely related to time-varying measures of new

charges and re-incarceration. In addition to these measures, the analysis

also controls for age, sex, and race and ethnicity. Finally, to study the risk of

nonresponse for those who have no correctional supervision after prison re-

lease, a dummy variable is included for those on probation or parole during

follow-up.

More formally, for respondent i in wave t , we can write the probability

of being late or missing, p j ( j = L, M) compared to being interviewed on

23

Table 6. Multinomial logit regression results in an analysis of late and missedinterviews, compared to on-time interviews, 3 panels of the Boston Reentry Study.

Model 1 Model 2Late Missing Late Missing

Intercept −.356 −2.179** −.675 −2.667**(.66) (2.80) (1.17) (3.21)

Lag late interview .474 1.213* .495 1.270*(1.35) (2.52) (1.37) (2.49)

Lag missed interview 1.880** 3.470** 1.858** 3.647**(3.30) (5.95) (3.30) (6.40)

Age −.048** −.031 −.047** −.025(2.94) (1.50) (2.76) (1.12)

Female −.056 −1.642 −.122 −1.673*(.16) (1.78) (.32) (2.05)

Addiction .420 1.067* .415 .919*(1.29) (2.53) (1.20) (1.96)

Mental illness .773* .0554 .850* .0172(2.51) (.13) (2.54) (.04)

Prior unstable housing −.863* −.818 −.823* −.926(2.38) (1.59) (2.09) (1.60)

No supervision .253 .326 .239 .529(.85) (.72) (.75) (1.14)

New charge .971* −.144(2.50) (.17)

New Incarceration 1.122 2.581*(1.91) (2.38)

Log likelihood -250.246 -237.502Respondent-waves (N) 366 366∗p < .05 ∗∗p < .01

Note: Numbers in parentheses are z statistics. Late respondents are those in the topquartile of the days-to-interview distribution. Standard errors have been adjustedfor clustering. One survey wave is omitted to allow lagged measures of interviewstatus.

24

time, pO,

log

(p j i t

pOi t

)=α1 j li t−1 +α2 j mi t−1 +x ′

iβ1 j + z ′i tβ2 j ,

where dummy variables indicating lagged interview status are included for

respondents who were late (li t−1 = 1) or missing (mi t−1 = 1) in the prior

wave, x i is a vector of time-invariant covariates, and z i t is a vector of time-

varying covariates. (With lagged predictors, analysis is confined to the last

three waves of the survey at t = 2 months, 6 months, and 12 months.)

Note that the multinomial logit model yields two sets of coefficients, j =L, M , one showing the odds of late respondents relative to those who were

on time, and another showing the odds of missing respondents relative to

those who were on time.

Multinomial logit results are reported in Table 6. We fit one model that

includes lagged interview status and baseline characteristics, and another

model that adds the effects of new charges and incarceration (Table 6).

Across both models, respondents who were late or missing in a given wave

were highly likely to be missing at the following wave. This suggests that

the propensity to nonresponse is in part an enduring trait that is observ-

able from wave to wave. Difficulty in scheduling an interview, resulting

in lateness, is also associated with the risk of nonresponse. The under-

lying risk of nonresponse is related to the baseline traits of a history of

drug addiction, mental illness, and a history of unstable housing. Unex-

pectedly, living in unstable housing prior to incarceration is asociated with

a relatively low risk of nonresponse, when other covariates are controlled.

Unusually, the BRS sample also includes a relatively large number of peo-

ple who have completed sentences and are not supervised by probation or

parole. Coefficients for no supervision are relatively small and insignificant

in both models indicating that respondents who have “maxed out” (com-

25

pleted their maximum sentence) face no higher risk of nonresponse than

the rest of the sample. This result underlines the utility of the sample re-

tention strategies that do not directly involve assistance from community

corrections agencies. Estimates from the final model show that the risk of

nonresponse—either being late for an interview or missing—are also asso-

ciated with new charges and reincarceration. The odds of being late for an

interview are more than doubled by a new arrest. Respondents who return

to prison or jail are also at very high risk of a noninterview. Although the

relative risk of a noninterview is very high, overall nonresponse rates in

the BRS were very low. Still, the positive association of new charges and

incarceration with the risk of nonresponse suggests that, in a sample with

less retention effort and a lower response rate, survey nonresponse may be

strongly nonignorable.

4 CONSEQUENCES OF RELUCTANCE AND NONRESPONSE

To study the effect of survey nonresponse on sample statistics, we can con-

struct a measure of the risk of nonresponse and examine the sensitivity of

sample quantities to different hypothetical levels of missing data.

An index for the risk of nonresponse can be constructed from data on

late interviews and actual missed interviews. Late interviews are completed

with additional retention effort, so treating late interview subjects as poten-

tial nonrespondents yields a sample that would have been observed with

less retention effort. We construct a summary index of respondent’s risk of

nonresponse using information on the days late for a scheduled interview

and nonresponse. Because the days late varies across waves, we transform

the measures to percentiles, Pi t within each wave and assign nonrespon-

26

dents to the highest percentile. A simple average across the four waves of

the survey yields a measure of the risk of nonresponse, P̄i , for each respon-

dent. High values of P̄i indicate respondents who were consistently very

late or who missed follow-up interviews. Low scores on P̄i indicate respon-

dents who completed all interviews on time and required relatively little

retention effort. Following the multinomial logit results on lagged effects,

the risk of nonresponse, P̄i , is treated as a stable trait. Still, nonresponse

risk also has a time-varying component. For example, respondents who re-

lapse to addiction at some point in the follow-up period may be more likely

to miss an interview compared to respondents with histories of addiction

who do not resume drug use. Sensitivity of sample statistics to the risk of

nonresponse is thus likely to be under-estimated by P̄i .

Figure 2 reports the means of key variables for respondents grouped in

the lower, middle and upper terciles of the risk of nonresponse, P̄i . Re-

spondents who are at low risk of nonresponse are in the bottom tercile of

the distribution of P̄i . Medium risk respondents are in the middle tercile

and high-risk respondents are in the top tercile. All variables except post-

incarceration employment are completely observed, either recorded at the

baseline interview or from criminal justice administrative records. Respon-

dents at high or medium risk of nonresponse reported high levels of drug

addiction, high school dropout, arrest, and incarceration compared to those

at low risk of nonresponse.

Mean differences in key variables across levels of the risk of nonre-

sponse may be associated with nonignorable nonresponse that drives sam-

ple selection bias in regression. Often in longitudinal studies with hard-

to-reach populations, regression analysis focuses on the effects of baseline

characteristics on outcomes at follow-up. We pool together the four follow-

27

Low

Addiction

Mental illness

Two parents at 14

HS dropout

Unstable housing

Employed

Probation/parole

Employed

Arrested

Incarcerated

Before Incarceration

After Incarceration

0 .20 .40 .60 .80

Proportion

Medium

0 .20 .40 .60 .80

Proportion

High

0 .20 .40 .60 .80

Proportion

Figure 2. Means of key pre-incarceration and post-incarceration variables by low,medium, and high risk of nonresponse, Boston Reentry Study.

28

up waves from the BRS and fit linear probability models to the pooled

data using as dependent variables measures of post-incarceration unstable

housing, drug use, and re-incarceration. Regression analysis focuses on the

coefficients for a history of drug addiction, unstable housing prior to arrest,

and probation and parole supervision after release. In addition, regressions

control for the wave of the survey, age, sex, and race and ethnicity.

To study the sensitivity of regression coefficients to different levels of

nonresponse, we conduct a simulation experiment analyzing subsets of the

data where observations are included in the analysis conditional on the

nonresponse risk, P̄i . As the average level of percentiles from four follow-

up surveys, the nonresponse risk, P̄i , varies from 10 to 97 with a mean of

50. We then define a nonresponse threshold, T , that is used to drop all

observations from respondent i for P̄i > T . With the reduced data set, we

estimate new regression coefficients, βT . These coefficients can be inter-

preted as the estimates that would have been calculated with a response

rate at a level of retention effort at threshold T . Allowing the retention ef-

fort to vary from T = 97,96, . . . ,50 we can examine how coefficients change

as nonresponse increases. With T varying over this range, the simulated

nonresponse rate varies from 0 to 45 percent, approximately the range of

nonresponse reported in recent longitudinal prisoner reentry studies. With

nonignorable nonresponse and the corresponding sample selection bias,

we expect coefficients to get larger in absolute value as nonresponse de-

clines. Statistical precision will also increase with larger sample sizes, so

confidence intervals will also shrink as nonresponse declines.

Figure 3 reports results from the simulation experiment. Results for

each dependent variable—housing instability, drug use, and re-incarceration—

are shown down the columns. The coefficients—for pre-arrest addiction,

29

pre-arrest unstable housing, and probation or parole status—are shown

across the rows. Hypothetical nonresponse rates are reported on the hori-

zontal axis and coefficients and confidence intervals are shown on the ver-

tical axis. In the model of housing instability, the effect of pre-arrest unsta-

ble housing and probation/parole behave consistently with classical sample

selection bias in which effects are significantly attenuated with high rates

of nonresponse. Neither effect would be detectable with a 55 percent re-

sponse rate, but both effects become large and significant as the response

rate approaches 100 percent. For post-incarceration drug use, the coeffi-

cient for pre-arrest addiction also becomes larger with the rising response

rate. The probation/parole coefficient gets slightly smaller, but the confi-

dence interval shrinks to statistical significance as sample size gets larger.

In the models for reincarceration, the coefficients for addiction, unstable

housing and probation/parole also get larger as sample selectivity declines

and sample size increases. These results indicate how high rates of non-

response in hard to reach populations contribute to sample selection bias,

driving null effects in the analysis of social disadvantage and vulnerability.

5 DISCUSSION

This paper explores study retention and the consequences of survey non-

response in a highly marginal population. Surveying a sample of released

state prisoners returning to neighborhoods in Boston, we describe a one-

year follow-up study involving a group of men and women with histories of

drug addiction, mental illness, and housing insecurity. The Boston Reentry

Study maintained a high response rate, over 90 percent, that provides an

opportunity to explore strategies for study retention in hard-to-reach pop-

30

Pre−Arrest Addiction

0.6 0.7 0.8 0.9 1.0

0.10

0.15

0.20

0.25

0.30

0.35

0.40

Response Rate

Housing Instability

0.6 0.7 0.8 0.9 1.0

0.00

0.05

0.10

0.15

0.20

Response Rate

Drug Use

0.6 0.7 0.8 0.9 1.0

−0.

050.

000.

05

Response Rate

Re−Incarceration

Pre−Arrest Unstable Housing

0.6 0.7 0.8 0.9 1.0

0.0

0.1

0.2

0.3

Response Rate

0.6 0.7 0.8 0.9 1.0

−0.

15−

0.10

−0.

050.

000.

05

Response Rate

0.6 0.7 0.8 0.9 1.0

−0.

10−

0.05

0.00

Response Rate

Probation or Parole

0.6 0.7 0.8 0.9 1.0

−0.

100.

000.

050.

100.

15

Response Rate

0.6 0.7 0.8 0.9 1.0

−0.

15−

0.10

−0.

050.

00

Response Rate

0.6 0.7 0.8 0.9 1.0

−0.

050.

000.

05

Response Rate

Figure 3. Coefficients for pre-arrest addiction, pre-arrest unstable housing, andprobation or parole status in panel regressions on post-incarceration housing in-stability, drug use, and re-incarceration. Coefficients vary with the response ratesorting sample respondents from low to high risk of nonresponse. (Dashed linesindicate 95 percent confidence intervals. A zero coefficient is marked by the solidhorizontal line.)

31

ulations as well as examine the effects of survey nonresponse that has been

common in prior research.

Three main conclusions can be drawn from the analysis of study reten-

tion in the BRS. First, formerly-incarcerated men and women show a very

high level of contact insecurity. Most of the BRS respondents experienced

significant residential instability, were weakly attached to traditional house-

holds, changed phones frequently, were periodically out of phone contact,

and in some cases returned to incarceration. These respondents would be

overlooked in conventional household surveys and we would expect very

high rates of attrition without special measures for study retention. High

rates of contact insecurity help explain the high rates of nonresponse re-

ported in prior studies of prisoner reentry.

Second, the challenges to study retention in hard-to-reach populations

can be successfully addressed through research design. Successful reten-

tion depends on a variety of redundant strategies for minimizing attrition

in a highly unstable sample facing extreme contact insecurity. Key strate-

gies in the Boston Reentry Study included significant interview incentives,

a nonjudgmental approach to interviews along with frequent phone check-

ins that helped build rapport, lists of proxy contacts, and in the last re-

sort, justice system and community streetworker queries. These measures

provided strong incentives for participation and helped quickly find those

whose contact information was lost.

Third, contact insecurity and the risk of survey nonresponse were closely

linked to the social risks and vulnerabilities of sociological interest. Thus

respondents at high risk of nonresponse had histories of drug addiction

and mental illness, and were also more likely to be arrested and incar-

cerated after prison release. Because the risk of a noninterview is closely

32

linked to the extreme social disadvantages of substantive interest, nonre-

sponse in such a socially marginal population is strongly nonignorable. In

the Boston Reentry Study we found that regression estimates were highly

sensitive to survey nonresponse in analyses of housing insecurity, drug use,

and re-incarceration. Regression estimates were greatly attenuated at hy-

pothetically high rates of nonresponse, around 35 to 45 percent.

Earlier studies of nonresponse in hard-to-reach populations report that

significant resources are required to collect data from the most elusive re-

spondents (Farrington et al. 1990; Teitler et al. 2003). Researchers have

suggested that the benefits in data quality that result from a higher reten-

tion rate may not be worth the cost of tracking down those who are most

difficult to contact (Teitler et al. 2003).

Our findings point to three additional considerations when evaluating

the benefits of high study retention. First, while retaining the most elusive

respondents does require significant effort, developing a culture within the

research project that promotes rapport and connectedness with study sub-

jects can improve retention at little extra financial cost. Second, the effort

to sustain 100 percent retention likely improves the quality of interviews

that are completed. Where interviews are conducted in a climate of trust

with an interviewer that a respondent has come to know through numer-

ous interactions, survey responses are likely to be more forthcoming and

complete, particularly in sensitive areas. Third, as our field period pro-

gressed, 100 percent study retention became as much a humanistic as a

methodological objective. A single missing observation out of nearly 600

interviews would barely increase bias or standard errors, but an opportu-

nity would be missed to register the voice and experience of a respondent

who has rarely been heard in science or policy. In very poor or socially

33

marginal research sites, building trust and connection with respondents

yields the benefits of accurate measurement and a high response rate, but

also gives voice to those who are largely invisible.

More generally, attrition in panel studies and other forms of nonre-

sponse represent a fundamental methodological challenge to the study of

extreme poverty and other hardship. Research in this area often features

large-scale data collections with household surveys or administrative data

bases (e.g., see the reviews of Brooks-Gunn and Duncan 1997 and Have-

man et al. 2015). With these data, the problems of under-enumeration and

survey nonresponse are well-documented (Groves and Couper 1998). Spe-

cialized data collections focused on those who are missed by a household

sampling frame or at risk of attrition in a panel study can usefully sup-

plement the large-scale data collection. Counting those who are the most

difficult to count holds the promise of observing the deepest disadvantage

and the full extent of its social contours. In these very marginal social

spaces—where nonresponse is strongly nonignorable—strategies for study

retention are critical for reducing bias and accurately observing extreme

material hardship and social insecurity.

34

REFERENCES

Brooks-Gunn, Jeanne and Greg J Duncan. 1997. “The Effects of Povertyon Children.” The Future of Children 7:55–71.

Brown-Peterside, Pamela, Evelyn Rivera, Debbie Lucy, Izzie Slaughter,Leigh Ren, Mary Ann Chiasson, and Beryl Koblin. 2001. “RetainingHard-to-Reach Women in HIV Prevention and Vaccine Trials: ProjectACHIEVE.” American Journal of Public Health 91:1377–1379.

Cotter, Robert B., Jeffrey D. Burke, Rolf Loeber, and Judith L. Navratil.2002. “Innovative Retention Methods in Longitudinal Research : ACase Study of the Developmental Trends Study.” Journal of Child andFamily Studies 11:485–498.

Cottler, Linda B., Wilson M. Compton, Arbi Ben-Abdallah, MalaikaHorne, and Daniel Claverie. 1996. “Achieving a 96.6 percent follow-up rate in a longitudinal study of drug abusers.” Drug and AlcoholDependence 41:209–217.

de la Puente, Manuel. 2004. “Census 2000 Ethnographic Studies.” Tech-nical Report 15, U.S. Census Bureau, Washington, DC.

Desmond, David P., James F. Maddux, Thomas H. Johnson, and Beth A.Confer. 1995. “Obtaining Follow-Up Interviews for Treatment Evalu-ation.” Journal of Substance Abuse Treatment 12:95–102.

Farrington, David P., Bernard Gallagher, Lynda Morley, Raymond J. St.Ledger, and Donald J. West. 1990. “Minimizing Attrition in Longi-tudinal Research: Methods of tracing and securing cooperation in a24-year follow-up study.” In Data Quality in Longitudinal Research,edited by David Magnusson and Lars Bergman, pp. 122–147. Cam-bridge, UK: Cambridge University Press.

Faugier, Jean and Mary Sargeant. 1997. “Sampling Hard to Reach Pop-ulations.” Journal of Advanced Nursing 26:790–797.

Fitzgerald, John, Peter Gottschalk, and Robert A. Moffitt. 1998. “AnAnalysis fo Sample Attrition in Panel Data: The Michigan Panel Studyof Income Dynamics.” Journal of Human Resources 33:251–299.

Groves, Robert M. and Mick P. Couper. 1998. Nonresponse in HouseholdInterview Surveys. New York: Wiley.

35

Harding, David, Jessica J B Wyse, Cheyney Dobson, and Jeffrey DMorenoff. 2014. “Making Ends Meet After Prison.” Journal of PolicyAnalysis and Management 33:440–470.

Haveman, Robert, Rebecca Blank, Robert Moffitt, Timothy Smeed-ing, and Geoffrey Wallace. 2015. “The War on Poverty: Measure-ment, Trends, and Policy.” Journal of Policy Analysis and Management34:593–638.

Heckman, James J. 1979. “Sample Selection Bias as a Specification Er-ror.” Econometrica 47:153–161.

Hirano, Keisuke, Guido W. Imbens, Geert Ridder, and Donald B. Rubin.2001. “Combining Panel Data Sets with Attrition and RefreshmentSamples.” Econometrica 69:1645–1659.

Hough, Richard L., Henry Tarke, Virginia Renker, Patricia Shields, andJeff Glatstein. 1996. “Recruitment and Retention of Homeless Men-tally Ill Participants in Research.” Journal of Consulting and ClinicalPsychology 64:881–891.

Hunter, Jennifer, Manuel de la Puente, and Matt Salo. 2003. “Compar-ative Ethnographic Research on Mobile Populations.” Technical re-port, U.S. Census Bureau.

Kachnowski, Vera. 2005. “Employment and Prisoner Reentry.” Technicalreport, Urban Institute, Washington, DC.

Kornfeld, Robert and Howard S Bloom. 1999. “Measuring Program Im-pacts on Earnings and Employment: Do Unemployment InsuranceWage Reports from Employers Agree with Surveys of Individuals?”Journal of Labor Economics 17:168–197.

La Vigne, Nancy G., Tracey L. Shollenberger, and Sara A. Debus. 2009.“One Year Out : Tracking the Experiences of Male Prisoners Return-ing to Houston, Texas.” Technical report, Urban Institute, Washing-ton, DC.

Lattimore, Pamela K and Danielle M. Steffey. 2009. “The Multi-Site Eval-uation of SVORI: Methodology and Analytic Approach.” Technicalreport, RTI International, Research Triangle Park.

Little, Roderick J.A. and Donald B. Rubin. 1987. Statistical Analysis ofMissing Data. New York: Wiley.

36

Lopoo, Leonard M and Bruce Western. 2005. “Incarceration and the For-mation and Stability of Marital Unions.” Journal of Marriage and theFamily 67:721–734.

Martin, Elizabeth, Denise Abreu, and Franklin Winters. 2001. “Moneyand Motive: Effects ofIncentives on Panel Attrition in the Surveyof Income and Program Participation.” Journal of Official Statistics17:267–284.

Martinez-Ebers, Valerie. 1997. “Using Monetary Incentives with Hard-to-Reach Populations in Panel Surveys.” International Journal of Pub-lic Opinion Research .

Messiah, Antoine, Helen Navaline, Annet Davis-Vogel, Danielle Tobin-Fiore, and David Metzger. 2003. “Sociodemographic and BehavioralCharacteristics Associated with Timeliness and Retention in a 6-Month Follow-up Study of High-Risk Injection Drug Users.” AmericanJournal of Epidemiology 157:930–939.

Metraux, Stephen, Caterina G Roman, and Richard S Cho. 2007. “Incar-ceration and Homelessness.” National Symposium on HomelessnessResearch pp. 1–31.

Nelson, Marta, Perry Deess, and Charlotte Allen. 1999. “The First MonthOut:post-Incarceration Experiences in New York City.” Vera Instituteof Justice Working Paper.

Papachristos, Andrew V., David M. Hureau, and Anthony A. Braga. 2013.“The Corner and the Crew: The Influence of Geography and SocialNetworks on Gang Violence.” American Sociological Review 78:1–31.

Pettit, Becky. 2012. Invisible Men: Mass Incarceration and the Myth ofBlack Progress. New York: Russell Sage Foundation.

Salganick, Matthew J. and Douglas D. Heckathorn. 2004. “Samplingand estimation in hidden populations using respondent-driven sam-pling.” Sociological Methodology 34:193–240.

Sirois, Catherine. 2015. “Household Dynamics in the Year after Prison.”Paper presented at the American Sociological Association Meetings,Chicago.

Sugie, Naomi F. 2014. Finding Work: A Smartphone Study of Job Search-ing, Social Contacts, and Wellbeing after Prison. Ph.D. thesis, Prince-ton University.

37

Teitler, Julien O., Nancy E. Reichman, and Susan Sprachman. 2003.“Costs and Benefits of Improving Response Rates for a Hard-to-ReachPopulation.” The Public Opinion Quarterly 67:126–138.

Travis, Jeremy. 2005. But They All Come Back: Facing the Challenges ofPrisoner Reentry. Washington DC: Urban Institute.

Travis, Jeremy, Bruce Western, and Stephens Redburn (eds.). 2014. TheGrowth of Incarceration in the United States: Exploring Causes andConsequences. Washington, DC: National Academy Press.

Visher, Christy, Vera Kachnowski, Nancy La Vigne, and Jeremy Travis.2004. Baltimore Prisoners’ Experiences Returning Home. Washington,DC: Urban Institute.

Visher, Christy A and Shannon M E Courtney. 2007. One Year Out: Ex-periences of Prisoners Returning to Cleveland. Washington, DC: UrbanInstitute.

Western, Bruce, Anthony Braga, Jaclyn Davis, and Catherine Sirois.2015. “Stress and Hardship After Prison.” American Journal of So-ciology 120:1512–1547.

38