From Homelessness to Housing:

Exploring Trends and Improving Data to

Strengthen Transition Projects’ Services

Prepared for Transition Projects, Inc.

By

Daniel Benson

Anna Brunner

Sara Nowakowski

Sara Sanders

Karina Virrueta

Workshop in Public Affairs

Spring 2017

©2017 Board of Regents of the University of Wisconsin System

All rights reserved.

For an online copy, see

http://www.lafollette.wisc.edu/outreach-public-service/workshops-in-public-affairs

publications@lafollette.wisc.edu

The Robert M. La Follette School of Public Affairs is a teaching and research department

of the University of Wisconsin–Madison. The school takes no stand on policy issues;

opinions expressed in these pages reflect the views of the authors.

The University of Wisconsin–Madison is an equal opportunity and affirmative-action educator and employer.

We promote excellence through diversity in all programs.

Table of Contents

List of Tables ...................................................................................................................................v

List of Figures ..................................................................................................................................v

Foreword ........................................................................................................................................ vi

Acknowledgements ....................................................................................................................... vii

Executive Summary ..................................................................................................................... viii

Introduction ......................................................................................................................................1

Background ......................................................................................................................................1

Risk Factors for Homelessness ....................................................................................................2

Programs Supporting Homeless Individuals to Housing .............................................................3

Placements and Rental Assistance ...................................................................................................4

Transition Projects’ Data .............................................................................................................4

Analysis of Placement and Subsidies ..........................................................................................5

Overview ..................................................................................................................................6

Housing Status .........................................................................................................................7

Service Provisions ....................................................................................................................9

Patterns in Being Placed or Receiving a Subsidy ............................................................... 9 Patterns in Receiving Multiple Placements and Multiple Subsidies ................................ 10 Patterns in Receiving an Ongoing Subsidy ....................................................................... 11

Transition Projects’ Data-Collection and Use ...............................................................................12

Discussion ......................................................................................................................................13

Recommendations ..........................................................................................................................14

Recommendation 1 ....................................................................................................................15

Promote an understanding of the importance of data ............................................................15

Create learning forums ...........................................................................................................15

Recommendation 2: Formalize data-collection processes .........................................................16

Example of Improving Data-Collection with Services ..........................................................17

Recommendation 3 ....................................................................................................................17

Process of Evaluating Findings ..............................................................................................18

Continuous Improvement .......................................................................................................18

Recommendation 4 ....................................................................................................................18

Thought Leadership ...............................................................................................................19

Conclusion .....................................................................................................................................19

Appendix A: Notes on Data Cleaning and Recoding ....................................................................20

Appendix B: TPI Demographics ....................................................................................................23

Appendix C: Placements and Subsidies Analysis Tables ..............................................................25

Appendix D: Housing Status Analysis Tables ...............................................................................27

Appendix E: Multiple Subsidy and Multiple Placements Analysis Tables ...................................29

Appendix F: Ongoing Subsidy Analysis Tables ............................................................................32

Appendix G: Survey Questions and Select Results .......................................................................33

Appendix H: Suggestions for Integrating Data into Decision-Making .........................................38

Works Cited ...................................................................................................................................39

v

List of Tables

Table 1. Number of Participants Who Received One or More Placements and/or Rental

Subsidies, 2007-2016 .................................................................................................................6

Table 2. Combining Intensive Support and Stable Housing Findings, 2007-2016 .......................11 Table 3. Demographics of All TPI Participants, TPI Participants from 2007-2016, and TPI

Participants in January 2015 ....................................................................................................24 Table 4. TPI Placements per Year, with Demographic Data .........................................................25 Table 5. TPI Subsidies per Year, with Demographic Data ............................................................25

Table 6. Receive Placement Services Regression Coefficients .....................................................26 Table 7. Receive a Subsidy Regression Coefficients .....................................................................26 Table 8. Housing Status Regression Coefficients Conditional on Receiving a Subsidy ...............27 Table 9. Housing Status Descriptive Statistics ..............................................................................28

Table 10. Multiple Subsidy Regression Coefficients ....................................................................29 Table 11. Multiple Subsidies Descriptive Statistics ......................................................................30

Table 12. Multiple Placements Regression Coefficients ...............................................................31 Table 13. Multiple Placements Descriptive Statistics....................................................................31 Table 14. Ongoing Subsidy Regression Coefficients ....................................................................32

Table 15. Ongoing Subsidy Descriptive Statistics .........................................................................32 Table 16. Q1: On average, how many hours a week do you typically spend entering data? (%) .33

Table 17. Q2: This week, how many hours did you spend on data entry? (%) .............................33 Table 18. Q3: How would you rate the quality (considering accuracy, completeness, usefulness)

of the data you enter? (%) ........................................................................................................33

Table 19. Q4: On average, how many hours a week do you spend using data (including

analyzing, cleaning, and using data to make decisions)? (%)..................................................34

Table 20. Q5: This week, how many hours did you spend using data (including analyzing,

cleaning, and using data to make decisions)? (%) ...................................................................34

Table 21. Q6: How would you rate the quality (considering accuracy, completeness, usefulness)

of the data you use? (%)...........................................................................................................34

Table 22. Q7: What is your position at Transition Projects? (n) ...................................................35 Table 23. Data Culture Questions, % Agree or Strongly Agree ....................................................35

List of Figures

Figure 1. Number of Placements and Subsidies, 2007-2016 ...........................................................6 Figure 2. Distribution of Subsidy Length .......................................................................................7 Figure 3. Proportion of Non-White Placement and Subsidy Participants .......................................7 Figure 4. Estimates of Various Subsidy Lengths on Stable Housing Outcomes compared to One-

Month Rental Assistance ...........................................................................................................8 Figure 5. Incorporating Data into Organization Decision-Making Model ....................................14

vi

Foreword

This report is the result of collaboration between the Robert M. La Follette School of Public

Affairs at the University of Wisconsin–Madison and Transition Projects, Inc, a nonprofit housing

organization in Portland, Oregon. The project provided graduate students at the La Follette

School the opportunity to improve their policy analysis skills while contributing to the capacity

of Transition Projects to better serve its community.

The La Follette School provides students with a rigorous two-year graduate program leading to a

master’s degree in public affairs. Students study policy analysis and public management as well

as concentrated study in at least one policy area. The authors of this report all are in their final

semester of their degree program and are enrolled in the Public Affairs 869 Workshop in Public

Affairs at UW–Madison. Although studying policy analysis is important, there is no substitute

for engaging actively in applied policy analysis as a means of developing policy analysis skills.

The Public Affairs 869 Workshop gives graduate students that opportunity.

Transition Projects is a national leader in providing life-changing services to people in need.

Transition Projects engaged with La Follette School of Public Affairs students to perform

rigorous research using data collected over the last decade in the process of delivering thousands

of services. This report includes an analysis of administrative data as well as interviews and other

research with the goal of helping Transition Projects improve services for individuals and

families in Portland.

I am grateful to Transition Projects for partnering with the La Follette School on this project. The

staff of Transition Projects have been exceptionally generous with their time to support this

project, including collaborating on data analysis. The students have collectively contributed

hundreds of hours to this project and in the process developed critical insights about state

policies and programs. The La Follette School is grateful for their efforts and hopes that this

report proves valuable for Transition Projects to improve the outcomes for the people it serves

and inform similar programs nationally.

J. Michael Collins

Professor of Public Affairs

May 2017

Madison, Wisconsin

vii

Acknowledgements

This report is the result of collaborative efforts from numerous contributors beyond the authors.

The La Follette School team would like to thank all those who provided support and expertise

throughout this project. The final product serves as a testament to these individuals’ commitment

to the pursuit of knowledge and the drive to induce positive, practical results. These individuals

provided critical advice and vital insight throughout our efforts, the results of which would not

have been possible without their guidance.

These dedicated individuals include, but are certainly not limited to:

Ryan Dunk, Logistics Coordinator and our client contact at Transition Projects;

Tony Bernal, Christian Heinlein, David Katz-Wigmore, Megan O’Keefe, and the entire

Transition Projects staff;

J. Michael Collins, our project advisor at the Robert M. La Follette School of Public Affairs,

whose expertise and unending encouragement served as indispensable resources for our team

throughout this project;

Jennifer Chang, Senior Policy Coordinator at the Portland Housing Bureau, for her critical

insight into Portland’s homeless crisis and Transition Projects’ place within the community;

Lisa Hildebrand, Senior University Relations Specialist for the La Follette School of Public

Affairs, for her editorial support; and,

Professors Rourke O’Brien, Hilary Shager, Timothy Smeeding, and our student colleagues at the

La Follette School of Public Affairs, who served as critical sounding boards for ideas and

provided invaluable counsel that was instrumental to ensuring the quality of this report.

viii

Executive Summary Effective organizations use quality data to learn more about their participants, better target their

services, and inform their decisions. Challenged by uniquely high and increasing levels of

homelessness, local leaders and homeless service providers in Portland, Oregon, are grappling

with how to better address the needs of the growing homeless population. One of the foremost

homeless service providers in the region, Transition Projects, Inc., has begun to explore how it

can best utilize its wealth of data as a resource to improve its homeless services.

Using program data, this report provides a descriptive analysis of Transition Projects’ key

services, with a focus on placement assistance and rental-subsidy services. Aligning with

prevailing research, some findings within the analysis suggest a systematic relationship between

level of assistance and stable housing; however, the strength of the relationship depends on the

person and his/her situation.

Inconsistencies within Transition Projects’ data challenge the validity of these findings and

prevent a more robust analysis of the organization’s services. To better understand the limitations

in the data, a survey explored how staff collect, use, and feel about data. This report uses

findings and observed limitations in the data, results from the survey, and prevailing research on

effective data use to provide the following actionable recommendations:

1. Further integrate data into Transition Projects’ culture: Promote a shared understanding

across the organization about the value of data. Encourage employees to share best practices

for collecting data.

2. Formalize data-collection processes: Discuss and address inconsistencies in current data-

collection and entry processes. Involve employees from all levels in defining data and

establishing consistent procedures, facilitate buy-in, accountability, and improved data

quality.

3. Evaluate data in terms of programmatic intentions and integrate into decision-making: Consider analysis findings, potential programmatic reasons that may explain them, and

alignment with organization intentions. Consider the outcomes of interest and think about how

to better collect existing data points to facilitate evaluation.

4. Garner sector-wide support for building the evidence base for homelessness programs:

Capitalize on Transition Projects’ position as a regional leader to foster venues for

information sharing across the county and the sector. Emphasize the current limitations of

existing data systems, and provoke discussions of improved strategies to more precisely

evaluate the effectiveness of assistance programs.

Quality data-collection and utilization can improve organizational decision-making, prompt

thoughtful and regular evaluations of programs, increase the efficiencies of service, and further

an organization’s mission. By implementing sustained changes to how the organization collects

and utilizes data, Transition Projects can improve its data to more effectively and efficiently

move people from homelessness to housing.

1

Introduction

Using quality data can help an organization learn about itself and be more effective. Quality data

allows an organization to “monitor program outcomes, expand system reporting capabilities, and

improve the validity of data analysis results” (OCPD, 2007). By evaluating its program

effectiveness and resource allocation, an organization has greater capacity to identify positive

outcomes and opportunities for improvement.

Quality data can also increase an organization’s credibility by establishing an evidence base that

demonstrates program outcomes and impacts. If an organization can highlight its successes and

point to data that verifies those successes, it can paint a more convincing picture for requesting

additional funding to expand programs. Alternatively, data can provide evidence of possible gaps

in services, such as high-risk populations that an organization is not reaching given its current

resource capacity.

Five factors maximize the utility of data an organization collects: (1) validity, (2) completeness,

(3) consistency, (4) accuracy, and (5) verifiability. Poor quality data that does not meet all five

components can impede an organization’s ability to understand its impact, possibly obstructing

its decision-making processes and hindering its mission. In short, “if [the data] is flawed […]

then decisions based on that data are more likely to be flawed” (Fisher, 2001).

Transition Projects (TPI) is a nonprofit homeless service provider based in Portland, Oregon.

The organization is concerned with identifying effective services that help individuals and

families maintain their housing. Additionally, TPI is interested in an evaluation of its own

services and how to enhance its data-collection processes.

This report details known risk factors associated with homelessness and identifies promising,

evidence-based programs for helping individuals experiencing homelessness re-establish housing

stability. To better target its own homelessness services, TPI is also interested in understanding

who it serves, who receives more intensive services, and how these services and service

intensities relate to outcomes. This report provides an analysis of TPI participant and service

data. However, because data limitations hinder the analysis, validity and interpretation of the

findings is limited. To further understand the data limitations as well as TPI’s current use and

attitudes toward data, this report also presents findings from a staff survey. Finally, the report

provides recommendations for TPI to move toward incorporating data into decision-making and

address data limitations, and, as a result, help TPI become a more effective homeless service

provider.

Background

Transition Projects (TPI) is a nonprofit agency with a mission of moving people from

homelessness to housing. Established in 1969, TPI is the largest homeless service provider in

Portland, Oregon, employing over 200 staff across nine program sites. Annually, TPI finds

permanent housing for 800 to 1,000 households, shelters several thousand people, and helps

more than 10,000 people meet their basic needs. The agency administers more than $2 million in

short- and long-term rental assistance to formerly homeless households and offers supportive

2

services such as case management, support groups, and classes to help people maintain their

housing (TPI, 2016).

Recent and dramatic increases in Portland’s population have exacerbated the need for TPI’s

homeless services. Since 2010, Portland’s population increased by 11 percent, from about

566,700 to 612,200 people in 2016 (ACS). This spike in population has put an increased strain

on the area’s housing market, which cannot keep up with demand. As a result, Portland’s rental

vacancy rates are low and gross rents are increasing. Between 2009 and 2015, the median gross

rent increased by 25 percent, to $971 per month (ACS, 2015).

Portland is also facing an affordable housing shortage. The number of people who are cost-

burdened by their rent, defined as those who pay more than 30 percent of their income for

housing, is increasing. The waitlist for public housing at Home Forward, the housing authority

for Portland, doubled in the last five years while the waitlist for Section 8 housing assistance

vouchers quintupled (Home Forward, 2010 2015). Additionally, the affordable housing shortage

has led to an increase in people facing homelessness so severe that the city of Portland

announced a State of Emergency on Housing and Homelessness in 2015.

Risk Factors for Homelessness

Nearly 3.5 million people nationally and 3,800 individuals in Portland and Multnomah County,

Oregon, experience homelessness in any given year (Smock, 2015). However, the risk of

homelessness is not experienced equally by all people – certain demographic sub-groups have a

greater risk of becoming homeless.

The main cause and primary risk factor of homelessness is economic hardship, such as losing a

job. However, a person does not need to be unemployed to be under threat of homelessness.

Underemployment and the low wages that come with it can similarly put individuals and families

at risk of becoming homeless. This is especially true in metropolitan areas where there is little

affordable housing and wages do not cover even the most basic living expenses (NPACH). The

most frequent causes of homelessness among participants in the 2015 point-in-time (PIT)

estimate were unemployment and inability to pay rent (Smock, 2015). Additionally, high

housing costs in Multnomah County have resulted in a need for 25,000 housing units that would

be affordable to the lowest-income renters (Smock, 2015).

While unemployment and underemployment are regarded as the leading causes of homelessness,

the correlated risk factors of mental health problems and substance abuse play critical roles in

elevating the odds that an individual may become homeless. The prevalence of mental health

issues is three to four times higher among homeless individuals, and the rate of alcohol abuse co-

occurring with one or more psychiatric conditions is five times greater (Shelton et al., 2015). In

Multnomah County, 57 percent of homeless individuals have disabling conditions, including

mental health, substance abuse, HIV/AIDS, developmental disabilities, and chronic health

conditions. In total, 49 percent of the homeless population in this area has substance abuse

problems and 36 percent has mental health problems (Smock, 2015).

Men and those identifying as black or African-American are also disproportionately more likely

to be homelessness. Over 60 percent of all sheltered homeless individuals are male and, despite

3

making up only 13 percent of the total U.S. population, 40 percent of the sheltered homeless

population was black or African-American in 2014 (Johnson et al., 1997) — more than three

times the rate of the general population. Multnomah County has a comparable proportion of male

homelessness (69 percent). Black or African-Americans are 24 percent of the homeless

population (Smock, 2015), which is about five times the rate of the area’s black and African-

American population overall (about 5 percent) (ACS).

While the number of veterans who are homeless decreased by 47 percent between 2009 and

2016, veterans still are more burdened with homelessness than any other demographic sub-

group, totaling about nine percent of the homeless population (HUD, 2016). According to a

Portland-Multnomah-Gresham Continuum of Care (CoC) PIT estimate, veterans make up a

slightly larger proportion of the homeless at 12 percent (Smock, 2015).

Programs Supporting Homeless Individuals to Housing

Since the McKinney-Vento Homeless Assistance Act of 1987, a range of innovative programs

have emerged nationally, including rapid re-housing, transitional housing, and permanent

supportive housing (National Coalition for the Homeless, 2006).

The U.S. Department of Housing and Urban Development (HUD) defines rapid re-housing as an

intervention that “rapidly connects families and individuals experiencing homelessness to

permanent housing through a tailored package of assistance that may include time-limited

financial assistance and targeted supportive services” (HUD Exchange, 2014). Transitional

housing is “a project that is designed to provide housing and appropriate supportive services to

homeless persons to facilitate movement to independent living within 24 months, or a longer

period approved by HUD” (HUD). Permanent supportive housing is permanent housing with

indefinite leasing or rental assistance paired with supportive services to assist homeless persons

with a disability or families with an adult or child with a disability achieve housing stability

(HUD Exchange, 2017).

HUD’s Family Options Study (2016) finds that these interventions may not be equally

appropriate for all homeless populations. For instance, temporary assistance provided by

transitional housing and eviction prevention may be a more effective solution for individuals and

families who are experiencing or under threat of homelessness due to a sudden and temporary

psychological, social, or economic shock (Gubits et al., 2016; Curtis et al., 2013). In these cases,

temporary assistance may go a long way toward re-establishing housing stability. Short-term

assistance, on the other hand, may provide fewer positive outcomes for individuals and families

with greater needs, such as chronic health problems or long-term unemployment. More long-

term assistance provided by permanent supportive housing can lead to greater housing stability

(Gubits et al., 2016). Additionally, a long-term subsidy, longer than 18 months, can increase

positive outcomes such as adult and child well-being (Gubits et al., 2016).

More recently, homelessness programs have tried new approaches to stabilizing housing for

individuals and families. Specifically, there has been an increased emphasis on Housing First

programs as well as rental subsidy programs for those struggling to pay for increasing rental

costs. The Housing First intervention “[provides] permanent, independent housing without

prerequisites for sobriety and treatment [… and] removes some of the major obstacles in

4

obtaining and maintaining housing for consumers who are chronically homeless” (Stefancic and

Tsemberis, 2007).

Due to drastic increases in rental costs, particularly in cities like Portland, rental subsidy

programs have become more prevalent in supporting those who are unable to afford rent. Studies

have shown some individuals or families are homeless only because their income was

insufficient to maintain housing (Smock 2015). Rental assistance programs can be used for a

one-time eviction-prevention payment or for ongoing subsidies for an individual or family.

Improving the efficacy and efficiency of homeless services in the Portland-Multnomah-Gresham

CoC necessitates a clear understanding of those who utilize homeless services in the greater

Portland area and how effective these services are in ensuring stable housing. In turn, a complete

picture of the area’s homeless population requires a robust data-collection system built upon a

data-friendly organizational culture. TPI, in its mission to move people from homelessness to

housing, requires understanding both the homeless populations they serve and effective use of its

collected data.

Placements and Rental Assistance

This section presents the following: a brief description of TPI’s data-collection system; an

overview of the demographic groups to whom TPI provides placement and rental assistance

services and how these have changed over time; and a deeper analysis into patterns in participant

characteristics and their relationship to housing status for whom housing status after receiving a

subsidy is known.

Transition Projects’ Data

TPI collects a wide range of data about its participants. As required by HUD and its national

CoC program, all participant data is entered in the Homeless Management Information System

(HMIS). HMIS has emerged as a valuable resource for homeless service providers, allowing

organizations to analyze participant, service, and housing data and track unduplicated data across

many projects in a CoC.

TPI’s HMIS software collects a wide range of data on each participant such as information on:

basic demographics, including race, ethnicity, and veteran status; health and disability status;

income and welfare; location and type of services received; and current or most recent housing

status. TPI uses this data to track participant interactions and demonstrate outputs and outcomes

to funders.

For this analysis, TPI provided data on about 18,200 participants who received services between

2000 and 2016. Due to inconsistencies in older data, this analysis uses data from only 2007 to

2016. That data was merged and cleaned to result in about 17,300 usable records (Appendix A).

See Appendix B for demographic information on all TPI participants during these years. In the

following analysis, a statistical method, called an OLS regression, was used to identify

significant characteristics related to the outcome of interest. This method permits identification

of relationships while controlling for program and participant characteristics, including time

trends, race, and service provisions.

5

TPI is interested in understanding who it serves, who receives more intensive services, and how

these services and service intensities relate to outcomes. However, due to significant data

limitations, the analysis focuses on two TPI services: placement assistance and subsidy

provision.

No causal relationships can be inferred between participants and TPI programs due to an

inability to control for factors such as self-selection and need, case manager discretion, and

resource availability.

Analysis of Placement and Subsidies

Research has shown the effectiveness of programs that prioritize rapid, permanent housing

placement and/or provide rental subsidies for people and families who are homeless or at-risk of

becoming homeless; both strategies are vital tools in TPI’s service toolbox. TPI’s placement

services include participants who self-resolve into housing, participate in rapid re-housing or

permanent supportive housing programs, and/or receive eviction prevention funds. Rental

assistance subsidies are a primary component of these programs. The subsidies are rent

payments, made by case managers, to landlords on behalf of participants. Some placement and

subsidy programs require TPI to follow up and track the participant’s housing status three, six,

and 12 months after receiving a subsidy.

This analysis begins with a brief overview of TPI’s placement and subsidy programs from 2007

to 2016. The analysis proceeds with an exploration of this service data and its relationship with

outcome data – housing status – to identify participants more likely to receive the services,

characteristics related to service provisions, and associations between these service provisions

and housing status. Specifically, the analysis is designed to answer the following three questions:

1. What participant characteristics and/or services are significantly associated with being in

stable housing at follow-up?

2. What participant characteristics are associated with service provisions, namely receiving

more intensive support?

3. Do the identified patterns provide possible explanations for participant characteristics

associated with being in stable housing?

The significant data limitations that restricted analysis to only two of TPI’s services further

hamper the validity of findings and interpretations for questions 1 and 3. More specifically,

because subsidy data is the only service data that includes housing status, the findings for

questions 1 and 3 are conditional on participants receiving a subsidy. Further, without an

appropriate comparison (or control) group, interpretation cannot account for stable housing

findings for those who do not receive a subsidy.

As a result, these findings and interpretations about housing status are presented with less

confidence. However, although causality cannot be inferred, understanding identified patterns,

potential programmatic reasons that may explain them, and if the patterns align with organization

intentions can be a useful step to guide thinking, incorporate data into decision-making, and help

TPI be more effective.

6

Overview

Between 2007 and 2016, TPI provided nearly 8,000 placements to 6,300 individual participants

and over 7,400 rental assistance subsidies to 5,500 participants (Appendix C). On average, TPI

provided almost 800 placements and over 550 subsidies each year (Figure 1).

Figure 1. Number of Placements and Subsidies, 2007-2016

Source: Transition Projects HMIS Data, 2007-2016

This discrepancy in the number of services provided and participants served shows some

participants were placed multiple times and/or received multiple subsidies. As Table 1 shows,

less than one-quarter of participants were placed or received a subsidy more than once.

Receiving multiple placements or multiple subsidies may indicate a higher level of service

provisions for some participants.

Table 1. Number of Participants Who Received One or More Placements and/or Rental Subsidies, 2007-2016

Number of Subsidies Received

Number of Participants

% of Total Subsidy

Participants

Number of Times Placed

Number of Participants

% of Total Placement Participants

1 4,337 78.5 1 4,921 78.0

2 906 16.4 2 1,054 16.7

3 220 4.0 3 259 4.1

4 49 1.0 4 56 1.0

5+ 14 <1 5+ 19 <1

Source: Transition Projects HMIS Data, 2007-2016 (Subsidies n = 5,526 participants; Placements n = 6,309 participants)

Similarly, the length of the subsidy a participant receives varies considerably. While the average

subsidy length is about four months, subsidy length ranges considerably from one month to

nearly seven years (Figure 2). The majority of subsidies provided one month of rental assistance

(64 percent); just over one-third of all subsides (36 percent) were two months or longer.

Conversations with TPI staff suggest this may result from a lack of available financial resources

to give participants longer-term, ongoing subsidies.

729

714

740 702650

861

701806 870

1,203

690

728

703631 630

869

665

927 892

682

2007 2008 2009 2010 2011 2012 2013 2014 2015 2016

Placements Subsidies

7

Figure 2. Distribution of Subsidy Length

Source: Transition Projects HMIS Data, 2007-2016 (n=6,561)

Since 2007, TPI has served more men, domestic violence survivors, and veterans experiencing

homelessness, with the sharpest increase (over 160%) occurring for veterans (Appendix C). TPI

has also served more non-white participants; however, when compared to the total number of

individuals served, the proportion who are non-white has decreased slightly (Figure 3).

Figure 3. Proportion of Non-White Placement and Subsidy Participants

Source: Transition Projects HMIS Data, 2007-2016 (Subsidies n = 5,526 participants; Placements n = 6,309 participants)

Housing Status

Of the approximately 5,500 subsidies provided, the overall rate of subsidy-receiving participants

in stable housing at follow-up is 80 percent. This section seeks to identify patterns in participant

characteristics and service provisions and their relationship to housing status. The analysis

utilizes data for only those participants given subsidies requiring follow-up. This is problematic

for the validity of the findings and interpretation presented. Each of the following estimates is the

relative higher (or lower) rate of likelihood of being in stable housing associated with each

factor, conditional on receiving a subsidy and controlling for other factors.

64%

15%

9% 7%2% 2%

1 month 2 to 6months

7 to 12months

13 to 18months

19 to 24months

Over 2 years

% o

f P

art

icip

ants

Subsidy Length

27%32%

23%

27%32%

29%

2007 2008 2009 2010 2011 2012 2013 2014 2015 2016

Placements Subsidies

8

Conditional on receiving a subsidy and receiving a follow-up, several demographic

characteristics are associated with being in stable housing at statistically significant levels

(Appendix D). Males and non-white participants are less likely to be in stable housing at follow-

up; males are 9 percentage points less likely, and non-white participants are 2 percentage points

less likely. Conversely, veterans are more likely to be in stable housing at follow-up (3

percentage points).

Additionally, conditional on receiving a subsidy, there are interesting patterns between service

provisions and stable housing. For example, being placed more than once is positively associated

with being in stable housing at follow-up (3 percentage points). This may suggest that it takes

time to find the best fit between a placement and a participant’s needs or characteristics.

However, multiple placements can also be interpreted as an ineffective use of resources which

does not align with being more likely to be in stable housing, suggesting more data is needed to

investigate this finding.

Subsidy length is also significantly related to housing status. When compared to receiving one

month of rental assistance, receiving a subsidy for two to six months is negatively related to

stable housing (-6 percentage points). While a subsidy of seven to 13 months is not any different

than receiving one month of rental assistance, a subsidy for 14 or more months is positively

related to being in stable housing (7 percentage points).

Figure 4. Estimates of Various Subsidy Lengths on Stable Housing Outcomes compared to One Month Rental Assistance

1 Month

Source: Transition Projects HMIS Data, 2007-2016 (n=6,561)

Note: Parameter estimates with 95% confidence limits

These findings suggest a longer-term subsidy (14 months or longer) is more positively related to

stable housing than receiving one month of rental assistance. Because subsidy length is

influenced by participant selection and need, case manager discretion, and resource availability,

these findings only identify patterns and trends, and do not confirm that a longer subsidy

contributes to being in stable housing. However, research such as the Family Options Study

9

(2016) does support the pattern stating that families that receive a long-term housing subsidy

(defined as 18 or more months) are significantly less likely to be homeless (Gubits et al., 2016).

These findings also suggest subsidies lasting between two and six months have a small but

statistically significantly negative relationship with stable housing than one month of rental

assistance. Though this negative relationship is unusual and unexpected, the findings may

support the notion that some individuals and families require only short-term financial assistance

to overcome a sudden and temporary shock that leads to homelessness (Gubits et al., 2016).

However, it is counterintuitive that a longer subsidy would be worse than a shorter one,

suggesting the finding could be the result of inconsistent data-collection and entry.

Together, these findings are consistent with previous research finding that the relationship

between financial assistance and stable housing depends on the person and his/her situation

(Gubits et al., 2016). However, data limitations challenge the validity of the findings and

underscore a need for improved data quality.

Service Provisions

Because levels of assistance and participant characteristics have varying relationships with a

participant’s likelihood to be in stable housing, it is worthwhile to investigate which participants

are more likely to receive the services as well as the levels of services that may lead to stable

housing. This section provides a descriptive analysis of participant characteristics and their

relationship to various service provisions, including:

1. Conditional on being a TPI participant, the likelihood of being placed and/or receiving a

subsidy,

2. Conditional on being placed and/or receiving a subsidy, the likelihood of receiving more

intensive support, specifically, a longer subsidy, multiple subsides, or multiple

placements.

After being analyzed on their own, each of these findings is subsequently considered in line with

the stable housing findings from above: Does understanding who receives services and the levels

of services explain why particular sub-groups are positively or negatively related to being in

stable housing? These interpretations are purely conjecture and are intended to prompt further

questions and discussion.

Patterns in Being Placed or Receiving a Subsidy

When compared to all TPI participants, several characteristics are more strongly correlated with

being placed or receiving a subsidy (Appendix C). As previously

noted, those who received a placement are also very likely to

have received a rental subsidy, and vice versa. In fact, this is the

strongest predictor for both being placed and receiving a subsidy

(nearly 90 percentage points in both cases).

Analysis also showed that veterans are more likely to receive

placement assistance than non-veterans (7 percentage points), but

are less likely to receive a subsidy (4 percentage points less).

While this could be due to a funding source that is not growing at

Patterns in Being Placed or Receiving a Subsidy

More likely to be placed: Veterans

More likely to receive subsidy: Domestic Violence Survivors

Less likely to be placed: Males

Less likely to receive subsidy: Veterans

10

the same rate as veteran participation, conversations with TPI staff revealed a recent focus on

increasing services for veterans. Because this shift only recently occurred, the data may not

reflect the impacts yet.

Males are slightly less likely to be placed (1 percentage points less). Domestic violence survivors

are slightly more likely to receive a rental subsidy (2 percentage points). While these findings are

statistically significant, they are relatively small and suggest no demographic sub-group is being

targeted for participation in these services. This is consistent with TPI’s client-driven practice for

determining service allocation.

Though receiving placement or subsidy services does not appear to target particular participants,

service provision is much more nuanced; participants may receive varied intensities of placement

and subsidy services. For example, a participant may receive more than one subsidy or housing

placement and/or a longer subsidy. The next sections explore these nuances, identifying

participant characteristics related to receiving more intense services and considering how these

nuances fit with housing-status findings.

Patterns in Receiving Multiple Placements and Multiple Subsidies

Although most participants are placed only once and receive only one subsidy, some receive a

higher level of service, such as being placed multiple times or receiving multiple subsidies.

Previously, multiple placements were also found to be positively associated with being in stable

housing at follow-up.

Conditional on being placed and/or receiving a subsidy, several

characteristics are more strongly associated with being placed or

receiving a subsidy more than once (Appendix E). There is a

strong relationship between being placed multiple times and

receiving multiple subsidies; if a participant received multiple

placements, he/she is 70 percentage points more likely to receive

multiple subsidies, and vice versa. This aligns with the fact that

the two services are largely provided in tandem.

Veterans and non-white participants are more likely to receive multiple subsidies (about 3

percentage points for both), but less likely to be placed multiple times (4 percentage points less

and 2 percentage points less, respectively). While this may indicate increased organizational

capacity to offer financial support and more effective placement, data restrictions and limitations

in the analysis make it difficult to interpret the findings as a positive or negative outcome. For

example, the data does not include the monetary value of a subsidy, and analysis does not

indicate whether a participant received multiple short-term or multiple long-term subsidies;

limited funding and varying funding cycles may call for multiple bursts of lower value or

shorter-term rental assistance. Similarly, the analysis could not control for the length of time

between additional placements. A participant returning for placement services a short time after

being placed initially would indicate a very different situation than someone who returned years

later.

Patterns in Receiving Multiple Placements/Subsidies

More likely to receive >1 subsidies: Veterans, Non-whites

Less likely to be placed >1 time: Veterans, Non-whites

11

It is more difficult to interpret the positive relationship between multiple placements/multiple

subsidies and stable housing at follow-up. For example, veterans and non-white participants have

the same patterns in receiving these multiple services, yet their trends for stable housing are

opposite: veterans are more likely to be in stable housing at follow-up, non-white participants are

less likely. This may indicate that the strength of the relationship between more intense services

and stable housing is dependent on the person and his/her situation. But, because of the above

limitations, further research is required to explore this puzzling finding.

Patterns in Receiving an Ongoing Subsidy

A longer-term subsidy is another indicator of a higher level of service and is positively

associated with housing status if the subsidy is 14 months or greater. Data limitations do not

permit analysis of participant characteristics related to receiving a 14-month or longer subsidy;

however, analysis of participant characteristics associated with receiving a subsidy for two or

more months reveals some interesting trends (Appendix F).

Conditional on receiving a subsidy, males are less likely to receive

an ongoing subsidy (3 percentage points less), while domestic

violence survivors are significantly more likely to receive an ongoing

subsidy (4 percentage points). These findings hold when controlling

for more demographic factors such as race and prior residence.

The above finding about the positive relationship between length of subsidy and stable housing

suggests that participants who experience more intensive services are more likely to be in stable

housing at follow-up. The finding that male participants are less likely to receive an ongoing

subsidy of two months or more may partly explain why they are less likely to be in stable housing.

However, there are some inconsistencies

in these trends. For instance, domestic

violence survivors are much more likely

to receive an ongoing subsidy but not

more likely to be in stable housing at

follow-up. This may suggest that

domestic violence survivors may be

receiving subsidies for 2 to 14 months or

that they may require additional services

rather than longer-term subsidies.

In summary, though causality cannot be

inferred from these results, certain

participant characteristics are

significantly related to receiving more

intensive support: multiple subsidies,

multiple placements, or longer subsidies. When considering this finding alongside housing-status

findings, a main theme emerged: service intensity is associated with housing status. Table 2

presents a combination of findings for service provision and stable housing.

Patterns in Receiving an Ongoing Subsidy

More likely: Domestic violence survivors

Less likely: Males

12

However, this is not the case for all participant characteristics or service provisions, suggesting

that the impact of more intensive services depends on the person and his/her situation. There

appear to be systematic relationships between greater intensity of service and stable housing,

although, the strength of the relationship depends on the person and his/her situation. Addressing

the data limitations encountered and finding opportunities to increase the quality of data can

permit a more robust analysis in the future and enable TPI to better target services.

Transition Projects’ Data-Collection and Use

In the above sections, this report sought to analyze TPI’s data to identify trends that may help the

organization improve its service delivery and programs, with the goal of improving TPI’s

effectiveness. However, numerous limitations were encountered that hindered the validity of the

findings and interpretations. To further understand the data limitations as well as TPI staff’s

current use and attitudes toward data, the following section presents findings from a staff survey.

Understanding data and data measurement is critical for making progress toward specified

outcomes. However, only the items that are measured, and measured well, ultimately receive

attention (Hatry, 2006). One key component of data-collection involves integrating a focus on

data into the organization’s culture. This ensures that data is collected, entered, and used

correctly and ultimately effectively. In doing so, it is also important to consider staff perceptions

and feelings about data and its use (Partnership for Public Service, 2012). The survey was

distributed to TPI staff to assess the existing culture regarding data-collection and use.

Approximately 60 staff members (25 percent of all TPI staff) responded to the survey. This

section summarizes some of the major themes. For more information regarding the survey

questions and results, see Appendix G.

Data use is relatively frequent among TPI employees. The majority of all employees spend up to

ten hours a week entering and using data for various tasks. About three out of four (74 percent)

employees surveyed perceive the quality of data they enter to be excellent, but the quality of data

used is rated equally often as neutral (46 percent) or excellent (46 percent). This suggests some

difference in judgment between the work individual employees do in entering data versus the

data that is ultimately part of the TPI data set.

All employees agreed that they want TPI to invest in good data quality and improve data usage

(82 percent). However, other opinions and ideas about data and the data system varied between

groups. While the majority of employees generally agreed that they were comfortable using the

data system, administrators were much more likely to be comfortable with it. Case managers and

shelter staff were more likely to have issues using the data system.

Summary of Survey Findings

Data use is frequent; however, perspectives on data quality vary.

Employees agree about investing in good data quality and improving data usage.

Many case managers and shelter staff find data entry takes too much time and seek additional resources to support their use of the data system.

Employees value data-collection, but also stress a desire to maintain client-driven focus.

13

Case managers described data entry as “tedious” and “frustrating,” and noted that “[the system]

changes frequently and it does not always produce an accurate snapshot of participants’

situations.” Half of all case managers surveyed agreed that data entry takes too much time. One-

quarter of shelter staff similarly felt that data entry is time consuming. One shelter employee

reported that “the data options available do not capture all the things [they] do in [their] role.”

Both case managers and shelter staff stated interest in improved data training as a strategy to help

lessen the time used in data entry and reduce duplicate and erroneous data points.

Administrators and shelter staff were more likely to feel that the most important parts of their

jobs require using data; case managers agreed with this statement only 33 percent of the time.

However, in the comments to this question, all employees shared similar perspectives on the

value of data. All survey participants echoed the idea that data is important to securing funding

and providing good services to TPI participants, but also identified compassion and interest in

the individuals as important. All survey participants indicated their hope that data could be used

to help improve TPI’s performance, increase their understanding of data’s role in providing

services to participants, and identify opportunities to better direct resources.

Case managers often have very different perspectives on the usefulness of data compared to

administrative staff and shelter staff. Case managers were more likely than the other groups to

agree that it takes too much time to enter data (53 percent) and that the most important purpose

of data is to satisfy external reporting requirements (67 percent). Many case managers, despite

demonstrating resistance to using data, describe finding value and wanting to use data to help

participants in open-ended questions.

While there is some disagreement among TPI employees about the value of being a data-driven

organization, most survey respondents found value in having data available. Many shelter staff

and case managers reported going beyond the required data-entry tasks to help participants find

the best possible programs and services. It is also encouraging to see an overwhelmingly positive

response toward investing in better data quality and use, signaling willingness across the

organization to make changes.

Discussion

TPI, in its mission to move people from homelessness to housing, requires understanding the

participants served as well as staff perceptions about data-collection and use. Analysis of TPI’s

placements and rental-subsidy programs identified relationships among participant

characteristics, service provisions, and outcomes. Though causality cannot be inferred from these

results, the level of assistance a participant receives is associated with his/her housing status.

Even controlling for other factors, there appear to be systematic relationships between assistance

and stable housing, but the strength of the relationship depends on the person and his/her

situation. However, inconsistencies within Transition Projects’ data challenge the validity of

these findings and prevented a more robust analysis of the organization’s services. Considering

the identified patterns can help guide thinking and organizational decision-making, but

improvement to TPI’s data-collection processes can ensure the accuracy of collected data and

help build TPI’s evidence base.

14

The survey findings of TPI’s data culture identified possible areas for improving data-collection.

While employees are concerned that increased data use may move the organization away from

being client-driven, they also would like TPI to invest in better data quality and use. These

survey findings informed the creation of actionable items in the report’s recommendations.

Alongside these recommendations are suggestions for how TPI can build upon the analysis

findings to position itself for better understanding its participants and service outcomes, and, as a

result, become an even more effective homeless service provider.

Recommendations

Incorporating a focus on data into an organization is a rather onerous process, from establishing

ways to collect useful data to integrating data into decision-making. A decision-making model

with structured steps is illustrated below. It simplifies an ideal decision-making process used by

an organization that efficiently collects and uses data. The model is circular because if the

process is repeated it leads to continuously improved services. Although the following

recommendations emphasize smaller, actionable steps for TPI, the model is provided purely for

reference.

Figure 5. Incorporating Data into Organization Decision-Making Model

Plan/Brainstorm

What do we want to know (outputs/outcomes/impact)? What data do we need to get

there?

Build

Build and standardize processes that allow

collection of useful and appropriate data.

Collect & Analyze Data

Run analyses, which may require hiring an analyst or

contracting outside the organization.

Evaluate

Compare results to programmatic goals. Are we performing how we want to

be?

Integrate

How can we share what we learned and be a thought leader in the

community?

15

Recommendation 1: Further integrate data into Transition Projects’ culture

To work toward the model above, TPI should continue integrating a focus on data into the

existing culture by promoting a shared understanding of the importance of data and by creating

learning forums.

All organizations have a culture, or a shared set of values that drive the mission, and TPI’s

values include being client-driven, compassionate, and equitable (Transition Projects, 2016).

Further integrating data as an organizational value can help TPI more effectively serve

participants and further its mission. From the survey, it is apparent that staff support data use.

TPI also has demonstrated capacity and support for data measurement, and its employees have

responded positively to investing in data quality.

These are among the key cultural components an organization needs to work toward integrating a

data focus (Hatry, 2006).

Promote an understanding of the importance of data

One way to sustain changes in data use is to reinforce the reason why quality data is important

and how it is used. Some staff expressed mixed feelings about data entry, fearing data use

conflicts with the organization’s client-driven approach. Additionally, staff may fear that data is

being used to evaluate their work. By being transparent about how data is used and educating

staff about the benefits of collecting data, TPI can help staff overcome some of these fears.

Additionally, the limitations of data should be made clear to all staff outcomes tell only what

happened, not why (Hatry, 2006). The literature includes a myriad of other strategies for

integrating data into the existing culture; however, these recommendations appear to be most

relevant to TPI, given survey findings.

Create learning forums

Another strategy to integrate data as an organizational value involves creating learning forums.

Learning forums provide opportunities to discuss data findings. The survey revealed that some

shelter staff integrate service data with participant meetings to guide decision-making about

service referrals. Additionally, learning forums may encourage employees to share best practices

for collecting data. Providing an opportunity for staff members and others to share how they are

using data may identify new techniques that could be applied across the organization. At the

Recommendations for Transition Projects

1. Further integrate data into Transition Projects’ culture.

2. Formalize data-collection processes.

3. Evaluate data in terms of programmatic intentions and integrate into decision-making.

4. Garner sector-wide support for building the evidence base for homelessness programs.

16

same time, this will increase staff buy-in to data-collection processes; staff will feel they are

valued and have a greater stake in how TPI uses data (Partnership for Public Service, 2012). By

encouraging TPI as an organization to learn from its employees, goals and processes can become

clearer to all stakeholders.

Recommendation 2: Formalize data-collection processes

This report examined TPI’s placements and rental-subsidy programs and identified relationships

among participant characteristics, service provisions, and outcomes. The La Follette School team

received more data from TPI than it used in the analysis. The following list highlights several

data limitations, grouped by whether these limitations were caused by inconsistencies in data

entry, data-collection, or data use. Addressing these limitations will increase TPI’s capacity to

produce a more robust analysis. The recommendation concludes by suggesting a specific

formalization process for collecting data on services.

Inconsistency in Data Entry

Service categories were tracked in three separate areas. Focusing on

one service was difficult because there was no clear pattern by which to

categorize them. Additionally, two specific areas for entering service

descriptions included over 50 unique service options, and combinations

of the two services entered per participant were inconsistent.

Subsidies were ultimately used by the La Follette Team for this report,

but there were some difficulties with this data as well. Subsidy start and

end dates were recorded inconsistently across TPI. Some were recorded

with an end date at the end of a month, while others had identical start

and end dates.

Inconsistency in Data-Collection Disability data was ultimately not able to be used for the analysis. Although TPI collects

data on disability diagnosis, it was lacking for most of the participant data. Additionally,

disabilities were recorded as whether a participant had a disabling condition, but lacked

information on the severity or current status of the disability. Lastly, due to changes in

disability definition and collection around 2014, the information was not able to be

incorporated into the report.

Inconsistency in Data Use

The purpose of data was unclear for many case managers and shelter staff surveyed.

This lack of clarity can lead to fear or resistance to using data in the future.

Decision-making was a common ideal goal for data use, but the suggested decisions

varied widely. Some case managers feared decisions based on data may be punitive,

performance-based decisions that would impact their job status. Shelter staff were

interested in seeing data used to make programmatic decisions, like planning new

shelters. Others wanted to know more about how the data help TPI as an organization

make decisions. Clarity in decision-making can make data goals more concrete and help

orient data-collection toward meeting agency and program goals.

Inconsistent Data

Race

Ethnicity

Disability

Services

Subsidy

End Date

17

As described in the analytical priorities for data-collection

to the left, additional data-collection could contribute to a

more robust analysis of participant characteristics, services,

and their impact on outcomes. For example, participant

income would add more depth to the analysis of subsidy

intensity, while causality could be potentially inferred if

housing status was recorded for all TPI participants.

By building capacity to track some analytic priority areas

and addressing some of the preceding inconsistencies, TPI

can strengthen its ability to perform more robust analyses

in the future. The following sub-sections and Appendix H provide more specific suggestions and

techniques for improving data-collection processes.

Example of Improving Data-Collection with Services

One approach to creating a more consistent data entry process might include creating clear

consensus around service definitions and the services most important for tracking across the

organization. Including all levels of employees in this discussion can provide several benefits.

For example, administrators, case managers, and shelter staff have unique and diverse

perspectives about different services and integrating these unique perspectives can help TPI gain

a more in-depth understanding of the most relevant service categories (Arygris and Schon,

1996).

Including all levels of staff also helps improve employee buy-in about data-collection, which in

turn improves the chances for better data quality (Partnership for Public Service, 2012). After

agreeing on service categories, disseminating the definitions across the agency and providing

TPI staff training can help maintain data quality. The checklist in Appendix H provides concrete

steps for TPI to use for other data-collection improvements.

Recommendation 3: Evaluate data in terms of programmatic intentions and integrate into decision-making TPI should consider its programmatic goals when evaluating the data findings in this report and

future analyses and should incorporate this evaluation into programmatic decisions. This report’s

analysis focused on three questions: 1. What participant characteristics and/or services are significantly associated with being in

stable housing at follow-up?

2. What participant characteristics are associated with service provisions, namely receiving

more intensive support?

3. Do the identified patterns provide possible explanations for participant characteristics

associated with being in stable housing?

These questions can serve as guidance for TPI when it begins to explore how to link its data with

its programmatic intentions. Integrating collected data, programmatic intentions, and decision-

Analytical Priorities for Data-Collection

Participant Income

Subsidy Value

Placement Intervention Type

Grant Allocation by Subgroup

Housing Status for Each Participant

18

making processes can help TPI leverage required data-collection to further its mission. This

recommendation concludes with a specific process for evaluating service data.

Process of Evaluating Findings

Staff can use this analysis to better understand who is currently served, who is likely to use

services and programs, and who is likely to experience positive outcomes, such as stable

housing. Specifically, TPI can consider why certain patterns are significant and how they align

with programmatic goals and expectations; doing so can help TPI be more intentional with its

programming and services.

The unique relationship between multiple placements and multiple subsidies is an example of a

finding worth investigating. Participants who come back for multiple placements are very likely

to receive multiple subsidies, and vice versa. This may make sense because rental assistance

often accompanies TPI’s placement services; but, this might spur case managers to ask why

participants are returning for additional placements? Further, in addition to receiving a subsidy,

participants who received more than one placement were more likely to be in stable housing. Is

receiving more than one placement supposed to be related to stable housing? Is there a

programmatic reason behind this finding? Or are participant characteristics influencing this

finding? Assessing program data in this manner is helpful for identifying program and service gaps. After

TPI understands more about the reach of its services, it can develop a strategy to better target

them.

Continuous Improvement

Upon developing a strategy, TPI can continue to evaluate how its programs and services are

performing. Routinizing this process allows for continuous process improvement, as illustrated

in the decision-making model above. One way TPI can institutionalize this process is by

incorporating findings into a strategic plan and then measuring program data against the strategic

plan’s goals. Another way would be to hold routine meetings to review and discuss program data

findings.

Recommendation 4: Garner sector-wide support for building the evidence base for homelessness programs

TPI’s knowledge of the network of service providers and city collaborations working to end

homelessness can be essential to continue developing the sector’s evidence base. For instance,

Portland’s city-wide initiative “A Home for Everyone” values using data to guide decision-

making. This creates a venue for TPI to share findings about both the opportunities for using data

and the fact that current organizational data may be restricted in its ability to identify causal

effects. More specifically, this data analysis was restricted by the inability to identify a

comparison group for investigating causal effects of subsidy length on housing status.

Investigating this question and others like it requires more rigorous, controlled evaluations.

19

Thought Leadership

TPI can use its position as a member, and often a leader, in collaborations to highlight the

importance of controlled studies for more precisely evaluating and identifying the most critical

and effective assistance programs. Current data can be used as evidence to encourage funder

support for additional, more rigorous analysis, such as experimental, or even randomized

designs, that may have the most potential to develop an evidence base regarding the impact of

programs.

Conclusion

Analysis of TPI’s placements and rental subsidy programs described relationships among

participant characteristics, service provisions, and housing stability outcomes. Some participant

sub-groups were found to receive more (or less) intensive services when compared to other

participants. However, due to data limitations, findings related to housing status were

inconclusive, counter-intuitive, or potentially false – limiting their validity. Improving data-

collection and further integrating data into the organization’s culture present opportunities to

improve TPI’s ability to perform more robust analyses in the future and continue to build the

evidence base. The report provided recommendations for how TPI can build upon the findings

and better evaluate its assistance programs, monitor successes, and learn from its data to become

a more effective homeless service provider.

Data can be a useful tool to increase knowledge about TPI and its participants, and how TPI can

better serve its community. However, more data for the sake of having data can be as hindering

as collecting no data or poorly collecting data. Being cognizant of the organization’s needs and

the variables that are necessary to further TPI’s mission and goals can help deter data overload

and meaningless data-collection. Also, finding avenues within current HMIS requirements to

collect data of interest can mitigate overloading staff.

TPI is a leader within the Portland community. The area can greatly benefit from TPI’s lead in

improving data quality and reliability, in turn increasing the ability to analyze data across

programs and organizations. TPI has a strong organizational culture. If it can leverage some of

the findings and recommended strategies, TPI can greatly increase its capacity for outcome

evaluation and move Portland closer to finding a home for everyone.

20

Appendix A: Notes on Data Cleaning and Recoding

Data Cleaning

To create the data set used for analysis, demographic data were first cleaned as described in the

following sections. After demographic data were finalized, they were merged onto the

placements and subsidy follow-up data sets. Only participants with demographic data are

considered in our analysis of TPI placements and rental subsidies.

Race

The original HMIS data records race and ethnicity across three variables. Two variables record

race, and a third records if a participant is Hispanic. When participants come to TPI, they are

allowed an open-ended response to the race category, which allows for participants to describe

their race as they like. As a result, these variables are not exclusive, because participants can be

double-counted in various categories. To simplify the categories, we recoded the 8-category race

variable into the following:

White, Non-Hispanic White

AND

Non-Hispanic

Non-White,

Non-Hispanic

Black or African American

Asian

American Indian or Alaska Native

Native Hawaiian or Other Pacific Islander

Other

AND

Non-Hispanic

Hispanic Hispanic

Disability

The HMIS data file on disability was originally in a long format, with multiple entries for each

participant. Disability type was captured in one categorical variable. From this variable, we

created indicators for each disability type. To convert the data to a wide format (one entry per

participant), we collapsed the data set to collect a count of each disability type for each

participant. As a result, our disability data is slightly different from how TPI uses it. Rather than

showing the disabilities a participant has over each visit, we have their cumulative disabilities up

to their most recent visit to TPI, so our counts for disability type may differ from TPI counts.

Disability data were not available for all TPI participants.

Prior Residence

We recoded the prior residence variable into a three-category variable, using HMIS definitions.

The following table shows the kinds of responses that fall into each of these three larger

categories.

21

Literally Homeless Place not meant for habitation (e.g., a vehicle, an abandoned

building, bus/train/subway station/airport, or anywhere

outside)

Emergency shelter, including hotel or motel paid for with

emergency shelter voucher

Save Haven

Interim housing

Institutional Situation Foster care home or foster care group home

Hospital or other residential non-psychiatric medical facility

Jail, prison, or juvenile detention facility

Long-term care facility or nursing home

Psychiatric hospital or other psychiatric facility

Substance abuse treatment facility or detox center

Transitional and

Permanent Housing

Situation

Hotel or motel paid for without emergency shelter voucher

Owned by client, with or without ongoing housing subsidy

Permanent housing for formerly homeless (such as: a CoC

project, HUD legacy programs, or Housing Opportunities for

Persons With Aids (HOPWA))

Rental by client, with or without subsidy

Residential project or halfway house with no homeless

criteria

Staying or living in a friend’s or family’s room, apartment, or

house

Transitional housing for homeless persons

Placements

For the purposes of this report, a participant is considered to have been placed in housing only

when he or she appears in TPI’s placement data. Conversely, any TPI participant who did not

appear in the placement data is not considered in any analysis using placement or housing status.

Within the data provided, some participants were recorded as having been placed in housing

multiple times in a single day. Multiple same-day placements were considered duplicative entries

and were dropped for the purposes of the analysis. Additionally, within the provided data, the

number of placement services recorded for 2016 diverged significantly from TPI’s internal

reporting on placements.

Multiple Placements

Multiple placements refers to whether a participant was placed in housing more than once within

the 10 years of TPI data analyzed. A participant who placed in housing more than once at any

point between 2007 and 2016 was recorded as having been placed multiple times and marked

with a 1. Participants who were placed only once in these 10 years were marked with a zero for

this variable. As stated above, some participants were recorded as having been placed multiple

22

times on a single day. These were considered to be data-entry errors, and one of the entries was

dropped for analysis purposes.

Subsidy Length

Subsidies in this report are only rental subsidies provided in follow-up data. Any TPI participant

who did not appear in the follow-up data is not considered in any analysis using subsidy or

housing-status data. Using the information provided on subsidy start and end dates, subsidy

lengths were counted as the number of days between start and end dates. Because some one-

month subsidies are recorded as ending either on the same day as the start date or at the end of

the month, any subsidy length of 30 days or less was considered a one-month subsidy.

Otherwise, subsidy-length days were converted to subsidy-length months. Based on observed

patterns in the number of participants with subsidies of varying lengths, subsidy lengths were

grouped into 1 month, 2- to 6-month, 6- to 14-month, and more than 14 months.

Multiple Subsidy

Multiple subsidy refers to the number of unique subsidy start dates for each TPI participant. Each

of these unique subsidy start dates was counted for each participant, resulting in a count of

subsidies. From here, an indicator was created to signal if a participant received one subsidy or

more than one subsidy. A 1 indicates that a participant had more than 1 subsidy, while a 0

indicates the participant had only 1 subsidy.

Housing Status

For the purpose of this report, only the housing status at the most recent participant follow-up

was considered. For some participants, housing status changes between follow-up dates, so our

analysis lacks this particular nuance. To determine stable housing, an indicator variable was

created, with a value of 1 for participants in stable housing and 0 if participants were not in

stable housing.

23

Appendix B: TPI Demographics

Overall, TPI serves a majority white population. However, non-white populations are

overrepresented in homeless populations across Oregon and within TPI populations. Most TPI

participants overall and within the timeframe of our analysis were men. Interestingly, in January

2015, TPI served a more equal population of men and women, which differs from the gender

breakdown of homeless individuals in the January Point in Time (PIT) estimate (70 percent

male, 30 percent female).

Roughly 40 percent of the TPI population has a disability, and many individuals have more than

one disability. The most common disabilities among TPI participants were addiction (including

drug, alcohol, and both drug and alcohol addictions), chronic health conditions, developmental

conditions, HIV/AIDS, and mental health problems.

The majority of TPI participants came to TPI after a period of being literally homeless, which

includes sleeping on the street or living in a place unfit for habitation. The next most common

prior residence was transitional and permanent housing solutions.

Domestic violence survivors and veterans were not uncommon in TPI. Nearly one-quarter of all

TPI participants were domestic violence survivors, and about one in five TPI participants were

veterans.

Using January 2015 data, TPI participants can be compared to the broader homelessness

demographics of its CoC. In this month, TPI served 195 participants. Like the CoC, TPI serves a

disproportionately high population of non-white individuals compared to the racial makeup of

Multnomah County and the city of Portland. Women made up 51 percent of TPI participants for

January 2015, a higher proportion than the proportion of homeless women on the PIT estimate.

Veterans made up 22 percent of the TPI population in January 2015.

A table summarizing the demographics of TPI overall, TPI in the last decade, and TPI in January

2015 follows.

24

Table 3. Demographics of All TPI Participants, TPI Participants from 2007-2016, and TPI Participants in January 2015 TPI Overall (n=18,346)

TPI, 2007-2016 (n=16,949)

TPI January 2015 (n=195)

Race White 62 58 55 Non-White 24 23 18 Hispanic 8 8 7 Gender Male 62 61 49 Female 38 39 51 Disabled (total) 42 40 26 Addiction 23 21 10 Chronic Health Condition 16 15 9 Developmental 15 13 5 HIV/AIDS 15 13 6 Mental Health Problem 28 27 18 Other 8 9 1 Prior Residence Literally Homeless 57 56 49 Institutional Situation 12 11 21 Transitional and Permanent Housing Solution

31 312 29

Domestic Violence Situation <1 <1 0 Other 1 1 2 Domestic Violence Survivor 28 27 26 Veteran 23 23 22

Source: TPI HMIS Data, 2007-2016 (n=18,346)

25

Appendix C: Placements and Subsidies Analysis Tables

The following tables (4 and 5) support the description of placements and rental-subsidy data as

described in the Placements and Rental Assistance section. Between 2007 and 2016, TPI

provided nearly 8,000 placements to 6,300 individual participants and more than 7,400 rental

assistance subsidies to 5,500 participants. Participants often receive both placements and

subsidies over time.

Table 4. TPI Placements per Year, with Demographic Data

Year Unique

Participants Total

Placements Male Male (%) Nonwhite

Nonwhite (%) Hispanic

Hispanic (%) Veterans

Veterans (%)

DV Survivor

DV Survivor

(%)

2007 719 729 395 55 195 27 40 6 152 21 150 21 2008 675 714 371 55 206 31 43 6 110 16 169 25 2009 687 740 441 64 203 30 35 5 164 24 186 27 2010 662 702 453 68 214 32 40 6 194 29 175 26 2011 611 650 384 63 198 32 40 7 120 20 178 29 2012 824 861 553 67 260 32 64 8 231 28 229 28 2013 663 701 407 61 202 30 36 5 195 29 206 31 2014 762 806 502 66 213 28 64 8 231 30 198 26 2015 820 870 556 68 210 26 46 6 335 41 197 24 2016 1160 1203 612 53 270 23 61 5 396 34 214 18

All Years 7583 7976 4674 62 2171 29 469 6 2128 28 1902 25

Source: TPI HMIS Data, 2007-2016 (n=6,309)

Table 5. TPI Subsidies per Year, with Demographic Data

Year Unique

Participants Total

Subsidies Male Male (%) Nonwhite

Nonwhite (%) Hispanic

Hispanic (%) Veteran

Veteran (%)

DV Survivor

DV Survivor

(%)

2007 478 690 280 59 128 27 29 6 99 21 106 22 2008 508 728 278 55 143 28 34 7 70 14 121 24 2009 470 703 295 63 123 26 26 6 102 22 128 27 2010 440 631 286 65 140 32 23 5 101 23 109 25 2011 441 630 264 60 138 31 31 7 67 15 129 29 2012 659 869 424 64 191 29 51 8 183 28 184 28 2013 494 665 315 64 138 28 29 6 167 34 142 29 2014 672 927 461 69 164 24 65 10 261 39 165 25 2015 715 892 504 70 176 25 38 5 321 45 161 23 2016 649 682 427 66 191 29 40 6 241 37 159 24

All Years 5526 7417 3534 64 1532 28 366 7 1612 29 1404 25

Source: TPI HMIS Data, 2007-2016 (n=5,526)

26

The following regression tables (6 and 7) show how participant characteristics predict the

likelihood of being placed or receiving a subsidy, when compared to all TPI participants.

Approximately 12,000 participants had complete data records of gender, race, ethnicity, and if

they are a veteran or domestic violence survivor.

Table 6. Receive Placement Services Regression Coefficients

In Placement In Placement In Placement In Placement

In Subsidy 0.885*** 0.909*** 0.912*** 0.905*** (253.79) (268.17) (262.10) (231.08) Male 0.013*** 0.016*** -0.014** (4.01) (4.83) (-2.89) Nonwhite 0.005 0.004 0.006 (1.36) (1.15) (1.43) Hispanic -0.004 -0.004 0.012

(-0.58) (-0.59) (1.28) Veteran 0.072*** (11.13)

Domestic Violence Survivor -0.010 (-1.91)

Constant 0.076*** 0.043*** -0.007* 0.002 (32.87) (15.13) (-2.24) (0.28) Year Fixed Effect No No Yes Yes

Observations 18608 16493 16493 11936 * p < 0.05, ** p < 0.01, *** p < 0.001

mean coefficients; t-score in parentheses

Source: TPI HMIS Data, 2007-2016 (n=18,608)

Table 7. Receive a Subsidy Regression Coefficients

In Subsidy In Subsidy In Subsidy In Subsidy

In Placement 0.824*** 0.878*** 0.866*** 0.882*** (173.76) (206.16) (189.76) (194.13) Male -0.005 -0.010** 0.006 (-1.51) (-3.04) (1.41) Nonwhite 0.004 0.004 0.002 (1.16) (1.16) (0.38) Hispanic -0.004 -0.001 -0.009 (-0.55) (-0.17) (-1.03) Veteran -0.036*** (-5.82) Domestic Violence Survivor 0.015** (2.96) Constant 0.017*** 0.020*** 0.010** 0.006 (14.79) (8.17) (3.02) (1.15) Year Fixed Effect No No Yes Yes

Observations 18608 16493 16493 11936 * p < 0.05, ** p < 0.01, *** p < 0.001

mean coefficients; t-score in parentheses

Source: TPI HMIS Data, 2007-2016 (n=18,608)

27

Appendix D: Housing Status Analysis Tables

The following tables (8 and 9) support the analysis described in the Housing Status section.

Approximately 4,800 participants had complete data records including gender, race, ethnicity,

prior residence, and if they are a veteran or domestic violence survivor.

Table 8. Housing Status Regression Coefficients Conditional on Receiving a Subsidy

Housing Status at Last Follow-Up

Housing Status at Last Follow-Up

Housing Status at Last Follow-Up

Housing Status at Last Follow-Up

2- to 6-Month Subsidy -0.046** -0.048** -0.047** -0.055*** (-3.09) (-3.25) (-3.17) (-3.51)

7- to 13-Month Subsidy 0.006 0.002 0.001 0.003 (0.39) (0.14) (0.07) (0.19) 14+ Month Subsidy 0.075*** 0.068** 0.064** 0.066** (3.49) (3.17) (2.91) (2.92) Given Placement -0.048 -0.048 -0.064

(-1.19) (-1.15) (-1.13) Placed more than 1 time 0.037*** 0.033** 0.034** (3.50) (3.07) (3.07) Male -0.061*** -0.085*** (-5.62) (-5.92) Nonwhite -0.030** -0.024* (-2.58) (-1.98) Hispanic -0.006 0.006 (-0.21) (0.22) Veteran 0.029* (2.16) Domestic Violence Survivor -0.015 (-1.02) Constant 0.610*** 0.647*** 0.698*** 0.713*** (38.30) (15.08) (15.45) (11.90) Year Fixed Effect Yes Yes Yes Yes

Observations 5513 5513 5295 4837 * p < 0.05, ** p < 0.01, *** p < 0.001

mean coefficients; t-score in parentheses

Source: TPI HMIS Data, 2007-2016 (n=5,513)

28

Table 9. Housing Status Descriptive Statistics

Not in Stable Housing

In Stable Housing

Total

Male 0.723 0.634 0.651 (0.448) (0.482) (0.477) Nonwhite 0.318 0.287 0.293 (0.466) (0.453) (0.455) Hispanic 0.0413 0.0431 0.0428 (0.199) (0.203) (0.202) Veteran 0.270 0.289 0.285 (0.444) (0.453) (0.452) Domestic Violence Survivor 0.243 0.297 0.287 (0.468) (0.479) (0.477) 2- to 6-Month Subsidy 0.173 0.157 0.160 (0.378) (0.364) (0.367) 7- to 13-Month Subsidy 0.112 0.126 0.123 (0.316) (0.332) (0.329) 14+ Month Subsidy 0.0381 0.0717 0.0653 (0.192) (0.258) (0.247) Given Placement 0.995 0.990 0.990 (0.0736) (0.102) (0.0971) Placed more than 1 time 0.396 0.465 0.452 (0.489) (0.499) (0.498)

Observations 1035 4478 5513 * p < 0.05, ** p < 0.01, *** p < 0.001

mean coefficients; t-score in parentheses

Source: TPI HMIS Data, 2007-2016 (n=1,279)

29

Appendix E: Multiple Subsidy and Multiple Placements Analysis Tables

The following tables (10-13) support the analysis described in the Service Provision section.

There are some characteristics more correlated with being placed or receiving a subsidy more

than one time. Approximately 4,600 participants had complete data records including gender,

race, ethnicity, prior residence, and if they are a veteran or domestic violence survivor.

Table 10. Multiple Subsidy Regression Coefficients

Given more than 1 Subsidy

Given more than 1 Subsidy

Given more than 1 Subsidy

Given more than 1 Subsidy

Given more than 1 Subsidy

Placed > 1 time 0.556*** 0.578*** 0.738*** 0.729*** 0.728*** (43.80) (43.80) (62.12) (58.81) (58.20) Male -0.003 -0.018 -0.012 (-0.34) (-1.84) (-1.16) Nonwhite 0.027*** 0.030*** 0.027** (3.43) (3.44) (3.12) Hispanic -0.017 -0.013 -0.015 (-0.88) (-0.67) (-0.75) Veteran 0.025** 0.028** (2.61) (2.88) Domestic Violence Survivor

-0.006 -0.002

(-0.55) (-0.17) Literally Homeless -0.035 (-0.87) Institutional Situation -0.016 (-0.40) Transitional and Permanent Housing Solution

0.003

(0.08) Constant 0.218*** 0.141*** 0.020* 0.032*** 0.046 (37.06) (9.46) (2.25) (3.46) (1.10) Year Fixed Effect No Yes Yes No Yes

Observations 6309 6177 5073 4589 4573 * p < 0.05, ** p < 0.01, *** p < 0.001

mean coefficients; t-score in parentheses

Source: TPI HMIS Data, 2007-2016 (n=6,309)

30

Table 11. Multiple Subsidies Descriptive Statistics

One Subsidy More than One Subsidy

Total

Placed > 1 time 0.119 0.848 0.358 (0.324) (0.359) (0.480) Male 0.662 0.637 0.654 (0.473) (0.481) (0.476) Nonwhite 0.242 0.309 0.264 (0.429) (0.463) (0.441) Hispanic 0.0429 0.0368 0.0409 (0.203) (0.188) (0.198) Veteran 0.417 0.338 0.391 (0.493) (0.474) (0.488) Domestic Violence Survivor 0.290 0.328 0.302 (0.454) (0.470) (0.459) Literally Homeless 0.616 0.480 0.572 (0.487) (0.500) (0.495) Institutional Situation 0.125 0.113 0.121 (0.331) (0.317) (0.326)

Transitional and Permanent Housing Solution

0.255 0.402 0.303

(0.436) (0.491) (0.460)

Observations 4337 2188 6525 * p < 0.05, ** p < 0.01, *** p < 0.001

mean coefficients; t-score in parentheses

Source: TPI HMIS Data, 2007-2016 (n=1,247)

31

Table 12. Multiple Placements Regression Coefficients

Placed more than 1 time

Placed more than 1 time

Placed more than 1 time

Placed more than 1 time

Given more than 1 Subsidy

0.425*** 0.483*** 0.806*** 0.795***

(36.78) (40.11) (77.25) (72.91) Male -0.006 0.019 (-0.74) (1.80) Nonwhite -0.023** -0.025** (-2.84) (-2.84) Hispanic 0.023 0.020 (1.13) (0.94) Veteran -0.044*** (-4.25) Domestic Violence Survivor

0.020

(1.78) Constant 0.075*** -0.056*** -0.004 -0.015 (18.43) (-7.15) (-0.54) (-1.43) Year Fixed Effect No Yes Yes Yes

Observations 6309 6177 5073 4589 * p < 0.05, ** p < 0.01, *** p < 0.001

mean coefficients; t-score in parentheses

Source: TPI HMIS Data, 2007-2016 (n=6,309)

Table 13. Multiple Placements Descriptive Statistics

One Placement More than One Placement

Total

Given more than 1 Subsidy 0.0774 0.776 0.327 (0.267) (0.418) (0.469) Male 0.660 0.641 0.654 (0.474) (0.480) (0.476) Nonwhite 0.261 0.269 0.264 (0.439) (0.444) (0.441) Hispanic 0.0375 0.0471 0.0409 (0.190) (0.212) (0.198) Veteran 0.438 0.307 0.391 (0.496) (0.462) (0.488) Domestic Violence Survivor 0.283 0.336 0.302 (0.451) (0.473) (0.459) Literally Homeless 0.609 0.504 0.572 (0.488) (0.501) (0.495) Institutional Situation 0.131 0.103 0.121 (0.338) (0.304) (0.326)

Transitional and Permanent Housing Solution

0.255 0.390 0.303

(0.436) (0.488) (0.460)

Observations 4921 1388 6309 * p < 0.05, ** p < 0.01, *** p < 0.001

mean coefficients; t-score in parentheses

Source: TPI HMIS Data, 2007-2016 (n=1,247)

32

Appendix F: Ongoing Subsidy Analysis Tables

The following tables (14 and 15) support the analysis of ongoing subsidies as described in the

Service Provision section. Of the approximately 7,400 subsidies provided between 2007 and 2016,

88 percent (6,500 subsidies) included an end date, permitting calculation of the length of the subsidy

a participant received. Approximately 5,700 participants had complete data records including

gender, race, ethnicity, prior residence, and if they are a veteran or domestic violence survivor.

Table 14. Ongoing Subsidy Regression Coefficients

Subsidy Received for more than 1 month

Subsidy Received for more than 1 month

Subsidy Received for more than 1 month

Male -0.026* -0.023 -0.034* (-2.02) (-1.44) (-2.06) Nonwhite -0.041** -0.026 -0.024 (-3.10) (-1.90) (-1.74) Hispanic 0.013 0.002 0.003 (0.44) (0.06) (0.11) Veteran 0.002 0.008 (0.15) (0.54) Domestic Violence Survivor 0.038* 0.036* (2.25) (2.14) Literally Homeless 0.052 (0.60) Institutional Situation -0.063 (-0.70) Transitional and Permanent Housing Solution

0.003

(0.03) Constant 0.389*** 0.262*** 0.241** (34.51) (10.37) (2.67) Year Fixed Effect No Yes Yes

Observations 6288 5730 5711 * p < 0.05, ** p < 0.01, *** p < 0.001

mean coefficients; t-score in parentheses

Source: TPI HMIS Data, 2007-2016 (n=6,288)

Table 15. Ongoing Subsidy Descriptive Statistics

1 Month Subsidy Subsidy for 1+ Month Total

Literally Homeless 0.526 0.521 0.523 (0.500) (0.500) (0.500) Institutional Situation 0.126 0.0975 0.111 (0.332) (0.297) (0.315) Transitional and Permanent Housing Solution

0.343 0.380 0.362

(0.475) (0.486) (0.481) Male 0.661 0.691 0.676 (0.474) (0.462) (0.468) Nonwhite 0.312 0.270 0.291 (0.464) (0.444) (0.454) Hispanic 0.0381 0.0404 0.0392 (0.192) (0.197) (0.194)

Observations 4210 2278 6488 * p < 0.05, ** p < 0.01, *** p < 0.001

mean coefficients; t-score in parentheses

Source: TPI HMIS Data, 2007-2016 (n=1,427)

33

Appendix G: Survey Questions and Select Results

Response rate: 68 out of 274, 25 percent

Table 16. Q1: On average, how many hours a week do you typically spend entering data? (%)

0 Hours 0 and 5 hours 5 and 10 hours More than 10

Administration 13 40 27 20

Case Manager 0 40 53 7

Shelter Staff 5 45 37 13

Total 5 45 37 13

Source: La Follette School Team Survey of TPI Staff, 2017

Table 17. Q2: This week, how many hours did you spend on data entry? (%)

0 Hours 0 and 5 hours 5 and 10 hours More than 10

Administration 27 47 0 28

Case Manager 0 50 36 14

Shelter Staff 3 49 34 14

Total 8 48 27 17

Source: La Follette School Team Survey of TPI Staff, 2017

Table 18. Q3: How would you rate the quality (considering accuracy, completeness, usefulness) of the data you enter? (%)

Poor Neutral Excellent

Administration 0 7 93

Case Manager 7 20 73

Shelter Staff 5 30 66

Total 4 22 74

Source: La Follette School Team Survey of TPI Staff, 2017

34

Table 19. Q4: On average, how many hours a week do you spend using data (including analyzing, cleaning, and using data to make decisions)? (%)

0 Hours 0 and 5 hours 5 and 10 hours More than 10

Administration 13 17 13 27

Case Manager 7 86 7 0

Shelter Staff 19 50 17 14

Total 15 57 14 14

Source: La Follette School Team Survey of TPI Staff, 2017

Table 20. Q5: This week, how many hours did you spend using data (including analyzing, cleaning, and using data to make decisions)? (%)

0 Hours 0 and 5 hours 5 and 10 hours More than 10

Administration 27 40 0 33

Case Manager 7 79 14 0

Shelter Staff 25 44 22 8

Total 22 51 15 12

Source: La Follette School Team Survey of TPI Staff, 2017

Table 21. Q6: How would you rate the quality (considering accuracy, completeness, usefulness) of the data you use? (%)

Poor Neutral Excellent

Administration 0 53 47

Case Manager 7 40 53

Shelter Staff 13 45 42

Total 9 46 46

Source: La Follette School Team Survey of TPI Staff, 2017

35

Table 22. Q7: What is your position at Transition Projects? (n)

Title n

Administration 15

Case Manager 15

Shelter Staff 38

Total 68

Source: La Follette School Team Survey of TPI Staff, 2017

Q8: Use the space below to provide any additional comments about data use or the data culture of TPI. Sample responses TPI needs to be really honest with itself regarding critical data entry problems at shelters. Low

performers must be coached or moved, and the managers do not have the time to coach. Something that will help us develop a data culture is training on data systems.

Table 23. Data culture questions, % agree or strongly agree

Administration Case Managers

Shelter Staff

Q9: I think data should have a large role in how decisions are made.

67 40 58

Q12: I am comfortable using the data system at TPI. 93 73 74

Q13: The most important part of my job requires using data.

73 33 68

Q15: I want TPI to invest resources to ensure good data quality and improve data use.

87 93 76

Q16: It takes too much time to enter data. 33 53 26

Q18: The most important purpose of data collection is to satisfy external reporting requirements.

53 67 45

Q19: It is clear how data are used by TPI to make decisions.

40 40 61

Source: La Follette Team Survey of TPI Staff, 2017

36

Q10: What decisions do you think should be made using data?

Most. What services we fund more, where we conduct outreach, how we modify our service

practices. Everything we do is essentially informed by the information regarding our clientele.

The expansion of shelters planning to be opened. Where funding is allocated. What other

services clients most need referrals to, including medical care, housing, food assistance, etc. and

therefore follow-up education for social workers.

Q11: Please use the space below to expand or clarify any of your answers from this section (Q9

and Q10—using data for decision-making).

Again, data is a good indicator on how well the organization is doing as a whole.

Demographic data can inform TPI about participants and their needs.

Q14: Please use the space below to expand or clarify any of your answers from this section (Q12

and Q13—using data at TPI).

The most important part of my job requires being compassionate and patient when assisting

clients with maintaining safe housing, and data entry is a key part of funding that safe housing.

I find the interaction with participants to be the most important part of my job. But I realize that

the data is essential in our ability to continue doing the work that we do in preparing grant

documentation and showing progress. Data shows us how effective our interactions are.

Q17: Please use the space below to expand or clarify any of your answers from this section (Q15

and Q16—attitudes toward data).

TPI is working on implementing better data training for staff, but we have a long way to go. Staff

are often entering more data than necessary by duplicating their responses which results in

people spending much more time on data entry. This can be resolved with better data quality

training.

Entering data is tedious, regardless of the length of time it takes.

We need to have someone to enter all data so that it isn’t the responsibility of case managers so

they can more effectively utilize their time.

37

Q20: Please use the space below to expand or clarify any of your answers from this section (Q18

and Q19—importance of data).

Although data is very important to collect for external funders, it is also important to collect for

our own information so that the agency can determine what is working well, what isn't and

where participants need the most support.

I think TPI can have too much faith in the data at times. There is plenty of user error and I don’t

believe the reports are that accurate a lot of the time. There is still a lot of variance around

definitions of what we are reporting i.e. does going into transitional housing count as a housing

placement? Data and reporting can be unclear or skewed at times when multiple people are

pulling the same data and coming up with different numbers. Managers and directors pulling

their own numbers can skew the clarity of our data.

Q20: What additional data would be helpful for you to use that is not already collected by TPI?

I think we need more accurate data collection, not necessarily more data.

I think we have too much data and that makes it hard [to] clean data.

Q21: Should TPI be a data-driven organization? What steps need to be taken to reach that goal?

I feel that TPI is already data-driven in many ways. If there were clear-cut guidelines on what

data to put in place when there is not a direct response, our data could be more clear and

concise. This giving us the opportunity to see clearly indicate the next step in the data-driven

process.

I disagree, I think data can only capture a small part of a human-based phenomenon. I think

hard data should be a part of what helps make decisions but we should not rely solely on hard

data. I believe that anecdotes, experiences, opinions etc. are all things that are very relevant and

important to include when working with people in a supportive and service-based organization.

38

Appendix H: Suggestions for Integrating Data into Decision-Making

Integrating a focus on data into the existing processes and organizational culture is a lengthy and

often difficult process. The following are specific, actionable items to help organizations collect,

evaluate, and use data.

PLAN: Identify a leader for the improvement process. Leaders should be able to:

Spend a few hours preparing for meetings, including reviewing data with staff.

Attend and regularly participate in scheduled meetings about data improvement.

Follow up and follow through with decisions made on data improvement.

Be comfortable working with data and employees across TPI, and be invested in improving

data-collection processes.

INTEGRATE: Working with TPI staff and administrative staff, the leader should:

Clarify decision-making goals and the role of data in making decisions by TPI administrators.

Ensure all staff understand the role of data in decision-making.

Examine the findings from the data. Are services meeting the goals of TPI? Are participants

getting enough help?

Use these findings and re-examine these services or characteristics of interest.

EVALUATE: With the leader’s assistance, identify program goals of interest

Start with a program, or even one service or participant characteristic that everyone uses or

finds valuable to facilitate buy-in from all employees.

Discuss the goals for this service or characteristic—what do you most want to know?

As a team, discuss how data can be collected and evaluated to learn what you want to know.

Work together to adjust the data-collection process to ensure data accuracy and quality for

answering your questions.

COLLECT & ANALYZE DATA: Start using the new processes for data-collection, and:

Collect feedback from staff on the new data process—what works? What doesn’t?

Examine patterns in the data to help identify categories for service or participant

characteristics of interest.

Develop a routine for cleaning and assessing data quality.

BUILD: Invite TPI staff to the conversation. Meet with all employees to:

Gain perspectives of data-collection from all sides.

Clarify goals for improving data-collection.

Find agreed-upon meaning and use for data collected. For example, is a “no” response a

missing data point, or is it equal to 0?

Standardize the data-collection process together, and train all staff to use the new process.

Additional strategies can be found in Harry Hatry’s 2011 report “A Guide to Data-Driven

Performance Reviews,” publicly available via the Urban Institute.

39

Works Cited

Argyris, Chris, and Donald Schon. 1996. "What is an Organization that it may Learn?" In

Organizational Learning II: Theory, Method, and Practice, by Chris Argyris and Donald

Schon.

Burt, Martha. 2010. "Life After Transitional Housing for Homeless Families." U.S. Department

of Housing and Urban Development. Urban Institute. Accessed April 2017.

http://www.huduser.gov/portal/publications/LifeAfterTransition.pdf.

Carman, Joanne. 2009. "Nonprofits, Funders, and Evaluation: Accountability in Action." The

American Review of Public Administration 39 (4): 374-390.

City of Portland, Oregon. 2013. A Home for Everyone: A United Community Plan to End

Homelessness to Portland/Multnomah County. Accessed April 2017.

https://www.portlandoregon.gov/phb/article/443314.

Curtis, Marah A., Hope Corman, Kelly Noonan, and Nancy E. Reichman. 2013. Life

Shocks and Homelessness. Demography 50(6), 2227-2253.

Fisher, Craig, and Bruce Kingma. 2001. "Criticality of Data Quality as Exemplified in

Disasters." Information and Management 39: 109-116.

Gubits, Daniel, et al. 2016. Family Options Study: 3-Year Impacts of Housing and Services

Interventions for Homeless Families. Accessed April 2017.

https://www.huduser.gov/portal/sites/default/files/pdf/FamilyOptionsStudySummary

Report.pdf.

Harbarger, Molly. 2016. Multnomah County Opens Women and Couples Homeless Shelters in

Southeast Portland. Accessed April 2017.

http://www.oregonlive.com/portland/index.ssf/2016/11/multnomah_county_opens_

women_a.html.

Hatry, Harry. 2006. Performance Measurement: Getting Results. Urban Institute Press.

Hatry, Harry. 2011. A Guide to Data-Driven Performance Reviews. Urban Institute.

Johnson, TP, SA Freels, and JA Parsons, et al. 1997. "Substance Abuse and Homelessness:

Social Selection or Social Adaptation." Addiction 92: 437-445.

National Coalition for the Homeless. 2006. McKinney-Vento Act. Accessed April 2017.

http://www.nationalhomeless.org/publications/facts/McKinney.pdf.

NPACH. 2017. Homelessness Risk Factors. Accessed April 2017.

http://www.npach.org/homelessness-risk-factors.html.

Partnership for Public Service. 2012. "From Data to Decisions II: Building an Analytics

Culture."

Sermons, M. William, and Peter Witte. 2011. State of Homelessness in America: A Research

Report on Homelessness. National Alliance to End Homelessness and Homeless

Research Institute.

Shelton, Katherine, Pamela J. Taylor, Adrian Bonner, and Marianne van den Bree. 2015. "Risk

Factors for Homelessness: Evidence from a Population-Based Study." Psychiatric

Services 60: 465-472.

Smock, Kristina. 2015. 2015 Point-in-Time: Count of Homelessness in

Portland/Gresham/Multnomah County, Oregon. Kristina Smock Consulting.

40

Stefancic, Ana, and Sam Tsemberis. 2007. "Housing First for Long-Term Shelter Dwellers with

Psychiatric Disabilities in a Suburban County: A Four-Year Study of Housing Access and

Retention." The Journal of Primary Prevention 28 (3): 265-279.

Transition Projects. 2016. About Transition Projects. Accessed April 2017.

http://www.tprojects.org/transitionprojects/.

Transition Projects. 2016. Housing and Homelessness: A Snapshot. Accessed April 2017.

http://www.tprojects.org/LiteratureRetrieve.aspx?ID=162627.

U.S. Department of Housing and Urban Development. 2016. Continuum of Care (CoC) Program

Eligibility Requirements. Accessed April 2017.

https://www.hudexchange.info/programs/coc/coc-program-eligibility-requirements/.

U.S. Department of Housing and Urban Development. 2011. "Part 91- Consolidated Submissions

for Community Planning and Development Programs."

U.S. Department of Housing and Urban Development. 2012. "Portland, Oregon: Innovative

Homeless Service Model at Bud Clark Commons."

U.S. Department of Housing and Urban Development. 2006. Predicting Staying in or Leaving

Permanent Supportive Housing that Serves Homeless People with Serious Mental

Illness." HUD Office of Policy Research.

U.S. Department of Housing and Urban Development. 2014. Rapid Re-Housing Brief. HUD

Exchange.