Arizona Department of Economic Security

Breakthrough Series Collaborative on Service Integration Participant Self-Assessment Survey Results

December 1, 2007

Prepared by Kathleen L. Andereck, Professor and Principal Investigator Richard C. Knopf, Professor and Director

ii

Breakthrough Series Collaborative on Service Integration PARTICIPANT SELF-ASSESSMENT SURVEY RESULTS

Prepared for Arizona Department of Economic Security

By:

Kathleen Andereck, Ph.D. Professor and Graduate Program Coordinator

Department of Recreation and Tourism Management School of Community Resources and Development

Richard C. Knopf, Ph.D. Associate Dean for Community Initiatives, Professor,

and Director, Partnership for Community Development

Arizona State University at the West Campus College of Human Services

Partnership for Community Development P.O. Box 37100

Phoenix, Arizona 85069

A Collaboration Between: ASU Partnership for Community Development

Arizona Department of Economic Security

December 11, 2007

iii

TABLE OF CONTENTS BREAKTHROUGH SERIES COLLABORATIVE ON SERVICE INTEGRATION

ARIZONA DEPARTMENT OF ECONOMIC SECURITY SELF-ASSESSMENT SURVEY RESULTS Executive Summary…………………………………………………….. 1 The Breakthrough Series Collaborative on Service Integration………... 3 The ASU Evaluation Process…………………………………………… 4 The Self-Assessment Questionnaire…………………………….……… 6

Questionnaire Administration………………………………….……….. 7 Survey Results………………………………………………………….. 8 Respondent Profiles…………………………………………….. 8 Program & Service Awareness..………………………………... 9 Importance – Performance Ratings of Principles of the BSCSI

Framework……………………………………………………… 10 Importance – Performance Ratings of BSCSI Components……. 10 Perceptions of Impact on Families……………………………… 18 Team Meetings……………..………………………………….... 21 Open-Ended Comments of Respondents……………………….. 29

Conclusions and Recommendations……………………………………. 29 The Qualitative Evaluation: A Measure of Success……………. 35 Credits………………………………………………………….. 35 References…………………..………………………………….. 35

Appendix A……………………………………………………………………..… 37 Self-Assessment Survey (Questionnaire) Appendix B……………………………………………………………………..… 49 Motivations for Joining the BSCSI Team Appendix C……………………………………………………………………..… 55 Services Respondents have Received through DES Appendix D……………………………………………………………………..… 61 Things that Worked Well for the Team Appendix E……………………………………………………………………….. 65 Things that Could Have Helped Team Achieve Successful Outcomes Appendix F……………………………………………………………………….. 70 Barriers to Achieving Outcomes Appendix G………………………………………………………………………. 75 Team Accomplishments that Could Not Have Happened Without BSCSI Appendix H………………………………………………………………………. 79 Additional Respondent Comments Appendix I……………………………………………………………………….. 84 Data Tables

EXECUTIVE SUMMARY This report summarizes the results of the quantitative evaluation of the State of Arizona Department of Economic Security’s (DES) Breakthrough Series Collaborative on Service Integration (BSCSI). The quantitative evaluation was one of four forms of evaluation conducted by ASU’s College of Human Services under the auspices of its Partnership for Community Development. The BSCSI was instituted by DES in January of 2006 to provide a mechanism for creating a more cohesive and coordinated human service delivery system to support the needs of Arizona families. To carry on the work of the BSCSI, twenty BSCSI Teams of 18 members each were established across the state of Arizona. The Teams were designed to bring together a diverse range of perspectives and opinions, ranging from DES staff to community-based organizations to former clients of human services. The ASU quantitative assessment process involved administration of a questionnaire to Team members near the beginning of the BSCSI process (April 2006), near the mid-point of the BSCSI process (November 2006), and near the end of the BSCSI process (May 2007). This report provides profiles of questionnaire responses across a variety of themes – demographics; BSCSI principles, practices, and desired outcomes; perceptions of impact on families; ratings of Team functioning and experiences; and program and service awareness. In addition to providing response distributions to a variety of measures, the report also provides a record of responses to many open-ended questions incorporated into the survey instrument (e.g., What was your motivation for joining the Team? Are there barriers to achieving successful outcomes? What could have helped your Team achieve successful outcomes?). In general, the research showed strong support and awareness of the BSCSI Framework for Change principles, practices, and desired outcomes. It also consistently showed a gap between the perceived importance of these attributes, and the degree to which these attributes were being manifested in their communities. The perceptions remained relatively consistent over time. Specific findings include:

Team members increased awareness of community services and how to access them as a result the BSCSI process.

There is strong support for all of the guiding principles of the BSCSI Framework for Change.

There are clear gaps between the importance attached to the BSCSI desired practices (Components and Sub-Components) and the perceived Performance of the community in offering these practices.

Team members have great resolve and are generally effective in taking action to increase Performance of these practices.

The Teams have created impact within all three desired outcomes of the BSCSI process, as articulated in the “Framework for Change”.

1

Team members see particular progress in giving families knowledge of resources in the community, increasing connections between families and these resources, and increasing communication among human service providers to ensure that resources are known and shared throughout the community.

Team members are moderately satisfied with the progress of their Teams. Some are strongly encouraged by Team progress; others see that there is much work yet to do.

There were uneven contributions emerging from the three Team Partner groups. DES Partners were perceived to be more engaged than Community Partners. At the same time, there was a clear recognition of the critical importance of all three Partner groups and a desire to find creative ways to equally engage all Partners.

Team members valued the diverse knowledge and experiences of fellow BSCSI participants, and valued the relationships that have emerged.

There was a call for more support for Team activities and better understanding of the role of the BSCSI process relative to the larger context of the DES system. At the same time, Team members were enthusiastic about what they were able to accomplish in spite of such barriers.

Teams had clear understanding of their purpose and goals. They wanted more resources to accomplish their goals, more Team members involved in the process, and more authority to make decisions.

On the whole, the motivation for affecting change was high. The report concludes with a discussion of key attributes of successful Team functioning including the degree to which Team members perceived these attributes were present in their Teams. In general, Teams appeared to have good clarity of mission, strong commitment and buy-in among the members, and high morale. They were less effective in mobilizing and engaging the talents of Team members and having access to ample resources and power as a basis for affecting change. The report concludes with recommendations for enhancing Team performance in pursuit of the desired practices and outcomes of the BSCSI process.

2

THE BREAKTHROUGH SERIES COLLABORATIVE ON SERVICE INTEGRATION In 2003, the Arizona Department of Economic Security (DES) embarked on a journey of innovation that would fundamentally revolutionize the way that the organization structures and delivers its services to better serve children, adults, and families. The agency instituted a new philosophy, called Service Integration, which focuses on integrating existing programs and services, creating effective Partnerships with families and Community Partners, and most importantly, increasing the benefits achieved by families utilizing DES services by creating a more cohesive and coordinated service delivery system. As a manifestation of the Service Integration philosophy, DES began searching for new strategies for incubating innovative ideas for change that would benefit families in local communities. In researching promising practices for innovative change, DES discovered the Breakthrough Series Collaborative (BSC) model being implemented by Casey Family Programs, a national foundation dedicated to serving children, youth, and families in the child welfare system. This model involves bringing together people with varying perspectives to assess existing services and/or programs within their local community, to identify multiple strategies for changes and/or improvements, and to test these strategies on a small scale in a real world setting. In the fall of 2005, DES approached Casey Family Programs with its desire to utilize this model for the purposes of forwarding the mission of Service Integration. The two organizations agreed to work collaboratively to infuse the BSC model into the DES Service Integration initiative in the hopes of creating transformative change in the way DES and other Community Partners serve Arizona families. As the conceptual framework for the DES Breakthrough Series Collaborative on Service Integration (BSCSI) was being developed, DES approached Arizona State University’s Partnership for Community Development (PCD) to manage the evaluative aspects of the process in December of 2005. The PCD agreed to provide both process and outcome-based assessments throughout the year-long process. Assessments were to include: (1) Evaluating BSCSI participant’s understanding and utilization of Service Integration principles and desired outcomes through participant self-assessment surveys, (2) Monitoring the innovative ideas that were produced by the BSCSI process throughout the developmental period, and (3) Assessing the ultimate impact of BSCSI activities on Arizona families through qualitative research methods. In January 2006, DES and Casey Family Programs officially launched the Breakthrough Series Collaborative on Service Integration (BSCSI). To carry on the work of the BSCSI, 20 BSCSI Teams of 18 members each were established across the state of Arizona. The Teams were intentionally designed to bring together a diverse range of perspectives and opinions -- including representation from different DES program areas (six Team members), previous or current families involved in DES programs and/or services (six Team members), and community organizations (nonprofits, faith-based organizations, and other government agencies) representing expertise in mental health, substance abuse treatment, domestic violence, and youth corrections (six Team members).

3

The BSCSI Teams received training on the primary methodology used in the Breakthrough Series model, which is referred to as a PDSA (Plan, Do, Study, Act) cycle. Utilizing a “train the trainer” process, each Team self-selected six members to attend two training sessions, referred to as Learning Summits, to receive instruction on the purpose of the BSCSI, the PDSA methodology, and how to apply the methodology while staying focused on a set of the family-based outcomes. The first Learning Summit was held in May 2006 and the second in October 2006. Three outcomes were identified: (1) Promoting family self-sufficiency, (2) Strengthening families, and (3) Developing the capacity of extended families and communities. In general, the Learning Summits asked participants to work with their Teams to create and test innovative strategies for better integration and coordination of DES and community-based programs and/or services with the ultimate goal of improving family outcomes in local communities. The Summit participants took the knowledge gained at the Learning Summits back to the other members of their Teams. Teams were provided with individual technical assistance, participated in monthly collaborative conference calls, and given access to the BSCSI Extranet to post their progress and monthly reports. Additional support for BSCSI Teams was provided by Senior Leaders, local program or office managers, who provided mentorship and assisted in overcoming barriers to the Team’s success. A pool of 27 Expert Faculty consisting of Casey Family Programs staff, DES staff from varying program areas, community development leaders, other government agencies, community-based organizations, Family Partners, and social work experts were also available for providing technical assistance and problem-solving. Over a 12 month period, the BSCSI Teams worked independently and collaboratively to create, test, and implement small tests of change focused on changes to processes and practices within the DES system in Arizona. Teams were encouraged to share information and “steal shamelessly” (a tenet of the BSCSI process) by taking good ideas and sculpting them to fit similar situations within their unique communities. Throughout the process, Teams received feedback on their application of the PDSA methodology and were encouraged to continue their innovative thinking. A third Learning Summit, held in June of 2007, celebrated successes that were accomplished during the first 12 months of the process. During this time, the BSCSI Teams developed 105 innovative strategies in the areas of process improvements, directories/resource guides, customer service, information gathering, DES office and lobby appearances, Team composition, education and training, and daycare/childcare. The enthusiasm, energy, and production created by the BSCSI Teams has made a profound impact on DES leading the agency to extend the process for an additional six months. THE ASU EVALUATION PROCESS ASU’s Partnership for Community Development (PCD) was contracted as an outside evaluator for the Breakthrough Series Collaborative on Service Integration (BSCSI). In this capacity, PCD staff coordinated an evaluation Team consisting of ASU faculty on the West campus with expertise in program evaluation, survey instrumentation, and social work to develop and implement an evaluation to assess both the BSCSI process and the outcomes achieved for families throughout Arizona.

4

In concert with expectations of DES, the ASU evaluation Team constructed a four-part evaluation process:

Self-Assessment Surveys (questionnaires) Qualitative Research (focus groups and personal interviews) Team Activity Monitoring (through PDSA Report Logs) Learning Summit Evaluations (participant evaluation forms)

Self-Assessment Surveys Mirroring the BSC evaluation methodology of Casey Family Programs, the ASU evaluation Team developed a Self-Assessment Questionnaire for Team members to complete at three intervals: initial, mid-term, and at the final stages of the process. In general, the purpose of this survey was to evaluate progress on understanding and moving forward the Service Integration philosophy, evaluate outcomes, and evaluate Team functioning and experiences throughout the BSCSI process. The format of the instrument, as well as the general methodology of administering the instruments, precisely followed protocols developed for use in BSC applications by Casey Family Programs. Summary results from the initial and mid-term assessments were provided to BSCSI participants at the Learning Summits and the BSCSI Planning Team. This report is the comprehensive assessment of Self-Assessment Survey results, which includes analysis of the questionnaires completed by participants at the end of the first 12 month period of BSCSI activity. Qualitative Research (Interviews/Focus Groups) The ASU Evaluation Team conducted qualitative research through interviews and focus groups with BSCSI Team members and BSCSI process beneficiaries across the state. Ten focus groups were conducted with each of ten BSCSI Teams; three focus groups were conducted with a sample of DES Team members involved in the BSCSI process; and 12 beneficiaries of the BSCSI process (Arizona family members who were impacted by Team activities). These conversations provided a framework for interpreting the effectiveness of the model BSCSI program including the impact of the BSCSI process and PDSA strategies on families within the local communities. The results of this qualitative research have been presented at a BSCSI Learning Summit and are reported in a separate PCD report: Department of Economic Security Breakthrough Series Collaborative on Service Integration Qualitative Evaluation Research Results. Team Activity Monitoring (PDSA Reporting Logs) Again, following Casey Family Programs BSC monitoring methodology, the ASU Evaluation Team created monthly reporting procedures for Teams to report PDSA activity and the level of Team involvement occurring within each Team unit. Teams accessed the BSCSI Extranet to complete two monthly reporting forms. One form for “public” display to all BSCSI participants and one of a more “confidential” nature to enable candid expressions of Team member participation (the latter was helpful in providing BSCSI Team leaders with helpful information on how to further encourage Team production). On a quarterly basis, the ASU Evaluation Team

5

provided assessment of the content and outcomes of PDSAs being produced by the Teams; their effectiveness in producing positive changes for families and their potential for “spread” or successful implementation as promising practices across Arizona. These quarterly reports were issued to the BSCSI Leadership Team and served as the basis for report outs to BSCSI Team participants during the Learning Summits. Learning Summit Evaluations The ASU evaluation Team also conducted assessments of evaluations completed by Team participants at all BSCSI Learning Summits. Learning Summit Evaluation forms were reviewed for common themes, suggestions, concerns, and challenges expressed by BSCSI Team members. One report was issued to DES for each of the three Learning Summits. THE SELF-ASSESSMENT QUESTIONNAIRE The questionnaire used in the Self-Assessment Survey followed Casey Family Programs BSC protocol and is reproduced in its entirety in Appendix A. It was administered to all Team members on three occasions (beginning of BSCSI process, mid-term, and end of BSCSI process). In general, the Self-Assessment Questionnaires were constructed to gather information on participants’ involvement with the BSC process and their perspective on the importance and implementation of several service delivery practices as articulated in the BSCSI Framework for Change (DES, 2005). Specifically, the questionnaire was comprised of seven sections (see Appendix A): Section 1. Program and Service Awareness (Questions 1 and 2)

Section 2. Importance-Performance Ratings: Principles of the BSCSI Framework for Change (Questions 3-8). Section 3. Importance-Performance Ratings of BSCSI Framework for Change

Components and Sub-Components (Questions 9-12) Section 4. Perceptions of Impact of the BSCSI Process on Families (questions 13-15) Section 5. BSCSI Team Composition and Outcome Ratings (Questions 16-20) Section 6. Team Experience Ratings (questions 21-26) Section 7. Demographic Information (questions 27-36)

The questionnaire remained consistent across each of the three periods of administration with only two exceptions. First, the ASU evaluation Team added Section 4 (Perception of Impact on Families) to the mid-term and final administrations. Its purpose was to assess forward movement on the three desired outcomes outlined in the BSCSI Framework for Change: increased self-sufficiency (Question 13), strengthening families (Question 14), and increased capacity to serve families (Question 15). Second, the ASU Evaluation Team added a series of questions about perceptions of Team functioning, experiences and barriers to Team effectiveness (Questions 17 through 26) to the final administration of the questionnaire. Included in this set of questions was a scientifically validated measure (Section V) of Team functioning to assess core strengths and weaknesses of Team functioning from a leadership development perspective (Blandin Foundation, 2006).

6

QUESTIONNAIRE ADMINISTRATION The initial Self-Assessment Questionnaire was administered by ASU on April 26, 2006. The questionnaires surveys were distributed to all Team members by e-mail (N=270) and regular surface mail (N=29) for a total distribution to 299 Team members – the total on record at the time of distribution. The surface mailings were made to Team members for which an e-mail address was not available. Team members were encouraged to complete the instrumentation through a variety of mechanisms: e-mails from BSCSI DES staff members, Team meetings, and telephone calls. In addition, the ASU evaluation Team sent out reminder emails to those participants who supplied an email address. Three sets of email reminders were sent: on May 3, shortly after the May 5 deadline suggested by the initial mailing and again on August 11. Overall, ASU staff received 220 return surveys for a response rate of 74% -- a relatively high response rate for an e-mail based survey of this nature. The mid-term Self-Assessment Questionnaire was administered to 319 Team members on November 15, 2006 (all Team members on record as of that date). Participants were asked to complete the online survey by December 31, 2006. DES staff sent out an e-mail reminder in mid-December, which was followed by another reminder sent by the ASU Evaluation Team on December 20, 2006. Overall, 130 participants returned surveys for a response rate of 41% -- significantly lower than the response rate achieved for the initial distribution. Nonetheless, a minimum of three members responded from each of the twenty Teams. The final Self-Assessment Survey was distributed on May 3, 2007. This was near the end of the initial 12 month BSCSI process period but prior to the third Learning Summit scheduled for early June of 2007. It was distributed to all Team members on record as of that date – a total of 315 individuals. Again, DES staff encouraged Team members to complete the questionnaire via e-mails and telephone conference calls. The ASU Evaluation Team also sent e-mail reminders on May 10 and 24, 2006. A total of 145 Team members responded by the deadline (May 30) for a response rate of 46%.

7

SURVEY RESULTS Respondent Profiles Respondent profiles were constructed from responses to questions included in Sections 5 and 7 of the questionnaire survey (specifically, Questions 16-17 and 27-36 -- see Appendix A). Using respondents to the final questionnaire administration as a base – 11% of the respondents were Family Partners, 61% were DES Partners, and 28% were Community Partners. Of the Community Partners about 23% were from faith-based organizations, 8% from community advocate organizations, 44% from other non-profit organizations, 23% from other government organizations, and 3% from Tribal organizations. The distribution of respondents partitioned by Team membership is shown in Table 1.

Table 1. Team Membership of Respondents

Team Initial (n=217)

Final (n=145) Team Initial

(n=217) Final

(n=145) Apache/St. Johns 3.6% 5.4% Maricopa/Golden Gate 2.7% 5.4% Cochise/Sierra Vista 5.4% 3.4% Maricopa/Avondale 6.3% 4.8%

Coconino/Flagstaff 5.4% 5.4% Mohave/Lake Havasu City 4.1% 3.4%

Gila/Globe 6.3% 6.8% Navajo/Show Low 6.8% 6.1% Graham/Safford 4.1% 4.1% Pima/Ft. Lowell 6.3% 7.5% Greenlee/Clifton 3.6% 2.7% Pinal/Coolidge 5.4% 7.5% La Paz/ Parker 4.1% 5.4% Santa Cruz/Nogales 5.9% 7.5% Maricopa/ASU Downtown Campus 6.3% 2.0% Yavapai/Prescott Valley 4.5% 4.8%

Maricopa/East Valley 2.7% 4.8% Yuma/Yuma 5.9% 4.1% Source: Question 16 of the Breakthrough Series Collaborative on Service Integration (BSCSI) Self-Assessment Questionnaire (Appendix A). Respondents were asked through an open-ended question what their motivations were for joining their BSCSI Team. The entire set of responses is reported in Table B1 of Appendix B. The general demographic profile for the respondents is shown in Table 2. The data includes all individuals who responded to either or both surveys. A majority of respondents were women, most either married or living with a Partner. The average age was 47 years old. The average size of households was three people which included a couple with one child. About a quarter of respondents had a four year college degree with another 46 percent having some level of college education. Incomes were moderate, with over half falling into the $20,000-$60,000 range. Most of the survey respondents were white, though quite a few Latinos were also represented. The group was about evenly split between those who had received DES services in the past and those who had not. The specific services respondents reported that they have received are listed in Appendix C.

8

Table 2. Demographic Profile of all Respondents

Characteristics Percent/Mean (n=212) Characteristics Percent/Mean

(n=212) Gender Income Level Female 78% $20,000 or less 10% Male 22% $20,001-40,000 31% Marital Status $40,001-60,000 26% Yes 68% $60,001-80,000 18% No 32% $80,001-100,000 10% Mean Age 47 yrs. $100,001 or more 5% Mean Household Size 3 people Race/Ethnicity Mean Number of Children 1 child White 60% Education Level Latino 29% Less than high school 1% American Indian 8% High school graduate 8% Asian/Pacific Islander 1% Some college 30% African American/Black 4% Two year college degree 16% Received DES Services Four year college degree 26% Yes 48% Graduate degree 19% No 52% Source: Question 27 and Questions 29 through 36 of the Breakthrough Series Collaborative on Service Integration (BSCSI) Self-Assessment Questionnaire (Appendix A). Program and Service Awareness Team members were asked to what extent they were aware of the diverse programs and services available in their communities to help families (Question 1, Appendix A). They were also asked to describe the extent to which they felt they knew how to access these services to help families (Question 2, Appendix A). Results are presented in Figure 1. On average, Team members felt they were moderately aware of services available. However, knowledge of how to access these services was somewhat lower than awareness. This indicates a continuing goal for service providers is to increase knowledge regarding access. It is important to note that awareness of programs and how to access them increased from the time of the initial survey to the time of the final survey, while the gap between awareness and knowledge decreased. This is a sign of positive progress along this dimension.

9

Figure 1. Awareness of community services and awareness of how to access them

2.9

3

3.1

3.2

3.3

3.4

3.5

3.6

Initial Final

AwareAccess

Source: Question 1 and 2 of the Breakthrough Series Collaborative on Service Integration (BSCSI) Self-Assessment Questionnaire (Appendix A). Note: “Awareness” and “Access” were measured on a four-point measurement scale, ranging from “1” – Not Aware/Knowledgeable to “4” – Very Aware/Knowledgeable. Importance-Performance Ratings of Principles of the BSCSI Framework Six principles of the BSCSI process were identified in the BSCSI Framework for Change (DES, 2005). The six principles were listed in Section 2 of the Self-Assessment Questionnaire (Questions 3-8, Appendix A). Team members were asked to rate their relative agreement with the Importance of each principle and to rate the degree to which they perceived it was being practiced (Performance) in their community (“Performance” – Question 2, Appendix A). Figure 2 is a graph that juxtaposes performance against Importance for judgments about each of the six principles. Importantly, the graph also juxtaposes data points generated by respondents during the initial survey administration against those generated by respondents to the final questionnaire administration. This yields the capacity to map change over time. From an interpretive standpoint, the greatest congruency between Importance and Performance runs along the diagonal running from the lower left corner to the upper right corner. Put another way, the Performance of the Principle in the community is roughly in sync with how important the respondent thinks it should be. As a general rule, all data points should be concentrated in the upper right portion of the diagonal – because this would indicate not only that the Team member values the principle, but also believes it is working well in his or her community. One might refer to the upper right quadrant as the “Keep up the Good Work!” quadrant. Data points concentrated in the lower left portion of the graph could be troubling for those wishing to promote Service Integration concepts – the respondent neither finds the principle important nor does the respondent see it being manifested in his or her community. This quadrant might be

10

Figure 2. Importance-Performance Analysis of BSCSI Principles

Importance-Performance Analysis

A

E

F

BDC

D

C

B

A

E

F

2.75

3.15

3.55

4.05 4.35 4.65

Importance

Perf

orm

ance

Initial Final

*Source: Questions 3 through 8 of the Breakthrough Series Collaborative on Service Integration (BSCSI) Self-Assessment Questionnaire (Appendix A). Notes: “Importance” and “Performance” were measured on a five-point measurement scale, ranging from “1” Strongly Disagree to “5” Strongly Agree. Data points “A” through “F” signify the six Principles assessed in Questions 3 through 8 of the questionnaire (Appendix A).

A=Services are provided to families in a complete, accessible and responsible manner. (Successful service integration is more than just locating services in the same place, blending funding from different sources, and making decisions based on common information.) B=Parents, children, youth, kin, communities and tribes have many strengths, resiliency and natural supports that can help resolve problems that arise. C=DES works together with families to develop and enhance independent, self-sustaining families and communities. D=Programs will emphasize prevention and early intervention services. At-risk children and families are identified early on to prevent later involvement with more complex child welfare and criminal justice agencies. E=Working with Community Partners is necessary to determine and fill gaps in the DES service delivery system. Collaborative efforts improve our ability to meet the needs of children and families. F=All families have something to contribute to the good of the community. Improved outcomes for children and families occur when families, community groups, faith-based groups, non-profit organizations, businesses and government work together with a common goal.

11

referred to as the “Needs Work!” quadrant. Another quadrant that could be referred to as a “Needs Work!” quadrant is the lower right quadrant. Relatively speaking, the judged Importance in this quadrant far outweighs the perceived Performance in the community. Thus, more work needs to be done to bring the Performance in line with the judged Importance. The remaining (fourth) quadrant is the upper left quadrant. Here, the perceived Performance outweighs the judged Importance. While service providers might take delight that their community is “over-performing” in this regard, it still might be important to continue to educate Team members about the importance of this principle as a core element of the BSCSI process. In general, for all principles, the ratings of Importance remained fairly stable over time, while ratings of Performance improved between the time of the initial survey to the time of the final survey. As the graph shows, there is still room for improvement with respect to the goal of achieving higher Performance and higher levels of Importance attached to the Principle. The Sub-Components receiving the highest Performing scores for Component 1 were Sub-Components E and F. Though the averages (means) for Performance fell at or above the scale mid-point for three principles, the fact remains that Team members rated Performance at lower than the scale mid-point (3.0) for three of the principles. Table B1 of Appendix I gives full response distributions for the data from which Figure 1 was constructed (Questions 1-2, Appendix A). Importance-Performance Ratings of BSCSI Components The BSCSI Framework for Change identifies many practices as targets for improvement under the BSCSI process. These practices are outlined in the BSCSI Framework for Change, framed into the context of four broad Components, and a series of Sub-Components for each (DES, 2005). Every Component and Sub-Component of the Framework for Change was incorporated into the Self-Assessment Questionnaire as Section 3 (see questions 9-12 of the Self-Assessment Questionnaire, Appendix A). Respondents were asked to rate each item with respect to the importance of the practice as well as the implementation (Performance) of the practice. Generally speaking, the four Components reflect the themes of “Intake and Assessment practices” (Question 9), “Coordination and Integration” practices (Question 10), “Accessibility and Accountability” practices (Question 11), and “Prevention and Intervention” practices (Question 12). As was true for the BSCSI principles incorporated into Section 2 of the Self-Assessment Questionnaire, an Importance-Performance analysis was conducted for each Question. The results are summarized reported below for both the initial distribution and final distribution of the Self-Assessment questionnaire (Figures 3-6). Complete response distributions for Importance and Performance ratings of each Sub-Component are detailed in Tables B2-B5 of Appendix I. In general, it is interesting to note that for all of the Sub-Components those related to coordination and relationships emerged as the practices perceived as most important. As was true for the rating of principles (Figure 2), the ratings for Performance averaged in many cases below the scale mid-point (3.0 or lower).

12

Figure 3. Importance-Performance Analysis of BSCSI Framework Component 1 (Intake and Assessment Practices)

Importance-Performance Analysis

C

D

EF

B

A

C

D

E

F

BA

3.35

3.63

3.90

4.08 4.32 4.56

Importance

Perf

orm

ance

Initial Final

*Source: Question 9a and 9b of the original version of the Breakthrough Series Collaborative on Service Integration (BSCSI) Self-Assessment Questionnaire (Appendix A). Notes: “Importance” was measured on a five-point measurement scale, ranging from “1” = Not Important to “5” = Extremely Important. “Performance” was measured on a five-point measurement scale, ranging from “1” = “Strongly Disagree” to “5” = Strongly Agree. Data points “A” through “F” signify the ranking results of the six Practices related to initial family intake and assessment evaluated in Questions 9a and 9b of the original version of the Breakthrough Series Collaborative on Service Integration (BSCSI) Self-Assessment Questionnaire (Appendix A).

A=Gather appropriate information during the initial contact with families in a uniform manner B=Gather appropriate information during the assessment process in a uniform manner C= Standardize intake and assessment documents across programs and service providers D=Engage families in a holistic manner during the intake and assessment process E=Encourage agency and service providers to recognize and understand the family's needs within the context of the family's culture F=Promote sharing of data and documents collected across DES programs

13

Figure 4. Importance-Performance Analysis of BSCSI Framework Component 2 (Coordination and Integration Practices)

Importance-Performance Analysis

A

D

C

B

EA

DC

B

E

3.25

3.50

3.75

4.00 4.25 4.50

Importance

Perf

orm

ance

Initial Final

*Source: Question 10a and 10b of the original version of the Breakthrough Series Collaborative on Service Integration (BSCSI)Self-Assessment Questionnaire (Appendix A). Notes: “Importance” was measured on a five-point measurement scale, ranging from “1” = Not Important to “5” = Extremely Important. “Performance” was measured on a five-point measurement scale, ranging from “1” = “Strongly Disagree” to “5” = Strongly Agree. Data points “A” through “E” signify the five Practices associated with coordination and integration of the delivery of services assessed in Questions 10a and 10b of the original version of the Breakthrough Series Collaborative on Service Integration (BSCSI) Self-Assessment Questionnaire (Appendix A).

A=Coordinate services by integrating the process and people involved B=Support a single case review/plan for families who receive multiple services from DES and the community C=Demonstrate that all DES programs connect families to community resources D=Have all DES programs connecting families to community resources E=Support co-locating DES and Community Partners in the same place to improve communication, planning and service delivery

14

Figure 5. Importance-Performance Analysis of BSCSI Framework Component 3 (Accessibility and Accountability Practices)

Importance-Performance Analysis

E

D

C

A

B

F

G

B

A

DC

F

G

E

3.40

3.60

3.80

4.20 4.38 4.55

Importance

Perf

orm

ance

Initial Final

*Source: Question 11a and 11b of the original version of the Breakthrough Series Collaborative on Service Integration (BSCSI)Self-Assessment Questionnaire (Appendix A). Notes: “Importance” was measured on a five-point measurement scale, ranging from “1” = Not Important to “5” = Extremely Important. “Performance” was measured on a five-point measurement scale, ranging from “1” = “Strongly Disagree” to “5” = Strongly Agree. Data points “A” through “G” signify the seven Practices of Accessibility of Services and Accountability of Service Partners assessed in Questions 11a and 11b of the original version of the Breakthrough Series Collaborative on Service Integration (BSCSI) Self-Assessment Questionnaire (Appendix A).

A=Promote systems that include the “family voice” in designing and implementing services B=Promote systems that include the "family voice" in assessing the quality and usefulness of services C=Ensure services are available to families that deal with all of their needs D=Promote services that recognize and include the diversity and strengths of families and their extended community E=Ensure services that support families who are identified as at-risk F=Build on existing relationships of families as well as extended families G=Create effective ways of sharing information about services and resources available through DES and other community organizations

15

Figure 6. Importance-Performance Analysis of BSCSI Framework Component 4 (Prevention and Intervention Practices)

Importance-Performance Analysis

E

D

CB

A

F

E

C

D

B

A

F

3.25

3.53

3.80

3.90 4.18 4.45

Importance

Perf

orm

ance

Initial Final

*Source: Question 12a and 12b of the original version of the Breakthrough Series Collaborative on Service Integration (BSCSI) Self-Assessment Questionnaire (Appendix A). Notes: “Importance” was measured on a five-point measurement scale, ranging from “1” = Not Important to “5” = Extremely Important. “Implementation” was measured on a five-point measurement scale, ranging from “1” = “Strongly Disagree” to “5” = Strongly Agree. Data points “A” through “F” signify the six Practices of prevention and intervention assessed in Questions 12a and 12b of the original version of the Breakthrough Series Collaborative on Service Integration (BSCSI) Self-Assessment Questionnaire (Appendix A).

A=Allow access to and use of information to predict family and community needs B=Allow the “family voice” to be heard when planning prevention and early-intervention services C=Ensure services provide for the needs of families before problems arise as well as immediately after problems arise D=Promote Partnerships with agencies that focus on prevention and early intervention E=Coordinate funding for innovative solutions to prevent crisis F=Allow the community to define the quality of services

16

Importance-Performance Ratings for BSCSI Component 1(Intake and Assessment) All of the intake and assessment practices of BSCSI Framework for Change Component 1 were rated relatively high in Importance (Figure 3 and Table B2 of Appendix I). The practice “promote sharing of data and documents collected across DES programs” was rated as the most Important item of all. The practices “Gather appropriate information during the initial contact with families in a uniform manner” and “Gather appropriate information during the assessment process in a uniform manner” were rated as having the highest Performance in the respondent’s communities. Conversely, the practices “Standardize intake and assessment documents across programs and service providers” and “Engage families in a holistic manner during the intake and assessment process” were rated lowest in both Importance and Performance. As a general rule, there is a clear gap between the level of perceived Importance and perceived Performance for each Sub-Component. Importance ratings were generally higher than Performance ratings. In terms of change over time, some differences were found in assessments during the initial distribution and the final. Yet none of these differences are statistically significant, this points to the relative stability of both the Importance and Performance ratings for Component 1 over time. Importance-Performance Ratings for BSCI Component 2 (Coordination and Integration) All of the Coordination and Integration practices of BSCSI Framework for Change Component 2 were rated relatively high in Importance (Figure 4 and Table B3 of Appendix I). The practice “Coordinate services by integrating the process and people involved” was rated as the most important item in both the initial and final distributions of the Self-Assessment Questionnaire. The practices “Coordinate services by integrating the process and people involved”, “Demonstrate that all DES programs connect families to community resources”, and “Have all DES programs connecting families to community resources” rated highest in perceived Performance on the initial distribution dropped a bit in perceived Performance in the final distribution. But they continued to be rated the highest in Performance in the final distribution. Conversely, the practice “Support co-locating DES and Community Partners in the same place to improve communication, planning and service delivery” was ranked lowest in perceived Performance in each questionnaire administration. The practice “Support co-locating DES and Community Partners in the same place to improve communication, planning, and service delivery” was rated lowest in perceived Importance in each case. As was true for Component 1 analyses, there is a gap between ratings of perceived Importance and perceived Performance. Again, Importance ratings were generally higher than Performance ratings. Neither Importance nor Performance ratings were significantly different between the initial and the final surveys. In terms of change over time, some differences were found in assessments during the initial distribution and the final. Yet none of these differences are statistically significant, this points to the relative stability of both the Importance and Performance ratings for Component 2 over time.

Importance-Performance Ratings for BSCSI Component 3 (Accessibility and Accountability) All of the Accessibility and Accountability practices of BSCSI Framework for Change Component 3 were rated relatively high in Importance (Figure 5 and Table B4 of Appendix I). In general, there was less variability among ratings of these practices than was true for ratings of practices within the other three Components. The practice “Ensure services that support families

17

who are identified as at-risk” was rated as most Important in both the initial and final distributions of the Self-Assessment questionnaire. The practice “Promote systems that include the "family voice" in assessing the quality and usefulness of services” was rated lowest in Importance on the final distribution, dropping a bit in Importance since the initial distribution. The practice “Ensure services that support families who are identified as at-risk” was rated highest in Performance for distributions of the questionnaire. The practice “Create effective ways of sharing information about services and resources available through DES and other community organizations” was rated lowest in Performance in each. As was true for Important-Performance analyses for Components 1 and 2 (above), there is a generally a gap between the level of perceived Importance and perceived Performance for each Sub-Component. And, as was true for the Importance-Performance analyses presented earlier, none of the Sub-Component ratings showed statistically significant differences between the initial and final distributions pointing to the relative stability of both the Importance and Performance ratings for the Component 3 practices over time. Importance-Performance Ratings for BSCSI Component 4 (Prevention and Intervention) All of the Prevention and Intervention practices of BSCSI Framework for Change Component 4 were rated as Important (Figure 6 and Table B6 of Appendix I). The three practices that tended to be rated as most Important in both distributions were “Promote Partnerships with agencies that focus on prevention and early intervention”, “Coordinate funding for innovative solutions to prevent crisis”, and “Ensure services provide for the needs of families before problems arise as well as immediately after problems arise.” The practice rated most highly in perceived Performance was “Promote Partnerships with agencies that focus on prevention and early intervention” in both distributions of the Questionnaire. Lowest in Importance were the practices “Allow access to and use of information to predict family and community needs” and “Allow the community to define the quality of services.” Lowest in Performance on the final distribution were “Coordinate funding for innovative solutions to prevent crisis” and “Allow the community to define the quality of services”, dropping slightly in perceived importance over the initial distribution. Neither Importance nor Performance ratings were significantly different between the initial and the final surveys. Both surveys found a gap between the levels of both Importance and Performance for all of the items suggesting improvements can be a goal for the future. As was true for Important-Performance analyses for all the Important-Performance analyses reported earlier, there is a generally a gap between the level of perceived Importance and perceived Performance for each Sub-Component. And, consistent with analyses for Components 1-3 above, none of the Sub-Component ratings showed statistically significant differences between the initial and final distributions. This again points to the relative stability of both the Importance and Performance ratings for the Component 3 practices over time. Perceptions of Impact on Families Team members were asked to assess the degree to which their Team’s activities have impacted the three desired outcomes of the BSCSI process. As described in the BSCSI Framework for Change and identified in the Learning Summits, these three outcomes are: (1) promoting self-sufficiency, (2) strengthening families, and (3) developing the capacity of extended families and communities to serve families. Based upon a scan of relevant scientific literature, a battery of

18

measures for each outcome was developed by the ASU Evaluation Team and incorporated into the mid-term and final distributions of the Self-Assessment Questionnaire (see Questions 13, 14 and 15, Appendix A). All of the items within each battery were measured on a five-point measurement scale where “1” indicates Strongly Disagree and “5” indicates Strongly Agree. Team members were asked to rate the degree to which they felt each item reflected a particular impact that their Team has had on families by engaging in the BSCSI process. Mean responses to the items for the final distribution are graphically presented in Figures 7, 8 and 9. Complete response distributions for each item in the final distribution are detailed in Tables B6 of Appendix I. Desired Outcome 1: Promoting Self-Sufficiency Respondents were given a list of 12 possible characteristics of family self-sufficiency and asked to rate the degree to which they felt their Team’s activities had impacted families in terms of that particular characteristic (see Question 13, Appendix A). In general, respondents tended to agree their Team had helped families achieve self-sufficiency along each of these 12 characteristics (Figure 7 and Table B6 of Appendix I). They felt they were particularly helpful in assisting families to “Have knowledge of resources in the community”. They felt they had lowest impact in helping families “have sufficient resources.” Otherwise, the remainder of the characteristics were rated roughly (and positively) the same. Figure 7. Perceptions of Team Impact on Family Self-Sufficiency (means)

1 1.5 2 2.5 3 3.5 4 4.5 5

Have knowledge of resources in the community

Know and recognize their needs

Know their assets and abilities

Envision their potential to be self-sufficient

Be innovative in overcoming barriers

Be proficient in accessing resources

Prioritize their needs

Prioritize their assets and abilities

Move toward their greatest potential

Be inspired to move beyond reliance on social services

Make things happen for themselves

Have sufficient resources

*Source: Question 13 of the Breakthrough Series Collaborative on Service Integration (BSCSI)Self-Assessment Questionnaire - final distribution (Appendix A).

19

Desired Outcome 2: Strengthening families Respondents were given a list of 11 possible characteristics of strengthened families and were asked to rate the degree to which they felt their Team’s activities had impacted families in terms of that particular characteristic (see Question 14 and Table of Appendix A). In general, respondents tended to agree their Team had helped strengthen families along each of these 11 characteristics (Figure 8 and Table B6 of Appendix I). The Team felt they had greatest impact in helping families “Increase connections between families and outside support networks” but felt less effective with helping “Increase spiritual well-being within the family” and encouraging families to “Spend more time together.” Ratings for other characteristics were relatively similar (and positive) among these impacts. Figure 8. Perceptions of Team Impact on Strengthening Families (means)

1 1.5 2 2.5 3 3.5 4 4.5 5

Increase connections between families and outside support networks

Create a safe environment for all family members

Become more socially connected

Become more engaged in their communities

Develop positive communication among family members

Adapt to hardship

Endure through hardship

Increase commitment between family members

Increase appreciation, affection and nurturing among family members

Spend more time together

Increase spiritual (not necessarily religious) well-being within the family

*Source: Question 14 of the Breakthrough Series Collaborative on Service Integration (BSCSI) Self-Assessment Questionnaire - final distribution (Appendix A). Desired Outcome 3: Developing capacity of extended families and communities to serve families Respondents were given a list of 18 possible characteristics of developing capacity of extended families and communities to serve families and were asked to rate the degree to which they felt their Team’s activities had impacted systems in terms of that particular characteristic (see Question 15, Appendix A). Once again, respondents tended to agree their Team had helped families achieve self-sufficiency along each of these 18 characteristics (Figure 9 and Table B6 of Appendix I). The Team felt they were particularly helpful in “Increasing communication to ensure that resources are known and shared throughout the community”, “Increasing services

20

Figure 9. Perceptions of Team Impact on Developing Capacity of Extended Families and Communities to Support Families (means)

1 1.5 2 2.5 3 3.5 4 4.5 5

Increasing communication to ensure that resources are known and shared throughoutthe community

Increasing services that are accessible

Increasing connections to resources

Increasing opportunities for families to receive services and support in their owncommunities

Increasing efficiency of services

Increasing the ability to engage in partnerships between families, communitypartners, and governments in working toward common goals

Increasing services that are coordinated (not fragmented)

Increasing timeliness of services

Increasing the ability to be innovative in solutions

Increasing the consideration of local strengths and needs in designing and deliveringservices

Increasing knowledge and building the assets of families and the communities inwhich they live

Increasing the ability to engage families in systems and solutions

Increasing “wrap around” services

Increase knowledge and awareness of prevention and early intervention services

Increasing culturally sound services

Safely increasing the number of families supported by extended family members

Safely decreasing the number of children in institutional placements

Safely decreasing the number of adult family members in institutional placements

*Source: Question 15 of the Breakthrough Series Collaborative on Service Integration (BSCSI)Self-Assessment Questionnaire - final distribution (Appendix A). that are accessible”, and “Increasing connections to resources.” The Team felt they had lowest impact in “Safely decreasing the number of adult family members in institutional placements” and “Safely decreasing the number of children in institutional placements.” The remainder of the characteristics were rated roughly (and positively) the same. Perceptions of Team Characteristics, Functioning, and Barriers A set of questions were incorporated into the final survey related to the functioning of Teams (see Questions 17-24, Appendix A). Team Meetings On average the Teams met twice a month with several of the Teams meeting just once a month. Nearly all met as an entire group in person (as opposed to meeting in sub-groups or by telephone conference).

21

Satisfaction with Team When asked how satisfied they were with their Team’s performance on a five-point scale (ranging from “1” Not Satisfied to “5” Extremely Satisfied), the responses were distributed in a bell-shaped curve along the rating scale (Figure 10). Most responses were on the mid-point of the scale (Moderately Satisfied). About ten percent reported they were “Not Satisfied” and approximately five percent reported they were “Extremely Satisfied”. In sum, at least 70 percent felt at least moderately satisfied with the Team experience. Figure 10. Satisfaction with Team Progress

0

10

20

30

40

Not Slightly Moder. Very Extrem.

*Source: Question 26 of the Breakthrough Series Collaborative on Service Integration (BSCSI)Self-Assessment Questionnaire - final distribution (Appendix A). Partner Engagement Respondents were asked to rate the degree to which different categories of Team members (DES Partners, Community Partners, and Family Partners) were engaged in the Team process. Four response categories were offered: “None”, “Low”, “Medium” and “High”. On the whole, respondents reported that the DES Partners were the most engaged in the process (Figure 11). Nearly half of the respondents reported high levels of engagement by DES staff. Family Partners were considered the least engaged. Nearly half of the respondents reported low levels of engagement by Family Partners. Increased involvement by the Family Partners must remain a continuing goal of the BSCSI process.

22

Figure 11. Partner Engagement

0

10

20

30

40

50

60

None Low Avg. High

DES

Community

Family

Overall

*Source: Question 23 of the Breakthrough Series Collaborative on Service Integration (BSCSI)Self-Assessment Questionnaire - final distribution (Appendix A). Team Functioning Just prior to the final distribution of the BSCSI Self-Assessment Questionnaire, the ASU Evaluation Team had completed several focus groups with Team members as a component of the Qualitative Research program of this evaluation. Based on some early results of these focus groups, the Evaluation Team created a battery of nine characteristics that appeared to be operational in determining how successfully the Team was functioning (Question 18, Appendix A). Respondents were asked to rate on a five-point scale ranging from “1” (Disagree) to “5” (Agree) the degree to which each characteristic described their Team. Figure 12 summarizes the results, with detailed response distributions reported in Table B7 of Appendix I. The highest rated characteristics were “Diverse knowledge and experience represented on the Team” and “Good relationships among Team members.” The lowest rated item was “Strong participation by Family Partners.” This finding is congruent with other findings of the BSCSI evaluation process, and Family Partner participation continues to exist as a desirable area of improvement. Question 18 also asked the Respondents to respond in an open-ended manner to the question “Are there other things that worked especially well for your Team?” The responses, which were plentiful, are reported in Appendix D.

23

Figure 12. Perception of Team Characteristics (means)

1 1.5 2 2.5 3 3.5 4 4.5 5

Diverse knowledge and experiencerepresented on the team

Good relationships among teammembers

Having fun during meetings

Commitment to the philosophy ofthe program

Maintaining flexibility during theprocess

Demonstrating innovation

Strong leadership for the team

Strong participation by communitypartners

Strong participation by familypartners

*Source: Question18 of the Breakthrough Series Collaborative on Service Integration (BSCSI)Self-Assessment Questionnaire - final distribution (Appendix A). Actions That Could Have Benefited the Teams The Self-Assessment Questionnaire included a list of five possible actions that hold potential for helping the Team achieve successful outcomes (Question 19, Appendix A). Respondents rated each action on a four-point scale, ranging from “1” (Not Helpful) to “4” (Very Helpful). These items were created based on what was learned from the ASU Evaluation Team from several focus groups conducted as part of the Qualitative Research program of this evaluation. A summary of the results is provided in Figure 13. A full distribution of responses is reported in Table B8 of Appendix I. The highest rated action was “Increased knowledge about this project relative to the larger DES system.” The lowest rated action was “More acknowledgement and encouragement for the Teams.” At the same time, there was not much variability among the items – all potential actions were rated very high. As part of Question 19, respondents were asked to respond in an open-ended format to the following question: “Are there other things you think could have helped your Team achieve successful outcomes?”. The responses again were plentiful and are reported in Appendix E.

24

Figure 13. Things That Could Have Helped the Teams (means)

1.00 1.50 2.00 2.50 3.00 3.50 4.00 4.50 5.00

Support from DES for engaging morefamily partners

Provide incentives for family partnerparticipation

More acknowledgment andencouragement for the teams

Increased knowledge about thisproject relative to the larger DES

system

Increased clarity of the project andprocess

*Source: Question 19 of the Breakthrough Series Collaborative on Service Integration (BSCSI) Self-Assessment Questionnaire - final distribution (Appendix A). Barriers to Achieving Outcomes The Self-Assessment Questionnaire included a list of 12 possible barriers that could have interfered with the Team’s achievement of successful outcomes (Question 20, Appendix A). Respondents rated the degree to which they perceived each barrier existed on a five-point scale, ranging from “1” (Strongly Disagree) to “4” (Very Helpful). These items were also created based on what was learned from the ASU Evaluation Team from several focus groups conducted as part of the Qualitative Research program of this evaluation. A summary of the results is provided in Figure 14. A full distribution of responses is reported in Table B9 of Appendix I. The highest rated barriers were “Difficulty engaging Family Partners”, “People moving in and out of the Team”, and “Lack of knowledge about the project by the larger DES community.” The lowest rated barriers were “Confusion about who was in charge of the Team”, “Not feeling the Team was valued”, and “More acknowledgement and encouragement for the Teams”. As part of Question 20, respondents were asked to respond in an open-ended format to the following question: “Are there other barriers to achieving outcomes that your Team encountered?”. The responses again were plentiful and are reported in Appendix F.

25

Figure 14. Barriers to Achieving Team Outcomes

1 1.5 2 2.5 3 3.5 4 4.5 5

Difficulty engaging family partners

People moving in and out of the team

Lack of knowledge about the project by the larger DES community

Lack of resources needed to implement actions

Took a while to get started

Lack of authority to implement actions

Establishing team cohesiveness took some time

Extranet/computer problems

Overly complex process

Confusion about the purpose of the project

Not feeling that the team was valued

Confusion about who was in charge of the team

*Source: Question 20 of the Breakthrough Series Collaborative on Service Integration (BSCSI) Self-Assessment Questionnaire - final distribution (Appendix A). Team Functioning The final distribution of the Self-Assessment Questionnaire included a scientifically validated set of measures of Team functioning developed by the Blandin Foundation (2006). Blandin’s 30-item set of measures was incorporated as Question 24 of the Questionnaire (Appendix A). The 30 items were designed to describe attributes that have been proven to be characteristic of “successful Teams” by researchers. Respondents were asked to rate the degree to which they experienced each attribute while being a member of their Team, using a five-point scale, ranging from “1” (Strongly Disagree) to “4” (Very Helpful). The highest ranked items were “I believe our Team has a high level of relevant knowledge, experience, and strong social connections”, “I believe my fellow Team members to be trustworthy”, “Team members are good at safeguarding confidential information”, and “I have a clear understanding of the Team’s goals.” The lowest rated items were “Our Team has the right number of people,” “Our Team has the resources necessary to achieve the Team's goals”, and “Our Team has the authority it needs to make important Team decisions.”

26

Table 3. Team Experiences

Experiences with your Team: Strongly disagree

%

Disagree %

Neither % Agree % Strongly

agree % Mean

I believe our Team has a high level of relevant knowledge, experience, and strong social connections

1.4 3.4 6.8 51.7 34.7 4.17

I believe my fellow Team members to be trustworthy 2.7 0.7 8.2 56.5 30.6 4.13

Team members are good at safeguarding confidential information 2 0.7 10.2 59.2 26.5 4.09

I have a clear understanding of our Team's purpose 1.4 6.1 6.1 59.9 25.2 4.03

Our Team works well together 3.4 2.7 12.2 51 29.3 4.01

The people on our Team trust each other 2 2.7 13.6 53.1 25.9 4.01

I understand our Team's goals 2 6.8 8.2 55.8 26.5 3.99

Team members back up the decisions made by the Team 2.7 3.4 13.6 55.1 23.1 3.94

Team members communicate with each other openly and directly 2 8.2 8.2 59.2 21.1 3.90

Our Team resolves conflict among Team members in a timely manner 2 3.4 23.1 49 20.4 3.84

Our Team takes the time to learn from both successes and failures 2 8.2 13.6 55.1 19 3.83

I know the indicators of success for our Team 1.4 10.9 13.6 51.7 21.8 3.82

People are proud to be part of this Team 2 9.5 17 46.9 22.4 3.80

Our Team produces high-quality work 2 6.8 21.8 46.3 21.1 3.79

Our Team adapts quickly to changing demands 3.4 8.8 15 52.4 18.4 3.75

Our Team gets results 4.8 6.8 17 51 16.3 3.7

*Source: Question 24 of the BSCSI Breakthrough Series Collaborative on Service Integration (BSCSI)Self-Assessment Questionnaire - final distribution (Appendix A).

27

Table 3. (Continued)

Experiences with your Team: Strongly disagree

%

Disagree %

Neither % Agree % Strongly

agree % Mean

Team members are passionate about our Team's purpose 3.4 9.5 20.4 44.9 19.7 3.69

Team members are actively involved with Team goal setting and decision making 3.4 12.9 13.6 47.6 19.7 3.69

Our Team plans and manages meetings to make efficient use of time 4.8 13.6 10.2 55.1 15 3.63

Our Team builds and executes plans to accomplish Team goals 3.4 9.5 19.7 53.1 12.9 3.63

Our Team has the right mix of skills, experience, and connections 4.8 15 10.9 52.4 16.3 3.61

Our Team uses agreed-upon processes for making decisions 4.1 11.6 13.6 58.5 10.2 3.60

Our Team makes sound and timely decisions 3.4 10.2 19 55.8 8.8 3.58

Our Team stays on track and meets deadlines 5.4 11.6 17.7 48.3 14.3 3.56

Our Team spends enough time working on Team goals versus dealing with more reactive issues

5.4 16.3 16.3 51 9.5 3.43

Our Team has had enough time to achieve results 7.5 13.6 20.4 47.6 8.8 3.38

Team members have clear roles and responsibilities 4.8 19 22.4 43.5 9.5 3.34

Our Team has the authority it needs to make important Team decisions 10.9 20.4 16.3 40.1 10.2 3.19

Our Team has the resources necessary to achieve the Team's goals 7.5 29.3 19.7 34 7.5 3.05

Our Team has the right number of people 5.4 36.7 20.4 28.6 8.2 2.97

*Source: Question 24 of the Breakthrough Series Collaborative on Service Integration (BSCSI) Self-Assessment Questionnaire- final distribution (Appendix A). Additional Accomplishments At the end of the series of questions on Team Functioning and Performance, respondents were asked whether they felt goals were accomplished via their participation in the BSCSI Team that could not have otherwise been accomplished (Question 25, Appendix A). Responses were in a Yes/No format. Sixty-one percent of the respondents stated “Yes”. Then they were asked the following open-ended question, “If yes, what is one example of such an accomplishment?”. A listing of their responses is provided in Appendix G.

28

Open Ended Comments of Respondents At the close of the Self-Assessment Questionnaire respondents were thanked for taking the time to share their perspectives. They were then offered additional opportunity to offer reflections with the open-ended probe “If there is anything more you would like to share with us, please write it in the space below”. Nearly 50 respondents offered comments. These comments are reproduced in Appendix H. CONCLUSIONS AND RECOMMENDATIONS The Self-Assessment Survey process created an illuminating glimpse of BSCSI Team member’s attitudes, perceptions, and reflections about Team functioning and needs. From a tracking perspective, substantial differences between perception of the importance and performance of core elements of the BSCSI Framework for Change did not emerge when comparing initial distribution responses to final distribution responses. This is surprising in light of accounts of clearly successful impact of the BSCSI process revealed in other three components of the ASU evaluation process – specifically through the Qualitative Research Component (focus groups and personal interviews), the Team Activity Monitoring Component (analysis of PDSA Report Logs), and the Learning Summit Evaluations (assessment of participant evaluation forms). Yet, in other ways, this finding may not be surprising. The methodology was complex – both in format and in the language drawn from the BSCSI Framework for Change which populated much of the Questionnaire. It could well be there was some form of bias response operating because of this reality. It is possible Team members who were more attuned to the concept and promise of the BSCSI process from the beginning were more likely to complete the Questionnaire at each stage of distribution – initial, mid-term and final. Thus, broader changes in attitude and perception among the wider range of Team members may not have been adequately captured. And, unlike the other components of the ASU evaluation, the methodology here focused more on measuring abstract concepts rather than concrete forms of impact. Other than the set of questions designed to measure perceptions of impact of the BSCSI process on families (Section 4, questions 13-15, Appendix A), the methodology was not designed to track specific outcomes of the BSCSI process and respondent perceptions of the specific outcomes of their Team activity. For this set of measures, responses are only available from the final distribution of the questionnaire. Thus, no pre-test/post-test information is available. What the results of the Self-Assessment Survey clearly convey is general support of Team members for the BSCSI process and enthusiasm about the promise of its impact. Among the highlights:

As a result of the BSCSI process, Team members increased awareness of community services and how to access them (Figure 1).

There is strong support for all of the guiding principles of the BSCSI Framework for Change (Figure 2).

29

There are clear gaps between the importance attached to the Desired Practices (Components and Sub-Components) and the perceived performance of the community in offering these practices.

There are clear gaps between the importance attached to the desired practices of the BSCSI (Components and Sub-Components) and the perceived performance of the community in offering these practices (Figures 3-6). Based upon motivations for joining the BSCSI Team (Appendix B), expressions of what worked well for the Teams, and other forms of enthusiasm expressed for the positive impacts of the BSCSI process (Appendices G and H) it is clear Team members have great resolve and are generally effective in taking action to increase performance of these practices.

There is perception the Teams have created impact within all three desired outcomes of the BSCSI process as articulated in the “Framework for Change” (Figures 7-9). Team members see particular progress in giving families knowledge of resources in the community (Figure 7), increasing connections between families and these resources (Figure 8), and increasing communication among human service providers to ensure that resources are known and shared throughout the community (Figure 9).

Team members are moderately satisfied with the progress of their Teams. Some are strongly encouraged by Team progress; others see that there is much work yet to do (Figure 10).

There were uneven contributions emerging from the three Team Partner groups (Figures 11 and 12). DES Partners were perceived to be more engaged than Community Partners. At the same time, there was a clear recognition of the critical importance of all three Partner groups and a desire to find creative ways to equally engage all Partners (Figure 14, Appendix E, and Appendix F).

Team members valued the diverse knowledge and experiences of BSCSI participants and valued the relationships that have emerged (Figure 12, Table 3).

There was a call for more support for Team activities and better understanding of the role of the BSCSI process relative to the larger context of the DES system (Figure 13 and 14). At the same time, they were enthusiastic about what they were able to accomplish in spite of such barriers (Appendix D).

Teams had clear understanding of their purpose and goals. They wanted more resources to accomplish their goals, more Team members involved in the process, and more authority to make decisions (Table 3). On the whole, the motivation for affecting change was high.

The Self-Assessment Survey has provided great insight into the yearnings, ambitions and potentials of the Teams. The necessity for successful Teamwork is at the heart of the success of the BSCSI vision and process. We would like to close with an analysis of the Team functioning data made available by the inclusion of the “successful” Team functioning measures developed by the Blandin Foundation (Krile 2006) and included as Question 24 in the Self-Assessment Questionnaire (see Table 3). The Blandin model, based upon extensive research, posits there are seven indicators of successful Team functioning – and that all indicators must be maximized for optimal Team performance and production. The indicators are:

30

Clarity of Mission Effective Mobilization of Talent Productive Norms Commitment and Buy-In Ample Resources and Power High Morale Accomplishment and Results

Based upon responses to the measures in Question 24, scale scores for each measure were calculated and are reported in Table 4. The scores are computed on a five-point scale, with a “1” representing “Strongly Disagree” and a “5” representing “Strongly Agree”. Through the self-assessment, respondents reported their Teams were most successful in having Clarity of Mission, Commitment and Buy-In from Team members, and High Morale. They rated their Teams lowest in having Ample Resources and Power and Effective Mobilization of Talent. Table 4. Relative Strength of Team Success Factors

Indicators of Team Success Mean

Clarity of Mission 3.94 Commitment and Buy-In 3.92 High Morale 3.92 Productive Norms 3.72 Accomplishments and Results 3.70 Effective Mobilization of Talent 3.31 Ample Resources and Power 3.31

*Source: Scaling of items (Krile, 2006) from Question 24 of the Breakthrough Series Collaborative on Service Integration (BSCSI)Self-Assessment Questionnaire - final distribution (Appendix A). Utilizing these findings as a context, we close with recommendations on how to enhance the capacity of BSCSI Teams to produce the desired outcomes of Service Integration for the purpose of strengthening families, building self-sufficiency of families, and building the capacity of extended families and communities to support families. Clarity of Mission. To be successful in producing desired outcomes a Team must have a clear purpose, set strategic objectives and benchmarks, and developed methods to track progress. In other words, the Team knows what it wants and has agreed on how it will know when it has accomplished its mission. The score for the clarity of mission scale (comprised of items 1-3 of Question 24, Appendix A) indicates that the Teams, in general, perceived they had a fairly clear sense of direction and reasonably clear indicators for tracking progress. The generally high levels of agreement with the BSC principles (Figure 2 and Table Bl of Appendix I) also support the notion that Team members have a clear sense of purpose and recognition of goals. Though they do not yet strongly feel that these principles are being practiced in their communities through their actions, they do know what they are trying to achieve. This is a positive step in the right direction and is a tribute to BSCSI managers and the

31

effectiveness of the Learning Summits. There is a feeling among Team members that they are making progress with respect to the principles being played out in practice. In a similar way, the respondents felt the BSCSI practices (Components and Sub-Components) are important, yet they do believe performance of these practices can be improved (Figures 3-6). And while they are heartened by their capacity to achieve the three outcomes of the BSCSI Framework for Change, there continues to be room for improvement in achieving these outcomes. Most of these improvements can be gained through strengthening other aspects of Team functioning parameters as provided by the Blandin model. We now discuss ways in which this can be accomplished. Effective Mobilization of Team Talent The Blandin model suggests it is imperative that Team members have clear roles and responsibilities and the Team has the right number of people with appropriate skills, experiences, and connections the Team needs to succeed. This factor was rated lowest among the indicators of success (Table 4). The score for the items that measure talent (items 4-6 of Question 24, Appendix A) barely moves into the positive end of the continuum, in fact, it is just above “Neutral”. This suggests the make-up of the Teams was not working at capacity along this dimension. Other results from the evaluation indicate several problems which undermined the performance of this indicator category: • Many respondents expressed concern that some of the Teams were too small resulting in

overworked existing Team members (Table B8 and B9, Appendices E and F). This points to the need to have increased Team size for the smaller Teams, so the workload for those existing members could have been decreased.

• There was also concern there are not enough Family Partners involved in the Teams (Table B9, Appendix F). The planning Team needs to continue to develop strategies to better engage Family Partners in the BSCSI process.

• Some respondents indicated a lack in clarity of in the roles each Team Member was to play in the Team (Appendix E and F). In particular, the data indicate Family Partners did not always know their roles or responsibilities. It is plausible to suggest that an underlying factor in the inability to retain some Family Partners could have been the perceived ambiguity of roles. Teams should have been encouraged to have well defined and articulated roles for all members, but particularly Family Partners.

• Based upon analysis of Team member lists utilized in the three distributions of the Self-Assessment Questionnaire, it is clear many Teams had a substantial amount of turnover during the timeline of the project. The number of people moving in and out of the Teams made it difficult to maintain flow and continue progression toward goals. This also points to the necessity to be persistent in developing strategies to involve, engage, reward, affirm, and maintain Team Members.

• At the same time, it was clear that Team Members had experience and diverse people as members. They valued, on the whole, the emerging relationships among them (Table B7, Appendix F).

32

Productive Norms. Norms are the “rules” for how members will work together to gain the commitment of Team members to progress toward the Team’s mission. To work effectively, Team rules should cover decision-making, communicating, and running meetings. The Team has agreed-upon rules for decision-making, communication, meetings, working hand-offs, and debriefings. Table 4 indicates the scale score for this dimension in moderately high (measured by Items 7-13 in Question 24, Appendix A). This suggests the Teams could have been helped by active facilitation of the management of norms. Overall, the evaluation suggests some areas for improvement: • Some open-ended comments reflected some Teams did not have clearly identified Team

Leaders (Appendices E and F). Though this appeared not to be a wide ranging concern, it does point once again to the need for clearly agreed upon processes for designating who should provide the leadership and clarifying the normative process for deciding this.

• There were also open-ended comments about the difficulty of finding meeting times when all Team members could meet (Appendices E and F). Teams need to be mentored in alternatives that might be available to “every member present” scenarios (e.g., sub-committees, virtual meetings, e-mail utilization).

• Teams can also be mentored to systematically evaluate their own normative formation processes and determine the root causes of any problems or mistakes with Team functioning. Learning Summits might have had more opportunities for exploring effective ways of managing norm formation processes to maximize success. More emphasis could have been placed on the processes for producing improved meeting management techniques, exploring processes for creating shared vision, and examining mechanisms for enhancing Team communication.

Commitment and Buy-in. The Blandin model suggests successful Teams build commitment through a compelling purpose and through Team members who are highly motivated in goal setting and decision making. Team members must find ways to build commitment and buy-in into the Team’s mission. Along with Mission, items comprising this scale were ranked high (items 14-17 of Question 24, Appendix A). Yet, there is always room for enhancement. Managers of the BSCSI could have enhanced Team performance on this dimension by: • Encouraging Teams to periodically review the Team’s purpose, goals and desired outcomes.

This brings sharpened focus of the purpose of the BSCSI process and the ultimate desired outcomes of it. This periodic review also facilitates a sense of shared destiny. Team members have the capacity to modify particular goals and actions being created in the context of the broader goals of the BSCSI process.

• Encouraging Teams to be certain that all members have the opportunity to be involved and contribute, especially Family Partners.

• Ensuring all Team members are comfortable with their individual roles and responsibilities. • Ensuring all Team Members understand the potential link between Team accomplishments

and community impact (development of a logic model to inform, channel, and guide Team activities).

Ample Resources and Power. Success in this indicator means the Team has enough money, equipment, time, and authority to accomplish its mission. Table 4 reveals this indicator was

33

ranked low buy the respondents – just barely into the agreement portion of the five-point scale (measured by items 18-21 of Question 24, Appendix A). The results of this evaluation make it clear many respondents felt they did not have the level of authority it would take to make appropriate decisions and implement innovative ideas (Table B6, B9, Appendices E and F). One of the most prevailing concerns was that mid-level (DES) managers were inadequately informed about the imperativeness of the BSCSI process and/or were not supportive of Team efforts as much as expected. BSCSI managers could have assisted by: • Devising ways to help the Teams assess the resources it needs to accomplish its goals and

assist the Teams in accessing these resources. • Continued persistence with mid-level management education and encouragement to support

the BSCSI process in general and Team members in particular. From the open-ended comments (Appendices E and F), it is clear that two many individuals in supervisory roles were not supportive of Team activity – or even presented obstacles to Team activity. Teams must be given enough autonomy and authority to act, and other personnel with supervisory responsibilities for Team Members need to be persistently informed about the importance of supporting Team member efforts. The process would have been enhanced by development of a formal communication plan to provide these mid-level managers a constant feed of insight about the demonstrated benefits of the BSCSI process.

High Morale. The Blandin model notes successful Teams have identified potential areas of conflict and have developed effective ways to deal with it. Scores for this indicator variable suggest the respondents rate Team functioning high in this dimension (items 22-26 in Question 24, Appendix A). The evaluation has shown individuals generally have fun as a member of the Team – and their Team works well together with minimal conflict (Table B7 and Appendix E.) BSCSI managers should continue to provide avenues for validating the successes of Teams, rewarding and affirming Team members for the many successes they have produced, and championing the communication of these success stories to the media and a broad array of constituencies. Accomplishments and Results. Finally, successful Teams have clear timelines for production and action plan achievement, document their success stories well, and are fully cognizant of the urgent and important ways they contribute to the resolution of a social need. Having a good sense of accomplishment and results means seeing that Team activities have indeed created what the Teams are striving toward. Responses to this measure indicate respondents only assigned moderate levels of performance to this attribute (items 27-30 of Question 24, Appendix A). The best method to improve the achievement of results is to focus efforts on the areas mentioned above. Ultimately, the clarity and production of results is contingent upon the Team’s capacity to have continued focus on the desired outcomes of the BSCSI process (both SI practices and family-oriented desired outcomes). BSCSI managers become most effective as they continue to lead Teams to frame all of their activities in the context of the driving forces underlying the BSCSI process, and in awareness of how outcomes of the process can lead to the positive transformation of communities. Only with an eye on desired outcomes, can Team members be effective in their action.

34

THE QUANTITATIVE EVALUATION: A MEASURE OF SUCCESS This quantitative evaluation is but one measure of the success of the BSCSI process. Through the Team Activity Monitoring component of the ASU BSCSI Evaluation Process, we learned that 105 innovative strategies were incubated by the 20 Teams. Breakthroughs were created in case management processes, mechanisms for communicating information to families, procedures for responding to customer needs, information gathering processes, environmental aesthetics and client support services, education and training programs, and childcare delivery services. Through the Learning Summit Evaluations, we learned that BSCSI participants appreciated the mentoring roles of BSCSI managers and demonstrated continual improvement in their capacity to incubate such innovative ideas in ways which would maximally impact society. Through the Qualitative Evaluation, we learned the Teams were successfully implementing the BSCSI process and making significant improvements to the community. We learned the Teams overcame significant obstacles to the innovation process to create change. Their activities were infused with a spirit of perseverance, collaboration and communication, effective leadership, creativity and flexibility, assistance from higher administration, a sense of hope, relationship building, and the ability to have fun. The impacts were broad in scope and far reaching in significance. Clients and service providers alike were transformed by increased insight, increased access, and streamlined procedures for meeting public needs. And now, in the Quantitative Evaluation, we learned that there was strong support by the Teams of the guiding principles, desired practices, and desired outcomes of the BSCSI Framework for Change. We also learned that the Teams were producing impact in each of the three desired outcomes of the BSCSI process: strengthening families, building self-sufficiency within families, and building capacity of extended families and communities. We learned that Team members saw particular progress in giving families knowledge of resources in the community, increasing connections between families and these resources, and increasing communication among human service providers to ensure that resources are known and shared throughout the community. And while there was a call for more support for their activities, the motivation for affecting change was high. Teams had clear understanding of their purpose and goals, and felt successful in accomplishing them.

In short, the quantitative evaluation mirrored other forms of BSCSI evaluation – pointing to overall success of the BSCSI process in spite of significant challenges. Breakthrough ideas for better serving Arizona families were created, tested, refined and ultimately implemented. Human service delivery systems of Arizona communities were positively impacted – Arizona families are the ultimate beneficiaries. CREDITS The ASU Partnership for Community Development wishes to thank the Arizona Department of Economic Security for funding this research. In particular, we are grateful to Judith Fritsch and Susan Hallett (Co-Program Administrators Office of Community Partnerships) for their commitment to the BSCSI process and contagious passion for Arizona families and

35

communities. Their wisdom, guidance and support throughout the research process has been deeply appreciated. We would also like to express gratitude for the many DES Partners, Community Partners, and Family Partners involved in the BSCSI process and their willingness to dedicate many hours to the completion of our assessment forms. Your responses gave us great insight about the impacts you have had on families in our Arizona communities. REFERENCES Krile, James F., The Community Leadership Handbook – Framing Ideas, Building Relationships

and Mobilizing Resources. (St. Paul, MN: Fieldstone Alliance Publishing Center, 2006). Arizona Department of Economic Security, The Breakthrough Series Collaborative on Service

Integration Framework for Change. (Phoenix, AZ: Arizona Department of Economic Security, 2005).

36

37

APPENDIX A

SELF-ASSESSMENT QUESTIONNAIRE

38

Breakthrough Series Collaborative on Service Integration(BSCSI)

Self-Assessment Survey Thanks for entering the online version of the Breakthrough Series Collaborative Service Integration Self-Assessment Survey. We appreciate your participation. Though this final evaluation is longer that the others, it is very important to have all Team members provide feedback on the program. Please enter the code number that was provided in the e-mail letter we sent you OR the number off the back of your paper questionnaire that you received in the mail. Code number: _______ Section 1 -- First, we would like answers to some general questions to help us put the rest of the information we collect in context. 1. To what extent are you aware of the diverse programs and services available in your community to help families? Not aware Somewhat aware Slightly aware Very aware 2. To what extent do you feel you know how to access these services to help families? Not knowledgeable Somewhat knowledgeable Slightly knowledgeable Very knowledgeable Section 2 -- This second section asks you about some key principles that guide all aspects of delivering services to families. Please let us know the extent to which you: 1) agree with each principle as being important when delivering services and 2) the extent to which you feel each principle is being practiced in your community. 3. Services are provided to families in a complete, accessible and responsible manner. (Successful service integration is more

than just locating services in the same place, blending funding from different sources, and making decisions based on common information.)

Strongly disagree Disagree Neither Agree Strongly agree As a principle Is being practiced 4. Parents, children, youth, kin, communities and tribes have many strengths, resiliency and natural supports that can help

resolve problems that arise. Strongly disagree Disagree Neither Agree Strongly agree As a principle Is being practiced 5. DES works together with families to develop and enhance independent, self-sustaining families and communities. Strongly disagree Disagree Neither Agree Strongly agree As a principle Is being practiced 6. Programs will emphasize prevention and early intervention services. At-risk children and families are identified early on

to prevent later involvement with more complex child welfare and criminal justice agencies. Strongly disagree Disagree Neither Agree Strongly agree As a principle Is being practiced

39

7. Working with Community Partners is necessary to determine and fill gaps in the DES service delivery system. Collaborative efforts improve our ability to meet the needs of children and families.

Strongly disagree Disagree Neither Agree Strongly agree As a principle Is being practiced 8. All families have something to contribute to the good of the community. Improved outcomes for children and families

occur when families, community groups, faith-based groups, non-profit organizations, businesses and government work together with a common goal.

Strongly disagree Disagree Neither Agree Strongly agree As a principle Is being practiced Section 3 -- In this section we ask you a series of questions about several areas we are working to improve. Please indicate: a) how important you think the specific practice is by selecting one answer for each item in the first set of questions and b) the extent to which you agree this practice is currently being implemented by selecting one answer for each item in the second set of questions. 9. The following practices are related to the initial contact (intake) with families and the assessment process for determining

their needs. a) Importance of practices that: Not Important Slightly

Important Important Very

Important Extremely Important

Gather appropriate information during the initial contact with families in a uniform manner

Gather appropriate information during the assessment process in a uniform manner

Standardize intake and assessment documents across programs and service providers

Engage families in a holistic manner during the intake and assessment process

Encourage agency and service providers to recognize and understand the family's needs within the context of the family's culture

Promote sharing of data and documents collected across DES programs

b) Implementation of practices that: Strongly

disagree Disagree Neither Agree Strongly

agree Gather appropriate information during the initial contact

with families in a uniform manner

Gather appropriate information during the assessment process in a uniform manner

Standardize intake and assessment documents across programs and service providers

Engage families in a holistic manner during the intake and assessment process

Encourage agency and service providers to recognize and understand the family's needs within the context of the family's culture

Promote sharing of data and documents collected across DES programs

40

10. These next few questions relate to the coordination and integration of the delivery of services to assist families. The process should be efficient with families and agencies working together effectively so that everyone has improved outcomes and the best experience possible.

a) Importance of practices that: Not Important Slightly Important

Important Very Important

Extremely Important

Coordinate services by integrating the process and people involved

Support a single case review/plan for families who receive multiple services from DES and the community

Demonstrate that all DES programs connect families to community resources

Have all DES programs connecting families to community resources

Support co-locating DES and Community Partners in the same place to improve communication, planning and service delivery

b) Implementation of practices that: Strongly

disagree Disagree Neither Agree Strongly

agree Coordinate services by integrating the process and people

involved

Support a single case review/plan for families who receive multiple services from DES and the community

Demonstrate that all DES programs connect families to community resources

Have all DES programs connecting families to community resources

Support co-locating DES and Community Partners in the same place to improve communication, planning and service delivery

11. Next, we are asking some questions about the accessibility of services and the accountability of service Partners in

meeting the needs of families. a) Importance of practices that: Not Important Slightly

Important Important Very

Important Extremely Important

Promote systems that include the “family voice” in designing and implementing services

Promote systems that include the "family voice" in assessing the quality and usefulness of services

Ensure services are available to families that deal with all of their needs

Promote services that recognize and include the diversity and strengths of families and their extended community

Ensure services that support families who are identified as at-risk

Build on existing relationships of families as well as extended families

Create effective ways of sharing information about services and resources available through DES and other community organizations

41

b) Implementation of practices that: Strongly disagree

Disagree Neither Agree Strongly agree

Promote systems that include the “family voice” in designing and implementing services

Promote systems that include the "family voice" in assessing the quality and usefulness of services

Ensure services are available to families that deal with all of their needs

Promote services that recognize and include the diversity and strengths of families and their extended community

Ensure services that support families who are identified as at-risk

Build on existing relationships of families as well as extended families

Create effective ways of sharing information about services and resources available through DES and other community organizations

12. The final group of questions relate to delivering services that emphasize prevention of problems within families and early

intervention by service providers when problems arise. a) Importance of practices that: Not Important Slightly

Important Important Very

Important Extremely Important

Allow access to and use of information to predict family and community needs

Allow the “family voice” to be heard when planning prevention and early-intervention services

Ensure services provide for the needs of families before problems arise as well as immediately after problems arise

Promote Partnerships with agencies that focus on prevention and early intervention

Coordinate funding for innovative solutions to prevent crisis

Allow the community to define the quality of services b) Implementation of practices that: Strongly

disagree Disagree Neither Agree Strongly

agree Allow access to and use of information to predict family

and community needs

Allow the “family voice” to be heard when planning prevention and early-intervention services

Ensure services provide for the needs of families before problems arise as well as immediately after problems arise

Promote Partnerships with agencies that focus on prevention and early intervention

Coordinate funding for innovative solutions to prevent crisis

Allow the community to define the quality of services Section 4 -- Three desired outcomes for the Service Integration project are: 1) promoting self-sufficiency, 2) strengthening families, and 3) developing the capacity of extended families and communities. The next three questions ask you to think of the families you have assisted in this project and assess how much your Team has helped them achieve these outcomes.

42

13. Please think about outcome one, promoting self-sufficiency. Now think about the families who have been impacted by your Team. Please indicate to what extent you feel your Team has impacted families in the following areas.

We have helped families to: Strongly disagree

Disagree Neither Agree Strongly agree

Know and recognize their needs Know their assets and abilities Prioritize their needs Prioritize their assets and abilities Envision their potential to be self-sufficient Have knowledge of resources in the community Be innovative in overcoming barriers Be proficient in accessing resources (e.g., money, people,

services)

Have sufficient resources Make things happen for themselves Move toward their greatest potential Be inspired to move beyond reliance on social services 14. Please think about outcome two, strengthening families. Now think about the families who have been impacted by your

Team. Please indicate to what extent you feel your Team has strengthened families in the following ways. We have helped our families strengthen their ability

to: Strongly disagree

Disagree Neither Agree Strongly agree

Create a safe environment for all family members Increase commitment between family members Develop positive communication among family members Increase appreciation, affection and nurturing among

family members

Adapt to hardship Endure through hardship Increase spiritual (not necessarily religious) well-being

within the family

Spend more time together Increase connections between families and outside

support networks

Become more socially connected (e.g., to neighborhood, other families, etc.)

Become more engaged in their communities (e.g., neighborhood watch, events, volunteering, etc.)

43

15. Please think about outcome three, developing the capacity of extended families and communities. Now think about the families who have been impacted by your Team. Please indicate to what extent you feel your Team has assisted developing the capacity of extended families and communities to support these families.

We have helped our families by: Strongly disagree

Disagree Neither Agree Strongly agree

Safely decreasing the number of children in institutional placements (e.g., group homes, shelters)

Safely decreasing the number of adult family members in institutional placements (e.g., group homes, shelters)

Safely increasing the number of families supported by extended family members

Increasing services that are accessible Increasing timeliness of services Increasing efficiency of services Increasing services that are coordinated (not fragmented) Increasing opportunities for families to receive services

and support in their own communities

Increasing “wrap around” services (e.g., smooth transition between services)

Increasing the consideration of local strengths and needs in designing and delivering services

Increasing communication to ensure that resources are known and shared throughout the community

Increasing culturally sound services Increasing connections to resources Increase knowledge and awareness of prevention and

early intervention services

Increasing knowledge and building the assets of families and the communities in which they live

Increasing the ability to engage families in systems and solutions

Increasing the ability to engage in Partnerships between families, Community Partners, and governments in working toward common goals

Increasing the ability to be innovative in solutions Section 5 -- The next section asks you a few questions about the BSC Team with which you are involved. 16. What BSC Team do you belong to? Apache/St. Johns Maricopa/ASU Downtown

Campus Pinal/Coolidge

Cochise/Sierra Vista Maricopa/East Valley Santa Cruz/Nogales Coconino/Flagstaff Maricopa/Golden Gate Yavapai/Prescott Valley Gila/Globe Maricopa/Avondale Yuma/Yuma Graham/Safford Mohave/Lake Havasu City Hopi Tribe/Kykotsmovi Greenlee/Clifton Navajo/Show Low Pascua Yaqui Tribe/ Valencia La Paz/Parker Pima/Ft. Lowell

44

17. What was your motivation for joining the Team? (Please write below). _____________________________________________________________________________________

______________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

______________________________________________________________ 18. Many things have contributed to successful Team outcomes. To what extent do you believe these characteristics describe

your Team? Team characteristics: Strongly

disagree Disagree Neither Agree Strongly

agree Strong participation by Family Partners Strong participation by Community Partners Strong leadership for the Team Commitment to the philosophy of the program Good relationships among Team members Diverse knowledge and experience represented on the

Team

Having fun during meetings Maintaining flexibility during the process Demonstrating innovation Are there other things that worked especially well for your Team? _____________________________________________________________________________________

________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

__________________ 19. To what extent do you feel the following actions could have helped your Team achieve successful outcomes? Things that could have helped: Not helpful Slightly helpful Moderately

helpful Very helpful

Provide incentives for Family Partner participation Support from DES for engaging more Family Partners Increased clarity of the project and process Increased knowledge about this project relative to the

larger DES system

More acknowledgment and encouragement for the Teams

45

Are there other things that you think could have helped your Team achieve successful outcomes? _____________________________________________________________________________________

________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

__________________ 20. To what extent do you think the following possible problems created barriers to achieving outcomes? Barriers: Strongly

disagree Disagree Neither Agree Strongly

agree Took a while to get started Establishing Team cohesiveness took some time People moving in and out of the Team Lack of knowledge about the project by the larger DES

community

Difficulty engaging Family Partners Confusion about who was in charge of the Team Confusion about the purpose of the project Lack of authority to implement actions Lack of resources needed to implement actions Extranet/computer problems Overly complex process Not feeling that the Team was valued Are there other barriers to achieving outcomes that your Team encountered? _____________________________________________________________________________________

________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

__________________ Section 6 -- This section asks you to evaluate various aspects of your Team. 21. On average, how often did your Team meet on a monthly basis? _______________ 22. How did your Team usually meet? In person as an entire group In person in sub-groups By telephone conference

46

23. What is your assessment of the level of engagement by the Team Partners in general? Partners: None Low Average High DES Team Partners Community Team Partners Family Team Partners Overall Team 24. Please respond to the following items with respect to the experience you have been having with your Team. Experiences with your Team: Strongly

disagree Disagree Neither Agree Strongly

agree I have a clear understanding of our Team's purpose I understand our Team's goals I know the indicators of success for our Team Our Team has the right number of people Our Team has the right mix of skills, experience, and

connections

Team members have clear roles and responsibilities Our Team plans and manages meetings to make efficient

use of time

Our Team spends enough time working on Team goals versus dealing with more reactive issues

Our Team uses agreed-upon processes for making decisions

Our Team makes sound and timely decisions Team members communicate with each other openly and

directly

Team members are good at safeguarding confidential information

Our Team takes the time to learn from both successes and failures

Team members are passionate about our Team's purpose I believe my fellow Team members to be trustworthy I believe our Team has a high level of relevant

knowledge, experience, and strong social connections

Team members are actively involved with Team goal setting and decision making

Our Team has the authority it needs to make important Team decisions

Our Team has the resources necessary to achieve the Team's goals

Our Team has had enough time to achieve results Our Team builds and executes plans to accomplish Team

goals

People are proud to be part of this Team Our Team works well together The people on our Team trust each other Our Team resolves conflict among Team members in a

timely manner

Team members back up the decisions made by the Team Our Team adapts quickly to changing demands Our Team produces high-quality work Our Team stays on track and meets deadlines Our Team gets results

47

25. Did you accomplish goals via participation with your BSC Team that you could not have otherwise accomplished? No Yes If yes, what is one example of such an accomplishment? _____________________________________________________________________________________

____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

______________________ 26. What is your overall assessment of your Team's progress? Not satisfied Slightly satisfied Moderately satisfied Very satisfied Extremely satisfied Section 7 -- Please complete this section only if you did not submit a questionnaire during the first or second rounds of the survey. This final section asks for some descriptive information about you. The information is completely confidential and we use it only to have a better understanding of our respondents. 27. Have you ever personally been involved with receiving services through DES? Yes No If yes, please explain _____________________________________________________________________________________

____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

__________________________________________________________ 28. Please tell us how you are associated with the Service Integration Breakthrough Series Team: Family Partner Department of Economic Security (DES) Partner Community Partner If you are a DES Partner, how many years have you been employed by DES? _____ If you are a Community Partner, how many years have you been in the social

services field? _____

If you are a Community Partner, with what type of organization are you affiliated? Faith-based Other non-profit organization Tribal: please

designate Tribal affiliation: Community advocate Other government _______________________________ 29. What is your gender? Female Male 30. Do you have a spouse or Partner with whom you live? Yes No 31. What is your age? (please write your age in the box clearly) _____

48

32. How many people are living in your household? (please write the number in the box clearly)

_____

33. How many children 18 years of age or under are living in your household?

(please write the number in the box clearly) _____

34. What is the highest education level you have achieved? (please X one) Less than high school Some college Four year college degree High school graduate Two year college degree Advanced degree 35. Of which ethnic or racial group(s) are you a member (please X all that apply) African American/Black American Indian Hispanic/Latino Asian American/Pacific Islander Euro American/White Other (specify:) 36. What was your annual household income in 2005 before taxes? (please X one) $20,000 or less $40,001 - 60,000 $80,001 - 100,000 $20,001 - 40,000 $60,001 - 80,000 $100,001 or more Thank you for taking the time to share your perspectives. We really appreciate your participation! If there is anything more you would like to share with us, please write it in the space provided below. __________________________________________________________________________________________

__________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

___________

APPENDIX B

MOTIVATIONS FOR JOINING THE BSCSI TEAM

(OPEN ENDED RESPONSES)

49

Motivation for Joining the BSCSI Team – Open Ended Responses Source: Question 17 of the BSCSI Self-Assessment Questionnaire (Appendix A). Note: Responses noted with an ellipsis indicate the respondent’s response exceeded the character space limits allotted for the survey question. • Integration of services. Better communication. Better service to our customers. Holistic services. • Told to do so. • To assist in improving the quality of life of those in need and through them improve the quality of

life for the entire community. • To help the community • Working and helping families. • I was volunteered. • Mandated that someone from my division be a part of the group. • To share information in assisting families, building Community Partnerships, gain knowledge on

services offered through other DES divisions. • I was asked by management if I wanted to join. I wanted to participate because service integration

has never truly happened during the 18 years I've worked with the state. • No motivation to join Team, this was an assignment given by supervisor. • I had no choice, was told we will attend even during staff shortages at the expense of client services

being delayed. Kind of defeated the purpose of the BSCs. • To help BSC make a change in our community (Tribal and Non-tribal). Even if the change would

be small, I wanted to make a difference. I wanted to help the community be more self-sufficient and make them aware of what resources are out there to help them reach their goals in life. I know they all have potential to be all they c…

• The hope that all DES agencies could combine efforts to make the entire application process as simple for clients and DES Partners.

• I was asked to join the Team. I didn't know what tit was all about until I got to go to the second meeting in Phoenix. It’s ok now that I know what it’s all about. Thank you....

• To assist with innovative changes for our consumers, learn more about our Community Partners, and work with DES personal in finding solutions for our customers.

• A strong interest in the theory and practice of service integration. • I was volunteered to attend. • To be apart of the solution to the current problem within or system of services provided to our

families in crisis. • To help people not go thru what I went thru. When I moved to AZ I was in Bullhead, sleeping in the

park and des had me waiting 9 days for an appt for help! • To improve coordination of services and engagement of families in a respectful way. • I was asked and when I found out what they Team was about, I wanted to be apart of something that

made a positive improvement to DES in Coolidge. • I thought I could make a difference and effect the services being offered. • I was hoping to make a difference in all of DES families and community all coming together to help

families • My motivation has been eliminating the barriers that our families deal with when looking for

assistance. • The Team was short staffed, I was asked to join by someone I have a lot of respect for. I had heard a

small amount of what the Team was doing & trying to accomplish & felt honored to be asked to join.

50

• My agency was approached by a DES representative, and my supervisor asked me to participate. Having worked in social services for about 15 years, I hoped to contribute to improving services for those in need.

• I wanted to help young families realize their self-worth and expand their horizons. This while making use of the local resources for self improvement and to enable them to move on out of the system.

• I wanted to join an effort to reach out to families and thought that this might be a way to do that. I also wanted to interact with other family workers.

• My supervisor attends a number of meetings; she could not attend and commit to one more. • Be connected with community. • Originally to develop more contacts and identify more service resources. • To help identify ways to improve services to families and increase a sense of community

empowerment. • Advocating for change and the desire for families to receive services timely without interruption

with hopes of creating open communication within the community in that there is no communication within the Departments. To learn from other agencies who are represented.

• Susan asked me. • I am a Family Member still receiving services and I wanted to be a voice for clients that would help

rather than hinder their welfare experience. • Help people help themselves, becoming self-sufficient while at the same time maintaining their

dignity and self-respect. • I didn't have a choice. My supervisor assigned me to be on the Team. However, I’m glad that she

did, because not only did I learn about all the services and opportunities out there, I also had an opportunity to meet some interesting and great people. I have always wanted to help people but wasn't sure what was out there but

• Was asked to join - part of our agency's responsibilities. • Continue Partnership with Pascua Yaqui to increase opportunities for tribal members to build self

sufficiency. • I am interested in seeing DES improve its services to families. • I wanted to learn as well as help with the improvement of the existing programs. However, I was

unable to stay with the Team till the end due to work responsibilities. • I was asked to participate and once I commit myself to something I make sure I make the best out of

it. • As a family member to be able to be apart of the change. Also to be educated on resources within

DES and the Community. I just plain LOVE the people involved. • My boss requested my attendance. • At first curiosity, wondering what this group would afford in services offered the community as a

whole and the hopes that changes and improvements would be addressed and made. What keeps me involved is the feeling we can and are making a difference.

• I was asked by 2 co-workers if I would be on the committee. Another member had resigned due to being too busy on their job.

• I was motivated to join the Pima/Ft. Lowell Team after my case with CPS was closed. I was asked to participate in the BSC by Ginger VanWinkle (CPS In-home/SVCS Supervisor) I was currently receiving services at the Ft. Lowell office and was not happy in the way that they were delivering services. The different programs did not see …

• To help make a difference. • I wanted to be part of something that would actually help someone or a family. I thought it was a

great idea to integrate DES services and that if we all worked together the goal could really be accomplished.

51

• I was volunteered into my Team. Once involved, I felt that only positive/improved changes could come from this process.

• Interested in the Service Integration principles. Ability to make small changes. Involving Community Partnerships in working with families to gain self-sufficiency.

• Improve family accessibility of services with in DES and the greater Tucson community. • To try to integrate our (DBME) service with the Community Partners for the benefit of our joint

participants. • My motivation for joining the Team is that I feel we can do a lot more in our community and DES

employees by learning about what each other offers in their agency. I feel that we all need to help one another and refer to each other. We are building our community and we all need each other to help out and work together to help t …

• I was asked by a representative of Congressman Rick Renzi office. I had never done anything like this before and wanted to see if I could help in another way in our community. I'm the employment director for the LDS church in the White Mts. We affect many people in so many ways for the entire community. It pleases me when we ….

• I enjoyed the collaboration & the networking. • Was asked to join at our equivalent of a Community Network Team. • As a social service provider/Family Partner, I know from personal experience that D.E.S. does not

provide holistic services, but rather addresses only what the client is there to apply for. I wanted to be a part of something bigger, something better for the good of my community. To make available, all resources in a holistic m….

• I have the need to be involved and want DES families with children to have the best services that are available to them.

• Feeling the need that we as a community definitely need to work together as a unit to help make resources more accessible and not so difficult to get the help that a person or family needs. It would not only benefit that person but would also benefit all the agencies that do work so hard to provide those services. To share infor…

• I really wanted to collaborate and have services integrated so families are assisted. I wanted to have families empowered to improve their lives.

• Was requested to. • I want to help families in need of a variety of services and make those services positive, informative,

helpful, and easily accessible. • To find, help my community to use the resources in our area. • To help empowered our families to empower themselves to be motivated to succeed in all they do. • Being able to make a difference in people’s lives and helping them to be successful in what they do.

Jointing hands with others the DES to make things happen. • To have community more involved in social solutions, rather than relying completely on

government. • To be able to make the community of programs that is beneficial to them. To help them overcome

the paperwork obstacle. • I wanted to help in moving families to be self sufficient • Was nominated. • Improve community and DES collaboration. • At the time I was assigned by my previous Program manage not know what the BSC was about.

After being assigned I have about other programs within DES and what they do for the community. It has been a great learning experience and look forward with working with other agencies.

• I work with a prevention program whose main recipients are at-risk families going through the DES system with much difficulty and backlash. I volunteered to be on the Team to see if I could help bring a more positive change to this system.

52

• I was told by my division that I would be participating in the Team. • I use to receive help from DES and now I’m on the other side of the glass and know how families do

need DES to be able to survive. • I was told to be part of this collaboration. • Experience new things. Get out of the daily routine. • I am optimistic and hope my involvement will help to promote a better understanding of the

workings of DES with the people who need services provided by DES by making it easier to access those services.

• The hope to make a difference. • I felt that the objectives were compatible with my beliefs. • To get involved with a community based committee to help the residents of this area. • My job that I am at now is a result of a DES program I have successfully gotten to self sufficiency

with these programs I feel that I wanted to do this integration more after we all knew what is going to accomplish and that I could be helpful in any way…

• Change. • My department supervisor requested I join, but due to employees leaving our department I was

unable to continue on a regular basis after November 2006. Also, I felt that there was a need to better help those seeking help.

• To understand their purpose and see how I can assist and or gain knowledge that may be helpful to others or myself someday.

• Wanted to see our office become more involved with the community and families. • Involving myself in the process of change by taking responsibility. • I volunteered to join the Team. My supervisor told me so!! Just kidding. Although I was appointed

to this Team, I have really enjoyed it and it has given me a greater awareness of community resources and what they can provide for our clients.

• I got a wonderful invitation and I felt like I could make a difference with my experience and understanding. I wasn't able to contribute as I liked, because the work was not in my own community. Also, I really didn't understand what the work was about, how it would last the whole year, etc. from the beginning.

• I was asked to start attending at the end of 3/07 because the original attendee was unable to continue due to office responsibilities. I came in very late after most PDAs had been done so I only know what the original people in the group have told me about their experiences.

• To help integrate work of Community Networks with Breakthrough Series... To increase collaborative impact of DES and Community Partners. To impact major systems reform.

• I transferred from another office where I was a Team member, and my supervisor here replaced himself with me.

• I wanted to help change and improve the delivery of services. I wanted to learn the DES system so as to better advocate for our participants. Also, a bullet on my agency's resume.

• My husband asked me to be part for input of info. • Due to low numbers of employees in Greenlee County I was the only one able to attend. • Develop relationship with Hopi tribal staff and Partners. • None, I was told that I had been put on this committee. • Feel privileged to be a part of making positive changes through DES for our families. • To share information and services to assist a family towards self-sufficiency. Better referrals for

quicker solutions, less stress for the family. Stop duplication of services, confusing to families. • The former Team member left this office. I was assigned this responsibility. • Helping DES staff and Clients to become more cohesive with community agency Partners in

empowering them with successful tools. • Personal betterment and to be involved with my community.

53

• Be involved in community and Partnering with other providers to assist community members. • Represent Community Partner to advocate for the needs of young children. • Because I have a good friend who asked me if I would be willing to take the time to do this as a

Family Partner. • To help make a difference in the system to be better services for the people who need help. • Community Network opportunity. • To work with DES to explore integrated services for at risk families that will lead to better long term

outcomes. • I was strongly encouraged by agency to join the group. • I believe in service integration. I have worked for DES/CPS for many years and I do not see how

the delivery of services can be provided other than in an integrated manner. The families DES serves have a multitude of needs but unfortunately we continue to provide fragmented services. Data and resources continue not to be shared.

• To have the opportunity to work on uniform communications and sharing of info between DES & Community Resources. Also to insure that all family needs were addressed holistically and thereby reducing the number of hurdles each family must face when receiving services from multiple sources.

• It was mandated that I attend. I have not attended in a long time. The main reason for this had to do with the Team. I felt that the members were trying to reinvent the wheel. Rural communities in AZ have been collaborating all along out of necessity.

• Appointed. • To further assist the families I deal with on a daily basis. To expand my knowledge on the variety of

resources to get them through daily life. • I was in the area and had become involved with several work Teams. • To see if we could make a difference. • Knowing I can help make a difference here in Nogales I have always wanted to help others and I

think it is important when people are in time of need to make things as easy as possible. • Trying to help the community that we serve. • I was assigned the task. • Build a strong community and see how my church can help. • To better serve our community and bring awareness of all services they are eligible for. • The fact that I could be the voice of our community's families (of course only the very few I could

contact) has been my motivation to participate in this Team. Hoping I could be listened to, and truly considered for my input.

• Find ways to better serve my clients. • None. • To be able to strengthen our community and be part of it to provide and assist my consumers much

better. • I was informed I needed to represent my division; I attended the first meeting and felt that it was

something that I could contribute too. • Helping families with resources within the community. Networking with other community members

to build an overall awareness of all available community resources. • Helping my community build stronger ties. • Desire to improve service delivery. Acquire better understanding on our community needs.

54

APPENDIX C

SERVICES RESPONDENTS HAVE RECEIVED THROUGH DES

(OPEN ENDED RESPONSES)

55

Services Respondents Have Received through DES – Open Ended Responses Source: Question 15 of the BSCSI Self-Assessment Questionnaire (Appendix A). Note: Responses noted with an ellipsis indicate the respondent’s response exceeded the character space limits allotted for the survey question. • I have been on cash assistance in the past, and now I am still receiving food stamps and Access. • Both with helping contact case managers at DES for clients and helping with applications or

information need for appointments at DES. • I have at one point in time have received food stamps, quimby and have been a vocational

rehabilitation client. • In 1982 I applied for UI benefits. Over-all the service was ok. • Although I am a DES employee, I still receive services (food stamps and AHCCCS). I am a single

mother of three children and could not make it without these services, as the state does not pay well enough for me and many others to remove ours…

• As a child I received services. • DDD is in our lives from birth. • Yes, my daughter has CP with a mental handicap. • My son is on SSA & AHCCCS and we have received food stamps in the past. However, more

currently I work as a Family Support Partner with many families in Pinal & Gila counties and I am somewhat concerned by the treatment they sometimes receive…

• When I was young our family was on Access and food stamps for a while. I was a foster parent for 8 years. I have a son with high functioning autism, he is doing very well now, and out of the program at 18. But I received a lot of help and t…

• Low income, unemployment service, Medical. • During my separation, I received assistance in Child Care and AHCCCS as secondary coverage. I

used these services for 6 months, until I got my situation resolved. • AHCCS, Food Stamps, Child Care, Child Support, CPS, and SSI Benefits. • Yes, 1990-1992. My family received help from DES when my husband was hurt during an accident,

and left responsibility on me. • No, but I hear everyone's experience that come to us for Community financial support. • Foster Parent. • Been on all three CASH, FS and MA cause I’m a single parent and the community resources always

ran out of money before you could ask for help - if you work the system you get the money first and those who for once in their life need help can't…

• I am the legal guardian for my grandchild and receive benefits for her (child care, medical care, and cash aid). I work and do not receive benefits for myself through DES.

• I am not young, so my dealings with DES cover many years and areas of service needs. I have used family assistance (FAA), unemployment, job services, child protective services (CPS), developmental disabilities (DDD), vocational rehabilitation…

• While attending college, I received IV-a services (food stamps, medical) and child care services. • My husband recently was laid-off and had to apply for benefits through DES. • Kids care health insurance. • Yes for medical, that is all I could qualify for. I feel I should have been able to receive food stamps

but they said I made to much money but I am a single mom with 2 children at the time and I am disabled. I would like to see people get more I…

• In 1974 I went thru a divorce and had 3 small children to raise alone with my families help. I applied for the Food Stamp, Medical and AFDC program due to no income and no job. The workers at

56

DES/FAA were very helpful and the Supervisor Lois… • I am currently receiving food stamps and AHCCCS for myself and my family as we are low income

and can barely make ends meet on our monthly income. • I have received cash assistance for and child support. • I received AHCCCS when I had my children, and I was on food stamps for a short period of time

when my husband was laid-off from a job (he collected unemployment as well until he got another job).

• Adoption Subsidy Services. • I received services from FAA. • I have a child in the DDD system. • I was 19, pregnant and without health insurance for my unborn child, back in 2001. • I received unemployment benefits for about one week. • I received Food Stamps, Child Care and Medical Assistance in the past while going through a

divorce. • When I was younger I was a single mother with two boys and need help for a while. • In the passed I have used AHCCCS, Food Stamp, Child Care and WIC. • I had to be on food stamps and medical for a while until I could find a "real" job to support our

family. • Family was on food stamps, and children were on AHCCCS when I was out of work, many years

ago. • Unemployment services 15 years ago. • My wife was unemployed for a period of time. • Foster child services. • I was provided with Child Care while in University. I could not have done it with out the service. I

also received food stamps for a brief time. Not so helpful for various reasons but still very useful at the time.

• For about 4 months in 1995 I received cash assistance right after my divorce and assistance with child care for a year or two. It really helped a lot.

• Received Food stamps and AHCCCS when I was laid of a previous job 6 years ago. • I have received food stamps and unemployment compensation in the past. • Unemployment benefits. • I have a son with a developmental disability and he was involved with AZEIP and is now

transitioned into the public school program. • Received services approximately 10 years ago for several summers when I was out of work and had

young children. • My son was born 6 weeks premature and my family received services via DDD, early intervention

program and I have also used employment services admin for years when looking for jobs, after high school, in college and post college.

• Received Food stamps, 23 years ago. • In the past I have received AFDC, Food Stamps, Medical coverage, Child Care and Child Support

services. • 20 Years back when I was unable to work due to high risk pregnancy. • 20+ years ago I was assisted with food stamps one time. I did not ever go back for help because of

the rude treatment I received. Also a couple years ago I did foster care for 2 nieces, CPS assisted with AHCCCS. I did not receive any cash as…

• When I first came to Tucson (1973), I used employment services to find a job. I was briefly unemployed and use Unemployment Insurance and job services again in 1982.

• I was on all services as a displaced homemaker. It was a night mare. I am educated and have always worked. I was treated as though I was incapable of making good decisions. I was honest with the

57

system and I ended up begging for food for myself… • My son receives services through DDD. He has a Developmental Disability. • About 40 years ago in California, received Food Stamps for about 3 to 4 months for my family. • Back in 1987 I received food stamps and medical coverage for my daughter. As of today, I’m

receiving services from the Child Support Enforcement office, … • Food Stamps and Medical during the strike for PHELPS Dodge in Morenci, Arizona, in 1983. But

because of lack of community resources in 1984, when my family moved to Tucson, we did not apply here and struggled terribly for a number of years until …

• I have never applied for or received benefits from des before, but I have family members who have. • AHCCCS and food stamps. • My husband and I have custody of our grandson and we initially used DES services for health

insurance. • In previous years applied for assistance with food stamps. I did not return to complete the process

due to treatment I received. Later applied and received assistance for child care in which I was treated very well and have since not require…

• I have received guardian subsidy and through the years have been through AHCCCS and other services.

• When my child needed medical assistance. • I have 2 children (adults now) that have CP. • I am currently receiving child care services & AHCCCS for my grandchild. • I have received AFDC for a grandchild that lived with me for a period of time. • I have personally gone through the intake systems with our clients (those in recovery from drug and

alcohol addiction) at numerous DES branches, both as an advocate and to learn hands on what they are experiencing.

• The Susie is under my care through CPS and I am receiving child care funding (paid directly to provider) to pay for her day care. I also receive a monthly reimbursement ranging from $38-$44 to pay for her expenses.

• When I first moved to Arizona I did collect unemployment for a few months before finding a job. • Food Stamps, medical, and currently going through child support services. • Many years ago I, with two children, was on Food Stamps but I got a job. Example to my children. • Food stamp and UI recipient. • I am currently in an FAA office for benefits. As well I have used CPS, Voc Rehab, DDD and many

other services under the DES umbrella for services important to maintaining a healthy family. • Many, many, years ago received FS for two months. • Not in Arizona. But back in Ohio. And I was not happy with the way I was treated. I was just a

number not a person. No help with other assistance programs just what I had come into the office to apply for.

• I received help thru medical services and child care for my grandchildren who I am raising. I was given other services, but I chose not to receive them.

• I have experience in receiving food stamps, child care and AHCCCS. I have helped others in the process of receiving other services.

• I have been on AHCCCS for many yrs for medical I don't know what I would have done if they did not offer this program.

• I was a single mother raising 6 children alone and needed help for a short time. • I received F/S, ACCESS, Schooling, and I'm now going to college, something I never thought I

would be doing! • Was involved in a DV household, left spouse went into shelter was at home mom. Know I'm unable

to work and am going into retraining. • Health care and disability.

58

• I receive food stamps and medical and over the summer I was getting child care as well. • 16 years ago, I was pregnant, had a toddler at home, unemployed, my spouse was abusive and didn't

always work. We didn't have health insurance. Kay Comer from DES was great and helped me get health care and food stamps to help me. It was f…

• I was receiving, at a time cash benefits and food stamps. Right now I am receiving medical care. • In CA my children and I received medical benefits, cash assistance, and food stamps. • I have been involved in several DES programs. The Jobs Program is what helped me get the job

where I am now and have been for 3 years now also have received food stamps and medical. • I am a foster parent. • I utilized DES Job Services when I relocated to Show Low in 1991 and that is how I became a State

employee. • I am raising 2 of my grandchildren and receive the Kinship benefit from DES as well a subsidy from

the State. My husband & I have had permanent guardianship for the last 4 years and have spent hours filling out paperwork and submitting forms …

• At a time in my life where income was an issue, I depended on the system to help myself and my children get through the daily struggle.

• Self and family. • Single mother of 4. Medical, child care, and EBT. • I currently utilize the services of DCSE and had used FAA benefits in the past. • My mother received AHCCCS Long-term care and I was the family member that applied for her.

The program provided her with such wonderful services performed by such wonderful, caring people. She passed away in June and I will be forever thankful.

• None. • Food stamps and some involved with child care and foster home C.P.S. • I have filed complaints of abuse with CPS. I have filed complaints against CPS. • I have foster children. I am pleased to say they have been good with me and on time. • My son has Spinal Bifida and hydrocephalus Hydrocepect. Trying to get services and help we

endured no calls back, rude workers, workers that act like we are bothering them and lots of nothing. Didn’t even know about a lot of services.

• Received TANF and Food Stamps at some point after my divorce. • When I was ill and needed services till I was able to get back on my feet to return back to work. • After getting away from Bullhead, I moved to Parker and received TANF, Food Stamps, Medical

until I found a job. I still receive food stamps and medical. • AHCCCS, food stamps, jobs program. • Before I was employed, and after my 1st husband left me with 4 kids, I needed assistance to

maintain my household, I received Food Stamps - I couldn't get any other services because I owned my house and my vehicle.

• AHCCCS (Value Options) for a family member. • AFDC for a grandchild. • This is a long story: when my daughter was 10 or 11 she fell and broke her arm. I was recently

divorced, working as a waitress and had no medical. The hospital referred me to DES. My initial experience was not pleasant to say the least. My…

• My father. • My family receives AHCCCS and also very little in Food Stamps. I am trying to be one of those

families of being self-efficient and it is very hard and frustrating. • Food Stamps, TANF, Jobs Service, Jobs Program, Childcare and Workshops. • I was on food stamps and medical for a short time. And had the help of job service in obtaining my

job with DES. • I came to Job Service to find a job.

59

• I have received DES benefits before from time to time, when my wages were not enough to support my family. Also, I got into this field (social services) because I was enrolled in the JOBS program. So, thank you very much DES!

• Cash Assistance, Food Stamps and Medical Assistance. • Younger received Medical. • I was made to feel small and unimportant that I had asked for my situation and didn't deserve to be

helped. It was awful. • Received cash assistance and child support services. • When I first moved here I lost my job and received AHCCCS, Food Stamps, Cash Assistance and

Child Care. No one told me about DCSE. • At one time I chose to receive DES benefits due to no self sufficiency in my family. I graduated

from ASU WEST while on benefits and am now self sufficient. • In 1976 as a single mother with 3 children and living with my parents and had no job or income. I

went to DES and applied for services and the supervisor of DES/FAA told me of a job with FAA, she helped me apply for the job and I was hired on…

• I have used DES for job application since my high school graduation. I used Family Assistance over 20 years ago.

• Jobs. • My grandchildren have received medical assistance. • When being a minor child living with a single parent. • Years ago I received AFDC, F/S and Medical due to a family emergency. It was a horrible

experience! • Many, many years ago received Food Stamps. • My son has autism. • Child protective services, food stamps, AHCCCS, Developmental Disabilities. • Food stamps about 20 years ago. • Received Child Care Services for 4 years and child support. • Food Stamps. • 1994-1996 in CA. I received Medical, Cash and Food Stamps for a family of 7. The Board of

Education in Riverside County offered childcare and I returned to college after a 20yr absence. Since I am writing this to you, I guess you know the res…

• Am receiving Unemployment, Food Stamps and AHCCCS. • My son has Spinal Bifida and Hydrocepho paraplegic. • Job services. • Was on benefits for a few years off and on before my employment with them. • FAA and Child Care services. • All programs by FAA. • Food Stamps and Cash Assistance Medical. • I have health care services. • When my children were younger, ACCHHS, food stamps, Unemployment benefits.

60

APPENDIX D

THINGS THAT WORKED WELL FOR THE TEAM

(OPEN ENDED RESPONSES)

61

Things that Worked Well for the Team – Open Ended Responses Source: Question 18 of the BSCSI Self-Assessment Questionnaire (Appendix A). Note: Responses noted with an ellipsis indicate the respondent’s response exceeded the character space limits allotted for the survey question. • The Team has chosen to focus on providing better resource information for clients so that they can

be more self sufficient and at the same time efficient in obtaining supportive services. All of our members seem to have enjoyed working in this area of improvement.

• Integration. • I feel that the communication is very helpful with in our Team and brain storming ideas. • Making each potluck culturally/ethnically different really helped. We had an ethnic theme each

time. • We had poor leadership and poor family and community participation. No lead DES staff (PM)

nothing worked for this Team. (It was a joke and waste of time). • Communication was very efficient for our Team. If we didn't have this we wouldn't have succeeded

in all we have done. Listening to our family member's did help out a lot! • No comment at this time. • Keith Smith was a strong leader, worked well with all the Team members, was willing to listen and

to learn. With his leadership, I believe the Tucson Tugboats worked hard to make small changes. • We all listened to each other, listened to everyone’s suggestions, and worked very well together. • We are all extremely dedicated to the families that we are given the opportunity to serve. • It was really great that the tribes and the town got together on this! • Emailing between each other to keep each of us connected seemed to work out pretty well. • N/A. • Brainstorming was our best tool - we all have good ideas, but brainstorming helped to define the

best ideas and how to get them done - also, everyone had excellent “Team spirit.” • I felt that for the most part, my Team was just going through the motions to meet the requests of

those overseeing the program. • Everyone was open in their communication and accepted. • Lively discussions about this effort and about service provision in DES. Understanding the variety

of programs and how those that participated operate. • Good Communication and collaboration. • Calling on each other to identify or support ideas. • Collaborative project where 18 people had critical roles towards determining the project's success. • In the majority the DES family consistently participated, not Community Partners. We had only one

strong Family Partner. Remember that in the rural areas, we work very differently that the metro areas.

• Our Team has a bunch of DES whiners that never wanted to break down silos. In fact, they put up silos by having a bad attitude about the whole process and not wanting to do more than sit at the table and gripe about how unfair it was that they were picked for this Team. Renae and Wes are the only DES Members that truly affected the Team and our common goal. I would love to continue with the …

• Mutual trust and respect. • The flexibility of when and where we can meet due to some individuals had to travel a distance to

attend meeting. • N/A. • Team work and flexibility among DES members. We all knew we had to attend the meeting, but

62

there were times we could not. • I feel our Team leader has a lot to do with it his willingness to listen and be open-minded. But most

of all supportive. I think the Tucson Access TV series has helped quite a bit in a lot of ways. The stories have educated and all have been inspirational to others dealing with the same problems. We have given hope.

• The desire to want to help our community, but, what we wished to do that would have helped the most, we could not do as we could not do anything that would cost the state any money and that PDSA would have been to have a Full-time Receptionist/Intake person available at a desk in the lobby of the DES office to meet and greet every single walk-in plus...complete a short questionnaire to det…

• Though we did have members from all area’s that did not hold in to the Team as would have been liked, the one’s that did become the core were excellent – with dependability and a positive can do attitude. I believe the folder aspect this Team (and most others from my understanding) was one of the best things we came up with. The positive response from the families and DES workers alike w…

• We stayed focused on the meeting, and accomplished the items we needed to work on. Everyone was committed to doing their assignment.

• Utilizing the Jobs program as a benefit I was able to engage myself and get credit for my participation in the BSC which allowed me to keep individuals connected either by phone or by email. We also reached out to the community through picnics and Public TV in our 6 part series show “Voices of Recovery.” We have tried to keep everyone engaged and have a “Project” or responsibility to be co…

• We all finally had the commitment but the leadership was in question and the motivation within the group was not there.

• Though Family Partner’s participation on the Team had its ups and downs, Community Partners and DES Partners worked well together. As a whole, with all Partners coming together at the table, creativity exploded and new ideas were thought of.

• Get up and do it family members!! • Our Team is small; we only have one Family Partner, three DES employees, and three Community

Partners that have truly committed to this project. The Team wants to make a difference in our community so by this terrific attitude our Team moves forward. So I think our caring and happy energy is what keeps our Team moving.

• We have had a number or our original members drop off the Team, however our remaining members have made it all worthwhile, including our leaders.

• Just that everyone on our Team is totally committed to the betterment of our community and community members. We will continue to work closely together, to provide the best possible services in a holistic manner here in Apache County. And I personally wish to thank all of you for this opportunity to serve my community and establish connections that will best serve Apache County.

• The friendships that have developed are what make this Team good. • We had a hard time getting people to the meeting, mainly because everyone has a job to do and

sometimes having the extra time to spare was hard, but to me, we’re worth it. I feel everyone on our Team does care about working together and making it a better community.

• There were no wrong answers which was good. I think we worked well together. • Having 4 committed core members. • Having different cultures represented at the table. • Keeping in touch even though we only met once a month. Email was good. • Knowing that we are all equal, and just because you were give a title of Coordinator, Senior Leader,

etc., that doesn’t make you bigger than anybody else. • We actually had a very good Team participation and well lead by our chairperson. She did not let

63

anyone not do anything. We all contributed. Kudos to our leader. • Overall, it was great! • Just all of the above. • The need or the wanting to make a change. • I believe that most, if not all, areas were covered in the question asked above. • It was strange, as our group was not consistent in attendance and it made it hard to accomplish our

goals. Then my office was short staffed due to medical problems and I could not continue on a regular basis. At first, it was hard to get the family members to attend, then in the middle 2 women stepped in as family members and they were great.

• Everyone got along. • Meeting several times a month with the distinct understanding that not all members will be able to

attend all meetings but are still well informed of what is going on. • Making sure that everyone had a place at the table. • I think that we work well together and everyone is ready to do their part. No one member has had to

do everything. • It seemed when someone dropped out they were able to come up with replacements. Sierra Vista

seemed to have a lot of community services, whom wanted to be involved. The Family Partners from Willcox were not included as well as we could have. We really didn't feel of much value either.

• When people could get away to attend our meetings there was good participation. Communication regarding when & where meetings were to be had some problems. Not everyone had email. Sometimes there were phone calls made early on, but fell away later due to excess work loads.

• It is not driven by one person. It is Team input. All comments are welcomed and voted on. • The lack of work ethic and motivation that exists overall in this community prevents change and

innovation from occurring except on the smallest scale. • PDSA Planning was very successful. • Just being involved. • No. • The Team was diverse and committed and it basically maintained limited objectives. • N/A • Food was a motivating factor. • There was great commitment to the Team in the beginning. • The DES Partners all got along well. • Most of the changes needed we could not address because DES continued to say only the legislature

can change that. We were handicapped in many areas. • We all worked very well together, we were a very diverse Team and it worked out well. We had a

great leader. • Tess' enthusiasm was greatly appreciated. • Strong support from (participating) DES Partners.

64

APPENDIX E

THINGS THAT COULD HAVE HELPED TEAM ACHIEVE SUCCESSFUL OUTCOMES

(OPEN ENDED RESPONSES)

65

Things that Could Have Helped Team Achieve Successful Outcomes – Open-Ended Responses Source: Question 19 of the BSCSI Self-Assessment Questionnaire (Appendix A). Note: Responses noted with an ellipsis indicate the respondent’s response exceeded the character space limits allotted for the survey question. • Consistent participation. • We need to have greater participation of both family members and Community Partners. By greater

I mean more consistent and more available. • Yes, the administration and the Team members knew what we were all about; however, the rest of

DES did not. This would have helped immensely with trying to follow through with our PDSA if all members of DES knew what we were attempting to accomplish, this way we could have all been headed the same way rather than having conflict as we forged ahead.

• I really thought we were going to have input on how to change DES to knock down barriers that DES presents to families.

• Ensuring DES Partners are committed to participation. Monitoring of participation by Program Managers or other Division Heads to ensure the Team is successful in achieving the goals and objectives of the BSC. I would have preferred that the process be explained in a follow up session after the original first meeting. A “Train the Trainer” process which would have involved the Team lead…

• Better participation from Family and community members. • DES could have listened to the Teams suggestions when then didn't it put up barriers and the

motivation Team decrease. Also this Team had very poor leadership when there were other Team’s members who are known for their successful leadership skills. In fact this group of people shouldn't even be defined or referred as a “Team.”

• More Community/outside suggestions could have helped our Team when we were stumped. • More involvement. • Stronger backing from DES management. • I think having more direction and having the funds or incentives for family members to attend. • Everyone seemed to be too busy to participate! The ones of us who wanted to be more involved

were at the mercy of the people who didn't have time! Lots of time, most people didn't show up. They claimed they weren't notified.

• We didn't have enough DES participation on our Teams. • We were told at the beginning that ideas we came up with would be enacted. Our first one was to

establish a person in the lobby to greet people, answer a live phone and assist people. We were turned down flat. This made some people upset. “No money” was the reason for turning us down. That's not what we were told would be DES’ answer. Coolidge used to have 1 person at the front,…

• More participation by all members. • Not many supervisors and/or Program Managers were very aware of what BSC was or that it even

existed. Also, if there were a clearer cut (less red tape) way to accomplish what is actually needed for a DES office to provide better services to the community.

• Better informing Team members of the jobs each Team member does and their relationship to the population in need.

• More commitment from the DES Partners. Food and prize incentives for the DES Team. • We should have had more and longer lasting participation from customers. It is certainly an

important consideration to expect people who are struggling with their own personal challenges to attend these types of activities unless they can understand to what extent these efforts will benefit

66

them in solving their own problems. I suspect that commitment and relegation of time is also somet…

• DES Management could be more supportive of their employees participating in the Team. The Team also should have been provided with a better outline of what they are able to do. Changes made were not approved or supported through DES Management while the employees are told to participate and the Team as a whole can make changes the outcomes and ideas were not supported therefore changes were …

• Budget. • We have a fantastic Planning Team but the supervisors at our local offices bucked us every inch of

the way. Our Team needed cohesiveness which, I don't think we ever attained because of resentment.

• Participation from District Program Manager. The Program Manager was on our Team until she retired. After that, there was no input or participation from the new Program Manager. I think that even though the new Program Manager wasn't on our Team, she should have at least attended one of our meetings or even asking about how our Team is doing. I think that our Team would have been more…

• I think that only the members of the BSC in DES were willing participants. Other DES staff didn't change anything including the way they think.

• Less meetings and more interaction from the family members. • I think for us everything has come together the way it was suppose to. Everything happens for a

reason. And sometimes we have no say so in the matter. • Given us the power to implement projects that DES/St. AZ would have funded. • It would have given a stronger message of possibilities had there been feedback that the Governor

herself was listening to what the Teams were doing and that at the end she did enact some change from all our work. This concept is a very positive one with great potential, but it had no teeth.

• We have only one more project on the table at this time, I think our Team needs a mentor from the higher ups that would tell us how to do things without all the RED TAPE. We are in the process of trying to establish a new position, one the families have said they wanted. With the old way of doing business this is almost impossible. It is hard to know what to do and where to get help.

• Not having the Bureaucratic RED TAPE to go through to get approval for most things. We found that just taking it upon ourselves as Family Partners we were more successful in reaching our completion on most of our PDSAs. Susan and Judith along with the help of Janice did keep the spirit of the BSC alive though even if in Phoenix.

• Strong support from DES upper management/hierarchy and a much more user friendly system for entering data.

• A clearer definition of what each Partner’s “role” on the Team was. • More support from the local office with the project. Not much from supervisors or local office

manager. • Yes, I think that with our Team needed help because the tribe did not understand what part we both

played in making this project work. As of today we have not been able to meet with them or the DES Family Assistant Office for one year we have tried to break this barrier and have had no success I really wish we had someone with the authority to make this happen. When the project started we…

• If had a small budget to accomplish some of the good ideas we came up with we would have reach our goals of our PDSAs.

• This was a new process for our DES Coordinators. Perhaps they should have more training before our first meeting to know more about the BSC.

• Well, believe it or not, I think we should do this every couple of years. One year was just not enough time. I think most of the Team members would be willing to continue this Partnership.

• More Family Partners could have helped our Team become more successful.

67

• Unfortunately, if we would have had more time to work on just putting this together and would have been able to be more active and not trying to keep up with all of your normal duties in your job that would have helped. More time to be more involved I think would have brought out more creativity and could have achieved more.

• Support from DES and real dedication to change. DES is unwilling to put money to assist families and to improve systems. DES was unwilling to listen to what was requested. Unwilling to change paperwork to integrate assessments. Unwilling to support our group at all. It was nice to Partner with each other but basically a waste of our time and efforts.

• Mini, local or regional, 1 day “Summits” would have kept the momentum and morale up. • More time to dedicate to the Team and its needs. • Having more Family Partners. • Getting the support from our local DES bosses. Having to find funding, and getting other agencies in

the community to commit in this process. Not being able to change some policies in the DES program.

• No, our jobs are too busy and it was hard to keep the same people at each meeting. • Local DES cooperation and support (employees, the entire hierarchy). “Upper” DES help and

support from Team leads, etc. • Yes, I think if the Teams had more cooperation by the hierarchy of the DES Administration. They

gave us full reigns to accomplish projects, but as long as we took care of them, no help in getting some of our PDSAs completed, namely, our Lobby Specialist.

• Yes. More cooperation from management. I bet that up until now they have no idea of what BSC is. They never had time to follow up on the Team progress; never cared.

• Stronger commitment from Family Partners. • The support of fellow co-workers and higher management. • We defiantly do need more Family Partners on our side. In our case, community and DES seem to

work fairly well in a crisis. • More commitment from DES. • A more clear and specific goal to accomplish. It was a frustrating process. After about half way

through we figured out a project that worked. So maybe an advisor who either attends by teleconference or actually is in attendance at meetings for guidance. I mean someone who really is trained in the process. We had Senior Leaders but they did not know too much about the process so it was confusin…

• Family members have no transportation or child care and could not attend meetings. • Having bigger dreams. • It would have been nice if we could have met more often, but everyone on our Team has so many

responsibilities and are from such a wide-spread area that it made it hard to get together more than once or twice a month. It was also hard because there were things we would like to have seen changed but that was beyond our capabilities.

• I am really not sure about the outcome, but I do think there could have been more communication from our own Team. I only get emails from the main program now.

• I was told originally that Maricopa East Valley was to be working with one of the Native American Tribes but the Tribe already had something like this in place so they backed out and the East Valley Group had to scramble to recruit Family Partners. Once the Family Partner provided their complaint, to be reviewed and a resolution determined, then the Family Partner never came back.

• More members and ones that actually attend and participate. • The key is to engage Family Partners. We had some assigned to our Team but they live over an

hour away (in the same county). This helped muddy the waters from the start. They had no knowledge how the Sierra Vista office worked. Became a barrier and distraction. More of a time commitment than I had anticipated.

68

• Just the time and support for our DES Partners. • It is difficult to focus on the extra projects when your individual job is demanding and you are under

staffed. • Solutions to keeping family and Community Partners interested in BSC. • I think our lack of success stems more from the complacency and lack of service providers, due to

our locale. Unfortunately, the CRIT people have not embraced prevention or responsibility. • Having Daycare and transportation vouchers for families to attend on a regular basis or some sort of

reward system for each family member attended they would earn points or credits for special or specific items that would encourage them to make meeting attendance: like a Wal-Mart Card $25, Star Bucks Card $5. etc. One handicap is that many of the families do not have internet access except t…

• Any support from DES upper hierarchy which was non existent. No support from DES upper management staff or any DES employees that were not on the Team. Only constant complaining and lack of interest in anything we tried to do to help customers.

• More Family Partners • Working together and try to keep the same people--each meeting someone was different--mostly

talked about themselves, never worked as a Team, always different people, only 1 or 2 parents showed up.

• Many DES Partners seemed to fear repercussions if they tried to change any of the current DES systems. Many seemed unconvinced that they had the support of their supervisors.

• Our Team was never really on the same page. The tribe seemed to want the Team to work on their needs only. The other Partners wanted to be more diverse and to include the surrounding community also. Clarification was sought but the tribe never seemed to buy into the established goal of serving other communities. I suggest if a tribal service integration Team is established again it be …

• To have better understood the process of developing PDA's. Closer liaison with project coordinators at State level including their attending monthly meetings.

• Better participation from the community service providers from the beginning. • Well we already have a similar program that does this same thing in the community. Why couldn't

we have just joined with them a lot of participation was not there because they were already on this other Team?

• Have little gifts, small stuff, for everyone who is there. • More support from upper management to help the negative feedback we received from local DES

departments. • Perhaps if there were more family members constantly participating in the process; instead of having

6 DES participants, 6 community members, 6 family members, changing it to be only 3 DES, 6 community members, but 9 family members? Because this BSC Team should lean more towards what families have to say about how they feel. DES members are mostly trying to work out their insufficiencies…

• Meaningful systems change must come from the top. • More training in the beginning. • Reaching out to more community members. • Greater commitment and participation from all DES Partners.

69

APPENDIX F

BARRIERS TO ACHIEVING OUTCOMES

(OPEN ENDED RESPONSES)

70

Barriers to Achieving Outcomes – Open-Ended Responses Source: Question 20 of the BSCSI Self-Assessment Questionnaire (Appendix A). Note: Responses noted with an ellipsis indicate the respondent’s response exceeded the character space limits allotted for the survey question. • No. • Lack of backing by the higher administration. All talk and no action. • Again, the major barriers are a lack of participation not only with family and Community

Partners but within DES Partners. Explain the roles and responsibilities of the Senior Leaders and encourage their participation. Being a Senior Leader and not following through is not setting an example for other Team members to encourage participation.

• Team members used their own money to purchase items for some of the PDSAs. • Not enough dedication to the Team by some DES Agencies and Family and Community

Partners. • This “Team” was a joke. There was distention among the DES staff and when brought to

the PM there was no action taken by the people in charge. • We started out with a full Team; just toward the end here we lost a couple of members. That

was a barrier to our Team, due to a lack of input on what families need. Somehow, someway, we overcame that and continued to make a small change.

• Yes, not enough involvement with the group; we needed more and new ideas. • Other than the DES Team members, many DES personnel and Managers did not understand

the Team and placed barriers to the implementation of projects on State property. • Just scheduling. I don't know who was in charge of it, but it wasn't consistent. I could never

count on it being on certain day or even close. It may have helped attendance if it was set up on a regular schedule.

• Our Team leader quit in August. Our new one didn't start till October. Joe seemed helpful at times, but it took two months to get a letter from him to submit for funds to get the TV/VCR paid for, which is totally showing to the rest of the Team that Joe never really cared about what the BSC Team really meant and was just showing up cause he was told he had to. Didn't feel that he really w…

• Having the time to be on the Team and keeping up with work. Felt like I did not have time for both.

• Mostly just the red tape that a government agency feels is necessary to initiate changes. • Scheduling of meetings seemed to be a major problem. Rarely did the DES Partners have

the ability to get away from their assigned jobs needs to attend. • Better directions were needed. More resources were needed: Meeting places, Drinks and

snacks, Supplies, Team reimbursement for out of pocket expenses. • You list them well. • Communication is a major barrier; although DES employees are co-located they are not

familiar with each others programs. The Team members especially the DES employees put barriers in the meeting times and some are very negative abut accommodating the rest of the Team so they choose not to participate. The meetings could be called at various times and it could be clearer as to the various id …

71

• Conflict between Team members – resolved. • Lack of willingness by “upper” people to help us get the job done. • Team members not following through with what they are assigned to do. Not everyone

shows up for meetings. • N/A • No, I don't think so. • Only a few. Possibly creating a bigger problem if engaged but still on our list to achieve it

just may take more time. • Not having DES supervisory buy-in (allowing DES employees to attend meetings, etc.).

Another barrier our Team encountered is that we were spread too far apart to making attendance at meetings convenient for all. Several Team members were from Apache Jct. which is approx. 1 hour away from Coolidge. These Partners all too often were unable to attend meetings. To overcome this type of sit …

• Not so much. Feeling the Team was not valued, but more so that the work we were doing did not serve as a means to an end – back to the teeth thing. No matter how important a concept was proven to be, it was hard to get the resources that helped make it a reality.

• We had a hard time getting family members to the table. The computer system was crazy. The process of updating was put on one person; no one else could get access. Too much paper work each meeting. Would have been better to use old forms a couple of times, lots of wasted paper. Maybe if we printed project out once, and then updated with one sheet instead of the whole thing over, and over.

• Some Community Partners were not able to be at meetings because of the times or days. • Strong support from DES upper management/hierarchy. • Focus changed from Community Based Outcomes to Individual Stories. • A community that does not want to recognize that there is a very large problem within the

community to help the people of our community. (City leaders believe there are no problems. Our community no longer has the different classes within the community. You are either upper class or lower class; there is no more middle class within our community.) Not enough jobs within the community to su…

• We had trouble on the meeting time and dates because everyone has such different schedules. Also the families did not get any money for time. One really big issue was the money for events. Our Team has had four events that needed supplies. The Team members needed to put in the money. Also the DES workers have put in most of the money for our Team. I think that two of the biggest barrier …

• No. • Getting together. • Lack of support from DES. DES wasted time and effort in this if they were unable to make

change. No money put towards the suggested changes/improvements. • DES members not wanting or willing to participate. • Community Partners having their own agendas for community services. • For me personally, it was understanding how the cycles worked. • Our Team was so spaced out in the Community Partners but we made it most of the time

and this was only once a month to work on the PDSAs. Not being able to work on things weekly.

• Feel we were let down by DES upper management to enforce attendance by the different

72

departments other than CPS and adaptation to the changes. There were departments that were very resistant and upper management let us down. When our small tests of change were resisted, it left us no where to go.

• Lack of Family Partners, lack of DES support and cooperation. • Inability in extranet/computer services. • I will always go back to management. Lack of support from the. They don’t’ care and they

want you to feel the same. • Lack of help from DES people who were not on the Team. • We did encounter the lack of funding to keep a good PDSA in the system which did speed

up the process for our area families to achieve assistance in an area at a faster pace. • We live in a rural community and some of us had to travel far and had other commitments

to attend to so teleconferencing could be a solution offered but it wasn’t. • Resistance. • We had a lot of problems with the extranet. We have only been able to get on once or twice.

I felt it was not very user friendly. Also we had a problem with getting everyone to the meetings. We have members in the Springerville/Eagar area, St. Johns area, Concho, and Flagstaff. Even for the closest areas it was a 60 mile round trip drive to get to meetings. A lot more for the Flagstaff pe…

• Like I said we just seemed to lose contact down here in Willcox. • Lack of time / perceived value relative to competing issues in the office among

management/decision-makers in Community Partner agencies for this project. • I find the people of Parker really do not want to put forth the effort to create their own

healthy lives, much less Partner with DES. For the majority of La Paz County's people, they have been trained to be helpless; they are either drug or alcohol soaked and are not caring very much about themselves or their children.

• Again major hurdles not enough Family Partners participating and that is what this is all about. Other than what I mentioned above is trying to make it on a more familiar home front so families are not intimidated with professional staff from both DES and Agencies. Some Family Partners felt uncomfortable and out of place at the DES office and other agency offices: Maybe at a more familiar …

• Several Team members came with their own very specific agendas and had no time or consideration for ideas of others.

• No one was in a group. One or two people went and did whatever without others aware. No place for parents to be or do--mostly DES workers among themselves. Meetings cancelled a lot.

• DES Partners rejected suggestions from community and Family Partners, and would not cooperate in exploring possible options. I believe all but one of our PSDAs that were acted upon were suggestions from DES Partners.

• To do service integration effectively, time and commitments had to be included in the process for the Team members. Instead it was just more of a “do this along with all your other work” position by our respective DES divisions. Service integration cannot be done 1/32 of the time during a work week. It needed to be the focus of the Partners.

• Not a clear commitment to achieving project goals by local DES and community resources staff. It did not seem that the project took hold and was more or less a prescribed exercise.

• I couldn't just go to certain things there was an email like the week before and I need at least

73

two weeks to make sure I had no clients. • There is one: DES members usually end up being the ones who know it all, therefore, do

most of the talking. I have learned a lot from them by listening to them, which I think is the good part.

• If the top wanted more integration of services, they would change the internal structure of DES.

• None worth mentioning.

74

APPENDIX G

TEAM ACCOMPLISHMENTS THAT COULD NOT HAVE HAPPENED WITHOUT BSCSI

(OPEN ENDED RESPONSES)

75

Team Accomplishments That Could Not Have Happened Without BSCSI – Open-Ended Responses Source: Question 25 of the BSCSI Self-Assessment Questionnaire (Appendix A). Note: Responses noted with an ellipsis indicate the respondent’s response exceeded the character space limits allotted for the survey question. • Improving the utilization, content and distribution of the primary community resource

guide. The Big Kids Book such that at, this time, one of our members was asked to join the volunteers who put out the book annually.

• We’ve made accomplishments that we probably would of not if not for the ideas of our Family Partners expressing their input.

• Co-location of AZ families first in the CPS office. • Increased knowledge of services offered within DES divisions. • Contact information within the different agencies on the Team. • Each Department working together to get cases done quicker and efficiently. Especially the

cases that were out-of-town that couldn't make the trip in. • Creating Family Folders for applicants new to DES which makes the entire application

process a less complicated and more speedy. • Working together, if not, things would not get done. • First, we learned from each other. We shared our successes, supported each other when we

felt failure. Second, we all learned about the strengths, weakness and limitations each of had. This help build a stronger Team. We…

• We got the children's area almost done in the Coolidge office. Will be putting up the TV and vcr/dvd player. The one goal we wanted was to get a receptionist here in the office to direct customers on where to go but due to f…

• Adjusting work schedule to accommodate IV-A Stanby's being seen quickly. Informing the tribes about more resources provided by the state and county.

• The list of things people need to bring to their appointment with DES. • Got a children's area established in the lobby, books for the kids to read with their parents. • Avondale Childcare unit was able to obtain access to FAA's view center system. This help

us view the information our clients have provided to the FAA programs and use it for our program instead of requesting the same informa…

• Our completed children's area for the Coolidge DES office - we have a wall mural, children's play table, bookcase and will soon have a TV to play children's videos. This helps many families as they come in to apply for servi…

• Job announcements & related employment information and scheduled events going out to all Team members and numerous Community Partners.

• Painting the FAA area. • DES and community training, appointment list flyer. • Processing of emergency assistance recipient. • Community and Faith Based information table at the Juvenile Court once per week,

Employee recognition - community prize bowl, E-mail notification to DES customers - wishful, Voices of Recovery - six part DES Breakthrough S…

• Survey of needs of people in the community. The outcome resulted needs of

76

education/employment and training. • We asked for handicapped parking and appropriate signs at our target site and although it

took awhile, we got it done. Other people had tried in the past and not gotten anywhere. • Finding out the extent of knowledge the public has of human services in the community. • An example would be, Outreach done by DES and Community Partners on a monthly basis

throughout the Hopi Community which includes, how to sign up for College classes, application process for higher education, service provided…

• Share information with other DES agencies. • Again for me I would say Tucson Access. To be able to share with others in Recovery

personal stories of success is such a huge impact for our families. • I will give you two examples, first the folders of information for families. The test project

that was put into place showed through very positive response, that it helped the clients feel they were valued. Not only did we…

• Employees smoking in front of the DES entrances was intimidating for clients. We were able to end that before the no-smoking ban come into affect. If the BSC Team didn't have the authority to make the changes, and order no …

• Staying sober for over a year and making good contacts thus leading to valuable relationships that will last a lifetime.

• Creating a customer friendly DES lobby. New furniture on order to replace the old, outdated furniture. The Team has put up new brochure/application holders on the walls which has made it easier for customers to access infor…

• Meeting with a LOM where there has been a history of public relation problems with front office staff.

• I'm still working on it. • None; traditional setting for conducting interviews. Went to the Inter-agency office to

conduct the interview. Since the Inter-agency is doing a remodel of their offices and they what to leave one of the (new office spaces) for…

• Our Team did have two informational fairs and they helped bring agencies together so they can serve the families and refer to one another. I also feel that the relationships between the Team DES employees, Community Partners…

• Yes, it has brought DES Partners, Community Partners and Family Partners together to share resources and ideas that otherwise may not have happened.

• Gaining full-time employment. • Goal about prescription drug co-pays was accomplished. • Police Officer for McDowell office. • Cultural training for DES employees. • Getting together a survey. • We produced an informational flyer about how parents can help children with ad/hd who

have issues in school. • We acquired a lobby specialist for the Avondale DES office. • Getting a children's corner in the DES office. • I personally accomplished more knowledge of how the DES system works. • Receptionist. • Flowerbeds in front of the DES office. • Not smoking in the front entrance - garden club - children's corner - implementation of jobs

77

board and lobby specialist. • The mutual communication with Unemployment and FAA agency was a good

accomplishment. • Housing a family by connecting with DES for placement into a Community Partner’s

resource. • Well, we had TANF recipients get a file to organize intake required information and that

went well. • A children's corner was placed in the DES lobby. • All clients are given folders to keep their important information in. This is for all clients not

program specific. • Our PDSAs worked together to improve the level of respect each person receives.

Fundamentally this improves the entire community by fostering the self respect of each member. Think ripples.

• A community survey completed by DES & Community Partners to help us see what people in our community feel they need to become self-sufficient.

• We were able to extend the service integration philosophy and Community Partnerships to agencies and players outside the original FFA test area.

• Informing the community of the resources and services that are available. • Having a person in the lobby of the FAA office to help navigate through the system or

perform triage for people entering the office. • But Team accomplished some goals. (There were no participants in the Team from tribal

social services or the clinic, which might have supported my work/goals more). • Trying to put a one community resource information guide for our families and Community

Partners. • Voices of recovery Series on Access TV. • DES office on Ft. Lowell has had its appearance changed; it is a better place for the people. • We put together some surveys for the community to learn about people's strengths and

weaknesses. • Lobby improvements including a children's area and television with DVD player. • The PDA'S. • A mural in FAA • I was able to speak directly to family members in the lobby to find out how they feel about

DES services. • Survey completed regarding Lobby. • Not only voicing concerns but developing a plan to make those changes. • Change telephone greeting to make friendlier to customers calling for information and or

services.

78

APPENDIX H

ADDITIONAL RESPONDENT COMMENTS

(OPEN ENDED RESPONSES)

79

Other Comments Made by Participants after Completing Questionnaire Source: End of BSCSI Self-Assessment Questionnaire (following Question 36; Appendix A). Note: Responses noted with an ellipsis indicate the respondent’s response exceeded the character space limits allotted for the survey question. • This committee had so many members that belong to other, long established community

committees, so it was basically wasting time duplicating efforts already being taken by other committees. Any baby steps taken were not worth the time and effort expended.

• The BSC program seems to lack ongoing enthusiasm that would assist us in gaining new and engaged members. It hasn't seemed to inspire our group thus far even though our leader is, in my view inspiring that being me. Perhaps I am in error and should take more responsibility for our lack of total excellence and involvement?

• I truly believe we would have had better and faster results if all of DES and the administration were behind us. Sometimes it felt like an uphill battle to accomplish the simplest PDSA.

• I believe the concept/goals and objectives of the BSC are necessary to ensure family successes are encouraged and supported by DES. I would like to see more of the information sharing between DES divisions on the types of services provided as well as sharing customer documents to avoid duplication of efforts.

• I think our Team spent too much time in the beginning trying to recruit and re-recruit family members and Community Partners. That was more of the focus than the PDSAs. By the time we really got going, the pilot was finished. We didn't have much Community Partnership participation. Instead of DES supporting community agencies to fulfill some of the roles to families, it seemed the focus of the support always came back to the DES agencies.

• I'm hoping that you compile all answers not just the positives ones. It seems that in the rural areas DES money could have been used more wisely. When they (DES, DAAC) closed the Adult protective office in a whole county that was oligarchy at its fines. Next time perhaps it would be wise to talk to field worker and on hands workers, community and the clients that will be affected by such decision making. In the meantime clients and staff are suffering.

• I am glad to be a participant in the BSC. Thank you! For allowing me to help our community make a small change and educate them in a lot of resources. Thank you!

• I participated in the jobs program and found it to be very counter productive to actually finding a job. The time that was required of me in terms volunteering, did not allow me enough time to actually look for a job.

• This is a wonderful concept but you need time to work at this and at des that is something we do not have.

• I appreciate the opportunity to work with such a great group of professional people. • I felt the group I was associated with was very ineffective. There was much more time

wasted on chit chat about DES internal matters than on the actual purpose. Also, sitting for 2 hours on monthly phone conferences was irritating and served almost no purpose other than to pat each other on the back or as a forum to be in the spot light. All things discussed

80

could have been presented in a short e-mail. I will not participate in this group again! • I have not participated consistently in the Team. I believe that part of the reason for not

making additional time to participate, was based on my assessment that the objectives were more institutional in nature/dealing with systems issues than I expected. Although this focus is absolutely of great importance to the effort, I was wanting to participate in a more hands on project solution approach that included a larger group of agency providers/change.

• See previous comments. Thanks. • DES members were unwilling to have meetings after work hours and bad-mouthed David

Bernes in front of Family and Community members. We lost all but 1 Community and 1 Family Member due in large part to this behavior of unwillingness. There are 3 people that are go-getters on this Team (they are the only ones that ever wrote PDSAs) and they are the ones that did everything. Several DES members deliberately scheduled meetings during Meetings and then co…

• I would just like to thank everyone involved. It has been a pleasure. • This program will never work until all of DES is brought into the program and it becomes

the way of doing business. If this does not happen, tomorrow, DES will go back to the way we've always done business and BSC will be forgotten and grouped with all other past endeavors of they tried that once, but, it didn't work, like everything else DES tries to do to make changes.

• I personally enjoyed being part of this endeavor! As a fledging project, I do feel this was a very beneficial attempt to help improve all aspects of the One Stop. Improving upon the area’s that have shown needed attention, there does seem to be a great future for this project. Thank you for allowing me to be a part of this Team!

• I believe that this program, and the awareness of service integration, listening to what client need and want, is a big step in the right direction. I am proud to have served on this Team.

• Thank you for extending the BSC another 6 months. • My final thoughts are if DES refuses to find the funding for a lobby specialist for our multi-

service building, then maybe this whole BSC process was all for nothing. I sure hope not! • I hope more Community Partners can be highlighted during the next 6 months. • I think that there are small and big changes that need to be addressed in our community and

when some things like this is being planned. It would be great to check with the Team and see who really wants to be part of this project. I think we had a lot of individuals that were on the Team only because they were put there not because they wanted to be there. If every DES agency just took the time to ask one questions I really feel we could change many lives.

• I'm sorry that we got off to a slow start. Once we learned the scope and purpose of our gathering it was a fun ride. We needed a budget to accomplish our goals. It would have been better to have someone who was not a DES employee to lead the Team, but only from a time standpoint. DES employees are busy. Maybe some one else could have kept us jazzed and ready to accomplish with more time to keep up the momentum. This has been a worthwhile effort and …

• It was an honor to be asked to participate in the DES Breakthrough Collaborative Series. Would be willing and happy to do this again. It has made a difference in our community.

• Thank you! • Community connection is important!

81

• Thank you for letting me be part of this Breakthrough Series and for all you help. • It is wonderful to have the consumer express their views. • Better customer service, one of the things that were really important to our DES office, was

getting a reliable friendly, front desk person. We even wanted to use a 2 person front desk where one is training the other for people skills; this could have been a job for someone who was looking at a receptionist job. What better way to teach confidents. But we were not able to get the position filled due to no agency in the DES having the money or even the jo…

• This BSC has taken so much of my time from work. I do not wish to participate in it after the Summit.

• This survey is extremely hard to read and track across to the correct answer line. You should highlight every other line. Need to put in option of 'somewhat agree or disagree'. On the DES involvement, it is not fair to say DES, it would be better to ask about department involvement because some departments were extremely cooperative while others were rude and not cooperative.

• Please do not continue the BSC Teams. • It has been a great adventure. God bless you all. • I do believe that small tests of change have been easily met in a lot of areas, but in a lot of

areas there are some changes that are to large for the BCS that maybe ought to be acknowledged so families can be self-sufficient faster instead of going downhill as in some cases that I have had the experience of learning about.

• My answers are not very consistent because the matrix of our group has changed. At first, great; then no more than 3 or 4 members were showing up (This went on for about 3 to 4 months). Then we got excellent Family Partners in that supplied …

• I am thankful to be included. • I was really excited about this project, even though the situation was not ideal for me. I am

sorry that some of my comments were negative. I really liked the Team and love the idea of this effort. Congratulations. Please don't hesitate to contact me in the future.

• If the program is to continue, give an estimate of time with duties, i.e. conference calls, meetings, extranet.

• I believe very strongly in service integration and this series. Prevention is better than treatment.

• I really believe that instead of offering the $100 a day for attending the Learning summit it could have been broken down to used to get Family Partners to attend local meetings. It is not good that most of the year some of the Family Partners did not come to the local meetings yet attend the Learning summit to get $100 a day when they have not been involved with the program at all.

• I never could see why you wanted the family member to be a part. Each worker did whatever and talked about it later. Mostly about their selves and what a great job they do. We never worked as a Team, there was always a power struggle and I would show up for meeting and they had been cancelled, or I would sit and listen to them talk about their selves. I did not see any different when I tried to get benefits for my son.

• I would hope that the BSC Project would continue. It seems that DES undertakes these projects that could have real value and impact and then fails to follow through.

• I have not attended this Team is a long time. I have many meetings and committees to attend in my work. I felt that the BSC was a waste of time.

82

• I do not think the project should be extended another 6 months. • I moved from the area and into a new district and was not able to make assessment of

progress as adequately as I would have liked. • I think Team is a very good idea. I think we need to keep going with the group. I think it

would be a good idea to the travel Team to meet like every two months. And let Teams know what kinds of PDA they are working on.

• Survey is extremely hard to read the lines and then track across to try to make sure I answered the questions the way I wanted to, I am not sure I accomplished that. So if I answered the questions 'wrong' that would not give you an accurate reflection of my opinions.

83

APPENDIX I

DATA TABLES

84

Table B1. Importance-Performance ratings of BSCSI Principles Services are provided to families in a complete, accessible and responsible manner. (Successful service integration is more than just locating services in the same place, blending funding from different sources, and making decisions based on common information.)

Strongly disagree % Disagree % Neither % Agree % Strongly

agree % Mean

Initial Final Initial Final Initial Final Initial Final Initial Final Initial Final

As a principle 2.3 2.0 6.3 5.4 4.5 2.0 42.5 43.5 42.5 44.9 4.20 4.26

Is being practiced 8.6 6.1 33.9 29.9 16.7 14.3 31.7 37.4 1.8 6.1 2.83 3.08

Source: Question 3 of the BSCSI Self-Assessment Questionnaire (Appendix A – final distribution). Parents, children, youth, kin, communities and tribes have many strengths, resiliency and natural supports that can help resolve problems that arise.

Strongly disagree % Disagree % Neither % Agree % Strongly

agree % Mean

Initial Final Initial Final Initial Final Initial Final Initial Final Initial Final

As a principle 2.3 2.0 6.3 5.4 4.5 2.0 42.5 43.5 42.5 44.9 4.24 4.26

Is being practiced 8.6 6.1 33.9 29.9 16.7 14.3 31.7 37.4 1.8 6.1 2.98 3.17

Source: Question 4 of the BSCSI Self-Assessment Questionnaire (Appendix A – final distribution). DES works together with families to develop and enhance independent, self-sustaining families and communities.

Strongly disagree % Disagree % Neither % Agree % Strongly

agree % Mean

Initial Final Initial Final Initial Final Initial Final Initial Final Initial Final

As a principle 1.8 0.7 6.3 8.2 8.6 4.1 45.2 46.3 35.7 37.4 4.09 4.15

Is being practiced 7.7 4.8 32.6 28.6 15.8 16.3 31.2 38.1 4.1 8.8 2.91 3.18

Source: Question 5 of the BSCSI Self-Assessment Questionnaire (Appendix A – final distribution).

85

Table B1. (Continued) Programs will emphasize prevention and early intervention services. At-risk children and families are identified early on to prevent later involvement with more complex child welfare and criminal justice agencies.

Strongly disagree % Disagree % Neither % Agree % Strongly

agree % Mean

Initial Final Initial Final Initial Final Initial Final Initial Final Initial Final

As a principle 1.8 0.7 5.9 3.4 9.0 9.5 40.3 43.5 41.2 39.5 4.15 4.22

Is being practiced 10.0 6.1 33.0 25.2 22.2 17.0 22.2 42.2 5.0

5.4

2.78 3.16

Source: Question 6 of the BSCSI Self-Assessment Questionnaire (Appendix A – final distribution). . Working with Community Partners is necessary to determine and fill gaps in the DES service delivery system. Collaborative efforts improve our ability to meet the needs of children and families.

Strongly disagree % Disagree % Neither % Agree % Strongly

agree % Mean

Initial Final Initial Final Initial Final Initial Final Initial Final Initial Final

As a principle 0.5 0.7 0.5 1.4 0.9 3.4 31.2 29.3 65.6 62.6 4.63 4.56

Is being practiced 5.9 2.7 21.3 19.7 17.6 15.0 35.7 44.2 11.3 15.6 3.27 3.52

Source: Question 7 of the BSCSI Self-Assessment Questionnaire (Appendix A– final distribution). All families have something to contribute to the good of the community. Improved outcomes for children and families occur when families, community groups, faith-based groups, non-profit organizations, businesses and government work together with a common goal.

Strongly disagree % Disagree % Neither % Agree % Strongly

agree % Mean

Initial Final Initial Final Initial Final Initial Final Initial Final Initial Final

As a principle 0.5 0.7 0.9 0.7 1.4 1.4 33.0 38.1 62.4 56.5 4.60 4.53

Is being practiced 4.1 4.8 28.1 23.1 20.8 19.0 29.0 40.1 9.5 10.2 3.12 3.29

Source: Question 8 of the BSCSI Self-Assessment Questionnaire (Appendix A – final distribution).

86

Table B2. Importance-Performance Ratings of BSCSI Component 1 (Intake/Assessment) Importance of practices that: Not

Important % Slightly

Important % Important % Very Important %

Extremely Important % Mean

Initial Final Initial Final Initial Final Initial Final Initial Final Initial Final

Promote sharing of data and documents collected across DES programs

0.9 0.7 1.8 0.7 8.6 8.2 21.3 24.5 65.6 63.3 4.52 4.53

Encourage agency and service providers to recognize and understand the family's needs within the context of the family's culture

1.4 0.0 3.2 0.7 10.9 15.6 27.6 29.3 54.3 53.1 4.35 4.37

Gather appropriate information during the initial contact with families in a uniform manner

1.4 0.7 0.9 2.0 14.0 12.9 34.8 30.6 47.5 52.4 4.28 4.34

Gather appropriate information during the assessment process in a uniform manner

0.9 0.7 1.4 0.7 12.2 14.3 36.7 34.7 47.1 48.3 4.31 4.31

Engage families in a holistic manner during the intake and assessment process

2.7 1.4 2.7 1.4 17.6 15.6 27.6 29.9 45.2 48.3 4.15 4.27

Standardize intake and assessment documents across programs and service providers

1.8 2.0 3.6 7.5 17.6 13.6 29.4 33.3 44.3 42.2 4.15 4.08

Source: Question 9(a) of the BSCSI Self-Assessment Questionnaire (Appendix A – final distribution). . Implementation of practices that:

Not Important %

Slightly Important % Important % Very

Important % Extremely

Important % Mean

Initial Final Initial Final Initial Final Initial Final Initial Final Initial Final

Gather appropriate information during the initial contact with families in a uniform manner

1.8 2.0 12.7 17.0 10.4 12.9 43.4 41.5 29.0 25.2 3.87 3.72

Gather appropriate information during the assessment process in a uniform manner

1.8 1.4 14.0 17.7 11.3 15.0 43.9 38.8 25.8 24.5 3.80 3.69

Encourage agency and service providers to recognize and understand the family's needs within the context of the family's culture

5.4 2.0 17.2 19.7 15.4 13.6 27.1 38.1 30.8 22.4 3.63 3.62

Engage families in a holistic manner during the intake and assessment process

8.6 3.4 15.4 22.4 18.1 17.7 27.6 33.3 24.0 19.0 3.46 3.44

Standardize intake and assessment documents across programs and service providers

7.7 8.8 19.5 20.4 11.3 15.0 30.3 32.7 26.2 21.1 3.50 3.38

Promote sharing of data and documents collected across DES programs

8.1 12.2 17.6 21.1 8.6 8.2 24.9 29.9 37.1 25.9 3.67 3.37

Source: Question 9(b) of the BSCSI Self-Assessment Questionnaire (Appendix A – final distribution ).

87

Table B3. Importance-Performance Ratings of BSCSI Component 2 (Coordination/Integration) Importance of practices that: Not

Important % Slightly

Important % Important % Very Important %

Extremely Important % Mean

Initial Final Initial Final Initial Final Initial Final Initial Final Initial Final

Coordinate services by integrating the process and people involved

0.9 0.0 0.0 0.0 7.2 12.9 39.8 32.7 50.2 52.4 4.41 4.40

Have all DES programs connecting families to community resources

0.9 0.7 0.5 0.7 15.8 14.3 27.1 27.2 52.5 53.7 4.35 4.37

Demonstrate that all DES programs connect families to community resources

0.9 0.0 0.9 0.7 12.7 17.7 33.0 28.6 49.8 50.3 4.34 4.32

Support a single case review/plan for families who receive multiple services from DES and the community

2.3 1.4 2.7 2.7 11.3 17.7 30.8 27.2 50.7 48.3 4.28 4.22

Support co-locating DES and Community Partners in the same place to improve communication, planning and service delivery

2.3 6.8 4.5 2.7 16.3 16.3 26.2 23.8 48.0 48.3 4.17 4.06

Source: Question 10a of the BSCSI Self-Assessment Questionnaire (Appendix A – final distribution). Implementation of practices that:

Not Important %

Slightly Important % Important % Very

Important % Extremely

Important % Mean

Initial Final Initial Final Initial Final Initial Final Initial Final Initial Final

Coordinate services by integrating the process and people involved

2.7 2.7 19.9 23.8 11.3 13.6 36.7 34.0 28.1 24.5 3.68 3.54

Have all DES programs connecting families to community resources

4.5 3.4 16.7 23.8 13.1 14.3 28.1 33.3 33.5 23.1 3.72 3.50

Demonstrate that all DES programs connect families to community resources

4.5 4.8 17.6 22.4 13.1 13.6 32.6 36.1 29.9 21.1 3.67 3.47

Support co-locating DES and Community Partners in the same place to improve communication, planning and service delivery

9.0 8.2 16.7 19.0 13.6 14.3 26.2 33.3 29.0 22.4 3.52 3.44

Support a single case review/plan for families who receive multiple services from DES and the community

10.9 8.8 19.0 23.1 12.2 15.6 25.3 25.9 29.9 24.5 3.45 3.35

Source: Question 10(b) of the BSCSI Self-Assessment Questionnaire (Appendix A – final distribution).

88

Table B4. Importance-Performance Ratings for BSCSI Component 3 (Accessibility/Accountability) Importance of practices that: Not

Important % Slightly

Important % Important % Very Important %

Extremely Important % Mean

Initial Final Initial Final Initial Final Initial Final Initial Final Initial Final

Ensure services that support families who are identified as at-risk

0.9 0.0 0.9 0.7 10.0 14.3 30.8 27.2 53.8 55.1 4.42 4.41

Create effective ways of sharing information about services and resources available through DES and other community organizations

0.9 0.0 0.0 0.0 8.1 17.7 25.8 23.8 60.2 55.1 4.53 4.39

Build on existing relationships of families as well as extended families

1.8 0.0 1.4 2.0 15.8 15.0 31.2 30.6 47.1 50.3 4.24 4.32

Ensure services are available to families that deal with all of their needs

1.8 0.0 0.0 2.0 12.2 17.0 32.6 29.3 50.7 48.3 4.35 4.28

Promote services that recognize and include the diversity and strengths of families and their extended community

0.9 0.0 0.5 0.7 14.9 18.4 27.1 33.3 53.8 44.9 4.37 4.26

Promote systems that include the “family voice” in designing and implementing services

0.9 0.0 1.4 0.0 18.6 20.4 27.6 33.3 49.3 44.2 4.27 4.24

Promote systems that include the "family voice" in assessing the quality and usefulness of services

0.9 0.0 1.8 2.0 16.3 18.4 27.1 33.3 51.6 44.2 4.30 4.22

Source: Question 1l(a) of the BSCSI Self-Assessment Questionnaire (Appendix A – final distribution).

89

Table B4. (Continued) Implementation of practices that:

Not Important %

Slightly Important % Important % Very

Important % Extremely

Important % Mean

Initial Final Initial Final Initial Final Initial Final Initial Final Initial Final

Ensure services that support families who are identified as at-risk

3.6 4.8 15.4 13.6 13.6 17.0 33.9 36.1 28.5 25.2 3.72 3.65

Promote services that recognize and include the diversity and strengths of families and their extended community

4.1 4.1 15.8 16.3 17.2 16.3 30.3 38.1 28.1 22.4 3.65 3.60

Build on existing relationships of families as well as extended families

3.6 4.8 15.8 15.0 20.4 19.7 27.6 33.3 28.5 24.5 3.64 3.59

Ensure services are available to families that deal with all of their needs

5.4 5.4 16.3 16.3 12.7 14.3 30.8 38.8 30.3 21.8 3.67 3.57

Promote systems that include the “family voice” in designing and implementing services

3.6 4.8 17.2 19.0 15.4 11.6 34.8 42.2 25.8 19.7 3.64 3.55

Promote systems that include the "family voice" in assessing the quality and usefulness of services

3.2 4.8 19.9 19.7 14.5 13.6 34.4 38.1 24.9 20.4 3.60 3.51

Create effective ways of sharing information about services and resources available through DES and other community organizations

5.4 7.5 15.8 20.4 12.7 12.9 28.5 29.3 32.6 25.9 3.70 3.48

Source: Question 1l(b) of the BSCSI Self-Assessment Questionnaire (Appendix A – final distribution).

90

Table B5. Importance-Performance Ratings of BSCSI Component 4 (Prevention/Intervention) Importance of practices that: Not

Important % Slightly

Important % Important % Very Important %

Extremely Important % Mean

Initial Final Initial Final Initial Final Initial Final Initial Final Initial Final

Coordinate funding for innovative solutions to prevent crisis

0.9 0.0 0.9 0.0 12.2 14.3 29.9 27.2 52.9 56.5 4.38 4.43

Promote Partnerships with agencies that focus on prevention and early intervention

0.9 0.0 0.5 0.0 12.7 13.6 24.4 31.3 58.8 53.1 4.44 4.40

Ensure services provide for the needs of families before problems arise as well as immediately after problems arise

0.9 0.0 0.9 0.7 14.5 15.6 29.9 27.9 51.1 53.7 4.34 4.38

Allow the “family voice” to be heard when planning prevention and early-intervention services

0.9 0.0 0.5 0.7 19.5 16.3 29.9 30.6 46.2 51.0 4.24 4.34

Allow access to and use of information to predict family and community needs

1.8 0.0 3.2 4.1 23.5 22.4 31.7 29.9 37.6 42.2 4.03 4.12

Allow the community to define the quality of services 1.4 1.4 6.3 6.8 19.5 25.2 31.7 28.6 36.2 36.1 4.00 3.93

Source: Question 12(a) of the BSCSI Self-Assessment Questionnaire (Appendix A - final distribution).

91

Table B5. (Continued) Implementation of practices that:

Not Important %

Slightly Important% Important% Very

Important% Extremely

Important% Mean

Initial Final Initial Final Initial Final Initial Final Initial Final Initial Final

Promote Partnerships with agencies that focus on prevention and early intervention

4.1 2.0 14.9 14.3 10.0 18.4 36.7 38.1 29.9 24.5 3.77 3.71

Allow the “family voice” to be heard when planning prevention and early-intervention services

3.6 3.4 18.1 19.7 14.9 15.6 33.0 37.4 26.2 22.4 3.63 3.57

Ensure services provide for the needs of families before problems arise as well as immediately after problems arise

6.3 6.8 17.2 19.7 12.2 14.3 29.9 32.7 30.3 24.5 3.63 3.49

Allow access to and use of information to predict family and community needs

4.5 3.4 19.9 22.4 13.1 15.0 33.5 40.8 25.8 17.0 3.58 3.46

Coordinate funding for innovative solutions to prevent crisis

9.0 12.2 19.0 18.4 9.5 15.6 29.4 28.6 29.4 23.8 3.53 3.34

Allow the community to define the quality of services

8.1 8.8 17.6 18.4 16.7 21.1 29.9 31.3 23.1 17.0 3.44 3.30

Source: Question 12(b) of the BSCSI Self-Assessment Questionnaire (Appendix A – final distribution).

92

Table B6. Achievement of BSCSI Desired Outcomes. Self-Sufficiency We have helped families gain self-sufficiency in these ways:

Strongly disagree

%

Disagree %

Neither %

Agree %

Strongly agree

% Mean

Have knowledge of resources in the community 1.4 4.1 16.3 48.3 27.9 3.99 Envision their potential to be self-sufficient 2.0 9.5 23.1 43.5 19.7 3.71 Know and recognize their needs 2.0 7.5 21.8 47.6 19.0 3.76 Know their assets and abilities 2.0 8.8 20.4 48.3 18.4 3.74 Prioritize their assets and abilities 2.0 10.9 25.2 42.9 17.0 3.63 Prioritize their needs 2.0 10.2 25.2 43.5 16.3 3.64 Be proficient in accessing resources (e.g., money, people, services) 2.7 10.2 17.7 50.3 16.3 3.69 Be inspired to move beyond reliance on social services 2.7 11.6 27.9 38.8 16.3 3.56 Be innovative in overcoming barriers 2.0 8.8 23.1 48.3 15.6 3.68 Move toward their greatest potential 2.0 13.6 25.2 41.5 15.6 3.56 Have sufficient resources 3.4 15.6 26.5 39.5 12.9 3.44 Make things happen for themselves 2.0 12.9 25.9 43.5 12.9 3.54 Source: Question 13 of the BSCSI Self-Assessment Questionnaire (Appendix A – final distribution).

93

Table B6. (Continued) Strengthening Families We have helped our families strengthen their ability to:

Strongly disagree

%

Disagree %

Neither %

Agree %

Strongly agree

% Mean

Increase connections between families and outside support networks 2.0 7.5 21.8 48.3 17.0 3.73 Become more socially connected (e.g., to neighborhood, other families, etc.)

2.7 10.9 28.6 39.5 15.0 3.55

Become more engaged in their communities (e.g., neighborhood watch, events, volunteering, etc.)

2.7 13.6 25.9 39.5 15.0 3.52

Create a safe environment for all family members 3.4 10.2 21.8 50.3 12.2 3.59 Develop positive communication among family members 3.4 10.2 29.3 42.9 11.6 3.50 Increase appreciation, affection and nurturing among family members 3.4 10.2 32.0 40.8 10.9 3.47 Increase commitment between family members 2.7 12.2 28.6 44.9 9.5 3.47 Endure through hardship 2.7 8.8 31.3 44.9 8.8 3.50 Adapt to hardship 2.7 8.2 32.7 46.3 8.2 3.50 Increase spiritual (not necessarily religious) well-being within the family

5.4 10.2 41.5 33.3 7.5 3.28

Spend more time together 3.4 9.5 38.8 38.8 6.8 3.37 Source: Question 14 of the BSCSI Self-Assessment Questionnaire (Appendix A – final distribution).

94

Table B6. (Continued) Building Capacity of Extended Families and Communities

We have helped our families by: Strongly disagree

%

Disagree %

Neither %

Agree %

Strongly agree

% Mean

Increasing connections to resources 4.8 8.2 23.1 42.2 16.3 3.60 Increasing communication to ensure that resources are known and shared throughout the community

3.4 7.5 21.8 49.0 15.6 3.68

Increasing the ability to engage in Partnerships between families, Community Partners, and governments in working toward common goals

4.8 9.5 25.2 41.5 15.6 3.56

Increasing services that are accessible 4.8 7.5 20.4 51.0 12.9 3.62 Increasing the ability to be innovative in solutions 6.1 8.2 26.5 42.9 12.9 3.50 Increasing opportunities for families to receive services and support in their own communities

4.8 6.8 25.9 46.9 12.2 3.57

Increasing efficiency of services 4.8 8.2 23.1 48.3 11.6 3.56 Increasing timeliness of services 4.8 10.9 22.4 48.3 10.2 3.50 Increasing services that are coordinated (not fragmented) 5.4 7.5 25.9 46.9 10.2 3.51 Increasing the consideration of local strengths and needs in designing and delivering services

4.8 10.2 25.2 46.3 10.2 3.49

Increasing knowledge and building the assets of families and the communities in which they live

6.1 8.8 26.5 44.9 10.2 3.46

Increasing the ability to engage families in systems and solutions 6.1 8.8 26.5 43.5 10.2 3.45 Increasing “wrap around” services (e.g., smooth transition between services)

6.8 8.2 31.3 40.1 10.2 3.40

Increasing culturally sound services 4.8 10.9 38.1 32.0 9.5 3.32 Increase knowledge and awareness of prevention and early intervention services

5.4 9.5 31.3 42.2 7.5 3.38

Safely increasing the number of families supported by extended family members

5.4 11.6 32.7 40.8 5.4 3.30

Safely decreasing the number of children in institutional placements (e.g., group homes, shelters)

6.8 14.3 40.8 29.3 5.4 3.13

Safely decreasing the number of adult family members in institutional placements (e.g., group homes, shltrs)

6.1 14.3 44.9 25.9 5.4 3.11

Source: Question 15 of the BSCSI Self-Assessment Questionnaire (Appendix A – final distribution).

95

Table B7. Perceptions of Team Characteristics

Team characteristics: Strongly disagree

%

Disagree %

Neither %

Agree %

Strongly agree

% Mean

Diverse knowledge and experience represented on the Team 0.7 1.4 4.8 52.4 40.1 4.31 Good relationships among Team members 2.0 2.0 8.8 51.7 34.7 4.16 Having fun during meetings 2.0 4.8 15.0 46.3 31.3 4.01 Maintaining flexibility during the process 3.4 4.8 13.6 50.3 27.2 3.94 Strong participation by Community Partners 4.1 16.3 11.6 40.8 26.5 3.70 Commitment to the philosophy of the program 2.0 4.1 17.0 49.0 26.5 3.95 Demonstrating innovation 2.7 6.8 16.3 45.6 26.5 3.88 Strong leadership for the Team 0.7 11.6 22.4 38.1 25.9 3.78 Strong participation by Family Partners 17.7 22.4 16.3 27.2 15.6 3.01

Source: Question 18 of the BSCSI Self-Assessment Questionnaire (Appendix A – final distribution). Table B8. Actions that Would Have Helped the Team Achieve Success

Things that could have helped: Not helpful %

Slightly helpful

%

Moderately helpful

%

Very helpful

% Mean

Increased knowledge about this project relative to the larger DES system 1.4 7.5 33.3 56.5 3.47

Provide incentives for Family Partner participation 1.4 14.3 24.5 57.8 3.42 Increased clarity of the project and process 2.0 9.5 35.4 51.7 3.39 Support from DES for engaging more Family Partners 4.8 10.9 27.2 55.1 3.35

More acknowledgment and encouragement for the Teams 2.7 19.0 32.0 43.5 3.20

Source: Question 19 of the BSCSI Self-Assessment Questionnaire (Appendix A – final distribution).

96

97

Table B9. Barriers to Achieving Team Outcomes

Barriers: Strongly disagree

%

Disagree %

Neither %

Agree %

Strongly agree

% Mean

Difficulty engaging Family Partners 0.7 10.2 12.9 25.2 49.7 4.14 Lack of knowledge about the project by the larger DES community 1.4 13.6 14.3 32.7 36.1 3.90 People moving in and out of the Team 1.4 8.2 12.9 42.2 32.0 3.99 Lack of authority to implement actions 5.4 13.6 21.8 25.9 32.0 3.66 Lack of resources needed to implement actions 2.7 15.0 20.4 32.7 27.2 3.68 Extranet/computer problems 3.4 16.3 27.9 22.4 27.2 3.55 Took a while to get started 1.4 19.7 15.6 36.1 25.9 3.66 Overly complex process 2.7 21.8 25.9 28.6 19.7 3.41 Establishing Team cohesiveness took some time 1.4 21.1 14.3 42.9 19.0 3.58 Confusion about the purpose of the project 4.8 21.8 20.4 34.7 17.0 3.38 Not feeling that the Team was valued 8.8 27.2 29.3 20.4 12.9 3.01 Confusion about who was in charge of the Team 10.9 34.0 20.4 20.4 12.9 2.90

Source: Question 20 of the BSCSI Self-Assessment Questionnaire (Appendix A – final distribution).