Evaluation Reportuccfiles.com/pdf/Mission1EvaluationReport.pdf · An online survey was created...

30
Evaluation Report March 1, 2012 A Report Commissioned by the UCC Collegium of Officers Submitted By: Rev. Kristina Lizardy-Hajbi, Ph.D. Minister for Christian Faith Formation Research Congregational Vitality and Discipleship, Local Church Ministries

Transcript of Evaluation Reportuccfiles.com/pdf/Mission1EvaluationReport.pdf · An online survey was created...

Page 1: Evaluation Reportuccfiles.com/pdf/Mission1EvaluationReport.pdf · An online survey was created using Polldaddy (an online survey service) and reviewed by several key national staff

Evaluation Report

March 1, 2012

A Report Commissioned by the UCC Collegium of Officers

Submitted By:

Rev. Kristina Lizardy-Hajbi, Ph.D.

Minister for Christian Faith Formation Research

Congregational Vitality and Discipleship, Local Church Ministries

Page 2: Evaluation Reportuccfiles.com/pdf/Mission1EvaluationReport.pdf · An online survey was created using Polldaddy (an online survey service) and reviewed by several key national staff

2 | P a g e

Contents Introduction and Overview 3

Research Methodology 4

Online Survey of Individuals/Congregations 5

Targeted Response Surveys 5

Online Survey of National Staff 7

Descriptive Data Collection 7

Data Analysis 7

Results 8

Online Survey of Individuals/Congregations 8

Targeted Response Surveys 14

Online Survey of National Staff 16

Conclusions 20

Recommendations 22

Appendix A: Mission: 1 Evaluation Plan 24

Appendix B: Participating Churches Survey Request 26

Appendix C: Sample Participating Churches 27

Appendix D: Non-Participating Churches Survey Request 28

Appendix E: Sample Non-Participating Churches 29

Page 3: Evaluation Reportuccfiles.com/pdf/Mission1EvaluationReport.pdf · An online survey was created using Polldaddy (an online survey service) and reviewed by several key national staff

3 | P a g e

Introduction and Overview Mission: 1 was created and implemented as “one united church on a shared

mission for 11 powerful days to feed the hungry and confront food-related

injustice.”1

Specifically, the online Mission: 1 Campaign Manual stated, “For the first 11 days

of November (11.1.11 - 11.11.11), United Church of Christ congregations,

colleges, seminaries, and health and human service agencies, will join together in a

collective mission campaign to collect more than 1 million food items for local

food banks, raise $111,111 in online donations for Neighbors in Need and

$111,111 for East Africa famine relief, and write 11,111 letters to Congress (in

support of Bread for the World’s 2011 Offering of Letters campaign) asking that

U.S. foreign assistance be reformed to more effectively serve the world’s poorest

people.” Congregations were also encouraged to creatively engage in local mission

projects, service opportunities, and educational events, making Mission: 1 “come

alive” in various local contexts.

Another stated hope for the project was that it would prompt “a time for intentional

theological reflection on our oneness as a church” and urged congregations and

other organizations to make “oneness” the central theme that undergirds the

worship, education, and prayer lives of its members.

Therefore, taking the above description and hopes into consideration, the main

purposes of this evaluation are to:

1. determine the overall impact of Mission: 1 on individuals, churches, and

other organizations that participated in the event, as well as on staff in the

national setting;

2. assess the degree to which the non-quantifiable hopes and goals for the

project were met; and

3. offer recommendations for future events based on the feedback received

from this assessment.

Overall, the main research question is: What was the impact of Mission: 1 beyond

its intended measurable outcomes?

1 This and subsequent quotes were taken from http://www.ucc.org/mission1 webpages and resource materials.

Page 4: Evaluation Reportuccfiles.com/pdf/Mission1EvaluationReport.pdf · An online survey was created using Polldaddy (an online survey service) and reviewed by several key national staff

4 | P a g e

Out of this central question, several specific questions guided the evaluation itself

and included:

- How did Mission: 1 impact individuals and congregations (their awareness

and commitment to food and hunger issues, sense of community,

faith/spiritual formation, sense of purpose, and relationships with local and

global entities)?

- What factors made Mission: 1 so successful and widespread across the life

of the church?

- How did Mission: 1 inform people’s understandings of what it means to be

part of a denomination?

- What impact did Mission: 1 have on staff in the national setting?

- Is Mission: 1 (or something similar to Mission: 1) replicable?

- Why did some congregations not participate in Mission: 1?

Research Methodology

To gain insight into the impact of Mission: 1 as a whole, a mixed methods

research approach was applied to this study. Mixed methods utilize both

quantitative (numbers-based) and qualitative (text-based) data. Numbers create an

overall sense of trends; and words in the form of stories, comments, opinions, and

other feedback offer a deeper understanding of those trends and experiences.

One specific type of research method that will be particularly crucial for this

evaluation is the use of a phenomenological approach. Phenomenological

approaches center on the interpretation of a particular event or experience, called

the phenomenon, shared by a group of people. Mission: 1, in this evaluation, serves

as that phenomenon by which the impacts are assessed and measured.

To respond to the research questions, a number of data collection methods and

strategies were utilized and included the following:

- Online surveys of individuals/congregations that participated in Mission: 1

- Targeted response surveys with a random purposeful sample of

congregations who participated in Mission: 1

- Targeted response surveys with a cluster sample of congregations who did

not participate in Mission: 1

- Targeted online survey of staff in the national setting

Page 5: Evaluation Reportuccfiles.com/pdf/Mission1EvaluationReport.pdf · An online survey was created using Polldaddy (an online survey service) and reviewed by several key national staff

5 | P a g e

- Descriptive data collection from comment boxes on Mission: 1 reporting and

donation forms (online and paper)

- Reviews of Mission: 1 website, including all documents, related news

features on the RSS feed, and submitted videos/photographs

Online Survey of Individuals/Congregations

An online survey was created using Polldaddy (an online survey service) and

reviewed by several key national staff members (Ben Guess, Mark Clark, Ann

Poston, Don Hill, Daniel Hazard, and Barb Powell) for editing prior to finalization.

The survey included multiple choice, matrix, and open-ended questions and was

designed for individuals, congregations, and other organizations who participated

in Mission: 1.

The survey was conducted from January 23 to February 15, 2012 for a total period

of three and a half weeks. To collect a sufficient amount of data, information about

the survey was advertised through the following formats and venues:

- Direct email announcements to all individuals who submitted letters, food

totals, and funds as part of Mission: 1 (total of 3,110 email addresses):

January 23 and February 10

- Announcements and links placed in Keeping You e-Posted (KYeP): January

24 and February 7

- Announcement and link prominently located on main page of www.ucc.org

- Announcement and link posted on UCC Facebook and Twitter pages

Targeted Response Surveys

Participant Congregations Survey

Two targeted response surveys were conducted with smaller samples of

congregations. The first response survey was conducted with congregations who

participated in Mission: 1 in order to gather similar feedback using a different

method (a practice known as data triangulation). This helps to ensure data

reliability (that the results are consistent).

For this survey, pastors of congregations that were featured on the Mission: 1

website—either in a news article or in a video/photograph—were sent emails

asking them to respond to two multiple-choice questions, with the opportunity to

provide comments in a third open-response question (see Appendix B). The

Page 6: Evaluation Reportuccfiles.com/pdf/Mission1EvaluationReport.pdf · An online survey was created using Polldaddy (an online survey service) and reviewed by several key national staff

6 | P a g e

questions were derived directly from the online survey in order to maintain

consistency and accuracy with regard to the reliability of data.

A random purposeful sampling method was utilized for this survey because: (a) it

provided a relatively diverse group of congregations in terms of geography and

church size; and (b) it added credibility since the potential sample of congregations

was too large for this kind of survey method.

Email addresses were obtained from the UCC 2011 Yearbook and various church

websites. The total number of churches in this sample was 41 (see Appendix C for

the complete list).

Two email requests were sent to pastors of the 41 congregations on February 2 and

10. In the event that a pastor’s direct email address was not located, the survey

request was sent to the church’s email address. Invalid email addresses that

bounced back to the sender were resent to church office email addresses.

Non-Participant Congregations Survey

The second targeted response survey was conducted with congregations who did

not participate in Mission: 1. This survey was performed in order to learn some of

the reasons why churches did not take part in this denominational initiative.

For this survey, congregations were identified based solely on their location in one

specific geographic area. Since it was difficult to determine non-participation

across the denomination, the researcher chose a particular conference (Rocky

Mountain Conference) that she was familiar with and compared a list of those

churches to the online participant list obtained through the Mission: 1 website to

determine which churches did not submit an online report.

This type of sampling method is known as cluster sampling and was utilized

because: (a) it provided a convenient sample; and (b) it offered enough

randomization that results could be generalized to the larger population of UCC

non-participant churches.

Pastors of the identified congregations were sent emails asking them to respond to

one multiple-choice question, with the opportunity to provide open-response

comments (see Appendix D). Email addresses were obtained from the UCC 2011

Yearbook and various church websites. The total number of churches in this

sample was 41 (see Appendix E for the complete list).

Page 7: Evaluation Reportuccfiles.com/pdf/Mission1EvaluationReport.pdf · An online survey was created using Polldaddy (an online survey service) and reviewed by several key national staff

7 | P a g e

Two email requests were sent to pastors of the 41 congregations on January 24 and

30. In the event that a pastor’s direct email address was not located, the survey

request was sent to the church’s email address. Invalid email addresses that

bounced back to the sender were resent to church office email addresses.

Online Survey of National Staff

In addition to assessing the impact of Mission: 1 on individuals and congregations

that participated, an assessment of the impact on individuals in the national

setting—those who directly or indirectly facilitated and supported Mission: 1—was

conducted as a complement to the larger survey and assisted in answering the

question of whether or not such an endeavor would be replicable in the future.

So, an online survey was created using Polldaddy and was designed specifically for

staff in the national setting to assess their participation, overall opinions of the

planning and execution process, and other various themes (see Appendix G for full

survey). The survey was sent to all Covenanted Ministries, CHHSM, Pension

Board, Insurance Board, and Cornerstone staff. The survey was announced via

email on February 13, with a reminder sent on February 21.

Descriptive Data Collection

Descriptive data submitted in the comment boxes on Mission: 1 reporting and

donation forms (both online and on paper) were reviewed as part of the overall

assessment regarding the impact of the project and were integrated into the

conclusions and recommendations of the evaluation.

Data Analysis

Descriptive data from surveys were reviewed and categorized into themes for the

report. Quantitative data gathered from online surveys was mainly analyzed within

Polldaddy; and higher-order analyses were conducted using PSPP, a specialized

quantitative data analysis software program.

Page 8: Evaluation Reportuccfiles.com/pdf/Mission1EvaluationReport.pdf · An online survey was created using Polldaddy (an online survey service) and reviewed by several key national staff

8 | P a g e

Results

Online Survey of Individuals/Congregations

Overall, a total of 873 respondents took the online survey, with a 92% completion

rate.

Of the individuals that were sent direct email notices resulting from online/paper

giving, letter writing, and food reports, here are the brief statistics on the response

rates based on the January 23 notice:

Total Recipients: 3,110

Recipient Opens: 1,654 (53.7%)

Recipient Click-Throughs: 659 (21.4%)

Forwarded Opens: 992 (includes auto-forwards)

Email Bounces: 30

Opt-Outs: 4

In terms of statistical surveying, 21.4% indicates a very solid response rate

(assuming the click-throughs resulted in completed surveys).2

Demographics. In the overall results of the survey, the following demographic

statistics were reported:

798 respondents (92%) completed the survey primarily as individuals who

were part of a larger group that participated in Mission: 1, while 74

respondents (8%) completed the survey as individuals who participated

without organizational support.

44% of survey participants identified as authorized ministers, and 56% did

not.

The greatest number of individuals who participated in the survey belonged

to churches located in a larger town or smaller city with a population

between 10,000 and 50,000 (24%), with the next greatest number of

participants belonging to churches located in villages or towns of less than

10,000 (23%). Together, those churches comprised almost half of the total

responses (47%) (see Figure 1).

2 This response rate could possibly be an indication of the effectiveness of the targeting strategies utilized, as well as

survey respondents’ interest in the Mission: 1 project as a whole. However, there is no way to determine a

completely causal link between the targeted emails and the response rate, given that there were other methods for

announcing the survey.

Page 9: Evaluation Reportuccfiles.com/pdf/Mission1EvaluationReport.pdf · An online survey was created using Polldaddy (an online survey service) and reviewed by several key national staff

9 | P a g e

57% of the churches to which survey participants belonged had an average

worship attendance of less than 100 people. 34% of the churches possessed

an average weekly worship attendance between 100 and 299 people (see

Figure 2).

0 5 10 15 20 25 30

N/A

Rural area or open country

Village or town of less than 10,000

Larger town or a small city with a population between

10,000 and 50,000

A suburb around a city with a population of 50,000 or

more

An older residential area in a city with a population of

50,000 or more

A downtown or central area of a city with a population

of 50,000 or more

FIGURE 1: Church Location Type (Percentages)

N/A or I don't know

500 or more

300-499

150-299

100-149

50-99

1-49

0 5 10 15 20 25 30 35 40

FIGURE 2: Average Weekly Worship Attendance

(Percentages)

Page 10: Evaluation Reportuccfiles.com/pdf/Mission1EvaluationReport.pdf · An online survey was created using Polldaddy (an online survey service) and reviewed by several key national staff

10 | P a g e

Activity. When participants were asked to indicate all of the ways Mission: 1 was

carried out by them personally or by their congregations, groups, or organizations,

94% reported that they engaged in food collection, 66% engaged in letter writing,

and 64% engaged in fundraising (see Figure 3).

9% of respondents listed activities in the “Other” category. Most of these

comments expanded upon the options that they selected above, but some

comments described activities such as hosting book discussion groups or other

educational efforts like speakers or forums (for both the congregation and the

wider community), pairing Mission: 1 with initiatives like the Let’s Move

Campaign or school backpack drives, and partnering with local businesses for

donations.

In addition, 53% of survey participants indicated the involvement of children and

youth in their congregations as part of Mission: 1; and 64% of all participants

utilized Mission: 1 congregational resources that were provided on the ucc.org

website.

Impact. Participants were asked to check the three most important ways that

Mission: 1 impacted their congregation or group, as well as the ways they were

personally impacted by the initiative (see Figure 4).

People who participated as individuals reported a greater sense of connection with

others across the UCC (71%) than people who participated as part of a larger group

0

10

20

30

40

50

60

70

80

90

100

Food Collection -

94%

Letter Writing -

66%

Fundraising -

64%

Partnership with

Local or Global

Non-Profit

Agency - 39%

Social

Action/Advocacy

in Local

Community -

29%

Other - 9%

FIGURE 3: Mission: 1 Activity

Page 11: Evaluation Reportuccfiles.com/pdf/Mission1EvaluationReport.pdf · An online survey was created using Polldaddy (an online survey service) and reviewed by several key national staff

11 | P a g e

(59%). Individual participants in Mission: 1 also cited an increased sense of

purpose (34%) and deepened spiritual life/connection with God (27%) at

statistically significantly higher rates when compared with individuals who

participated with a larger group (19% and 9%, respectively).

FIGURE 4: Impact of Mission: 1 (Percentages)

In a separate survey question, participants were asked to rate their level of

connection and affiliation with the wider UCC as a result of Mission: 1. 74% of

respondents indicated they felt “Somewhat More” or “Significantly More”

connection, and 25% felt the “Same” sense of connection and affiliation with the

wider UCC. Less than 1% (only 4 respondents) said they felt “Less” connection.

Primary Goal. 77% of respondents indicated that Mission: 1 was first and

foremost designed to engage the whole UCC in one shared mission with

measurable outcomes, and 19% saw Mission: 1 as deepening UCC churches’

involvement in hunger and poverty ministries (see Figure 5).

0

10

20

30

40

50

60

70

80

Congregation/Group

Individual

Page 12: Evaluation Reportuccfiles.com/pdf/Mission1EvaluationReport.pdf · An online survey was created using Polldaddy (an online survey service) and reviewed by several key national staff

12 | P a g e

Prior Engagement. 97% of respondents indicated that they/their congregation or

group were already engaged in food and hunger ministry prior to Mission: 1.

Conditions for Participation. The three conditions that a majority of survey

participants indicated as “Very Important” in terms of their participation in

Mission: 1 were: (a) it provided an opportunity to be a part of something larger

than themselves (60%); (b) it was something that all members of the church could

do (60%); and (c) there were clear and concrete goals (50%). Other conditions

were ranked as “Important” by a majority of respondents (see Figure 6).

When asked to provide comments on their response choices, participants offered

thoughts that were ultimately categorized into four major topic areas:

Reiteration and endorsement of Mission: 1 as a national and global effort

that tied churches to the wider UCC.

Reiteration and endorsement of Mission: 1 as a flexible initiative, yet it was

one that possessed concrete goals and timelines.

Description of factors that mitigated churches’ own participation such as

competing campaigns (stewardship, mission), poor timing, or general lack of

enthusiasm on the part of members.

Description of issues that churches encountered in the Mission: 1 initiative

such as confusion around Neighbors in Need and East Africa famine relief

funding and various issues with online congregational resources.

FIGURE 5: Primary Goal of Mission: 1

Engage the whole UCC in one

shared mission with

measurable outcomes - 77%

Deepen UCC churches'

involvement in hunger and

poverty ministries - 19%

Utilize online technology to

connect UCC members and

churches - 3%

Page 13: Evaluation Reportuccfiles.com/pdf/Mission1EvaluationReport.pdf · An online survey was created using Polldaddy (an online survey service) and reviewed by several key national staff

13 | P a g e

FIGURE 6: Conditions for Mission: 1 Participation

Future Participation. 89% indicated that they/their congregation or group would

participate in future denomination-wide events similar to Mission: 1. 10% were not

sure, and 1% said they would not participate.

When asked why or why not, participants responded with comments that focused

on two major types of sentiments:

- Positive sentiments: Most respondents expressed the ways in which Mission:

1 was a positive experience for them/their congregations and groups and

again talked about the sense of connection to the wider church while being

able to make a difference locally. A few congregations even chose to make

Mission: 1 an annual event. Several people commented with regard to the

hunger focus of Mission: 1, “This is just the right thing to do” or “This is

what being Christian is about.”

- “It depends” sentiments: Several respondents stated that their participation

would depend—on the topic of another similar effort, on whether the theme

matched their churches’ mission and goals or could be linked with pre-

existing activities (both related to hunger and not related to hunger), or if

new and transitioning church leadership would promote such efforts. A few

comments also mentioned continued struggles to connect their congregation

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Sufficient

advance

notice and

time to

prepare

Clear and

concrete

goals

Set

timeframe

for

participation

Something

that all

members of

the church

could do

Room for

creativity

and

flexibility

depending

on context

Informative,

attractive,

and useful

resources

Provided

opportunity

to be part of

something

larger than

ourselves

Very Important

Important

Somewhat Important

Not Important

Page 14: Evaluation Reportuccfiles.com/pdf/Mission1EvaluationReport.pdf · An online survey was created using Polldaddy (an online survey service) and reviewed by several key national staff

14 | P a g e

with the wider UCC. One participant stated that such efforts could become

“too cumbersome if done too often.”

Final Thoughts. When prompted to offer any final thoughts regarding Mission: 1,

357 individuals chose to provide comments. The nature of these comments was

overwhelmingly positive, with many sentiments of gratitude to the wider UCC for

the planning and execution of Mission: 1 and requests to initiate similar initiatives

in the future. The following comment summarizes these sentiments: “Thank you

for a well-thought out vision, great resources, measurable goals, flexible and

personally adjustable implementation, and a great way to connect with others in the

UCC around a common mission. This is the church at its best.”

There were several comments that offered suggestions for future similar efforts

based on their experiences with Mission: 1. These included requests for more lead

time in planning for churches (perhaps providing announcements in the spring for

a fall endeavor), suggestions regarding the reporting of tallies for online food and

fund donations, and some clarity around giving (specifically regarding the

relationship between Neighbors in Need, East Africa famine relief, and Mission: 1

giving). Some respondents also requested follow-up information and

communication on the impact of letters written, funds donated, and food collected.

Targeted Response Surveys

Participant Congregations Survey

Of the 41 congregations targeted for this survey, 11 congregations responded

(26.8% response rate). Pastors were asked two multiple-choice questions from the

large-scale Mission: 1 Survey: (a) What is the single most important way that

Mission: 1 impacted your congregation; and (b) As you saw it, Mission: 1 was first

and foremost designed for what purpose?

In response to the first survey question, 45% (5 out of 11 pastors) emphasized that

Mission: 1 increased a sense of connection with others across the UCC, and 3

pastors (27%) responded that Mission: 1 increased/deepened their churches’

commitments to addressing food and hunger issues (see Figure 7).

One minister offered these comments with regard to an increased sense of

connection to the wider UCC: “There was excitement that we were doing

something with other UCC churches across the country. We shared news stories

Page 15: Evaluation Reportuccfiles.com/pdf/Mission1EvaluationReport.pdf · An online survey was created using Polldaddy (an online survey service) and reviewed by several key national staff

15 | P a g e

and updates of what others were doing. There was a connection. Having our church

featured in United Church News was exciting to many members, and many began

to check out the UCC website for more stories.” Other pastors made similar

comments in this regard.

For the second question on the primary purpose of Mission: 1, all 11 respondents

(100%) indicated that Mission: 1 was meant to “engage the whole UCC in one

shared mission with measurable outcomes.” Two respondents also indicated that

another central purpose of the initiative was to “deepen UCC churches’

involvement in hunger and poverty ministries.”

Pastors expressed that an initiative with measurable outcomes that also allowed for

creativity and contextualization was a key factor in their churches’ involvement:

“Our congregation has been committed to both direct service and advocacy work

around hunger issues both locally and globally, so the opportunity to share that

concern in a finite, measurable, denomination-wide effort helped reinforce and

affirm our relationship with others in the UCC. It was a creative idea that

encouraged creativity as well.”

Non-Participant Congregations Survey

Of the 41 congregations targeted for this survey, 8 congregations responded

(19.5% response rate). Pastors were asked the following question and given several

choices for response (see Appendix D for complete list of options): What was the

primary reason your congregation did not participate in Mission: 1?

0

10

20

30

40

50

Sense of

connection with

others across the

UCC

Commitment to

addressing food

and hunger issues

Sense of

connection as a

congregation

Sense of purpose Commitment to

mission and

service in general

FIGURE 7: Impact on Participating

Congregations (Percentages)

Page 16: Evaluation Reportuccfiles.com/pdf/Mission1EvaluationReport.pdf · An online survey was created using Polldaddy (an online survey service) and reviewed by several key national staff

16 | P a g e

The great majority of pastors (7 out of 8) articulated that their churches were

already engaged in food and hunger ministries (see Figure 8). These churches did

not feel a need to report the labors of their already-existing efforts as part of

Mission: 1. The following comment summarizes the sentiments of this response:

“This issue is a core issue for our church, and we did not feel that we needed to

participate in a special campaign to emphasize the issue any more than we already

do.” Some stated that the participation required for Mission: 1 was “just more

reporting” in an already busy season for their churches.

5 out of 8 pastors also emphasized that their congregations were focused on other

mission efforts besides food and hunger. Individuals listed ministries such as

assistance with transitional housing, a clothing bank, mission trips and global

partnerships, community organizing efforts, and Habitat for Humanity work.

Online Survey of National Staff

For the online survey of UCC national staff, 29 individuals responded, with an

81% completion rate.

Demographics. The following information was captured from survey respondents:

18 participants (64%) were female, 9 participants (32%) were male, and

1(4%) identified as other.

19 individuals (66%) self-identified as European American/White, 6 (21%)

as African American/Black, 2 (7%) as Latino/Hispanic, 1 (3%) as Asian

American or Pacific Islander, and 1 (3%) as Other.

0102030405060708090

100

Already engaged in

addressing food and

hunger issues (7

responses)

Focused on other

mission efforts

besides food and

hunger (5 responses)

Church in a state of

transition at this time

(1 response)

Not aware of

Mission: 1 (1

response)

FIGURE 8 : Responses of Non-

Participating Churches (Percentages)

Page 17: Evaluation Reportuccfiles.com/pdf/Mission1EvaluationReport.pdf · An online survey was created using Polldaddy (an online survey service) and reviewed by several key national staff

17 | P a g e

13 individuals (45%) identified as program staff with a focused portfolio, 7

(24%) as administrative support staff, 5 (17%) as a supervisor of multiple

staff, and 4 (14%) as other.

Activity. Individuals were asked to describe the role(s) they played in the

execution of Mission: 1. Responses ranged from the creation of congregational and

justice-related resources and news articles, to assisting with the promotion in

various regions of the country and at key events like Synod and Bless!, and to

engaging in mission and advocacy work during the 11 days of Mission: 1.

When asked what role(s)/activities individuals’ ministry teams engaged in

regarding Mission: 1, respondents reported ministry-specific online and newsletter

promotions, assisting with reporting of online contributions, participation in food

collection and advocacy, creation of information and resources, and assistance with

“coordination of side stories to promote and broaden the scope of the work.”

Impact. Survey participants were asked to check the single most important way

that Mission: 1 impacted them personally. 41% of respondents indicated that the

initiative created an increased sense of connection with others across the UCC, and

28% indicated that Mission: 1 increased their commitment to addressing food and

hunger issues (see Figure 9).

0

5

10

15

20

25

30

35

40

45

Sense of

connection

with others

across the

UCC

Commitment

to addressing

food and

hunger issues

Mission: 1 did

not have a

personal

impact on me

Sense of

purpose

Sense of

connection

with my

colleagues

Spiritual

life/connection

with God

Commitment

to mission and

service in

general

Other

FIGURE 9: Impact of Mission: 1 (Percentages)

Page 18: Evaluation Reportuccfiles.com/pdf/Mission1EvaluationReport.pdf · An online survey was created using Polldaddy (an online survey service) and reviewed by several key national staff

18 | P a g e

Primary Goal. 26 respondents (90%) indicated that Mission: 1 was first and

foremost designed to engage the whole UCC in one shared mission with

measurable outcomes. 3 respondents (10%) listed other primary goals.

Characteristics and Opinions. Individuals were asked to view several statements

and indicate the level of truth these statements held for them personally. 59% of

respondents responded with “Very True” to the statement that Mission: 1 was an

initiative that they fully supported, and 52% responded “Very True” to the

statement that they were easily able to incorporate Mission: 1 into their portfolios.

On the statement that staff workloads increased during the 11 days of Mission: 1,

responses were split between those who answered “Not True At All” (31%) and

“Mostly True” (28%) (see Figure 10).

FIGURE 10: Mission: 1 Characteristics

Comments on the above responses were generally grouped into three main themes:

- Supportive: Several comments indicated their overall support of Mission: 1

and found the initiative to be exciting and successful in reaching its stated

goals.

- Supportive, but…: Staff indicated their overall support but commented on

ways that Mission: 1 highlighted organizational and communication issues

in the national setting (in the planning process, vision, and execution).

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Mission: 1

was an

initiative that I

fully

embraced and

supported.

I was easily

able to

incorporate

Mission: 1

into my

portfolio.

The in-service

training for

Mission: 1

adequately

prepared me

to support this

initiative.

Mission: 1

created an

increased

sense of unity

among staff in

the national

setting.

My workload

greatly

increased

during the 11

days of

Mission: 1.

Not True At All

Fairly True

Somewhat True

Mostly True

Very True

Page 19: Evaluation Reportuccfiles.com/pdf/Mission1EvaluationReport.pdf · An online survey was created using Polldaddy (an online survey service) and reviewed by several key national staff

19 | P a g e

- Other: Several staff who served as deployed staff or were on sabbatical

made comments regarding their inability to fully respond to some of the

questions that were targeted toward those located in the Cleveland offices at

the time.

Overall, when staff members were asked whether they wished Mission: 1 had been

executed differently, responses were evenly split between Yes (31%), No (34%),

and Not Sure (34%), with no statistically significant differences between

percentage variance.

However, when asked whether individuals believed the UCC should engage in

another all-church initiative similar to Mission: 1, 25 respondents (86%) said Yes,

3 (10%) were Not Sure, and 1 (3%) said No.

The single most important characteristic of an “all-church initiative” such as

Mission: 1 was that “all settings of the church have a clear understanding of what

is expected and how they can participate” (18 responses, 33%). All other options

chosen indicated a reduced response rate by 50% or more (see Figure 11).

FIGURE 11: All-Church Initiative Characteristics

(Percentages) All settings of the church have a clear understanding of what is

expected and how they can participate.

All settings of the church have adequate advance time to prepare and

inspire broadest possible participation.

The goals and outcomes are measureable and realized using real-

time reporting and accounting.

Outcomes are multi-faceted (such as local service, justice advocacy,

fundraising, and global action).

Broadest possible buy-in is sought from Collegium, Mission

Planning Council, Boards of Directors, Conference Ministers, local

churches, and associated/affiliated ministries, schools, and agencies.

All settings of the church are consulted about the

idea/vision/direction/decision as to what constitutes an “all-church

initiative.”

The dates are specific, not ongoing.

Page 20: Evaluation Reportuccfiles.com/pdf/Mission1EvaluationReport.pdf · An online survey was created using Polldaddy (an online survey service) and reviewed by several key national staff

20 | P a g e

Conclusions

Based upon the data collection methods utilized (large-scale surveys, targeted

surveys, multiple-choice/matrix responses, and open-ended responses) and the

populations surveyed (participating individuals and congregations/groups, targeted

participating congregations, targeted non-participating congregations, and national

staff), several key statements and themes regarding Mission: 1 were realized.

These statements and themes directly respond to the main purposes highlighted at

the beginning of the evaluation.

Respondents, across the board, felt that Mission: 1 was first and foremost

designed to engage the whole UCC in one shared mission with measurable

outcomes. Mission: 1 individuals and groups, church pastors from the targeted

survey, and UCC national staff all indicated, overwhelmingly, that this was the

main goal of the initiative. This theme is nearly identical to the official stated goal

of Mission: 1: “One united church on a shared mission for 11 powerful days to

feed the hungry and confront food-related injustice.”

The greatest impact of Mission: 1 was the sense of connection it created with

others across the life of the United Church of Christ. Not only was this the

primary impact indicated by all sampled participant groups, individuals and

congregations, but respondents of the large-scale online survey also indicated that

the “opportunity to be part of something larger than ourselves” was the single most

important aspect of their participation. While national staff members were slightly

less enthusiastic about the sense of unity and connectedness that Mission: 1 created

among their colleagues in the national setting, the majority of survey respondents

still indicated that this was an impact of Mission: 1.

The food and hunger focus of Mission: 1 increased/deepened individuals’ and

congregations’ commitments to addressing this particular issue of injustice. While people felt that this was not the greatest outcome or impact of Mission: 1, it

strongly ranked second. This was clearly an issue that many individuals and groups

were already addressing in both local and global contexts, yet Mission: 1 increased

and deepened their awareness and involvement in this regard. As several

commented, this was a general, widespread issue that could be readily addressed

by all members of the church.

Inversely, Mission: 1 was less about fostering an individual or collective sense of

purpose or deepening spirituality and connection with God. This is not to say that

Page 21: Evaluation Reportuccfiles.com/pdf/Mission1EvaluationReport.pdf · An online survey was created using Polldaddy (an online survey service) and reviewed by several key national staff

21 | P a g e

this was not an implicit outcome, but respondents across the board did not indicate

that Mission: 1 had this particular impact on them or their congregations/groups in

a statistically significant way. However, this was not a stated goal of the initiative.

Mission: 1 was a successful endeavor due, in large part, to the overall

structure of the initiative. Clear and concrete goals, a set timeframe for

participation, advance notice and preparation time, and useful resources were all

cited in open-ended responses from each surveyed group as key logistical features

that encouraged participation. In addition, the theme of creativity was frequently

mentioned as a crucial aspect of success. Whether individuals or churches

described the ways in which they were adapting Mission: 1 to their own contexts

and cultures, or directly mentioned the notions of creativity and flexibility in their

responses, this theme played a central role in the execution of Mission: 1 in all

settings.

Among the congregations surveyed that did not participate in Mission: 1, it

was clear that their lack of involvement in this initiative did not signal a lack

of commitment to mission in general. On the contrary, congregations were

involved in mission work (even involving food and hunger issues) in both local

and global contexts; but they felt that they did not need to engage in Mission: 1 at

that particular time. For these congregations, it may be that a sense of connection

and unity to the larger denomination was not a priority at this point in their

ministries; however, this is only speculation and was not verified.

National staff indicated that they envisioned all-church initiatives in general

to be events in which all settings of the church have a clear understanding of

what is expected and how they can participate. While this is not specific to

Mission: 1 in particular, it may be an indication of ambiguities regarding specific

roles of various staff in the national setting. For example, the query regarding the

preparation that the in-service training provided received a mixed response overall,

as did the question on staff workload increasing during the 11 days of Mission: 1.

Page 22: Evaluation Reportuccfiles.com/pdf/Mission1EvaluationReport.pdf · An online survey was created using Polldaddy (an online survey service) and reviewed by several key national staff

22 | P a g e

Recommendations

The following recommendations are based on the data collected and the

conclusions offered:

1. Mission: 1, or something similar in nature to Mission: 1, should be

executed again. This was the overwhelming response of individuals and

congregations/groups that were surveyed. Many respondents talked about a

“Mission: 2,” and several congregations already committed to continuing

Mission: 1 this coming year in their local settings. Because the structural

aspects of this initiative determined set goals and timelines, yet encouraged

creativity and flexibility, individuals and groups were able to adapt Mission:

1 to their respective contexts.

However, regardless of the focus issue or structural aspects of a future

initiative, what inspired the great success of Mission: 1 was that it created a

sense of connection and unity within the denomination as a whole. Whatever

initiatives are proposed in the future, this aspect must be a fundamental

component of both the endeavor itself and the communication about such a

project.

Another key aspect of any future initiative must be a focus on addressing

injustice. This an essential identity marker of the United Church of Christ,

and congregations committed to participating in Mission: 1 because it

actively engaged members in addressing injustice through mission and

advocacy.

One note of caution: Several respondents expressed that something similar

to Mission: 1 should not be executed too often, as individuals and churches

may not appreciate the additional activity involved with such efforts on a

frequent basis.

2. If another similar initiative is undertaken, greater clarity is needed with

regard to financial giving aspects of the project. Participants indicated

some confusion with regard to various funding options (particularly for

Neighbors In Need and East Africa Famine Relief) and also requested

options for reviewing their giving totals online to assist them in keeping

track of overall congregational giving.

Page 23: Evaluation Reportuccfiles.com/pdf/Mission1EvaluationReport.pdf · An online survey was created using Polldaddy (an online survey service) and reviewed by several key national staff

23 | P a g e

3. It may be helpful for UCC staff in the national setting to be provided

with clearer guidelines and examples of ways to participate and

integrate future initiatives into their particular roles and

responsibilities. This may assist in an increased sense of unity and

connection with colleagues in other ministry areas.

Page 24: Evaluation Reportuccfiles.com/pdf/Mission1EvaluationReport.pdf · An online survey was created using Polldaddy (an online survey service) and reviewed by several key national staff

24 | P a g e

Appendix A

Mission: 1 Evaluation Plan

Purpose

The central purpose in conducting this particular evaluation of Mission: 1 is to answer the following

question: What was the impact of Mission: 1 beyond its intended measurable outcomes? In order words,

what were individual and congregational experiences of Mission: 1? How did it shape, form, and

transform people with regard to their faith, mission commitments, and sense of unity/community?

Research Questions

Several specific research questions will guide the evaluation. They include, but are not limited to, the

following:

- How did Mission: 1 impact individuals and congregations (their awareness and commitment to

food and hunger issues, sense of community, faith/spiritual formation, sense of purpose, and

relationships with local and global entities)?

- What factors made Mission: 1so successful and widespread across the life of the church?

- How did Mission: 1 inform people’s understandings of what it means to be part of a

denomination?

- Is Mission: 1 (or something similar to Mission: 1) replicable?

- Why did some congregations not participate in Mission: 1?

Methodology

To gain insight into the impact of Mission: 1 on individuals and congregations, a qualitative research

methodology will be applied. Qualitative methods draw from text-based data rather than numbers-based,

or quantitative, data. Words in the form of stories, comments, opinions, and other feedback offer an

understanding of experiences not obtainable through quantitative methods.

One specific type of qualitative research method that will be particularly crucial for this evaluation is the

use of a phenomenological approach. Phenomenological approaches center on the interpretation of a

particular event or experience, called the phenomenon, shared by a group of people. Mission: 1, in this

evaluation, serves as that shared phenomenon encountered by all participants.

To respond to the research questions, a number of data collection methods and strategies will be utilized

and include, but are not limited to, the following:

- Online surveys of individuals/congregations who participated in Mission: 1

- Descriptive data collection from comment boxes on Mission: 1 reporting and donation forms

(online and paper)

- One-on-one interviews/open-ended response surveys with a purposeful, representative sample of

congregations who participated in Mission: 1

- One-on-one interviews/open-ended response surveys with a purposeful, representative sample of

congregations who did not participate in Mission: 1

Page 25: Evaluation Reportuccfiles.com/pdf/Mission1EvaluationReport.pdf · An online survey was created using Polldaddy (an online survey service) and reviewed by several key national staff

25 | P a g e

Timeline

The 2012 evaluation timeline below is subject to change given varying contextual issues; however, this

provides a framework from which to implement and complete the final report.

Mid-January – February 15 Data Collection

Mission: 1 Online Survey

Interviews/Response Surveys

February 16 – March 6 Analysis of Data/Report Writing

March 7 Preliminary Report Completed

March 13 Preliminary Report Presented to Executive Council

Development Committee

April 1 Final Report Completed

April 15 – 18 Final Report Presented at Annual Consultation

This is only a preliminary plan of action and is subject to change depending upon information and

resources available. If you have questions, please contact Kristina Lizardy-Hajbi, Minister for Christian

Faith Formation Research, at [email protected] or 216-736-3866, Toll Free 1-866-822-8224 Ext. 3866.

Page 26: Evaluation Reportuccfiles.com/pdf/Mission1EvaluationReport.pdf · An online survey was created using Polldaddy (an online survey service) and reviewed by several key national staff

26 | P a g e

Appendix B

Participating Churches Survey Request

Dear Colleague,

My name is Kristina Lizardy-Hajbi, and I serve as Minister for Christian Faith Formation

Research in Local Church Ministries. I am conducting an evaluation of the United Church of

Christ’s recent Mission: 1 effort, and the ministry that your congregation engaged in for this

event was featured on the Mission: 1 website (either in an article or a photograph/video that was

submitted).

In addition to the large survey that is currently being conducted on Mission: 1 (you can complete

it by clicking on this link: http://6404460.polldaddy.com/s/mission-1-survey), I am also

attempting to gain information from a smaller group of pastors whose congregations participated

in this event. Specifically, I’d like to learn more about the impact that you feel Mission: 1 had on

the people in your church and would greatly appreciate your time and effort in responding to the

following two questions.

1. What is the SINGLE most important way that Mission: 1 impacted your congregation?

It deepened/increased our…

a. Sense of connection with others across the UCC

b. Commitment to addressing food and hunger issues

c. Awareness of food and hunger issues

d. Commitment to mission and service in general

e. Sense of connection as a congregation

f. Sense of purpose

g. Spiritual lives/connection with God

h. Other (please describe)

2. As you saw it, Mission: 1 was FIRST and FOREMOST designed to:

a. Engage the whole UCC in one shared mission with measurable outcomes

b. Deepen UCC churches’ involvement in hunger and poverty ministries

c. Utilize online technology to connect UCC members and churches

d. Invite new people into relationship with the UCC

3. Thoughts and comments on your above responses:

Thank you for your time and collaboration in this assessment effort, and I look forward to

hearing from you soon. If you have any additional questions, please do not hesitate to contact

me. Blessings on your ministries—God Is Still Speaking!

Rev. Dr. Kristina Lizardy-Hajbi

Minister for Christian Faith Formation Research

Congregational Vitality and Discipleship

Local Church Ministries

United Church of Christ

Page 27: Evaluation Reportuccfiles.com/pdf/Mission1EvaluationReport.pdf · An online survey was created using Polldaddy (an online survey service) and reviewed by several key national staff

27 | P a g e

Appendix C

Sample Participating Churches

CHURCH CITY ST

Zion Philadelphia UCC Ritzville WA

Congregational UCC Rochester MI

First Congregational UCC Eldora IA

Wantagh Memorial Congregational UCC Long Island NY

United Church on the Green New Haven CT

St. Paul's Community UCC Cleveland OH

First Congregational UCC South Portland ME

Danville Congregational UCC Danville CA

Rock Spring Congregational UCC Arlington VA

St. Paul's UCC Somerset PA

Trinitarian Congregational UCC Concord MA

Hendersonville First Congregational Church* Hendersonville NC

Congregational UCC Tolland CT

Kirkwood UCC Atlanta GA

First Congregational UCC South Haven MI

First Congregational UCC Haworth NJ

Naples UCC Naples FL

Smyrna UCC Canby OR

Chewelah UCC Chewelah WA

Immanuels UCC-Holstein Marthasville MO

St. John UCC Wood River IL

Sun Lakes UCC Sun Lakes AZ

First Congregational UCC Silverton CO

Ascension UCC West Norriton PA

First Congregational UCC Cedar Rapids IA

Spirit on the Lakes/Minnehaha UCC Minneapolis MN

First Church UCC Lombard IL

Pensacola Beach Community UCC Pensacola FL

Sycamore UCC Sycamore OH

Trinity UCC McCutchenville OH

First Congregational UCC Bloomfield CT

First Congregational UCC Eugene OR

Mt. Sinai UCC Long Island NY

Church of the Good Shepherd UCC Ann Arbor MI

United Church of Hinesburg Hinesburg VT

Salem UCC Steinauer NE

First Congregational Church of Berkeley Berkeley CA

Rolling Hills Community Church Lago Vista TX

Page 28: Evaluation Reportuccfiles.com/pdf/Mission1EvaluationReport.pdf · An online survey was created using Polldaddy (an online survey service) and reviewed by several key national staff

28 | P a g e

Congregational UCC Ogden UT

West Congregational Church Phoenix AZ

Old South Church Boston MA

*Did not participate in Mission: 1.

Appendix D

Non-Participating Churches Survey Request

Dear Colleague,

My name is Kristina Lizardy-Hajbi, and I serve as Minister for Christian Faith Formation

Research in Local Church Ministries. I am conducting an evaluation of the United Church of

Christ’s recent Mission: 1 effort (if you are unfamiliar with this project, please go to

http://www.ucc.org/mission1/ to learn more).

I am looking for information from congregations in the Rocky Mountain Conference that did

NOT participate in this endeavor. If you are a pastor of such a church, would you mind

responding to the following question?

What was the PRIMARY reason your congregation did not participate in Mission: 1?

a. The church I serve is in a state of transition at this time.

b. The church I serve is already engaged in addressing food and hunger issues (briefly

describe).

c. The church I serve is focused on other mission efforts besides food and hunger (briefly

describe).

d. There are other more pressing issues within the church that needed our attention (briefly

describe).

e. I serve a congregation that maintains membership in more than one denomination.

f. I am a minister ordained by another denomination and engage the congregation in those

mission efforts.

g. I was not aware of Mission: 1.

h. Other (briefly describe)

I’d greatly appreciate your response and any other comments you’d like to share in this regard. If

your congregation did participate in Mission: 1, please disregard this message.

I look forward to hearing from you soon. Thank you for your participation in this brief survey!

Rev. Dr. Kristina Lizardy-Hajbi

Minister for Christian Faith Formation Research

Congregational Vitality and Discipleship

Local Church Ministries

United Church of Christ

Page 29: Evaluation Reportuccfiles.com/pdf/Mission1EvaluationReport.pdf · An online survey was created using Polldaddy (an online survey service) and reviewed by several key national staff

29 | P a g e

Appendix E

Sample Non-Participating Churches

CHURCH CITY ST

Christ Congregational UCC Denver CO

Berkeley Community Congregational Church Denver CO

Douglas Congregational UCC Douglas WY

Casper UCC Casper WY

Wheatland UCC Wheatland WY

St. John's UCC Greeley CO

St. John UCC Idalia CO

Buena Vista Congregational UCC Buena Vista CO

Black Forest Community UCC Black Forest CO

Vista Grande Community UCC Colorado Springs CO

United Church of Broomfield Broomfield CO

Seventh Avenue UCC Denver CO

United Church of Montbello Denver CO

Washington Park UCC Denver CO

First Congregational UCC Eastlake CO

Mayflower Congregational UCC Englewood CO

First Plymouth Congregational UCC Englewood CO

Rockland Community Church Golden CO

Columbine United Church Littleton CO

Northglenn UCC Northglenn CO

Parker UCC Parker CO

Wheat Ridge Congregation of the UCC Wheat Ridge CO

First Congregational UCC Longmont CO

Faith UCC Windsor CO

Broadmoor Community UCC Colorado Springs CO

Creede Community Church Creede CO

New Jerusalem Missionary Baptist UCC Fountain CO

The Church in the Wildwood Green Mountain Falls CO

Hilltop UCC Parker CO

First Congregational UCC Craig CO

Union Congregational UCC Crested Butte CO

Hillcrest Congregational UCC Montrose CO

The United Church of the San Juans Ridgway CO

Congregational Church of the Valley Silt CO

Holladay UCC Holladay UT

United Church of Kanab-Fredonia Kanab UT

Ogden Japanese Christian Church Ogden UT

Page 30: Evaluation Reportuccfiles.com/pdf/Mission1EvaluationReport.pdf · An online survey was created using Polldaddy (an online survey service) and reviewed by several key national staff

30 | P a g e

Orem Community UCC Orem UT

Midvale EFKS New Jerusalem UCC Midvale UT

Japanese Church of Christ Salt Lake City UT

First Congregational UCC Rock Springs WY