Post on 19-Aug-2015
Running head: PROGRAM EVALUATION PAPER1
Program Evaluation Paper
Children In Action Sports Club: Barnabus Club
Jessica L. Hipchen
The University of Georgia
PROGRAM EVALUATION PAPER2
Literature ReviewThe children who participate in the Barnabus Club, a program provided by Children In
Action Sports Club (CIA) located in Brunswick, Ga, are generally from neighborhoods where
drugs, alcohol, and violence are a commonality. The criteria for participant selection in the
program includes: single parent family, one or both parents incarcerated, family in crisis (i.e.
drug/alcohol abuse), low self-esteem, weight/coordination issues, and/or poor school
performance.
Many of these factors have not been measured in the community. There is existing data on
single parent family households within 5-10 miles of our facility. In total, there are an estimated
2,475 single-parent households with children in this 10 mile radius. Something that is important
for these children is programs that are age appropriate and provide and clean and safe
environment (Nowak B., 2014, p. 13).
These children have little to no positive influences in their lives. One of the factors that
influences children’s behavior in the schools is the importance that is placed on an education
within the home and in their social life. Within the ten mile radius mentioned earlier, there is an
estimated total of 3,741 individuals who do not have a high school degree (Nowak, 2014, p. 8).
Additionally, there is an estimated total of 2,580 children between the ages of 6-11 in this area
(Nowak, 2014, p.5). This is not including those that are farther than the 10 mile radius. If they
do not have time invested in them by outside sources such as the personnel within the Barnabus
Club afterschool program, they will likely fall into the cycle of the behaviors mentioned above.
Barnabus Club was created so that these children would have a safe place to go after school.
The kids are given an opportunity for structured activities to learn how to behave appropriately
and to work in a group setting. The goal of this program is to teach the children team building
and behavior, academic, and service skills. This will in turn increase their self-esteem and
PROGRAM EVALUATION PAPER3
motivation, decrease their rate of behavior problems in school and the home, improve their
grades, expand their knowledge of how to work as a team, and give back to the community in
which they live. Most importantly, the program helps the children to understand that Jesus Christ
is the motivator behind all that is done.
Program TheoryThere is no written program theory or guiding philosophy for the Barnabus Club. Their
activities are not based on hard data and/or research. The source of the impetus for the program
and associated assumptions is the Gospel of Jesus Christ. The personnel of Barnabus Club are
Christians who whole heartedly believe that Jesus is the one who gave them a passion for helping
at-risk children such as the ones involved in the program. They are able to make a difference in
the lives of children because they are merely a reflection of Jesus Christ’s character. Barnabus
Club believes that when the children have a circle of influence that reflects the love of Jesus
through activities such as mentoring, group sports, and academics, positive changes such as the
ability to make healthy choices are made within the children. Staff have seen and recorded these
positive changes throughout the length of the program, which will be further discussed in the
data collection portion of the paper.
Program Description:
PurposeThe mission of Children in Action Sports Club (CIA) is “to utilize sports and physical
activities as an outreach to children, youth, and adults to build character, excellence, and
teamwork” (Children In Action Sports Club, 2014). The vision of the organization “is to provide
a favorable environment to experience, accept, and exercise the application of a Christ-centered
lifestyle” (Children In Action Sports Club, 2014). The purpose of the program is to be a Christ-
centered ministry that is intentional in building relationships with children ages seven to twelve
PROGRAM EVALUATION PAPER4
and helping them to make healthy choices. One of CIA’s programs that was created to
encompass this mission and vision and create an impact in the community is the Barnabus Club.
ObjectivesThere are no existing written objectives. I have come up with a list of goals and
objectives that I would suggest creating based on the information provided about the program.
These include:
Goal 1: To promote the gospel of Jesus with participantsOutcome Objective 1a: To increase basic knowledge of the gospel of participants by 100% by 2016.
Outcome Objective 1b: To increase the number of children in the program who believe that Jesus Christ is their Savior by 50% by 2016.
Goal 2: To decrease the number of behavior problems with participants, specifically those who were referred to the program because of behavior issues
Outcome Objective 2a: To decrease the number of behavior problems experienced by participants within the schools by 50% by 2016.Outcome Objective 2b: To decrease the number of behavior problems experienced within the home by 50% by 2016.
Goal 3: To increase self-esteem of participantsOutcome Objective 3a: To decrease the rate of obesity of children involved in the program, specifically those referred to the program because of weight issues, by 20% by 2016. Outcome Objective 3b: To increase the number of children who believe that they can have a successful future by 90% by 2016.
Goal 4: To increase ability of participants to work as a teamOutcome Objective 4: To decrease the amount of negative interactions between/among participants involved in recreational activities by 75% by 2016.
Goal 5: To increase knowledge of participants on how to make healthy decisions
Outcome Objective 5: To increase participants’ knowledge on how to make healthy decisions by 100% by 2016.
Goal 6: To increase activeness of participants in the community
PROGRAM EVALUATION PAPER5
Outcome Objective 6: To increase the amount of participants who spend time in community service activities to total 100% by 2015.
ParticipantsThe participants are children between the ages of seven and twelve who have participated
in the Summer Outreach program, another CIA program, and are identified as high risk. The
criteria for participant selection includes: single parent family, one or both parents incarcerated,
family in crisis (i.e., drug/alcohol abuse), low self-esteem, weight/coordination issues, and/or
poor school performance. The children have shown themselves to be receptive to the
programming (Summer Outreach) yet needing more time with personnel to maintain their
positive behavior and attitude changes. As of 2014, a total of 70 kids participate in the program
with an average of 40 children per day.
StaffingThe program has a total of two full time and four part time staff. In addition to paid staff,
there is an average of five to seven volunteers per week (A. Benner, personal communication,
October 30, 2014).
In addition to the staff and volunteers mentioned above, an evaluator will also be
considered as part time staff in the Barnabus Club’s program. The tasks of the evaluator will be
to collect behavior reports from the school that involve the participants in the program, create
and administer surveys for teachers, parents, and children, provide and distribute log books for
teachers and parents, collect data on progress reports and report cards of the participants, record
the amount of times that a CIA staff member has had to visit the school to mentor a participant,
and lastly to combine all of the information into a structured format that can then be coded and
interpreted for distribution.
PROGRAM EVALUATION PAPER6
SettingThe Barnabus Club program is located in a gymnasium leased by CIA. The gymnasium
includes a kitchen, a dining area for snacks/meals, and additional areas for homework, devotions,
and physical activities.
ActivitiesAs mentioned before, CIA is an organization dedicated to reaching out to high risk
children and youth to build character, a desire for excellence, and an appreciation for teamwork
through physical activity in a team setting which mirrors Jesus Christ in relationship and
character building. In addition, the Barnabus Club, now in its third year of operation, involves
one-on-one mentoring with participants during practices, games, and devotion times as well as
picking children up from schools around Glynn County and transporting them to the facility for
the after school program. The daily program involves 30 minutes of free time for the children to
unwind followed by a break for nutritious snacks and 45 minutes of supervised homework/study
time. At the end of their study time, the program provides a 30 minute devotion time. The final
45 minutes are dedicated to a variety of sports activities where the importance of physical
activity is stressed. Barnabus Club students who experience difficulties in school, academic
and/or behavioral, receive additional support from CIA staff who make school visits and
collaborate with school teachers and other school personnel. Based on their suggestions, CIA
works more directly with the children to help resolve their problems. Additionally, the children
are provided opportunities for community service each week. This may involve doing things
such as serving food to the homeless, raking leaves for the elderly, picking up trash, or
packaging medical supplies at Map International.
Logic modelSee Chart 1 in the Appendix
PROGRAM EVALUATION PAPER7
Focus/Aim of the Evaluation The focus of the evaluation is to determine if CIA’s Barnabus Club program is achieving the
goals and initial, intermediate, and long-term objectives that I created based on the program’s
mission, vision, and activities. There is no written list of goals and objectives; therefore, I created
my own. The results of the evaluation will show whether or not the Barnabus Club program is
having the positive impact on the participants’ behavior and attitudes that are intended. Once the
results have been identified, they can be used to show the areas where impact is abundant or
lacking so that changes can be made to be the most efficient and effective possible.
Research Questions and Hypotheses If students participate in the Barnabus Club then they will gain a greater
understanding of God’s word.IV-Information provided during the huddle/devotion sessionDV- Score on survey
If students participate in Barnabus Club then there will be a decrease in behavior problems at home.
IV-Participation in mentoringDV-Combination of a survey and a parent/guardian log book
If students participate in Barnabus Club then there will be a decrease in behavior problems at school.
IV-Participation in mentoring DV- Combination of behavior reports, a survey, and a teacher logbook
If students participate in the Barnabus Club then their grades in school will increase.
IV-Participation in homework sessionsDV-Combination of progress reports and report cards
If students participate in the Barnabus Club then they will acquire the skills to work on a team.
IV-Participation in sports activitiesDV-Survey
Measurable Indicators. See Chart 2 in Appendix
PROGRAM EVALUATION PAPER8
Design Selection The design selection for this evaluation will be a group research design (GRD). This
strategy is best for the Barnabus Club environment because it allows the program to be evaluated
immediately when a child enrolls in the program and then again a year later. Barnabus Club
would not want to utilize a GRD that delayed the intervention for a child, therefore the strategy
to begin the assessment and data collection at the time of enrollment was chosen. There would be
ethical issues if Barnabus Club were to choose to have a control group. This is due to denying
services that would help the high-risk children. Therefore, all designs containing control groups
were eliminated from the list of options. Furthermore, there will be continuous data kept by
parents, teachers, and staff which will be gathered together and added to the results of the pre-
and post-surveys given to the children to measure different areas of growth. This will show if the
objectives were reached. This is an important factor to show the effectiveness of the program on
the areas intended. There are some threats to internal validity when using a GRD. It is not always
the case that the results are generalizable. To overcome these difficulties there will be a pilot test
before the actual evaluation is conducted. Results cannot be labeled as causal because the
evaluation is not as rigorous as an experimental design (Royce D., Thyer B., Padgett D., p.222).
If necessary, this can be overcome by conducting an evaluation using an experimental research
design. Additionally, in the future, the organization will be able to use the same materials and
evaluation plan. This process is referred to as replication (Royce D., Thyer B., Padgett D., 164).
Data Collection In an effort to control external validity there will be a pilot test completed beforehand.
Data will start being collected upon enrollment into the program. The process of data collection
can be found in Chart 2 of the Appendix.
PROGRAM EVALUATION PAPER9
Data Analysis StrategiesThe surveys will be given to children at the time of enrollment into the program. The
evaluator will distribute the same surveys at the one year mark. Throughout the year, staff will be
responsible for keeping a log book on the behavior of children during group sports activities.
This will be indicated by listing with tally marks the number of times that a child has a negative
behavioral interaction with a team mate. Teachers and parent(s) will also be responsible for
keeping behavior log books and will record the same way as staff for the child in the school or
home. Grades will be collected from the schools at the time of distribution of progress reports
and report cards. Lastly, BMI tests will be conducted at the same times as the surveys. The
results will then be given to the evaluator who will add up the scores and interpret the findings so
that the organization can understand the effectiveness of its program on all levels.
LimitationsThe limitations that the evaluation may come into contact with are that students may not
attend the afterschool program every day. This could skew the data for behavior during sports
activity times. Additionally, each parent, teacher, and staff may have different definitions of
good and bad behavior. This could result in a misuse of the terms when recording data. An
additional issue could be that the parent(s), teacher(s), and staff do not record like they are
supposed to in the log book. If a student involved in the evaluation drops out of the program, the
data for that child will not be included in the results.
Potential Ethical Issues One potential ethical issue to keep in mind is confidentiality. The evaluator and staff
would be required to not mention ongoing data and results that are tied to any one child. This
would be unethical and could threaten the well-being of the child. An additional potential ethical
PROGRAM EVALUATION PAPER10
issue is informed consent. The child would need to give assent and the parent(s)/guardian(s) of
the participant would need to give consent. Because the child is not 18 years of age, this is
required. Not following these guidelines would be unethical and would cause the data to be
thrown out.
Plan for Involving StakeholdersTo involve stakeholders, CIA could host a family night. This would be done so that the
families of the children involved in the program could have a chance to come and visit the
facility and learn about what their children are participating in each day after school and see the
impact that it is having on their child/children’s life. All staff and volunteers would be required
to attend so that the parent(s)/guardian(s) of the children can ask any questions and meet the
hands and feet that are behind the program.
An additional way to involve stakeholders would be to host an appreciation dinner for
staff and volunteers. Food would be provided through donations from local organizations such as
those that already provide the healthy snacks for the children in the Barnabus Club program. The
dinner would be free of charge and would be a time of thanks and community for the staff and
volunteers.
Plan for Disseminating Results The results will be distributed to all key stakeholders of CIA. This includes
parent(s)/guardian(s), teachers involved with the program, staff, board members, volunteers, and
donors. This is so that each stakeholder can see the impact that their time and service are making.
Because board members have a bi-monthly meeting and will soon be switching to a monthly
meeting, results will be disseminated through a PowerPoint presentation during the first board
meeting after the results have been finalized. All staff will be required to attend the board
meeting. Additionally, results will be sent to all board members and staff via email. Teachers,
PROGRAM EVALUATION PAPER11
parent(s)/guardian(s), volunteers, and donors will have a summary of results that will be placed
into a document containing overall percentages and major findings. This is so that the document
will not contain jargon and a multitude of hard data that may not be easily understood. This
document will also be posted on the website.
TimelineSee Chart 2 in Appendix
Budget
Children in Action Sports Club: Barnabus Club program
Requested Funding
Total
Non-Personnel Expenses
Printing (surveys, presentation materials) $700.00 $700.00Log Books $300.00 $300.00BMI Chart $0 $0Total Personnel Expenses $1000
Personnel ExpensesEvaluator (surveys, coding, data documents)
$15,000 $15,000
Total Personnel Expenses $15,000
Total $16,000
PROGRAM EVALUATION PAPER12
References
(2014). About Us: Mission &Vision. Retrieved from http://www.ciasportsclub.net/mission--
vision.html.
Georgia Baptist Convention. (2014). Community Outreach Profile. Comprehensive
Demographic Data and Lifestyle Analysis. 2505 Norwich St., Brunswick,
Ga. :Nowak.
Royse, D., Thyer, B. A., Padgett, D. K., & Logan, T.K. (2010). Program evaluation: An
Introduction (fifth edition). Thomson Brooks/Cole.
PROGRAM EVALUATION PAPER13
Appendix
Chart 1: Logic Model
Inputs Activities Outputs InitialOutcomes
Intermediate Outcomes
Long-term Outcomes
1. Staff2. Funding
Entities3. Volunteers4. Gymnasium5. Sports
Equipment6. Healthy
Food for Snacks
7. Transportation
8. Collaborating Partners
9. Existing Organizational or Interpersonal Networks
10. Time11. Supplies12. Parent(s)/
Guardian(s)13. Computers
1. Huddle/Devotion and Character Development Sessions
2. Homework Sessions
3. Physical Activity Sessions
4. School Visits Sessions
5. Service Learning Activities
1. Number of huddle/devotion and character development sessions held
2. Number of homework sessions held
3. Number of physical activities sessions held
4. Number of school visits made
5. Number of service learning sessions
1. Increased awareness of the gospel
2. Increased awareness of positive character development
3. Changes in attitude about homework
4. Increase in physical activity
5. Changes in attitude about being mentored
6. Increase in participation in service learning activities
1. Increased knowledge of the gospel
2. Increased knowledge of a positive character
3. Increased educational skills
4. Increased ability to participate in sports activities
5. Increased ability to work as a team
6. Increase in positive behavior at school and home
7. Increase in willingness to help others
1. Increase in belief that Jesus is our savior
2. Increase in self-esteem
3. Increased positive character development
4. Increase in grades
5. Ability to work as a team
PROGRAM EVALUATION PAPER14
Chart 2: Measurement Indicators
Time of Outcomes Indicators Target Sample Data Source Data Evaluation Collection
MethodAt registry into program and one year
If students participate in the Barnabus Club then there will be a decrease in problems experienced by participants within the school.
Ongoing mentoring within the program and during school as needed
50% decrease in behavior problems
High-Risk children
School
Teacher
Child
Behavior Reports
Behavior Log-Book
Survey
At registry into the program and one year
If students participate in the Barnabus Club there will be a decrease in the number of behavior problems experienced within the home.
Ongoing mentoring afterschool within the program
50% decrease in behavior problems
High-Risk children
Parent(s)
Child
Behavior Log-Book
Survey
At registry into the program and one year
If students participate in the Barnabus Club then there will be a decrease in the rate of overweight/obesity of children involved in the program, specifically those referred to the program because of weight issues.
Organized sports activities during the program
20% decrease in the rate of overweight/obesity
High-Risk children
Child BMI Test
At registry into the program and one year
If students participate in the Barnabus Club then
Ongoing mentoring and huddle/devoti
90% increase
High-Risk children
Child Survey
PROGRAM EVALUATION PAPER15
there will be an increase in the number of participants who believe that they can have a successful future.
on sessions
At registry into the program and one year
If students participate in the Barnabus Club then there will be a decrease in the amount of negative interactions between participants involved in recreational activities.
Organized sports activities during the program
75% decrease
High-Risk children
Child
Staff
Survey
Behavior Log-Book
At registry into the program and one year
If students participate in the Barnabus Club program then there will be an overall increase in knowledge of how to make healthy decisions.
Ongoing mentoring and huddle/devotion sessions
100% will increase their knowledge
High-Risk children
Child Survey
At registry into the program and one year
If students participate in the Barnabus Club then there will be an increase in the number of participants who spend time in community service activities.
Weekly Service Learning Activities
100% of students will increase time spent
High-Risk children
Staff Time Log
At registry into the program and one year
If students participate in the Barnabus Club then there will be an increase in basic knowledge of
Ongoing huddle/devotion sessions
100% will increase their knowledge
High-risk children
Child Survey
PROGRAM EVALUATION PAPER16
the gospel of participants.
At registry into the program and one year
If students participate in the Barnabus Club then there will be an increase in the number of children in the program who believe that Jesus Christ is their Savior.
Ongoing huddle/devotion sessions
50% of children will believe this
High-Risk children
Child Survey