1    

 

 

 

Program  Evaluation  Plan  

for  

Harrisburg  Parks  and  Recreation  

Youth  Soccer  Program  

 

 

 

By:  Shawn  Marble  

February  17,  2012  

 

 

 

 

2    

Table  of  Contents  

History  

 3-­‐4  

Programs  to  be  Evaluated  

 4-­‐5  

Stakeholders  

 5-­‐6  

Purpose  of  the  Evaluation  

 7-­‐8  

Structure  and  Context  of  the  Program  

8  

Evaluation  Questions  

9-­‐11  

Population  Description  

12-­‐14  

Body  of  Knowledge  

14-­‐16  

Evidence  Reliability  and  Validity  

16  

Resources  Needed  

17  

Appendix  A  

18  

 

 

 

3    

History:

The Harrisburg Parks and Recreation Department was formed in January 2007 with one full time

staff member. In January 2009, the Town of Harrisburg hired their first Parks and Recreation

Director to bring the department up to two full time people. This was done because the Town

was planning on undertaking the operations of the Harrisburg Youth Association (HYA). HYA

had existed in the Town of Harrisburg for sixteen years. It was a youth association run entirely

by volunteers and parents. The youth association had provided youth athletics in the spring, fall

and winter seasons including baseball, softball, soccer, basketball, cheerleading and football. The

participant numbers had grown so much that it was no longer manageable by volunteers. The

HYA board approached the Town and after months of preparation and a vote, the Town of

Harrisburg took over the operations of the youth athletic program in January 2009.

The Harrisburg Parks and Recreation Department offers a variety of programs to include special

events such as the Harrisburg Halloween Festival and the Harrisburg July 4th Celebration. It

offers classes such as art, yoga and bridge. By far, the biggest program the Parks and Recreation

Department runs is the youth athletic program. In the spring of each year, the department offers

baseball for age 3-15, softball for ages 7-15 and soccer for ages 3-15. In the fall the department

offers those three sports along with Pop Warner Cheerleading and Pop Warner Football for

children age 7-15. In the winter the department offers basketball for ages 5-15. In 2011, the

department registered 3,281 for youth athletics. Soccer was the largest activity with 1,400

participants.

volunteer coaches and several volunteer coordinators. There are sport specific coordinators that

4    

volunteer to help direct each league. For soccer there is a ¾ and 5/6 coordinator, a 7/8 and 9/10

coordinator and an 11/12 and 13/15 coordinator. These coordinators work very closely with the

parents, coaches, participants and staff and help with the draft process, forming practice and

game schedules and overseeing the league in play.

Program to be Evaluated:

The program that this evaluation plan will focus on is the Harrisburg Parks and Recreation youth

soccer program for ages 11, 12 and 13 (11/13) male and female division and the 14, 15 and 16

(14/16) coed division. Up until spring 2011, the Harrisburg Parks and Recreation Department

offered soccer for the following age groups: ¾ coed, 5/6 coed, 7/8 coed, 9/10 coed, 11/12 coed

and 13/15 coed. In spring of 2011, the 9/10 age group was split into boys and girls divisions.

There was an overwhelming support of this and there were more than enough participants

registered to make this happen. This continues today. The reason this split occurred is because

the 9/10 soccer coordinator had heard from enough parents that they felt this age group should be

split up by gender. He also had observed that the registration numbers had dropped off for

females when they went from 7/8 to 9/10. It was decided that by splitting this age group, a better

opportunity could be provided to the males and females in this league.

In January 2012, before registration for the spring season began, the department decided to make

some additional changes. The 11/12 age division was changed to 11/13. It was also split up by

gender. The oldest age group was changed from 13/15 to 14/16. These changes were made for

several reasons. The 11/12 age group was split up to continue what had been done in 9/10. The

coordinators and staff felt that since the males and females were playing separately in 9/10, they

should continue that in 11/12.

5    

We also added 13 year olds to that division. That was so that we could add 16 year olds to the

older league by forming a 14-16 division. It was thought that many sixteen year olds may still

want to play recreational soccer, but that once they turned sixteen they could no longer play in

Harrisburg. By changing the age breakdowns a little, we could include these sixteen year olds.

The only reason this age division was not broken down into male/female is because it has been

difficult in the past to get enough participants registered in this age group to have enough teams.

It was feared that if it was split, there would not be enough females or males alone to have a

league. So this league was left coed.

The ultimate goal of these changes it to provide a better program for the youth involved. These

decisions were made based on parent and participant requests and staff and coordinator

observations. Since spring 2012 will be the first season these changes will be in effect, the

Harrisburg Parks and Recreation Department feels it would be highly beneficial to evaluate the

program at the end of the season to make sure that participants and parents are satisfied and that

the participants are benefitting from the changes that have been made. This will also tell the

Parks and Recreation Department whether to continue with these changes for the fall 2012

season, or whether to go back to the original age groups.

Stakeholders:

A successful program evaluation is developed using questions posed by the key stakeholders,

individuals, groups, or organizations that have a significant interest in how well a program

Freeman, 2004, p. 18).

The major stakeholders in this case are the parents, soccer participants, coaches, coordinators,

parks and recreation staff, the Town Administrator and Town Council members.

6    

The parent, participants, coaches and coordinators all have a stake in this project because

changes to the program directly affect them. They will be the ones providing the majority of the

feedback regarding the changes to the program. The staff have a stake because it is their

responsibility to run a youth soccer program that benefits its participants in the best way

possible. The Town Administrator and Town Council have a stake because they are managing

the tax dollars that fund this project and are held accountable by citizens of the Town of

Harrisburg.

As the recreation coordinator for the Parks and Recreation Department I have a strong

relationship to most of the stakeholders involved and the program. As a staff member I work

under the Town Council and Town Administrator. It is my responsibility to make sure I am using

our budget wisely and providing the best customer service I can. I work closely with parents and

participants during registration and the season to ensure that each participant is having a good

experience. I address complaints and issues with any parents, players or coaches. I work hand in

hand on a daily basis with the coordinators and coaches to help with the draft process and

forming practice and game schedules. We all support each other to make sure these tasks are

website. I work daily with the other staff in the department to make sure each detail of the

program from registration to ordering uniforms and equipment to scheduling picture day is done

proficiently. The organization and management of this program is largely a group effort and

every piece of the puzzle must be in place for the program to run smoothly.

7    

Purpose of the Evaluation:

The type of evaluation that is being done is a summative evaluation. This main purpose of this

evaluation process is to determine whether the programs performance meets the criteria that are

expected. This evaluation will help the department and upper management make a decision about

whether to continue the changes made to the youth soccer program or whether or not it needs to

be re-structured. It could also affect the funding and other resources, such as staff, that is

provided (Rossi, Lipsey and Freeman, 2004).

In this case, the staff would be primarily responsible for conducting the evaluation. Certain

stakeholders such as program participants, parents and volunteers may provide input but will not

provide direct input in the decision making process. Therefore it is especially important that the

information that is found be scientifically credible. It could potentially be used by the Town

Council or Town Manager to make important decisions about the continuation of the program. If

the program evaluation results are not what the Council wants to see, staff positions and other

important resources such as money and equipment, could be eliminated. However, positive

results from the evaluation process could help convince decision makers that additional resources

are needed. Even if the program changes are not successful, staff will be needed to re-structure

the program and reallocate resources. In either situation, this evaluation can help provide

important information needed to make educated decisions by upper management and elected

officials (Rossi, Lipsey and Freeman, 2004).

8    

Structure and Context of the Program:

The purpose of the evaluation is to see if the outcomes projected by the Parks and Recreation

staff occur from the changes that are made in the 11/13 and 14/16 age groups for soccer. The

short term goals of the youth soccer program are the same for all participants: to learn the basic

fundamentals of soccer; to get weekly physical activity and interaction with other soccer players;

to get weekly exposure to the outdoors and Harrisburg parks; and to learn to follow directions

and play as a member of a team. It will be challenging to measure the number of participants

who learn the basic fundamentals of soccer and if they learn to play as a member of a team, but

this may be the individual opinion of each participant. The department may be able to determine

this by the number of participants who continue in the program season after season. Additional

intermediate and long term outcomes can be seen in the program logic model in Appendix A.

By making the said changes to the soccer program, it is not expected that the outcomes will

necessarily change, but that they will be met. The overall goal of the department would be for

each child to meet the short term, intermediate and long term outcomes of the program. These

changes to the program will hopefully allow more participants to succeed in having these

outcomes than if the changes had not been made.

9    

Evaluation Questions:

Below is a list of sample questions that could be used to evaluate the changes to the youth soccer

program. These questionnaires would only be given to participants (or the parents) of the 11/13

male and female division and the 14/16 coed division.

1. Have you participated in the Harrisburg Youth Soccer program prior to this season?

A. Yes B. No

If you answered yes, how many seasons have you played?

Answer: _______________________________________

2. What age group did you participate in?

Answer: _______________________________________

3. How would you rate your experience this season?

A. Great B. Good C. Neutral C. Ok D. Poor

4. (11/13 Participants Only) Did you agree or disagree with the change made to the division

of the 11/13 coed league into 2 divisions: 11/13 male and 11/13 female?

A. Strongly Disagree

B. Disagree

C. Neutral

D. Agree

10    

E. Strongly Agree

5. (11/13 Participants Only) Would you like to see the 11/13 age group continue to be split

into male and female?

A. Yes B. No

6. (11/13 Participants Only) Would you or your child continue to play in the Harrisburg

Parks and Recreation Youth Soccer Program if the 11/13 age division stayed divided by

gender?

A. Yes B. No

Why or Why Not?

Answer: _______________________________________

7. (14/16 Participants Only) This age division was previously 13/15. This season it changed

to 14/16. Did you agree with the inclusion of 16 year olds in this age division?

A. Yes B. No

8. (14/16 Participants Only) Would you like to see this age group continued to be for 14, 15

and 16 year olds next season?

A. Yes B. No

11    

9. (14/16 Participants Only) Would you or your child continue to play in the Harrisburg

Parks and Recreation Youth Soccer Program if the 14/16 age division stayed the same

next season?

A. Yes B. No

Why or Why Not?

Answer: _______________________________________

10. Approximately how many days did you or your child meet to have soccer practice?

A. 0-5 B. 6-10 C. 11-15 D. 16-20

11. Do you or your child feel they have learned the fundamentals of soccer this season?

A. Yes B. No

12. Did you or your child feel they were part of the team?

A. Yes B. No

13. Please feel free to provide any additional comments or feedback regarding the changes

made to the 11/13 male/female or 14/16 coed soccer divisions.

________________________________________________________________________

________________________________________________________________________

12    

The questions that are listed here will help determine how many participants felt satisfied with

the age group they or their child was in. It also provides them a chance to let us know how many

days there child was at soccer practice which also tells us how many times during the soccer

season they were outside at Harrisburg Parks. In addition, by providing a blank question at the

end for feedback, parents or participants can write anything additional they wish to include about

the program.

t we could set a definite percentage that would determine if the program would

continue or not. It would depend on the number of respondents we received and the use of other

evaluation methods such as registration numbers, observations made by staff and coordinators

and the feedback received from participants and their parents. However, I would feel confident

that if over 60% of respondents were satisfied or strongly satisfied with the program that we

should continue it. I would also conclude that if 60-70% said they plan on registering for the

program next season that we should continue with the current changes. If the response rate is

lower than that, I would be hesitant to continue without some hard thought and discussion with

other staff members, participants, upper management, parents and volunteers.

Population Description:

The population that I will be evaluating will be that of the participants and parents of the 11/13

male and female soccer division as well as those participating in the 14/16 coed soccer division

for the spring 2012 season. This population consists of 11-16 year olds and their parents who live

in Cabarrus and Mecklenburg Counties. They are predominately Caucasian and African

American, middle class citizens.

13    

It will be easy to identify this population because they will all be registered with the Harrisburg

Parks and Recreation Department and are in a computer database system which has their address,

email and phone number. For the evaluation questions, it will be easiest to reach them by the use

of email and the internet. Questionnaires will be emailed out to each participant in the 11/13 and

well as paper copies could be kept at the Parks and Recreation office. Coaches could assist by

handing out questionnaires at their practices and games. We will not use a sample, but will

collect date from the entire population.

Data Collection Techniques and M easurement Tools:

The  main  forms  of  evaluation  instruments  that  I  plan  on  using  to  evaluate  the  Harrisburg  Parks  

and  Recreation  youth  soccer  program  would  include  questionnaires,  observations  and  the  use  

of  registration  numbers.  From  the  evaluation  questions  listed  on  page  9-­‐11,  both  quantitative  

and  qualitative  date  will  be  collected.  The  quantitative  date  will  include  the  numbers  of  

participants  who  have  participated  before,  those  that  plan  on  participating  again  and  the  

number  of  participants  who  answered  yes  or  no  to  each  of  the  yes/no  questions.  The  

qualitative  data  will  be  the  information  gathered  that  describes  why  or  why  not  the  participant  

does  or  does  not  plan  to  participate  again  and  the  open  ended  question  at  the  end  of  the  

survey  which  allows  the  participant  to  provide  feedback.    

We  will  use  quantitative  data  also  when  looking  at  the  registration  numbers  for  each  age  group  

when  comparing  them  from  the  fall  of  2011  to  the  spring  of  2012.  We  will  gather  additional  

14    

qualitative  date  when  observing  the  11/13  and  14/16  age  divisions  during  practice  and  games.  

We  will  also  gain  qualitative  date  from    feedback  during  these  observations.    

We  plan  on  collecting  the  majority  of  our  data  by  the  use  of  questionnaires  that  will  be  

distributed  at  the  end  of  the  spring  soccer  season.  This  would  occur  in  late  May  or  early  June  

2012.  In  order  to  ensure  a  strong  response  rate,  an  incentive  may  be  offered,  such  as  a  small  

discount  on  a  future  registration  fee  or  a  parks  and  recreation  item  such  as  a  t-­‐shirt.  If  an  

incentive  cannot  be  offered,  parents  would  be  emailed  and/or  called  to  reiterate  the  

importance  of  providing  feedback  on  the  questionnaires.  If  coaches  distribute  and  collect  these  

questionnaires  at  the  last  few  practices,  they  can  help  the  department  ensure  they  are  

collecting  the  responses  from  the  majority  of  participants.    

Body of K nowledge:

Title IX played a huge role in bringing sports equality to both males and females. In an excerpt

taken from Title IX gender-

permitted prior to puberty and in coed sports where there are equal numbers of males and

females on both teams

par. 6).

It has been found that prior to puberty, there is no physiological reason to separate boys and girls

in sports leagues. However, sometimes it is important to split these leagues based on differences

in experience or skill level. Furthermore, when soccer first became popular in the United States

in the 1970s, it was rare that there were enough girls to form female only divisions. However,

15    

with the continued growth of soccer, especially since the 1990s, it is feasible in some leagues to

have enough males and females to split divisions by gender (Snow, 2007).

According to Snow, males and females should play each other prior to puberty if the competition

many cases, girls have not had the same experience or skill instruction

interest of the participants to split the league by gender. This is not to say that boys and girls

cannot play together in a coed league, however, the main goal would be that each group is of the

performance and skill level and of the relatively same size (Snow, 2007).

From this research, a case can be formed as to why the 11/13 age group should be divided by

gender. Through observations and parental and participant feedback, it was determined that the

size of the participants and the skill level was not the same when children reached this age

division. Therefore, it was more beneficial to both genders to divide this division by gender.

Again, this was also done because there were enough females to form their own division. For the

spring 2012 season so far, there are 55 girls registered. This is enough for five teams of eleven

which is more than enough for a league. The male division has 103 11/13 year olds registered.

Together, this totals 158 participants. In the fall 2011 season, when this age group was coed,

there were only 111 participants.

The important of youth athletics for teenagers seems like an obvious thing. Team sports help

teenagers grow physically, emotionally and socially. Team sports teach important skills such as

competition. Although the main goal of youth athletics should be for participants to learn to work

as a team, it is important to learn competition a skill most will face in life in academics, sports

children an opportunity to understand the healthy aspects of competition in a friendly

-2).

16    

Physically, youth who participate in athletics are more likely to describe themselves as being in

good shape and are more likely to make wise healthy food choices. Teenagers also benefit

including those in high school, are likely to receive an end of season trophy for participating in

the team sport. Receiving a trophy with their teammates is intended to help youth feel a sense of

This is another case for why youth athletics should be offered for teens of all ages. Previously

our leagues stopped at age fifteen. By extending that age group to sixteen, we are allowing youth

to get the benefits of youth athletics for one more year.

Evidence  Reliability  and  Validity:  

In  order  for  data  collection  procedures  to  be  considered  accurate,  they  must  be  valid  and  

reliable.  Some  participants  may  be  hesitant  to  answer  the  questionnaires  honestly  and  may  

want  to  answer  the  questions  the  way  they  think  the  evaluator  wants  or  expects  (Evaluation  

Primer,  1998,  par,  1).  One  way  to  ensure  this  will  not  occur  is  to  make  the  questionnaires  

anonymous.      

We  could  also  try  the  questionnaire  out  on  a  small  group  of  participants  and  give  them  the  

same  questionnaire  twice  at  different  times.  If  they  respond  with  the  same  answers,  this  will  

help  ensure  that  the  questionnaire  is  reliable  and  valid.  We  would  also  want  to  ensure  that  the  

questions  we  are  asking  are  providing  the  information  that  is  needed  (Evaluation  Primer,  1998).  

This  can  be  accomplished  by  allowing  input  from  the  stakeholders  and  making  sure  the  

17    

questionnaire  is  tested  by  staff  and  coordinators  to  make  sure  the  questions  are  easily  

understandable.  

Resources  Needed:  

The  biggest  resource  that  will  be  needed  will  be  staff  and  time.  We  will  need  to  have  at  least  

one  staff  person  in  place  who  can  take  the  time  needed  to  develop  a  good  questionnaire  and  to  

analyze  the  results.  This  person  will  also  need  to  be  available  to  observe  practices  and  games  

and  to  spend  time  talking  with  parents  and  coaches.  This  would  preferably  be  someone  who  

currently  works  in  the  parks  and  recreation  department  who  has  thorough  knowledge  of  the  

youth  soccer  program  and  the  changes  that  will  be  made.  A  small  budget  will  be  needed  to  

print  questionnaires.    

In  addition,  support  from  the  Town  Administrator  and  Town  Council  will  be  needed.  It  would  

not  be  worth  the  time  and  effort  to  collect  this  data  if  upper  management  and  the  elected  

officials  did  not  support  staff  in  doing  so.  However,  it  is  to  their  benefit  to  evaluate  their  

programs.  If  staff  does  not  have  the  support  of  its  stakeholders  or  the  necessary  resources  to  

complete  the  evaluation,  the  evaluation  will  be  difficult  to  conduct.  In  turn,  there  will  be  no  way  

for  the  staff  to  know  whether  to  continue  with  program  changes.  If  they  do  continue  and  

registration  numbers  drop,  the  Parks  and  Recreation  Department  could  lose  funding  and  

support  for  the  Harrisburg  Youth  Soccer  program.  Elected  officials  may  need  some  convincing  in  

order  to  allocate  staff  time  and  money  to  performing  a  large  evaluation  such  as  this,  but  

ultimately  it  could  save  the  Town  money  and  any  long  term  negative  repercussions  of  not  

evaluating  the  program.    

18    

APPENDIX A

19    

References

Evaluation primer - ensuring evaluations yield valid and reliable f indings. (1998). Retrieved

2/18, 2012, from http://www2.ed.gov.prox.lib.ncsu.edu/offices/OUS/PES/primer6.html

Payne, C. & Fogarty, K. Importance of youth involvement in sports. Retrieved 2/17, 2012, from

http://edis.ifas.ufl.edu/fy1001

Rossi, P., Lipsey, M., & Freeman, H. (2004). In Shawn L. C. (Ed.), Evaluation A systematic

approach (7th ed.). Thousand Oaks, CA: Sage Publications, Inc.

Snow, S. (2007). The coed question. Retrieved 2/17/12, 2012, from

www.socceramerica.com/article/22428/the-coed-question.html