ED 454 130

AUTHORTITLE

INSTITUTION

REPORT NOPUB DATENOTECONTRACTAVAILABLE FROM

PUB TYPEEDRS PRICEDESCRIPTORS

IDENTIFIERS

ABSTRACT

DOCUMENT RESUME

SO 032 834

Clawson, Heather J.; Coolbaugh, KathleenThe YouthARTS Development Project. Juvenile JusticeBulletin.Department of Justice, Washington, DC. Office of JuvenileJustice and Delinquency Prevention.NCJ-1866682001-05-0018p.

OJP-95-C-006Juvenile Justice Clearinghouse, PublicationReprint/Feedback, P.O. Box 6000, Rockville, MD 20849-6000.Tel: 800-638-8736 (Toll Free); Fax: 301-519-5600; e-mail:tellncjrs@ncjrs.org; For Full Text:http://virlib.ncjrs.org/JuvenileJustice.asp.Reports Descriptive (141)MF01/PC01 Plus Postage.*Art Activities; Art Education; *At Risk Persons;*Delinquency; *Delinquency Prevention; Fine Arts; IndividualDevelopment; Program Descriptions; *Program Evaluation;*Youth ProblemsGeorgia (Atlanta); Oregon (Portland); Texas (San Antonio)

The arts enrich the culture and individual livesimmeasurably, but what impact do arts-based programs have in preventingjuvenile delinquency? To address this question, the YouthARTS DevelopmentProject, with the technical assistance of the Office of Juvenile Justice andDelinquency Prevention (OJJDP), brought together Federal agencies, nationalart organizations, and a consortium of local arts agencies to develop andassess arts-based prevention programs for at-risk youth. This bulletindescribes the evaluation and its positive findings for YouthARTS programs inAtlanta (Georgia); Portland (Oregon), and San Antonio (Texas). The bulletinstates that the lessons learned by the Art-as-Work (Atlanta), Youth ArtsPublic Arts (Portland), and Urban smARTS (San Antonio) programs will helpother agencies to improve their arts programs, achieve project goals, andrecognize the importance of evaluating arts-based programs for at-risk youth.According to the bulletin, arts-based delinquency programs have a promisingfuture, and objective assessments such as these show that people are on theright path toward realizing that future. (BT)

Reproductions supplied by EDRS are the best that can be madefrom the original document.

(A

O

Vl

;

The YouthARTS Development Project.

OJJDP Juvenile Justice Bulletin; May 2001

Clawson, Heather J.Coolbaugh, Kathleen

Ift

U.S. DEPARTMENT OF EDUCATIONOffice of Educational Research and Improvement

EDUCATIONAL RESOURCES INFORMATIONCENTER (ERIC)

)2(This document has been reproduced asreceived from the person or organizationoriginating it.

Minor changes have been made toimprove reproduction quality.

Points of view or opinions stated in thisdocument do not necessarily representofficial OERI position or policy.

EST COPY AVAILABLE

U.S. Department of Justice

Office of Justice Programs

Office of Juvenile Justice and Delinquency Prevention

So

f;

QD Mbr g C.9 0 1 I c4) 'V @

The YouthARTSDevelopment ProjectHeather J. Clawson and Kathleen CoolbaughThe YouthARTS Development Project, ini-tiated in 1995, is a collaborative effortamong Federal agencies, national arts or-ganizations, and a consortium of threelocal arts agencies in Atlanta, GA; Port-land, OR; and San Antonio, TX, designedto identify, implement, and refine effectivearts-based delinquency prevention pro-grams. To support this purpose and, atthe same time, to provide much-neededinformation to the broader arts com-munity about the efficacy of such pro-grams, the National Endowment forthe Arts (NEA) and the U.S. Departmentof Justice's Office of Juvenile Justiceand Delinquency Prevention (OJJDP) part-nered in 1995 to conduct a national evalu-ation of the project.

The YouthARTS Development Project wasspearheaded by the National Assembly ofLocal Arts Agencies (NALAA), now Ameri-cans for the Arts, in recognition of theexpanding role of local arts agencies inusing the arts to address social and com-munity development issues. Using datafrom a 1995 survey of its 3,800 members,NALAA found that increasing numbers oflocal arts agencies were pursuing socialchange projects, most of which were pro-grams for youth with the common goal ofreducing problem behaviors such asschool failure, drug use, delinquency, andteen pregnancy (Mulcahy, 1996).

Despite the growing interest and invest-ment in these programs, however, little

was known about their effectiveness inactually preventing juvenile problem be-haviors. In 1996, the President's Commit-tee on the Arts and the Humanities foundthat "only a handful of studies havebegun to document the positive relation-ship between program participation andcognitive development, motivation tolearn, organization, self-perception, andresiliency" (Weitz, 1996). After a compre-hensive review of existing evaluationresearch on the impact of arts-basedprograms for at-risk youth, The RANDCorporation similarly concluded that "in-teresting arts programs abound, [but]few provide good evaluations of their out-comes" (McArthur and Law, 1996). Al-though these studies found some evi-dence to substantiate the hypothesis thatarts-based programs could foster desir-able participant outcomes, both thequantity and quality of existing evalua-tions made it difficult to verify the pro-grams' success. According to both thePresident's Committee on the Arts andthe Humanities and RAND, more con-trolled evaluations of arts programs forat-risk youth were needed to determinewhether such programs are effective and,if so, what features of the programs aremost important to their success.

The YouthARTS Development Project waslaunched with three overarching goals:to enhance program development andcapacity-building in local arts agencies, toidentify effective arts-based delinquency

3

Eft

OD V' 0 ar_

A Message From OJJDPThe arts enrich our culture and ourlives immeasurably, but what impactdo arts-based programs have in pre-venting juvenile delinquency? Untilrecently, there has been little objec-tive evidence available to determinewhether youth arts programs en-hance participant skills that reducethe risk of involvement in delinquency.

To address this need, the YouthARTSDevelopment Project brought togeth-er Federal agencies, national artorganizations, and a consortium oflocal arts agencies to develop andassess arts-based prevention pro-grams for at-risk youth.

OJJDP has provided technical assist-ance and funding in support of anational evaluation of the YouthARTSDevelopment Project. This Bulletindescribes the evaluation and its pos-itive findings for YouthARTS programsin Atlanta, GA; Portland, OR; and SanAntonio, TX.

The lessons learned by the Art-at-Work (Atlanta), Youth Arts Public Arts(Portland), and Urban smARTS (SanAntonio) programs will help otheragencies to improve their arts pro-grams, achieve project goals, and re-cognize the importance of evaluatingarts-based programs for at-risk youth.

Arts-based delinquency preventionprograms have a promising future.Objective assessments, such asthose featured in these pages, showthat we are on the right path towardrealizing that future.

prevention and intervention programs,and to disseminate information about pro-gram planning, implementation, and evalu-ation nationally. To accomplish thesegoals, three local arts agenciesthe FultonCounty Arts Council in Atlanta, GA; theRegional Arts and Culture Council in Port-land, OR; and the San Antonio Departmentof Arts and Cultural Affairs in San Antonio,TXwith support from the National En-dowment for the Arts and several founda-tions, each collaborated with local socialservice agencies beginning in the fall of1996.' The arts agencies implemented arts-based demonstration programs for youthwho were at risk of engaging in problemand delinquent behaviors. Figure 1 depictsthe overall YouthARTS program model.Although all three of the YouthARTSprogramsArt-at-Work in Atlanta, GA;Youth Arts Public Art in Portland, OR; andUrban smARTS in San Antonio, TXwerebased on the same risk- and protection-focused approach, they served differenttarget populations and provided differentarts-based activities.

National Evaluationof the YouthARTSDevelopment ProjectThe national evaluation of the YouthARTSDevelopment Project was designed as across-site evaluation with both processand outcome components. The processcomponent gathered information on pro-gram implementation and operations, andthe outcome component assessed theextent to which the three YouthARTSprograms had immediate and long-termpositive effects on the knowledge, atti-tudes, and behaviors of program partici-pants. To both conserve limited evalua-tion resources and engage local programstaff in the process, the evaluation designrelied heavily on local staff and local datacollectors hired by the programs to assistwith much of the data collection effort.

The process component of the evaluationused a qualitative approach, including areview of program documents and inter-views with staff, participants, and keystakeholders, to document program op-erations. The outcome component of theevaluation used a quasi-experimental de-sign, supported by qualitative data, todetermine the extent to which:

The programs achieved the desiredimmediate effects on participants (e.g.,increased art knowledge and improvedprogram-related skills such as commu-nication and cooperation).

Figure 1: YouthARTS Development Project Design

Community andindividual risk

factors

Conditions

Communityresources

Collaborationamong community

organizations

V

Program ActivitiesArts-based afterschool, weekend, and summer programsproviding:

o Training in:

Art (e.g., visual arts, drama, dance, graphics,photography).

Vocational areas.

Entrepreneurship.

o Opportunity to use skills.

o Contact with adult role models.

o Supervision in a safe environment.

Immediate Outcomeso Development and improvement of art skills and

program-related skills (i.e., expressing angerappropriately, communicating effectively with adultsand peers, cooperating with others, participating insessions, and working on tasks from start to finish).

o Recognition for new competencies.

Intermediate Outcomeso Improved attitude toward the future.

o Improved attitude toward school.

o Healthier attitude about drug use.o Increased positive peer associations.o Increased resistance to peer pressure.o Increased positive associations with adults.o Reduced alienation from others.o Improved self-esteem.o Improved self-efficacy.

o Increased interest in healthy activities.

o Increased community involvement.

V

Impactso Increased academic success.o Reduced problem and delinquent behaviors.

2

The programs had the desired interme-diate effects on the attitudes and be-haviors that affect delinquency andacademic performance (e.g., healthierattitudes about drug use, increasedpositive peer and adult associations,improved self-esteem).

The programs had the desired long-term impacts on juvenile delinquencyand academic performance (i.e., de-creased court referrals and increasedacademic achievement).

The goal of using a quasi-experimentaldesign was to compare the attitudes andbehaviors of program participants withthose of matched comparison-groupyouth at multiple time points to attributeimprovements in skills, attitudes, and be-haviors, with some degree of confidence,to program participation. Implementinga quasi-experimental design with theseprograms proved to be a significant chal-lenge, as described below in the evalua-tion site summaries and the section onevaluation lessons learned.

Although some processes and instru-ments were tailored for each site, theoverall evaluation used a common set ofdata collection procedures and instru-ments designed by the national evaluator.For the process evaluation, all interviewswere conducted by the national evaluator.For the outcome evaluation, the nationalevaluator provided the following sevenstandardized instruments for use by thelocal data collectors:

Art knowledge survey tailored foreach site.Participant skills assessments to docu-ment the artists' perceptions of partici-pants' art and program-related skills(e.g., expressing anger appropriately,communicating effectively with adultsand peers, cooperating with others,participating in sessions, working ontasks from start to finish).

Your Opinions Count (YOC) survey tomeasure changes in participants' self-reported attitudes and behaviors (e.g.,attitudes about school, drug use, andthe future; self-esteem; peer pressure).

Participant focus group interviewguide to collect supporting qualitativedata about program operations andoutcomes.Probation officer/caseworker feed-back survey to obtain probation offic-ers' and caseworkers' perceptions ofprogram outcomes.

Academic data form to gather aca-demic data for participants and com-parison youth.Court information form to recordbasic demographic data and courtinformation for participants andcomparison youth.

The evaluation was conducted from fall1996 through spring 1999. The overallevaluation design called for data to becollected before program participation(pretest), at the end of the program cycle(posttest), and for an identified periodafter the program cycle (14 months in At-lanta, 19 months in Portland, and 22months in San Antonio).2 The nationalevaluator provided training and detaileddata collection manuals to the local staffon how to use each of these data collec-tion instruments. The national evaluatoralso assisted the local sites in their selec-tion of procedures for choosing appropri-ate comparison groups and obtainingdata from and about them.

In fact, identifying comparison groups andrelying on local data collectors posed themost formidable challenges for the out-come evaluation. Selecting appropriatecomparison groups was difficult in each ofthe sites, primarily because of the need toidentify youth similar to the program par-ticipants (because random assignmentwas not possible) and then to retain theirinvolvement throughout the course of thestudy. Given the populations involved inthe YouthARTS Development Project (i.e.,system-involved or at-risk youth), this wasalso a time-consuming undertaking. Com-peting demands on staff time and atten-tion, turnover, and lack of data collectionexperience and/or commitment to theevaluation process all affected the timeli-ness and quality of some of the data col-lected. These constraints, coupled withthe relatively few participants served inthe Atlanta and Portland programs, re-sulted in findings that, although promisingfor arts-based prevention programs,should be considered preliminary. Thesmall sample sizes and lack of truecomparison groups limit evaluators' abil-ity to generalize the findings or attributepositive outcomes unequivocally to thearts program interventions. Nonetheless,the findings presented below for eachprogram are encouraging and support theunderlying theory that arts-based pro-grams help contribute to reduced juveniledelinquency. In 1998, the Americans forthe Arts published The YouthARTS Tool

Kit, containing information on programplanning, staff training, evaluation, andcosts and resources for arts-based preven-tion and intervention programs.' The ToolKit includes a section on how to evaluatearts-based programs based on what waslearned from the national evaluation.

Each of the programs is described in thefollowing sections, with a summary pre-sented in the table. These overviews de-scribe each program's first year of opera-tion, including startup activities, programgoals, youth served, and program activi-ties. They also describe the evaluationprocess and findings for each site, fol-lowed by important lessons learned inAtlanta, Portland, and San Antonio. Otheragencies may find these lessons helpfulas they evaluate their own arts-basedprograms for at-risk youth.

Art-at-WorkProgram DescriptionIn September 1996, the Fulton CountyArts Council entered into a collaborativepartnership with the Fulton County Juve-nile Court to implement Art-at-Work inAtlanta, GA. This program was designedto provide art instruction, job training,and literacy education to a small group offirst-time status offenders, ages 14 to 16,whose most serious juvenile offense wastruancy. Program participants were re-ferred by probation officers of youth whowere first-time truants on probation for 2years, a period consistent with the lengthof the Art-at-Work program. The programwas limited to 15 youth to allow for inten-sive one-on-one interaction between theartists and the youth. Art-at-Work wasdesigned to provide participants with:

Improved art and employment skills(e.g., goal setting, communication,sales/marketing).Opportunities to use their new skills toproduce, exhibit, and sell their ownart.Opportunities to display artwork andreceive public recognition for theirwork.

Exposure to career opportunities inthe arts.Opportunities to develop positiverelationships with adult role modelsand peers.Improved or increased self-esteem,pride, discipline, commitment,

3

Summary of YouthARTS Programs

ParameterArt-at-WorkAtlanta, GA

Youth Arts Public ArtPortland, OR

Urban smARTSSan Antonio, TX

Local collaborativepartners

Fulton County Arts Counciland Fulton County JuvenileCourt.

Portland Regional Artsand Culture Council andMultnomah County Divisionof Juvenile Justice Service.

City of San Antonio Depart-ment of Arts and CulturalAffairs, Department of Corn-munity Initiatives, and SanAntonio Independent SchoolDistrict.

Target population Truant youth ages 14-16referred by current probationcounselors.

Adjudicated youth (exceptthose adjudicated for sexoffenses) ages 14-16 referredby current probation officers.

Nonadjudicated, at-risk youthages 10-12 referred byteachers, principals, andself-referrals.

Capacity 15 youth per program period. 15 youth per unit per session(gang reduction unit, Northunit, and Southeast unit).

60 youth at each of 7 schools.

Duration 4 sessions (8-12 weeks each)with the same group ofparticipants.

1 session per unit (12 weekseach).

1 session (16 weeks) perschool per year.

Frequency 8 hours per week duringschool, 25 hours per weekduring summer.

6 hours per week. 9 hours per week.

Staffing Program director, projectmanager, lead artist/programcoordinator, professionalartists, and probationcounselors.

Program director, projectmanager, professional artists,and probation officers.

Program director, projectmanager, teacher liaison,professional artists, andcaseworkers.

Training Two-day artist trainingfocusing on child develop-ment, conflict resolution,problem solving, and class-room management.

Informal training of artistsand probation officers focus-ing on program design, goalsand objectives, backgroundof participants, and rules andregulations.

Five-day cross-training ofartists and caseworkersfocusing on working with at-risk youth, child management,curriculum development, andschool rules and regulations.

responsibility, and attitudes aboutschool and the future.

Increased prosocial behavior and re-duced alienation from others.

In addition, youth were expected to showincreased academic, vocational, and so-cial success and decreased truancy andother delinquent behaviors.

During the planning stages, the ArtsCouncil worked closely with the FultonCounty Juvenile Court. The newly hiredproject manager attended juvenile hear-ings, met with judges and probationofficers, and shadowed probation officersto learn more about the juvenile justice

system. The chief judge assigned the di-rector of intake to serve as the key liaisonto the Art-at-Work program. Over time,court officials, including the chief judge,supported the program by publicizing itsachievements, visiting classes, and at-tending exhibits of participants' work atthe end of each program session. In addi-tion, juvenile court probation officerswho referred youth to the program peri-odically conferred with the lead artist toensure that youth were participating inthe program and to help the artists ad-dress behavioral problems.

Art-at-Work staff positions included aproject manager, a program coordinator,

continued on next page

artist instructors, and a local data collec-tor. Although Art-at-Work planned to hirea social worker to assist with the pro-gram, the position was never filled. Thedirector of the Fulton County Arts Councilserved as the executive director of theprogram. Four full-time and three part-time artists were carefully selected by theproject manager and program coordi-nator from a pool of qualified artists whohad experience working with at-riskpopulations.

A week before the program started, pro-gram staff conducted an artist trainingsession. The curriculum included topics

4

Summary of YouthARTS ProgramsContinued

ParameterArt-at-WorkAtlanta, GA

Youth Arts Public ArtPortland, OR

Urban smARTSSan Antonio, TX

Approach Arts-based afterschool andsummer education and job-training program designed toserve one group of youth for2 years.

Part of each day is spentlearning art skills andproducing saleable art.Projects have includedfurniture design and appli-cation, ceramics, mosaics,photography, drama, andcomputer graphics. Part ofeach day is spent learningentrepreneurship andplanning exhibits. Studentshelp organize exhibits atthe end of each session.All proceeds support theprogram.

Arts program designed toinvolve youth in the produc-tion and administration of apublic arts project, fromdesign to production andpublic exhibition.

Youth work with the artiststwice per week during after-school hours. Probationcounselors are present ateach session to help artistscontrol problem behaviors.

Each session focuses on adifferent art medium. Themedia include printmaking,photography, poetry, drama,and videography.

A final exhibit or presentationdesigned to provide youthwith recognition for theiraccomplishments is scheduledfor the end of each session.

Afterschool arts educationprogram for youth at sevenschools.

Each school is assignedthree artists who design andimplement the art activities(i.e., dance, visual arts, drama,creative writing, and story-telling). The program alsoprovides educational fieldtrips.

Transportation home from theprogram and snacks duringprogram hours are provided.

Case management is providedby the Youth Services Divisionof the Community InitiativesDepartment. Every youthreferred to the program re-ceives a home visit froma caseworker for intake andassessment of the youth andfamily.

Incentive Students receive $5 per hourduring school year and $100per week during summervacation.

Participants from the Northand Southeast units receivetime off probation or commu-nity service hours for suc-cessful completion.

Gang reduction unit partici-pants are given a $100 incentiveand are required to participate.

N/A

Intended outcome Art skills

Vocational/entrepreneurialskills

Life skills

Prosocial behavior

Art skills

Vocational/entrepreneurialskills

Life skills

Prosocial behavior

Art skills

Vocational/entrepreneurialskills

Life skills

Prosocial behavior

such as child development, conflict reso-lution, problem solving, and classroommanagement.

In the fall of 1996, probation officers re-ferred 15 youth to the program, only 10 ofwhom were still participating regularly byDecember. Youth were not required toattend the program, and overall atten-dance was lower and more sporadic than

expected by program staff. Reasons forsporadic attendance included problemsfor teen mothers in finding affordablechildcare, schedule conflicts with otherafterschool activities, and problems withtransportation. During the evaluation,analysis of the backgrounds of youthwho regularly attended the programrevealed that several were not first-time

status offenders but rather had commit-ted more serious offenses before programparticipation.

The Arts Council allocated space withinits West End Performing Arts Centeralocation easily accessible by public trans-portation and within walking distance ofmany participants' homesfor program

5

Creating a clay sculpture in the Art-at-Workprogram in Atlanta, GA.

activities. The Art-at-Work program activi-ties included arts instruction in variousart disciplines, a literacy component, andjob skills training. A 12-week art sessionwas provided during the fall-winter schoolterm, followed by a 12-week spring ses-sion, with a break between sessions thatcorresponded with winter vacation forstudents. Each of the 12-week sessionsoperated for 2 hours after school onTuesdays and Thursdays and 4 hours onSaturdays. During the summer, an inten-sive 8-week session met for 5 hours perday, Monday through Friday. Although notpart of the original program design, anadditional 8-week session, operating onthe same schedule as the 12-week ses-sions, was added to the program in Mayto keep youth involved during the transi-tion from the end of the school year untilthe start of the summer session. On alter-nate Saturdays throughout the program,visiting artists provided additional artsinstruction or field trip opportunities forthe participants. The field trips includedvisits to the Atlanta College of Art, theNexus Press, Seven Stages and other localtheaters, and art exhibits. The youth par-ticipants, who were called apprentice art-ists, were paid $5 per hour during theschool year and $100 per week during thesummer for participating in the classesand producing marketable art.

Throughout the program year, the youthwere divided between two studios, each ofwhich focused on a particular art discipline.Half of the participants were assigned toeach studio, and halfway through the ses-sion, the two groups switched studios.Separating the participants into two stu-dios decreased the apprentice-instructorratios, and switching between studios pro-vided youth with an opportunity to workwith new media and different instructors.The art disciplines covered during theyear were graphic design, drawing, paint-ing, mosaics, sculpture, artist chairs, mu-rals, photography, and drama. In the stu-dios, each of the apprentices developedmultiple pieces of art to display for sale atan exhibition at the end of the cycle and atleast one piece that he or she could takehome. Proceeds from the sale of the arthelped cover the program's operatingcosts, including stipends. This process ofproducing and selling art was designed toteach the participants practical businessconcepts, including production goals, in-ventory, and marketing.

Evaluation Activities andFindingsThe outcome evaluation in Atlanta wasalmost immediately hampered bydifficulties in identifying an appropriatecontrol group. A control group could notbe formed from the 15 youth initiallyidentified as program candidates becausethey all accepted the invitation to partici-pate. Probation officers were then asked toidentify another group of youth who weresimilar to the program participants withrespect to age, sex, race, and court involve-ment to serve as the control group. Theseyouth were offered a modest stipend toparticipate. Because only three youth ac-cepted, evaluators decided not to includethem in the evaluation. Probation officerswere asked to identify another group ofyouth for whom academic and court datacould be collected from existing schooland court records; they identified 12 youthfor the control group. Of the 10 core pro-gram participants, complete evaluationdata were collected for 7. Thus, the partici-pant and control groups for whom evalua-tion data were available included 7 and12 youth, respectively. Moreover, only

People underestimate the power of art and the power of youth to express them-selves. Art-at-Work gives youth a voice.

Art-at-Work Program Manager

Art gives you a way to express yourself. You can get your feelings out through yourart without hurting anyone.

Art-at-Work Participant

I think we often underestimate youth. I was surprised at how kids with no art experi-ence could not only increase their knowledge of art concepts but apply what theylearned in a critique of a mural.

Art-at-Work Artist

I learned how to work with other people. The people skills I learned in the programhave helped me. I am working and want to go to business school.

Art-at-Work Participant

If I wasn't in the program I probably would have gotten pregnant again and droppedout of school. Now I'm still in school and doing good.

Art-at-Work Participant

Since I started the program, I've started back to school regularly. I've started beingresponsible.

Art-at-Work Participant

If you expect very little from youth, that is what you will get! We expected high-quality [art] products, and that is what the youth produced. They more than met ourexpectations.

Art-at-Work Artist

6

archival data were collected for the con-trol group youth; they did not completethe YOC survey. Participant survey andarchival data were augmented with inter-view data from the art staff about theprogram participants and from the partici-pants themselves.

According to the end-of-program partici-pant skills assessment, all seven of theyoung artists had gained the skills neces-sary to produce quality art, had producedart, and had received public recognitionfor their work. The art staff also report-ed remarkable improvement in the parti-cipants' enthusiasm and appreciationfor art over the course of the program.As depicted in figure 2, all seven youthshowed improvement in four of theprogram-related skills assessed by theevaluation: art skills, cooperating withothers, participating, and communicatingeffectively with their peers.

Responses from the YOC participant sur-vey reflect improvements in attitudes to-ward school, attitudes about drug use,and the frequency with which the youthengaged in delinquent behaviors. Feed-back from the probation officers tends tosuggest Art-at-Work had a positive impacton the attitudes and behaviors of youth.Probation officers noted that youthwho participated in the program

Refurbishing a chair in the Art-at-Workprogram in Atlanta, GA.

Figure 2: Program-Related Skills Exhibited by Art-at-Work Participants

Art

Angerexpression

Communicationwith adults

Cooperation

Participation

Communicationwith peers

Task completion

Note: n=7 participants.

0 20 40 60 80

Percentage of Youth Exhibiting Skills

Preprogram El Postprogram

100

demonstrated increased self-esteem andan increased sense of accomplishmentand pride and showed improved relation-ships with their peers and family mem-bers. They also believed the program wasvaluable because, in addition to being afun, safe, and challenging afterschool ac-tivity, it enhanced future opportunities forsome of the participants by providingskills that will help them academicallyand vocationally. Although the limitedacademic data collected for the twogroups preclude drawing any meaningfulconclusions about academic achieve-ment, participants indicated in interviewsthat their participation in the programhad a positive influence on their aca-demic performance. Two participants de-cided to continue their education beyondhigh school as a result of the arts pro-gram, with one attending business schooland the other enrolling in art classes atcommunity college.

Despite the disappointing quality andquantity of evaluation data from the Art-at-Work program, even cautious inter-pretation of the information provided bythe youth participants and program staff

7

suggests the program was beneficial.Recognizing the limitations of this evalu-ation but still committed to arts-basedprevention, Art-at-Work is continuing toevaluate its program.

Plans for the FutureAfter completion of the first Art-at-Workprogram period, the project manager helda meeting to discuss the findings from theevaluation with partners from the com-munity, including judges and probationofficers. In addition, former program par-ticipants and program staff were invitedto provide input. The focus of the meetingwas to reassess the program design basedon lessons learned from the evaluation.As a result, the group decided to makeseveral changes.

Greater attention to recruitment.Although the program's original designcalled for provision of services to truantyouth, the court data collected for theevaluation verified that, indeed, youthwho participated in the Art-at-Workprogram were more involved with thejuvenile justice system than the program

9

originally anticipated. Under the new pro-gram design, greater attention is given torecruitment and assessment of partici-pants with minor delinquent offenses in-cluding status offenses such as truancy.Program staff determined that the pro-gram is not equipped to address the moreserious offenders who tend to have mul-tiple problems such as substance abuseand dependency issues.

Limited duration of the program. Ratherthan expecting youth to commit to a 2-year period, program staff have rede-signed the program to operate for 1 year.The attrition rates suggested that a 12-month period was "long enough" foryouth. The demands on youth's time (e.g.,afterschool activities, full-time work, fam-ily commitments) and changes in interestmade a 2-year commitment unrealistic formost youth.

Continued evaluation. The project man-ager and others involved with the Art-at-Work program saw clearly how importantevaluation results were for obtaining on-going program support and funding andfor improving the overall quality of theprogram. Recognizing many problemswith the initial evaluation of Art-at-Work(e.g., missing data, lack of academic data,poor selection of comparison group), theproject manager approached Caliber As-sociates to conduct a new evaluation tobegin in late September 2000. This evalua-tion is in progress. Caliber is currentlyaddressing the problems of the initialevaluation with the project manager, andplans are in place for conducting a rigor-ous evaluation using either random as-signment or a true matched comparisondesign. Plans call for building the evalua-tion into the program's overall design toallow for accumulation of data on moreyouth participants over several years,which will permit program staff to makestronger statements about the signifi-cance of the arts program in changingyouth attitudes and behaviors. This ap-proach will not compromise the ability ofthe artists to continue working with smallnumbers of youth on a one-on-one basis.

Youth Arts Public ArtProgram DescriptionIn 1996, the Portland Regional Arts andCulture Council, in collaboration with theMultnomah County Division of JuvenileJustice Services and other local arts

organizations, initiated the Youth ArtsPublic Art program in Portland, OR, withfunding from Percent for Public Art.'Youth Arts Public Art was designed toserve small groups of youth ages 14 to 16who were on probation in the Portlandjuvenile justice system for any status ordelinquency offense except sex offenses.The goal of the program was to achievethe following participant outcomes:

Improved art skills.

Increased awareness of art educationand careers.

Recognition of new skills.

Positive relationships with adult rolemodels and peers.

Improved self-esteem and attitude to-ward the future.

Improved social skills (e.g., communi-cation, teamwork, empathy).

Improvement and gains in these areaswere expected to enhance academic andsocial success and reduce involvement indelinquent behaviors.

The Regional Arts and Culture Councilworked closely with the Division of Juve-nile Justice Services and arts organiza-tions to develop Youth Arts Public Art,determine its goals and objectives, anddefine the roles of the collaborative part-ners and their staff. Staff positions in-cluded a project manager, probationcounselors, artist instructors, and localdata collectors. The director of the Port-land Regional Arts and Culture Councilserved as program director. The councilwas responsible for daily program

operations and hiring and training artistsand other program staff, and it allocatedPercent for Public Art funds to providemost of the art supplies (e.g., video equip-ment, costumes) and pay the artist in-structors. The Division of Juvenile JusticeServices provided probation counselors torefer youth to the program, provide casemanagement, and assist the artists in con-ducting the workshops. In addition, theprobation counselors often provided trans-portation for participants. The Division ofJuvenile Justice Services also providedsnacks and gift certificates as incentivesfor youth participation. Other local artsorganizations allocated space in their fa-cilities for program workshops, exhibits,and performances and provided videoequipment and costumes for theaterproductions.

Artists and counselors received orienta-tion training before the program sessions.The training focused on program design,goals and objectives, background of par-ticipants, and rules and regulations. Inaddition to this training, several artistsattended an orientation for probationofficers that was conducted by the Divi-sion of Juvenile Justice Services to helpthem better understand the programparticipants and the role of the probationcounselors.

The Regional Arts and Culture Counciland the Division of Juvenile JusticeServices identified three probation unitsto participate in the program: Southeast,North, and the Gang Resource Interven-tion Team (GRIT). The Southeast andNorth units included youth from at-risk

We need to teach our youth that all behaviors have consequences, not just the badbehaviors. Youth Arts Public Art provides youth with new art skills, gives them achance to use those skills in developing quality art, and gives youth a chance tosucceed and to be rewarded for their positive behavior.

Youth Arts Public Art Probation Officer

I learned what I can do with photography. I'm pretty good with the camera. I hope toget a job taking pictures for a newspaper or magazine.

Youth Arts Public Art Participant

In the beginning, I wanted to be sent back to court. I didn't want to have anything todo with this program. Now I'm very involved and consider myself a leader.

Youth Arts Public Art Participant

A lot of people got to see my work. It made me feel very proud of what I had done,and that doesn't happen very often.

Youth Arts Public Art Participant

8

10

O

Creating monoprints in the Youth Arts Public Art project in Portland, OR.

areas in Portland who were on probationfor status and delinquent offenses. TheGRIT unit included youth identified asgang members or involved in gang activ-ity. Probation officers in each unit re-ferred 15 youth from their probationcaseloads to the program, based on theirterm of probation (i.e., the youth had tobe on probation for the duration of theprogram). Although 45 youth were re-ferred (15 from each unit), a total of 37actually participated in the sessions (12from North, 11 from Southeast, and 14from GRIT). Attendance was not manda-tory for youth from the Southeast andNorth units, but once the youth agreed toparticipate in the program, they were ex-pected to commit to the entire 12 weeks.To promote attendance, these units of-fered time off probation and/or commu-nity service hours in exchange for activeparticipation. For the GRIT unit, atten-dance was mandatory, and a $100 gift cer-tificate was offered as an incentive forparticipating in the program. Despite thisincentive, attendance by the GRIT unitreferrals was low and sporadic.

Youth from each unit participated in a 12-week session that met for 2 hours, 3 daysa week. The program sessions focusedon different art forms, selected by theproject manager and the supervisor ofeach probation unit. The North unit re-ceived instruction in photography (e.g.,use of equipment, style) and poetry (e.g.,presentation, forms of poetry), the South-east unit studied videography (e.g., use ofequipment, techniques, sound, editing),and the GRIT unit studied theater (e.g.,set design, costumes, script writing,movement). In each session, the youth

worked with the artists and counselors insmall groups to develop quality public artand to prepare for a final exhibition orperformance. The North unit participantspublished a book of their photographyand poetry that focused on aspects oftheir everyday lives. The Southeast unitdeveloped a short informational videoabout new juvenile justice legislation af-fecting youth in Portland, including inter-views with local politicians and judges.The GRIT unit performed a play that de-picted life in a gang and that involvedboth music and dance.

Evaluation Activities andFindingsIn January 1997, probation counselors foreach of the three probation units selectedyouth for the comparison group fromtheir existing caseloads. They attemptedto identify youth who were similar to par-ticipants with respect to age, sex, andother important factors (e.g., court his-tory, living in high-crime areas). Althoughthe counselors invited 45 youth to volun-teer for the study, only 22 agreed to par-ticipate. These youth were given a modeststipend for their participation.

As in Atlanta, data were not uniformlyavailable for all youth in the two groups.Complete preprogram and postprogramdata from the YOC survey were availablefor only 19 of the 37 core participants and13 of the 22 comparison youth. Program-related skills assessments were availablefor 21 participants. Only nine programparticipants were available to participatein data collection for the followup evalua-tion. Participant survey and archival data

9

were augmented with interview data fromthe staff artists and with data from focusgroups of the participants themselves.

Based on the end-of-program participantskills assessment, all 21 of the participantshad gained the skills necessary to producequality art, had produced art, and had re-ceived public recognition for their work.According to the program staff, the partici-pants took great pride in their work andlooked forward to the public recognitionfor their achievements. In fact, for theGRIT unit, it was only after the artiststhreatened to cancel the final productionbecause of poor rehearsal attendance thatthe group became serious about theirwork and participation. Every youth in theGRIT unit was present for the final publicperformance. As shown in figure 3, most ofthe youth also showed improvement in allprogram-related skills during the programperiod, particularly in their ability to co-operate with others. The project managerand artists also observed improvement inthe participants' ability to work as a teamand form new friendships.

Findings from the YOC survey were alsopromising. Although the number of pro-gram participants and comparison youthwho completed the survey was relativelysmall, there were noticeable improve-ments in participants' self-reported in-volvement in delinquent behavior duringthe program period. Additionally, a greaterproportion of participants than compari-son youth showed improvement in theirattitudes toward school, resistance topeer pressure, and self-efficacy. These find-ings were supported by focus group datafrom the nine participants interviewed atthe 19-month followup. These youth saidthe program taught them self-respect, waysto get along with others and work in a teamenvironment, and, especially, the impor-tance of taking responsibility for their ac-tions. In addition, they said the programhelped them recognize their talents andmost importantopened their eyes toopportunities and career options.

More sobering was the fact that, al-though most of the youth acknowledgedthat dealing and using drugs were wrongand illegal (as reflected in their surveyand focus group responses), many con-sidered these activities a necessityformoney or to escape everyday life. Theseyouth were very open about their daily"reality," which for many involved guns,violence, poverty, homelessness, andfear. For these youth, involvement in theprogram was very important because, as

Figure 3: Program-Related Skills Exhibited by Youth ArtsPublic Art Participants

Art

Angerexpression

Communicationwith adults

Cooperation

Participation

Communicationwith peers

Task completion

Note: n=21 participants.

20 40 60 80 100

Percentage of Youth Exhibiting Skills

Preprogram Postprogram

one youth explained, "It gave us a safeplace to spend time and it provided uswith an opportunity to feel good aboutourselves."

Because of difficulty in locating academicdata for participant and comparisonyouth, especially those who did not regu-larly attend school or who attended alter-native schools, little can be said aboutthe impact of the program on academicachievement. Anecdotal information fromparticipants suggests, however, that theprogram was successful in changing howsome of the youth felt about education.As a result of their program participation,some youth reported that they are gettingtheir priorities straight and trying to dobetter in school.

Although limited by the quality and quan-tity of evaluation data, findings for theYouth Arts Public Art program suggestthe program may benefit some youthinvolved in the juvenile justice system.Using what has been learned about pro-gram implementation and impact, programstaff are refining the program to empha-size what works and change what doesnot work. Additionally, reassessing which

outcomes are realistic from a 12-weekarts-based program with high-risk youthhas become a priority for program staffduring the redevelopment phase. Theyacknowledge that the problems facingmany of the youth involved in the pro-gram (e.g., neglect, physical abuse, home-lessness) are more serious and requiregreater intervention than Youth Arts Pub-lic Art can offer. Staff are also reevaluatingthe appropriate target population to beserved by the program.

Plans for the FutureThe Youth Arts Public Art program usedthe evaluation results and lessons learnedfrom the first year of operation to makeimportant changes to the program.

Greater awareness of implications of artmedia being selected. The project man-ager and staff learned that they need tobe more careful when selecting the artmedia to use with different populationsof youth. For example, they discoveredthat the use of theater with youth in thegang unit, who are guarded and reluctantto put themselves in vulnerable situationson stage or off, was probably a poor

match. A less threatening art form, suchas the videography project, might havebeen better suited for this group of high-risk youth.

More emphasis on recruitment and at-tendance. Program staff determined thatattendance rates would need to improvefor the program to be successful. By serv-ing less serious offenders and targetingyouth who express an interest in the arts,staff hope to have greater success atmaintaining participants' interest and in-creasing involvement in the program.

Greater collaboration across sessions.The project manager is working on thedevelopment of increased collaborationamong the various youth groups involvedin the different sessions, which would ex-pose youth to more art forms without sig-nificantly increasing program cost. Addi-tionally, the joint project would provideopportunities for youth from diversebackgrounds to work together toward acommon goal.

Urban smARTSProgram DescriptionThe City of San Antonio Department ofArts and Cultural Affairs initiated the Ur-ban smARTS afterschool program in 1993to prevent high-risk students, ages 10 to12, in San Antonio, TX, neighborhoodsfrom engaging in delinquent behaviors.The Department of Arts and Cultural Af-fairs joined the YouthARTS DevelopmentProject in 1995 and began operating Ur-ban smARTS as a YouthARTS program in1996. Teachers and counselors at eachparticipating school from the San AntonioIndependent School District referred tothe program primarily sixth grade stu-dents who were experiencing academicfailure, had poor school attendance, andengaged in antisocial behavior. UrbansmARTS provided afterschool art work-shops that were designed to achieve:

Improved social skills (e.g., com-munication with peers and adults,teamwork).

Improved academic performance andcommitment to school.

Improved art skills.

Recognition for new skills.

Positive relationships with adult rolemodels and peers.

Improved self-esteem and attitudetoward the future.

10

12

Artwork from the Urban smARTS program inSan Antonio, TX.

These outcomes were expected to leadto academic and social success and de-creased delinquent behaviors.

Since the inception of Urban smARTS in1993, the City of San Antonio Departmentof Arts and Cultural Affairs, the Depart-ment of Community Initiatives, and theSan Antonio Independent School Districthave worked together to develop the Ur-ban smARTS program curriculum and de-fine the roles and responsibilities of thepartnering agencies and program staff.Staff positions included a project man-ager, teacher liaison, caseworkers, andartist instructors. The director of the SanAntonio Department of Arts and CulturalAffairs served as program director, andthe department was responsible for hiringand training staff. The Department ofCommunity Initiatives provided casework-ers for case management and for facilita-tion of prevention-oriented discussionsduring the program. The school districtreferred students from seven schools tothe program and provided teacher liaisonsto help the artists conduct the workshops,space in the schools for the art work-shops and performances, transportationfor participants who lived farther thanwalking distance from the school, andsnacks during the afterschool workshops.

The program director assigned a teamconsisting of three artists (including onelead artist), four caseworkers, and oneteacher liaison to each of the sevenschools. The team was responsible formaking decisions about program format,discipline, and schedules.

Before the start of the Urban smARTS pro-gram in 1996, a 5-day training session wasoffered to all artists and caseworkers.This training focused on issues related toworking with at-risk youth (e.g., behaviormanagement), curriculum development,and school rules and regulations. It alsoprovided an opportunity for the artistsand caseworkers to learn more abouteach other's disciplines and to developstrategies to integrate the art andprevention-focused curriculums.

Although more than 400 youth were re-ferred to the program in the fall of 1996(approximately 60 youth per school), only112 participated regularly. Youth droppedout of the program because of lack of inter-est, family obligations, involvement inother programs, and/or problems inschool. Although disappointing, the attri-tion of students from the program resultedin more one-on-one interaction betweenstaff and the youth who did participate,

which proved helpful for the artists, case-workers, and participants. Although some-what inconsistent in the beginning, atten-dance and involvement in the workshopsincreased as the 112 youth became moreinvolved in the activities and bonded withthe artists and other participants.

Urban smARTS conducted 3-hour after-school art workshops on Tuesdays,Wednesdays, and Thursdays for 16 weeksat each of the seven schools. The primaryart forms offered at each school weredrama, dance, and visual arts. During thefirst few weeks of the program, the partici-pants spent part of each 3-hour workshopwith each of the artists to learn about thedifferent art forms. After several weeks,the youth were divided into groups basedon their level of interest in these art forms.The youth then worked with the artist todevelop skills and produce their own art.All of the youth also helped prepare thefinal public performance and/or exhibitthat concluded the 16-week session. Dur-ing 13 of the program's 16 weeks, case-workers provided weekly 30-minute ses-sions in a group setting in which theydiscussed prevention-oriented topicssuch as decisionmaking, conflict resolu-tion, self-respect, and substance abuse.

Children need to experience a sense of accomplishment and be recognized for theirefforts. This program enables kids to succeed at something and provides thempositive feedback.

Urban smARTS Artist

Being in the program has made me a better writer. I was never interested in writingbefore this program. Now I love to write.

Urban smARTS Participant

I had never performed in front of a crowd before. It made me feel good and proud ofwhat I had done.

Urban smARTS Participant

Before I started the program, I was hanging out with the wrong crowd and gettinginto trouble. Now I do better in school and see myself as an artist.

Urban smARTS Participant

You hear people say art saves lives. That may sound funny, but programs like UrbansmARTS keep youth off the streets and away from dangerous situations, like gangactivities. Our youth feel safe in our program and have positive outlooks on the future.

Urban smARTS Executive Director

In October 1998, the Urban smARTS program joined nine other youth arts programsat the White House in being recognized with the "Coming Up Taller Award," an awardgiven to outstanding arts and humanities programs for young people.

11

13

Evaluation Activities andFindingsConducting the evaluation of the UrbansmARTS program proved to be a signifi-cant challenge, complicated by logistical,legal, program management, and method-ological issues. Of the seven participatingUrban smARTS schools, only five partici-pated in the evaluation (one lacked thecase management component, and theother was excluded because it was an el-ementary school). Project managers ex-cused two of these five schools from hav-ing to identify a control group, believingthat, as new Urban smARTS programs,they should not be burdened with thisrequirement. In the three schools wherecontrol groups were identified, teacherliaisons selected sixth grade classes withteachers willing to administer the YOCsurvey and participate in subsequentdata collection activities. Recognizingthat these comparison youth were neitherrandomly assigned nor matched on keycharacteristics, project managers none-theless believed the selected youth wouldbe comparable to the program partici-pants because most of the youth at eachschool shared similar demographic char-acteristics and faced similar risk factors.This proved to be true. The participantand comparison youth were similar onkey factors. Ultimately, only one of thefive schools participated in the followupevaluation, which included only programparticipants (i.e., no comparison group).

For various reasons, evaluation data forUrban smARTS were somewhat limited.Late and uneven administration of somedata collection activities, combined withmisplacement of many postprogram sur-veys by the project manager, circum-scribed the analyses that could be con-ducted. Although complete preprogramand postprogram YOC survey data wereavailable for all 112 regular program par-ticipants, data were provided for only29 comparison youth. Art knowledge,academic, and skills assessment datawere available for only 93, 86, and 78 par-ticipants, respectively. Followup datawere available for only 22 participantsfrom one school. Survey and archivaldata were augmented with interview datafrom the artist staff and with focus groupdata from program participants.

Participants' gains in art knowledge weredifficult to determine because of lateadministration of the preprogram artknowledge survey. Testimonies from par-ticipants and art instructors suggest,

however, that the youth did learn a consid-erable amount about the different art formsthat were taught, and, more important,youth had a greater interest in and appre-ciation for the arts after being involved inthe program. The desire expressed byyouth for the Urban smARTS program to beoffered in sixth grade and ninth grade, bothtransitional years for most students, wasevidence of their enthusiasm for theprogram.

Findings from the end-of-program partici-pant skills assessment show that mostparticipants demonstrated program-related skills throughout the program,and youth who did not demonstrate thenecessary program skills at the start ofthe program mastered these skills by theend. As shown in figure 4, the greatestchanges were reported in two skill areas,working on tasks from start to finish anddemonstrating art skills necessary to pro-duce quality artwork (i.e., skills identifiedby artists as essential, given the art formbeing taught). Many artists reportedthat most participants showed markedimprovement in their overall social andart skills.

The results of the YOC survey support theeffectiveness of the Urban smARTS pro-gram as a prevention initiative. Most par-ticipant and comparison youth report-ed having favorable attitudes and engagingin positive behaviors at program startup.During the program period, however, moreparticipants than comparison youthshowed improvement in their attitudesabout school and in their behavior (i.e., lessself-reported involvement in delinquentbehavior). Additionally, participants weremore likely than comparison youth to sus-tain their appropriate attitudes about druguse, resistance to peer pressure, and senseof self-efficacy. The fact that program par-ticipants could maintain their high scoresover the course of the program better thancomparison group members suggests thatthe program may help buffer these youthagainst many of the risk factors that canaffect them at this age. Official court re-cords also supported this finding. A re-view of the records found no court involve-ment before or during the program periodfor any of the 112 participants. Only twoparticipants had committed an offenseduring the 22-month followup period (bothcases of shoplifting).

Figure 4: Program-Related Skills Exhibited by Urban smARTSParticipants

Art

Angerexpression

Communicationwith adults

Cooperation

Participation

Communicationwith peers

Task completion

Note: n=78 participants.

0 20 40 60 80

Percentage of Youth Exhibiting Skills

Preprogram E Postprogram

100

12

4

Based on the survey data available fromparticipants at one school at the 22-month evaluation followup, the youthmaintained their appropriate attitudesabout drug use, positive peer associa-tions, resistance to peer pressure, fre-quency of delinquent behaviors, andpositive relationships with adults aftercompleting the program. These findingswere echoed during focus groups with the22 followup participants. Most of theseyouth reported that they felt more ma-ture, worked harder at school, and triedto be better people as a result of beinginvolved in Urban smARTS.

During these focus group discussions,students reiterated their enthusiasm andsupport for the Urban smARTS program.They enjoyed their participation and theopportunity to learn about the arts. Inaddition, they said the program taughtthem how to appreciate things aroundthem, get along better with others, dealwith emotions in a constructive way, andcommunicate through music and the arts.Several participants indicated the pro-gram had taught them to be more self-confident and helped them believe inthemselves. Several students said thatthey "hung out with the wrong crowd"before participating in the program andthat the program had helped them to stayout of trouble by keeping them off thestreets and giving them something to doafter school.

These findings suggest that the UrbansmARTS program was successful in meet-ing its goal of keeping the youth partici-pants engaged in positive afterschool ac-tivities and preventing their involvement indelinquent behaviors. Program casework-ers concurred, noting that during the stu-dents' participation in the program, theirbehavior and attitudes improved and theybecame more respectful of others.

Analysis of the academic data revealedless promising findings. Average grades forparticipants did not change during or afterthe program. This was not surprising tomost of the program staff, however, be-cause the Urban smARTS program did notfocus on core academic subjects such asmath, science, and English. Some evidencesuggests that program participation mayhave helped improve school attendance.Program staff are reconsidering the rela-tionship between participation in UrbansmARTS and academic achievement andhope to refine the intended academicoutcomes of the program in the future.

Plans for the FutureAfter completing the transition of theUrban smARTS program administrationfrom the San Antonio Department of Artsand Cultural Affairs to the Department ofCommunity Initiatives and hiring a perma-nent program director, program staff ex-amined the evaluation results and fo-cused their attention on improvingprogram implementation.

Expansion of the program in schools.Based on the positive findings from theevaluation and increased local interest inthe program, the Department of Commu-nity Initiatives has expanded the programinto other local school districts and isconsidering expanding into the San Anto-nio high school system. Teachers, princi-pals, and former participants have ex-pressed interest and identified a need toexpand the program into high school toprovide services during ninth grade.

Redistribution of resources. Expansion ofthe program into more schools has al-ready resulted in the need to redistributeresources. For example, to extend thereach of the program to more schools,one artist works in each school ratherthan the multiple artists who worked ineach school during the evaluation period.Although the implications of this changeare unknown, it was necessary given thecurrent availability of resources.

Greater collaboration among key agen-cies and organizations. The program di-rector is focusing on collaboration amongthe caseworkers, artists, and schoolpersonnel (i.e., teachers, principal, coun-selors). Improving the working relation-ships among the parties is expected toimprove program recruitment, retention,and operation. With greater attendanceand better coordination of services,youth are more likely to experiencepositive change.

Lessons LearnedDo arts-based programs prevent or re-duce delinquent behavior among youth?This key question remains only partiallyanswered by this evaluation. Although itproduced considerable evidence to sup-port the hypothesis that such programscan contribute to the avoidance orreduction of delinquent behavior, theevaluation suffered several problems thatlimited the ability to definitively answerthe question. However, the lessonslearned from this evaluationabout both

the programs themselves and the processof evaluating themcan be immenselyvaluable to the design, implementation,and evaluation of other youth-focusedarts-based efforts.

Evaluation LessonsEvaluating any social service program ischallenging, especially when the programdeals with populations who are at risk ofbecoming delinquent or already in trou-ble. Evaluating programs that deal withjuvenile delinquency poses special chal-lenges because they serve adolescents, apopulation that requires special attentionto data collection protocols, such as ob-taining parental permission for their chil-dren to participate in interviews or sur-veys and gaining the cooperation ofsystem-wary youth. Moreover, these pro-grams are often modestly funded, withstaff stretched in many directions, work-ing with too many youth in too little time.In order for an evaluation component tosucceed, there needs to be careful atten-tion to the design, timing, and resources.Following are the lessons learned regard-ing several interrelated issuescontrolgroups, sample size, data collection, andprogram stability.

Control groups. Answering questionsabout the effectiveness of delinquencyprevention and intervention programsrequires the most rigorous evaluationdesign possible. An experimental designwith random assignment of youth intoparticipant and control groups, the designthat best allows for testing whether theprogram "causes" changes in partici-pants, was originally planned for thisevaluation. Because of limited recruit-ment (i.e., not enough youth to use ran-dom assignment) and concerns amongsome of the program partners about theethics of not providing services to all eli-gible youth, a quasi-experimental designwith matched comparison groups wasadopted. To enable evaluators to at-tribute changes in participants' attitudesand behaviors to the program, the com-parison group had to be similar to thegroup of participant youth from the startof the evaluation in several key factors(e.g., age, race, grade level, history ofdelinquency, attitudes, and behaviors).Although attempts were made to matchyouth on key characteristics in Atlantaand Portland, the participant and controlgroups had some important differences(e.g., history of delinquency). Thesedifferences and the small sample sizes

13'

15

made it difficult to attribute with cer-tainty a direct relationship between pro-gram participation and the observed out-comes. Implementing rigorous evaluationdesigns requires considerable planningand, equally important, commitment oftime and resources to ensure that the in-tegrity of the design is maintainedthroughout the course of the evaluation.

Sample size. To some extent, the smallsample sizes for the evaluation of thethree YouthARTS programs were due toprogram design and were thus unavoid-able. An important feature of many artsprograms, like Art-at-Work in Atlanta andYouth Arts Public Art in Portland, is thesmall youth-to-artist ratio. Although thisallows for quality one-on-one instructionof the participants, it creates a challengefor the evaluator. Small sample sizes pre-clude testing for statistical significance(determining whether observed changesoccur by chance or because of the inter-vention). Additionally, small sample sizesmake it difficult to generalize the findingsto the broader population of at-risk youth.Although an evaluator cannot control thesize of a program, steps can be taken toavoid small sample sizes resulting frommissing data. Because data were not uni-formly collected for all youth at all datacollection points, many youth had to beexcluded from the analyses. Every effortmust be made to obtain complete datafor all youth involved in the program andcomparison groups.

Data collection. One of the most criticalaspects of any evaluation is data collec-tion. Even the most rigorous design willnot provide meaningful results withoutthe necessary data. Every evaluationfaces resource-driven tradeoffs, however,and data collection is often an area wherecompromises are made. The YouthARTSevaluation used program staff, supportedby local data collectors hired by eachsite, to collect the YouthARTS evaluationdata.

Even with detailed training and how-tomanuals provided to assist the local staffwith their data collection tasks, the datacollection effort suffered. Program staffwere very busy, focused primarily on pro-gram operations and service delivery.Many of the local data collectors wereinexperienced and, despite guidance andmanuals, were often unable to managethe challenges posed in collecting thenecessary academic and court data. Fre-

quent technical assistance throughout thedata collection process could help avoidsuch data collection problems in similarsituations in the future.

Program stability. Outcome evaluationsseek to answer the question, Did this pro-gram work? Implicit in answering thequestion is an understanding of what"this program" is. If the program itself isyoung and still undergoing developmentalchanges (as the YouthARTS programswere, especially in Atlanta and Portland),it is difficult to determine which programworked or did not work. Moreover, youngprograms face enormous challenges withhiring and training staff, program logis-tics, recruitment, and day-to-day opera-tions. These evolving programs resultedin two key problems for the evaluation:

First, it was difficult for project manag-ers to oversee the evaluation, ensurecompliance with data collection re-quirements, and deal with the day-to-day demands of program operations.

Second, because of the first problem,both identifying the intervention itselfand ensuring that the data collectioninstruments were measuring the appro-priate outcomesthose that reason-ably could be expected to occur as aresult of the interventionweredifficult.

Future evaluations of arts-based pro-grams should be done on those programsthat have reached a level of stability thatensures the intervention and its desiredoutcomes are well defined. Also, if pro-gram staff are to be responsible for datacollection, they must have the capacityand training.

Program LessonsMany important lessons were learnedabout planning and implementing arts-based juvenile delinquency prevention/intervention programs. The experiencesof these three YouthARTS programs pro-vide useful insights for future programoperation and evaluation. Informationfrom both the process and outcomeevaluation components helped programstaff identify common factors that led tosuccessful program implementation.

Collaboration. All of the programs incor-porated some form of collaborative pro-cess. Participants noted that having avoice in establishing program rules,regulations, and workshop activities and

having frequent opportunities to produceand exhibit their art were factors thathelped them engage in the program. Pro-gram staff recognized the need for im-proved collaboration among program part-ners (e.g., school district, juvenile justiceagency, art agency) and the importance ofidentifying the roles and responsibilities ofeach partner. Lack of "buy in" and under-standing of the program and the evalua-tion from the beginning resulted in unex-pected complications (e.g., lack of accessto data, difficulty accessing facilities, over-extension of staff). Additionally, programstaff believed that the partnership neededto be broadened to include other youth-serving agencies. They quickly learnedthat many of the youth they were servinghad problems beyond the scope of the artsprogram and that any impact the programmay have had on improving behavior wasreduced by other circumstances that af-fected the lives of the participants. Widen-ing the circle of collaborative partnerswould have improved the program's abilityto refer youth and their families to otherservices in the community.

Skilled, qualified artists. Artists who hadboth experience in the arts and the abilityto work with at-risk youth tended to excelat engaging youth in art activities, adapt-ing activities to meet participants' specificneeds, handling problem behaviors, bond-ing with participants, and establishing mu-tual respect with the youth. YouthARTSparticipants gravitated to those artistswho exhibited expertise in their field andcommunicated easily and respectfullywith them.

Onsite caseworkers and probationofficers or counselors. Onsite casework-ers and probation officers or counselorswere invaluable assets to the programs.These individuals served as positive rolemodels for the participants, and by col-laborating closely with the artists, theyhelped them provide individualized artinstruction to smaller groups of partici-pants. Involvement of these key playersalso provided youth with an opportunityto develop positive relationships withtheir probation officers or counselors.Participants were more involved in theinstructional activities when the proba-tion officers tried to learn the materialwith them. The youth seemed more likelyto take risks (e.g., reveal personal infor-mation, try new things that might embar-rass them) when they saw their probationofficers taking the same risks. Finally,close involvement of the caseworkers and

14

16

probation officers or counselors allowedthem to handle behavioral problems dur-ing the workshops, provide referrals, andwork with the youth and their families toensure that the participants received anyadditional support needed to develop ormaintain positive attitudes and behaviors.

Comprehensive training for all programstaff. Comprehensive training coveringtopics such as at-risk youth, risk factorsand problem behaviors, classroom man-agement, conflict resolution educationand collaborations with other youth-serving agencies was essential for all pro-gram staff. The training not only served toorient staff to the program but also pro-vided them with an opportunity to discussexpectations, roles, and responsibilities.These discussions ensured that programstaff understood the importance of col-laboration to achieve common goals andthat everyone shared realistic, appropriateexpectations about what the programcould accomplish.

Range of arts programs and services. Pro-gram staff and youth participants from allthree programs recognized the need forexpanded arts services in the community.Many of the youth expressed a continuedinterest in the arts and identified the de-sire to be connected with other services inthe community that would allow them topursue their art interests (e.g., additionalart classes, jobs, internships, scholar-ships). Getting connected to the com-munity was important for these youth,and, although each of the programs hadmade some progress in connecting youthto other services, most youth felt nothingwas available for them once the programended. Program staff in all three cities areworking to identify ways to keep youthconnected to the arts and to their com-munities in the future.

Transportation for participants. One ofthe most common reasons for poor

program attendance cited by participantsand staff was lack of transportation.Youth attendance and participation is es-sential to a program's success. Providingreliable transportation (e.g., school buses,city transportation, volunteer-drivenvehicles) makes higher attendance ratespossible.

ConclusionThe future of arts-based delinquency pre-vention and intervention programs ispromising. Through continued investmentin rigorous long-term evaluations, thestrengths of YouthARTS and other arts-based programs can be identified, weak-nesses can be corrected, services can beadjusted to meet the growing needs ofyouth and communities, and effectivemodels can be replicated across the Na-tion. Just as the lessons learned from thenational evaluation have helped the threeYouthARTS programs redesign and imple-ment their programs, the information canbenefit others in the field who are strug-gling with new ways to combat the prob-lem of juvenile delinquency. The nationalevaluation of the YouthARTS Develop-ment Project has shown that providingyouth with new skills, giving them theopportunities to use these skills, and of-fering them positive feedback and recog-nition for their hard work can potentiallylead to healthier attitudes and positivebehaviors.

Endnotes1. Although the YouthARTS DevelopmentProject was initiated in 1995, programsdid not begin operating until fall 1996.

2. The followup period varied by site as aresult of differences in program length,availability of followup data, and otheradministrative issues (e.g., changes inprogram management) that affected thetiming of the followup data collection.

15

3. For more information on The YouthARTSTool Kit, contact Americans for the Arts at800-321-4510 or visit their Web site atwww.artsusa.org.

4. Portland's Percent for Public Art legis-lation requires that 1 percent of the valueof all public construction projects be setaside for the installation of public art.

For FurtherInformationFor information about the YouthARTS De-velopment Project Evaluation Report, con-tact the Juvenile Justice Clearinghouse:

Juvenile Justice ClearinghouseP.O. Box 6000Rockville, MD 20849-6000800-638-8736301-519-5212 (fax)www.puborder.ncjrs.org (Internet)

ReferencesMcArthur, D., and Law, S.A. 1996. The Artsand Public Safety Impact Study: A Reviewof Current Programs and Literature. Pre-pared for the Los Angeles Department ofCultural Affairs. Santa Monica, CA: RANDCorporation, Institute on Education andTraining.

Mulcahy, K.V. 1996. The arts and commu-nity development: Local arts agencies ascultural extension agents. Symposiumpresentation. Washington, DC: NationalAssembly of Local Arts Agencies (nowAmericans for the Arts), January 23.

Weitz, J.H. 1996. Coming Up Taller: Artsand Humanities Programs for Childrenand Youth At Risk. Washington, DC: ThePresident's Committee on the Arts andthe Humanities.

U.S. Department of Justice

Office of Justice Programs

Office of Juvenile Justice and Delinquency Prevention

Washington, DC 20531

Official BusinessPenalty for Private Use $300

AcknowledgmentsHeather J. Clawson is a Senior Associate and Kathleen Coolbaugh is a Principalat Caliber Associates in Fairfax, VA.

All photographs in this Bulletin were provided by the Regional Arts and CultureCouncil.

Share With Your ColleaguesUnless otherwise noted, OJJDP publications are not copyright protected. Weencourage you to reproduce this document, share it with your colleagues, andreprint it in your newsletter or journal. However, if you reprint, please cite OJJDPand the authors of this Bulletin. We are also interested in your feedback, such ashow you received a copy, how you intend to use the information, and how OJJDPmaterials meet your individual or agency needs. Please direct your comments andquestions to:

Juvenile Justice ClearinghousePublication Reprint/FeedbackP.O. Box 6000Rockville, MD 20849-6000800-638-8736301-519-5600 (fax)E-mail: tellncjrs@ncjrs.org

18

PRESORTED STANDARD

POSTAGE & FEES PAID

DOJ/OJJDP

PERMIT NO. G-91

This Bulletin was prepared under grantnumber OJP-95-C-006 from the Office ofJuvenile Justice and Delinquency Prevention,U.S. Department of Justice.

Points of view or opinions expressed in thisdocument are those of the authors and do notnecessarily represent the official position orpolicies of OJJDP or the U.S. Department ofJustice.

The Office of Juvenile Justice and Delin-quency Prevention is a component of the Of-fice of Justice Programs, which also includesthe Bureau of Justice Assistance, the Bureauof Justice Statistics, the National Institute ofJustice, and the Office for Victims of Crime.

U.S. Department of EducationOffice of Educational Research and Improvement (OERI)

National Library of Education (NLE)Educational Resources information Center (ERIC)

NOTICE

Reproduction Basis

ERIC

This document is covered by a signed "Reproduction Release(Blanket)" form (on file within the ERIC system), encompassing allor classes of documents from its source organization and, therefore,does not require a "Specific Document" Release form.

This document is Federally-funded, or carries its own permission toreproduce, or is otherwise in the public domain and, therefore, maybe reproduced by ERIC without a signed Reproduction Release form(either "Specific Document" or "Blanket").

EFF-089 (3t2000)