Evaluating the Aboriginal Programs and Initiatives at the

Department of Canadian Heritage

Canadian Evaluation Society Conference 2003Vancouver, British Columbia

Purpose of Presentation

Provide context to the evaluations within the Aboriginal Affairs Branch at PCH

Present two methodological approaches Aboriginal Languages Initiative – project

site visits Aboriginal Women’s Program – Family

Violence Initiative – learning circle

Discuss evaluation challenges and lessons learned

Discuss next steps

Aboriginal Affairs Branch - PCH

13 programs and initiatives Focussed mainly off-reserve Pan-Aboriginal in nature

Developed over some thirty years Multifaceted approach to complex

Aboriginal issues Facilitate participation in Canadian society Strengthen Aboriginal cultural

distinctiveness

Types of Program Support

Sustaining – established Aboriginal organizations Representative, women’s, friendship centres,

broadcasting National, provincial/territorial, regional

Project

Partnering Accords with territorial governments –

mutual goals Aboriginal organizations

Program Delivery – 3 approaches

Canadian Heritage administration Aboriginal Affairs Branch Regional offices

Aboriginal organizations Six western cities – youth advisory

councils

Comprehensive Review

Retrospective study Environmental scan Consultations – Aboriginal

organizations/groups Evaluation of 13 programs and initiatives

Evaluation Challenges

Complex approach to program delivery

Ensuring a collaborative approach (involvement of delivery partners)

Ensuring cultural sensitivity

Availability of baseline data and performance indicators

Evaluation of the Aboriginal Languages Initiative

Evaluation Methods Document and File Review Secondary Research and Statistical Review Key Informant Interviews Case studies of delivery organizations (AFN,

ITK and MNC) Project Site Visits – 10

Working group and steering committee comprised of representatives from PCH (Aboriginal Affairs Branch and Corporate Review Branch) AFN, ITK, MNC)

ALI Project Site Visits

2 Inuktitut sites, 2 Michif sites, 6 First Nations languages sites

Sites were selected to include all regions and geographic locations (near urban, rural, and isolated) and from each type of “project cluster”

Opportunities and Benefits Opportunities

Conduct face-to fact interviews with those involved directly and indirectly with the project

Conduct focus groups/dialogue circles with project beneficiaries and other community people

Benefits Observe projects and activities directly Place project in context of the community within which

it operates Obtain information on program impact directly from

beneficiaries - how they define success - Obtain information in a culturally sensitive manner Communities see themselves as part of evaluation

process

Evaluation of the AWP Family Violence Initiative

Evaluation Methods

Document& File

Review

Key Informant Interviews

LearningCircle

What is a Learning Circle?

Gathering of people from different organizations

Opportunity for open sharing and exchange Inclusion of range of projects and

perspectives

Learning Circle Process

Sharing knowledge

What have we learned?

Issues and challenges

Stories and journeys

Common threads

Future direction

Learning Circle Opportunities and Benefits

Opportunities to:

Share experiences and insights

Learn

Explore ideas

Reduce isolation

Benefits:

Connection at the community

level

Meaningful results

Lessons learned

Inspiration & momentum

Future program directions

Disadvantage of methodological approaches

Costly and time consuming

Results from site visits or learning circles are not statistically representative of the results of the program as a whole – findings need to be balance with other lines of evidence

One time assessment rather then continuous measurement of results

Importance of including Aboriginal communities in evaluation work – legitimizes results

Qualitative data should be balanced with quantitative information

Evaluations can be designed that meet both government and Aboriginal community information needs

Importance of information sharing

Lessons Learned

Analysis of information Development of position paper Consultation Development of a new consolidated

policy

Aboriginal Affairs BranchNext Steps…

Evaluation reports available electronically

Department of Canadian Heritage Website

www.pch.gc.ca

Presenters Audrey Greyeyes,

Aboriginal Affairs Branch, PCH(819) 994-2121 audrey_greyeyes@pch.gc.ca

Dianne Lepa, Corporate Review Branch, PCH (819) 994-7485, dianne_lepa@pch.gc.ca

Wanda Jamieson, JamiesonHartGraves Consulting (613) 730-2595 wjamieson@jhgconsulting.com