Post on 01-Feb-2016
description
NWT Literacy Strategy Evaluation and Renewal
Overview
International Adult Literacy and Skills Survey (IALSS)
NWT Literacy Strategy Summative Evaluation
NWT Literacy Strategy Renewal
International Adult Literacy and Skills Survey (IALSS)
IALSS measures adult skill proficiencies on a 5-point scale (with 5 as the highest level).
Level 3 is considered the international standard for functional literacy.
Prior to IALSS, the best data on adult literacy available to the NWT was self-reported grade levels from censuses.
Data from the 2003 IALSS provides direct measurements of NWT adult skill levels.
International Adult Literacy and Skills Survey (IALSS)
NWT adult literacy levels were close to the Canadian average in the all skill domains.
IALSS confirmed a wide discrepancy between NWT Aboriginal and non-Aboriginal adult literacy levels in all skill domains and for all age groups.
NWT Adult Skill Proficiencies
Prose Literacy
Document Literacy Numeracy
Problem Solving
Level 1 19.3 20.2 25.0 35.7Level 2 25.8 25.9 28.0 36.3Level 3 35.1 33.8 31.7 24.1Level 4/5 19.8 20.1 15.3 3.9
Level 3 Document Literacy Task
Describe the relationship between
fireworks sales and injuries
Prose Literacy
2003 IALSS NWT Prose Proficiency, 16+
0%10%20%30%40%50%60%70%80%90%
100%
Aboriginal Non-Aboriginal
Level 4/5
Level 3
Level 2
Level 1
Document Literacy
2003 IALSS NWT Document Literacy, 16+
0%10%20%30%40%50%60%70%80%90%
100%
Aboriginal Non-Aboriginal
Level 4/5
Level 3
Level 2
Level 1
Numeracy
2003 IALSS NWT Numeracy Proficiency, 16+
0%10%20%30%40%50%60%70%80%90%
100%
Aboriginal Non-Aboriginal
Level 4/5
Level 3
Level 2
Level 1
Problem Solving
NWT Problem Solving Proficiency, 16+
0%10%20%30%40%50%60%70%80%90%
100%
Aboriginal Non-Aboriginal
Level 4/5
Level 3
Level 2
Level 1
IALSS and the Literacy Strategy
IALSS data has been cross-referenced with goals, objectives and actions from the 2001 NWT Literacy Strategy.
Recommendations arising from IALSS research are informing the renewal of the NWT Literacy Strategy.
Example 1: NWT Adult Participation in Training
2003 IALSS Percentage of NWT Population, 16-65,
Receiving Adult Education and Training in 12 Months Preceding IALSS Testing
0
10
20
30
40
50
60
70
80
90
Prose Literacy
Document Literacy
Numeracy Problem Solving
Level 1
Level 2
Level 3
Level 4/5
Example 1: NWT Adult Participation in Training
IALSS data on NWT adult participation in reveals that low literacy adults, who need training the most are accessing it the least.
Recommendations for new Literacy Strategy: – Targeted recruitment of low literacy NWT
adults into training; – Promotion of a lifelong learning perspective.
Example 2: Workplace Literacy IALSS reveals that a significant percentage of NWT Level
1 and 2 adults are in the workplace, so effective workplace literacy programs are crucial to increasing adult skill levels.
Level 1 Level 2 Level 3 Level 4/5Prose 12.4 24.5 39.0 24.1Document 13.3 24.4 37.8 24.5Numeracy 17.7 28.0 35.8 18.6Problem Solving 28.4 38.6 28.1 4.9
2003 IALSS Percent Distribution of Employed NWT Adults,
16 - 65, by Skill Level
Example 2: Workplace Literacy
IALSS draws attention to the issue of skills losses:
“Skills are like muscles that need to be exercised in order to be developed and maintained.”
Of particular concern is the low workplace reading, writing and numeracy engagement rates of NWT Level 1 and 2 adults.
Example 2: Workplace Literacy
-0.8-0.6-0.4-0.2
00.20.40.6
ReadingIntensity
Writing Intensity NumeracyIntensity
2003 IALSS NWT Reading, Writing and Numeracy Engagement
at Work by Prose Literacy Level
Level 1
Level 2
Level 3
Level 4/5
Example 2: Workplace Literacy
Continuing to deliver workplace literacy programs that only address skills deficiencies is akin to pouring water into a leaky bucket.
Recommendation for new Literacy Strategy:– Educate employers about skills losses and
encourage them to provide employees with opportunities to use and retain their skills in the workplace.
NWT Literacy Strategy Summative Evaluation
Background Purpose Methodology Successes Challenges Recommendations
Background
Motion – July 7, 2000. January 2001, the GNWT approved the
Northwest Territories Literacy Strategy. Allocation of an additional $2.4 million
annually for literacy initiatives in the NWT.
Towards Literacy: A Strategy Framework 2001-2005 was developed in consultation with the social envelope departments and other GNWT departments and agencies.
Background
Accountability framework was developed in 2002/2003 fiscal year.
Results-based Management and Accountability Framework (RMAF).
RMAF is the de facto standard for evaluation and accountability in Canada.
RMAF used for Literacy Strategy interim and summative evaluations.
Background
Annual reports. Interim evaluation of the Literacy
Strategy (2004)– covered the first 2 fiscal years (2001/2002
and 2002/03) .
Summative evaluation (2007)– covers first 5 fiscal years (2001/02 to
2005/06).
Purpose Assess progress after 5 years of
NWT Literacy Strategy programming.
Gather feedback from stakeholders.
Inform renewal of the Literacy Strategy.
Methodology 4 goals of Towards Literacy: A Strategy
Framework – 2001-2005:1. to increase the number of people in the NWT who are literate2. to increase awareness of the importance of literacy in the official languages of the NWT3. to ensure government departments work together to increase literacy levels in the NWT 4. to develop literacy partnerships across society.
8 target groups – 5 under the Goal #1.
Methodology
Document and data review – program stats, budgets, formative
evaluation results, annual reports, etc.
Consultations in 10 NWT communities:– Aklavik, Inuvik, Norman Wells, Deline,
Fort Simpson, Fort Providence, Fort Smith, Hay River, Hay River Reserve, Yellowknife.
Methodology
Interviews and focus groups with program managers and stakeholders– NGOs, ECE regional offices, ECE HQ, Aurora
College, Aboriginal Language Communities.– Focus on relevance, success, cost-effectiveness.
Surveys of stakeholders – Focus on impact of Literacy Strategy activities
on objectives, sub-goals, goals.
Methodology
Report analyzed and synthesized the information collected to determine relevance, success, and cost-effectiveness.
Examined 4 Literacy Strategy goals, including a roll-up of the 5 sub-goals under Goal #1.
Literacy Strategy overall.
Successes
Increased awareness of literacy throughout the NWT .
$2.4M was a significant investment in literacy (literacy resources, research, capacity).
Increased partnerships and networking. Evaluation and reporting are positive
steps to Literacy Strategy improvement.
Successes
Some individual programs were very successful:– NWTLC – Northern Parenting & Literacy
Program– NWT IALSS data.
Literacy more than just reading and writing– Technology and Internet.
Challenges
Literacy Strategy was a good idea on paper – but complex and difficult to implement– interlinking strategies not well understood– unfunded areas were problematic– literacy needs are long term – no quick fixes– lack of baseline data.
Data collection and reporting were difficult at all levels.
Challenges
Stakeholders indicated there were not enough resources to achieve all goals and objectives– spread thin, rather than focussed on 1 or 2
areas. Staff turnover. No multi-year funding at the project level. Communication and coordination of the
Literacy Strategy were not consistent.
Recommendations
Renew the Literacy Strategy based on results of IALSS and NWT Literacy Strategy Summative Evaluation.
Need a dedicated Literacy Strategy Coordinator.
Take a “targeted” approach.
Recommendations
Design goals and objectives realistically. Adopt a multi-year funding approach. Ensure proper data collection. Design and implement Literacy Strategy
communications plan. Ensure Literacy Strategy partners play an
active role in design and delivery of renewed Literacy Strategy.
NWT Literacy Strategy Renewal
The new NWT Literacy Strategy will be informed by IALSS data and research from the Literacy Strategy summative evaluation.
The Literacy Strategy renewal process provides an opportunity to build on literacy initiatives of the past six years and make improvements to literacy programming in the NWT.
Literacy Strategy Renewal
Acting upon recommendations from the summative evaluation, ECE established a multi-stakeholder Literacy Strategy renewal working group.
Representation from ECE, Health and Social Services, Aurora College, NWT Literacy Council, NGO literacy service providers, Aboriginal organizations and industry.
NWT Literacy Strategy Renewal
Technical subgroup drafting goals, objectives and actions.
Draft objectives and actions will be presented to larger NWT Literacy Strategy Renewal Working Group on December 13 – 14.
NWT Literacy Strategy Renewal
Literacy Strategy Renewal Working Group recommends a 10-year literacy strategy.
Evaluation framework for new Literacy Strategy will be developed concurrent with the Literacy Strategy.
Target date for completion of draft NWT Literacy Strategy is February 2008.
Thank You!