Enhancing Practice in Work with Offenders: the Role of Evaluation

Jean Hine, De Montfort University

Workstream 1: Aim

• To learn:– Better understanding of what evaluation is

undertaken, who does it, how, why, with what result

– What facilitates/inhibits– What might help

• To develop– Evaluation guidance/toolkit– Library of examples of evaluation practice

Four Workstreams

Good and Effective Practice

3. Literature Review

1. Evaluation 4, EPR

2. SEED

• Interviews across the EU• Mapping of evaluation• Working with 2 pilot evaluations• Co-ordination with workstreams 2, 3

and 4• Development of evaluation

guidance

Workstream 1: Components

Mapping

• More scoping than mapping.• Not comprehensive but does

include good range countries, organisations, individuals

• Complexity – language and meaning– range organisations involved– structuring of work with offenders in

different countries

Evaluation takes many forms

• Specific independent research studies• MoJ national studies/reviews• Monitoring/management information• Target assessment• Compliance • Project outcomes• Individual casework• Serious case reviews

Who evaluates• Some countries have strong

commitment to evaluation and active programme, some do not see evaluation as important

• Most countries have some governmental level unit undertaking some data work, some basic statistical, some special centres/units

• Most countries have some university involvement but approach varies by discipline

• Some use of independent organisations

What is evaluated?

• Specific programmes and practices– Electronic monitoring, thinking skills,

domestic violence, sex offender programmes, etc

• Much focus on recidivism/outcomes– But different abilities to assess

• Competition with prison evaluation• Generally short term impact

What’s not?

Limited:• Systematic evaluation of general supervision• Service user evaluation• Detailed analysis of routinely collected data

Key learning from Mapping

• Wide variation• Most evaluation is centrally led– Central commitment important to enable– But prioritises central concerns– Tends to be large scale and quantitative

• There are interesting pieces of work which need to be more widely known

• General commitment to a desire for more evaluation, sometimes externally driven

Facilitators for evaluation

• Central commitment• External pressures– EU– Funders– Government threat

• Individual enthusiasm

Inhibitors for evaluation

• Cultural/practitioner resistance• Limited Resources– Time as well as money

• Lack of skills/knowledge• Lack of interested audience• Overestimation of the task• Limited awareness of possibilities• Lack of clarity about objectives of practice

Pilot work• Shared process• Working together• In some ways artificial but beneficial– for pilots– for guidance

• Useful differences and similarities• Two partners

Italy

• NGO, large scale drug rehabilitation community• Offending not a direct focus but desistance an

anticipated benefit• Formative evaluation• How to retain more alternative sentencees at end of

sentence• Mixed approach– Focus groups– Data analysis

• Highlighted importance of shared language

England

• Social enterprise offering peer support for employability

• Offending not direct focus but an anticipated benefit

• Formative evaluation• How to improve success of specific project

with probationers

Common issues

• Identifying evaluation question• Data system limitations• Designing manageable project• Resources• Time frame• Insider role tension• Support • Changing context

Pilots’ Focus Group

• Right time to evaluate• Political context• Availability of respondents• Rigour and confidence• Ethical issues• Written guidance too ‘academic’• Cop or coach?• Critical friend/benevolent challenging• Evaluation is time and labour intensive (but worth it!)

How does this inform guidance?

• Clarity• Flexibility• Range of uses/users• Different starting points• Making the most of existing data• Making evaluation manageable

International comparability difficult

• Differing terms and meanings• Differing criminal justice processes• Different criteria for sanctions• Different legal contexts

Example of Recidivism

• Usually means reconviction• Availability of data:– Routine standard collection or not– Data protection laws

• Definition of conviction– Legal frameworks

• Time frames– Judicial procedures

The Hope

Quality evaluation

Sound evidence

Informed practice

Good outcomes