Developing Workflow Process Diagrams To Target Interventions
Developing a Critical Interventions Framework
-
Upload
national-centre-for-student-equity-in-higher-education -
Category
Education
-
view
393 -
download
1
description
Transcript of Developing a Critical Interventions Framework
![Page 1: Developing a Critical Interventions Framework](https://reader036.fdocuments.us/reader036/viewer/2022070317/5561da16d8b42aa83f8b5c3e/html5/thumbnails/1.jpg)
Critical Interventions FrameworkDr Ryan Naylor, CSHE, University of Melbourne 05/02/2014
Developing a
Curtin University is a trademark of Curtin University of TechnologyCRICOS Provider Code 00301J
![Page 2: Developing a Critical Interventions Framework](https://reader036.fdocuments.us/reader036/viewer/2022070317/5561da16d8b42aa83f8b5c3e/html5/thumbnails/2.jpg)
Project Background
• Commissioned project for DIICCSRTE, completed by Dr Ryan Naylor, Dr Chi Baik, Professor Richard James
• Are we on track in achieving national social inclusion goals?
1. Where are we?
2. How effective are our current initiatives? (What appears to work well? What doesn’t?)i. Is it possible to generate a typology of equity initiatives to allow consolidation of research evidence?
ii. Is there evidence in the literature or from HEPPP evaluations to support their efficacy?
![Page 3: Developing a Critical Interventions Framework](https://reader036.fdocuments.us/reader036/viewer/2022070317/5561da16d8b42aa83f8b5c3e/html5/thumbnails/3.jpg)
Caution!
• Some necessary simplifications had to be made in creating the typology and fitting the literature to it
• “Intervention” is a contentious term with troubling associations
• New coalition government changes the policy context
![Page 4: Developing a Critical Interventions Framework](https://reader036.fdocuments.us/reader036/viewer/2022070317/5561da16d8b42aa83f8b5c3e/html5/thumbnails/4.jpg)
What has had the biggest effect on equity?
• Uncapping/deregulation of volume of undergraduate places?
• National target for low SES participation (and associated Mission Based Compacts)?• HEPPP funding?• Wider societal trends in community beliefs about the value of undertaking higher education, entry requirements, eligibility for participation?• Efficacy of equity initiatives depends on underlying factors such as these• Many variables, highly inter-related
![Page 5: Developing a Critical Interventions Framework](https://reader036.fdocuments.us/reader036/viewer/2022070317/5561da16d8b42aa83f8b5c3e/html5/thumbnails/5.jpg)
Where are we?A quick look at the numbers
![Page 6: Developing a Critical Interventions Framework](https://reader036.fdocuments.us/reader036/viewer/2022070317/5561da16d8b42aa83f8b5c3e/html5/thumbnails/6.jpg)
Since 2007, there has been an explosion in domestic student numbers
2007: approx. 722,000 domestic students
2011: approx. 888,000 domestic students
= An increase of 23% over 4 years, oran annual growth rate of 5%
This level of growth is unprecedented in Australian HE
![Page 7: Developing a Critical Interventions Framework](https://reader036.fdocuments.us/reader036/viewer/2022070317/5561da16d8b42aa83f8b5c3e/html5/thumbnails/7.jpg)
Gains in participation share have been made…
Participation Ratio (2011)
![Page 8: Developing a Critical Interventions Framework](https://reader036.fdocuments.us/reader036/viewer/2022070317/5561da16d8b42aa83f8b5c3e/html5/thumbnails/8.jpg)
…But they have been relatively modest and not universal
Participation Ratio (2011)
![Page 9: Developing a Critical Interventions Framework](https://reader036.fdocuments.us/reader036/viewer/2022070317/5561da16d8b42aa83f8b5c3e/html5/thumbnails/9.jpg)
It is difficult to improve equity during growth periods
Stud
ents
with
a d
isab
ility
Indi
geno
us s
tude
nts
Stud
ents
from
a N
on E
nglis
h Sp
eaki
ng
Back
grou
nd
Wom
en in
Non
-Tra
ditio
nal A
reas
Low
SES
(pos
tcod
e m
easu
re)
Low
SES
(CD
mea
sure
)
Regi
onal
Rem
ote
Tota
l
0
2
4
6
8
10
12
Tota
l
Average grow
th rate (2008-2011) (%)
![Page 10: Developing a Critical Interventions Framework](https://reader036.fdocuments.us/reader036/viewer/2022070317/5561da16d8b42aa83f8b5c3e/html5/thumbnails/10.jpg)
But some groups whose share has historically been stable have
increased their participation share
Stud
ents
with
a d
isabi
lity
Indi
geno
us st
uden
ts
Stud
ents
from
a N
on E
nglis
h Sp
eaki
ng B
ackg
roun
d
Wom
en in
Non
-Tra
ditio
nal
Area
s
Low
SES
(pos
tcod
e m
easu
re)
Low
SES
(CD
mea
sure
)
Regi
onal
Rem
ote
Tota
l
0
2
4
6
8
10
12
Tota
lA
verage growth rate (2008-2011) (%
)
![Page 11: Developing a Critical Interventions Framework](https://reader036.fdocuments.us/reader036/viewer/2022070317/5561da16d8b42aa83f8b5c3e/html5/thumbnails/11.jpg)
This change has not been uniform – low SES
Access rate (2011) (%)
Change since 2007 (%)
A 4.25 -0.01
B 6.92 2.38
C 7.10 -0.39
D 15.14 -0.14
E 20.49 -1.38
F 32.82 0.66
G 33.83 2.88
Sector average 16.87 0.75
• No correlation between access rate (2011, 2007) and change• Complex factors – different geographical contexts, access policies,
etc• Traditional strong performers didn’t do better
![Page 12: Developing a Critical Interventions Framework](https://reader036.fdocuments.us/reader036/viewer/2022070317/5561da16d8b42aa83f8b5c3e/html5/thumbnails/12.jpg)
Most groups appear no less likely to succeed
Tota
l
![Page 13: Developing a Critical Interventions Framework](https://reader036.fdocuments.us/reader036/viewer/2022070317/5561da16d8b42aa83f8b5c3e/html5/thumbnails/13.jpg)
Most groups (bar 2) are no less likely to succeed
Tota
l
May not have seen full effects yet – not all students from DDS cohort have moved through the system yet (early days yet!)
![Page 14: Developing a Critical Interventions Framework](https://reader036.fdocuments.us/reader036/viewer/2022070317/5561da16d8b42aa83f8b5c3e/html5/thumbnails/14.jpg)
For most groups, the key problem continues to be access
Equity group Participation Retention Success
Students with a disability 0.48 0.96 0.93
Indigenous students 0.55 0.85 0.81
Students from Non English Speaking Background
0.82 1.04 0.97
Women in Non-Traditional Areas 0.35 1.01 0.99
Low SES (postcode measure) 0.67 0.98 0.97
Low SES (CD measure) 0.62 0.97 0.96
Regional 0.64 0.98 0.99
Remote 0.39 0.91 0.94
Participation and access are (should be?) key focal points in student equity and social inclusion
![Page 15: Developing a Critical Interventions Framework](https://reader036.fdocuments.us/reader036/viewer/2022070317/5561da16d8b42aa83f8b5c3e/html5/thumbnails/15.jpg)
Equity group Participation Retention Success
Students with a disability 0.48 0.96 0.93
Indigenous students 0.55 0.85 0.81
Students from Non English Speaking Background
0.82 1.04 0.97
Women in Non-Traditional Areas 0.35 1.01 0.99
Low SES (postcode measure) 0.67 0.98 0.97
Low SES (CD measure) 0.62 0.97 0.96
Regional 0.64 0.98 0.99
Remote 0.39 0.91 0.94
This is not to argue they don’t need support once enrolled• less academically well prepared students from any background• Indigenous students
For most groups, the key problem continues to be access
![Page 16: Developing a Critical Interventions Framework](https://reader036.fdocuments.us/reader036/viewer/2022070317/5561da16d8b42aa83f8b5c3e/html5/thumbnails/16.jpg)
How effective are our current initiatives:The Critical Interventions Framework
![Page 17: Developing a Critical Interventions Framework](https://reader036.fdocuments.us/reader036/viewer/2022070317/5561da16d8b42aa83f8b5c3e/html5/thumbnails/17.jpg)
The Critical Interventions Framework
• Is it possible to create a typology of equity initiatives?• Can we find sufficient evidence in the literature to point to the efficacy of particular types of initiatives?• Can we identify the initiatives that are most effective?• We don’t know for sure, so we had to make some guesses
![Page 18: Developing a Critical Interventions Framework](https://reader036.fdocuments.us/reader036/viewer/2022070317/5561da16d8b42aa83f8b5c3e/html5/thumbnails/18.jpg)
The Critical Interventions Framework
• Is it possible to create a typology of equity initiatives?• Yes
• Can we find sufficient evidence in the literature to point to the efficacy of particular types of initiatives?• Often no
• Can we identify the initiatives that are most effective?• No. There simply isn’t enough evidence.
![Page 19: Developing a Critical Interventions Framework](https://reader036.fdocuments.us/reader036/viewer/2022070317/5561da16d8b42aa83f8b5c3e/html5/thumbnails/19.jpg)
The equity initiative terrain across a notional student lifecycle
![Page 20: Developing a Critical Interventions Framework](https://reader036.fdocuments.us/reader036/viewer/2022070317/5561da16d8b42aa83f8b5c3e/html5/thumbnails/20.jpg)
The Critical Interventions Framework (one small section)
![Page 21: Developing a Critical Interventions Framework](https://reader036.fdocuments.us/reader036/viewer/2022070317/5561da16d8b42aa83f8b5c3e/html5/thumbnails/21.jpg)
The Critical Interventions Framework (one small section)
![Page 22: Developing a Critical Interventions Framework](https://reader036.fdocuments.us/reader036/viewer/2022070317/5561da16d8b42aa83f8b5c3e/html5/thumbnails/22.jpg)
The Critical Interventions Framework (one small section)
![Page 23: Developing a Critical Interventions Framework](https://reader036.fdocuments.us/reader036/viewer/2022070317/5561da16d8b42aa83f8b5c3e/html5/thumbnails/23.jpg)
• We all have intuitions about what works • Attempted to base framework on evaluative science, not intuition• Unfortunately, the science is largely not there, so some estimation and judgement was involved
The Critical Interventions Framework (one small section)
![Page 24: Developing a Critical Interventions Framework](https://reader036.fdocuments.us/reader036/viewer/2022070317/5561da16d8b42aa83f8b5c3e/html5/thumbnails/24.jpg)
All things considered, what did we rate highly?
High
1B. Later-year outreach (Years 10-12)
2B. Bridging/foundation programs
2D. Scholarships
4C. Student services provision
5A. Monitoring student completion rates
Very High
1D. School curriculum enhancement/support
2A. Pathway/articulation programs
2C. Alternate selection criteria and tools
3A. First year orientation/transition support
• Not intended to narrow or homogenise people’s efforts
![Page 25: Developing a Critical Interventions Framework](https://reader036.fdocuments.us/reader036/viewer/2022070317/5561da16d8b42aa83f8b5c3e/html5/thumbnails/25.jpg)
Red - rated very high; green – rated high
![Page 26: Developing a Critical Interventions Framework](https://reader036.fdocuments.us/reader036/viewer/2022070317/5561da16d8b42aa83f8b5c3e/html5/thumbnails/26.jpg)
Do we have good evidence?
High Quality of Evidence
1B. Later-year outreach (Years 10-12) Limited
2B. Bridging/foundation programs Some (from US)
2D. Scholarships Strong (needs-based, not merit)
4C. Student services provision Varies depending on service
5A. Monitoring student completion rates N/A
Very High
1D. School curriculum enhancement/support Some (from US); strong for need
2A. Pathway/articulation programs Mixed
2C. Alternate selection criteria and tools Strong
3A. First year orientation/transition support Strong (from US)
![Page 27: Developing a Critical Interventions Framework](https://reader036.fdocuments.us/reader036/viewer/2022070317/5561da16d8b42aa83f8b5c3e/html5/thumbnails/27.jpg)
Red – strong evidence
![Page 28: Developing a Critical Interventions Framework](https://reader036.fdocuments.us/reader036/viewer/2022070317/5561da16d8b42aa83f8b5c3e/html5/thumbnails/28.jpg)
Green – some evidence
![Page 29: Developing a Critical Interventions Framework](https://reader036.fdocuments.us/reader036/viewer/2022070317/5561da16d8b42aa83f8b5c3e/html5/thumbnails/29.jpg)
Blue – may be impossible to get evidence
![Page 30: Developing a Critical Interventions Framework](https://reader036.fdocuments.us/reader036/viewer/2022070317/5561da16d8b42aa83f8b5c3e/html5/thumbnails/30.jpg)
How are we spending our HEPPP funding?
High Proportion of HEPPP funding
1B. Later-year outreach (Years 10-12) 14
2B. Bridging/foundation programs 7
2D. Scholarships 14
4C. Student services provision 15
5A. Monitoring student completion rates N/A
Very High
1D. School curriculum enhancement/support 1
2A. Pathway/articulation programs 3
2C. Alternate selection criteria and tools 2
3A. First year orientation/transition support 10Total: 66%
• This is purely descriptive, not normative!• Not all initiatives require the same amount of funding
![Page 31: Developing a Critical Interventions Framework](https://reader036.fdocuments.us/reader036/viewer/2022070317/5561da16d8b42aa83f8b5c3e/html5/thumbnails/31.jpg)
Where to next?
• Written for national policy purposes – not meant to function at the institutional level • We need better evidence of program efficacy (and this is where institutions come in)
o Detailed, rigorous and published evaluationso To enable a sector-wide conversation about
equity initiatives• How can the CIF be used in an institutional context? Is this typology helpful?
![Page 32: Developing a Critical Interventions Framework](https://reader036.fdocuments.us/reader036/viewer/2022070317/5561da16d8b42aa83f8b5c3e/html5/thumbnails/32.jpg)
Where to next?
• Funding sought to research: o Process and attitudinal factors affecting program
evaluationo Resources for embedding evaluation into core
business• First year experience survey• Collaborations with NCSEHE staff