Breakout  groups  feedback  

Reten%on  and  success  

•  Reten%on  and  success  are  dis%nct,  but  linked.  Qualita%ve  vs  binary.  

•  Applica%ons:  quick/early  drop-­‐out,  adapa%ve  learning.  

•  Ethical  issues.  •  Media%ng  feedback,  using  analy%cs  to  present  the  model  with  the  ra%onale,  used  as  the  basis  for  a  personalised  conversa%on.  

Photo  (CC)  Trey  Ratcliff  hJp://www.flickr.com/photos/stuckincustoms/4622806283/  

Dashboards  

Mul%ple  audiences  •  Different  purposes  •  Same  data  sets  •  Interpreta%on  and  clarity  •  Training  and  sense  making  

Mul%ple  Purposes  • Aggrega%on  • Interven%on  • Mo%va%on  • Informed  decision  making  

• ‘De-­‐modularisa%on’  (holis%c  informa%on)  

• Ipsa%ve  vs  norm  informa%on  

Ethics  • Emo%ons  • Anxiety  • Surveillance  

• Privacy  • Transparency  

Opera%onalisa%on  • Selec%ng  data  sets  • Timeliness  and  efficacy  • evalua%on  

• Granularity  • Interac%vity  • Proprietary  tool  providers  preemp%ng  our  needs/wants  

• Pedagogically  drivers  

Dashboard  Examples  • How  am  I  doing  compared  to  cohort?  Student  •  Is  what  I’m  doing  with  my  students  working?  Tutor  • Which  students  are  most  likely  to  drop  out?  Ins%tu%on  • Are  any  students  gradua%ng  from  this  ins%tu%on  without  all  of  the  required  learning  outcomes?  PSRB  

• Across  the  sector  which  ins%tu%ons  produce  the  best  graduates  in  each  discipline?  Researchers  

Dangers  of  a  Pre-­‐Crime  Unit  Pre-­‐fail  

Analy5cs  for  Student  Success  &  Reten5on:  Issues  

Beware  self-­‐fulfilling  failure  prophecies!  

Informed  interven%ons  hopefully  changing  learners’  futures  for  the  beJer…  But  what  does  that  do  for  datasets  and  historical  comparison?  Important  to  collect  data  about  interven%ons  to  assess  their  impact  amongst  other  variables    

Ethics  of  interven5on:    Just  for  those  who  are  failing?  What  about  the  rest?  

“Dear  <field1>…”  

Beware  back-­‐firing  personalisa%on  expecta%ons:  “So  I  really  am  just  a  number”  

Beware:  can’t  count,  doesn’t  count:  we’re  in  a  complex  people  business!  

Pedagogy  &  LA  

Issues  •  How  do  we  measure  learning  (rather  than  ‘success’  in  

assessments)  •  Approximate  proxies  for  learning…  •  Shouldn’t  assessment  be  our  ‘best  measure’  of  learning  –  

well,  perhaps  it  should  be  a  suite  of  analy%cs  •  What  ‘knowledge’  do  we  want  from  our  graduates  •  ‘Recipe’  issue  of  LA?  –  so  we  have  to  make  sure  we’re  

looking  for  the  ‘right’  processes  •  Assessment/analy%cs:  Snapshots,  con%nuity,  and  change  

metrics;  how  can  they  be  used?  •  Analy%cs  driven  by  what  we  want  to  achieve  rather  than  

what  data  is  available  

Examples  

•  Dialogue  analysis,  perhaps  analysis  of  use  of  social  networks  

•  LA  as  pedagogy  v  LA  for  pedagogy  –  LA  which  feeds  back  in  to  ‘improving’/adap%ng.  LA  can  help  us  challenge  our  assump%ons  about  how  the  learning  is  taking  place.    Can  LA  allow  us  to  hypothesis  test  our  (as  teachers)  assump%ons  about  learning?  

•  Pass  rate  and  online  ac%vity  has  a  correla%on  –  effec%ve  ‘proxy’?  

Data  sources  

Issues  •  Availability  •  Quality  •  Enrich  (combining  data)  •  Private    •  Paying  to  access  your  own  

data  •  Need?  •  Data  ownership  •  Not  everything  is  online  –  

no  footprint  (overall  visibility  of  interac%ons)  

•  Volume  

•  Awareness  of  data  collec%on  

•  Sharing  (ethics,  commercially  sensi%ve)  

•  Infrastructure  •  Planning  in  rapidly  evolving  

area  (itera%ons)  •  Granularity  (nano)  •  Purpose  •  Culture  change  

Examples  

•  TINCAN  API  

•  IBM  –  (data  don’t  ask,  don’t  get)  

•  midata