Post on 09-Apr-2017
Observa(on Compara've Analysis of Forma've Assessment Apps
Defini(on
Issues
Applica(ons
Sugges(ons
Future
Challenges
TESOL Arabia Conference 11 March, 2016 HyaC Regency Hotel Dubai, UAE
James Buckingham
Observa(on
Defini(on
Issues
Applica(ons
Sugges(ons
Future
Challenges
TESOL Arabia Conference 11 March, 2016 HyaC Regency Hotel Dubai, UAE
James Buckingham
Instructor Educa'on Technologist Digital Badge Specialist Zayed University Abu Dhabi, UAE
Compara(ve Analysis of Forma(ve Assessment Apps
Backchannel / list of links hBps://todaysmeet.com/TACON2016
Observa(on
Defini(on
Issues
Applica(ons
Sugges(ons
Future
Challenges
Compara(ve Analysis of Forma(ve Assessment Apps
increasing interest in the use of online applica'ons in K-‐20 driven by? • increased availability of smartphones / table devices / laptops
• increased reliability and availability of network connec'vity to support it
• increased interest in using such apps to mo'vate students
• perceived "new ability" to carry out low / no cost computer based assessment (Sharkey, 2005)
Observa(on
Defini(on
Issues
Applica(ons
Sugges(ons
Future
Challenges
Compara(ve Analysis of Forma(ve Assessment Apps
increasing interest in the use of online applica'ons in K-‐20 driven by? • increased availability of smartphones / table devices / laptops
• increased reliability and availability of network connec'vity to support it
• increased interest in using such apps to mo(vate students
• perceived "new ability" to carry out low / no cost computer based assessment (Sharkey, 2005)
Observa(on
Defini(on
Issues
Applica(ons
Sugges(ons
Future
Challenges
Compara(ve Analysis of Forma(ve Assessment Apps
increasing interest in the use of online applica'ons in K-‐20 today's buzzwords
Observa(on
Defini(on
Issues
Applica(ons
Sugges(ons
Future
Challenges
Compara(ve Analysis of Forma(ve Assessment Apps
increasing interest in the use of online applica'ons in K-‐20 today's buzzwords • engaging students; • promo'ng “interac(on”
Observa(on
Defini(on
Issues
Applica(ons
Sugges(ons
Future
Challenges
Compara(ve Analysis of Forma(ve Assessment Apps
increasing interest in the use of online applica'ons in K-‐20 today's applica(ons
• ROI -‐ rela'vely low cost in 'me w/ rela'vely high speed of return
• (me -‐ easy to use, create, collaborate and share
• money -‐ low or no cost investment
Observa(on
Defini(on
Issues
Applica(ons
Sugges(ons
Future
Challenges
Compara(ve Analysis of Forma(ve Assessment Apps
increasing interest in the use of online applica'ons in K-‐20 today's applica(ons
• ROI -‐ rela'vely low cost in 'me w/ rela'vely high speed of return
• (me -‐ easy to use, create, collaborate and share
• money -‐ low or no cost investment
What’s not to like?
Observa(on
Defini(on
Issues
Applica(ons
Sugges(ons
Future
Challenges
Compara(ve Analysis of Forma(ve Assessment Apps
But what is “forma(ve” assessment? not so easy to answer • different defini'ons • even different terms for it • "informa've assessment" (Winkler, 2010)
Observa(on
Defini(on
Issues
Applica(ons
Sugges(ons
Future
Challenges
Compara(ve Analysis of Forma(ve Assessment Apps
But what is “forma(ve” assessment? not so easy to answer • different focuses • learner • instruc'onal design • course design • program design
Observa(on
Defini(on
Issues
Applica(ons
Sugges(ons
Future
Challenges
Compara(ve Analysis of Forma(ve Assessment Apps
But what is “forma(ve” assessment? my focus • learner • instruc'onal design • course design • program design
Observa(on
Defini(on
Issues
Applica(ons
Sugges(ons
Future
Challenges
Compara(ve Analysis of Forma(ve Assessment Apps
But what is “forma(ve” assessment? my focus • learner • a student’s review of their learning • check how well they have
understood • use “data” to • guide (focus) their next move • improve their work
Observa(on
Defini(on
Issues
Applica(ons
Sugges(ons
Future
Challenges
Compara(ve Analysis of Forma(ve Assessment Apps
But what is “forma(ve” assessment? my focus • learner • to provide feedback that will • build learner confidence • promote learner improvement
(Wehlburg, 2011)
Observa(on
Defini(on
Issues
Applica(ons
Sugges(ons
Future
Challenges
Compara(ve Analysis of Forma(ve Assessment Apps
But what is “forma(ve” assessment? my focus • learner • theory • helps make student learning processes more transparent (see what students know) so it benefits BOTH student and instructor
Observa(on
Defini(on
Issues
Applica(ons
Sugges(ons
Future
Challenges
Compara(ve Analysis of Forma(ve Assessment Apps
But what is “forma(ve” assessment? my focus • learner • research • in top three of most effec've ways to impact on student learning
1. make learning fun 2. effec've PD 3. forma(ve assessment (Winkler, 2010)
Observa(on
Defini(on
Issues
Applica(ons
Sugges(ons
Future
Challenges
Compara(ve Analysis of Forma(ve Assessment Apps
But what is “forma(ve” assessment? my focus • learner • methods • feedback without grades • self assessment • peer assessment • ques'oning
Observa(on
Defini(on
Issues
Applica(ons
Sugges(ons
Future
Challenges
Compara(ve Analysis of Forma(ve Assessment Apps
Key “forma(ve” assessment issues? Observa(on #1 -‐ lack of formal training • oben guided by teacher's percep(on of what "works"
• availability of formal instruc'on or PD on how to conduct effec've FA very low
(Volante & BeckeC)
Observa(on
Defini(on
Issues
Applica(ons
Sugges(ons
Future
Challenges
Compara(ve Analysis of Forma(ve Assessment Apps
Key “forma(ve” assessment issues? Observa(on #1 -‐ lack of formal training • few chances to learn / explore FA • high dependency on HE courses to
direct assessment methods • …. even then, few focus on FA ..
choosing instead to focus on SA
(Volante & BeckeC, 2011)
Observa(on
Defini(on
Issues
Applica(ons
Sugges(ons
Future
Challenges
Compara(ve Analysis of Forma(ve Assessment Apps
Key “forma(ve” assessment issues? Observa(on #2 -‐ assessing vs grading • Tempta(on • to "grade / measure” • valued so highly by school admin
(Whelburg, 2011) (Volante & BeckeC, 2011)
Observa(on
Defini(on
Issues
Applica(ons
Sugges(ons
Future
Challenges
Compara(ve Analysis of Forma(ve Assessment Apps
Key “forma(ve” assessment issues? Observa(on #2 -‐ assessing vs grading • Tempta(on • to "grade / measure” • overemphasis on SA =
environment of high accountability • excessive tes'ng = neglect of
cri'cal thinking (far more difficult and 'me consuming to test?)
• more SA = less feedback (?) lower quality feedback (?) (Whelburg, 2011) (Volante & BeckeC, 2011)
Observa(on
Defini(on
Issues
Applica(ons
Sugges(ons
Future
Challenges
Compara(ve Analysis of Forma(ve Assessment Apps
Key “forma(ve” assessment issues? Observa(on #3 -‐ limited methods
(Torrance & Pryor, 2001)
Observa(on
Defini(on
Issues
Applica(ons
Sugges(ons
Future
Challenges
Compara(ve Analysis of Forma(ve Assessment Apps
Selec'ng “FA” applica'ons criteria? • Key traits • easy to use • easy to realize results • low cost / no cost = accessible to all
Observa(on
Defini(on
Issues
Applica(ons
Sugges(ons
Future
Challenges
Compara(ve Analysis of Forma(ve Assessment Apps
Selec'ng “FA” applica'ons criteria? • Key affordances (design supports ac'on) • provides feedback to both instructor / student • data produced ... downloadable .. for interpreta'on
• supports instructor authored feedback • supports collabora'on / sharing with peer/instructors
• supports both convergence and divergence ques'oning
Observa(on
Defini(on
Issues
Applica(ons
Sugges(ons
Future
Challenges
Compara(ve Analysis of Forma(ve Assessment Apps
Selec'ng “FA” applica'ons criteria? • popular among educators • social cura'on • currently ranked highest in educator
based surveys
Observa(on
Defini(on
Issues
Applica(ons
Sugges(ons
Future
Challenges
Compara(ve Analysis of Forma(ve Assessment Apps
Selec'ng “FA” applica'ons criteria? • popular among educators • Edshelf • Graphite • Jane Hart's 100 tools for 2015
Observa(on
Defini(on
Issues
Applica(ons
Sugges(ons
Future
Challenges
Compara(ve Analysis of Forma(ve Assessment Apps
Selec'ng “FA” applica'ons criteria? • Edshelf
social review / cura'on by educators
Observa(on
Defini(on
Issues
Applica(ons
Sugges(ons
Future
Challenges
Compara(ve Analysis of Forma(ve Assessment Apps
Selec'ng “FA” applica'ons criteria? • Edshelf
social review / cura'on by educators
Observa(on
Defini(on
Issues
Applica(ons
Sugges(ons
Future
Challenges
Compara(ve Analysis of Forma(ve Assessment Apps
Selec'ng “FA” applica'ons criteria? • Graphite
social review / cura'on by reviewers & educators
Observa(on
Defini(on
Issues
Applica(ons
Sugges(ons
Future
Challenges
Compara(ve Analysis of Forma(ve Assessment Apps
Selec'ng “FA” applica'ons criteria? • Graphite
social review / cura'on by reviewers & educators
Observa(on
Defini(on
Issues
Applica(ons
Sugges(ons
Future
Challenges
Compara(ve Analysis of Forma(ve Assessment Apps
Selec'ng “FA” applica'ons criteria? • Graphite
social review / cura'on by reviewers & educators Ranking Kahoot 3 / 4 Socra've 3 / 4 Nearpod 4 / 5
Observa(on
Defini(on
Issues
Applica(ons
Sugges(ons
Future
Challenges
Compara(ve Analysis of Forma(ve Assessment Apps
Selec'ng “FA” applica'ons criteria? • Jane Hart's 100 tools for 2015
nomina'ons & reviews by educators
Observa(on
Defini(on
Issues
Applica(ons
Sugges(ons
Future
Challenges
Compara(ve Analysis of Forma(ve Assessment Apps
Selec'ng “FA” applica'ons criteria? • Jane Hart's 100 tools for 2015
nomina'ons & reviews by educators Ranking 17 – Kahoot 32 – Socra've 50 – Nearpod
Observa(on
Defini(on
Issues
Applica'ons
Sugges(ons
Future
Challenges
Compara(ve Analysis of Forma(ve Assessment Apps
Challenges facing “FA” adop'on Address training needs • educators may lack skills on • how to interpret student data results • offer feedback that will lead to improved student
results Unclear next steps? • lack of incen(ves for teachers to invite learning
how to measure effec'veness of feedback • lack of consensus among academics on how to
move forward (Sharkey & Murnane, 2006)
Observa(on
Defini(on
Issues
Applica'ons
Sugges(ons
Future
Challenges
Compara(ve Analysis of Forma(ve Assessment Apps
Sugges'ons – My “current” review Kahoot #17 • supports “convergence” type ques'ons • does NOT support divergence type ques'ons
• maximum 4 choices – “right / wrong” feedback • does NOT support authored feedback • could be adapted to realize peer feedback •
Observa(on
Defini(on
Issues
Applica'ons
Sugges(ons
Future
Challenges
Compara(ve Analysis of Forma(ve Assessment Apps
Sugges'ons – My “current” review Socra(ve #32 • supports BOTH “convergence” and “divergence”
ques'ons • does support authored feedback • could be adapted to realize peer feedback
Observa(on
Defini(on
Issues
Applica'ons
Sugges(ons
Future
Challenges
Compara(ve Analysis of Forma(ve Assessment Apps
Sugges'ons – My “current” review Nearpod #50 • supports BOTH “convergence” and “divergence”
ques'ons • does support authored feedback • could be adapted to realize peer feedback
Observa(on
Defini(on
Issues
Applica'ons
Sugges(ons
Future
Challenges
Compara(ve Analysis of Forma(ve Assessment Apps
Sugges'ons -‐ General Focus on the learner’s needs • nature of feedback needs to be • more suppor've (in small groups?) • more informa've
leads to • less convergence in FA design • True / False • Right / Wrong
Observa(on
Defini(on
Issues
Applica'ons
Sugges(ons
Future
Challenges
Compara(ve Analysis of Forma(ve Assessment Apps
Sugges'ons -‐ General Focus on the learner’s needs • nature of feedback needs to be • more suppor've (in small groups?) • more informa've
leads to • more divergence in FA design • open ques'ons / open answers
Observa(on
Defini(on
Issues
Applica'ons
Sugges'ons
Future
Challenges
Compara(ve Analysis of Forma(ve Assessment Apps
The Future? • more variety in FA strategies (growing segment of app market)
• more personalized & self directed instruc'on
Observa(on
Defini(on
Issues
Applica'ons
Sugges'ons
Future
Challenges
Compara(ve Analysis of Forma(ve Assessment Apps
The Future? • more variety in FA strategies (growing segment of app market)
• more personalized & self directed instruc'on
• Current examples?
Interac've video applica'ons
Observa(on
Defini(on
Issues
Applica(ons
Sugges(ons
Future
Challenges
List of references
Compara(ve Analysis of Forma(ve Assessment Apps
Abuid, B. A. (2014). A student par'cipa'on assessment scheme for effec've teaching and learning. Learning and Teaching in Higher Educa'on: Gulf Perspec'ves, 11(1). Retrieved from hCp://lthe.zu.ac.ae/index.php/lthehome/ar'cle/view/109 Foggo, L. (2007). Using Blogs for Forma've Assessment and Interac've Teaching. Ariadne, (51). Retrieved from hCp://www.ariadne.ac.uk/issue51/foggo/ Laborda, J. G., Sampson, D. G., Hambleton, R. K., & Guzman, E. (2015). Guest Editorial: Technology Supported Assessment in Formal and Informal Learning. Journal of Educa'onal Technology & Society, 18(2), 1–2. l’Anson, S. (2016, February 24). Forma've Assessment Archives. Retrieved from hCp://interac'veachievement.com/category/forma've-‐assessment/
Observa(on
Defini(on
Issues
Applica(ons
Sugges(ons
Future
Challenges
List of references
Compara(ve Analysis of Forma(ve Assessment Apps
Sharkey, N. S., & Murnane, R. J. (2006). Tough Choices in Designing a Forma've Assessment System. American Journal of Educa'on, 112(4), 572–588. hCp://doi.org/10.1086/505060 Torrance, H., & Pryor, J. (2001). Developing Forma've Assessment in the Classroom: Using ac'on research to explore and modify theory. Bri'sh Educa'onal Research Journal, 27(5), 615–631. hCp://doi.org/10.1080/01411920120095780 Volante, L., & BeckeC, D. (2011). Forma've assessment and the contemporary classroom: Synergies and tensions between research and prac'ce. Canadian Journal of Educa'on/Revue Canadienne de L’éduca'on, 34(2), 239–255. Wehlburg, C. (2011). A scholarly approach to assessing learning. Interna'onal Journal for the Scholarship of Teaching and Learning, 5(2), 2.
Observa(on
Defini(on
Issues
Applica(ons
Sugges(ons
Future
Challenges
List of references
Compara(ve Analysis of Forma(ve Assessment Apps
Winkler, C. C. (2010). Examples of Effec've Forma've Assessment. Mar'n Luther College. Retrieved from hCps://www.mlc-‐wels.edu/library/search-‐find-‐2/special-‐collec'ons/pdf-‐files/LitReviewWinkler2010.pdf
Observa(on
Defini(on
Issues
Applica(ons
Sugges(ons
Future
Challenges
TESOL Arabia Conference 11 March, 2016 HyaC Regency Hotel Dubai, UAE
James Buckingham
Instructor Educa'on Technologist Digital Badge Specialist Zayed University Abu Dhabi, UAE
Compara(ve Analysis of Forma(ve Assessment Apps
Backchannel / list of links hBps://todaysmeet.com/TACON2016