State and Directions of Learning Analytics Adoption (Second edition)

State and Directions of Learning Analytics Adoption

(Second edition)

Dragan Gašević@dgasevic

March 21, 2017ISoTL, UBCVancouver, BC, Canada

Educational Landscape Today

Growing need for educationActive learning

Education defunding

Feedback loops between students and instructors

are missing/weak!

LEARNING ANALYTICS

Learning environment

Educators

LearnersStudent

Information Systems

Blogs

Videos/slides

Mobile

Search

Educators

Learners

Networks

Student Information

Systems


Blogs

Mobile

Search

Networks

Educators

LearnersStudent

Information Systems


Videos/slides

CASE STUDIES

Student retention

Year 1 Year 2 Year 3 Year 40.00%

10.00%20.00%30.00%40.00%50.00%60.00%70.00%80.00%90.00%

100.00%

Course SignalsNo Course Signals

Arnold, K. E., & Pistilli, M. D. (2012, April). Course Signals at Purdue: Using learning analytics to increase student success. In Proceedings of the 2nd International Conference on Learning Analytics and Knowledge (pp. 267-270).

Tanes, Z., Arnold, K. E., King, A. S., & Remnet, M. A. (2011). Using Signals for appropriate feedback: Perceptions and practices. Computers & Education, 57(4), 2414-2422.

Can teaching be improved?

Wright, M. C., McKay, T., Hershock, C., Miller, K., & Tritz, J. (2014). Better Than Expected: Using Learning Analytics to Promote Student Success in Gateway Science. Change: The Magazine of Higher Learning, 46(1), 28-34.

INSTITUTIONAL ADOPTION: CURRENT STATE

Current state – Oz and Europe

http://sheilaproject.eu/http://he-analytics.com

Very few institution-wide examples of adoption

Tsai, Y. S., & Gasevic, D. (2017). Learning analytics in higher education – challenges and policies: a review of eight learning analytics policies. In Proceedings of the Seventh International Learning Analytics & Knowledge Conference (pp. 233-242).

Sophistication model

Siemens, G., Dawson, S., & Lynch, G. (2014). Improving the Quality and Productivity of the Higher Education Sector - Policy and Strategy for Systems-Level Deployment of Learning Analytics. Canberra, Australia: Office of Learning and Teaching, Australian Government. Retrieved from http://solaresearch.org/Policy_Strategy_Analytics.pdf

Adoption challenge

Leadership for strategic implementation & monitoring

Lack of leadership

Bought an analytics product.

Analytics box ticked!

Leadership challenge

Adoption challenge

Equal engagement with different stakeholders

Adoption challenge

Training opportunities to use learning analytics

Adoption challenge

Policies for learning analytics practice

What’s necessary to move forward?

DIRECTIONS

Data – Model – Transformation

Barton, D., & Court, D. (Oct 2012). Making Advanced Analytics Work for You. Harvard Business Review, 79-83, https://hbr.org/2012/10/making-advanced-analytics-work-for-you/ar/1

https://hbr.org/2012/10/making-advanced-analytics-work-for-you/ar/1


Creative data sourcing

Gašević, D., Dawson, S., Pardo, A. (2016). How do we start? State and Directions of Learning Analytics Adoption. Oslo, Norway: International Council for Open and Distance Education. http://bit.ly/icde_la_16

Social networks are everywhere

Gašević, D., Zouaq, A., Jenzen, R. (2013). ‘Choose your Classmates, your GPA is at Stake!’ The Association of Cross-Class Social Ties and Academic Performance. American Behavioral Scientist, 57(10), 1459–1478.


Creative data sourcingNecessary IT support


Awareness of limitations and challenging assumptions

Kovanović, V., Gašević, D., Dawson, S., Joksimović, S., Baker, R. (2015). Does Time-on-task Estimation Matter? Implications on Validity of Learning Analytics Findings. Journal of Learning Analytics, 2(3), 81-110.


Question-driven, not data-driven


Field of research and practice

Gašević, D., Kovanović, V., & Joksimović, S. (2017). Piecing the Learning Analytics Puzzle: A Consolidated Model of a Field of Research and Practice. Learning: Research and Practice, 3(2), in press. doi:10.1080/23735082.2017.1286142

Learning analytics is about learning

Gašević, D., Dawson, S., Siemens, G. (2015). Let’s not forget: Learning analytics are about learning. TechTrends, 59(1), 64-71.

One size fits all does not work in learning analytics

Gašević, D., Dawson, S., Rogers, T., Gašević, D. (2016). Learning analytics should not promote one size fits all: The effects of course-specific technology use in predicting academic success. The Internet and Higher Education, 28, 68–84.

Learning context

Instructional conditions shape learning analytics results

Learner agency

Jovanović, J., Gašević, D., Dawson, S., Pardo, A., & Mirriahi, N. (2017). Learning analytics to unveil learning strategies in a flipped classroom. The Internet and Higher Education, 33, 74-85.

More time online does not always mean better learning

Systemic Adoption Model

Colvin, C., et al. (2015). Student retention and learning analytics: A snapshot of Australian practices and a framework for advancement. Sydney: Australian Office for Learning and Teaching.

Strategic capability


Solution-focused Model


Process-focused Model



Inclusive approaches to adoption


What do students want?

Representation on committeesStudent expectation of learning analytics

Focus group interviews Whitelock-Wainwright, A., Gašević, D., & Tejeiro, R. (2017). What do students want?: towards an instrument for students' evaluation of quality of learning analytics services. In Proceedings of the Seventh International Learning Analytics & Knowledge Conference (pp. 368-372).

Expert’s perspective to LA policyimportance ease

privacy & transparency

privacy & transparency

risks & challenges

risks & challenges

roles & responsibilities (of all stakeholders)

roles & responsibilities (of all stakeholders)

objectives of LA (learner and teacher support)

objectives of LA (learner and teacher support)

data management

data management

research & data analysis

research & data analysis

3.79 3.79

6.03 6.03

r = 0.66

Learning analytics purposes

Quality, equity, personalized feedback, coping with scale, student experience,

skills, and efficiency

The University of Edinburgh (2017). Learning Analytics Policy, http://www.ed.ac.uk/academic-services/projects/learning-analytics-policy


Inclusive approaches to adoptionAnalytics tools for non-statistics experts


Visualizations can be harmful

Corrin, L., & de Barba, P. (2014). Exploring students’ interpretation of feedback delivered through learning analytics dashboards. In Proceedings of the ascilite 2014 conference (pp. 629-633). ascilite.

Students don’t perceive dashboards as feedback

Pardo, A., Jovanovic, J. Dawson, S., Gasevic, D. (in preparation). Using Learning Analytics to Scale the Provision of Personalised Feedback.


Participatory design of analytics toolsAnalytics tools for non-statistics expertsDevelop capabilities to exploit (big) data


Marr, B. (Oct 2015). Forget Data Scientists - Make Everyone Data Savvy, http://www.datasciencecentral.com/m/blogpost?id=6448529%3ABlogPost%3A337288

http://www.datasciencecentral.com/m/blogpost?id=6448529:BlogPost:337288

FINAL REMARKS

Rhetoric of simplistic technological fixes

is unproductive

Embracing complexity of educational systems

Capacity development

Multidisciplinary teams in institutions critical

Ethical and privacy considerationDevelopment of data privacy agency

Prinsloo, P., & Slade, S. (2015). Student privacy self-management: implications for learning analytics. In Proceedings of the Fifth International Conference on Learning Analytics And Knowledge (pp. 83-92). ACM.

Sclater, N. (2014). Code of practice for learning analytics: A literature review of the ethical and legal issues. http://repository.jisc.ac.uk/5661/1/Learning_Analytics_A-_Literature_Review.pdf

Development of analytics culture

Manyika, J. et al. (2011). Big Data: The Next Frontier for Innovation, Competition, and Productivity. McKinsey Global Institute, http://goo.gl/Lue3qs

Thank you!

State and Directions of Learning Analytics Adoption (Second edition)

Education

Transcript of State and Directions of Learning Analytics Adoption (Second edition)