Community Learning Analytics – A New Research Field in TEL

Lehrstuhl Informatik 5 (Information Systems)

Prof. Dr. M. Jarke 1

Learning Layers

This slide deck is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.

Community Learning Analytics –

A New Research Field in TEL

Ralf Klamma Advanced Community Information Systems (ACIS)

RWTH Aachen University, Germany [email protected]

JTEL Summer School, Malta, April 28, 2014



Learning Layers

Abstract Learning Analytics has become a major research area recently. In particular learning institutions seek ways to collect, manage, analyze and exploit data from learners and instructors for the facilitation of formal learning processes. However, in the world of informal learning at the workplace, knowledge gained from formal learning analytics is only applicable on a commodity level. Since professional communities need learning support beyond this level, we need a deep understanding of interactions between learners and other entities in community-regulated learning processes - a conceptual extension of self-regulated learning processes. In this presentation, we discuss scaling challenges for community learning analytics and give both conceptual and technical solutions. We report experiences from ongoing research in this area, in particular from the two EU integrating project ROLE (Responsive Open Learning Environments) and Learning Layers (Scaling up Technologies for Informal Learning in SME Clusters).



Learning Layers

Responsive Open

Community Information

Systems

Community Visualization

and Simulation

Community Analytics

Community

Support

Web Analytics W

eb E

ngin

eerin

g

Advanced Community Information Systems (ACIS) Group @ RWTH Aachen

Requirements Engineering



Learning Layers

Agenda

Lear

ning A

nalyt

ics

Comm

unity

Lear

ning A

nalyt

ics

ROLE

& Le

arnin

g Lay

ers

Expe

rts in

Com

munit

y Info

rmati

on

Syste

ms

Conc

lusion

s & O

utloo

k



Learning Layers

A PHD STUDENT VIEW ON THE RESEARCH FIELD



Learning Layers Motivations for Doing

PhD Research in TEL ■  Some reasons (more?)

–  My supervisor told me … (research interest of person paying me) –  My own research interest –  Good career perspectives (get famous, get rich, or both)

■  Formal Learning –  Close to my own practice and experience as a teacher, researcher –  Research settings easier to control (classroom as a lab)

■  Informal Learning –  Better funding opportunities (H2020, industry) –  More innovative (mobile, Web, micro, games) –  Real impact expected



Learning Layers

LEARNING ANALYTICS



Learning Layers Self- and Community Regulated

Learning Processes

Based on [Fruhmann, Nussbaumer & Albert, 2010]

Learner profile information is

defined or revised

Learner finds and selects learning resources

Learner works on selected learning resources

Learner reflects and reacts on

strategies, achievements and usefulness

plan

learnreflect

The Horizon Report – 2011 Edition



Learning Layers The Long Tail of Personal Knowledge

in Lifelong Learning

■  Zillions of new learning opportunities ■  Abundance of learning materials ■  But: Extremely challenging to find & navigate

High-quality, specially designed, learning materials like books or course material

Gaps in personal knowledge identified mostly by real-world practice



Learning Layers

Personal Learning Environment (PLE) PLE describes the tools, communities, and services that constitute the individual educational platforms learners use to direct their own learning and pursue educational goals LMS – course-centric vs. PLE – learner-centric:

•  Extension of individual research •  Students in charge of their learning process

•  self-direction, responsibility •  Promotes authentic learning (incorporating expert feedback) •  Student’s scholarly work + own critical reflection + the work and voice of others •  Web 2.0 influence on educational process

•  customizable portals/dashboards, iGoogle, My Yahoo! •  Learning is a collaborative exercise in collection, orchestration, remixing, & integration of data into knowledge building •  Emphasis on metacognition in learning



Learning Layers ROLE Approach to the Design

of Learning Experiences

guidance & freedom of

learner

motivation of learner (intrinsic,

extrinsic)

stimulation of learner’s meta-

cognition

collaboration & good practice sharing among

peers

personalization & adaptability to learner & context What is the impact of these

findings from behavioral & cognitive psychology on

design of learning?

Goal setting Planning Reflection

Control & Responsibility Recommendation



Learning Layers ROLE Approach to the Design

of Learning Experiences What is the impact of these findings from behavioral & cognitive psychology on

design of Personal Learning Environments?

learner profile information is defined and revised

learner finds and selects learning resources

learner works on selected learning resources

plan

learn reflect

learner input regarding goals, preferences, …

creating PLE

recommendations from peers or tutors

assessment and self-assessment

evaluation and self-evaluation

feedback (from different sources)

learner should understand and control own learning process

ROLE infrastructure should provide adaptive guidance

attaining skills using different learning events (8LEM)

learner reflects and reacts on strategies, achievements,

and usefulness

monitoring recommen-dations

be aware of



Learning Layers Learning Analytics vs. Community

Learning Analytics Formal Learning Learning Analytics Community

Regulated Learning

Community Learning Analytics

Environment LMS EDM/VA CIS/ROLE DM/VA/SNA/Role Mining

Tools Fixed LMS Specific Eco-System Tool Recommender

Activities Fixed Content Recommender

Dynamic Content Recommender / Expert Recommender

Goals Fixed Progress Dynamic Progess / Goal Mining / Refinement

Communities Fixed Not applicable Dynamic (Overlapping) Community Detection

Use Cases Courses Learning Paths Peer Production / Scaffolding

Semantic Networks of Learners / Annotations



Learning Layers

COMMUNITY LEARNING ANALYTICS – A GENERAL APPROACH



Learning Layers

Communities of Practice ■ Communities of practice (CoP) are groups of people

who share a concern or a passion for something they do and who interact regularly to learn how to do it better (Wenger, 1998)

■  Characterization of experts in CoP –  Shared competence in the domain –  Shared practice over time by interactions –  Expertise based on gaining and having reputation within the CoP –  Being an expert vs. being a layman, a newcomer, an amateur etc. –  Informal leadership –  Identity as an expert depends on the lifecycle of the communities

Expertise in highly dynamic, locally distributed multi-disciplinary and heterogeneous communities?



Learning Layers Proposed Development of the

Community Learning Analytics Field ■  Will happen J Big Data by Digital Eco Systems (Quantitative Analysis)

–  A plethora of targets (Small Birds) –  Professional Communities are distributed in a long tail –  Professional Communities use a digital eco system

–  An arsenal of weapons (Big Guns) –  A growing number of community learning analytics methods –  Combined methods from machine intelligence and knowledge representation

■  May not happen L Deep Involvment with community (Qualitative Analysis) –  Domain knowledge for sense making –  Passion for community and sense of belonging –  Community learns as a whole

→ Community Learning Analytics for the Community by the Community



Learning Layers

Web 2.0 Competence Development Cultural and Technological

Shift by Social Software Impact on

Knowledge Work Impact on

Professional Communities

Web 1.0 Web 2.0 Microcontent Providing

commentary Personal knowledge

publishing Establishing personal

networks Testing Ideas

Social learning Identifying competences Emergent Collaboration

Trust & Social capital

personal website and content management

blogging and wikis User generated content Participation

directories (taxonomy) and stickiness

Tagging ("folksonomy") and syndication

Ranking Sense-making

Remixing Aggregation Embedding

Emergent Metadata Collective intelligence Wisdom of the Crowd Collaborative Filtering Visualizing Knowledge

Networks



Learning Layers Interdisciplinary Multidimensional

Model of Communities ■  Collection of CoP Digital Traces in a MediaBase

–  Post-Mortem Crawlers –  Real-time, mobile, protocol-based (MobSOS) –  (Automatic) metadata generation by Social Network Analysis

■  Social Requirements Engineering with i* Framework for defining goals and dependencies in CoP

Social Software Cross-Media Social Network Analysis on Wiki, Blog, Podcast, IM, Chat, Email, Newsgroup, Chat …

Web 2.0 Business Processes (i*) (Structural, Cross-media)

Members (Social Network Analysis: Centrality,

Efficiency, Community Detection)

Network of Artifacts Content Analysis on Microcontent, Blog entry, Message,

Burst, Thread, Comment, Conversation, Feedback (Rating)

Network of Members

Communities of practice

Media Networks



Learning Layers Community Learning Analytics

in CoP ■  User-to-Service Communication

•  CoP-aware Usage Statistics •  Identification of successful CoP services •  Identification of CoP service usage patterns

■  User-to-User Communication •  CoP-aware Social Network Analysis •  Identification of influential CoP members •  Identification of CoP member interaction/learning patterns

+



Learning Layers Supporting Community Practice

with the MobSOS Success Model



Learning Layers Community SRE Processes–

i* Strategic Rationale



Learning Layers

RESPONSIVE OPEN LEARNING ENVIRONMENTS



Learning Layers Responsive Open Learning

Enviroments (ROLE) 2009-2012

•  Empower the learner to build their own responsive learning environment ROLE Vision

•  Awareness and reflection of own learning process Responsiveness

•  Individually adapted composition of personal learning environment User-Centered



Learning Layers ROLE

Technical Infrastructure

■  Sucessfully deployed in industry and education ■ Open Source Software Development Kit ■ ROLE Widget Store (role-widgetstore.eu) ■ ROLE Sandbox (role-sandbox.eu)



Learning Layers ROLE Sandbox – Geospatial &

Temporal Access

§  Users: 5787 (95% external) §  Widgets: 1475 (71.5% external) §  Spaces: 1283 (64.3% external) §  Shared Resources: 18922 (6% external)



Learning Layers

ROLE Requirements Bazaar – Community-aware Requirements Prioritization

Factors influencing requirements ranking

User-controlled weighting of ranking factors

Community-dependent requirements ranking lists

http://requirements-bazaar.org



Learning Layers Learning Analytics Visualization –

Dashboards

1.  Database Selection

2.  Filter Selection/Definition

3.  Adapted Visualization



Learning Layers

LEARNING LAYERS – SCALING UP TECHNOLOGIES FOR INFORMAL LEARNING IN SME CLUSTERS



Learning Layers

Maturing

Interacting with People at the workplace Paul discovers a problem at the construction site with PLC equipment ...

Generating dynamic Learning Material The regional training center observes the Q&A and links it to their course material ...

Q: How to use PLC equipment …? •  I have seen this before here … • Last time I did it, I … • Here is something helpful

Social Semantic Layer Emerging shared meaning,

giving context Energy Consump.on

Lightning

X3-‐PVQ X3-‐PJC

X3-‐POZ PLC Equipment Instructional Taxonomy

• What is … • How to … • Example of …

Tutorial: How to Use PLC What is PLC How to use it? Examples Further Information Hot Questions and Answers

Work Practice Taxonomy •  Installation • Testing • Operation

Peter

Paul

Mary

Interacting in the Physical Workplace Physical workplace is equipped with QR tags, learning materials are delivered just in time ...

A list of helpful resources • Tutorials: How to use … • Persons: Peter, Mary, … • Work Practice: Installation,.. • Concepts: PLC, Lightning • Q&A: …,

Learning Layers in the Construction Industry



Learning Layers

Learning Layers – Scaling Technologies for Informal Learning

Learning Layers – Scaling up Technologies for Informal Learning in SME Clusters



Learning Layers

Space (shared by multiple users)

Using the ROLE Framework for Semantic Video Annotation

Web application (composed of widgets)

Widget (collaborative web component)

http://role-sandbox.eu/



Learning Layers

SeViAnno Prototypes SeViAnno (Web)

SeViAnno 2.0 (Widgets)

AnViAnno (Android)

AchSo! (Android)



Learning Layers

COMMUNITY LEARNING ANALYTICS – EXPERT IDENTIFICATION



Learning Layers Experts in

Learning Communities ■  In learning communities

many experts from different fields meet –  Intergenerational learning –  Interdisciplinary learning

■  New Openness for Amateur Contributions

■  Methods, Tools & CoP co-develop –  Expert role models needed –  Expert identification based

on complex media traces



Learning Layers YouTell - A Web 2.0 Service for

Collaborative Storytelling §  Collaborative storytelling §  Web 2.0 Service §  Story search and

“pro-sumption”

§  Tagging §  Ranking/Feedback §  Expert finding §  Recommending

Klamma, Cao, Jarke: Storytelling on the Web 2.0 as a New Means of Creating Arts Handbook of Multimedia for Digital Entertainment and Arts, Springer, 2009



Learning Layers Expert Finding – Computation of

Actual Knowledge ■  Data vector consists of

–  Personal data vector –  Competences, skills,

qualification profile –  Self-entered data

–  Story data vector –  Visits of stories –  Involvement in projects

–  Expert data vector –  Advice given –  Advice received

–  Value = #Keywords � Date Decay � Feedback

Motivation PESE: Web 2.0 –Anwen- dung für community- basiertes Storytelling Der PESE- Prototyp Evaluierung des Prototypen Zusammen- fassung Ausblick

Find the most appropriate expert

Data vector represents knowledge of the expert



Learning Layers Knowledge-Dependent

Learning Behaviour in Communities

Renzel, Cao, Lottko, Klamma: Collaborative Video Annotation for Multimedia Sharing between Experts and Amateurs, WISMA 2010, Barcelona, Spain, May 19-20, 2010

§  Expert finding algorithm: Knowledge value of community sorted by keywords §  Community behavior: Experts spent more time on the services §  Experts prefers semantic tags while amateurs uses “simple” tags frequently §  Community tags: Experts use more precise tags



Learning Layers

Threads to Expert Finding ■  Compromising techniques

—  Sybil attack [Douc 2002], Reputation theft, Whitewashing attack, etc.. —  Compromising the input and the output of the expert identification algorithm

■  Example: Sybil attacks —  Fundamental problem in open collaborative Web systems —  A malicious user creates many fake accounts (Sybils) which all reference the user to

boost his reputation (attacker’s goal is to be higher up in the rankings)

Sybil region Honest region ABack edges



Learning Layers

Conclusions & Outlook

■ Community Learning Analytics –  Informal learning more challenging for learning analytics –  New research challenges and funding opportunities –  Highly interdisciplinary and multi-method research

■ Case Studies –  Responsive Open Learning Environments – ROLE SDK for Near Real-Time Widget-Based Web Applications

–  Learning Layers - Scaling up Technologies for Informal Learning in SME Clusters –  Informal Learning on the Workplace – Collaborative Semantic Video Annotation

Community Learning Analytics – A New Research Field in TEL

Science

Transcript of Community Learning Analytics – A New Research Field in TEL