Using Analytics to Examine Course Design and Identify Effective Practices (166234657)
Transcript of Using Analytics to Examine Course Design and Identify Effective Practices (166234657)
![Page 1: Using Analytics to Examine Course Design and Identify Effective Practices (166234657)](https://reader035.fdocuments.us/reader035/viewer/2022081401/577cd9121a28ab9e78a29f7e/html5/thumbnails/1.jpg)
7/29/2019 Using Analytics to Examine Course Design and Identify Effective Practices (166234657)
http://slidepdf.com/reader/full/using-analytics-to-examine-course-design-and-identify-effective-practices-166234657 1/31
Using Analytics to Examine CourseDesign and Identify Effective Practices
Feb. 6, 2013
Karin Readel, University of Maryland, Baltimore County
![Page 2: Using Analytics to Examine Course Design and Identify Effective Practices (166234657)](https://reader035.fdocuments.us/reader035/viewer/2022081401/577cd9121a28ab9e78a29f7e/html5/thumbnails/2.jpg)
7/29/2019 Using Analytics to Examine Course Design and Identify Effective Practices (166234657)
http://slidepdf.com/reader/full/using-analytics-to-examine-course-design-and-identify-effective-practices-166234657 2/31
For Audience Participation:
http://rwpoll.com
Session ID = eli2013
![Page 3: Using Analytics to Examine Course Design and Identify Effective Practices (166234657)](https://reader035.fdocuments.us/reader035/viewer/2022081401/577cd9121a28ab9e78a29f7e/html5/thumbnails/3.jpg)
7/29/2019 Using Analytics to Examine Course Design and Identify Effective Practices (166234657)
http://slidepdf.com/reader/full/using-analytics-to-examine-course-design-and-identify-effective-practices-166234657 3/31
Which best describes your feelings about theresults of Superbowl 2013?
1. Go Ravens! Woot!2. Total depression over the loss
3. Meh
4. Superbowl, schmooper-bowl. It’s all aboutthe Puppybowl!
![Page 4: Using Analytics to Examine Course Design and Identify Effective Practices (166234657)](https://reader035.fdocuments.us/reader035/viewer/2022081401/577cd9121a28ab9e78a29f7e/html5/thumbnails/4.jpg)
7/29/2019 Using Analytics to Examine Course Design and Identify Effective Practices (166234657)
http://slidepdf.com/reader/full/using-analytics-to-examine-course-design-and-identify-effective-practices-166234657 4/31
About Blackboard @ UMBC
Began using Bb in SP2000 Campus Adoption
95% of all students
75% of all instructors 65% of all courses
350 Organizations
![Page 5: Using Analytics to Examine Course Design and Identify Effective Practices (166234657)](https://reader035.fdocuments.us/reader035/viewer/2022081401/577cd9121a28ab9e78a29f7e/html5/thumbnails/5.jpg)
7/29/2019 Using Analytics to Examine Course Design and Identify Effective Practices (166234657)
http://slidepdf.com/reader/full/using-analytics-to-examine-course-design-and-identify-effective-practices-166234657 5/31
www.umbc.edu/blackboard/reports
![Page 6: Using Analytics to Examine Course Design and Identify Effective Practices (166234657)](https://reader035.fdocuments.us/reader035/viewer/2022081401/577cd9121a28ab9e78a29f7e/html5/thumbnails/6.jpg)
7/29/2019 Using Analytics to Examine Course Design and Identify Effective Practices (166234657)
http://slidepdf.com/reader/full/using-analytics-to-examine-course-design-and-identify-effective-practices-166234657 6/31
Most Active Bb Courses
![Page 7: Using Analytics to Examine Course Design and Identify Effective Practices (166234657)](https://reader035.fdocuments.us/reader035/viewer/2022081401/577cd9121a28ab9e78a29f7e/html5/thumbnails/7.jpg)
7/29/2019 Using Analytics to Examine Course Design and Identify Effective Practices (166234657)
http://slidepdf.com/reader/full/using-analytics-to-examine-course-design-and-identify-effective-practices-166234657 7/31
Bb Analytics for Learn
In 2012 UMBC participated in the LFT for BA4L
Implementation almost complete Interested in what the data could tell us
about the impact of our faculty training
efforts
![Page 8: Using Analytics to Examine Course Design and Identify Effective Practices (166234657)](https://reader035.fdocuments.us/reader035/viewer/2022081401/577cd9121a28ab9e78a29f7e/html5/thumbnails/8.jpg)
7/29/2019 Using Analytics to Examine Course Design and Identify Effective Practices (166234657)
http://slidepdf.com/reader/full/using-analytics-to-examine-course-design-and-identify-effective-practices-166234657 8/31
Alternate Delivery Program (ADP)
Training Program for faculty moving fromF2F to Hybrid or Online format
Provides 1 time $2500 stipend to redesigncourse
1 day “Hybrid Workshop”
Self-reflection on pedagogical problem posed
Interview with IT and FDC staff
Create & present 2 online activities
Teach redesigned course
Evaluate ADP process
![Page 9: Using Analytics to Examine Course Design and Identify Effective Practices (166234657)](https://reader035.fdocuments.us/reader035/viewer/2022081401/577cd9121a28ab9e78a29f7e/html5/thumbnails/9.jpg)
7/29/2019 Using Analytics to Examine Course Design and Identify Effective Practices (166234657)
http://slidepdf.com/reader/full/using-analytics-to-examine-course-design-and-identify-effective-practices-166234657 9/31
Questions:
Are there differences in course measuresbetween the various modes of instruction?
Does training make a difference?
Can we use BA4L data to identify effectivepractitioners?
What other (non-LMS) data can be included
in analysis?
![Page 10: Using Analytics to Examine Course Design and Identify Effective Practices (166234657)](https://reader035.fdocuments.us/reader035/viewer/2022081401/577cd9121a28ab9e78a29f7e/html5/thumbnails/10.jpg)
7/29/2019 Using Analytics to Examine Course Design and Identify Effective Practices (166234657)
http://slidepdf.com/reader/full/using-analytics-to-examine-course-design-and-identify-effective-practices-166234657 10/31
rwpoll.com session ID= eli2013
Which mode of instruction would you expect tofind the highest average level of courseinteraction?
1. F2F
2. Hybrid
3. Online
![Page 11: Using Analytics to Examine Course Design and Identify Effective Practices (166234657)](https://reader035.fdocuments.us/reader035/viewer/2022081401/577cd9121a28ab9e78a29f7e/html5/thumbnails/11.jpg)
7/29/2019 Using Analytics to Examine Course Design and Identify Effective Practices (166234657)
http://slidepdf.com/reader/full/using-analytics-to-examine-course-design-and-identify-effective-practices-166234657 11/31
rwpoll.com session ID= eli2013
Would you expect to see higher levels of course interaction with Graduate or Undergraduate students?
1. Graduate students
2. Undergraduate students
![Page 12: Using Analytics to Examine Course Design and Identify Effective Practices (166234657)](https://reader035.fdocuments.us/reader035/viewer/2022081401/577cd9121a28ab9e78a29f7e/html5/thumbnails/12.jpg)
7/29/2019 Using Analytics to Examine Course Design and Identify Effective Practices (166234657)
http://slidepdf.com/reader/full/using-analytics-to-examine-course-design-and-identify-effective-practices-166234657 12/31
F2F Hybrid Online
Undergrad 131.6 306.9 480.6
Grad 125 245.7 532.2
0
200
400
600
800
1000
1200
A v e r a g e # o f I n t e r a c t i o n s
Average # of Interactions - All Faculty
![Page 13: Using Analytics to Examine Course Design and Identify Effective Practices (166234657)](https://reader035.fdocuments.us/reader035/viewer/2022081401/577cd9121a28ab9e78a29f7e/html5/thumbnails/13.jpg)
7/29/2019 Using Analytics to Examine Course Design and Identify Effective Practices (166234657)
http://slidepdf.com/reader/full/using-analytics-to-examine-course-design-and-identify-effective-practices-166234657 13/31
All Trained All Trained All Trained
F2F Hybrid Online
Undergrad 131.6 149.8 306.9 468 480.6 659.6
Grad 125 132.5 245.7 250.8 532.2 401.6
0
200
400
600
800
1000
1200
A v e r a g e #
o f I n t e r a c t i o n s
Ave Interactions- All vs. ADP Trained
![Page 14: Using Analytics to Examine Course Design and Identify Effective Practices (166234657)](https://reader035.fdocuments.us/reader035/viewer/2022081401/577cd9121a28ab9e78a29f7e/html5/thumbnails/14.jpg)
7/29/2019 Using Analytics to Examine Course Design and Identify Effective Practices (166234657)
http://slidepdf.com/reader/full/using-analytics-to-examine-course-design-and-identify-effective-practices-166234657 14/31
0
10
20
30
40
50
60
F2F Hybrid Online
Ave Course Measures by Instruction Mode
for All Faculty
Ave Content
Ave Course Accesses
Ave Min/Access
![Page 15: Using Analytics to Examine Course Design and Identify Effective Practices (166234657)](https://reader035.fdocuments.us/reader035/viewer/2022081401/577cd9121a28ab9e78a29f7e/html5/thumbnails/15.jpg)
7/29/2019 Using Analytics to Examine Course Design and Identify Effective Practices (166234657)
http://slidepdf.com/reader/full/using-analytics-to-examine-course-design-and-identify-effective-practices-166234657 15/31
0
10
20
30
40
50
60
70
80
F2F Hybrid Online
Ave Course Measures by Instruction Mode for ADP
Faculty
Ave Content
Ave Course Accesses
Ave Min/Access
![Page 16: Using Analytics to Examine Course Design and Identify Effective Practices (166234657)](https://reader035.fdocuments.us/reader035/viewer/2022081401/577cd9121a28ab9e78a29f7e/html5/thumbnails/16.jpg)
7/29/2019 Using Analytics to Examine Course Design and Identify Effective Practices (166234657)
http://slidepdf.com/reader/full/using-analytics-to-examine-course-design-and-identify-effective-practices-166234657 16/31
0
10
20
30
40
50
60
70
80
Ave Content Ave Course
Accesses
Ave Min/Access
Effect of Training on Average Course
Measures
F2F All
F2F Trained
Hybrid All
Hybrid Trained
Online All
Online Trained
![Page 17: Using Analytics to Examine Course Design and Identify Effective Practices (166234657)](https://reader035.fdocuments.us/reader035/viewer/2022081401/577cd9121a28ab9e78a29f7e/html5/thumbnails/17.jpg)
7/29/2019 Using Analytics to Examine Course Design and Identify Effective Practices (166234657)
http://slidepdf.com/reader/full/using-analytics-to-examine-course-design-and-identify-effective-practices-166234657 17/31
content% content
accessedgradecenter tools assessments
All 69.6 41.3 37.9 64.3 40
Trained 81 43.3 54 77 58.3
0
10
20
30
40
50
60
70
80
90
% o
f C o u
r s e s U s i n g
Effect of Training on Course Measures
![Page 18: Using Analytics to Examine Course Design and Identify Effective Practices (166234657)](https://reader035.fdocuments.us/reader035/viewer/2022081401/577cd9121a28ab9e78a29f7e/html5/thumbnails/18.jpg)
7/29/2019 Using Analytics to Examine Course Design and Identify Effective Practices (166234657)
http://slidepdf.com/reader/full/using-analytics-to-examine-course-design-and-identify-effective-practices-166234657 18/31
F2F Hybrid Online All Modes
All Instructors 5.9 9.7 6.5 6.3
Trained 7.6 14.4 6.1 8.4
0
2
4
6
8
10
12
14
16
Effect of Training on Folder Depth Ratio
![Page 19: Using Analytics to Examine Course Design and Identify Effective Practices (166234657)](https://reader035.fdocuments.us/reader035/viewer/2022081401/577cd9121a28ab9e78a29f7e/html5/thumbnails/19.jpg)
7/29/2019 Using Analytics to Examine Course Design and Identify Effective Practices (166234657)
http://slidepdf.com/reader/full/using-analytics-to-examine-course-design-and-identify-effective-practices-166234657 19/31
BA4L Built in Reports
![Page 20: Using Analytics to Examine Course Design and Identify Effective Practices (166234657)](https://reader035.fdocuments.us/reader035/viewer/2022081401/577cd9121a28ab9e78a29f7e/html5/thumbnails/20.jpg)
7/29/2019 Using Analytics to Examine Course Design and Identify Effective Practices (166234657)
http://slidepdf.com/reader/full/using-analytics-to-examine-course-design-and-identify-effective-practices-166234657 20/31
Ability to Drill Down to Course
![Page 21: Using Analytics to Examine Course Design and Identify Effective Practices (166234657)](https://reader035.fdocuments.us/reader035/viewer/2022081401/577cd9121a28ab9e78a29f7e/html5/thumbnails/21.jpg)
7/29/2019 Using Analytics to Examine Course Design and Identify Effective Practices (166234657)
http://slidepdf.com/reader/full/using-analytics-to-examine-course-design-and-identify-effective-practices-166234657 21/31
rwpoll.com session ID= eli2013
Would you expect to see higher levels of course interaction in the LMS in courses using“clickers in class?
1. Yes
2. No
![Page 22: Using Analytics to Examine Course Design and Identify Effective Practices (166234657)](https://reader035.fdocuments.us/reader035/viewer/2022081401/577cd9121a28ab9e78a29f7e/html5/thumbnails/22.jpg)
7/29/2019 Using Analytics to Examine Course Design and Identify Effective Practices (166234657)
http://slidepdf.com/reader/full/using-analytics-to-examine-course-design-and-identify-effective-practices-166234657 22/31
Trends For Courses Using Clickers…
No difference in average interactions,course accesses, % of items accessed vs.non-clicker courses.
“Clicker courses” were more likely to useassessments (79% vs. 40%)
![Page 23: Using Analytics to Examine Course Design and Identify Effective Practices (166234657)](https://reader035.fdocuments.us/reader035/viewer/2022081401/577cd9121a28ab9e78a29f7e/html5/thumbnails/23.jpg)
7/29/2019 Using Analytics to Examine Course Design and Identify Effective Practices (166234657)
http://slidepdf.com/reader/full/using-analytics-to-examine-course-design-and-identify-effective-practices-166234657 23/31
What About Smart Room Use?
Smart Room = fixed data projector Lecture Rooms aren’t all centrally scheduled
79 Registrar; 55 Departmental
68 of 79 monitored by GVE
Projector use data was crossmapped with
the course schedule data to produce reportsshowing exactly when projectors were usedin specific courses
![Page 24: Using Analytics to Examine Course Design and Identify Effective Practices (166234657)](https://reader035.fdocuments.us/reader035/viewer/2022081401/577cd9121a28ab9e78a29f7e/html5/thumbnails/24.jpg)
7/29/2019 Using Analytics to Examine Course Design and Identify Effective Practices (166234657)
http://slidepdf.com/reader/full/using-analytics-to-examine-course-design-and-identify-effective-practices-166234657 24/31
![Page 25: Using Analytics to Examine Course Design and Identify Effective Practices (166234657)](https://reader035.fdocuments.us/reader035/viewer/2022081401/577cd9121a28ab9e78a29f7e/html5/thumbnails/25.jpg)
7/29/2019 Using Analytics to Examine Course Design and Identify Effective Practices (166234657)
http://slidepdf.com/reader/full/using-analytics-to-examine-course-design-and-identify-effective-practices-166234657 25/31
% Projector Usage for Courses in GVE
Monitored Rooms FA2011- SP2012
0 0-10 10-20 20-30 30-40 40-50 50-60 60-70 70-80 80-90 90-100
FA2011 11.5 14.4 11.1 8.8 5.9 7.7 7.4 6.7 12.0 9.4 5.1
SP2012 8.3 17.7 11.6 9.1 12.6 12.2 12.8 13.3 2.3 0.1 0.0
0.0
2.0
4.0
6.0
8.0
10.0
12.0
14.0
16.0
18.0
20.0
% C
o u r s
e s
![Page 26: Using Analytics to Examine Course Design and Identify Effective Practices (166234657)](https://reader035.fdocuments.us/reader035/viewer/2022081401/577cd9121a28ab9e78a29f7e/html5/thumbnails/26.jpg)
7/29/2019 Using Analytics to Examine Course Design and Identify Effective Practices (166234657)
http://slidepdf.com/reader/full/using-analytics-to-examine-course-design-and-identify-effective-practices-166234657 26/31
Room Size of Courses with <10%
Projector Use
<40 40-60 60-80 80-100 100-150 >150
FA2011 26.6 47.3 10.4 10.0 2.9 2.9
SP2012 39.6 38.3 8.7 4.8 4.3 4.3
0.0
5.0
10.0
15.0
20.0
25.0
30.0
35.0
40.0
45.0
50.0
% o
f C o u r s e s
![Page 27: Using Analytics to Examine Course Design and Identify Effective Practices (166234657)](https://reader035.fdocuments.us/reader035/viewer/2022081401/577cd9121a28ab9e78a29f7e/html5/thumbnails/27.jpg)
7/29/2019 Using Analytics to Examine Course Design and Identify Effective Practices (166234657)
http://slidepdf.com/reader/full/using-analytics-to-examine-course-design-and-identify-effective-practices-166234657 27/31
rwpoll.com session ID= eli2013
Do you think that hybrid courses would usedata projectors more or less than F2Fcourses?
1. More
2. Less
3. No Difference
![Page 28: Using Analytics to Examine Course Design and Identify Effective Practices (166234657)](https://reader035.fdocuments.us/reader035/viewer/2022081401/577cd9121a28ab9e78a29f7e/html5/thumbnails/28.jpg)
7/29/2019 Using Analytics to Examine Course Design and Identify Effective Practices (166234657)
http://slidepdf.com/reader/full/using-analytics-to-examine-course-design-and-identify-effective-practices-166234657 28/31
0
5
10
15
20
25
30
35
SP2012 FA2012
A v e % P
r o j e c t o r U s e
% Projector Use - All Courses vs. Hybrid
All Courses
Hybrid
![Page 29: Using Analytics to Examine Course Design and Identify Effective Practices (166234657)](https://reader035.fdocuments.us/reader035/viewer/2022081401/577cd9121a28ab9e78a29f7e/html5/thumbnails/29.jpg)
7/29/2019 Using Analytics to Examine Course Design and Identify Effective Practices (166234657)
http://slidepdf.com/reader/full/using-analytics-to-examine-course-design-and-identify-effective-practices-166234657 29/31
0
2
4
6
8
10
12
14
16
0 1-10 11-20 21-30 31-40 41-50 51-60 61-70 71-80 81-90 91-100
% o
f H y b r i d C o
u r s e s
% Projector Use Hybrid Courses SU2012
![Page 30: Using Analytics to Examine Course Design and Identify Effective Practices (166234657)](https://reader035.fdocuments.us/reader035/viewer/2022081401/577cd9121a28ab9e78a29f7e/html5/thumbnails/30.jpg)
7/29/2019 Using Analytics to Examine Course Design and Identify Effective Practices (166234657)
http://slidepdf.com/reader/full/using-analytics-to-examine-course-design-and-identify-effective-practices-166234657 30/31
Next Steps:
Look at impact of training over time onspecific faculty
Look at impact of “redesign” on specific
courses Add clicker and projector use data to iBBLA
cube
![Page 31: Using Analytics to Examine Course Design and Identify Effective Practices (166234657)](https://reader035.fdocuments.us/reader035/viewer/2022081401/577cd9121a28ab9e78a29f7e/html5/thumbnails/31.jpg)
7/29/2019 Using Analytics to Examine Course Design and Identify Effective Practices (166234657)
http://slidepdf.com/reader/full/using-analytics-to-examine-course-design-and-identify-effective-practices-166234657 31/31
Using Analytics to Examine CourseDesign and Identify Effective Practices
Karin Readel
DoIT, UMBC