Tracking Pathways to Success
Identifying Learning Success Factors Across Course Delivery Formats
Peter Usinger, State Assessment Meeting 2013
Student Success in College Environment
High W/F
As high as 50% attrition
of FTIC cohort
45% of Withdrawal Reasons are
Course Related
Student Attributes
Financially Limited
Non-traditional
Academically Underprepared
Study Context
Reviewing High F/W
Engagement in College Prep
F2F vs. Online Instruction
Nursing OrientationSLS Course
Support
Research Questions
Q1
• Delivery format based differences between self-directed behaviors and learning strategies.
Q2
• Similarities and Differences across subject domains/disciplines and their sensitivity to course delivery formats.
Q3
• Relationships among MSLQ constructs, and with student characteristics, course characteristics, and delivery formats.
Q4
• “Predictive” model to explain the relationships for developing support mechanisms and change curricular design for greater success/completion
(Online) Course Completion
Student Characteristics
Demographic variables
Previous e-learning experience
Self-Directed Learning Habits
Metacognitive skills
Motivation
Self-discipline
Autonomy
Self-regulated behaviors
Sample drawn from…
Mixed withdrawal/failure rates136 Courses• Humanities, English/Letters, Mathematics and Statistics,
Sciences, Social Sciences, Nursing; Hybrids and Workforce Dev. excluded from analysis due to low frequencies.
Delivery• Online (216 Sections)• Face to Face (931 Sections)
>20,000 StudentsPopulation• Invited via email to complete MSLQ online (15 min.)
1,179 Course sections (total)
Sample
by
Academic A
rea
2,200 Participants:11% Response Rate
Discipline F2F Online
Humanities 43 29
Letters 314 84
Math 957 82
Nursing 136 33
Sciences 207 56
Social Sciences 85 64
MSLQ Scales and Reliability
Motivated Strategies for Learning Questionnaire (MSLQ)
Original MSLQ Applied in Study
Items α Items α
A. Motivational Constructs1. Intrinsic Goal Orientation 4 0.74 4 0.752. Extrinsic Goal Orientation 4 0.62 2 0.633. Task Value 6 0.90 2 0.904. Control of Learning Beliefs 4 0.68 3 0.765. Self-Efficacy for Learning & Performance 8 0.93 8 0.966. Test Anxiety 5 0.80 4 0.75 B. Learning Strategies Constructs 1. Rehearsal 4 0.69 2 0.692. Elaboration 6 0.75 2 0.583. Organization 4 0.64 2 0.544. Critical Thinking 5 0.80 5 0.805. Metacognitive Self-Regulation 12 0.79 10 0.846. Time/Study Environmental Management 8 0.76 7 0.827. Effort Regulation 4 0.69 4 0.718. Peer Learning 3 0.76 3 0.779. Help Seeking 4 0.52 3 0.72Total Items in Questionnaire 81 61
Motivational Correlations
Pearson Correlation Coefficients (N=2,200)
…between Grades Passing Online+
…and Motivational Constructs
1. Intrinsic Goal Orientation 0.19*** 0.16*** 0.04
2. Extrinsic Goal Orientation 0.03 0.08*** -0.04
3. Task Value 0.24*** 0.19*** 0.04*
4. Control of Learning Beliefs 0.30*** 0.25*** 0.10***
5. Self-Efficacy for Learning & Performance 0.46*** 0.37*** 0.07**
6. Test Anxiety -0.23*** -0.15*** -0.01
Note 1: *p < .05. **p < .01. ***p < .001. Note 2: +Method: F2F=0, Online=1.
Learning Strategies
Pearson Correlation Coefficients (N=2,200)
…between Grades Passing Online
…and Learning Strategies Constructs
1. Rehearsal 0.00 0.02 -0.06**
2. Elaboration 0.09*** 0.07** 0.02
3. Organization 0.04 0.03 -0.06*
4. Critical Thinking 0.04 0.01 0.06*
5. Metacognitive Self-Regulation 0.04 0.04* 0.00
6. Time/Study Environmental Management 0.19*** 0.13*** 0.07**
7. Effort Regulation 0.27*** 0.21*** 0.06*
8. Peer Learning -0.04 -0.02 -0.19***
9. Help Seeking -0.04 0.00 -0.32***
(where *p < .05. **p < .01. ***p < .001.)
Due to the very minor statistical differences in the results for grades vs. course success, and the fact that a significantly higher proportion of C and D students across most discipline areas are subsequently failing, we’ve created a Student Performance aggregate variable that combines student’s Course Performance into three levels:
• Level 0: Failure or Withdrawal from Course• Level 1: Course Grade Equals C or D• Level 2: Course Grade Equals A or B
This Performance variable is used in the student success analysis following.
Data Aggregation Rules
Comparison by Delivery Type
MSLQ Correlation w/ Course Performance
Online Face to Face
Mean Correlation Mean Correlation
Intrinsic Goal Or. 5.11 0.18*** 4.99 0.19***
Extrinsic Goal Or. 5.82 0.01 5.94 0.06*
Task Value 4.99 0.23*** 4.79 0.23***
Control Beliefs 5.83 0.31*** 5.49 0.28***
Self-Efficacy 5.54 0.43*** 5.20 0.44***
Test Anxiety 4.42 -0.21*** 4.47 -0.21***
Rehearsal 4.79 -0.04 5.04 0.03
Elaboration 5.28 0.12* 5.20 0.06*
Organization 5.00 0.04 5.21 0.03
Critical Thinking 4.33 0.07 4.13 0.01
Metacog. S-Reg. 4.86 0.03 4.84 0.04
Time/Study Mgmt. 5.48 0.22*** 5.26 0.15***
Effort Regulation 5.86 0.25*** 5.69 0.24***
Peer Learning 2.58 -0.07 3.41 0.00
Help Seeking 3.10 -0.02 4.52 0.01
N= 366 1,751
Comparison by Course Level
MSLQ Correlation w/ Course Performance
Developmental College Level
Mean R Coeff. Mean R Coeff.
Intrinsic Goal Or. 4.97 0.21*** 5.03 0.17***
Extrinsic Goal Or. 6.04 0.07 5.84 0.05*
Task Value 4.59 0.24*** 4.94 0.21***
Control Beliefs 5.48 0.31*** 5.57 0.28***
Self-Efficacy 5.15 0.43*** 5.32 0.44***
Test Anxiety 4.56 -0.23*** 4.40 -0.19***
Rehearsal 5.13 0.07 4.92 0.00
Elaboration 5.14 0.06 5.26 0.09***
Organization 5.33 0.05 5.10 0.04
Critical Thinking 4.13 -0.02 4.21 0.04
Metacog. S-Reg. 4.94 0.10* 4.80 0.03
Time/Study Mgmt. 5.42 0.23*** 5.24 0.17***
Effort Regulation 5.79 0.27*** 5.69 0.25***
Peer Learning 3.33 0.00 3.22 -0.04
Help Seeking 4.45 0.00 4.19 -0.01
N= 551 962
Important Note by Author:Please note that the following slide has been corrected since it contained a transfer error due to a truncated spreadsheet field. Affected is one of the interesting findings with regard to the role of test anxiety in the course performance of Nursing students. The original presentation slides showed a positive correlation between Test Anxiety and Course Performance for the whole Nursing group in this sample. However, this is only true for the 2nd year cohort of the RN program involved. Pre-admission, 1st year, and BSN students show patterns similar to other academic areas.
This is an extremely valuable research result since this correlation also goes along with a positive correlation between Test Anxiety and Rehearsal Strategies and Peer Learning. This indicates that our 2nd year Nursing students were not negatively overwhelmed with anxiety, but had learned to offset their anxiety levels successfully by rehearsing the subject matter at hand, and working with their peers to master the course content.
We will follow-up on these findings with an appropriate discipline-specific pathway model in the next phase of this longitudinal research project.
Correlations by Academic Area
Pearson Correlation with Course Performance
Humanities (N=75)
Letters (N=416)
Math (N=1043)
Nursing (N=174)
Sciences (N=276)
Soc. Sci. (N=156)
Motivational Constructs
1. Intrinsic Goal Orientation 0.14 0.07 0.22 0.22 -0.04 0.24
2. Extrinsic Goal Orientation -0.01 0.11 0.06 0.06 0.00 0.12
3. Task Value 0.25 0.10 0.21 0.27 -0.03 0.32
4. Control of Learning Beliefs 0.30 0.16 0.31 0.48 0.15 0.27
5. Self-Efficacy Learn. & Perf. 0.32 0.29 0.46 0.55 0.34 0.33
6. Test Anxiety -0.15 -0.04 -0.27 -0.20 -0.16 -0.09
Learning Strategies
1. Rehearsal -0.06 0.03 0.01 0.06 -0.08 0.12
2. Elaboration 0.23 0.11 0.04 0.13 -0.04 0.12
3. Organization -0.05 0.10 0.02 0.02 -0.04 0.11
4. Critical Thinking 0.09 0.00 -0.02 0.07 -0.13 0.00
5. Metacognitive Self-Reg. 0.02 0.03 0.07 0.11 -0.12 0.11
6. Time/Study Management 0.20 0.25 0.15 0.28 0.09 0.31
7. Effort Regulation 0.19 0.22 0.27 0.39 0.05 0.32
8. Peer Learning -0.12 -0.11 0.02 0.02 -0.17 0.02
9. Help Seeking -0.15 0.03 0.02 -0.03 -0.14 0.08
Success Comparison in Math
Example: Math Differences Failure/Withdrawal Passing Grade Difference
Motivational Constructs
Intrinsic Goal Orientation 58.3% 66.8% 8.5%
Extrinsic Goal Orientation 79.6% 84.0% 4.5%
Task Value 48.1% 59.7% 11.6%
Control Beliefs 63.4% 77.9% 14.5%
Self-Efficacy 50.0% 73.8% 23.7%
Test Anxiety 67.6% 55.6% -11.9%
Learning Strategies Constructs
Rehearsal 65.1% 67.1% 2.0%
Elaboration 65.9% 68.3% 2.4%
Organization 67.6% 70.1% 2.5%
Critical Thinking 50.0% 48.9% -1.1%
Meta-Cognitive Self-Regulation 61.5% 64.8% 3.3%
Time/Study Management 67.1% 73.4% 6.3%
Effort Regulation 70.4% 81.2% 10.7%
Peer Learning 36.0% 37.3% 1.4%
Help Seeking 53.7% 56.1% 2.5%
Multivariate Regression I
Multivariate Regression Analyses (Total Sample)
Predicting Course Performance
F-Value Pr > F R-Square
Fit of Model 42.1 <.0001 0.24
Parameter Estimates t-Value Pr > |t| St-B
Intrinsic Goal Orientation -1.97 0.05 -0.06
Extrinsic Goal Orientation -2.52 0.01 -0.06
Task Value 0.51 0.61 0.01
Control of Learning Beliefs -1.84 0.07 -0.06
Self-Efficacy for Learning 14.84 <.0001 0.52
Test Anxiety -1.46 0.14 -0.03
Rehearsal 1.27 0.20 0.03
Elaboration -0.47 0.64 -0.01
Organization -1.09 0.27 -0.03
Critical Thinking -0.75 0.45 -0.02
Metacognitive Self-Regulation -1.84 0.07 -0.07
Time/Study Management 2.26 0.02 0.07
Effort Regulation 2.95 0.00 0.09
Peer Learning -1.07 0.29 -0.03
Help Seeking -0.42 0.67 -0.01
Multivariate Regression II
Multivariate Regression Analyses w/ Performance
Online Courses Face-to-Face Classes
F-Value Pr > F R-Square F-Value Pr > F R-Square
Fit of Model 7.1 <.0001 0.22 33.2 <.0001 0.24
Parameter Estimates t-Value Pr > |t| St-B t-Value Pr > |t| St-B
Intrinsic Goal Orientation -0.35 0.73 -0.03 -1.47 0.14 -0.05
Extrinsic Goal Orientation -0.54 0.59 -0.03 -1.66 0.10 -0.04
Task Value -0.25 0.80 -0.02 0.27 0.78 0.01
Control of Learning Beliefs 0.57 0.57 0.04 -2.30 0.02 -0.08
Self-Efficacy for Learning 5.36 <.0001 0.44 13.60 <.0001 0.54
Test Anxiety -0.40 0.69 -0.02 -1.83 0.07 -0.05
Rehearsal -0.77 0.44 -0.05 2.14 0.03 0.07
Elaboration -0.26 0.80 -0.02 -0.76 0.45 -0.02
Organization -0.18 0.86 -0.01 -1.17 0.24 -0.04
Critical Thinking 1.54 0.13 0.11 -1.88 0.06 -0.06
Metacognitive Self-Regulation -1.67 0.10 -0.15 -1.12 0.26 -0.05
Time/Study Management 2.37 0.02 0.17 0.85 0.39 0.03Effort Regulation -0.05 0.96 0.00 2.92 0.00 0.10Peer Learning -0.99 0.33 -0.07 -0.48 0.63 -0.01
Help Seeking -0.04 0.97 0.00 -0.14 0.89 0.00
Multivariate Regression III
Multivariate Regression Analyses w/ Performance
Developmental College Level
F-Value Pr > F R-Square F-Value Pr > F R-Square
Fit of Model 14.0 <.0001 0.25 30.0 <.0001 0.24
Parameter Estimates t-Value Pr > |t| St-B t-Value Pr > |t| St-B
Intrinsic Goal Orientation -0.51 0.61 -0.03 -2.07 0.04 -0.07
Extrinsic Goal Orientation -2.20 0.03 -0.09 -1.21 0.23 -0.03
Task Value 0.34 0.73 0.02 -0.27 0.78 -0.01
Control of Learning Beliefs -0.42 0.67 -0.02 -2.08 0.04 -0.08
Self-Efficacy for Learning 7.31 <.0001 0.48 13.29 <.0001 0.55
Test Anxiety -1.90 0.06 -0.08 -0.72 0.47 -0.02
Rehearsal 1.64 0.10 0.08 0.74 0.46 0.02
Elaboration -1.97 0.05 -0.11 -0.30 0.76 -0.01
Organization -0.36 0.72 -0.02 -0.39 0.70 -0.01
Critical Thinking -2.03 0.04 -0.11 0.14 0.89 0.00
Metacognitive Self-Regulation 1.08 0.28 0.07 -2.55 0.01 -0.11
Time/Study Management 2.33 0.02 0.13 1.44 0.15 0.05
Effort Regulation 0.69 0.49 0.04 3.07 0.00 0.11
Peer Learning -0.33 0.75 -0.02 -1.04 0.30 -0.03
Help Seeking -0.77 0.44 -0.04 0.47 0.64 0.02
• MSLQ scales: excellent reliability & context sensitivity• Motivational factors clearly driving student success• Learning Strategies (cognitive skills) show multi-faceted instructional improvement (learning) opportunities
• Peer Learning/Help Seeking: Underrepresented within cohorts, not an integrated/facilitated college activity
• Many underprepared students succeed largely via intrinsic motivation and confidence/self-efficacy, combined with hard work and study management
• The lack of exposure to meta-cognitive abilities and critical thinking skills nurtures course environments in support of compliant learners instead of self-directed learners!
Conclusions
Study Impact / Next Steps
• Online: Assess Quality Matters impact across delivery formats and feed information back into process
• 1st Year/EWS: Compare demographic differences associated with the various success patterns
• SLS: Evaluate impact with SLS courses when applying the MSLQ as formative assessment tool early in Student Success or Developmental Ed classes
• Overall: Conduct additional focus groups at college sites; seek expansion of study to inform interventions
• FCS: Seek study replication across service areas
Top Related