Post on 23-Dec-2015
Faculty Perceptionsof Fall 2011 IDEA
Preliminary Report to CNU Faculty Senate
April 2012
By Deborah Moore, Director, OAEA for IDEA TaskForce & CNU Faculty
Purposes of the Survey
• gather information about Fall 2011 IDEA experience . . . targeting the online. . .
• learn what information you would like to have about IDEA timeline, reports, etc.
• identify what you need to know about the reports to help you improve instruction in your classroom and those planned collectively by your department.
Background Information
Response Rate: 192 submitted of 394 invited (49%); 8 reported “none of my classes were evaluated this fall”)By subgroup: (not all respondents provided a response to the status item)
Adjunct, 30 of 148 (20%)Restricted, 44 of 78 (56%)Probationary, 31/53 (58%)Tenured, 80 of 115 (70%)
Which best describes your status in Fall 2011?
Answer Response %
adjunct 29 15%
restricted 47 24%
probationary, hired before Fall 2011 26 14%
probationary, new hire as of Fall 2011 8 4%
tenured 82 43%
Total 192 100%
Which type of administration did you experience this Fall?
Answer Res-ponse
%
none of my classes were evaluated this fall 8 4%
online only, short form 82 43%
online only, diagnostic form 40 21%
online, both short and diagnostic forms 34 18%
paper only, diagnostic form 24 13%
paper and online, short form 2 1%
paper and online, both short & diagnostic 2 1%
Total 192 100%
Importance: Teaching EffectivenessOverall, how important is IDEA to you with respect to your own teaching efforts and improvement practices.
Satisfaction: Teaching EffectivenessOverall, how satisfied are you with IDEA with respect to your own teaching efforts and improvement practices.
Importance: EvaluationOverall, how important is IDEA to you with respect to CNU's
process for instructor/course evaluation?
Satisfaction: EvaluationOverall, how satisfied are you with IDEA with respect to CNU's process for
instructor/course evaluation?
Teaching Effectiveness: Importance by Satisfaction
(Q1.6)Overall, how satisfied are you with IDEA with respect to your own teaching efforts and improvement p...
Very Dissatisfied Dissatisfied Somewhat Dissatisfied Neutral Somewhat
Satisfied Satisfied Very Satisfied Total
Not at all Important 4 0 0 0 0 0 0 4
(Q1.5)Overall, how important is IDEA to you
with respect to your own teaching efforts and
improvement prac...
Very Unimportant 0 1 0 1 1 3 0 6
Somewhat Unimportant 1 4 3 2 4 0 1 15
Neither Important nor Unimportant 0 0 0 3 0 0 0 3
Somewhat Important 1 3 6 15 14 17 1 57
Very Important 2 3 5 3 19 41 1 74
Extremely Important 0 0 1 0 3 8 5 17
Total
8 11 15 24 41 69 8 176
Evaluation: Importance by Satisfaction
(Q1.8)Overall, how satisfied are you with IDEA . . .instructor/course evaluation
Very Dissatisfied Dissa-tisfied
Somewhat Dissatisfied Neutral
Somewhat Satisfied Satisfied
Very Satisfied Total
Not at all Important
4 1 0 0 0 0 0 5
(Q1.7)Overall, how important is IDEA .
. . for instructor/course
evaluation
Very Unimportant2 4 0 2 0 0 0 8
Somewhat Unimportant
3 3 1 0 0 0 0 7
Neither Important nor Unimportant
0 2 4 7 1 1 0 15
Somewhat Important
1 5 6 7 20 4 0 43
Very Important1 6 9 6 17 20 3 62
Extremely Important
3 5 9 2 8 7 2 36
Total 14 26 29 24 46 32 5 176
Please rate the following aspects of the your IDEA experience:
Statistic
I knew
how to
select
objecti
ves f
or each
of my
course
s.
The on
line F
IF was
easy to
comple
te.
My colle
agues
were ab
le to help
me w
ith th
e FIF,
if I
needed
it.
The Office
of Asse
ssment
was able
to re
solve
issues
I had
with
completi
ng the F
IF.
I knew
when
the ID
EA ad
ministrati
on was
schedule
d.
I know how to
use m
y repo
rts to
improve
instr
uct-
tion.I k
now how to use
my I
DEA re
ports f
or ten
ure an
d
personne
l acti
vities.
Overal
l I though
t that
the onlin
e exp
erienc
e was
more co
nvenien
t than i
nconve
nient.
Online, Short FormMean 4.14 4.14 3.63 3.36 4.03 3.82 3.60 3.28
SD 0.90 0.63 0.80 0.81 0.93 0.94 0.99 1.25N 73 72 71 72 73 72 72 72
Mean 3.77 4.05 3.92 3.21 3.81 3.74 3.42 3.33SD 1.04 0.69 0.75 0.57 0.60 0.97 1.06 1.42N 39 39 38 39 39 39 38 39
Mean 3.78 4.09 3.65 3.41 4.00 3.79 3.47 3.26SD 0.88 0.58 0.98 0.88 0.95 0.77 0.90 1.11N 34 33 32 33 34 34 34 34
Mean 4.17 3.06 3.37 3.11 4.04 3.96 3.83SD 0.76 0.64 1.07 0.68 0.91 0.91 1.01 NAN 24 18 18 18 24 24 24
Scale valuesStrongly Disagree
=1
Disagree=2
Neither Agree/
Disagree=3
Agree=4Strongly Agree=5
Online, Short Form
Online, Short & Diagnostic Forms
Online, Diagnostic Form
Paper, Diagnostic
What did you do to encourage or ensure that your students completed their IDEAs for your classes? (Select all that
apply.)
Statistic
Discussed the
importance of student
participation in IDEA
evaluations
Discussed my choice
of objectives and how
they related to the class
Offered rewards for
participation
Set aside classroom time for students to bring laptops
and complete
the evaluation
I didn't do anything to encourage
IDEA participation
Online, Short Form (N=72)
% 88 21 4 6 10
Online, Diagnostic Form (N=39)
% 72 26 18 15 15
Online, Short & Diagnostic Forms
(N=34)% 76 24 9 9 12
Paper, Diagnostic (N=21) % 76 33 10 10 10
Stat-istic All 1st Page
Stats at end and
items I added Other
Comments primarily
Online, Short Form (N=74)
% 51 18 7 1 23
Online, Diagnostic Form (N=39)
% 77 5 0 3 15
Online, Short & Diagnostic Forms
(N=34)
% 59 12 12 3 15
Paper, Diagnostic (N=24)
% 50 8 13 17 13
Which elements of your individual course reports do you use?
Statistic
weeks 11&12
weeks 12&13
weeks 13&14 (don't
change the
timeframe)
weeks 14&15 other
Online, Short Form (N=71) % 3 6 61 25 6Online, Diagnostic Form
(N=38)% 3 5 45 42 5
Online, Short & Diagnostic Forms (N=34)
% 9 6 50 26 9
Paper, Diagnostic (N=24) % 8 13 50 21 8
Currently the IDEA is administered weeks 13 and 14 of the term. What weeks would you prefer?
IDEA Webpages
• http://www.cnu.edu/assessment/idea/index.asp
• http://www.theideacenter.org/
Preliminary Summary/Highlights
• Survey response rates were acceptable, although low (56%-70%) for subgroups except adjunct (20%).
• Nearly every department had faculty participating in the survey.
• As expected more respondents indicated having participated with an online experience (160/192 or 83%) and paper was limited to a small subgroup (26/192 or 14%). A small group had no classes evaluated and ended their survey after reporting their faculty status (8/192 or 4%).
• Although not ideal, these values can serve as a benchmark for comparison about faculty perceptions of the online application of the IDEA.
• The majority of the respondents indicate good knowledge about IDEA and related procedures (75%); however 25% indicate the need information about both the form and procedures.
• Faculty were asked to provide an overall rating of importance and satisfaction about two contexts of IDEA use: IDEA for personal teaching improvement, and IDEA as used by CNU for course/instructor evaluation.
• In the 1st of 2 contexts, use of IDEA for personal teaching improvement efforts, 84% or 148/176 respondents indicated the tool was important (EI + VI + SI). Within that subgroup, 74% were satisfied and 14% were dissatisfied with IDEA. Although room for improvement, overall this is a positive outcome.
• Similarly 80% or 141/176 respondents indicated the tool was important (EI + VI + SI) in the context of CNUS use of IDEA for course/instructor evaluation. Within this subgroup of those who indicate CNU’s use of IDEA for course/instructor evaluation is important, 57% were satisfied and 32% were dissatisfied.
Continued• Efforts to determine what is
dissatisfying—and addressing the responses should help to shift these ratios in a more positive direction
• Strategies instructors used to encourage student completion of IDEA form are similar except those involved with the Diagnostic form (either paper or online) used slightly more strategies than those involved with the Short form.
• Weeks 13 and 14 (the current timeframe) was preferred by the largest percent of respondents in the various subgroups (by form), however there was also support for pushing the administration to weeks 14 and 15.
• Regardless of format subgroup, comments were fewer in quantity and less helpful qualitatively.