Transcript of Symposium visual final
- 1. NEERO 45th Annual Confernece Symposium 2.2 | Thursday,
10:15-11:45am Symposium Title: Student Evaluations of Higher and
Basic Education Instructors: Three Case Studies Presenters:Tom
NeuschaferNichol MurrrayDiane Onorato
- 2. Objectives of Presentation Secondary Heading 1. To
demonstrate that high school students can recognize good teaching
and can collaborate to build an assessment tool which parallels
professionally tested instruments 2. To demonstrate evidence that
college students must have buy-in and confidence that their
opinions are valued and to share strategies for increasing the
response rates of evaluations. 3. To demonstrate a process by which
an online or blended course can be improved by student evaluations
using an instrument which correlates to the course design and
management
- 3. Student-Designed Evaluations The Evaluation The need for an
evaluation Wrights article The evaluation What they were told /
what they werent told Questions on parenthesis Grading system
Explanation sections
- 4. Student-Designed Evaluations The Evaluation Lets take a look
at the actual evaluation! Sections of the evaluation About the
instructor About the course About the programNow lets hear some of
their reasons, in their own words!
- 5. Student-Designed Evaluations Student Reasons
- 6. Top Complaints of Faculty General Instructors arriving late
or unprepared Instructors that lecture for an entire class period
Instructors taking too long to respond to messages and provide
feedback on work Instructors who do not provide enough examples or
case studies Instructors who are boring Instructors with poor
speaking skills/Cannot understand instructor When an instructor
thinks that he or she is better and above the students
- 7. Top Complaints of Faculty Online Courses Instructions that
are not clear Waiting too long for grades/feedback Missing or
disengaged instructors (lack of support) Poor course design
navigation is confusing, hard to find assignments
- 8. Top Praises of Faculty Professionalism dress, punctuality,
behavior A detailed, organized syllabus that clearly conveys
expectations Flexibility when an instructor is willing to make
changes to the schedule and content when appropriate Interesting
and relevant course material Varied methods of instruction When an
instructor shows a passion for teaching Ample interaction with
students Respect and fairness Source: Butler County Community
College Office of Academic Affairs
- 9. Student Evaluations of Higher Education Faculty Concerns
Students are being selective in who they choose to
evaluate.Evaluations are now administered online.Ineffective
Feedback Students are not taking the evaluations
seriously.Instructors do not receive results until after the
semester is over.
- 10. Online Course Evaluation linkActions Taken Discussed
concerns with VP of Academic Affairs Looked at the current
evaluation instrument for alignment with my teaching Added
additional questions to the evaluation instrument: Please rate the
following statement as it applies to you: 1. I feel that my
instructor cares about my success in this course. 2. My instructor
responds to my questions in a timely manner. 3. My instructor takes
time to include lessons that are purposeful and can be applied to
real-life situations. 4. My instructor values me as an
individual.
- 11. Case Studies Fall 2012Spring 2013 4 course sections (81
students) 4 different approaches to encourage participation
Administered ongoing course feedback survey 2 course sections (27
students) Reflective writing activity regarding attitudes toward
instructor evaluations Class discussion about evaluations
- 12. Results Fall 2012 Response Rate 6/25 = 24%Section L01
Announced that the evaluation window has opened9/22 = 41%Section
L02 Announced that the evaluation window opened; sent weekly e-mail
reminders9/17 = 53%Section C51 Announced that the evaluation window
opened; sent students a link to the online evaluation site (through
their personal e-mail)13/17 = 76%Section B01 Class discussion on
instructor evaluations; gave students opportunity to complete
evaluations during class time in the computer lab
- 13. Respose Rates
- 14. Ongoing Feedback Survey Administered 2/3 into semester
Important to share and discuss comments with students 1. My
instructor is available to me in person, by phone, and via e-mail.
2. Participation in this class is valued. 3. I feel comfortable in
this class. 4. Which of the following were the most interest and
helpful methods of instruction for you? (methods listed) 5. Comment
on any suggestions you may have to improve your experience in this
course.
- 15. Spring 2013 Reflective Writing Activity Survey on Faculty
and Course Evaluations How seriously do you approach faculty
evaluations? When the time comes to complete evaluations, do you
complete them for all of your instructors, some of them, or none?
Do you think the instructors read and take your comments into
consideration? Think of any negative experiences you have had with
a course/instructor. What made it a negative experience? How would
you describe your ideal instructor? If you could add one question
to a faculty evaluation, what would it be? If you were the
President of the college, what methods would you use to evaluate
instructors?
- 16. Findings from Survey The majority stated that they complete
evaluations for the instructors that they either had a very
positive experience with or one that they thought was very bad. I
complete them if I had a bad experience with a teacher. If they
dont ask, I dont care. If they do ask, then Ill do it. Students do
not learn or care about classes that the instructor does not care
about.
- 17. Findings from Survey When describing an ideal instructor:
Respectful, on time, easy-to-reach, interesting, varies
instruction, posts grades online One who applies it to real-life
rather than just making me memorize I like professors who make a
lasting impression. Spring 2013: Results will be available after
May 7.
- 18. Strategies to Increase Awareness of Faculty Evaluations
Take evaluation questions into consideration when developing your
course syllabus, objectives, content, and assessments. Provide
opportunities during class time to complete evaluations. Reflective
writing activities In-class discussions Mid-semester surveys Share
findings with students
- 19. Evaluation of Blended College Courses Using a Community of
Inquiry Survey: Opportunity for Improvement Diane Z. Onorato
Indiana University of PA Mercyhurst University, North East,
PASurvey retrieved from http://communitiesofinquiry.com/model
- 20. Babson Survey Research Group and the College Boards 2011
report on the status of online learning in the US, 1. 6.1 million
college students took an online course in Fall 2010 2. 31 percent
of all college students were taking at least one online course in
2011 3. Online education had 10% growth rate; traditional
education, a 2% growth rate 4. 65 percent of all reporting tertiary
level schools include online education as part of their strategic
plan. 5. According to the most recent distance and online learning
report from Converge (2011), in five years, if the percentages of
increase stay the same online students will outnumber traditional
students. 6. 1.8 million online basic education student in 2009-10
(Converge, 2011, p. 5)
- 21. Theoretical framework of a Community of Inquiry (CoI)
collaborativeconstructivist learning experience through the
development of three interdependent elements social, cognitive and
teaching presence.
- 22. Definition of Blended Learning Blended learning is the
organic integration of thoughtfully selected and complementary
face-to-face and online approaches and technologies. Blended
learning is seen as an opportunity to fundamentally redesign how we
approach teaching and learning in ways that higher education
institutions may benefit from increased effectiveness, convenience
and efficiency. (Garrison & Vaughan, 2008)
- 23. Teaching Presence is the design, facilitation, and
direction of cognitive and social processes for the purpose of
realizing personally meaningful and educationally worthwhile
learning outcomes (Anderson, Rourke, Garrison, & Archer, 2001,
p. 5). CategoriesExamples of IndicatorsFacilitating
DiscourseDefining, focusing, motivating with knowledge of students
efficacy; Clarifying ideas, shared experiences; Managing
netiquette; Providing feedback and directionsDirect
InstructionDiscussion; Questioning, Injection of new
knowledgeInstructional Design & OrganizationSetting curriculum,
standards, targets, scaffolding; Calendar; Designing methods,
activities; Use of medium; Offering direction for technical
support
- 24. Teaching Presence: Survey Items Design & Organization
1. Clearly communicated important course goals 2. Clearly
communicated important course topics 3. Provided clear instructions
on how to participate 4. Clearly communicated important due
dates/time frames Facilitation 5. Identifying areas of agreement
and disagreement to help me to learn 6. Guiding class towards
understanding course topics to clarify my thinking 7. Keeping
participants engaged and participating in productive dialogue 8.
Keeping the course participants on task 9. Encouraged course
participants to explore new concepts 10. Reinforced the development
of a sense of community Direct Instruction 11. Focus discussion on
relevant issues in a way that helped me to learn 12. Feedback that
helped me understand my strengths and weaknesses Retrieved from
http://communitiesofinquiry.com/methodology 13. Feedback in a
timely fashion
- 25. Social presence is the ability of participants to identify
with the community (e.g., course of study), communicate
purposefully in a trusting environment, and develop inter-personal
relationships(Garrison, 2007; van Shie, 2008, retrieved from by way
of projecting their http://communitiesofinqu
iry.com/sites/community individual personalities
ofinquiry.com/files/conce pt-map.gif) (Garrison, 2009, p. 352).
CategoriesExamples of Indicatorssense of self and audienceEmotional
Expression Using humor, self-disclosure, emoticons; Expressions of
feelings; a sense of safety to speak without unfair treatment Open
CommunicationContinuing threads; Quoting and referring to each
others statements; Complimenting and agreeing; Risk-free
ExpressionGroup CohesionEncouraging and including others; Using
names and greetings
- 26. Social Presence: Survey ItemsRetrieved from
http://communitiesofinquiry.com /methodologyAffective expression
14. Getting to know other[s] gave me a sense of belonging 15. Able
to form distinct impressions of [others] 16. Online . . . is an
excellent medium for social interaction Open communication 17.
Comfortable conversing 18. Comfortable participating in . . .
discussions 19. Comfortable interacting with other[s] Group
cohesion 20. Comfortable disagreeing . . . while still maintaining
sense of trust 21. My point of view was acknowledged by other[s]
22. Develop a sense of collaboration
- 27. Cognitive Presence is the extent to which learners are able
to construct and confirm meaning through sustained reflection and
discourse (Garrison, Anderson, & Archer, 2001, p. 11).
CategoryExamples of Indicatorreflection and interactionTriggering
EventSense of puzzlement about a topic; Recognizing the problem but
not yet beginning an activity on the topicExplorationInformation
exchange through discussion; Recognition of divergent views;
Offering of suggestions; BrainstormingIntegrationConnecting ideas;
Finding points of convergence; Offering tentative conclusions or
solutionsResolutionTest and defend solutions; Apply new ideas
- 28. Cognitive Presence: Survey Items Triggering event 23.
Problems posed increased my interest 24. Activities piqued my
curiosity 25. Felt motivated to explore content related questions.
Exploration 26. Utilized a variety of information sources 27.
Brainstorming and finding relevant information helped me resolve
content-related questions 28. Discussions help[ed] me appreciate
different perspectives.
- 29. Cognitive Presence: Survey Items (continued) Integration
29. Combining new information helped me answer questions 30.
Activities helped [with]. . . explanations/ solutions 31.
Reflection . . . helped me understand fundamental concepts
Resolution 32. Can describe ways to test and apply [course ideas]
33. Have developed [practical] solutions 34. Can apply the
knowledge created in this course to my work or . . . other
activities
- 30. Results from CoI Survey IndicatorClass Class Class Class
Class Class Class Class Class #1 #2 #3 #4 #5 #6 #7 #8 #9TP design
& organization8%0%0%4%10%11%10%10%15%TP
facilitation5%25%4%6%17%12%8%13%17%TP direct
instruction13%17%7%13%13%13%9%13%20%SP affective
expression3%17%4%8%3%11%7%10%27%SP open
communication0%0%4%8%10%11%9%13%20%SP group
cohesion0%0%7%9%13%13%11%10%27%CP triggering
event17%0%7%12%10%11%11%12%13%CP
exploration17%0%4%13%20%14%9%13%20%CP
integration0%0%0%11%10%9%7%8%20%CP
resolution10%0%7%10%13%15%11%13%33%
- 31. Findings & Implications Scaffolding Time for reflection
Clear expectations, models Active and collaborative learning Manage
the learning environment; summarize the conversation/new direction
Provide feedback beyond grades Students self-efficacy
- 32. Findings & Implications Terry Ann Morris (2011) 65%
completion for online CC and 72% for traditional. Survey says, 1.
Enjoyed discussion activities (multiple days) 2. Instructor
feedback was the high point (24 hrs.) 3. Dont like chat rooms,
social networking with no instructional purpose 4. Self-efficacy is
determining factor in successful completion 5. Varied favorite
activities underscore the diversity of the population
- 33. To enhance social & cognitive presence, use a Wiki
using a scaffold to create a pro/con debate
- 34. Cognitive PresenceArbaugh (2007) most difficult Elder and
Pauls Model of Critical Thinking (2010) Carefully design word tasks
to promote critical thinking. They developed a model of questions
based on standards that improve learners thinking skills and help
them question ideas before they decide if they should accept them.
Questions include clarification, accuracy, precision, relevance,
logic, significance, fairness.
- 35. To keep learners engaged, Survey (Bonk)
- 36. Teaching presence Welcome email Scavenger hunt Wiki
introductions Once upon a time Guest lectures Templates Scaffolding
Multiple discussion forums Pride Wall
- 37. Importance of Teaching Presence The body of evidence is
growing rapidly attesting to the importance of teaching presence
for successful online learning The consensus is that teaching
presence is a significant determinate of student satisfaction,
perceived learning, and sense of community. Dr. Marti
Cleveland-Innes, Athabasca University Dr. Randy Garrison,
University of Calgary (2007) 13th Annual Sloan-C Conference
- 38. Students must feel a sense of participation Students know
and can articulate what good teaching, learning, and collaboration
is at the high school level. If there is no student evaluation
procedure, a high school class can collaborate to produce a
suitable instrument. Students must have buy-in or an accessible,
secure means of participation in course evaluation in order for
significant numbers to provide usable feedback.
- 39. The evaluation must measure class procedures to allow for
improvement The evaluation instrument should measure what happens
in class and be a standard across similarly constructed classes so
all faculty can be evaluated fairly. Results of a course evaluation
by a student should provide data for teachers to improve their
courses with respect to increased perceived learning and student
satisfaction.
- 40. NEERO 45th Annual Confernece Symposium 2.2 | Thursday,
10:15-11:45am Symposium Title: Student Evaluations of Higher and
Basic Education Instructors: Three Case Studies Thank you for your
attention!Tom NeuschaferNichol MurrrayDiane Onorato