Conversion of Faculty Evaluations to an On- Line Format Catherine Hackett Renner SUNY Geneseo Larry...
-
date post
15-Jan-2016 -
Category
Documents
-
view
214 -
download
0
Transcript of Conversion of Faculty Evaluations to an On- Line Format Catherine Hackett Renner SUNY Geneseo Larry...
![Page 1: Conversion of Faculty Evaluations to an On- Line Format Catherine Hackett Renner SUNY Geneseo Larry Piegza Gap Technologies, Inc. OnlineCourseEvaluations.com.](https://reader035.fdocuments.us/reader035/viewer/2022062518/56649d495503460f94a262de/html5/thumbnails/1.jpg)
Conversion of Faculty Evaluations to an On-Line FormatCatherine Hackett Renner SUNY Geneseo
Larry PiegzaGap Technologies, Inc.OnlineCourseEvaluations.com
![Page 2: Conversion of Faculty Evaluations to an On- Line Format Catherine Hackett Renner SUNY Geneseo Larry Piegza Gap Technologies, Inc. OnlineCourseEvaluations.com.](https://reader035.fdocuments.us/reader035/viewer/2022062518/56649d495503460f94a262de/html5/thumbnails/2.jpg)
The Paper Process ~22,000-25,000 forms per term
were hand sorted into course sections
Distributed to faculty the last 2 weeks of classes Faculty returned them to IR
Office Forms were separated, scanned,
then hand sorted back into course sections Invariably evaluations would
be turned in late Faculty received the results
back mid-February
![Page 3: Conversion of Faculty Evaluations to an On- Line Format Catherine Hackett Renner SUNY Geneseo Larry Piegza Gap Technologies, Inc. OnlineCourseEvaluations.com.](https://reader035.fdocuments.us/reader035/viewer/2022062518/56649d495503460f94a262de/html5/thumbnails/3.jpg)
Complaints with Process Faculty
Time delay in getting results back Late returns were not processed
Students Anonymity Detail in comments
Classroom time did not allow for thoughtful detail in comments
Convenience Would like to see results online
![Page 4: Conversion of Faculty Evaluations to an On- Line Format Catherine Hackett Renner SUNY Geneseo Larry Piegza Gap Technologies, Inc. OnlineCourseEvaluations.com.](https://reader035.fdocuments.us/reader035/viewer/2022062518/56649d495503460f94a262de/html5/thumbnails/4.jpg)
Conversion to On-line Format
Internal conversion was not feasible Migration to new system Internal survey software was not secure
enough for this process Decision was made to outsource
Constraints on company Had to use our own instrument Had to be able to conform to the levels of
access that were dictated by Faculty Senate
![Page 5: Conversion of Faculty Evaluations to an On- Line Format Catherine Hackett Renner SUNY Geneseo Larry Piegza Gap Technologies, Inc. OnlineCourseEvaluations.com.](https://reader035.fdocuments.us/reader035/viewer/2022062518/56649d495503460f94a262de/html5/thumbnails/5.jpg)
Company Choice
Best met our needs Could take our current evaluation system and upload it Could reproduce our levels of access Very customer-oriented
![Page 6: Conversion of Faculty Evaluations to an On- Line Format Catherine Hackett Renner SUNY Geneseo Larry Piegza Gap Technologies, Inc. OnlineCourseEvaluations.com.](https://reader035.fdocuments.us/reader035/viewer/2022062518/56649d495503460f94a262de/html5/thumbnails/6.jpg)
Spring 2006 Pilot
Students enrolled in Psychology classes and Astronomy Laboratories provided their Student Evaluation of Faculty Instruction (SOFI) ratings in an on-line format rather than in-class paper format 51 course sections 1,199 individual students providing1,806
evaluations These 1,199 students were also able to
compare the on-line version to in-class paper format (used in their other classes)
![Page 7: Conversion of Faculty Evaluations to an On- Line Format Catherine Hackett Renner SUNY Geneseo Larry Piegza Gap Technologies, Inc. OnlineCourseEvaluations.com.](https://reader035.fdocuments.us/reader035/viewer/2022062518/56649d495503460f94a262de/html5/thumbnails/7.jpg)
Getting the Word Out Students were informed of the pilot by their course
instructors at the beginning of the semester One week prior to the SOFI evaluation period
informational flyers were posted within the classrooms of the 51 sections
Article on the process was published in the student newspaper
Student Government Association endorsed the process and was involved with disseminating information
On the first evaluation day faculty were given handouts to distribute to their students explaining the process
The evaluation period started on April 17th and ended on May 3rd
![Page 8: Conversion of Faculty Evaluations to an On- Line Format Catherine Hackett Renner SUNY Geneseo Larry Piegza Gap Technologies, Inc. OnlineCourseEvaluations.com.](https://reader035.fdocuments.us/reader035/viewer/2022062518/56649d495503460f94a262de/html5/thumbnails/8.jpg)
Pilot Procedure
Students received an e-mail with their Login ID and password E-mail contained direct link the login webpage
E-mail reminders 48 hours Reminders stopped once all evaluations were
performed
Faculty received e-mails requesting they remind the students of the evaluation period Within the e-mail faculty also received the
response rate for each of their courses
![Page 9: Conversion of Faculty Evaluations to an On- Line Format Catherine Hackett Renner SUNY Geneseo Larry Piegza Gap Technologies, Inc. OnlineCourseEvaluations.com.](https://reader035.fdocuments.us/reader035/viewer/2022062518/56649d495503460f94a262de/html5/thumbnails/9.jpg)
Pilot Results
The response rate for the pilot was 76%! This was consistent
with the response rate for the in-class SOFI (70-73%)
Some bumps in the road All occurred within the
first 2 days All were minor All were resolved in
less than 24 hours
![Page 10: Conversion of Faculty Evaluations to an On- Line Format Catherine Hackett Renner SUNY Geneseo Larry Piegza Gap Technologies, Inc. OnlineCourseEvaluations.com.](https://reader035.fdocuments.us/reader035/viewer/2022062518/56649d495503460f94a262de/html5/thumbnails/10.jpg)
Student Satisfaction Survey
Seven survey questions designed to assess the following: Anonymity Detail of Comments Convenience Choice
This final question was designed to be the “litmus” test of the pilot
This question asks students which format they would prefer to use if given the choice
![Page 11: Conversion of Faculty Evaluations to an On- Line Format Catherine Hackett Renner SUNY Geneseo Larry Piegza Gap Technologies, Inc. OnlineCourseEvaluations.com.](https://reader035.fdocuments.us/reader035/viewer/2022062518/56649d495503460f94a262de/html5/thumbnails/11.jpg)
Anonymity
In general, I believe in-class SOFIs are: Percent
In general, I believe on-line SOFIs are: Percent
not anonymous 3.7 not anonymous 6.2
slightly anonymous 13.6 slightly anonymous 7.4
somewhat anonymous 57.4 somewhat anonymous 34.6
completely anonymous 25.3 completely anonymous 51.9
![Page 12: Conversion of Faculty Evaluations to an On- Line Format Catherine Hackett Renner SUNY Geneseo Larry Piegza Gap Technologies, Inc. OnlineCourseEvaluations.com.](https://reader035.fdocuments.us/reader035/viewer/2022062518/56649d495503460f94a262de/html5/thumbnails/12.jpg)
Detail of Comments
Was the amount of detail you provided in your comments different in the on-line version
compared to the in-class version? Percent
Yes, I was able to provide more detail than in class 27.8
Yes, I provided less detail than in class 13.6
No, it was about the same 57.4
![Page 13: Conversion of Faculty Evaluations to an On- Line Format Catherine Hackett Renner SUNY Geneseo Larry Piegza Gap Technologies, Inc. OnlineCourseEvaluations.com.](https://reader035.fdocuments.us/reader035/viewer/2022062518/56649d495503460f94a262de/html5/thumbnails/13.jpg)
Convenience
How convenient was it to use the on-line method compared to the in-class method? Percent
Not convenient at all 4.9
somewhat inconvenient 14.8
somewhat convenient 32.1
very convenient 47.5
![Page 14: Conversion of Faculty Evaluations to an On- Line Format Catherine Hackett Renner SUNY Geneseo Larry Piegza Gap Technologies, Inc. OnlineCourseEvaluations.com.](https://reader035.fdocuments.us/reader035/viewer/2022062518/56649d495503460f94a262de/html5/thumbnails/14.jpg)
Results
How often have you referred to the SOFI results in any given term? Percent
never 70.4
occasionally 22.2
often 3.7
very often 3.1
![Page 15: Conversion of Faculty Evaluations to an On- Line Format Catherine Hackett Renner SUNY Geneseo Larry Piegza Gap Technologies, Inc. OnlineCourseEvaluations.com.](https://reader035.fdocuments.us/reader035/viewer/2022062518/56649d495503460f94a262de/html5/thumbnails/15.jpg)
Results
SOFI results will be available for students within 3 weeks of the last day of completion and will be accessible on-line. Given this, how likely will you refer to the on-line SOFI results?
Percent
never 13.6
occasionally 38.3
often 28.4
very often 19.1
![Page 16: Conversion of Faculty Evaluations to an On- Line Format Catherine Hackett Renner SUNY Geneseo Larry Piegza Gap Technologies, Inc. OnlineCourseEvaluations.com.](https://reader035.fdocuments.us/reader035/viewer/2022062518/56649d495503460f94a262de/html5/thumbnails/16.jpg)
Choice
If you could choose which SOFI format
would you prefer to use? Percent
on-line 53.7
in-class paper 17.3
doesn't matter 27.2
![Page 17: Conversion of Faculty Evaluations to an On- Line Format Catherine Hackett Renner SUNY Geneseo Larry Piegza Gap Technologies, Inc. OnlineCourseEvaluations.com.](https://reader035.fdocuments.us/reader035/viewer/2022062518/56649d495503460f94a262de/html5/thumbnails/17.jpg)
Decision
We went to a full implementation Fall 2006
80% response rate Smooth run
![Page 18: Conversion of Faculty Evaluations to an On- Line Format Catherine Hackett Renner SUNY Geneseo Larry Piegza Gap Technologies, Inc. OnlineCourseEvaluations.com.](https://reader035.fdocuments.us/reader035/viewer/2022062518/56649d495503460f94a262de/html5/thumbnails/18.jpg)
What SUNY Geneseo Did Well
Create a login link from their web portal Erased student confidentiality concerns
by selecting an off-site vendor Communicated with their teachers Shared results to their students
![Page 19: Conversion of Faculty Evaluations to an On- Line Format Catherine Hackett Renner SUNY Geneseo Larry Piegza Gap Technologies, Inc. OnlineCourseEvaluations.com.](https://reader035.fdocuments.us/reader035/viewer/2022062518/56649d495503460f94a262de/html5/thumbnails/19.jpg)
What SUNY Geneseo Did Well
Sent out regular emails to their students Have teachers hand out directions Ran ad in school newspaper Mobilized their student government
![Page 20: Conversion of Faculty Evaluations to an On- Line Format Catherine Hackett Renner SUNY Geneseo Larry Piegza Gap Technologies, Inc. OnlineCourseEvaluations.com.](https://reader035.fdocuments.us/reader035/viewer/2022062518/56649d495503460f94a262de/html5/thumbnails/20.jpg)
Top 10 Things Not To Do When Going Online:
10) Have a one week evaluation window for your first semester
9) Decide on a vendor two weeks before your evaluation starts
8) Spam your students
7) Change your course of action based on the teacher suggesting things at the current time
![Page 21: Conversion of Faculty Evaluations to an On- Line Format Catherine Hackett Renner SUNY Geneseo Larry Piegza Gap Technologies, Inc. OnlineCourseEvaluations.com.](https://reader035.fdocuments.us/reader035/viewer/2022062518/56649d495503460f94a262de/html5/thumbnails/21.jpg)
Top 10 Things Not To Do When Going Online:
6) Run concurrent with paper
Actual school's response rates
2004
20062005
0%
10%
20%
30%
40%
50%
60%
70%
![Page 22: Conversion of Faculty Evaluations to an On- Line Format Catherine Hackett Renner SUNY Geneseo Larry Piegza Gap Technologies, Inc. OnlineCourseEvaluations.com.](https://reader035.fdocuments.us/reader035/viewer/2022062518/56649d495503460f94a262de/html5/thumbnails/22.jpg)
Top 10 Things Not To Do When Going Online:
5) Buy a system that doesn't allow your students to click a "I dropped this class" button.
4) Don't communicate to your teachers
3) Ignore your vendor’s advice / Go it alone
2) Ask 100 questions on each survey
![Page 23: Conversion of Faculty Evaluations to an On- Line Format Catherine Hackett Renner SUNY Geneseo Larry Piegza Gap Technologies, Inc. OnlineCourseEvaluations.com.](https://reader035.fdocuments.us/reader035/viewer/2022062518/56649d495503460f94a262de/html5/thumbnails/23.jpg)
Top 10 Things Not To Do When Going Online:
1) Send a two page email lecturing your students on their civic responsibility to fill out course evaluations
![Page 24: Conversion of Faculty Evaluations to an On- Line Format Catherine Hackett Renner SUNY Geneseo Larry Piegza Gap Technologies, Inc. OnlineCourseEvaluations.com.](https://reader035.fdocuments.us/reader035/viewer/2022062518/56649d495503460f94a262de/html5/thumbnails/24.jpg)
8 Must Haves
1) Patience to change your school’s culture
Response Rates For Edgewood College
0%
10%
20%
30%
40%
50%
60%
70%
2004 2005 2006
Year
Fall
Spring
Summer
![Page 25: Conversion of Faculty Evaluations to an On- Line Format Catherine Hackett Renner SUNY Geneseo Larry Piegza Gap Technologies, Inc. OnlineCourseEvaluations.com.](https://reader035.fdocuments.us/reader035/viewer/2022062518/56649d495503460f94a262de/html5/thumbnails/25.jpg)
8 Must Haves
2) Eliminate common questions for team taught courses
3) Prizes to give away to students
4) Share results to students
5) Allow teachers to email to their students
6) Have a contest
![Page 26: Conversion of Faculty Evaluations to an On- Line Format Catherine Hackett Renner SUNY Geneseo Larry Piegza Gap Technologies, Inc. OnlineCourseEvaluations.com.](https://reader035.fdocuments.us/reader035/viewer/2022062518/56649d495503460f94a262de/html5/thumbnails/26.jpg)
8 Must Haves
7) Follow-up question technology
Did the instructor speak audibly and clearly?
What could the instructor have done to speak more clearly?
Get a microphone Not face the blackboard
Speak slower Speak faster
![Page 27: Conversion of Faculty Evaluations to an On- Line Format Catherine Hackett Renner SUNY Geneseo Larry Piegza Gap Technologies, Inc. OnlineCourseEvaluations.com.](https://reader035.fdocuments.us/reader035/viewer/2022062518/56649d495503460f94a262de/html5/thumbnails/27.jpg)
8 Must Haves
8) Dropped class survey
What was the primary reason why you dropped this class?
Course related Instructor related Financial issue
Scheduling issue Something about me Other