Grading:
Assessment Techniques to Enhance Learning KURT F. GEISINGER, PH.D.
AUGUST, 2015, KING FAHD UNIVERSITY
My History in Grading My own history studying grading…as a psychometrician
◦ Published several studies on grade inflation in late 1970s and early 1980s
◦ Developed scales that measure faculty members orientations to grading
◦ Wrote monograph that became “Marking Systems” entry in the Encyclopedia of Educational Research
My work as a dean and vice president for academic affairs/ provost where I worry far more about student learning
The Essence of Grading Grades matter!
◦ To Students
◦ To Programs
◦ To the institutions
Grades are specific to given courses
Students take grades very seriously
Faculty do too, although grading is an onerous task
Grading is deeply embedded in higher education
Grading or Marking Systems There are many systems in place:
◦ Letter grades are most common (A-F is most typical)
◦ Numerical grades also exist in some setting (1-100) ◦ Sometimes schools align numerical grades with letter grades
◦ PASS-FAIL grading also exists ◦ At some schools, all courses are pass-fail
◦ Most schools offer limited numbers of pass-fail classes
◦ A few schools only offer descriptive grading; at one school I know, every student in every class gets a brief letter written to him/her telling him/her how they have done in the class, what skills they have shown, and so forth. It is a small liberal arts school with very small classes.
Grading at King Fahd University
Letter Grades Points Descriptive Grades
A+ 4.00 Exceptional
A 3.75 Excellent
B+ 3.50 Superior
B 3.00 Very Good
C+ 2.50 Above Average
C 2.00 Good
D+ 1.50 High-Pass
D 1.00 Pass
F 0.00 Fail
IP - In Progress
IC - Incomplete
DN 0.00 Denial
NP - No grade-Pass
NF - No grade-Fail
W - Withdrawn
WP - Withdrawn with Pass
WF 0.00 Withdrawn with Fail
AU - Audit
Purposes of Grading To provide information to students about their performance
Administrative institutional purposes
Sorting and selecting students
Motivating students
Research tools
Preparation for life
General Principles
Context ◦ Consider what you plan to do with data collected, scores or grades?
◦ Classroom Assessment
◦ Formative Assessment
◦ Summative Assessment
◦ Qualitative vs Quantitative
◦ Curricular representation ◦ Achievement & Ability
◦ Preassigned weights or proportional values
Assessments ◦ Plan ahead & be clear and specific
◦ Create Test Blueprints
◦ Tasks / item responses will generate relevant evidence?
◦ Make items that are construct relevant ◦ Create scoring rubrics
Assigning Grades ◦ Qualitative Approach ◦ Quantitative Approaches ◦ Defensibility, Validity & Fairness
Context
◦ Consider Content ◦ Unit
◦ Curriculum / Course Outline
◦ Consider what you plan to do with data collected, scores or grades?
◦ Classroom Assessment
◦ Formative Assessment
◦ Summative Assessment
◦ Qualitative vs Quantitative
◦ Curricular representation ◦ Achievement & Ability
◦ Preassigned weights or proportional values
What is the main purpose of assessment in relation to curricula?
• End of program or exit criteria
• End of semester
• End of unit
• In progress
Clarify: ◦ Who will write, administer, and score each assessment?
◦ What decisions will be made using the results (high/low stakes)?
◦ When will the assessment be administered? (timing & frequency)
◦ Where will the test be administered? (environment and resources)
◦ Why focus on various levels of learning?
◦ How will the scores be reported to the students?
What Kinds of Things Should be Included in Grades?
Affective values? ◦ Students’ interest in the course
Classroom behavior? ◦ Speaking up in class
◦ Participating actively in class, seeming interested
Attendance?
Student growth—how much an individual student learns in a class relative to what they knew at the beginning of the course
Student learning!
Grade Inflation One big concern that began in the 1970s
Average grades at institutions increased on average from about 2.20-2.30 (on a four-point scale) to 2.70-2.80
Popular press and some higher education theorists argued that this probably happened due to the US military draft and the Vietnam War
◦ Limited evidence to support this hypothesis
Changes in grading policies have much to do with this (non-punitive grading standards)
◦ Pass-fail grading
◦ The ability to drop courses late in a semester
Changes in the approach/attitudes of faculty members
For Grading to Be Meaningful Faculty must identify the most meaningful aspects of learning in a class
Faculty must assign weights to different components of grading that represent the learning in a class
◦ E.G., if tests and projects are both included, then they should be weighted in a manner consistent with their importance in terms of student learning
Faculty must construct tests and assignments that assess learning in accordance with the more meaningful aspects of learning
Faculty should set reasonable standards
Faculty should use assignments to guide student learning
Grading as a Motivator Without question, grades affect student motivation
◦ “What’s going to be on the test?”
◦ “Will this be on the test?”
In psychology, we talk about Intrinsic and Extrinsic Motivation
It’s sometimes hard as a faculty member to try to motivate students in terms of their intrinsic motivation
It is also easy to fall into the pattern of telling students or implying to students what is likely to be on the test
Assessment as Part of Grading In this age of accountability, accrediting bodies insist on outcomes assessment. Requirements include:
◦ Assessing student learning with both direct and indirect measures. ◦ An indirect measure might be an evaluation of a graduate’s work performance.
◦ Connecting assessments to the mission, goals, and objectives of the program.
◦ Attending to reliability and validity of measures.
◦ Demonstrating widespread faculty involvement
◦ Using assessment information for improvements.
◦ Integrating assessment and planning and budgeting.
Intrinsic Motivation Intrinsic motivation refers to motivation that is driven by an interest or enjoyment in the task itself, and exists within the individual rather than relying on external pressures or a desire for reward.
Intrinsic motivation is a natural motivational tendency and is a critical element in cognitive, social, and physical development. Students who are intrinsically motivated are more likely to engage in the task willingly as well as work to improve their skills, which will increase their capabilities.
Students are likely to be intrinsically motivated if they: ◦ attribute their educational results to factors under their own control, also known as
autonomy
◦ believe they have the skills to be effective agents in reaching their desired goals, also known as self-efficacy beliefs
◦ are interested in mastering a topic, not just in achieving good grades
Extrinsic Motivation Extrinsic motivation refers to the performance of an activity in order to attain an outcome, whether or not that activity is also intrinsically motivated.
Extrinsic motivation comes from outside of the individual. ◦ Rewards (for example, money or grades) for showing the desired behavior
◦ Punishment following misbehavior.
◦ Competition, because it encourages the performer to win and to beat others.
◦ Praise and recognition, a cheering crowd, or the desire to win a trophy are also extrinsic incentives.
Are grades important to everyone?
Classroom Assessment Techniques
Another presentation, but a big part of this one too
Keeping faculty knowledgeable about student learning through frequent, short, ungraded assessments in class
◦ One minute quiz where students describe the one thing that they did not understand well from that day’s class
In this presentation, I am more focused upon “graded” assignments, assignments where students perform and faculty members assign marks to their work.
Three False Hopes to be Dismissed Grading can be totally objective
Faculty can reach 100% consensus on grading
Grading is the ideal way to motivate students
Twelve Principles of Good Grading: 1/12
1. Appreciate the complexity of grading; Use it to help students learn
◦ Survey by Milton, Pollio and Eison demonstrated that among 6,000 students and faculty, there were many different perceptions of grading and of the roles of grading
◦ Students perceptions of grading differed according to their previous experience, motivation and so on
◦ Evaluation is part of a system that is based on the beliefs of those who participate in the system
◦ We need a system that meets our needs
◦ There is no perfect evaluation system
◦ The four primary roles of grades: evaluation, communication, motivation, and organization
Twelve Principles of Good Grading: 2/12
2. Substitute judgment for objectivity
There is no perfect standard
There is no way to assess all students with perfect reliability using any standard
◦ Reasons for questionable objectivity ◦ Even with objective question, one cannot ask them all
Job of faculty member: to render a judgment in as informed and professional a manner as possible
In a world where faculty do not wish to stress memory to the exclusion of critical thinking and problem solving, judgment is important to higher order thinking and learning
Twelve Principles of Good Grading: 3/12
3. Use your grading time effectively
Make the best judgment you can and then move on
You have other work to do.
Twelve Principles of Good Grading: 4/12
4. Be open to change
Grade inflation happened in the United States.
At one time a “C” was an average grade and GPA averages throughout the United States were about 2.20
Today at most schools they are about 2.70
At some schools, including Stanford University, they are about 3.50
Imagine moving to another university that has a vastly different approach to grading
Twelve Principles of Good Grading: 5/12
5. Listen and Observe ◦ Dressel’s advice to young faculty members in Handbook of Academic Evaluation
It is the meaning that students attach to grades that most affects student learning
Observe students and listen to them
Grades are not part of the laws of the Universe
Rather, they carry specific meanings to specific individuals
If we are trying to use grades to improve student learning, we should listen to students and hear how they interpret various grades
Twelve Principles of Good Grading: 6/12
6. Communicate and collaborate with students
Grading can be antagonistic. No one wants that.
The fair and helpful evaluation of students is what students both seek and need.
Student-faculty relationships and student-peer relationships are critically important for students.
Try to find ways to collaborate with students on goals.
Seek to get students actively involved in the learning process.
Ray Bucko’s approach to having students evaluate the work of each other as a learning activity.
Students should know on the syllabus exactly how they will be evaluated, with rubrics and percentages, if possible
Twelve Principles of Good Grading: 7/12
7. Integrate Grading with Other Key Processes
Grading should be integrated with teaching
Go over tests so that students understand what they did not learn effectively initially
Have testing and other evaluative components of your class integrated with planning, teaching, and other class activities
Twelve Principles of Good Grading: 8/12
8. S e i z e t h e t e a c h a b l e m o m e n t
Informal feedback can be tremendously important to students
Remember how influential some of your own faculty were to you!
Think about the values what you wish to communicate to students and try to embody them
When students are emotional about the grades they have received or your evaluation of them, try to stay calm and determine what you wish to communicate to them
What about plagiarism?
Twelve Principles of Good Grading: 9/12
9. Make Student Learning the Primary Goal
Grades are sometimes important for external reasons: potential employers wish to see the grades, scholarship and graduate admissions committees want to see them, and so on.
Nevertheless, directing student learning should be the primary use of grades and grading
Grading can influence learning if learning is the primary goal and grading is used to enhance it
If grades are the goal rather than learning, there is less of a chance that learning will be enhanced, and intrinsic motivation certainly will not be
Twelve Principles of Good Grading: 10/12
10. Be a teacher first and a grader second
Grades serve as part of the gatekeeper function in higher education
Sorting should occur only after every student has had an equal opportunity to learn material
We do serve as gatekeepers, but only at the end of the educational process; we are teachers until that time
Throughout a semester, we should remain focused upon helping students to learn, determining what they need to learn, and advising them
Twelve Principles of Good Grading: 11/12
11. Encourage learner-centered motivation
If one holds grades over students’ heads, that discourages student-centered learning and motivation
Students can become more self-motivated if it is due to their engagement with the instructor in the learning process
We should encourage students that hard work does have positive affects and that they can largely control their own futures
Students’ attitudes toward grades can negatively affect their motivation to learn
Twelve Principles of Good Grading: 12/12
12. Emphasize Student Involvement
Whatever we do, we should attempt to seek students to invest their time and energy in the academic process
Astin’s Conditions for Excellence in Grading
Astin, summarizing 100s of studies of undergraduate grading and learning cited 3 conditions of excellence in grading. The three are:
◦ Student involvement in learning (most important)—relating to the amount of time and energy students’ invest
◦ High expectations by the faculty member
◦ Assessment and feedback—students need to improve and the only way that they can do so is if they get feedback—think about the comments that you make on student papers and examinations
Six Suggestions for Making Assignments Part of the Grading Process
1. Begin by considering what you want students to learn
2. Select tests and assignments that both teach and test the learning you value most.
3. Construct a course outline that shows the nature and sequence of major tests and assignments.
4. Check that the tests and assignments fit your learning goals and are feasible in terms of workload.
5. Collaborate with your students to set and achieve goals.
6. Give students explicit directions for their assignments.
Considering What One Wants Students to Learn
Faculty need to be able to say, “At the end of this course, I want students to be able to …”
Verbs to use include “define, argue, solve, create”
Reduction in the need to recall/remember facts (in many cases)
A good exercise is to sit and do this for courses that you are presently teaching and list out these student learning outcomes
Selecting Assignments and Tests that Measure What the Instructor Values Most
This relates to using your time most effectively
Faculty report that they are most interested in teaching students to analyze, synthesize and think critically.
Reviews of assignments, however, indicate that they stress students to acquire knowledge, comprehend basic concepts and ideas and terms, and to apply such knowledge
Evaluate your own examinations to check what you are measuring
Even essay tests can simply assess students abilities to expound upon learned facts, theories and so on
Choose assignments that are interesting and provide students with the opportunity to learn new skills
Permit peer collaboration when it is appropriate
Constructing a Course Outline Start with a very basic skeleton of the course to be taught
Ask yourself if the workload is manageable
Identify assignments that cause students to use their higher-order thinking skills
Ask “What should students learn?” rather than “What content should I cover?”
Draft assignments to maximize the learning you desire your students to achieve
Check Tests and Assignments for Fit and Feasibility
Ask yourself two questions: ◦ Do my tests and assignments fit the kind of learning I want for students? (Fit)
◦ Is the workload I am planning for the students (and myself) reasonable and sustainable? (Feasibility)
For example, are the tests that you are building the kinds on which students can succeed if they stay up all night the night before the examination?
Can you maintain the workload for grading if your class goes from 15 students to 45?
Collaborating with Student to Set and Achieve Goals
Some faculty meet with students before finalizing their goals
They discuss with students what the students want from the class in terms of learning and skills
Even a discussion of these goals could be quite useful
Try to reach agreement
Have students write their own personal goals in the class
Sometimes this is difficult if courses are in a sequence
Making Assignments and Tests Clear to Students
Give the students ample opportunity to ask questions in advance of assignments and tests
Be as explicit as possible
If you are asking students to run an experiment, for example, ◦ Tell them exactly what they are asked to do, that is, how they should conduct
the experiment
◦ What analyses of the results should be
◦ What the resultant report should look like
Example of an Assignment to Design and Report on an Experiment
Your written report should demonstrate that you have formulated a hypothesis, designed a solid experiment, controlled variables, operationally defined terms, and interpreted data appropriately. You should also demonstrate that you understand the score and sequence of the scientific report format and the importance of quantification to scientific writing.
Workshop Activity: Assignments
Rubrics, Etc.
What obligations do we have at the undergraduate level for teaching communications skills?
Rubrics, Etc.
What obligations do we have at the undergraduate level for teaching communications skills? Plenty…
Rubrics, Etc.
Modern View: • Get students ready for grad school • Write APA style papers Reality Check: How many students will write APA papers after graduation?
Rubrics, Etc.
Traditional Writing Communication Modes: • Experimental Paper • Theoretical Review • Posters • Reaction Papers • Journal Entries
Rubrics, Etc.
Professional Writing Communication Modes: • Case Notes • Grant Proposals • Business Memo/Letters • Project Reports • Web Sites
Rubrics, Etc.
Speaking Communication Modes: • Formal Presentation • Brief Presentation • Debate • Panel Presentation • Paired Presentation • Interviewing
Rubrics, Etc.
SANITY-SAVING STRATEGY: The Common Rubric
Get department to converge on good writing quality
Simplify assessment tracking to 3 categories ◦ Strength
◦ Acceptable
◦ Needs development
CRITERIA
Clarity of purpose
Situational context
Argument organization
Conclusion quality
Language management
Quality of evidence
Disciplinary style
Delivery/format
AREA CRITERIA 2 1 0 PointContribution/Notes:
Fostering Motivation A first principle is student involvement
The more active student learning is, the better
Research by Milton, Pollio, and Eison categorized students as grade-oriented or learning-oriented
Some students also see themselves as helpless, and that their efforts will not pay off
Other students do feel that they have control of their lives, over what happens to them, and that they can earn success
Grade-Oriented Students I think that without regularly scheduled examinations I would not learn and remember much
I do not find studying at home to be interesting
I will withdraw from an interesting class rather than risk a low grade
I get irritated by students who ask questions that go beyond what we need to know on examinations
How can we best motivate such students?
Should we attempt to help them to become more learning-oriented?
Learning-Oriented Students I find the process of learning new material fun
I enjoy classes in which the instructor attempts to relate material to concerns beyond the classroom
I often discuss interesting material that I have learned in class with friends and family
I try to make time for outside reading despite the demanding coursework
Faculty are Mostly Learning-Oriented
To get to where we are, we tend to enjoy learning
Now, we read things in our field for curiosity sake
We often read or get into conversations on intellectual topics outside of our areas of expertise
In US, faculty are very likely to read newspapers…students not
Two best ways to motivate students emerge from academic faculty-student contact and student-peer contact
It helps to tell students that you know that they can succeed
If you can get students to read material before class, then you can get them to discuss the material in class, to argue it, to debate it, and the like
At Some Level, Does it Matter?
Establishing Criteria and Standards
If one has clear criteria:
It saves grading time
The process becomes fairer and more consistent
Helps students know what to expect
Help students get feedback about their learning and evaluate their own learning
Help professors teaching sequenced courses communicate with each other
Help peers provide feedback to each other
Can explain grading more efficiently and accurately
Example: scoring master’s comprehensive essay questions
Rubrics and Clear Criteria Having a rubric moves one from unstated criteria to highly explicit criteria
Moves from Norm-referenced grading to more of a Criterion-referenced perspective
Identify exactly what you wish to measure in any given test or assignment
Work Activity: Developing a Rubric
Example Rubric for a Science Report
5 ◦ Is appropriate in tone and structure
for a science journal
◦ Contains all necessary names
◦ Identifies the method of experimentation
◦ Allows reader to understand the design
4 ◦ Is appropriate in tone and structure
for a science journal
◦ Contains most descriptors
◦ May lack some names
◦ Identifies the function of the experimentation
◦ Suggests the design
Example Rubric for a Science Report
3 ◦ Identifies function and names but
does not allow the reader to anticipate the design
2 ◦ Identifies function or name, but
not both
◦ Lacks design information or is misleading
1 ◦ Is patterned after a non-scientific
discipline or missing
Notice that all of the descriptors begin with a verb, mostly active verbs
One can also override the scale score if one disagrees with it
What about Multiple-Choice Tests (Or Other Objectively Scored Tests)
Several methods
One method begins by categorizing items as to whether they assess higher-order thinking or not
Then one identifies the percentage of questions in the higher-order and lower-order questions one would expect students at various grade levels to get correct
Can also use certain students about whom one has an idea of their likely grade and see how they do
Or both of the above and then adjust
Why Develop a Rubric? Makes grading more consistent and fair
Saves time in the actual grading process
Helps to diagnosis student strengths and weaknesses
Permits teacher to suggest readings for students to deal with their weaknesses
Can track changes in students individually and classes over time
Makes it easier for graduate assistants to do the grading
Helps to make grading more consistent, professor-to-professor
Can help later with programmatic outcomes assessments
Grading Using a Rubric Can also assign grades using a rubric in a manner similar to the previous slide
List out the characteristics of an A+ student, an A, a B, and so on…
What Kind of Grader Are You? Norm-referenced grader
Criterion-referenced grader
Self-referenced grader
Each of these is considered a frame of reference
They have implications for the grades you assign and how high they are
There was probably a change in philosophy happening in the 1970s and 1980s
Are You Norm-Referenced? Grading “on the cure” is the best way to evaluate student performance.
It is an obligation of the university to grade students in a comparative manner to aid prospective employers in locating the best candidate for jobs.
A final examination on which almost all students score quite high is probably too easy, no matter how well the students know the material.
Are You Criterion-Referenced? A student should be able to know in advance how well he must do to receive a high grade.
The extent to which a student has met the requirements of a course should determine his grade.
Before a course begins, a professor should have already determined the criteria that he plans to use to grade students.
Inspecting grades awarded in a particular course should indicate which students have an acceptable mastery of the subject matter and which students do not.
Are You Self-Referenced? A professor should take a student’s effort and ability into account when grading.
A student should be graded in relation to his potential to succeed in a class.
A student who is utilizing all of his ability should receive a high grade, regardless of his overall ability.
What kinds of assessments and assignments do you utilize?
Essay tests?
Multiple-choice tests?
Other objective tests? (e.g., fill-in the blanks, true-false)
Term papers?
Projects?
Do you test primarily for memory or problem solving?
The implications in terms of grading are…
Why?
Differences across Disciplines There are large differences across disciplines
Sciences tend to grade more stringently than other disciplines
Education is the highest
These tend to be stable and consistent across institutions
They may relate to the attitudinal and frame of reference issues
Conclusions Grading should be used to help students learn
This approach requires that students are involved in their learning, that they are active learners, and they wish to have communication with faculty members and peers
Motivating students with grades has traditionally been tied to extrinsic motivation; we want students to have intrinsic motivation
Following the principles enclosed in this presentation should help increase student learning
Top Related