ChemEd DL is the Chemical Education Pathway of the National STEM Distributed Learning
ChemEd 2003 Auburn University, July 27-31, 2003
description
Transcript of ChemEd 2003 Auburn University, July 27-31, 2003
![Page 1: ChemEd 2003 Auburn University, July 27-31, 2003](https://reader035.fdocuments.us/reader035/viewer/2022062409/568146e8550346895db4222b/html5/thumbnails/1.jpg)
1
ChemEd 2003Auburn University, July 27-31, 2003
![Page 2: ChemEd 2003 Auburn University, July 27-31, 2003](https://reader035.fdocuments.us/reader035/viewer/2022062409/568146e8550346895db4222b/html5/thumbnails/2.jpg)
2
A Methodology for the Simultaneous Assessment of
Cognitive and Content Domains
Ray A. Gross, Jr.
Prince George’s Community CollegeLargo, Maryland
![Page 3: ChemEd 2003 Auburn University, July 27-31, 2003](https://reader035.fdocuments.us/reader035/viewer/2022062409/568146e8550346895db4222b/html5/thumbnails/3.jpg)
3
My Final Objective for Today
• To provide a methodology for identifying student weaknesses--both in content areas and cognitive skills
• So that you may use the methodology to find weaknesses and then devise strategies to overcome the weaknesses
![Page 4: ChemEd 2003 Auburn University, July 27-31, 2003](https://reader035.fdocuments.us/reader035/viewer/2022062409/568146e8550346895db4222b/html5/thumbnails/4.jpg)
4
My Intermediate Objective
• To show how to develop a set of desired learning outcomes (DLOs) that reflect what is actually tested and can also be assessed
![Page 5: ChemEd 2003 Auburn University, July 27-31, 2003](https://reader035.fdocuments.us/reader035/viewer/2022062409/568146e8550346895db4222b/html5/thumbnails/5.jpg)
5
Outline of Presentation
1. Define terms
2. Develop content categories
3. Marry content and cognitive categories in a matrix
4. Write DLOs from the matrix
5. Code exam questions for assessment
6. Prepare an assessment matrix
![Page 6: ChemEd 2003 Auburn University, July 27-31, 2003](https://reader035.fdocuments.us/reader035/viewer/2022062409/568146e8550346895db4222b/html5/thumbnails/6.jpg)
6
What is a DLO?
A statement written by an instructor to a student telling the student by an action verb (related to a cognitive skill) what the instructor expects the student to be able to demonstrate during a semester for a specific topic (related to a content area).
![Page 7: ChemEd 2003 Auburn University, July 27-31, 2003](https://reader035.fdocuments.us/reader035/viewer/2022062409/568146e8550346895db4222b/html5/thumbnails/7.jpg)
7
What is a DLO?
• A statement the instructor can use to code exam questions so that the questions can be assessed by content area and cognitive skill in a systematic and consistent fashion.
![Page 8: ChemEd 2003 Auburn University, July 27-31, 2003](https://reader035.fdocuments.us/reader035/viewer/2022062409/568146e8550346895db4222b/html5/thumbnails/8.jpg)
8
Example of a DLO
• Predict products for name reactions such as the Grignard and Wittig reactions.
• A DLO completes the phrase—“Upon successful completion of this course, a student should (or will) be able to:”
![Page 9: ChemEd 2003 Auburn University, July 27-31, 2003](https://reader035.fdocuments.us/reader035/viewer/2022062409/568146e8550346895db4222b/html5/thumbnails/9.jpg)
9
A Set of DLOs
• The number of DLOs determined by the instructor to cover the content domain of the course.
![Page 10: ChemEd 2003 Auburn University, July 27-31, 2003](https://reader035.fdocuments.us/reader035/viewer/2022062409/568146e8550346895db4222b/html5/thumbnails/10.jpg)
10
Two Elements in a DLO
• Cognitive Skill—A mental process used by a person in problem solving (general – applies to all courses)
• Content Area—A portion of a course defined by the instructor (specific – applies to a given course)
• Time—A semester
![Page 11: ChemEd 2003 Auburn University, July 27-31, 2003](https://reader035.fdocuments.us/reader035/viewer/2022062409/568146e8550346895db4222b/html5/thumbnails/11.jpg)
11
What are the Cognitive Skills?
• Some person’s idea of the mental processes humans use to solve problems (i.e., learn).
• Bloom’s Taxonomy: Six ways of thinking or six cognitive skills humans use to solve problems, rank ordered by complexity. These six skills span all disciplines and are adopted for use herein.
![Page 12: ChemEd 2003 Auburn University, July 27-31, 2003](https://reader035.fdocuments.us/reader035/viewer/2022062409/568146e8550346895db4222b/html5/thumbnails/12.jpg)
12
Bloom’s Cognitive Domain
A. Knowledge--rote
B. Comprehension—grasp meaning
C. Application—use material in new situations
D. Analysis—break material into components
E. Synthesis—put parts together
F. Evaluation—use judgment
![Page 13: ChemEd 2003 Auburn University, July 27-31, 2003](https://reader035.fdocuments.us/reader035/viewer/2022062409/568146e8550346895db4222b/html5/thumbnails/13.jpg)
13
Content Domain
• A list of content areas that can be subdivided into course topics
• The content domain is course specific and must be determined by the instructor.
![Page 14: ChemEd 2003 Auburn University, July 27-31, 2003](https://reader035.fdocuments.us/reader035/viewer/2022062409/568146e8550346895db4222b/html5/thumbnails/14.jpg)
14
Content Domain
• Course topics are derived from content areas in the same way action verbs are derived from cognitive areas.
• An example of a content domain is shown on the next slide for organic chemistry.
![Page 15: ChemEd 2003 Auburn University, July 27-31, 2003](https://reader035.fdocuments.us/reader035/viewer/2022062409/568146e8550346895db4222b/html5/thumbnails/15.jpg)
15
A Content Domain (Orgo)
1. Nomenclature2. Equations3. Mechanisms4. Problems 5. Synthesis6. Explanations7. Correlations8. Classifications9. Definitions
![Page 16: ChemEd 2003 Auburn University, July 27-31, 2003](https://reader035.fdocuments.us/reader035/viewer/2022062409/568146e8550346895db4222b/html5/thumbnails/16.jpg)
16
Your Content Domain
• Determined initially by what you have tested in the past
![Page 17: ChemEd 2003 Auburn University, July 27-31, 2003](https://reader035.fdocuments.us/reader035/viewer/2022062409/568146e8550346895db4222b/html5/thumbnails/17.jpg)
17
Limits of Assessment
• The cognitive domain (six alphabetic items) and content domain (nine numeric items) define the level to which the course is subdivided for assessment purposes.
• For the example, a 6 x 9 matrix defines the parameters for assessment and the corresponding DLOs.
![Page 18: ChemEd 2003 Auburn University, July 27-31, 2003](https://reader035.fdocuments.us/reader035/viewer/2022062409/568146e8550346895db4222b/html5/thumbnails/18.jpg)
18
What is a Cognitive-Content Matrix?
• A way of showing how the content and cognitive domains are interrelated
• A way of showing how the two independent variables that make up a set of DLOs are related
• A tool for qualitatively or quantitatively assessing an evaluation or combined evaluations
![Page 19: ChemEd 2003 Auburn University, July 27-31, 2003](https://reader035.fdocuments.us/reader035/viewer/2022062409/568146e8550346895db4222b/html5/thumbnails/19.jpg)
19
Cog
Cont
Kno
A
Com
B
App
C
Ana
D
Syn
E
Evl
F
1-- x x x
2-- x x
3-- x
4-- x
5-- x x x
6-- x
Hypothetical Cognitive-Content Matrix
![Page 20: ChemEd 2003 Auburn University, July 27-31, 2003](https://reader035.fdocuments.us/reader035/viewer/2022062409/568146e8550346895db4222b/html5/thumbnails/20.jpg)
20
Before Developing Content Domain--we will use Organic Nomenclature to:
• Explore how a given content area may be assessed by all six cognitive skills
• Learn how to dissect a question into its cognitive and content components
• Learn how to code a question by its content and cognitive components
![Page 21: ChemEd 2003 Auburn University, July 27-31, 2003](https://reader035.fdocuments.us/reader035/viewer/2022062409/568146e8550346895db4222b/html5/thumbnails/21.jpg)
21
Six Cognitive Codes for each Content Code
• Content area 1 = Nomenclature
• DLOs may be coded from 1-A to 1-F
![Page 22: ChemEd 2003 Auburn University, July 27-31, 2003](https://reader035.fdocuments.us/reader035/viewer/2022062409/568146e8550346895db4222b/html5/thumbnails/22.jpg)
22
A. Knowledge—rote
Q1. Write the name of CH4?
Code = 1. Nomenclature
A. Knowledge
• The student is intimately familiar
with the item.
![Page 23: ChemEd 2003 Auburn University, July 27-31, 2003](https://reader035.fdocuments.us/reader035/viewer/2022062409/568146e8550346895db4222b/html5/thumbnails/23.jpg)
23
B. Comprehension—grasp meaning
Q2. Give the IUPAC name of CH3CH(CH3)CH3.
Code = 1. Nomenclature
B. Comprehension
• The formula is not new to the student; the instructor discussed it prior to the exam.
![Page 24: ChemEd 2003 Auburn University, July 27-31, 2003](https://reader035.fdocuments.us/reader035/viewer/2022062409/568146e8550346895db4222b/html5/thumbnails/24.jpg)
24
C. Application—use material in new situations
Q3. Give the IUPAC name for CH3C(CH3)2CH(CH3)CH3.
Code = 1. Nomenclature
C. Application
• Student has never seen this structure before; nomenclature rules are applied.
![Page 25: ChemEd 2003 Auburn University, July 27-31, 2003](https://reader035.fdocuments.us/reader035/viewer/2022062409/568146e8550346895db4222b/html5/thumbnails/25.jpg)
25
D. Analysis—break material into components
Q4. What is the parent hydrocarbon of 3-isopropyl-4-methylheptane.
Code = 1. Nomenclature
D. Analysis
• The name is broken into its component parts to find parent name.
![Page 26: ChemEd 2003 Auburn University, July 27-31, 2003](https://reader035.fdocuments.us/reader035/viewer/2022062409/568146e8550346895db4222b/html5/thumbnails/26.jpg)
26
E. Synthesis—put parts together
• Q5. What is the IUPAC name of the structure on your right?
Code = 1. Nomenclature
E. Synthesis
• Component parts must be assembled into a name; requires analysis then synthesis.
![Page 27: ChemEd 2003 Auburn University, July 27-31, 2003](https://reader035.fdocuments.us/reader035/viewer/2022062409/568146e8550346895db4222b/html5/thumbnails/27.jpg)
27
F. Evaluation—use judgment
O
N
OH
O
OHCH3O O
N
OH
O
CH3Odehydration rearrangement
?
8 9
Figure 1
• A research group studied the mechanism for the dehydration and rearrangement of the aglucone 2,4-dihydroxy-7-methoxy-2H-1,4-benzoxazin-3(4H)-one (8) into 3-formyl-6-methoxybenzoxazolin-2(3H)-one (9) by the reagent N-ethoxycarbonyltrichloroacetaldimine.
![Page 28: ChemEd 2003 Auburn University, July 27-31, 2003](https://reader035.fdocuments.us/reader035/viewer/2022062409/568146e8550346895db4222b/html5/thumbnails/28.jpg)
28
F. Evaluation—use judgment(continued)
• Q6. The carbon atom in 8, indicated by the question mark in Figure 1, should be given what number by nomenclature rules?
• Code = 1. Nomenclature
F. Evaluation
• Judgment is required in comparing two ring systems, which students haven’t seen before.
![Page 29: ChemEd 2003 Auburn University, July 27-31, 2003](https://reader035.fdocuments.us/reader035/viewer/2022062409/568146e8550346895db4222b/html5/thumbnails/29.jpg)
29
Summary
• All six cognitive skills (codes A-F) might be tested against nomenclature (code 1).
• In theory, 54 (6 x 9) question types might be formulated for organic chemistry.
• For any course in which Bloom’s taxonomy is adopted, a similar matrix is theoretically possible.
• In practice, only a fraction of the possible codes is needed.
![Page 30: ChemEd 2003 Auburn University, July 27-31, 2003](https://reader035.fdocuments.us/reader035/viewer/2022062409/568146e8550346895db4222b/html5/thumbnails/30.jpg)
30
How is the Content Domain Developed?
• By analysis not synthesis
• Analyze previous exams, categorize each Q by content
• Consolidate the content in the questions into a content domain
• Also, categorize each Q by cognitive category as you go (Bloom’s A-F)
![Page 31: ChemEd 2003 Auburn University, July 27-31, 2003](https://reader035.fdocuments.us/reader035/viewer/2022062409/568146e8550346895db4222b/html5/thumbnails/31.jpg)
31
Content Development Process
1. Make a content domain of about 8 categories.
2. Code previous questions into one of the categories by number.
3. Add categories if necessary.
4. Consolidate categories if necessary.
![Page 32: ChemEd 2003 Auburn University, July 27-31, 2003](https://reader035.fdocuments.us/reader035/viewer/2022062409/568146e8550346895db4222b/html5/thumbnails/32.jpg)
32
The Process is a Retro-Analysis of each Question
5. Concurrently, code each Q by letter—a Bloom cognitive skill (e.g., C).
6. Use the codes to make a qualitative content—cognitive matrix (see slide 18).
![Page 33: ChemEd 2003 Auburn University, July 27-31, 2003](https://reader035.fdocuments.us/reader035/viewer/2022062409/568146e8550346895db4222b/html5/thumbnails/33.jpg)
33
Build a Qualitative Cognitive-Content Matrix
• Lay out a content vs cognitive matrix by number and letter codes.
• Each dual-coded Q (e.g., 7D, 4C, and 6F) equates to a box in the matrix.
• Place an X in each box that is represented by a dual-coded Q.
• The result is a qualitative matrix• It shows what you actually tested by content
and cognitive categories.
![Page 34: ChemEd 2003 Auburn University, July 27-31, 2003](https://reader035.fdocuments.us/reader035/viewer/2022062409/568146e8550346895db4222b/html5/thumbnails/34.jpg)
34
Qualitative Matrix
• Shows which content and cognitive categories are to be assessed or were assessed
![Page 35: ChemEd 2003 Auburn University, July 27-31, 2003](https://reader035.fdocuments.us/reader035/viewer/2022062409/568146e8550346895db4222b/html5/thumbnails/35.jpg)
35
Cog
Cont
Kno
A
Com
B
App
C
Ana
D
Syn
E
Evl
F
1-- x x x
2-- x x
3-- x
4-- x
5-- x x x
6-- x
![Page 36: ChemEd 2003 Auburn University, July 27-31, 2003](https://reader035.fdocuments.us/reader035/viewer/2022062409/568146e8550346895db4222b/html5/thumbnails/36.jpg)
36
Subordinate Cognitive Domain to Content Domain
• In order to minimize the number of DLOs, the cognitive domain is subordinated to the content domain, so that only nine DLOs are needed for the 54 possibilities.
• The cognitive skill for a specific question will be evident by the letter code you assign when the question is formulated.
![Page 37: ChemEd 2003 Auburn University, July 27-31, 2003](https://reader035.fdocuments.us/reader035/viewer/2022062409/568146e8550346895db4222b/html5/thumbnails/37.jpg)
37
Use the Qualitative Matrix to Develop DLOs
Cognitive
Content
Know.
A
Comp.
B
Appl.
C
Nomen.
1.
X
Identify
Name or Draw
X
(Re)write
Name or Draw
X
Construct or Formulate
Name or Draw
![Page 38: ChemEd 2003 Auburn University, July 27-31, 2003](https://reader035.fdocuments.us/reader035/viewer/2022062409/568146e8550346895db4222b/html5/thumbnails/38.jpg)
38
Consolidation of Cognitive Skills
• One or more verbs that encompass all of the cognitive skills tested are married with the content category to arrive at a single DLO for nomenclature.
![Page 39: ChemEd 2003 Auburn University, July 27-31, 2003](https://reader035.fdocuments.us/reader035/viewer/2022062409/568146e8550346895db4222b/html5/thumbnails/39.jpg)
39
Construct DLO
• Name chemical species from structures and draw structures from names.
• This is the DLO for nomenclature.
• Develop DLOs for each content category in the same way.
![Page 40: ChemEd 2003 Auburn University, July 27-31, 2003](https://reader035.fdocuments.us/reader035/viewer/2022062409/568146e8550346895db4222b/html5/thumbnails/40.jpg)
40
Cognitive-Content Linkage
Application
Applies principles to new situtations
CognitiveGeneral Instructional Objectives Verb for
Specific LO
Predict
Reaction products
Content
Equations
![Page 41: ChemEd 2003 Auburn University, July 27-31, 2003](https://reader035.fdocuments.us/reader035/viewer/2022062409/568146e8550346895db4222b/html5/thumbnails/41.jpg)
41
Process Marries Domains
• For the next exam, dual code each Q.
• Assign points for each Q and develop a quantitative matrix.
• Determine max points for each box.
• Calculate student percentages for each box.
• Both content and cognitive skills are assessable.
![Page 42: ChemEd 2003 Auburn University, July 27-31, 2003](https://reader035.fdocuments.us/reader035/viewer/2022062409/568146e8550346895db4222b/html5/thumbnails/42.jpg)
42
Dual Coding Process
• Allows instructor to predetermine the mix of content and cognitive skills to be assessed for each evaluation
![Page 43: ChemEd 2003 Auburn University, July 27-31, 2003](https://reader035.fdocuments.us/reader035/viewer/2022062409/568146e8550346895db4222b/html5/thumbnails/43.jpg)
43
Coding Example 1
• Which compound, ethanol, ethane or diethyl ether has the highest boiling point?
• Content = 7. Correlation (property—structure) • Cognitive = D. Analysis or B. Comprehension
![Page 44: ChemEd 2003 Auburn University, July 27-31, 2003](https://reader035.fdocuments.us/reader035/viewer/2022062409/568146e8550346895db4222b/html5/thumbnails/44.jpg)
44
Note
• In the previous example, an instructor who covered the specific question in class would code the question as a comprehension, whereas an instructor who covered the principles involved but not the specific question would code the question as an application.
• (Added per ChemEd 2003 participation!)
![Page 45: ChemEd 2003 Auburn University, July 27-31, 2003](https://reader035.fdocuments.us/reader035/viewer/2022062409/568146e8550346895db4222b/html5/thumbnails/45.jpg)
45
Coding Example 2
• Calculate the degrees of unsaturation in C5H10O.
• Content = 4. Problem• Cognitive = C. Application
![Page 46: ChemEd 2003 Auburn University, July 27-31, 2003](https://reader035.fdocuments.us/reader035/viewer/2022062409/568146e8550346895db4222b/html5/thumbnails/46.jpg)
46
Coding Example 3
• Explain why the nitration of bromobenzene is slower than that of benzene yet produces an ortho,para product.
• Cognitive = 6. Evaluation • Content = F. Explanation
![Page 47: ChemEd 2003 Auburn University, July 27-31, 2003](https://reader035.fdocuments.us/reader035/viewer/2022062409/568146e8550346895db4222b/html5/thumbnails/47.jpg)
47
A Partial Quantitative Matrix
CognitiveContent
Totals ↓ A B C D E F
1 80% 90 70 80
2 70% 70
3 48% 48
4 70% 66 67 75
Totals %
→
90 70 55 69 80 75
![Page 48: ChemEd 2003 Auburn University, July 27-31, 2003](https://reader035.fdocuments.us/reader035/viewer/2022062409/568146e8550346895db4222b/html5/thumbnails/48.jpg)
48
Quantitative Matrix
• See the Word document (in the folder with these PowerPoint slides) for an actual quantitative matrix.
• The quantitative matrix identifies student strengths and weaknesses.
• A strategy for improving weaknesses can be developed from the identified weaknesses.
![Page 49: ChemEd 2003 Auburn University, July 27-31, 2003](https://reader035.fdocuments.us/reader035/viewer/2022062409/568146e8550346895db4222b/html5/thumbnails/49.jpg)
49
Summary• Content areas may be assessed against each
applicable cognitive skill by dual coding test questions.
• Content-cognitive weaknesses can be identified and actions formulated to improve student learning in those areas.
• The skill levels (i.e., degree of higher-order thinking) for a given exam can be determined in advance by systematically coding exam questions as they are prepared.
![Page 50: ChemEd 2003 Auburn University, July 27-31, 2003](https://reader035.fdocuments.us/reader035/viewer/2022062409/568146e8550346895db4222b/html5/thumbnails/50.jpg)
50
Comments
• The DLOs provide a methodology for doing a macro-assessment that is useful for instructors.
• For the DLOs to be meaningful for students, examples of coded test questions for each teaching block are probably necessary.