HEURISTICS AS WARRANTS: Leveraging Sourcing and Corroboration Heuristics as Warrants in Historical...
-
Upload
ruth-johnson -
Category
Documents
-
view
216 -
download
0
Transcript of HEURISTICS AS WARRANTS: Leveraging Sourcing and Corroboration Heuristics as Warrants in Historical...
HEURISTICS AS WARRANTS:
Leveraging Sourcing and Corroboration Heuristics as
Warrants in Historical Argumentative Writing
Ryan McCarty@RyanP_McCarty
[email protected] of Illinois at Chicago
LRA Annual Conference12/4/2015
Agenda
My Path to this Research Theoretical Framework Methods Findings and Discussion Implications
My Path to this Research
My time as a teacher
The importance of warrants, and warrants that may influence those in power
Using historical thinking heuristics to warrant claims
Historical Thinking Heuristics (& Close Reading) as Warrants
Sourcing identifying and evaluating the credibility of a source
based on its author or origin (Reisman, 2012)
Corroboration reading across multiple historical accounts to compare
content and identify differences, giving more weight to information common across accounts (Britt & Aglinskas, 2002)
Close Reading analyzing an author’s language use for three purposes:
to learn what a text says, to determine how a text works, and to evaluate its larger significance in relation to other texts and ideas (Adler, 1965; T. Shanahan, 2013)
Theoretical and Empirical Basis
Disciplinary Literacy (e.g. Moje, 2008; Shanahan & Shanahan, 2008)
Discourse Perspectives (e.g. Gee, 1989; Lankshear et al., 1997)
Multiple Text Comprehension (e.g. Perfetti et al., 1999; Wineburg, 1991)
Reading and Writing Connection (e.g. Fitzgerald & Shanahan, 2000; Tierney & Shanahan, 1996)
Argumentation & Historical Argumentation (Freeman, 2005; Toulmin, 1958; De La Paz & Felton, 2010; Monte-Sano, 2008)
Study Design
Quasi-Experimental and Mixed Methods 89 11th grade students, US History class,
diverse setting near major urban area Two instructors; 8 days total
Treatment Condition: Heuristics as warrants, warrant-generating questions (Freeman, 2011; Toulmin, 1958) and sentence frames (Graff & Birkenstein, 2007)
Comparison Treatment: Finding evidence to support different claims; Evaluating evidence
Research Questions and Hypotheses
Question 1: Does a treatment in using heuristics as warrants improve student performance on selecting and writing effective warrants, compared to a treatment in finding effective evidence?
Question 2: How do students perform on warrant selection and warrant writing items meant to reflect specific types of historical thinking?
Question 3: What sort of thinking does a subset of students in the treatment group engage in during the warrant selection and warrant writing tasks?
Hypothesis- Students in the treatment condition will select and write warrants more effectively than students in the comparison treatment
Central Questions of Unit
Did the explosion of the U.S.S. Maine cause the U.S. to invade Cuba?
Yes, the explosion of the U.S.S. Maine caused the U.S. to invade Cuba
No, the explosion of the U.S.S. Maine did not cause the U.S. to invade Cuba
Should the U.S. have invaded Cuba following the explosion of the U.S.S. Maine?
Yes, the U.S. should have invaded Cuba… No, the U.S. should not have invaded Cuba…
Text Set for Assessment
http://historicalthinkingmatters.org/spanishamericanwar/
Two newspaper articles, a song, two speeches, two telegrams, and a textbook excerpt
Methods
Figure 3.1. Example of a Warrant Selection Task
Example Warrant Selection Task
Figure 3.2. Example of Warrant Writing Task
Example Warrant Writing Task
Impact of Treatment on Warrant Selection & Writing
1a. Ability to Select Effective Warrants Warrant Selection Task
Ind. Samples t-test Warrant Selection Task “Correct” vs. “Not
Wrong” answers Ind. Samples t-test
1b. Ability to Write More Effective Warrants Coding for Aspects of Quality
Examine Means Coding for Historical Thinking
Ind. Samples t-test
Impact on Specific Types of Thinking
2a. Warrant Selection Performance by Type of Thinking Item Analysis by type of thinking
Ind. Samples t-test
2b. Warrant Writing Performance by Type of Thinking Coding for type of thinking
Ind. Samples t-test
Student Thinking During Tasks; Student Thoughts on Treatment 3a. Types of Thinking Displayed During
Task Subset of 4 students from Treatment Condition Think Aloud Protocol
Coding for type and percentage of historical thinking Coding for effectiveness
3b. Student Thoughts on Intervention Semi-structured Interview
Coding for type and percentages of historical thinking Coding for what learned, how different from typical
instruction Coding for what would make this easier (item 10)
Findings and Discussion
Research Question 1: Findings and Discussion
Ability to Select Effective Warrants Selection overall: No Statistically Significant Difference Heuristics M = 39.32 vs. Evidence M = 36.63, range = 26
Difficulty of foils
“Correct” vs. “Not Wrong”: No Statistically Significant Difference Heuristics M = 19.16 vs. Evidence M = 15.97, range = 22
p = 0.01; = .005 Ability to Write More Effective Warrants
Quality: No Statistically Significant Difference Accuracy slightly lower for Heuristics
Overall Hist. Thinking: No Statistically Significant Difference (M = 2.20 vs. M = 1.36), range = 4
p = 0.01; = .003
Claim: No, explosion of U.S.S. Maine did not cause the U.S. to invade Cuba
Evidence: The New York Times (Doc B) reported that Navy Leader Captain Schuley felt the explosion was an “accident”, likely caused by a coal bunker fire that ignited the ship’s ammunition. However, the New York Journal (Doc A) stated that “all” naval officers thought the Maine was “purposely” destroyed by a Spanish mine. It says the “brutal” Spaniards may have triggered the explosion while the men were sleeping.
Student Warrant: “While Doc A says it wasn't an accident, Doc B says it was an accident. The New York Times tells the truth about certain events that actually did happen. Unlike the New York Journal they always exaggerate and make up stories to make it more reliable. Doc B provides more evidence to show why because of a fire that could have occurred. Just because Doc A uses the word brutal doesn't mean - the explosion of the U.S.S Maine (caused) the U.S to invade Cuba.”
Writing Task Example: Heuristics Used as Warrant
Research Question 2: Findings and Discussion
2b. Warrant Selection Performance by Type of Thinking
Mean performance was slightly higher for treatment across types of thinking
Mean difference largest for corroboration 2c. Warrant Writing Performance by Type of
Thinking No difference in thinking types overall
Items cued different types of thinking Lowest performance on “Yes, should have caused”
Predicate/side
Table 4.6
Performance on Warrant Selection Task by Type of Thinking
Total Items
Group
M (SD)
Sourcing 16 Treatment 10.32 (2.27)
Comparison 9.73 (2.36)
Close Reading 16 Treatment
Comparison
10.60 (1.89)
9.80 (2.16)
Corroboration 16 Treatment
Comparison
10.68 (2.17)
9.67 (2.12)
Non-Historical 12 Treatment 7.84 (1.52)
Comparison 7.57 (2.27)
Note. n for Treatment =25. n for Comparison Treatment =30
Table 4.9
Performance on Warrant Writing Task by Type of Thinking- Items 1&2 Total
Highest Possible Total
Group
M (SD)
Sourcing 2 Treatment 1.00 (0.71)
Comparison 0.68 (0.61)
Close Reading 2 Treatment
Comparison
0.64 (0.70)
0.36 (0.56)
Corroboration 2 Treatment
Comparison
0.56 (0.58)
0.32 (0.48)
Sum of Historical Thinking 6 Treatment 2.20 (1.29)
Comparison 1.36 (1.03)
Note. For treatment, n=25. For Comparison Treatment, N=28
Student Thinking During Task; Student Thoughts on Treatment
3a. Heuristic Use During Think Aloud Warrant Selection:
All used at least 2 types of historical thinking Close reading effectiveness ranged from 67% to 100%
Warrant Writing: All used Sourcing; Consistently effective (14 of 15 idea units) Task may be easier for students
3b. Student Responses During Interview Heuristic Use
All show some hist. thinking; Primarily sourcing No close reading; Did not refer to text during responses
Suggestions for Making Work Easier: Time, practice, feedback Changes to task design
What They Learned: 2 emphasized heuristics as warrants, one said argument structure one found it confusing
Ivan on what he learned
I learned how to use sourcing, like how to connect the evidence and that could make a strong argument, just from sourcing. I didn’t know that before… now we used sourcing, as forwarding your claim [sic].
… So we used sourcing to see if it was reliable or not, but now we’re like, we’re seeing “So this is reliable… because this person was there, and when was this written, and how far or how close it was, would that change their viewpoint.
... I would like to say, “Hey this guy, he was there, he survived, he would know a little more about what happened there, than someone who was like running around ten years later, based on like information they got offline or something like that [sic].
Sonia on what she learned
I’m not really sure. Like the warrants thing confused me a little because we’ve been using reasoning from the beginning of the school year. So… having a new term (was) kind of, well confusing, how to use it, when to use it or what it is [sic].
It was somewhat easier, but it didn’t feel like I was doing much, like it was coming from me, so it was copy and paste… We usually write our own claims, well like the whole week we usually have our own claims, our own evidence, and then our own like reasoning and concluding sentence, everything was there for us (but here) we had to use what we were given…
Implications
Limitations
Design limitations Narrow definition of historical
thinking Refine Instrumentation
Warrant selection Warrant writing Think Aloud & Interview
Implications
Implications for Research Innovative approach to instrumentation; refine
further Refine treatment to positively impact student
learning Continued research of warrants, a neglected yet
essential area of study Implications for Instruction
Implications limited due to nonsignificant results Explicitly teaching of hist. argument writing Use of heuristics in combination; Close reading
in discipline
Next Steps
Qualitative analysis of essay writing data
Sampling of essays from each quartile Breaking into idea units, coding for aspect of
argumentation (i.e. claim, evidence, warrant) More refined analysis of type of historical
thinking present in warrant Looking across data sets Combining measures for MANOVA
analysis Formative Design-Based Research;
Transfer to issues students choose