Critical Evaluation: Critical Reading and Critical Thinking (web version)
-
Upload
jamie-bisset -
Category
Education
-
view
1.074 -
download
2
description
Transcript of Critical Evaluation: Critical Reading and Critical Thinking (web version)
Critical EvaluationCritical ReadingCritical Thinking
James Bisset ([email protected] ) Academic Liaison Librarian (Research Support)
Session outline- What is Critical Reading / Critical
Thinking- Ecology of Resources
- Cognitive bias and you, the researcher
- Evaluating Research Information
The non-critical reader
- Reads a text as a source for...- memorising facts & statements - repeating facts & statements- building a narrative around facts & statements without analysing validity, reliability or applicability
The critical reader- Reads a text as a...
- One interpretation of facts
- Recognises the importance of...- what a text says- how the text evidences and portrays the subject matter
Critical Reading“ Critical Reading involves understanding the content of a text as well as how the subject matter is developed. Critical reading takes in the facts, but goes further. “
http://www.rimt.edu.au/studyandlearningcentre/
Critical Thinking“Critical thinking involves reflecting on the validity of what you have read in light of our prior knowledge and understanding of the world.“
http://www.criticalreading.com/critical_reading_thinking.htm
• Much training is about directing you to the right information = searching and retrieval
• As postgraduate researchers you have to be critical and reflect on what you find.
• Be aware of your impact on your own research, and the research of others.
• What defines your evaluative criteria?
The need to evaluate information
• Resources are interconnected and they evolve
• Information resources are transformed into knowledge
• Knowledge becomes a resource
• Therefore prior knowledge shapes what we go on to create
Ecology of Resources
• Resources are interconnected and they evolve
• Information resources are transformed into knowledge
• Knowledge becomes a resource
• Therefore prior knowledge shapes what we go on to create
Ecology of Resources
• In theory we can select almost any information to complete a task
• In practice we filter it by selecting resources we think most appropriate
• Motivation - affected by the learning we have already done
Role of the researcher
Role of the researcher
• In theory we can select almost any information to complete a task
• In practice we filter it by selecting resources we think most appropriate
• Motivation - affected by the learning we have already done
Other factors
• But, filtering is done for us BEFORE we get the chance to make a judgement
• People• Technologies• Cost• Skills• Copyright, IP
Other factors
• But, filtering is done for us BEFORE we get the chance to make a judgement
• People• Technologies• Cost• Skills• Copyright, IP
Other factors
• But, filtering is done for us BEFORE we get the chance to make a judgement
• People• Technologies• Cost• Skills• Copyright, IP
Other factors
• But, filtering is done for us BEFORE we get the chance to make a judgement
• People• Technologies• Cost• Skills• Copyright, IP
Other factors
• But, filtering is done for us BEFORE we get the chance to make a judgement
• People• Technologies• Cost• Skills• Copyright, IP
• You need to be critical and reflect on all of the sources you find and use.
• You have a professional responsibility, as your research will impact on others.
• You are creating knowledge…… which evolves, and will shape what others create…… similarly, the information you discover will shape the knowledge you create.
• You need to be aware of the filters already impacting upon the information you use.
Part 1 Summary
Cognitive biases• Subjectivity is vulnerable to bias &
hunches• Concept of cognitive bias was
developed in 1970s by Tversky and Kahneman
• Four main groups- Social - Probability/belief - Memory - Decision making
Social biases• Ascribe positive or negative traits to
self, individuals or groups• Loading values or anticipating action
based on prior experience or a bias against self, individuals or groups
• Academic impact: need to verify information and not rely on own views; important to remember when analysing human subjects
Memory biases• How you perceive past events• False memory, positive memory,
imbalanced memory• E.g. A Photo, a Suggestion, a False
Memory • Academic impact: importance of
accurate record keeping and note taking
Probability and belief
• To disregard or to pay too much attention to probability
• Academic impact: need to treat each research finding as distinct and to judge it in its own right
Decision-making biases
• Influences on your decisions by own biases or those of a group
• Academic impact: need to be objective and consider all possible routes of enquiry and treat all research findings as valid until proved otherwise e.g. Semmelweis reflex
Cognitive biases• On your table, group the
forty cards into four piles of ten
• Social • Memory • Probability• Decision
• Explored some of the key bias which may subconsciously be impacting upon how your (subjective) measure of ‘value’ might effect how you filter information and your evaluative criteria.#
Part 2 Summary
Evaluating information In a literature review you need to
evaluate:• Relevance to the topic• Authority of the author, publisher etc• Objectivity• Presentation• Methodology• Currency
Evaluating information In a literature review you need to
evaluate:• Relevance to the topic• Authority of the author, publisher etc• Objectivity• Presentation• Methodology• Currency
• Before reading the text…• Read the abstract, introduction or
summary.• Scan the bibliographic information
which may highlight key subject areas not specifically alluded to.
• Emphasis may not be clear until you read in full.
Relevance to the topic
• Upon reading the text…• What level is the information at?• Does it contain, and discuss in enough
detail the information you are seeking?• Is the research relevant to the subject
domain / geographical area / demographic / time period you are interested in?
• Three ways of reading: Comprehension, Analysis, Interpretation
Relevance to the topic
“In the course of a series of investigations into various aspects of mental inheritance an intensive study has been made of so-called ‘identical’ twins. The cases examined fall into two main groups: first, those reared together in their parents' homes; secondly, those separated in early infancy, and brought up apart. With the latter, despite wide differences in environmental conditions, the correlations for intelligence, unlike those for school attainments, prove to be surprisingly high. It is argued that this implies that ‘intelligence’, when adequately assessed, is largely dependent on genetic constitution.” http://10.1111/j.2044-8295.1966.tb01014.x“ Burt’s study of monozygotic twins reared apart … involved the largest number of separated twin pairs at the time and produced the highest estimate of heritability for IQ”
Relevance to the topic Exampl
e
Comprehension: Burke identified a link between IQ and inherited genes.
Analysis: This article compares the evidence for IQ being determined by inherited genes as opposed to IQ being affected by external environmental factors.
Interpretation: Evidence for IQ being an inherited trait rather than affected by external factors has potential implications for the development of social and education policy.
Relevance to the topic Exampl
e
• Key topics and ideas• Level of information presented• Relevance in terms of location/subject/scope.• Three ways of reading: Comprehension,
Analysis, Interpretation
Relevance to the topic
Be aware of what is filtering your choices… - Vocabulary and broadness of interpretation. Are you under-estimating the value of a source because it doesn’t match your choice of keywords precisely?
“The trouble with quotes from the internet is that
you never know if they are genuine.”
Abraham Lincoln(1809-1865, President of the United States of
America).
Authority
• Are the authors acknowledged experts in the field? - frequently cited? do they have an h-index? - have you or colleagues heard of them? - can you find any profile information where they work? - how well respected is the author, and their work, in their related field of research?
• Where is it published? - impact factors for a journal (not always an accurate measure of quality, but potentially one of prestige) - is it peer reviewed?
Authority
• Sir Cyril Burt• Fellow of British Academy• Author of over 350 articles and a number of books.
• “pioneer research on the inheritance of mental ability”
AuthorityExampl
e
””the most satisfactory attempt” to estimate hereditability of IQ” and
“”the most valuable” of all the separated twin research.”
”the largest of its kind and the only one where “the distribution of children into
foster homes was random ” “
Richard J Herrnstein
Arthur R Jenson Exampl
e
””the most satisfactory attempt” to estimate hereditability of IQ” and
“”the most valuable” of all the separated twin research.”
• Professor at the University of California, Berkeley. • Author of 400+ peer reviewed papers. • In 2002, was listed in the Review of General Psychology’s top 50 “most eminent psychologists of the 20th century”
”the largest of its kind and the only one where “the distribution of children into
foster homes was random ” “
• Edge Pierce Professor of Psychology at Harvard University.• Author of several peer reviewed papers and books. • Credited with discovering and developing several models and theories as one of the founding researchers in the field of quantitative analysis of behaviour.
Richard J Herrnstein
Arthur R Jenson
”the only one of its kind in which the calculation of heritability had any
meaning.”
””the best data“ on separated twins.”
William B Shockley
Hans Eysenck Exampl
e
”the only one of its kind in which the calculation of heritability had any
meaning.”
• Professor of Psychology at the Institute of Psychiatry, King’s College, London. • In 2002, was listed in the Review of General Psychology’s top 100 “most eminent psychologists of the 20th century” as the most cited living psychologist at the time of his death.
””the best data“ on separated twins.”
• Alexander M. Poniatoff Professor of Engineering and Applied Science at Stanford University.• Joint awarded Nobel Prize in Physics in 1956.• 1960’s and 1970’s moved also into area of hereditary behaviour.
William B Shockley
Hans Eysenck
- British Journal of Mathematical and Statistical Psychology - Impact Factor 1.258 - 5th of 13 journals in category “Psychology, Mathematical”
- British Journal of Educational Psychology - Impact Factor 2.093 - 11th of 50 journals in category “Psychology, Educational”
- British Journal of Psychology - Impact Factor 2.103 - 26th of 126 journals in category “Psychology, Multi-disciplinary”
Authority – Impact Factors
Exampl
e
• Are the authors acknowledged experts in the field? - frequently cited? do they have an h-index? - have you or colleagues heard of them? - can you find any profile information where they work? - how well respected is the author, and their work, in their related field of research?
• Where is it published? - Impact factors for a journal (not always an accurate measure of quality, but potentially one of prestige) and is it peer reviewed?
Authority
Be aware of what is filtering your choices y - Is the prestige of the author or the publication impacting on how you evaluate the content?
• Is the subject controversial?
• What is the authors purpose in writing the paper?
• If there are differing views on the subject area, does the author consistently fall into one ‘camp’?
Objectivity
** This is an over-simplification…. **
Objectivity
IQ is inherited IQ is affected by external
factorsBurt, C (1943) “Ability and Income” British Journal of Educational Psychology
Burt, C.L. (1957) “Heredity and Intelligence; A reply to criticisms” British Journal of Statistical Psychology
Burt, C.L. (1958). "The inheritance of mental ability", American Psychologist,
Burt, C.L. (1972). "Inheritance of general intelligence", American Psychologist,
Burt C (1966) “The Genetic Determination of Differences in Intelligence: A Study of Monozygotic Twins Reared Apart and Together.” British Journal of Psychology
Exampl
e
• Is the subject controversial?
• What is the authors purpose in writing the paper?
• If there are differing views on the subject area, does the author consistently fall into one ‘camp’?
Objectivity
Be aware of what is filtering your choices - Does the author demonstrate any hidden bias on the topic? - Evaluate yourself? Are you subconsciously over-valuing the resource because it confirms your own prejudices? Are you being objective?
• are the methodology and sources of data used clearly identified or explained?
• what evidence is presented to support the ideas and conclusions expressed?
• are the conclusions and assumptions made by the author consistent, logical and justified?
• does the research raise any unanswered questions?
Methodology
“ tests of the usual type… as a means of estimating genotypic differences, even the most carefully constructed tests are highly
fallible instruments, and … their verdicts are far less trustworthy than the judgments of the
pupil’s own teachers.”(on measuring IQ of subject children]
Methodology Exampl
e
“ tests of the usual type… as a means of estimating genotypic differences, even the most carefully
constructed tests are highly fallible instruments, and … their verdicts are far less trustworthy than the
judgments of the pupil’s own teachers.”Burt, C. (1957)British Journal of Statistical Psychology
“the unaided judgments even of the most experienced teachers … are nevertheless far less
trustworthy in the long run that the results obtained by properly applied intelligence tests.”
Burt, C. (1943)British Journal of Educational Psychology
Methodology Exampl
e
“ tests of the usual type… as a means of estimating genotypic differences, even the most carefully
constructed tests are highly fallible instruments, and … their verdicts are far less trustworthy than the
judgments of the pupil’s own teachers.”Burt, C. (1957)British Journal of Statistical Psychology
“the unaided judgments even of the most experienced teachers … are nevertheless far less
trustworthy in the long run that the results obtained by properly applied intelligence tests.”
Burt, C. (1943)British Journal of Educational Psychology
Methodology Exampl
e
2.23.39 – 2.25.56http://
news.bbc.co.uk/1/hi/help/3681938.stm
• Various criteria you can assess a resource by.
- a lot more ‘citation’ tools available for journal literature.
• How much time do you realistically have?
Part 3 Summary
• All quotes and opinions were taken from one article:
Tucker, W.H.(1994) “Fact and fiction in the discoevry of Sir Cyril Burt’s Flaws” Journal of the History of the Behavioral Sciences (30).
• Does that change your opinion on some of the previous slides assertions and emphasis?
Part 3 Summary
Bibliography• Kahneman, D. and Amos, A. (1972) ‘Subjective probability: a judgment of
representativeness’, Cognitive Psychology. 3(3): 430-454.
• Luckin, R. (2010) Redesigning learning contexts: technology-rich, learner-centred ecologies. Abingdon: Routledge.
• Strange, D., Hayne, H. and Garry, M. (2008) ’A photo, a suggestion, a false memory’, Applied Cognitive Psychology. 22: 587–603.
• Whitworth, D (2010) “The three domains of value: Why IL practitioners must take a holistic approach” Available at: http://prezi.com/rxqnzpoooolb/the-three-domains-of-value-why-il-practitioners-must-take-a-holistic-approach/
• Tucker, W.H.(1994) “Fact and fiction in the discoevry of Sir Cyril Burt’s Flaws” Journal of the History of the Behavioral Sciences (30).
• http://www.informationliteracy.ie/
Image Credits[Slide 11, 12] Via Flickr Creative Commons, by Martin LaBar. Original available at http://www.flickr.com/photos/32454422@N00/163107859/
[Slide 3, 8, 21, 36] Via Flickr Creative Commons, by Kevin Dooley. Original available at http://www.flickr.com/photos/12836528@N00/2577006675
[Slide 14] Via Flickr Creative Commons, by shellorz. Original available at http://www.flickr.com/photos/59198719@N00/2192821345
[Slide 17] Via Flickr Creative Commons, by Richard Cocks. Original available at http://www.flickr.com/photos/richardland/3999234316/
[Slide 16] Via Flickr Creative Commons, by Photo Extremist. Original available at http://www.flickr.com/photos/thevlue/4839060646/
[Slide 15] Via Flickr Creative Commons, by What Dave Sees. Original available at http://www.flickr.com/photos/whatdavesees/2487875504/
Image CreditsPicture of Professor Arthur Jensen at the 2002 ISIR meeting. Author: Tim Bates. Available at http://en.wikipedia.org/wiki/File:Arthur_Jensen_Vanderbilt_2002.jpg
Picture of Hans Eysenck. Author: Sirswindon. Available at https://en.wikipedia.org/wiki/File:Hans.Eysenck.jpg
Picture of William B Shockley. Author Chuck Painter / Stanford News Service. Available at http://en.wikipedia.org/wiki/File:William_Shockley,_Stanford_University.jpg
[Slide 18] Via Flickr Creative Commons, by vl8189. Original available at http://www.flickr.com/photos/27630470@N03/
[Slide 19] Via Flickr Creative Commons, by opensourceway. Original available at http://www.flickr.com/photos/47691521@N07/4371001458/
[Slide 22] From tdifh.blogspot.com
[Slide 28-29] Provided by colleague
[Slide 68] ‘Vitae®, © 2010 Careers Research and Advisory Centre (CRAC) Limited‘ Available at www.vitae.ac.uk/rdf