Post on 19-Dec-2015
Critical evaluation of research information
Laura Jeffrey Researcher Training
Course outline
• Importance of evaluation• Forms of value– Subjective• Cognitive bias
– Objective– Intersubjective
• Relationship between the forms of value
The need to evaluate information
• Much training is about directing you to the right information = searching and retrieval
• One of the distinctive elements of postgraduate research is that you have to be critical and reflect on what you find
• What defines your evaluative criteria?
Ecology of resources
• Term developed by Rosemary Luckin• Resources are interconnected and they evolve
e.g. natural resources• Information resources are transformed into
knowledge• Knowledge becomes a resource• Therefore prior knowledge shapes
what we go on to create
Role of the researcher
• In theory we can select almost any information to complete a task
• In practice we filter it by selecting resources we think most appropriate
• Motivation - affected by the learning we have already done
Other factors
• But, filtering is done for us BEFORE we get the chance to make a judgement
• People• Technologies• Cost• Skills• Copyright, IP
Value
• Filtering process = value judgement– By researcher– Made on their behalf
• What forms of value are there and how do they work together to create information literate researcher?
Subjective form of value
• Decisions we make– Is this what I want, do I need this, is it relevant?
• Privileges you as the researcher in the decision making process
• If we omit it we get groupthink (Janis, 1972) or battery cognition (Blaug, 2007)
• Importance therefore of asserting individual criticality
Cognitive bias
• SO… subjectivity is vulnerable to bias and hunches
• Concept of cognitive bias was developed in 1970s by Tversky and Kahneman
• Four main groups– Social– Memory– Probability/belief – Decision making
Social biases
• Ascribe positive or negative traits to self, individuals or groups
• Loading values or anticipating action based on prior experience or a bias against self, individuals or groups
• Academic impact: need to verify information and not rely on own views; important to remember when analysing human subjects
Memory biases
• How you perceive past events• False memory, positive memory, imbalanced
memory• E.g. A Photo, a Suggestion, a False Memory • Academic impact: importance of
accurate record keeping and note taking
Probability and belief
• To disregard or to pay too much attention to probability
• Academic impact: need to treat each research finding as distinct and to judge it in its own right
Decision making biases
• Influences on your decisions by own biases or those of a group
• Academic impact: need to be objective and consider all possible routes of enquiry and treat all research findings as valid until proved otherwise e.g. Semmelweis reflex
Cognitive biases
• On your table, group the forty cards into four piles of ten– Social – Memory – Probability– Decision
Objective form of value
• Scientific measures of validity or reliability• Exists so that subjective values don’t unduly
influence work • Omit this scheme of value and we risk
information (and knowledge formed from it) becoming counterknowledge (Thompson, 2008).
Intersubjective form of value
• Based on the shared values of a community e.g. morals, ethics, laws, economics
• Allows for discussion of scientific method as it can’t explain everything
• Acceptance in a community• If we omit this then values are relativist
Three forms of value
Subjective
Inter-subjectiveObjective
Summary of relationship of values
• Awareness of benefits of all three values• Awareness of drawbacks of each individual
value• Experience variation in your engagement with
information • Understand when different forms of value
should be prioritised or used in conjunction• Application to the research process