Critical evaluation of research information

Laura Jeffrey Researcher Training

Course outline

• Importance of evaluation• Forms of value– Subjective• Cognitive bias

– Objective– Intersubjective

• Relationship between the forms of value

The need to evaluate information

• Much training is about directing you to the right information = searching and retrieval

• One of the distinctive elements of postgraduate research is that you have to be critical and reflect on what you find

• What defines your evaluative criteria?

Ecology of resources

• Term developed by Rosemary Luckin• Resources are interconnected and they evolve

e.g. natural resources• Information resources are transformed into

knowledge• Knowledge becomes a resource• Therefore prior knowledge shapes

what we go on to create

Role of the researcher

• In theory we can select almost any information to complete a task

• In practice we filter it by selecting resources we think most appropriate

• Motivation - affected by the learning we have already done

Other factors

• But, filtering is done for us BEFORE we get the chance to make a judgement

• People• Technologies• Cost• Skills• Copyright, IP

Value

• Filtering process = value judgement– By researcher– Made on their behalf

• What forms of value are there and how do they work together to create information literate researcher?

Subjective form of value

• Decisions we make– Is this what I want, do I need this, is it relevant?

• Privileges you as the researcher in the decision making process

• If we omit it we get groupthink (Janis, 1972) or battery cognition (Blaug, 2007)

• Importance therefore of asserting individual criticality

Cognitive bias

• SO… subjectivity is vulnerable to bias and hunches

• Concept of cognitive bias was developed in 1970s by Tversky and Kahneman

• Four main groups– Social– Memory– Probability/belief – Decision making

Social biases

• Ascribe positive or negative traits to self, individuals or groups

• Loading values or anticipating action based on prior experience or a bias against self, individuals or groups

• Academic impact: need to verify information and not rely on own views; important to remember when analysing human subjects

Memory biases

• How you perceive past events• False memory, positive memory, imbalanced

memory• E.g. A Photo, a Suggestion, a False Memory • Academic impact: importance of

accurate record keeping and note taking

Probability and belief

• To disregard or to pay too much attention to probability

• Academic impact: need to treat each research finding as distinct and to judge it in its own right

Decision making biases

• Influences on your decisions by own biases or those of a group

• Academic impact: need to be objective and consider all possible routes of enquiry and treat all research findings as valid until proved otherwise e.g. Semmelweis reflex

Cognitive biases

• On your table, group the forty cards into four piles of ten– Social – Memory – Probability– Decision

Objective form of value

• Scientific measures of validity or reliability• Exists so that subjective values don’t unduly

influence work • Omit this scheme of value and we risk

information (and knowledge formed from it) becoming counterknowledge (Thompson, 2008).

Intersubjective form of value

• Based on the shared values of a community e.g. morals, ethics, laws, economics

• Allows for discussion of scientific method as it can’t explain everything

• Acceptance in a community• If we omit this then values are relativist

Three forms of value

Subjective

Inter-subjectiveObjective

Summary of relationship of values

• Awareness of benefits of all three values• Awareness of drawbacks of each individual

value• Experience variation in your engagement with

information • Understand when different forms of value

should be prioritised or used in conjunction• Application to the research process