Innovative Design & Manufacturing Research Centre

University of Bath

“World-leading research in engineering design and manufacture.”

IdMRC Social Research MethodsAutumn Lecture-Workshop Series

Science

• Aim?• When is knowledge scientific knowledge?

• Criteria?• Knowledge sources?• When is research scientific research?

Henri Christiaans

Science

• RealismWhat we observe is real

• InstrumentalismWhat we observe doesn’t need to be real

• Social constructivismTheories only get meaning through social and political context

What is Knowledge?

• Justified true belief (Plato’s Theaetetus)• The Greeks classify knowledge into 2 types:• Doxa (believed to be true)• Episteme (known to be true)

• Doxa Epistime• Through Scientific process of inquiry

• How do we know what we know?• Define knowledge alternatively• Supported by evidence (usually empirical)• Conceive knowledge claims in a probabilistic sense

• Knowledge is a matter of societal acceptance

How is Knowledge Acquired?

• Role of science, where science is a convention, related to societal norms, expectations, values, etc.

• Thus, is science equals any scholarly attempt at acquiring knowledge

• Science requires conventions to be followed

How is Knowledge Acquired?

• Role of science, where science is a convention, related to societal norms, expectations, values, etc.

• Thus, is science equals any scholarly attempt at acquiring knowledge

• Science requires conventions to be followed

Knowledge in design

• Implicit prioritisation of the (language-based mode of) propositional knowledge (justified true beliefs) seems to exclude certain kinds or formats of knowledge associated with practice, which are often called practical, experiential, personal, or tacit knowledge and which evade verbal articulation.

Knowledge sources

• Observation• Experiments/measurements

• The Reason• Mathematics/logical reasoning

• Intuition• Authority• (Divine) Revelation

Science based on empirism

Empirism:Knowledge derived from how the world is experienced. Scientific statements are controlled by and derived from our experiences and observations. en

Scientific theories developed and tested by experiments and observations through empirical methods

Questions to be asked

1. Which methods do we plan to use?

2. Which methodology defines the use of methods?

3. Which theoretical perspective do we start from in order to apply the right methodology?

4. Which epistemology feeds this theoretical perspective?

Ontology

1. A systematic account of Existence. Nature of the world around us.

2. (From philosophy) An explicit formal specification of how to represent the objects, concepts and other entities that are assumed to exist in some area of interest and the relationships that hold among them.

3. The hierarchical structuring of knowledge about things by subcategorising them according to their essential (or at least relevant and/or cognitive) qualities.

Epistemology and ontology

The way of understanding and interpreting how we know what we know.

Particular methodologies tend to entail (subscribe to) particular epistemologies and, in their turn, particular forms of ontology

Ontology in Computing Terms

• For AI systems, what "exists" is that which can be represented.

• We can describe the ontology of a program by defining a set of representational terms. Definitions associate the names of entities in the universe of discourse (e.g. classes, relations, functions or other objects) with human-readable text describing what the names mean, and formal axioms that constrain the interpretation and well-formed use of these terms. Formally, an ontology is the statement of a logical theory.

• A set of agents that share the same ontology will be able to communicate about a domain of discourse without necessarily operating on a globally shared theory. The idea of ontological commitment is based on the Knowledge-Level perspective.

Epistemology

• From the Greek words episteme (knowledge) and logos (word/speech) is the branch of philosophy that deals with the nature, origin and scope of knowledge.

• Refers to our theory of knowledge, in particular, how we acquire knowledge (Hirschheim, 1992).

Research background

Epistemology

objectivism

subjectivism

Theoretical perspectivepositivism

Interpretativism

symbolic interactionism

phenomenology

hermeneutics

feminism

(post)modernism

Social-constructivism

Methodology

experimentaldescriptivesurveyethnographyheuristicaction researchdiscourse anal.evaluation

Methods

scalingquestionnairesobservationinterviewfocus groupcase studynarrativesethnographicstat analysisdata reductioncognitive mappinginterpretative methdocument analysiscontent analysisconversation anal.

Crotty, 1998

Research background

Epistemology

objectivism

subjectivism

Theoretical perspectivepositivism

Interpretativism

symbolic interactionism

phenomenology

hermeneutics

feminism

(post)modernism

Social-constructivism

Methodology

experimentaldescriptivesurveyethnographyheuristicaction researchdiscourse anal.evaluation

Methods

scalingquestionnairesobservationinterviewfocus groupcase studynarrativesethnographicstat analysisdata reductioncognitive mappinginterpretative methdocument analysiscontent analysisconversation anal.

Theoretical perspective

Philosophical point of view which feeds the methodology and offers a context for the process and the logics, and gives our criteria a basis.

Cultural differences play a role

Research background

Epistemology

objectivism

subjectivism

Theoretical perspectivepositivism

Interpretivism

symbolic interactionism

phenomenology

hermeneutics

feminism

(post)modernism

Social-constructivism

Methodology

experimentaldescriptivesurveyethnographyheuristicaction researchdiscourse anal.evaluation

Methods

scalingquestionnairesobservationinterviewfocus groupcase studynarrativesethnographicstatistical analysisdata reductioncognitive mappinginterpretative methdocument analysiscontent analysisconversation anal.

Crotty, 1998

Three Main Epistemologies

Positivist Interpretivist Critical

Interpretivism

Interpretivism rests upon idealism:

•the world is interpreted through the mind; e.g., classificatory schemes of species;•the social world cannot be described without investigating how people use language and symbols to construct what social practices; i.e., understand their experience;•the social world becomes the creation of the purposeful actions of conscious agents; and•no social explanation was complete unless it could adequately describe the role of meanings in human actions•Actions are not governed by discrete patterns of cause and effect (as in positivism), but by rules that social actors use to interpret the world

Positivist Science

• 5 Pillars• Unity of scientific method• Causal Relationships • Empiricism• Science and its process is Value-Free• Foundation of science is based on logic and maths

Ontology of Positivism

• Realism• Universe comprised of objectively given, immutable

objects and structures, existing as empirical entities, on their own, independent of the observer’s appreciation of them.

• Contrasts with relativism or instrumentalism, where reality is a subjective construction of the mind, thus varying with different languages and cultures.

• While hugely successful in physical sciences, it is not as successful for social science.

Anti-Positivism

• Latter part of 19th century• Man as an actor could not be studied through the methods of

natural sciences that focus on establishing general laws. In the cultural sphere man is free (Burrell and Morgan, 1979)

Post-Positivism

• Based on the concept of critical realism, that there is a real world out there independent of our perception of it and that the objective of science is to try and understand it,

• combined with triangulation, i.e., the recognition that observations and measurements are inherently imperfect and hence the need to measure phenomena in many ways.

• The post-positivist epistemology regards the acquisition of knowledge as a process that is more than mere deduction. Knowledge is acquired through both deduction and induction.

10-04-23

Objective Analysis

design Problem

design Solution

Designer

objective Analysis

Rational Solving Problem Paradigm

Design Task(= problem +

situation+ teime)

design Solution

Designer

subjective Interpretation

Reflection in Action Paradigm

POSITIVISM PHENOMENOLOGY

Constructivist Root

Rationalist Root

Simon versus Schon

Methodology

Our strategy and action plans, the design process which defines what specific methods we will choose

Research background

Epistemology

objectivism

subjectivism

Theoretical perspectivepositivism

Interpretativism

symbolic interactionism

phenomenology

hermeneutics

feminism

(post)modernism

Social-constructivism

Methodology

experimentaldescriptivesurveyethnographyheuristicaction researchdiscourse anal.evaluation

Methods

scalingquestionnairesobservationinterviewfocus groupcase studynarrativesethnographicstatistical analysisdata reductioncognitive mappinginterpretative methdocument analysiscontent analysisconversation anal.

Crotty, 1998

Types of Research

Analytical Historical PhilosophicalLiterature study Meta-analysis

-------------------------------------------------------------------------------------------------------Descriptive Survey (questionnaire, interview)

Case study Task analysisDocument analysis Correlation anal.Observation Etnographics

-------------------------------------------------------------------------------------------------------Explorative Survey Correlational

Case study Experimental-------------------------------------------------------------------------------------------------------Experimental Pre-experimental

True-experimentalQuasi-experimental

-------------------------------------------------------------------------------------------------------

Ethnographics

Types of research methods

idiographic nomothetic

rational

empirical

unbiased

participatory

qualitative

quantitative

inductive

deductive

prescriptive

descriptive

knowledge problem

Fundamental Research: the Empirical cycle

‘t Hart c.s.

testing

deductionprediction

theory

generalising

modelling

Explaining/interpretin

g

modelling

testing

evaluating

describing/

interpreting

evaluation

specifying

inductionhypotheses

Practice oriented Research: The regulative cycle

problem from practice

evaluation

intervention

plan(problem solving)

generalising

modelling

designing

deciding

process evaluation

describing/

interpreting

action-process supportingobserving

evaluating

diagnosis

‘t Hart c.s.

Method

The technique to gather data, related to the research question.

Research background

Epistemology

objectivism

subjectivism

Theoretical perspectivepositivism

Interpretativism

symbolic interactionism

phenomenology

hermeneutics

feminism

(post)modernism

Methodology

experimentaldescriptivesurveyethnographyheuristicaction researchdiscourse anal.evaluation

Methods

scalingquestionnairesobservationinterviewfocus groupcase studynarrativesethnographicstat analysisdata reductioncognitive mappinginterpretative methdocument analysiscontent analysisconversation anal.

Crotty, 1998

35

Qualitative Positivist Research versus Non-Qualitative Positivist Research

QPR Methods Non-QPR Methods

Field experiment Math Modeling (analytical modeling)

Lab experiment Group feedback

Free simulation experiment Participative research

Experimental simulation Case study

Adaptive experiment Philosophical research

Field study  

Opinion research  

Archival research  

Table 1. QPR versus Non-QPR Methods (Click on the method for its definition)

Type of Research, General Research Approaches, Data Collection Techniques, & Data Analysis Techniques

Research-Led

Design-Led

ParticipatoryDesign

generative toolsDesign

and Emotion

Critical Design

User-centered Design

contextual

enquiryLead-user inovation

appliedethnography

Usability testing

Human factors and ergonomics

Dutch/Scandinavia

n design

Research-Led

Part

icip

ato

ry m

ind

set

Exp

ert

min

dset

Probes

Sanders, 2002

Design-Led

Research-Led

Design-Led

ParticipatoryDesign

generative toolsDesign

and Emotion

Critical Design

User-centered Design

contextual

enquiryLead-user inovation

appliedethnography

Usability testing

Human factors and ergonomics

Dutch/Scandinavia

n design

Research-Led

Part

icip

ato

ry m

ind

set

Exp

ert

min

dset

Probes

Sanders, 2002

Design-Led

Research background

Epistemology

objectivism

subjectivism

Theoretical perspectivepositivism

Interpretativism

symbolic interactionism

phenomenology

hermeneutics

feminism

(post)modernism

Methodology

experimentaldescriptivesurveyethnographyheuristicaction researchdiscourse anal.evaluation

Methods

scalingquestionnairesobservationinterviewfocus groupcase studynarrativesethnographicstatistic. analysisdata reductioncognitive mappinginterpretative methdocument analysiscontent analysisconversation anal.

Crotty, 1998

Definitions

• ‘Research’ = the systematic inquiry to the end of gaining new knowledge

• a ‘researcher’ = a person who pursues research (e.g., in design).

• Practice’ = professional practice (e.g., in design) or to processes usually used in professional practice to produce professional work for any purpose other than the (deliberate) acquisition of knowledge.

• ‘Practitioner’ = anyone who works in professional practice.

Process (design methodology)

product people

Design Knowledge

designers

Design knowledge

• Design knowledge resides firstly in people: in designers especially. Therefore, we study human ability - of how people design. This suggests, for example, empirical studies of design behaviour, but it also includes theoretical deliberation and reflection on the nature of design ability. It also relates strongly to considerations of how people learn to design

Design knowledge

• Design knowledge resides firstly in people: in designers especially. Therefore, we study of human ability - of how people design. This suggests, for example, empirical studies of design behaviour, but it also includes theoretical deliberation and reflection on the nature of design ability. It also relates strongly to considerations of how people learn to design.

• Design knowledge resides secondly in processes: in the tactics and strategies of designing. A major area of design research is methodology: the study of the processes of design, and the development and application of techniques which aid the designer.

Design knowledge

• Design knowledge resides firstly in people: in designers especially. Therefore, we study of human ability - of how people design. This suggests, for example, empirical studies of design behaviour, but it also includes theoretical deliberation and reflection on the nature of design ability. It also relates strongly to considerations of how people learn to design

• Design knowledge resides secondly in processes: in the tactics and strategies of designing. A major area of design research is methodology: the study of the processes of design, and the development and application of techniques which aid the designer.

• The product dimension asks for forms and materials, and finishes with the embodiment of design attributes: both the intentional world (teleological and functional –wishes and needs–) in relation with the principal, partial and elementary function and the man’s connection with the systemic formal and material part (structure, organization, parts and connections).

Scientific Interpretive

Theoretical perspective

Scientific, usually based on physics Interpretive, focusing on individuals’ experiences, their construction of understanding, perceptions and interpretation of reality. Often centres on individual creativity and subjective perceptions relating to being creative.

Focus Empirical realities of the design processes, design objects, design brief and contexts.

The core concept of ‘design’ is defined in terms of these activities.

Experiences of designers and other design constituents. Tries to identify form of internal creative design activities from observation of externalities.

Typically defines design in terms of creativity, art, individual genius and socio-cultural influences

Design Research

Scientific Interpretive

View of Design Design is a process.

May or may not include creativity.

Intuitive, involving hidden aspects of human subjective thinking and affective activity.

View of creativity

‘Something, or a specification for something, is “created”’.

Creation can be achieved mechanically, by automation or intuitively.

Human internal activity that results in ideas for new, unusual, highly valued, never before created things, emerging ‘magically’ from the genius of designers.

Focus on ‘individual creativity’ attributed to specific ‘designers’ and socio-cultural influences.

Design Research

Scientific Interpretive

Data collection Similar to physics and natural sciences.

Drawn from various qualitative traditions, e.g. anthropology, ethnography, history, includes self reporting data collection.

Analysis methods

Similar to physics and natural sciences.

Drawn from various qualitative traditions, e.g. anthropology, ethnography, history, includes reflective analysis of self reports and self perception.

Knowledge focus

Discipline specific empirical information (along with)

elicited representations of tacit information and data that designers use.

Tacit and embodied skills of designers and users.

Culturally-determined knowledge.

Embedded meanings.

Design Research

Scientific Interpretive

Strengths 1. Techniques to investigate phenomena in ways that are transparent, repeatable, testable, and verifiable.

2. Research methods are expressed in a formal language that enables precise critique of the data collection techniques, methods of analysis, processes that lead to abstractions, and the theory abstractions and conclusions.

3. Correspondence between characteristics of phenomena and the formal defined symbolic language of concepts and operations in which mathematically theories and representations of the phenomena are expressed.

1. Focus on human considerations, such as the human creative aspects of design, and how users and other interpret designed outcomes.

2. Interpretive methods give space for designers and users to explain, in their own words, and from their own perspectives, how they design and use designed outcomes and how they communicate with others about designs.

3. Interpretive methods also allow exploration of opinions of users about cultural aspects of particular designs.

4. The interpretive approach can be extended to draw strength from the use of large data sets by which correlations and measures of confidence in them can be established between individuals’ ‘stories’ and the phenomena being studied.

Scientific Interpretive

Weaknesses Scientific empirical method does not adequately address human subjective, interpretive and experiential phenomena except via physiological substrates.

Main weakness is lack of reliability of individuals’ evidence, perceptions and interpretations i.e. lack of correlation between what people say and reality.

Evidence of this problem in studies of e.g. witness testimony, reliability of memory, relationships between reported thoughts and physiological evidence, influence of subconscious ‘thinking’, mental illusions and delusions in normal people.

‘False consciousness’: people’s representations of themselves are inaccurate or simply wrong.

Extends to individuals descriptions of processes, and the social activities that they undertake.

Scientific Interpretive

Contradictions There is an incompatibility between scientific modelling of design process and inclusion of a process element ‘create a new solution’ as a subjective human activity.

Claims that all sub-fields of design are incommensurate as they use different knowledge (and that the broader field of design is fundamentally fragmented) is at odds with scientific representation of designers working across disciplines and in multi-cross- and trans-disciplinary teams.

There is tension between interpretive approaches that focus on experiential subjective phenomenological aspects of human creative design activity and the frequent shift of emphasis onto aspects of design and creative activity that are more accessible empirically using a physical science approach.

There is an epistemological inconsistency in claims that Design exists of itself as a phenomena capable of creative agency and action.

Love’s proposal:

a unified basis for design theory bridging these two incompatible approaches.

Advantages

• It provides a coherent epistemological basis for new theories

• It recasts prior research and theory within a justified integrated framework with a clear epistemology and ontology.

• This in turn provides the basis for developing a design field.

Design Research

• Designs (i.e. the specification for creating or doing something) • Designed outcomes (after they are manufactured/actualised) • Design activity • Design processes • The skills of designers • The role of design activity • Cognitive design processes • Behaviour of designers as individuals and in social groups • Combinations of the above

Foundations for a unified basis

Epistemologies Assumptions for Qualitative and Quantitative Research

Researcher tests or verifies a theory

Researcher tests hypothesesor research questions

Researcher defines and operationalizesvariables derived from the theory

Researcher measures or observesvariables using an instrument

to obtain scores

Creswell, 2003

Deductive logic of quantitative research

Generalizations or theoriesto past experiences and literature

Researcher looks for broad patterns.Generalizations or Theories from

Themes or Categories

Researcher analyze data toform themes or categories

Researcher asks open-ended questionsof participants or records field-notes

Researcher gathers informatione.g. interviews, observations Creswell, 2003

Inductive logic of qualitative research

Qualitative vs Quantitative

Purpose

Perspective

Procedures

Quantitative

General Laws

Test Hypotheses

Predict behavior

Outsider-Objective

Structured

formal measures

probability samples

statistical analysis

Qualitative

Unique/Individual case

Understanding

Meanings/Intentions

Insider-Subjective

Unstructured

open ended measures

judgement samples

interpretation of data

Qualitative Research

Triangulation By using several data collecting methods – field notes, interviews, narratives – a complete picture of the phenomenon can be provided

Interpretation:observation of species

• -

-

a

Interpretation

• -

-

a b

c d

Interpretation:observation of discourse

J (reading) pack is firmly attached to the bike positioning of the backpack was alright fact that the centre of gravity of the backpack is placed rather far to the back of the bike (inaudible)I do we have any … em...J there's a problem with potholes .. the backpack tends to slide up and down which adversely influences stability I guess when you hit bumpsI isn't that in the negative?J mm yeah well the product was considered ugly well that's solvable (laughter) we can fix that one if nothing else ... it takes a while to get used to cycling with this weight; mistakes are made attaching the fastening device to the bike so it has to be easy to attachK with only one yeah gotta be fool proof so that's part of ourJ yeah that should be in our specK functional spec

The role of interpretation

Gap between objects and our representations, in 3 forms ('methodological horrors', Woolgar '88):

1. Indexicality2. Inconcludability3. Reflexivity

THANK YOU!