research methods

LECTURE THREE

Michael Poku-Boansi, Ph.D.

September, 2012

RESEARCH DESIGN FRAMEWORKS

TODAY2

Research designs are concerned with turning the research question into a testing/testable project.

Any research design deals with at least four (4) problems:

INTRODUCTION

TODAY3

- What questions to study;

- What data are relevant;

- What data should you collect; and

- How to analyze the results.

INTRODUCTION

TODAY4

Every research design has its positive and negative sides.

Research design can be divided into fixed and flexible designs or in other instances, they are referred to as “Quantitative” and ‘Qualitative” research designs.

INTRODUCTION

TODAY5

Fixed designs do not need to be quantitative and flexible design need not be qualitative.

In fixed designs the design of the study is fixed before the main stage of data collection takes place.

INTRODUCTION

TODAY6

Fixed designs are normally theory driven; otherwise it is impossible to know in advance which variables need to be controlled and measured (often these variables are quantitative).

Flexible designs allow for more freedom during data collection. One reason for using the type of design is that the variable of interest is not quantitatively measurable (e.g. culture, decision making process, plan approval process, Case Studies etc)

INTRODUCTION

TODAY7

In other cases, theory might not be available before one starts the research.

So you can see that there are several classifications and models of research design but we will deal with the following three classical ones:

INTRODUCTION

TODAY8

- Quantitative – Qualitative Design;

- Classical Experimental Design; and

- Cross – Sectional Design

INTRODUCTION

TODAY9

Introduction

There are several classifications and models but will deal with three that are mostly used in planning:

- Classical Experimental Design;

- Cross-Section Design; and

- Quantitative - Qualitative Design.

TYPICAL RESEARCH DESIGNS IN PLANNING

TODAY10

Key Elements

a)Two comparable groups or situations: experimental and control groups

b)The experiment group is exposed to an independent variable (treatment or change agent); control group is not.

CLASSICAL EXPERIMENTAL DESIGN

TODAY11

c) Cases are randomly selected in both experimental and control groups

d) Effect of the independent variable on the dependent variable is measured on 2 occasions.


TODAY12

a) Pre-test - before introduction of change agent/intervention

b) Post-test - after introduction or injection of change/intervention .


TODAY13

e)Differences between the two measurements are compared and inferences made under some critical assumptions. What are they?

f) If the difference in the experimental group is significantly larger than the control group, it is inferred that the independent variable is causally related to the dependent variable


TODAY14

Groups Pre-test Post-test Difference

Experimental Qi x Q2 Q2-Q1 = de Control Q3 Q4 Q4-Q3 =

dc

DIAGRAMMATIC DESIGN

TODAY15

Why Experimental Designs?

Experiments are more associated with natural sciences - physical and biological/natural sciences.

Why experimental designs in Social Sciences?

TODAY16

Two (2) reasons:

a)Helps us to understand the logic of all research designs, the model against which we can evaluate other designs.

b)Allows the investigator to draw causal inferences, and establish whether or not the independent variables

caused changes in the dependent variable.

TODAY17

Experimental design is used less frequently by social scientists primarily because of its rigid structure which often cannot easily be adapted to the social sciences

KEY COMPONENTS OF EXPERIMENTAL DESIGN

18

a) Comparison – Normal

- Necessary to establish that two variables are correlated or

causally related. - E.g. compare behaviour of two groups of youth

who smoke and those who do not or socio economic

conditions along a rehabilitated road corridor and a deteriorated

road corridor.

- Both internal (time-wise) and external comparison.


19

b) Manipulation

- Helps to establish time-order of events

- Manipulate X and Y is disturbed or also changed.

- Can then say that change in Y is caused by change in X


20

b) Manipulation

- E.g. if drug abuse among the youth causes

indiscipline, then it follows that the use of drugs precedes the

expression of indecent behaviour.

- It is important to note that change occurs after the

activation or injection of independent variable.


21

c) Control - Internal Validity

- Third criterion of causality requires that rival explanations

are ruled out.

- Enables us to determine that the observed variation is non-

spurious.

- Non-spurious relation is a relationship between two (2)

variables that cannot be explained by a third variable.


22

- So control means → the operation that enables the researcher to rule out other factors as rival explanations of the observed association between variables.

→ this issue of internal validity

- To establish internal validity, the researcher must answer

the question that the independent variable caused the

changes in dependent variable.


23

- Factors that may affect Internal Validity are:

a) Extrinsic factors - those occurring prior to the research

operation; and

b) Intrinsic factors - affect the results during the study

period.


24

d) Generalisation: External Validity.

- The extent to which the research findings can be

generalised to larger populations and applied to different

social and political settings.


25

- Two (2) conditions must be satisfied

a) Representativeness of the sample

b) Minimisation of reactive elements - reducing artificial disturbances in the natural setting.


26

Related research approach

a)Explanatory research - desire to establish causality - e.g. epidemiology - causes of morbidity; infant mortality.

b) Impact Studies:

- Establishing the degree of change resulting from the injection of change variable


27

- Trying to establish the explanatory variables ‘with’ and ‘without’ and ‘Before’ and ‘After’

E.g. Agric experiments, Micro-finance, community participation

CROSS - SECTIONAL DESIGNS (CSD)28

Introduction

a)The most predominant design used in social sciences. - Often identified with survey research - a method of data collection and analysis common in social science studies.

- Research mostly aimed at describing the pattern of relations before any attempt at casual inference is made


b) Comparison with Experimental Design.

E.g. The effect of gender/literacy/alcoholism on communal

violence.

- Researcher cannot a) Manipulate the independent variable

b) Cannot incorporate control (time element of change is

not possible)


c)To overcome methodological limitations associated with CSDs, the researcher uses statistical analysis to approximate some of the operations that are in-built in Experimental design.

- Statistical analysis is used to show the interrelationship

- Using statistics to organise describe, and summarise observations.


→ E.g. Cross-tabulation or bivariate percentage analysis (20% women, 40% men are involved in communal violence).

- Though this statistical analysis will give us the behaviour of men and women, we cannot:

a) Establish causality between the variables

b) We cannot explain why?


- When Cross - Sectional Designs (CSD) are used, explanatory factors are statistically controlled unlike experimental design.

- In Cross - Sectional Designs (CSD), multivariate methods of statistical analysis are used as alternatives to experimental methods of control and drawing of inferences e.g. elaboration by cross-tabulation, multiple regression etc


Main advantage:

a)can be carried out in natural settings and permit researchers to employ random probability samples.

b) This allows statistical inferences to broader populations and permits generalisation of findings to real life situations and thereby increasing external validity.


Disadvantage:

- Lack of adequate control over rival explanations - internal

validity. Related Research Approaches

Panel (Longitudinal) Research

- A vigorous solutions to the time dilemma or problem in

cross-sectional studies


Designed to collect time series data from the same sample - collect data before injection of the change agent and then after

E.g. yearly E.g. snapshot and Video - A snapshot is like single case study whiles a Panel Study is like a video


Advantages

a)A good way of studying trends - behaviour, population etc

b) Measures changes with high degree precision - Better than changing samples


Problems

a)The likelihood of flagging commitment of respondents - Affects representativeness of sample

b) Panel conditioning - Respondents giving a given set of answers to please researcher.


Example: Case Study

Definition:

- It’s an empirical enquiry that allows the researcher to investigate and understand the dynamics of a particular system


- It has 3 attributes:

a) Investigates a contemporary phenomenon within its real- life context.

b) When the boundaries between phenomenon and context are not clearly evident. c) Multiple sources of evidence are used.


Case Study is preferred when:

a) “how” and “why” questions are being asked.

b) the researcher has little control over events

c) the focus is on contemporary phenomenon within a real life context.


Relevant to Planning:

- It enables the researcher to learn from practice in order to inform the theory on which that practice is based.


Choice of Appropriate Design - Selection of appropriate research design or strategy depends on 6 conditions:

a) The type of research question.

b) The control an investigator has over actual behavioural events


c) The focus on contemporary as opposed to historical phenomena

d) Purpose of research

e) Time available

f) Type of data involved.

QUANTITATIVE (Q1) AND QUALITATIVE (Q2) APPROACHES

44

General Definitional Difference

Qi - Behaviour can be explained by objective facts

Q2 - There are multiple realities that are socially defined.


45

The 2 are further compared in four ways:

- Assumptions of the world

Qi - There are social facts with objective reality.

Q2 - Reality is socially constructed through individual or collective definition of the situation


46

Purpose

Qi - Explains the causes of change of social facts through objective measurements and quantitative analysis

Q2 - Concerned with understanding of social phenomenon from actors perspectives through participation


47

Approach-focus of research

Qi - Often employs experimental and correlational designs to reduce error - people are objects of investigation.

Q2 - Ethnographic description of process


48

Choice of Appropriate Design

Researchers role

Qi - Detached to avoid bias

Q2 - Immersed in the phenomenon of interest

PRACTICAL LEVEL 49

Practical level

i. Nature of variables

Qi - Quantifiable e.g. income, output etc

Q2 - Perceptional variables reflecting attitudes, preferences and priorities.

PRACTICAL LEVEL 50

ii. Interviews format

Qi - Structured, formal, pre-designed questionnaire

Q2 - Open-end, semi-structured, interactive

PRACTICAL LEVEL 51

iii. Sampling

Qi - Probability Sampling

Q2 - Purposive Sampling

MEASUREMENTS52

Introduction

The concept of ISOMORPHISM: concerns how measurement instruments relate to the reality being measured. The researcher is always confronted with the need to:

a) Search for an already developed measure and reported in literature;

MEASUREMENTS53

b)Develop a new or original measure to convert empirical observations in relation to the research problem; and

C) Provide evidence that the measures used are valid and reliable.

MEASUREMENTS54

Nature of Measurement

Definition: “measurement is a procedure in which a researcher assigns numerals - numbers or other symbols - to empirical properties (or variables) according to rules”

Example: Job applicant develops a rating system to help him to select the most appropriate job offer

MEASUREMENTS55

Nature of Measurement

→ Scale: 1, 2, 3, 4

Criteria: Job satisfaction, Salary, Location Jobs obtained:

Teaching, NGO, public service.

MEASUREMENTS56

Basic concepts in Measurements

The basic concepts used in defining measurement include Numerals, Assignments and Rules.

a) Numerals

- A numeral has no quantitative meaning unless you give it

such a meaning e.g. I, II, III.

MEASUREMENTS57

- It can be used to identify phenomena, objects or persons and thus can be used to designate months, books, streets, etc.

- Numerals that are given quantitative meaning become numbers that the researchers can use to explain, describe and predict phenomenon.

MEASUREMENTS58

b)Assignment - in definition of measurement, it means “mapping”. Numerals or numbers are mapped onto objects or events:

MEASUREMENTS59

MEASUREMENTS60

c)Rules - It specifies the procedure a researcher used to assign numerals or numbers to objects or events.

E.g. Assign numerals 1 to 5 to a level of impact or satisfaction with a Job.

MEASUREMENTS61

→ Rules are the most significant component of the measurement procedure because they determine the quality of measurement. → Poor rules make measurements meaningless.

→ Measurement is assignment of numerals or numbers to objects, events or variables according to rules.

MEASUREMENTS62

Levels of Measurement

- Scientist distinguish between different ways of measuring: Levels or scale of measurement

- 4 levels or scales or “Measurements”

MEASUREMENTS63

a)Nominal Scale

- It is the lowest and simplest level.

- Numbers or other symbols are used to classify objects or observations into a number of categories.

- These numbers only constitute a nominal or classificatory scale.

MEASUREMENTS64

- E.g. - classification of population into sex (1,2), religions (1, 2, 3), ethnicity (1, 2, 3, 5, 5), level of education.

- Mathematically, the basic property of the nominal scale is that the properties of objects in one category are designated as identical for all its cases..

MEASUREMENTS65b) Ordinal scale

- Unlike nominal, it brings in the question of relationship between variables (order).

E.g. higher, greater, lower etc.

- It allows for ranking of variables in either ascending or descending. E.g. Highly Favourable, Favourable, Not Favourable, etc..

MEASUREMENTS66

c) Interval Scale

- Apart from being able to rank a set of observations, it also gives the exact distance between each of the observations and the distance is constant.

- It is characterised by a common and constant unit of measurement that assigns real numbers to all (pairs of) the objects in the ordered set.

MEASUREMENTS67

- Income, exams results, weight, output, population size, age, mileage etc.

MEASUREMENTS68d) Ratio Scale

- Variables that have natural Zero-point can be measured on a

ratio scale. E.g. freezing point of water

- Examples are weight, time and length. Interval scale - the difference of amount measured

is applied to an arbitrary point → whereas with Ratio, it is

applied to an obsolete zero-point

MEASUREMENTS69

- General Principle: Properties that can be measured at higher level can also be measured at lower levels but not vice versa.

- E.g. - If a variable can be measured at Ratio level, it can also be measured at nominal level.

MEASUREMENTS70

Measurement Error

Errors are associated with measurements

- Definition: differences in measurement scores that are due to anything other than real differences.

MEASUREMENTS71

Reasons

1.The score obtained maybe influenced by an associated attribute, an attribute the researcher did not intend to measure. E.g. unfairness in judgement about an event - football

2. Differences in temporal situation - e.g. health, mood of the person.

MEASUREMENTS72

Reasons

3.Differences in the settings in which the measure is used e.g. – age, probability, gender.

4. Difference in the administration of the measuring instrument – e.g. poor lighting, tiredness of interviewer etc.

MEASUREMENTS73

Reasons

5.Difference in processing - wrong or inconsistent coding of responses

6.When different people interpret the measuring instrument differently.

MEASUREMENTS74

Validity

- This is concerned with: “Am I measuring what 1 intend to measure?”.

MEASUREMENTS75

- This comes about because many measurements in the Social Science are indirect. Under this situation, researchers are never fully certain that they are measuring the variable for which they designed their measurement operation. E.g. does voter turnover really measure democracy? Community preference - various types e.g. content relevance to characteristics Empirical: Prediction of results and same Construct etc: Relation to thematic framework

MEASUREMENTS76

Reliability

Reliability is a central concern to the social Scientist because the instruments used are rarely valid - i.e. problem free.

It refers to the extent to which a measuring instrument contains variable errors - that is errors that appear inconsistent from observation to observation during any one measurement attempt or that vary each time measurement is taken.

MEASUREMENTS77

E.g. if you use a ruler to measure 2 points in time and there is a difference, then the ruler contains a variable error.

Errors are common because the variables in Social Science are indirect or qualitative. - e.g. momentary distraction, ambiguous construction etc.

MEASUREMENTS78

Each measurement consists of two Parts: true or error components

Reliability: the ratio of the true-score variance to

the total variable in the score as measured.

TYPES OF SCALE 791.Rating

According to Gyedu et al (1999), the rating scale is a device used in evaluating products, attitude or other characteristics of variables. Mugenda and Mugenda (1999) also define it as a scale used to measure perception, attitude, values and behaviour.

TYPES OF SCALE 80

Con’t

Rating scales are type of scales that consists of numbers and descriptions which are use to rate or rank the subjectivity and intangible components in research. Rating scales are limited by characteristics of human memory, sensitivity and accuracy of observation. Essentially, there are three types of rating scales, namely numeral scales, graphic scales and forced choice scale.

TYPES OF SCALE 81

i Numeral Rating Scale

This method measure characteristic by assigning numbers to the specific rating categories. For example rating a teacher’s performance along a scale of 1 to 10. It is very easy to construct and the length of the scale or number of points along the scale is an arbitrary decision.

TYPES OF SCALE 82

Con’t

A disadvantage of this scale is that, individual respondents have different frame for referencing which may affect rater reliability or consistency, that is, a rating of seven of an object by a person may be equal to a rating of five by another person on the same scale.

TYPES OF SCALE 83

ii Graphic Rating Scale

Is similar to numerical scale except that the scale itself contains words rather than numbers. It has an objective and a straight line with rating categories positioned along the line.

TYPES OF SCALE 84

iii Forced Choice Rating Scale

It is a scale that presents the respondents with a series of choices and requires him to choose one over the others. It is much more complex to develop and score than the numerical and graphical scale. Example can be as follows:

TYPES OF SCALE 85

The teacher is

{ } Always on time for class

{ } Pleasant in class

{ } Very sincere when talking with students

{ } Well read

TYPES OF SCALE 86

This method provides an overall ranking to the factor being studied without asking the respondents to rank order. This method has two advantages:

It overcomes the rater’s tendency to leniency because he is forced to make some kind of judgements; and

The distribution of ratings is much more realistic than some other rating scale distribution.

TYPES OF SCALE 87

2. Thurstone Scale

Thurstone is one of the first and most productive scaling theorists. He actually invented three different methods for scaling a uni-dimensional scale: the method of equal appearing intervals, the method of successive intervals and the method of paired comparisons. The three methods differed in how the scale values for items were constructed, but in all three cases, the resulting scale was rated the same way by respondents.

TYPES OF SCALE 88It was used in measuring core attitude when you have multiple

dimensions or concerns around that attitude.

In Thurstone scaling, the researcher would obtain a panel of judges (say 100 of them) and then dream up every conceivable question you can ask about a phenomenon.

By administering the questionnaire to the panel, the researcher can analyse inter-item agreement among judges and then even use the discrimination index to weed out what are called the non-homogenous items.

It is based on the premise that, scaling is all about homogeneity, a term sometimes used as synonymous with being one-dimensional.

TYPES OF SCALE 89

3. Likert Scale

They were developed in 1932 as the familiar five-point bipolar response format most people are familiar with today. It is a rating scale used in measuring the strength of agreement with a clear statement.

Likert scaling is a one-dimensional scaling which usually ask people how they agree or disagree, approve or disapprove, believe to be true or false.

TYPES OF SCALE 90

4. Guttmann’s Scale

It was developed in the 1940s and is a technique of mixing questions up in the sequence they are asked so that respondents don’t see that several questions are related.

It is sometime known as cumulative scaling or scalogram analysis.

research methods

Data & Analytics

Transcript of research methods