Fitting Research Design to Research Purpose

Research Purpose

• The overall purpose of most research is to investigate a predicted relationship between the occurance of some variation of one variable, A, and the occurance of variations of another variable, B, in the same setting.

• Variables may be – states of the physical or social environment (e.g.,

weather conditions, the number of people present in the situation),

– properties of a stimulus (e.g., the facial expression in a photograph, the content of a message), or

– characteristics of a person or a person’s behavior (e.g., mood state, degree of aggression).

• Relationships can be between – two environmental variables (e.g., the relationship

between variations in the coldness of the weather on the number of people who are in an outdoor setting),

– between an environmental or stimulus variable and an individual characteristic or trait (e.g., the relationship between the state of the weather and the average mood of people exposed to it),

– or between two characteristics of an individual (e.g., the relationship between mood and aggressiveness).

• To say that there is a relationship between two such variables means that if the state of one variable differs or changes, we can expect that the state of the other will also change or differ.

• So, for example, if we measure people’s mood on a sunny day and then again on a cloudy day and there is a difference in mood such that mood is more negative on the second occasion, then we can say we have shown a relationship between the state of the weather and individuals’ moods.

The nature of the relationship may be specified in terms of the form it will take, that is, what kind of changes in B will accompany particular changes in A and what the causal direction of the relationship will be. Directionality may be differentiated into three types.1. Unidirectional causation, in which changes in A are predicted

to produce subsequent changes in B, but changes in B are not expected to influence A

(e.g., increases in the temperature-humidity index are accompanied by an increase in aggressive responses of rats, but the degree of aggressiveness of rats does not affect weather conditions).

Source: Neumann, 2011)

2. Bidirectional causation, in which changes in A lead to changes in B and, in addition, changing B produces changes in A

(e.g., perceiving threat produces feelings of anxiety, and increasing anxiety enhances the perception of threat).

3. Noncausal covariation (or third-variable causation), in which changes in A are indirectly accompanied by changes in B because both A and B are determined by changes in a third variable, C

(e.g., birth rate and consumption of beef steak rise or fall with increases or decreases in the cost of living index).

Diagrams of Causal Relations

Positive relationship

Level of stress (financial, social, emotional, etc.) is positively associated with the likelihood that a couple will divorce.

Positive and negative relationship

Positive path relationship (MODERATOR, MEDIATOR…soon)

Level of stress is positively associated with the likelihood that a couple will divorce, but the amount of resources (financial, social, emotional, etc.) they possess is negatively associated with it.

Level of stress is positively associated with frequency of fighting by a couple, which is associated with the likelihood that the couple will divorce.

Complex relationships

Level of stress is positively associated with the likelihood that a couple will divorce and negatively associated with the likelihood that the couple will have emotionally well-adjusted children. In addition, the divorce process itself has a negative effect on the emotional adjustment of children.

Level of stress and amount of resources are negatively associated with each other (ie people who tend to have many resources are less likely to experience or better able to deal with stress). Level of stress is positively associated with the frequency of fighting by a couple, but the amount of resources is negatively associated with it. Amount of fighting is positively associated with the likelihood that a couple will divorce. Both fighting and divorce itself are negatively associated with the likelihood that the couple will have emotionally well-adjusted children.

Returning back to the simplest case: how can you justify the existence of this relationship? What about falsification?

Moderators and Mediators

• In addition to specifying the nature and direction of a causal relationship under study, it also is important to distinguish between two different types of “third variables” that can influence causal relationships—moderators and mediators

• Sometimes causal relationships can be either augmented or blocked by the presence or absence of factors that serve as moderator variables.

• To take another weather-related illustration, consider the causal relationship between exposure to sun and sunburn. Although there is a well-established cause–effect link here, it can be moderated by a number of factors.

• For instance, the relationship is much stronger for fair-skinned individuals than for dark-skinned persons. Thus, fair skin is a moderator variable that enhances the causal relationship between sun exposure and burning. However, this does not mean that the sun–sunburn relationship is spurious.

• The moderator variable (skin pigmentation) does not cause the effect in the absence of the independent variable (sun exposure). Other moderator variables can reduce or block a causal sequence.

• For instance, the use of effective suntan lotions literally “blocks” (or at least retards) the causal link between the sun’s ultraviolet rays and burning. Thus, a researcher who assesses the correlation between sun exposure and sunburn among a sample of fair-skinned people who never venture outdoors without a thick coat of 30 SPF sunblock would be ill-advised to conclude that the absence of correlation implied the absence of causation.

Moderator relationships can be represented notationally as follows:

It is important here to distinguish between third variables that serve as moderators and those that serve as mediators of a cause–effect relationship.

With moderator effects, the causal link is actually between X and Y, but the observed relationship between these two variables is qualified by levels of variable C, which either enhances or blocks the causal process. A mediational relation, on the other hand, is represented as follows:

In this case, the presence of C is necessary to complete the causal process that links X and Y. In effect, varying X causes variations in C, which, in turn, causes changes in Y.

To return to our weather examples, the effect of rain (X) on depression (Y) may be mediated by social factors (C). Rain (X) causes people to stay indoors or to hide behind big umbrellas, hence reducing social contact (C). Social isolation (C) may, in turn, produce depression (Y). However, rain may not be the only cause of social isolation. In this case, rain as an independent variable is a sufficient, but not necessary, cause in its link to depression. To demonstrate that X causes Y only if C occurs does not invalidate the claim that X and Y have a causal relationship; it only explicates the causal chain involved.

FORMS OF VALIDITY

• The research strategy should be be guided by considerations of two types of validity—internal and external validity

• Internal validity has to do with the certainty with which one can attribute a research outcome to the application of a treatment or manipulation that is under the rigid control of the researcher.

• Internal validity is about the extent to which causal inferences can legitimately be made about the nature of the relationship between the treatment and the outcome.

Structural Explanation

• A causal explanation (those that we have seen so far) says, B happens because A causes B. (positivist)

• A structural explanation says that B happens because positioned inside a larger structure that either blocks off or provides B openings to other areas in the structure. (critical)

• In a causal explanation, one or more factors may cause a response in other factors. This is like one ball that rolls and hits others, causing them to begin rolling.

• In structural explanation, in contrast, the logic operates within a larger structure.

• Three major types of theories that use structural explanation are

• Sequential theories• Network theories• Functional theories

Sequential Theory

• Emphasizes the order or sequence by which events occur

• identifies the necessary earlier steps and possible subsequent steps in an unfolding pattern of development across time.

• A sequential theory maps out an ordered set of stages.

• aka `Path Dependence`

• Comes in two forms1. Self reinforcing: once set into motion events,

events continue to operate on their own or propel later events in a direction that resist external factors. An initial trigger event constraints or places limits on the direction of a process. Once a process begins, inertia comes into play (classical example is QWERTY-designed to prevent primitive machines from jamming)

2. Reactive

• Comes in two forms1. Self reinforcing2. Reactive: It focuses on how each event responds to an

immediately preceding one. Thus instead of tracing a process back to its origins, it studies each step in the process to see how one influences the immediate next step. • The interest is in whether the moving sequence of events

transforms or reverses the flow of direction from the initial event.

• The path does not have to be unidirectional or linear: it can bend or even reverse course to negate its previous direction.

Examples to sequential theory

• Oesterle, Johnson, and Mortimer (2004) offer a sequential theory in their panel study on voluntarism among young people. The authors adopted a `life course` perspective in which `the meanings of roles and activities differ across life stage`. – The impact of an event at a specific phase of a

person’s life differs from the same event from happening in other phases, and the same event will shape events in later phases.

Oesterle, Johnson, and Mortimer (2004)

• DATA: They examined a panel data of student (15-16 years old) for 9 years.

• RESULT: People who worked or who were parenting full-time at an earlier stage (18-19 years old) were less likely to volunteer at a later stage (26-27 years old) than people whose major activity was to attend school full-time.

• CAUTION: What is the worth of this work? What does less likely means?

• Three major types of theories that use structural explanation are

• Sequential theories• Network theories• Functional theories

Network Theory

• Explains social relations in terms of placement in a network

• Explains by referring to relational positions (within the network of course) or its size, shape, type and existence of connections among positions, density of connections, centrality in a network, gaps etc.

• The positions (nodes) might be people, organizations, cities or even an heterogeneous collection of them

Examples

• Entwisle et.al. (2007) studied networks in villages in a region of Thailand.– They found that networks connecting people, through

kinship or other social ties varied by village: `Networks are sparse in some, dense in others; porous (transitive) in some, less so in others.

– The networks had many consequences for relations with nearby villages, for the economic activities in a village, for whether people migrated out of a village.

– Network structure shaped the flow of activities, degree of intravillage cooperation

Entwisle et.al. (2007)

• Solid lines indicate people related as brother or sister and dotted lines indicate those helping with the rice harvest.

• Households a,b,d,e work together• No direct family connection between a and d

or between b and e but they cooperate due to indirect ties

• More ties implies more cohesion

• Three major types of theories that use structural explanation are

• Sequential theories• Network theories• Functional theories

Functional Theory

• Uses the idea of systems with a set of mutually interdependent relations

• Various parts depend on another and in combination all parts function together as a whole.

• Emphasis is on how interdependent parts fit into and operate to sustain an overall system with specific parts serving complementary and specialized supporting roles for the whole

Example• Kalmijn (1991) explained a shift in the way that Americans select marriage

partners using a functional explanation. He relied on modernization theory which holds that the historical processes of modernization (industialization, urbanization, and secularization) shape societal development.– As part of Modernization people rely less on traditional ways of doing things.– Religious beliefs and social ties weaken.– Young adults gain independence from their families.– Education is a major socialization agent in modern society– It effects person’s future earnings, moral beliefs, and values, and leisure time

interests.– In the past parents and religion had a major role in selecting marriage partners– Young adults spent more time in school– Over time, the trend has been that people are less likely to marry persons within

the same religion and increasingly more likely to marry persons with a similar level of education.

So far…

• We have seen– Causal explanations (positive)– Structural (critical)

The third option – Interpretive explanation

Interpretive Explanation

• Places what we wish to explain (e.g. a social relationship, event, cultural practice…) within a specific social context and setting that have a meaning system.

• Each person’s subjective view shapes how she acts, so the goal is to discern others’ reasoning

Example

• Futrell and Simi (2004) study US white power movement, collective identity, or a shared sense of `we`

• They examined members of racist movements that are fragmented into many organizations (Ku Klux Klan, Christian identity groups, Aryan Nation, neo-Nazi groups)

• They investigated how members communicate their belief and engage in activism.

• After interviewing and collecting data on 56 activists

Futrell and Simi (2004)

• After interviewing and collecting data on 56 activists– Discovered that the members participated in small

domestic gatherings (study groups, ritual parties, etc) in which they reaffirmed their commitments to the group and discouraged conformity to outsiders.

– Hence they could create free spaces for their radical belief to florish

Research Design Strategies

• Just as the choice of research method must be conditioned on considerations of the nature of the phenomenon of interest, so too must the role of statistical techniques be evaluated with respect to the general goal of eliminating or reducing the plausibility of rival alternative hypotheses for the events under investigation.

• One potential rival explanation that plagues social research at all stages of investigation is the operation of “chance.”

• The phenomena of interest to the social sciences are generally subject to considerable nonsystematic variation, that is, variations from individual to individual and, within individuals, from time to time.

• The purpose of most inferential statistical tests is to assess the validity of this rival explanation of results in terms of the probability, or likelihood, that the obtained data pattern could have occurred by chance.

• The results of a statistical inference test tell us the probability of a Type I error of inference—the likelihood that a result would be obtained when the null hypothesis (no true relationship between the independent and dependent variable) is actually valid.

• Statistical significance is achieved when this probability is so low as to render the chance explanation implausible.

• External validity is concerned with the issue of generalizability. Assuming that a research finding is internally valid, external validity has to do with the extent that it can be generalized to other respondent groups, to other settings, and to different ways of operationalizing the conceptual variables.

• Even when internal validity is high, however, there may arise questions about the validity of interpretations of causal effects obtained in any given study, particularly their applicability or generalizability outside of the experimental setting. These concerns constitute questions of external validity, which can be further divided into questions of

(1) generalizability of operationalizations and (2) generalizability of results to other places

and participant populations

• Validity of Operationalizations corresponds to misusing a statistical technique such as scale vs. technique conflict or failure of assumptions that the technique rests on.

• Once a research study has been completed, the investigator is usually interested in reaching conclusions that are generalizable across people and across settings.

• Threats to this form of external validity arise from possible interaction effects between the treatment variable of interest and the context in which it is delivered, or the type of participant population involved. (a’ka random sampling)

• An experimental finding lacks external validity if the nature of the effect of the independent variable would be reduced or altered if the setting or the participant population were changed.

• Now we switch to technical matters and review or learn the necessary background to discuss further topics on research design strategies.