THE EXPERIMENTAL

METHOD

Experimental Method - is basically a collection of research

designs, guidelines for using them, principles and procedures for determining statistical significance, and criteria for determining the quality of a study

For some researchers , the experimental method is the premier method, all others being ‘ground clearing’ operations, that is, preliminary data collection and interpretation exercises to prepare for a formal experiment.

Experiments are generally conducted in order to

test the strength of relationships between variables.

We also saw that when the researcher is testing the

influence of one variable on another , the variable

doing the influencing is called the independent

variable, while the other being influenced is

called the dependent variable.Example:

“In a study of the effect of two different methods for teaching grammar, the teaching method would me the independent variable, and the students’ performance on a test of grammar knowledge would be the dependent variable”.

QUALITY CONTROL ISSUES:

THREATS TO INTERNAL VALIDITY

Quality Control – in the experimental method, is largely a matter of understanding the many things that can go wrong and taking steps to prevent or minimize those threats.

Threats to internal validity can be divided to three categories:

Experience Bias factorsParticipant BiasInstrumentation Bias

Threats to Internal Validity Based on Experience

Three Experience Bias Factors

(1) HISTORY - refers to events –things that happen during

an experiment which may influence the results.- can also have an unintended influence on

the outcomes of the treatment.

Example : “If classes in the study are disrupted due

to natural disasters or political unrest, the research will be affected”

Experience Bias factors are those “based on what occurs within a research study as it progresses”.

(2) TESTING- refers to the fact that taking a pre-test may

influence the subjects’ performance on post-test. That is in addition to learning from the treatment, learners may do better on the post-test because the pre-test alerted them to what was being investigated in the study.

(3) EXPECTANCY

- A treatment may appear to increase learning effectiveness as compared to that of a control or comparison group, not because it really boosts effectiveness but because either the experimenter or the subjects believe that it does and behave according to this expectation ( Tuckman, 1999)

Threats to Internal Validity Based on Participants

(1)Selection

Participant Bias is a result of the “characteristics of the people on whom the study is conducted”

Five participant bias factors:

-as a threat to Internal validity is the idea that somehow the groups to be compared turn out to be different before the treatment

Safeguard: Random selection and random assignment

are used to combat this problem.

(2) Maturation-refers to the normal development people undergo

whether or not they are receiving a treatment of some kind.

Safeguard:This problem is addressed through the use of control

group, which is at the same development level as the treatment group and goes through the same experiences for the period of time.

(3) Statistical regression-is the name of a tendency for people’s test scores to

change whether or not the knowledge, skill or ability to being measured truly changes.

Safeguard:Using subjects who represent an entire range of

ability levels in your design is one way to avoid this problem.

( 4)Experimental mortality- Is the problem of losing subjects from the study. It

can be especially worrisome if the groups end up being quite different sizes because people dropped out.

(5) Interactive combination of factors-happens when more than one threat is present in

a study.

Safeguard:To deal with this threat, researchers often try

to recruit more people for a study than they may actually need.

Safeguard: Careful planning is needed to avoid these sorts

of problems.

Threats to Internal Validity Based on Instrumentation

INSTRUMENTATION refers to the measurement or observation procedures used during an experiment ( Tuckman, 1999)

Instrumentation Bias has to do with “the way the data are collected”

The threat of instrumentation is also sometimes called instability of measures, because it occurs if the measurement or recording processes change during the experiment.

Safeguard:Observer training, rater training, and the careful

piloting of all questionnaires and data collection devices.

QUALITY CONTROL ISSUES:

THREATS TO EXTERNAL VALITY

There are four issues concern about external validity:

(1) Reactive effects of testing

- It is related to the testing threat ( or practice effect to internal validity. In this situation, the presence of a pre-test may give the effects of the treatment boost. Then, when the outcomes of the experiment are transferred to the real world where no pre-test is involved, that extra boost will be lacking.

(2) Reactive effects of experimental arrangements

-This threat is very interesting problem. This idea refers to the fact that sometimes just knowing that one is in an experiment is enough to cause a difference that may be captured by the dependent variable , whether or not one is in the treatment group.

(3) Interaction effects of selection bias

-occurs when the sample in an experiment is not really representative of the population from which it was drawn.

Stratified random sampling

-This term means that before we select people from the population to be in the sample, we determine what the relevant characteristics of the population are and make sure that the levels in the sample represents the population appropriately.

(4) Multiple-treatment interaction

-This threat is hard to manage in the classroom research , especially in second language settings where students have access to the target language outside of class.

RESEARCH DESIGNS IN THE EXPERIMENTAL METHOD

In order to deal with these threats, the experimental method includes many different research designs to counteract the possible confounding variables that could influence the internal and external validity of a study. The various designs have different strength and weaknesses. Anyone who chooses to do an experiment must balance the focus of the research question against the time resources available for conducting the study in order to choose the best design.

Scenario 1: The One-Shot Case Study Design

Scenario 2: The One-Group Pre-Test Post-Test Design

Scenario3: The Intact Groups Design

Scenario 4 : The nonequivalent Control(comparison) Groups design

Scenario 5:The Time Series Design

Scenario 6: The Equivalent Time Samples Design

Scenario 7: Post- Test Only Control Group DesignScenario 8:Pre-test Post Test Control Design