Tuesday, December 23, 20141

The Slides discuss• Comparing two means to ascertain which mean is of greater

statistical significance.• The statistical test used for this type of analysis is the t test,

and the statistic that is computed is called a t value.• Three research situations in which the t test can be used to

analyze the data and compare the means froma) Two independent groups (e.g., experimental and control);b) Two paired samples (e.g., pretest and posttest scores); andc) a sample and a population (e.g., comparing a mean of a

sample to the mean of the population).• Examples with numerical data to illustrate each of the three

types of t tests. • After t value is computed, you have decided whether it is

statistically significant and whether your research hypothesis was confirmed.

Steps For Comparing Groups

Assumptions of t-Test• The groups are independent of each other.• A person (or case) may appear in only one group.• When the two group are approximately the same size,

there is no need for the homogeneity of variance.• The independent variables are interval or ratio.• The population from which samples are drawn is

normally distributed.• Samples are randomly selected.• The groups have equal variance (Homogeneity of

variance).• The t-statistic is robust (it is reasonably reliable even if

assumptions are not fully met.

A t-test is used to compare two means in three different situations

t-tests with Two Samples

Independent Samples t-test

Independent Samples t-test• Used when we have two independent samples, e.g., a

comparison of experimental (treatment) and control groups.• The two groups whose means are being compared are

independent of each other.• Formula is: or

The second formula is used when the two groups have the same size of scores (i.e., n1=n2)

• Terms in the numerator are the sample means. • Term in the denominator is the standard error of the

difference between means.

An example of a t Test for Independent samples

A new test preparation company, called Bright Future (BF), wants to convince high school students studying for the American College Testing (ACT) test that enrolling in their test preparation course would significantly improve the students’ ACT scores.

BF selects twenty students at random and assign ten to an experimental group and ten to a control group.

The students in the experimental group participate in the test preparation course conducted by BF. At conclusion of the course, both groups of students take the ACT test, which was given to high school students the previous year. BF conducts a t test

For independent samples to compare the scores of Group 1 (Experimental E) to those of Group 2 (Control C).

The study’s research (alternative) hypothesis (HA) is directional and can be described as HA: µE>µC and Ho: µE=µC

The t- value is computed using the following formulas:

Review 7 Steps for Significance Testing

1. Set alpha (p level).

2. State the Research question

3. Set up hypotheses, Null and Alternative.

4. Calculate the test statistic (sample value).

4. Find the critical value of the statistic.

5. State the decision rule.

6. State the conclusion.

ACT Scores of Experimental Group (n=10) and Control Group (n=10)

Experimental Group 1 Control Group 2

26 19

27 24

21 18

31 23

21 22

25 24

29 32

32 29

34 15

23 20

Independent Sample Steps

1. Set alpha. Alpha(α) = .05

2. State Research Question.

Does students who participated in the test taking course score significantly higher on the practice form of the ACT than did the control group students?

Independent Sample Steps

3. State Hypotheses.

Null is H0: µ1 = µ2. Students who participated in the test taking course did not score significantly higher on the practice form of the ACT than did the control group students.

Alternative is H1: µ1 ≠ µ2. Students who participated in the test taking course scored significantly higher on the practice form of the ACT than did the control group students.

Independent Sample Steps (2)

4a. Calculate test statistic:

Independent Sample Steps (2)

4b. Calculate test statistic:

Independent Sample Steps

4. Determine the critical value. Alpha is .05, one tailed test, and df = N1+N2-2 or 10+10-2 = 18. The value is 1.734

5. State decision rule. If t-obtained (t-value) > t-critical or 1.992 > 1.734, then reject the null hypothesis. Such a large different between two groups could have occurred by chance alone less than 5 percent of the time.

6. Conclusion: Reject the null. It means than “Students who participated in the test taking course scored significantly higher on the practice form of the ACT than did the control group students” is confirmed

Interpretation

Based on the results of this study, we can interpret that at least 95 percent confident that the course offered by Bright Future (BF) is indeed helpful to students similar to those who participated in the study.

Based on the results of this study, we can interpret that at least 95 percent confident that the course offered by Bright Future (BF) is indeed helpful to students similar to those who participated in the study.

We can conclude that participating in the BF test-taking preparation course is effective and can bring about a statistically and practically significant increase in students’ ACT scores.

We can conclude that participating in the BF test-taking preparation course is effective and can bring about a statistically and practically significant increase in students’ ACT scores.

Cohen’s Effect Size

• Effect size of 0.2 is small, 0.5 moderate, 0.8 large

• The control group is used to compute SD because it is not contaminated by the treatment effect.

• Our ES is 0.85 which is considered high. This ES confirms that the difference of 4.30 points between the two groups is practically significant, in addition to being statistically significant (at p<0.05)

• In addition to inspecting and evaluating the difference between the means of the two groups, we can use the index of effect size (ES) to evaluate the practical significance of our study and the effectiveness of the intervention.

The language of StatisticsLearning about statistics is much like learning a new language. Many of the terms used in statistics may be completely new to you, like quartile. Also some words you already know will be used in new combination, like standard deviation. Finally, some words will have a new meaning when used in the context of statistics. For example, the word significance has a different meaning in statistical context. When used outside of statistics, significance is used as a value judgment. If something is said to be significant, then we usually mean that it is important or of consequence. The opposite of significant is usually insignificant. However in statistics, the word significance has a different meaning. Significance refers to an effect that has occurred that is not likely due to chance. In statistics, the opposite of significance is nonsignificant, and this mean that an effect is likely due to chance. It will be important for you to remember that the word insignificant is a value judgment, and it typically has no place in your statistical vocabulary

SPSS Results

Independent Samples Test

Levene's Test for Equality of Variances

t-test for Equality of

Means F Sig. t dfScores Equal variances

assumed.007 .937 1.992 18

Equal variances not assumed

1.992 17.791

Group Statistics

Group N Mean Std. DeviationStd. Error

MeanScores 1 10 26.90 4.557 1.441

2 10 22.60 5.082 1.607

Task of t-test for Independent samples

The data below represent the number of sentences remembered correctly by two groups of subjects, tested under two different conditions. The groups, one of 10 and one of 9 subjects, were constituted by random allocation of the 19 subjects. We wish to test the null hypothesis that the mean number of sentences recalled correctly is the same under the two conditions, at the p< 0.05 level.

Group 1 (n1=10) Group 2 (n2=9)

18 13

15 14

13 12

17 6

14 11

8 13

10 17

11 16

7 5

17

Table of Scores on a sentence recall test

Do 6 Steps for Significance Testing

1. Set alpha (p level).

2. State the Research question

3. Set up hypotheses, Null and Alternative.

4. Calculate the test statistic (sample value).

4. Find the critical value of the statistic.

5. State the decision rule.

6. State the conclusion.

ReferencesReferences Main Sources

Coolidge, F. L.2000. Statistics: A gentle introduction. London: Sage.Kranzler, G & Moursund, J .1999. Statistics for the terrified. (2nd ed.). Upper Saddle River, NJ: Prentice Hall.Butler Christopher.1985. Statistics in Linguistics. Oxford: Basil Blackwell.Hatch Evelyn & Hossein Farhady.1982. Research design and Statistics for Applied Linguistics. Massachusetts: Newbury House Publishers, Inc.Ravid Ruth.2011. Practical Statistics for Educators, fourth Ed. New York: Rowman & Littlefield Publisher, Inc.Quirk Thomas. 2012. Excel 2010 for Educational and Psychological Statistics: A Guide to Solving Practical Problem. New York: Springer.

Other relevant sources

Agresi A, & B. Finlay.1986. Statistical methods for the social sciences. San Francisco, CA: Dellen Publishing Company.Bachman, L.F. 2004. Statistical Analysis for Language Assessment. New York: Cambridge University Press.Field, A. (2005). Discovering statistics using SPSS (2nd ed.). London: Sage. Moore, D. S. (2000). The basic practice of statistics (2nd ed.). New York: W. H. Freeman and Company. 

  Tuesday, December 23, 2014