multi variate discriminant

7/28/2019 multi variate discriminant

1/46

9. Discriminant Analysis

Example 9.1: Consider the following data on financial

ration for solvent and bankrupted companies

Financial Ratios of Bankrupt and Solvent Companies, Altman (1968)Source: Morrison (1990). Multivariate Statistical Methods,

3rd ed. McGraw-Hill

X1 = Working Capital / Total AssetsX2 = Retained Earnings / Total AssetsX3 = Earnings Before Interest and Taxes / Total AssetsX4 = Market Value of Equity / Total Value of LiabilitiesX5 = Sales / Total AssetsGroup, 1 = Bankrupt 2 = Solvent

1


2/46

Group X1 X2 X3 X4 X5

1 36.7 -62.8 -89.5 54.1 1.71 24.0 3.3 -3.5 20.9 1.11 -61.6 -120.8 -103.2 24.7 2.51 -1.0 -18.1 -28.8 36.2 1.11 18.9 -3.8 -50.6 26.4 0.91 -57.2 -61.2 -56.6 11.0 1.71 3.0 -20.3 -17.4 8.0 1.01 -5.1 -194.5 -25.8 6.5 0.51 17.9 20.8 -4.3 22.6 1.01 5.4 -106.1 -22.9 23.8 1.5

1 23.0 -39.4 -35.7 69.1 1.21 -67.6 -164.1 -17.7 8.7 1.31 -185.1 -308.9 -65.8 35.7 0.81 13.5 7.2 -22.6 96.1 2.01 -5.7 -118.3 -34.2 21.7 1.51 72.4 -185.9 -280.0 12.5 6.71 17.0 -34.6 -19.4 35.5 3.41 -31.2 -27.9 6.3 7.0 1.31 14.1 -48.2 6.8 16.6 1.6

1 -60.6 -49.2 -17.2 7.2 0.31 26.2 -19.2 -36.7 90.4 0.81 7.0 -18.1 -6.5 16.5 0.91 53.1 -98.0 -20.8 26.6 1.71 -17.2 -129.0 -14.2 267.9 1.31 32.7 -4.0 -15.8 177.4 2.11 26.7 -8.7 -36.3 32.5 2.81 -7.7 -59.2 -12.8 21.3 2.11 18.0 -13.1 -17.6 14.6 0.9

1 2.0 -38.0 1.6 7.7 1.21 -35.3 -57.9 0.7 13.7 0.81 5.1 -8.8 -9.1 100.9 0.91 0.0 -64.7 -4.0 0.7 0.11 25.2 -11.4 4.8 7.0 0.9

2


3/46

2 35.2 43.0 16.4 99.1 1.3

2 38.8 47.0 16.0 126.5 1.92 14.0 -3.3 4.0 91.7 2.72 55.1 35.0 20.8 72.3 1.92 59.3 46.7 12.6 724.1 0.92 33.6 20.8 12.5 152.8 2.42 52.8 33.0 23.6 475.9 1.52 45.6 26.1 10.4 287.9 2.12 47.4 68.6 13.8 581.3 1.62 40.0 37.3 33.4 228.8 3.52 69.0 59.0 23.1 406.0 5.5

2 34.2 49.6 23.8 126.6 1.92 47.0 12.5 7.0 53.4 1.82 15.4 37.3 34.1 570.1 1.52 56.9 35.3 4.2 240.3 0.92 43.8 49.5 25.1 115.0 2.62 20.7 18.1 13.5 63.1 4.02 33.8 31.4 15.7 144.8 1.92 35.8 21.5 -14.4 90.0 1.02 24.4 8.5 5.8 149.1 1.5

2 48.9 40.6 5.8 82.0 1.82 49.9 34.6 26.4 310.0 1.82 54.8 19.9 26.7 239.9 2.32 39.0 17.4 12.6 60.5 1.32 53.0 54.7 14.6 771.7 1.72 20.1 53.5 20.6 307.5 1.12 53.7 35.6 26.4 289.5 2.02 46.1 39.4 30.5 700.0 1.92 48.3 53.1 7.1 164.4 1.9

2 46.7 39.8 13.8 229.1 1.22 60.3 59.5 7.0 226.6 2.02 17.9 16.3 20.4 105.6 1.02 24.7 21.7 -7.8 118.6 1.6

3


4/46

Relevant questions then are:

How do the companies in these two groups differfrom each other?

Which ratios best discriminate the groups?

Are the ratios useful for predicting bankruptcies?

Partial answers to can be obtained by examining each

single variable at a time.

4


5/46

For example sample statistics for each group

are

Sample Statistics of Bankrupt data

Statistic X1 X2 X3 X4 X5

Bankrupt Mean -2.83 -62.51 -31.78 40.05 1.50

Solvent 41.40 35.24 15.32 254.67 1.94

Bankrupt Median 5.40 -39.40 -17.70 21.70 1.20

Solvent 45.60 35.60 14.60 164.40 1.80

Bankrupt Standard Deviation 45.88 71.31 51.35 54.94 1.16

Solvent 14.21 16.51 10.87 206.57 0.93

Bankrupt Sample Variance 2104.57 5085.48 2637.18 3018.22 1.35

Solvent 201.99 272.50 118.11 42669.19 0.86

Bankrupt Kurtosis 6.95 3.31 17.55 9.51 12.30

Solvent -0.63 -0.33 0.71 0.72 6.29

Bankrupt Skewness -2.09 -1.69 -3.82 2.91 3.03

Solvent -0.37 -0.18 -0.56 1.31 2.18

Bankrupt Range 257.50 329.70 286.80 267.20 6.60

Solvent 55.00 71.90 48.50 718.30 4.60

Bankrupt Minimum -185.10 -308.90 -280.00 0.70 0.10

Solvent 14.00 -3.30 -14.40 53.40 0.90

Bankrupt Maximum 72.40 20.80 6.80 267.90 6.70

Solvent 69.00 68.60 34.10 771.70 5.50

Bankrupt Count 33 33 33 33 33

Solvent 33 33 33 33 33

t-Test: Two-Sample Assuming Equal Variances

Sales / Total Assets

Bankrupt Solvent

Mean 1.50303 1.939394

Variance 1.350928 0.864962

Observations 33 33

Pooled Variance 1.107945

df 64

t Stat -1.68396

P(T


6/46

Some graphics may also be helpful. For ex-

ample,

Class limits Bankrupt Solvent

< -51 6 0

-35 3 0

-20 6 0

-5 10 2

10 8 7

25 0 17

40 0 7

41 > 0 0

Histogram

EBIT / Total Assets

0

2

4

6

8

10

12

14

16

18

F

X1 27.9892 0.0001X2 58.8555 0.0001X3 26.5698 0.0001

X4 33.2726 0.0001X5 2.8357 0.0971

Average R-Squared: Unweighted = 0.2922351Weighted by Variance = 0.3546308

Multivariate Statistics and Exact F Statistics

S=1 M=1.5 N=29

Statistic Value F Num DF Den DF Pr > F

Wilks Lambda 0.369760775 20.4534 5 60 0.0001Pillais Trace 0.630239225 20.4534 5 60 0.0001Hotelling-Lawley Trace 1.704451275 20.4534 5 60 0.0001Roys Greatest Root 1.704451275 20.4534 5 60 0.0001

17


18/46

Example: Discriminant analysis applied to bankrupt data

Canonical Discriminant Analysis

Adjusted Approx SquaredCanonical Canonical Standard Canonical

Correlation Correlation Error Correlation

1 0.793876 0.781803 0.045863 0.630239

Eigenvalues of INV(E)*H

= CanRsq/(1-CanRsq)

Eigenvalue Difference Proportion Cumulative

1 1.7045 . 1.0000 1.0000

Test of H0: The canonical correlations in thecurrent row and all that follow are zero

LikelihoodRatio Approx F Num DF Den DF Pr > F

1 0.36976078 20.4534 5 60 0.0001

NOTE: The F statistic is exact.

Total Canonical Structure

CAN1

X1 0.694823X2 0.871854X3 0.682260X4 0.736708X5 0.259462

18


19/46


20/46

Raw Canonical Coefficients

CAN1

X1 0.0034765558X2 0.0084720383X3 0.0152812900X4 0.0030378872X5 0.4984713894

Class Means on Canonical Variables

GROUP CAN1

1 -1.2856131752 1.285613175

20


21/46

The output includes several coefficient ma-

trices.

The structure matrices describe the correla-

tions of the original variables with the dis-

criminant function.

The most useful of these for interpretation

purposes is the within canonical structure.

In the case of multiple groups also between

canonical structure may give useful additionalinformation.

This structure tells how the means of vari-

ables and means of discriminant functions are

correlated.

21


22/46

The standardized coefficients are obtained by

dividing the raw coefficients by the standarddeviations of the variables.

These coefficient tell the marginal effect of

the (standardized) variable on the discrimi-

nant function.

Labeling the discriminant function is based

on those variables having largest correlations

and largest standardized coefficients.

22


23/46


24/46

It should be noted that the basic assumption

in the discriminant analysis is that the vari-ables are normally distributed in each of the

groups, and that the covariance matrices are

the same.

The former assumption is harder to test. Thelatter is easier (in SPSS select Box M from

the options).

If the covariance matrices are not the same

the linear discriminant function analysis is in-valid.

One should move to the quadratic discrimi-

nant function analysis.

This method, however, is planned for classi-

fication purposes.

24


25/46

Example 9.4. Testing for the equality of the popula-

tion covariance matrices.H0 : 1 = 2,(4)

where i is the population covariance matrix of the

population i (i = 1, 2).

SPSS give the result: Test Chi-Square Value = 186.18

with 15 degrees of freedom and p-value = 0.0001

We observe that the null hypothesis is rejected, hence

one analysis results should be interpreted with caution.

25


26/46

Number of Discriminant Functions

In a case of multiple group (> 2) the question

is: in how many dimension the groups are

different.

In the case of two groups this is not a majorproblem, because the groups can differenti-

ate only in one dimension.

Generally, however, there can be more dis-

criminating dimensions, if q > 2.

Example 9.5: The following data is a classic exampleconsidering different species of Iris Setosa.

The following measures were made:

SL: Sepal length

SW: Sepal WIdthPL: Pedal LengthPW: Pedal Width

26


27/46

The CANDISC procedure produces the following re-

sults.

title;

data iris;title Discriminant Analysis of Fisher (1936) Iris Data;input sepallen sepalwid petallen petalwid spec_no @@;if spec_no=1 then species=SETOSA ;if spec_no=2 then species=VERSICOLOR;

if spec_no=3 then species=VIRGINICA ;label sepallen=Sepal Length in mm.

sepalwid=Sepal Width in mm.petallen=Petal Length in mm.petalwid=Petal Width in mm.;

datalines;50 33 14 02 1 64 28 56 22 3 65 28 46 15 2 67 31 56 24 363 28 51 15 3 46 34 14 03 1 69 31 51 23 3 62 22 45 15 259 32 48 18 2 46 36 10 02 1 61 30 46 14 2 60 27 51 16 2

65 30 52 20 3 56 25 39 11 2 65 30 55 18 3 58 27 51 19 368 32 59 23 3 51 33 17 05 1 57 28 45 13 2 62 34 54 23 377 38 67 22 3 63 33 47 16 2 67 33 57 25 3 76 30 66 21 349 25 45 17 3 55 35 13 02 1 67 30 52 23 3 70 32 47 14 264 32 45 15 2 61 28 40 13 2 48 31 16 02 1 59 30 51 18 355 24 38 11 2 63 25 50 19 3 64 32 53 23 3 52 34 14 02 149 36 14 01 1 54 30 45 15 2 79 38 64 20 3 44 32 13 02 167 33 57 21 3 50 35 16 06 1 58 26 40 12 2 44 30 13 02 177 28 67 20 3 63 27 49 18 3 47 32 16 02 1 55 26 44 12 250 23 33 10 2 72 32 60 18 3 48 30 14 03 1 51 38 16 02 1

61 30 49 18 3 48 34 19 02 1 50 30 16 02 1 50 32 12 02 161 26 56 14 3 64 28 56 21 3 43 30 11 01 1 58 40 12 02 151 38 19 04 1 67 31 44 14 2 62 28 48 18 3 49 30 14 02 151 35 14 02 1 56 30 45 15 2 58 27 41 10 2 50 34 16 04 1...;

27


28/46

title Canonical Discriminant Analysis of IRIS data;

proc candisc data = iris;class species;var sepallen--petalwid;

run;

Which gives the results:

Canonical Discriminant Analysis of IRIS data


150 Observations 149 DF Total4 Variables 147 DF Within Classes3 Classes 2 DF Between Classes

Class Level Information

SPECIES Frequency Weight Proportion

SETOSA 50 50.0000 0.333333VERSICOLOR 50 50.0000 0.333333VIRGINICA 50 50.0000 0.333333


Multivariate Statistics and F Approximations

S=2 M=0.5 N=71

Statistic Value F Num DF Den DF Pr > F

Wilks Lambda 0.023438631 199.145 8 288 0.0001Pillais Trace 1.191898825 53.4665 8 290 0.0001Hotelling-Lawley Trace 32.47732024 580.532 8 286 0.0001Roys Greatest Root 32.1919292 1166.96 4 145 0.0001

NOTE: F Statistic for Roys Greatest Root is an upper bound.

NOTE: F Statistic for Wilks Lambda is exact.

28


29/46

Adjusted Approx Squared

Canonical Canonical Standard CanonicalCorrelation Correlation Error Correlation

1 0.984821 0.984508 0.002468 0.9698722 0.471197 0.461445 0.063734 0.222027

Eigenvalues of INV(E)*H= CanRsq/(1-CanRsq)

Eigenvalue Difference Proportion Cumulative

1 32.1919 31.9065 0.9912 0.99122 0.2854 . 0.0088 1.0000

Test of H0: The canonical correlations in thecurrent row and all that follow are zero

LikelihoodRatio Approx F Num DF Den DF Pr > F

1 0.02343863 199.1453 8 288 0.00012 0.77797337 13.7939 3 145 0.0001

Total Canonical Structure

CAN1 CAN2

SEPALLEN 0.791888 0.217593 Sepal Length in mm.

SEPALWID -0.530759 0.757989 Sepal Width in mm.PETALLEN 0.984951 0.046037 Petal Length in mm.PETALWID 0.972812 0.222902 Petal Width in mm.

29


30/46

Between Canonical Structure

CAN1 CAN2

SEPALLEN 0.991468 0.130348 Sepal Length in mm.SEPALWID -0.825658 0.564171 Sepal Width in mm.PETALLEN 0.999750 0.022358 Petal Length in mm.PETALWID 0.994044 0.108977 Petal Width in mm.

Pooled Within Canonical Structure

CAN1 CAN2

SEPALLEN 0.222596 0.310812 Sepal Length in mm.SEPALWID -0.119012 0.863681 Sepal Width in mm.PETALLEN 0.706065 0.167701 Petal Length in mm.PETALWID 0.633178 0.737242 Petal Width in mm.

30


31/46

Total-Sample Standardized Canonical Coefficients

CAN1 CAN2

SEPALLEN -0.686779533 0.019958173 Sepal Length in mm.SEPALWID -0.668825075 0.943441829 Sepal Width in mm.PETALLEN 3.885795047 -1.645118866 Petal Length in mm.PETALWID 2.142238715 2.164135931 Petal Width in mm.

Pooled Within-Class Standardized Canonical Coefficients

CAN1 CAN2

SEPALLEN -.4269548486 0.0124075316 Sepal Length in mm.SEPALWID -.5212416758 0.7352613085 Sepal Width in mm.PETALLEN 0.9472572487 -.4010378190 Petal Length in mm.PETALWID 0.5751607719 0.5810398645 Petal Width in mm.

Raw Canonical Coefficients

CAN1 CAN2

SEPALLEN -.0829377642 0.0024102149 Sepal Length in mm.SEPALWID -.1534473068 0.2164521235 Sepal Width in mm.PETALLEN 0.2201211656 -.0931921210 Petal Length in mm.PETALWID 0.2810460309 0.2839187853 Petal Width in mm.

Class Means on Canonical Variables

SPECIES CAN1 CAN2

SETOSA -7.607599927 0.215133017VERSICOLOR 1.825049490 -0.727899622VIRGINICA 5.782550437 0.512766605

31


32/46

The Wilks lambda test indicates that there

are two statistically significant discriminatorson the five percent level.

Generally the hypotheses to be tested is like

in the factor analysis

H0 : The number of discriminators = m

H1 : More is needed(5)

On the basis of the within-matrices the first

discriminator indicates that the species differwith respect to the overall size of the leaves

and the second discriminator that species dif-

fer also with respect to the width of the

leaves.

32


33/46

Example 9.6: Bankruptcy risk and signal to reorga-

nization of a company (Laitinen, Luoma, Pynnonen1996, UV, Discussion Papers 200)

Thus we have four groups.

33


34/46

The used ratios are:

34


35/46

Sample statistics:

B2 (n=20) N3 (n=17) N4 (n=23) F for eq

Variable Mean Std Dev Mean Std Dev Mean Std Dev Mean Std Dev of means

ROI -10.24 8.60 3.52 5.59 2.27 7.14 12.02 5.96 37.66***

TCF -13.32 10.83 0.13 2.31 0.97 5.00 6.47 5.67 32.48***

QRA 0.58 0.39 0.57 0.55 1.14 0.70 0.85 0.42 4.95**SCA -0.61 20.22 -4.75 18.79 13.62 13.19 23.13 19.55 10.39***

DSR 1.09 0.55 0.69 0.25 0.88 0.34 0.57 0.28 7.62***

**=significant at level 0.01

***=significant at level 0.001

B1 (n=20)

35


36/46

Number of canonical discriminant functions:

The results indicate that also the third canonical dis-

criminant function is statistically significant.

36


37/46

Canonical structure and standardized coefficients:

Table 11. Canonical structure and Standardized canonical coefficients both as pooled within.

Canonical structure Standardized coefficient

Variable CAN1 CAN2 CAN3 CAN1 CAN2 CAN3

ROI 0.702 0.036 0.004 0.717 0.013 -0.737

TCF 0.643 0.059 0.467 0.372 -0.458 0.983QRA 0.101 0.513 0.653 -0.061 0.563 0.661

SCA 0.252 0.773 -0.168 0.169 0.946 -0.522

DSR -0.306 0.203 0.149 -0.722 0.034 0.16

*Correlation coefficients between original variables and canonical variables.

37


38/46

Interpretation of the discriminant functions:

38


39/46


40/46

CAN1, the financial performance, shows that the fi-

nancial performance is the main characteristic differ-

entiating healthy and bankruptcy firms (as expected).

CAN2, controversy dynamic liquidity and static ratios,

is differentiating characteristic between reorganizable

non-bankrupt and reorganizable bankrupt firms.

CAN3, controversy between liquidity and other ratios,

reorganizable non-bankrupt firms and healthy firms.

The distinction is probably due to the fact that non-

bankrupt firms may have cash reserves (high liquidity),

but do not use it profitably.

40


41/46

Classification

The other main usage of discriminant anal-

ysis is to predict from which of the given

classes a given observation is coming from

(decease diagnostics, bankruptcy prediction,

etc.).

The goal is to minimize the misclassification

rate, (two groups labeled as 1 and 2)

P(E) = p1P(2|1) +p2P(1|2),(6)

where P(E) denotes the misclassification prob-

ability, pi is the probability that an obser-

vation is from group i, and P(j|i) denotes

the probability that an observation coming

from the group j is classified to the group i,

i, j = 1, 2, and p1 +p2 = 1.

The probabilities pi indicate the prior prob-

abilities or the population proportion of the

group i.

41


42/46

In the SAS-system procedure DISCRIM can

be used for classification purposes.

Example 9.7: Consider the bankruptcy example.

OPTIONS LS = 72;TITLE Example: Discriminant analysis applied to bankrupt data;DATA bankrupt;

INFILE d:\tex\opetus\tmmt\bankrupt.dat firstobs = 11;INPUT group x1-x5;

PROC DISCRIM CROSSVALIDATE;CLASS group;VAR x1-x5;

RUN;

The results are

Example: Discriminant analysis applied to bankrupt data

Discriminant Analysis66 Observations 65 DF Total

5 Variables 64 DF Within Classes2 Classes 1 DF Between Classes

Class Level Information

PriorGROUP Frequency Weight Proportion Probability1 33 33.0000 0.500000 0.5000002 33 33.0000 0.500000 0.500000

42


43/46

Discriminant Analysis Pooled Covariance Matrix Information

Covariance Natural Log of the DeterminantMatrix Rank of the Covariance Matrix

5 31.011359

Pairwise Generalized Squared Distances Between Groups2 _ _ -1 _ _

D (i|j) = (X - X ) COV (X - X )i j i j

Generalized Squared Distance to GROUP

From GROUP 1 21 0 6.611202 6.61120 0

Discriminant Analysis Linear Discriminant Function_ -1 _ -1 _

Constant = -.5 X COV X Coefficient Vector = COV Xj j j

GROUP1 2

CONSTANT -1.76280 -4.67181X1 0.01113 0.02007X2 -0.03003 -0.00825X3 0.01810 0.05739X4 0.00266 0.01047X5 1.42947 2.71115

43


44/46

Remark 9.3: In the two groups classification problem,

the logit (or probit) regression is more popular.

Example 9.8: Logit regression of the bankruptcy data

(we use only variable x2 here because of the conver-

gence problems).

proc logistic data = a.bankruptcy;

* wcta (x1) reta (x2) ebitta (x3) mvetvl (x4) sta (x5);model group = reta / ctable;

run;

Response Profile

Ordered TotalValue Group Frequency

1 1 33

2 2 33

Probability modeled is Group=2.

Model Convergence Status

Convergence criterion (GCONV=1E-8) satisfied.

Model Fit Statistics

InterceptIntercept and

Criterion Only Covariates

AIC 93.495 19.804SC 95.685 24.183-2 Log L 91.495 15.804

44


45/46

T

esting

Global

Null

Hypothesis:

BETA=0

Test

Chi-Square

DF

Pr

>

ChiSq

Likelihoo

d

Ratio

75.6917

1

multi variate discriminant

Documents

Transcript of multi variate discriminant