SHIFTED FACTOR ANALYSIS: A TEST OF MODELS AND ALGORlTHMS

Sungjin Hong Department of Psychology

Submitted in partial filfiliment of the requirements for the degree of

Master of Arts

Faculty of Graduate Studies The University of Western Ontario

London, Ontario Septernber 1997

O Sungjin Hong 1997

National Library 1*1 of Canada Bibliothèque nationale du Canada

Acquisitions and Acquisitions et Bibliographie Services services bibliographiques

395 Wellington Street 395. me Wellington OttawaON K1AON4 OttawaON K 1 A W Canada Canada

Yourlm V o n e r e Y ~

Our fi& Notre retereme

The author has granted a non- L'auteur a accordé une licence non exclusive licence allowing the exclusive permettant à la National Library of Canada to Bibliothèque nationale du Canada de reproduce, loan, distribute or sell reproduire, prêter, distribuer ou copies of this thesis in microform, vendre des copies de cette thèse sous paper or electronic formats. la forme de rnicrofiche/film, de

reproduction sur papier ou sur format électronique.

The author retains ownership of the L'auteur conserve la propriété du copyright in ths thesis. Neither the droit d'auteur qui protège cette thèse. thesis nor substantial extracts fkom it Ni la thèse ni des extraits substantiels may be printed or otherwise de celle-ci ne doivent être imprimés reproduced without the author's ou autrement reproduits sans son permission. autorisation.

ABSTRACT

The position shifi of the sequential facton in some sequential data can violate the

standard factor model's requirement of proportionality of the effect of factor weights.

Harshrnan has proposed a Shifted Factor model and analysis procedure to resolve this

problern. It would unshift the latent factors to recover the proportionality of factor effects.

He has also conjectured that the shift adjustment of the factors would provide an

additional advantage: the factor solution given by fitting the Shifted Factor model may

not be rotationally indeterminate. This thesis tests the feasibility of the "unshifiing" idea

by developing and testing cornputer programs that estimate shifi values in addition to

standard factor loadings and then applies these shifls to factors when predicting data.

Two types of programs are tested: Quasi Alternating Least Squares (QALS) and

nonlinear optimization (OPT). Both types of programs are implemented for two-way and

three-way sequentially organized data. Error-free and fallible synthetic data are created

and then analyzed to test the ability of the programs to recover latent factors and shifis,

and to test the hypothesized uniqueness property of the two-way Shified Factor model.

Both QALS and OPT have perfectly (fi-orn the error-free synthetic data) and successfully

( from the fallible synthetic data) recovered factor loadings and the shifts. The

hypothesized uniqueness of the Shifted Factor solution has been numerically proved by

showing many perfect solutions from the error-fiee data and many matching solutions

From the fallible data, starting fiom a few different random positions. Brain evoked

potential time series data are analyzed as a "real data" application of the model. The real

data analysis was not successful perhaps partly because not only shifts but shape changes

of factors seemed to be present. Provision for the shape changes (e.g., width variation in

latent peaks) is a possible fûture extension of this method. In the meantime, other kinds

of reai data (EEG spectra, chernical spectra, chromatographie series, etc) should be

explained by the Shifted Factor mode1 because they are likely to have shifts without

widthhhape changes.

Keywords: Factor analysis, Principal component analysis, P W A C , Uniqueness, Time

series, Lag.

ACKOWLEDGEMENTS

I would like to thank my advisor, Dr. Richard Harshman, for his deep insight and

cntical comments throughout this study. Without his thoughtful guide and w a m help,

this thesis would have not been finished. 1 would also thank Dr. Robert Gardner, Dr. Sam

Paunonen, and Dr. Stan Leung for their patience in understanding many arnbiguous

expressions in the cirafi of this thesis, and their critical suggestions. ln particular, it was

gratefully appreciated that Dr. Stan Leung kindly provided his data for this study and

taught me many basics necessary for analyzing the data.

TABLE OF CONTENTS

Page

CERTITICATE OF EXAMINATION ................................................ ABSTRACT .............................................................................. ACKNOWLEDGEMENT ............................................................. TABLE OF CONTENTS ............................................................... LIST OF TABLES ..................................................................... LIST OF FIGURES ..................................................................... LIST OF APPENDICES ...................................... .. .......................

1 . INTRODUCTION

. 2 STATEMENT OF PROBLEM ................................................... ........................... 2.1. Proportionality of the Standard Factor Model ........................... 2.2. An Example of Sequentially Organized Data

............ 2.3. Violation of the Proportionality in Some Sequential Data

3 . SHIFTED FACTOR MODELS ...................................................... 3.1. Two-way Shified Factor Mode1 ............................................. 3.2. Three-way Shified Factor Models ........................................... 3.3. Uniqueness in the Two-way Shifted Factor Mode1 .....................

4 . ALGORITHM TESTTNG AND DEVELOPMENT ............................... 4.1. Quasi-ALS .....................................................................

............ 4.1.1. Altemating Estimation for the Shifted Factor Models .................................... 4.1.2. Brute Force Estimation of Shifis

............ 4.1.3. Polishing the Sequential Mode Estimation Procedure 4.1.4. Fractional Line Search for the Shift Estimation .................. 4.1 .5 . "Super Iteration" of Recurrent BF and FLS .....................

.................................... 4.1.6. Alternating Estimation for TSF1 4.2. Initial Test of Nonlinear Optimization: Simultaneous

............................................. Optimization of Al1 Parameters

............................................. . 5 ANALYSIS OF SYNTHETIC DATA 5.1. Quasi-ALS ...................................................................

................................. 5.1.1. Two-way Synthetic Data Analyses .............................. 5.1.2. Three-way Synthetic Data Analyses

...................................................... 5.2. Nonlinear Optimization .............................. 5.2.1. Two-way Synthetic Data Analyses .............................. 5 -2.2. Three-way S ynthetic Data Analyses

6 . AN APPLICATION ..................................................................

7 . DISCUSSION

REFERENCES APPENDIX A APPENDK B APPENDIX C APPENDIX D APPENDIX E APPENDIX F APPENDIX G VITA .......................................................................................

vii

Table

LIST OF TABLES

Description Page

Nurnber of Perfect Solutions and Averaged Model Fit and Recovenes of Non-perfect Solutions .................................... Model Fits (R2) and the Recoveries of the Two-way

................................................... Super QALS Analyses

Model Fits (R~) and the Recoverks of the Three-way ................................................... Super QALS Analyses

Model Fits (R') and the Recovenes of the Two-way Nonlinear Optimization Analyses ..................... .. ............-.

Model Fits (R') and the Recoveries of the Three-way Nonlinear Optirnization Analyses .......................................

Factor Loadings (B) on the Measuring Conditions and ......................................................... Shift Estimates (S)

An Example of Mean Offset of Shifts and ................................................... Their Mode B Loadings

LIST OF FIGURES

Figure Description

Basic Data Relation Matrix (BDRM) ....................................

Flash Evoked Potentids Recorded at Different Scalp Locations ...

....................................... Three Shapes of Synthetic Curves

The 25 Evoked Potential Data C w e s ................................. Recovered Sequential Factors by the Two-way Shifted

..................... Factor Mode1 and by the Standard Two-way PCA

The Data Curves in Each Condition o f the Paired Pulses ............

Page

LIST OF APPENDICES

Appendix Description Page

A Fractional Line Search (FLS) for the Shifi Estimation .................. 68

B Oscillating Mode1 Fits in FLS and Super QALS ........................ 71

.......................................... C AIternating Estimation for TSF 1 74

.................. D Singular Value Decomposition for the A Estimation 77

E Nonlinear Optimization Procedures ....................................... 79

F MATLAB PrograrnsofQALS ........................................... 81

G Mean Shift Offset and Shift Outliers ....................................... 92

INTRODUCTION

Cattell (1 946) showed various ways of getting syrnmeû-ical data to factor analyze,

fiom his BDRM (Basic Data Relation Matrix): R- and Q-, P- and O-, and S- and T-

technique. He suggested P-technique as a method to define personality facton unique1 to

one person. P-technique factor analyzes the correlations (or covariances) among variables

but across occasions, while the conventional factor analysis extracts factors fiom the

correlations (or covariances) among variables but across subjects. He called the

traditional factor analysis R-technique. Different techniques defined by Cattell are based

on the various ways to get a symmetrical cross product type data matrix. In other words,

they are different in terms of how to capture the covariation of measures due to the

underlying factors in the data. Figure 1 shows how the different versions of correlations

(or covariances) can capture facton fiom the different facets.

Considenng that time senes variables are likely to be lagged dong the time axis

but differentially according to levels of the variable mode, Cattell (1963) suggested time-

corrected (or lead-and-lag) P-technique in order to correct the mismatch of time senes

variables before factoring them by using the standard factor analysis. But the potential

lagging was supposed to exist in the relation among variables, i.e., in correlations. Time-

corrected P-technique analyzes a corrected version of across-time correlations, maximum

cross correlations. To get a maximum cross correlation for two time senes variables,

' The term "unique factor'' to a person or a group (when each occasion measure is averaged across perçons in a gmup) should not be confused with "specific factor" to a variable. "Unique" is to restrict the factor solution to the sample (a person or a group), while "specific" is to indicate the variance accounted for by a variable but none of the other variables.

People

Variables

F i w e l . Basic Data Relation Matrix (BDRM): adopted from Cattell (1946). Each pair of two parallel lines represents how to capture the covariation from one of six facets of BDRM.

one of the two tirne senes variables is first lagged by various amounts oflag and then

correlated with the other variable. Among the cross correlations, the maximum cross

correlation is then used for factor analysis. Tirne-corrected P-technique, however, does

not take care of dzfferential lagging of the underlying time senes factors because the

varying amounts of lagging are applied to the observed time series variables, not to the

underlying time senes factors. In other words, it assumes that the underlying time senes

factors are simultaneously lagged.

Molenaar ( 1985) generalized Cattell 's time corrected P-technique by allowing

varying arnounts of lagging of the underlying time series factors and called his model

dynarnic factor analysis. Dynamic factor analysis assumes that an observed time series is

a weighted surn of underlying time senes factors that persist during several subsequent

times. In other words, the contribution of the latent time series factor to the data at time r

is assumed to take several consecutive times t, t+ 1, . . . to W e a r itself out. Dynarnic factor

analysis successfully resolved the limitation of the time corrected P-technique: the latent

time series factors are simultaneously lagged (or shifted). However, dynamic factor

analysis also cannot uniquely detetmine the model parameters, as is the case in the

solution of principal component analysis (PCA), because an arbitrary rotation of the

solution of dynarnic factor analysis keeps the data recovered by the model invariant.

More detailed discussion will be provided in the Shifted Factor model section.

STATEMENT OF PROBLEM

Pro~ortionaliîy of the Standard Factor Model

A fundamental feature of the standard factor model is the "proportionality" of the

effects of factor weights, i.e., of factor Ioadings and factor scores. The loading of factor r

on variable j is a weight applied multiplicatively to each subject7s factor score on factor r.

across al1 subjects; it represents the relative contribution of the rth factor to the jth

variable, and the proportional contribution to the data for al1 subjects at the jth level. In

the same way, the rth factor score of the ith subject holds for al1 variables as the relative

and proportional contribution of the rth factor to the ith subject at al1 variable levels.

Thus, the product of the jth factor loading and ith factor score for rth factor determines

the contribution of that factor to the observed score of the ith subject at jth variable. This

proportionality of the factor model c m be considered as a generalization fiom multiple

regression in the sense that both factor analysis and multiple regression search for

coefficients (or weights) that define some linear combination(s) of a set of given

variables so as to maximally account for the target variable(s). Factor analysis differs

fiom multiple regression in that it has to estimate two sets of unknown coefficients (i.e..

factor loadings and factor scores), but multiple regression estimates only one set of

coefficients (Le., regression coefficients).

The proportional relationship between the two sets of factor weights c m be

descnbed as a "bilinear combination," and written in scalar form as

where a represents a factor loading, b represents a factor score, and e represents a

residual not fit by the model. The subscnpts, i, j, and r, respectively, represent the ith

subject, the jth variable, and the rth factor. The italic upper case letter R is used to

represent the maximum value of r, Le., the nurnber of factors. By this convention, this

paper also uses the itdic upper case letter I to stand for the maximum value of i (e-g., the

nurnber of subjects), and J to represent the maximum value o f j (e-g., the nurnber of

variables). Note that Equation 1 is not an expression for the typical correlation or

covariance data as is usually the case in the traditional factor analysis, but instead is an

expression for the original "raw score" or profile data (as in Horst, 1965). Kniskal(1978)

differentiated two types of data fitting methods in factor analysis: direct and indirect

fitting. Indirect fitting represents the conventional way of performing factor analysis in

which a correlation or covariance matrix is factor analyzed, whereas direct fitting means

to fit a factor model directly to the original profile or "raw" score data without converting

them to correlations or covariances. Equation 1 represents the factor rnodel for direct

fitting in his taxonomy. The appropnateness of direct and indirect fitting for some

sequentially organized data will be discussed in the following section.

An Example of Sequentiallv Or~anized Data

Some psychological measures can be obtained at a series of closely spaced successive

points across an interval of tirne, fiequency, or distance. For exarnple, consider an

electrical signal evoked in the brain by a stimulus, such as a flash of light; the data consist

of a set of voltage measures obtained at I successive time points. Then suppose we have a

collection of J senes of such sequential measures frorn different sources, for example, J

6

different electrode locations on the scalp, but al1 obtained during the same set of l times.

We can then compose an I by J data matrix X, by adjoining the J column vectors of

length I side by side. These sequential data can be M e r assumed to consist of J linear

combinations of a few latent sequential variables called factors. An example of evoked

potential data is presented in Figure 2, where 8 time series measures are drawn against a

set of the sarne time points after a flash of light. Figure 2 is drawn based on the profiles

provided in Field and Graupe (1 991). The sequential mode could be frequencies or

distances instead of time points. For instance, EEG power spectral data in physiological

psychology or fluorescence spectral data in chemistry may be examples of typical

frequency data. III environmental science, the levels of the sequential mode could be

many geographical locations that are apart from a source-location by varying distances,

which is supposed to create a few pollutants.

Violation of the Proportionality in Some Sesuential Data

Some kinds of sequential measures rnay not be appropnate to fit by the standard

factor model. A problem arises when the underlying sequential factors are not simply

added together to produce the observed columns of X, but are first subject to shitts in

position of varying amounts along the sequential axis before being surnmed to produce

the observed data in any column. The fact that row position shifts (or position offsets) can

be different for each of the factors underlying such sequential measures will cause the

effects of the sequentiai factors to Molate the proportionality across cuves (e.g., across

electrode locations) required by the standard factor model. For example, the effects of a

sequential factor could be shifted down by 3 rows (relative, Say, to the average shifi

Time after flash

Figure 2. Flash Evoked Potentials Recorded at Di fferent Scalp Locations (C3, C,, C4, P3, Pz, P4, 01, Oz). From Field and Graupe (1 99 1 ).

8

position of that factor) when measured at an electrode located in the temporal lobe, while

it could be shifted up by 2 rows when measured at an electrode located in the occipital

lobe. Likewise, the shift amounts may Vary by factors at the same level in the non-

sequential mode. For example, a sequential factor could be shifted down by 3 rows when

measured at an electrode located in the temporal lobe but another sequential factor at the

same location might be shifted by a different nurnber of rows.

Furthemore, the standard indirect fitting factor analysis is also not feasible if

shifiing factors are present. This is because it is not appropriate to compute cross

products, covariances, or correlations among variables by matching ith scores meaçured

at different variables (e.g., different electrode locations) without adjusting for the row

position shi fis, again because underlying sequential factors will be di fferentially shi fted

From one variable to the next. If the sequential factors are differentially shifled across

variables, the "true" association between two sets of sequential measures would not be

known without adjusting the row position offset before computing one of the cross

product type measures. In addition, if the sequential factors are differentially shifted

across variables, it is not possible to know the 'bue" association between variables

without revealing the nature of the latent shifted factors. This is because each sequential

variable is a sum of a few sets of bi-products (&,ajr) and because the latent sequential

factors are differentially shifted before producing the bi-products. Thus, raw profile data

(1 by J for the above example) would be preferred to the symrnetrical data of cross

products ( Jby J for the above exarnple) when decomposing the sequential data to reveal

the latent shified factors.

SHIFTED FACTOR MODELS

Two-way Shifted Factor Mode1

Harshrnan (1 997) has proposed a "Shifted Factor" type of model to incorporate

the row position shifis in the description of the latent factors underlying the sequential

data when the proportionality is violated. He derived the Shifted Factor model from the

standard bilinear factor model shown in Equation 1 as follows,

(2) xii = x a,. 1+sjr 1 t- b . ~r +eV -

This differs from Equation 1 only in that (i+sjr) replaces i (the row subscnpt of the

sequential mode factor loading a is parenthesized for clarity). There is a shift value SIr,

added to the standard row subscnpt i, which is an additional mode1 parameter that is not

part of the standard factor mode1 such as the standard hvo-way factor analysis presented

in Equation 1. Note that the shifi value s,, is added to the row subscnpt i of the factor

loading for the sequential mode, air, so as to adjust the row position of factor r at colurnn

j. In the case of the above example, sjr is the shiR parameter defining the amount of the

row position shifting of the rth latent time senes factor at jth electrode location. The shifi

pararneters sjr can be collected in a J by R parameter matrix S; these pararneters form a

two-way matrix because shift values are assumed to Vary by levels of the non-sequential

modeL (Mode B in the above equation) as well as by factors. Therefore, the element in the

' Mode will be used to represent each axis in the data matrix (or array) but only for the raw profile data, not for cross product type symrnetrical data. Thus, each mode will be a distinct entit-y, e.g., subjects, variables, measuring conditions, occasions, etc.

10

jth row and the rth column of S represents the size of the row position shift that is applied

to al1 elements of the jth time series, but only for the rth factor.

The two-way Shifted Factor mode1 is different f?om both Cattell's time corrected

P-technique and Molenaar's dynamic factor anaiysis in the sense that it directly fits the

model to the raw profile data rather than to a cross product type of symmetncal data. The

direct fitting of the Shifted Factor model then gives a "direct" solution of factor scores

without any fùrther estimation. Dynamic factor analysis has been reported not to be able

to determine uniquely the solution of factor scores aven a fixed solution of factor

loadings (Molenaar, 1985). The two-way Shif€ed Factor model is the same as dynamic

factor analysis in the sense that both of them adjust for the row position shifi in the latent

factor level, but not in the observed variable level as time corrected P-technique does.

However, the two-way Shifted Factor model is different From dynamic factor analysis

because the former c m give a uniquely determined solution of factor scores and loadings,

but the latter cannot. The discussion about the indeterminacy in dynamic factor analysis

will be followed in more detail later.

Three-wav Shifted Factor Models

Like the two-way Shifted Factor model, the three-way Shified Factor models can

be derived fiom a three-way proportional factor mode1 (Le., trilinear factor model) such

as PARAFAC (ParaIlel Factor analysis) . The PARAFAC model for direct fitting of 'raw'

profile data can be presented in scalar form as,

I I

Equation 3 shows that each data point in a three-way array3 X is approximated by a sum

of R triple-products except, of course, the residual e p which is not accounted for by the

model. The proportionality of the effect of the three sets of factor loadings4 in PARAFAC

holds in the same way as it does for the two sets of factor loadings in the two-way

standard factor analysis s h o w in Equation 1. To facilitate describing three-way Shified

Factor models, the above hvo-way example of sequential data can be extended to a three-

way sequential data array by introducing another non-sequential mode. The third mode

could be various subjects from whorn the evoked potentials are to be measured or

different types of stimuli presented to a single subject. Thus, the evoked potential

rneasures are obtained from K different stimuli but at the sarne 1 time points after each

stimulus and at the sarne J electrode locations, forming an I by J by K three-way

sequential data array X.

The size of the shifi parameter matrix S (or of three-way array S) in three-way

Shified Factor models will Vary depending on how one defines the relation of shifi

parameters between the two non-sequential modes. The simpler model assumes that the

shifi, sjr holds for all levels of the third mode. For exarnple, the shifl parameter of the jth

electrode location for the rth factor is a constant across ail K stimuli. With this

assumption, a three-way Shifted Factor model (TSF1 , which stands for Three-way

Shifted Factor model 1) can be derived as,

One convention is to use a bold and underlined upper case letter for three-way arrays, whereas a bold upper case Ietter represents a two-way array. ' "Loading" will be used to stand for factor weights in any mode in a more general sense since each mode does not have any special meaning in the rnodel level.

TSF1 differs fiorn Equation 3 in the same way that Equation 1 differs fiom Equation 2.

narnely that shift parameter sjr is added to the row subscript i of the sequential mode

parameter a. The shift parameter has two subscripts just as in the two-way Shifted Factor

model shown in Equation 2. In other words, the rth factor underlying the jth 'slab' of the

three-way sequential data is shified by a constant amount s, for al1 levels of the third

mode. Because the shif? value for the rth factor at the jth level of Mode B (e.g., electrode

location mode) is common to al1 levels of Mode C (e.g., stimuli mode), shift parameters

can be collected in a J by R mahix as in the two-way Shifted Factor model.

A bit more complicated but general three-way Shifted Factor mode1 can be

defined by allowing a different amount of shift at sach specific level of Mode C for each

level of Mode B. In the above three-way example, when producing the time series of

evoked potential responses obtained at the jth electrode location but only fiom the kth

stimulus, the rth sequential factor is assumed to be subject to a distinct arnount of shifi.

Thus. the shifi parameters are assumed to Vary across levels of Mode C as well as across

levels of Mode B. Consequently, one needs to collect the shifi parameters in a J by K by

R three-way array, 3. This three-way Shifted Factor model (TSFZ) can be written in

scalar form as follows,

Note that the shifi parameter added to the row subscript i of the sequential mode ioading

a has additional subscnpt k, which is introduced to represent the varying shifl amounts

across levels of Mode C (e.g., across K stimuli). In TSF2, the rth sequential factor

underlying the jth level of Mode B and the kth level of Mode C is assurned to be shifted

by a distinct amount, s,b. In other words, shift pararneters are independent across each

combination of levels f?om both non-sequential modes as well as across factors. TSF:! is

more general than TSF1 ; TSF1 can be considered as a special case where the K J-by-R

slabs in the "three-way" shid parameter array S are constrained to be identical, thus,

TSF2 subsumes TSF1 . However, TSF1 is simpler and since it has fewer mode1

pararneters, the ratio of data degrees of fkeedom to mode1 parameters is higher and hence

better in TSF 1 than inTSF2.

The third three-way Shifted Factor mode1 (TSF3) c m be defined by açsuming that

the shifis result from two independent effects: the fint due to one non-sequential mode

(e-g., the electrode location mode), and the second due to the other non-sequential mode

(e.g., the stimuli mode). Shid values are allowed to vary across levels of both non-

sequential modes but the shifi value for the rth factor at the jth level of Mode B and the

kth level of Mode C is a sum of two shift components from different sources, sjr and sb.

As a result, shift pararneters can be collected in two 2-way shifi matrices: one, SfB, for

Mode B and the other, Stcj for Mode C. TSF3 can be written in scalar form as,

R

14

where si, is added to the row subscnpt i of the sequential mode factor loading a to adjust

the row position shift by an arnount due to the jth level of Mode B. and sb is to adjust the

row position shift by an arnount due to the kth level of Mode C. The parameter s, is the

element in row j and colurnn r of SfB,, and s b is the element in row k and column r of S,o.

Which, if any, of these three-way Shifted Factor models is appropriate for a

dataset will depend on the specific properties of the factors underlying those data. Thus,

it is an empirical question, to be investigated by comparing the results from fitting those

models to a given three-way sequential dataset.

Uniqueness in the Two-wav Shifted Factor Mode1

The "rotation problem", that is, the indeterminacy in fixing the orientation of

factor axes, is one of the most fundamental and potentially controversial issues in the

two-way standard factor model. Harshman (1 994b) has conjectured that the additional

parameters collected in S will help determine the unique orientation of factors underlying

the sequential data, even in the two-way case. Although the shift parameten are

introduced to provide for offsets of factors along the sequential axis, they c m provide

additional information useful in fixing the factor orientation for two-way sequential data.

In this sense, they can be considered an analogue of the third mode factor loadings in

PARAFAC. Thus, the recovered factor loadings of the two-way Shifted Factor mode1

will not be rotationally indeterminate when factors are differentially shifted at each level

of the non-sequential mode, enough to fix the factor orientation.

To show the uniqueness of the two-way Shifted Factor model, it is useful to first

think of the indeterminacy of factor orientation in two-way factor analysis or principal

15

component analysis (PCA). The standard two-way model for direct fitting analysis of raw

score data cm be written in matrix notation as,

where F, A, and E are, respectively, the factor score matrix, the factor loading matrix,

and the error (or unique) matrix not fit by the model. Note that equation 7 is for raw

profile data rather than for the cross product type of symrnetrical data. This is to facilitate

cornparison with the uniqueness property of the huo-way Shifted Factor mode1 which

also involves direct fitting of raw profile data. The rotational indeterminacy of the factor

score and the factor loading matrices can be explained by showing that the sarne X is

recovered no matter what rotation matrix has been applied. Let T be some nonsingular

'rotation' or transformation matrix. We can insert I (or TT-') in Equation 7 to obtain the

same X as follows,

where F and A represent, respectively, the rotated version of F and A. Thus, any

transformation of F and the cornpensating rotation of A leaves the fitted part of the data

X invariant, even though an infinite number of rotated factor solutions are possible.

However, this rotational invariance of the recovered X does not hold in the two-

way Shifted Factor rnodel. Because J differently shified versions of A are required to

produce X, there is no longer only one pair of rotation matrices, i.e., a rotation matrix T

and its inverse T-' as the compensating rotation matrix, to produce always the sarne

16

recovered X as in Equation 8 for the standard two-way model. The rotated version of the

two-way Shifted Factor model can be derived as follows,

where [s,(A)] and 8; represent, respectively, the shifted and then rotated version of A

and the rotated b,. Note that the columns of A in Equation 9 must be shifted first by the

amounts defined in the j th row of S before being rotated. Because the shifted version of A

[ s,(A)] holds for only the jth colurnn of data X and the jth row of B, it is not possible in

the two-way Shifted Factor model to keep the fitted part of data X invariant by applying a

pair of rotation matrices, Le., T and T-'. Because the shifted versions of A are not the

same across levels of Mode B, the idea of rotation (by only one pair of rotation matrices)

does not make sense in the two-way Shifted Factor model. in other words, it is not

possible to rotate a single A (not multiple versions of A each of which is for the

corresponding row of B) and the whole B (not b,) in the two-way Shifted Factor model.

The differential shifting of A makes it impossible to rotate the factor loading and score

matrices while keeping the fitted part of X invariant, and consequently gives a unique

solution. Thus, the two-way Shified Factor model has an important additional property,

"miqueness" of the orientation of factor axes in addition that it adjusts the row position

ofiset of sequential factors.

Note that Molenaar's dynamic factor analysis is also able to provide for the

position offsets of factors along the sequential axis by assurning that the contribution of

the latent time series factor to the data X at time t penists during several subsequent

17

times r, r+I, . . . to Wear itself out. As a result, the factor contribution to the data at time r

is a sum of a few previous factor contribution at t, t-1, . . . Because factor loadings are

supposed to vary depending on different amounts of lags, the recovered data by dynamic

factor analysis can be considered a surn of the factor contribution across several

subsequent times. The rotational invarince of the recovered data X by dynarnic factor

analysis can also be written in the sarne way as Equation 8, as follows,

O

where S is the maximum nurnber of lags (or shifts). F, and As represent, respectively, the

rotated version of Fs and A, (respectively, factor scores and factor loadings at time s).

Thus, any transformation of F,'s and the cornpensating rotation of As's will leave the

fitted part of the data X invariant as is the case in the standard two-way factor analysis.

Likewise, the provision for and estimation of such row position shifts will be able

to help resolve the rotation problem in some three-way sequential data particularly when

the sequential factors are differentially shified along the sequential mode and when the

systernatic proportional factor variation in the third mode is too weak to determine

uniquely the factor orientation. However, note that the shifi patterns must be distinct from

one factor to the next.

ALGORITHM TESTTNG AND DEVELOPMENT

Clearly, when the underlying structure of a dataset involves shif'ting latent factors.

standard factor analysis will not be suitable. New factor estimation methods are needed;

ones that incorporate d l of the parameters of the Shified Factor model. Two different

types of estimation methods have been tested here: QALS (Quasi AItemating Least

Squares) and nonlinear optimization (OPT). ALS is prefixed with "Quasi" because the

row position shifi of the underlying sequential factors makes the parameter estimation for

the sequential mode an approximate but not an exact least squares solution, conditional

on the other set of parameters B. This issue will be discussed in the following sections in

detail. To arrive at the current version of the QALS program, a series of algorithms have

been developed in a cumulative way; thus, the final version of the QALS prograrn

includes al1 the developed feahires. In addition, initial versions of the OPT programs have

been developed to implement the Shifted Factor models by another estimation rnethod. A

versatile matrix language program MATLAB (e.g., see MathWorks, 1996a and 1996b)

has been used for both the QALS and the OPT programs.

The altemating least squares (ALS) method is a popular and straightforward

technique to estimate multiple sets of unlaiown parameters. That is, one part of the

unhown pararneter set is estimated at a time by using regular regression estimation,

given the curent estimates of the other set(s) of variables5. Each set of unknown

variables is altemately estimated, updating the estirnate of some part of the unknowns at a

time. This altemating estimation keeps iterating until the iterative solution reaches a

prescribed stopping criterion. One of the most important properties of the ALS estimation

is that the estimate of each set of unlmowns is always a least squares (i.e., multiple

regression) solution given the data and the cwen t estimates of the other unknowns.

For the standard two-way factor analysis, ALS altematively estimates factor

scores and loadings. The alternating idea of the estimation of loadings on each mode

holds for the Shifted Factor models. However, the position of sequential factor loadings

along the sequential axis must be adjusted before using them as the fixed parameters in

the regression estimation. in sirnilar fashion, the columns of the data must be shified

before the data are used to estimate the sequential factors.

Altemating Estimation for the Shifled Factor Models

Harshman (1 997) suggested the following three features of the estimation

procedure for the two-way Shifted Factor mode1 in terms of the alternating least squares

method. These features are what have to be adapted to the ordinary ALS factor analysis

program;

1. The loadings for the sequential mode (Mode A) are subject to position shifts

along the sequential axis when it is used in the estimation of rows of non-

ALS is used for multilinear models, with each subset of parameters chosen so that the estimation problem is Iinear for those parameters when the othen are held fixed (Carroll & Pruzansky, 1984)

sequential mode (Mode B);

2. The data X is subject to position shifts along the sequential mis when used in

the estimation of colurnns of A;

3. There is a new step in the major iteration that estimates elements in S, the size

of the shifis for the latent factors of Mode A at each level of Mode B.

When estimating the R factor loadings for a specific level of the non-sequential

mode (e-g., b,,, . . ., bjR, giving the relative importance of the factors at a specific electrode

location), a differentially shifted version of A should be used as the predictors, Le., the

fixed (or independent) variables in the regression equation. Consequently, each row of

loadings for the non-sequential mode (i.e., each row of B) must be estimated individually

by using a version of A that is shifted for that row so that it will give the least squares

solution for that row of B and the corresponding colurnn of X. Conversely, for the A

estimation, colurnns of the data X must be differentially "mshifted" to be aligned in the

same row position as a particular colurnn in A, where the proportionality of the effect of

factor weights is restored. Now, each colurnn of A must be estimated at a time because

the amount of shifting is not the same across factors. These two altemating estimations

iterate until the QALS solution reaches a prescribed stopping criterion.

Brute-Force Estimation of Shifis

As mentioned in Harshman (1 997), the estimation of the size of shifts for a

specific row of S requires a different type of estimation procedure from the ordinary

regression method. As a crude exhaustive search type of estimation, a "brute-force" (BF)

method has been developed to estimate the size of shifls. The BF estimation is exhaustive

3 1

and simultaneous with B estimation in the sense that al1 possible combinations of shifts in

a given range are tried when estirnating each row of B. For example, when the

predetermined range of allowed shifi candidates is From -5 to 5 ( 1 1 integers) and the

number of factors to extract is 2, the number of al1 possible combinations of shifis to try

for the estimation of one row of B becomes 121 (= 1 1'). The corresponding row of S is

then updated with the best combination of shift values, namely the one that gives the least

sum of squared residuals for x,, the corresponding colurnn of X. The corresponding row

of B, bj is also updated with the resulting regression weights given the best combination

of shi fis, the fixed A, and x,.

Poiishing the Sequential Mode Estimation Procedure

When estimating the rth column of A, unshifting the columns of X by the amount

defined in the rth column of S is not enough to estimate precisely a colurnn of A. This is

because only the contribution of the rth factor is subject to the position shifts by the

amounts given in the rih colurnn of S; the contributions of other factors are shified by

different patterns of shifts, as given (approximately) by the values in different columns of

S. Thus, it is necessary to partial out the contribution of the other factors before

unshifling the data to restore each column to the "true" row position as in A. The

contribution of the rth factor X, is then estimated by subtracting the other factors'

contribution fkom X.

However, the estimate of the rth column of A, a,, is still not an exact least squares

solution. During the intermediate iterations, the estimate of Xr (a "polished" version of X

for the rth factor's contribution) includes some systematic variance due to the other

37 -- factors as well as the error variance. This is because inaccurate estimate of shifis during

these iterations c m cause the estimate of the rth factor contribution to be confounded

with the contribution of other factors. Likewise, some of the systematic variance due to

the rth factor can also be subtracted fkom X. This problem in the A estimation of Q M S

prograrn has not yet been overcome. Nevertheless, it does not prevent a perfect solution

fiom error-kee data. The inaccurate estimate of A violates the property of the iterative

ALS estimation: monotonie improvement of mode1 fit (reduction in sum of squared

residuals). This is also the reaçon why "quasi" has to be attached in fiont of ALS for the

name of the ALS type of Shifted Factor programs.

Fractional Line Search for the shift estimation

Another estimation procedure for shifts, called Fractional Line Search (FLS), has

been developed to increase further the mode1 fit. The purpose of FLS is to allow the

estimate of shifis to be fractional values rather than only integer values as assurned in BF.

where they are the numbers of rows to shift along the sequential axis. Considering that

sequential factors usually refiect processes that are continuous along the sequential axis

in nature or in the population, the interval (or unit) of shifi and the nurnber of levels of

the sequential mode can be considered as an artificial choice among many possibilities;

selected for the convenience of measurement, by convention, or due to a limitation of the

measuring tools. We c m then consider the sequential values as a sample fiom the

continuous population values, at least in theory. Thus, it makes sense to interpolate the

factor loadings in the sequential mode to obtain estimates for fractional shift values,

23

hopefully, in order to obtain the more flexible shifl estimation and thus B estimation. See

Appendix A for the details about the FLS estimation procedure.

A single shifi value is estimated at a time in FLS rather than a combination of

shifis for al1 factors at the jth column of the data because the combinations of fiactional

shifls are infinite. Necessarily, there is an order effect of factors in ternis of the fractional

shifl estimation and the resulting Mode B estimation. The order effect can be thought as a

cost of the more flexible estimation of the shift values

"Super Iteration" Using: Recurrent BF and FLS

Fit values during the iterations in FLS sometimes oscillated by the every nth

iteration, where n varied case by case. For example, the fit values bounced back and forth

between the sarne two nurnbers dunng the iterations in the analyses of synthetic data

when the oscillating symptom was strong. This symptom seems to be due to both the

order effect of factors in the shift and the B estimations in FLS and the "quasi" least

squares solution of A due to the inaccurate estimate of the rth factor contribution to the

data. Refer to Appendix B for the details about the oscillating symptom and an

interpretation of it.

As a provision for the oscillation problem, a "super" iterative method has been

developed. The program went through higher level iterations, altemating between BF and

FLS fittings. The oscillating or non-converging solutions obtained after a prescribed

maximum number of iterations in FLS are again fed into BF as a starting position. This

"super" level iteration of recurrent BF and FLS resolved most oscillating problems and

sometimes non-converging solutions of error-fiee data in a given number of iterations

24

(e.g., 1000). The super iterative method seems to take a distinct advantage kom both BF

and FLS. BF in the super iteration can estimate shifts without the order effect of factors

because of the exhaustive method of estimation. The solution by FLS is less accurate than

is the solution by BF because of the order effect. Thus. the ' k o n g " or less accurate FLS

solution could be considered as a kind of different initial starting position in the recurrent

BF that gets the BF solution in another path to the final solution. Thus, the less accurate

FLS solution c m give BF another chance to try a better path and then to avoid reaching a

local minimum solution. If the FLS solution is "bad" enough to give BF a chance to find

an efficient path to the grand minimum, the recurrent fitting by BF but with a small

maximum number of iterations will be better than a single BF fitting with a fairly large

maximum number of iterations. The recurrent fitting by BF in the super iteration could be

equivalent to trying a number of different starting positions and then picking the best

arnong them. However, the less accurate FLS solution must not be considered as a totally

different starting position because it is still partly a least squares solution.

Three-wav Alternatinn Estimation for TSF 1

A three-way QALS program has also been developed to implement the sirnplest

three-way Shified Factor rnodel, TSFl. The idea of the altemating estimation works for

the three-way Shifted Factor models in the sarne way as for the two-way Shifted Factor

model. For the ordinary ALS estimation method, see Appendix C. For the estimation of

each row of B, columns of the current estimate of the fixed pararneter set (A and C) must

first be shifted by the amount given in the corresponding row of S. This is because TSFl

assumes that a specific shift si, holds across al1 levels of Mode C but at the jth level of

Mode B and for the rth factor. Thus, B is estimated one row at a time by using the

properly shifted version of the fixed part. However, when estimating the factor loadings

of the other non-sequential mode C, the shifting procedure is not the sarne as that for the

Mode B estimation. The columns in the fixed parameter set of A and B must be

differentially shified according to the levels of Mode B because T SFl assumes the

sequential factors to be differentially shified across levels of Mode B. Another difference

between the C estimation and the B estimation is that al1 elements in C are

simultaneously estimated rather than by individual rows because the amount of shi Ring is

common across al1 levels of Mode C. A is estimated in the same way as in the two-way

QALS. That is, each colurnn of A is estimated separately by using the row-position

corrected version of X that is unshifted appropriately to line up the latent contributions of

factor r. This is done by applying the values defined in the rth column of S to restore the

"true" row position in the data.

Lnitial Test of Nonlinear &timization: Simultaneous Estimation of Al1 Parameters

The QALS Shifted Factor program usually requires a long time to converge. Most

computation time spent in the QALS programs has been for the BF estimation of shifts

because al1 possible combinations of shifts (at each level) must be tried. 'fhe computation

time of the QALS programs is approximately a function of the numbér of the allowed

shift estimates in BF and the number of factors. For example, when the shifts are allowed

to range fiom -5 to 5 (1 1 integen) and the nurnber of factors is set to 3, 133 1 1-by-3 row

vecton of integer shifts must be tried in estimating each row of S and the resulting Mode

B loadings. The three-way QALS program requires a huge amount of computation time

26

in general, particularly, in inverting (by using generalized inverse) the "big" matrix of the

fixed pararneter set of A and C for Mode B estimation in BF. For example, for b,, the IxK

by R fixed pararneter rnatrix (factonvise bi-products of a, and ch) must be inverted and

then premultiplied by x,, hr' times (where N is the number of possible integer shifts) to

get the best least squares solution of b,.

To develop, hopehilly, a much faster program and, more importantly, as another

method to implement the Shifted Factor models, a couple of nonlinear optirnization

programs have been tentatively tested. In addition, one important advantage of using

nonlinear optimization for the Shifted Factor models is that one c m easily impose

constraints on the Shified Factor models that might be appropriate to some sequential

data. For instance, non-negative factors cm be assumed for some chernical or

physiological data based on a relevant theory or in nature. Consequently, the non-

negativity constraint on the model would be suitable to those types of data. Of course, the

constraints on the Shifted Factor models must be carefully applied based on a sound

extemal source of information. If the constraints imposed on the model are correct, they

will make the model more stable in the presence of excess noise or systematic error. They

will also be able to Save some of computation time. More irnportantly, the correct

constraints will also be able to make the estimation more accurate in the sense that a

correct constraint reduces the degrees of keedom of the model to estimate. For exarnple.

if the '?rue" underlying factors are al1 positive, the expected value of the sum of squared

residuals of random positive factor estimates from the positive '?me" factor loadings will

be smaller than that of the unconstrained random factor Ioadings. This is because the

27

unconstrained factor estimates could be either positive or negative so that the expected

distance in the factor space between the unconsaained factor estimate and the tnie

loading is longer than that between the constrained estimate and the true. The expected

distance is equivalent to the square root of the mean square error of factor loadings.

Most optimization programs tested so far simultaneously estirnate al1 parameters,

A, B, (and C in the three-way case) and S by nonlinearly optimizing the parameters to

minimize an objective fûnction, the sum of squared residuals of the whole data matrix X

( ~ r X in the three-way case) rather than a part of parameters as is the case in the QALS

program. The A L S estimation is not simultaneous because the estimation is for one mode

at a time. The estimation is not simultaneous even within Mode A because each column

of A must be estimated at a time by using a correctly polished and then unshifted version

of X. The updated columns of A are then used in the estimation of the other columns of

A, inevitably having an effect on the estimation of the remaining columns of A.

The objective function (Le., sum of squared residuals of a column of the data or

the whole data) is minimized by a numencal 'hi11 climbing' procedure, simultaneous

optimizing al1 parameters of the Shifted Factor model. Details are presented in Appendix

E.

One common requirement in using these rninirnization programs is that the

objective fünction must be continuous. This continuity is a requirement to evaluate the

gradient of the objective function to minimize. It could be appropnate to assume that the

variation of factor loading parameten (A and B) results in continuous function values

because it is always possible to define a value between two factor loadings. Thus, the

28

continuity of the objective function will be guaranteed because the factor weight is

infinitely continuous in theory. However, the continuity does not hold for the Shifted

Factor rnodels as long as the size of shifts are defined to be a number of rows in a given

dataset. However, the linear interpolation of A or X by using the fiactional shifis has

resolved this discontinuity problem. If one c m define a fairly small amount of fiactional

shifl, the resulting surn of squared residuals will be continuous. This is because, given a,,

and bjr for xo, the sum of squared residual of x, is a continuous function of continuos the

fractional shifts, which can be any real number.

ANALYSIS OF SYNTHETIC DATA

Sequentially organized synthetic error-fiee data have been used to determine if

the cornputer programs work, and if the Shified Factor solution is unique. These tests

have been performed with both two-way and three-way error-free synthetic data. The

progressive improvements in algorithmic methods for the QALS technique have each

been tested with the synthetic data, and the changes have been accurnulated to complete

the final version of QALS prograrn. That is, the finally completed QALS program (Super

Iterative QALS with the recurrent brute force and the fractional line search estimations)

utilizes al1 the methods mentioned earlier in the algorithm section of Quasi-ALS. The

completed QALS prograrn also has been used to analyze synthetic fallible data to see

how well the prograrn can recover the parameters when some amount of error is present.

or in other words, to see how robust the Shifled Factor models are against various

amounts of error. The nonlinear optimization programs also have been used to analyze

the sarne error-fiee and fallible synthetic datasets to compare the two types of estimation

methods in tems of both the ability to recover the parameters and the efficiency of the

programs. In addition, three different types of c w e s were used in producing the error-

fiee sequential datasets to see if the shape of sequential factors has a differential effect on

the ease of the recovery of parameters. Because both QALS and OPT are likely to reach

local minima, a few different starting positions have been tried for each analysis.

The standard two-way factor analysis solutions of the synthetic two-way datasets.

and the P W A C solutions of the synthetic three-way datasets have been used as a

reference or baseline for cornparison when evaluating the solutions given by the Shifted

30

Factor models. The reference solutions make it possible to see how much additional fit is

attained, and more importantly, how much the recovery of parameters used to generate

the data is improved by the Shified Factor model over and above that given by the

standard two-way factor analysis or PARAFAC. Single-scored overall fit values such as

R', mean square error, or STRESS (so called '8adness of fit") give valuable information

,\ about how well X recovered by a model resembles the data, but they do not give more

detailed information on how well the pararneters recovered or estimated by the analysis

resemble those used to generate the (synthetic) data. In particular, because of the

rotational problem in the standard two-way factor model, it is often possible to get a

fairly strong agreement between the data and the recovered X, even though the estimate

of factors is certainly far fiom the true parameters which have produced the data. For this

reason, the results bom analyzing the synthetic datasets have been investigated in terms

of not only the rnodel fits but also the agreement between the true pararneters and the

estimates of them.

Two-way error-flee datasets were generated based on the Shifted Factor model. In

other words, differential row position shifts of factors were used in generating the

synthetic datasets. The same error-fiee synthetic data were used to test the successively

developed algorithmic features. The final version of the QALS program (the super

iterative QALS program) both for the two-way Shifted Factor model and for TSF1 was

then used to analyze the synthetic fallible data with varying amounts of error.

Two-way Synthetic Data Analvsis

Data Three different shapes of curves were used when constructing the factor loadings of -

the sequential mode (see Figure 3). The fint set of c w e s , called "Handrnade";

subsequently shortened to "Hand", was generated by continuously mouse clicking 100

times in a two-dimensional geornetric plane ranging from -1 to 1 in both axes. The

resulting coordinates in each axis formed a 100x 1 sequential vector. Equally spaced sine

values were setected to generate the second and the third sets of curves, called "Hump"

and "Sine" respectively. Each hump in the "Hurnp" curves consists of sine values fiom O

to xbut the fint c w e (i.e., the first factor that is named F1 in Figure 3) was multiplied

by 0.5 to give a different magnitude of the factor contribution in the sequential mode. In

addition, the humps in the two factors are located in different levels in the sequential a i s

to make the "Hump" curves clearly different from each other. Non-zero values in the first

"Sine" curve are sine values ranging fiom O to Zxand those in the second from O to x.

Again, the first factor was multiplied by 0.5 for the sarne reason. Even though both the

"Hump" curves and "Sine" curves consist of evenly spaced sine values, they are quite

different fiom each other with respect to the slope of curves. The "Sine" c w e s are

smoother (Le., have more slowly changing factor loadings along the sequential axis) than

are the "Hurnp" curves. The resulting three types of synthetic data have been then used to

explore some possible effects of the shape of the sequential factors on the ease of

recovery of parameters. Of course, al1 the other characteristics in organizing the synthetic

data were fixed to avoid potential confounding of the effect of different shapes with those

"Handmade" curves

"Hump" curves

"Sine" curves

Fimire 3. Three Shapes of Synthetic Curves

by the other conditions, such as number of factors, numben of levels and loadings for

each mode, tnie shift values, and so on.

The number of factors was set to 2 so as to Save the computation time in the BF

estimation. The number of levels (Le., time points or successive values) of the "Hand"

curves was set to 100 and that of the "Hurnp" and the "Sine" curves to 60. The true

loadings of the non-sequential mode, B and the true shifts, S were set to be the same for

the three different types of datasets. The number of levels of Mode B was set to 20. The

true factor loadings of Mode B were sampled from uniformly distributed random

nurnbers ranging between O and 1. The true shift values for the 20 levels of Mode B were

also sampled from the evenly distributed integen ranging from -3 to 3.

The different sets of synthetic sequential data have been further tested at three

different error levels (5%, IO%, and 25%) by adding different amount of error to the

error-fiee rank6 2 synthetic datasets as described above. Normally distributed random

numbers were used as the error part in generating fallible synthetic data. The size of both

the mie and the error parts of the synthetic data was then set to unit variance by dividing

each part by its corresponding "grand" standard deviation before muitiplying relative size

weights. "Grand" is appended to represent the standard deviation of the whole rnatrix of

the mie or the error part. The scale free true and error parts (Le., unit-varianced tnie and

error parts) were then multiplied by the square root of their desired proportions in the

total variance.

6 The rank of a matrix is defined as the maximum number of linearly independent rows (or c01um.n~) of the matrix (Green & Carroll, 1976: p 167).

34

Results. The error-f?ee synthetic data were analyzed first to see if the programs could

recover the true factor loadings and shifi values perfectly without providing any

constraint such as the orthogonality among factors. The QALS program could recover the

true factor loadings and shift values perfectly after applying the polishing procedure for

the A estimation. Even though the BF procedure could sometimes alrnost recover the true

factor loadings and the true shifis perfectly without the polishing procedure (recovery

correlation of more than 0.9), it was only after applying the polishing procedure that the

perfect solutions were obtained. The nurnber of perfect solutions out of 4 and the

averaged recovery correlations of parameters are summarized in Table 1. The standard

two-way factor analysis solutions are also given as a reference to compare with the

QALS solutions.

The recovery correlations of the non-perfect solutions have been averaged across

factors as well as across solutions. The uniqueness property of the two-way Shified

Factor mode1 has been proved by showing that the two-way QALS programs can recover

the parameters (A, B, and S) perfectly as well as the data from error-free data without any

extemal constraint such as the orthogonality among factors. The successive fitting of BF

and then FLS but only once for each (BF-FLS) could not get additional perfect solutions

that had not been perfect when fitting BF once. However, when BF and FLS were used

recurrently (i.e., the Super iterative QALS: Super QALS), the number of perfect solutions

was much increased as shown in Table 1. Thus, the hypothesized distinct advantages of

BF and FLS in Super QALS have been numerically confimed. That is, FLS could give

the following BF a chance to try another "starting position" by aggravating the better

Table 1

Number of Perfect Solutions and Averaged Mode1 Fit and Recoveries of Non-perfect

Solutions

Non-perfect Solutions No. Perfect

Fit ( R ~ ) A B S

PCA

Super QALS

"Hand" "Hurnp" "Sine"

"Hand" "Hump" "Sine"

"Hand" "Hump" "Sine"

Note: BF and Super QALS represent, respectively, fitting only by BF vs. the recurrent fittings by BF and FLS. PCA is for the model fits and the averaged recoveries across factors of the standard PCA. The number of perfect solutions is out of 4 different starting positions. The decirnal point is dropped fiorn the model fits (R') and the recovery correlations for A, B, and S. The model fits and the recovery correlations of the non- perfect solutions are averaged across factors as well as across the non-perfect solutions.

solution kom the previous BF fitting in Super QALS. However, FLS. when used just

once after fitting BF (i.e., in BF-FLS), fit a little additional variance without further

improving the recovery of the parameters. This is the reason why the nurnbers of perfect

solutions by the two QALS fitting methods (i-e., BF and BF-FLS) are compared with

each other in Table 1.

A specific amount of shifi cm have a totally different effect on the factor

contribution when it is applied to two different curves. The more fluctuating the

successive loadings in the sequential mode or the steeper the dope of curves, the larger

the effect of an arnount of shift when applied to the sequential factor. Because the "Sine"

curves are smoother than the "Hump" curves and are Iess fluctuating than the "Hand"

curves, it seems to be harder to find the correct arnount of shifts even though the tme

shifts are the same for al1 types of data.

Based on the results sumrnarized in Table 1, Super iterative QALS was selected

for further testing. Ten different starting positions were tried for each different error level

and different type of data, in order to estimate better how many times Super QALS can

reach the unique optimum solution and to see how much closer the optimum solutions

were to the true parameters. Table 2 surnmarizes the mode1 fits ( R ~ ) and the recovery

correlations of parameters, A, B, and S obtained in these 10 m s . The recovery

correlations are the correlations between the true parameters and their estimates. A lot of

results have been sumrnarized in Table 2 to facilitate more direct and simultaneous

cornparisons across different e m r levels, across the three different types of data, and

with the standard PCA solutions. The column heading "Uniq." represents the number of

Table 2

Mode1 Fits I R ) and the Recoveries of the Two-way Super QALS Analyses

Fit Uniq. A B S Error (%) R~ (Bad) F1 F2 (Bad) F1 F2 (Bad) F1 F2 (Bad)

"Hand" O PCA SF

5 PCA SF

10 PCA SF

25 PCA SF

"Hump" O PCA 879 SF P (902)

5 PCA 845 SF 961 (953)

10 PCA 811 SF 923(916)

25 PCA 711 SF 816(802)

"Sine" O PCA SF

5 PCA SF

10 PCA SF

25 PCA SF

Note: Error (%) means the proportion of error in the total variance. PCA and SF stand for, respectively, the PCA solution and the two-way super iterative QALS solution. The column heading Uniq. represents the nurnber of perfect solutions out of 10. The averaged mode1 fits and recovery correlations of the non-perfect solutions are given in the parentheses next to those of their perfect or best solution. The decimal points are dropped From the mode1 fits and the recovery correlations. The capital letter P represents the perfect fit or recovery.

38

the perfect solutions for the error-fiee data (0%) and the nurnber of the best and

presumably unique irnperfect solutions for the fallible data (5%, IO%, and 25%). The

counts are out of the 10 solutions starting fiom different initial values for the parameters.

The non-perfect (in the error-fkee data case) and the bad non-matching solutions with the

best solutions (in the fallible data case) are averaged across different solutions as well as

across both factors, and then reported in the parentheses. Solutions have been decided to

be identical when they were correlated with each other by more than 0.99. The capital

letter P in the columns for the recoveries represents the perfect correlation.

First of all, the true parameters could, again, be recovered perfectly fiom al1 error-free

data generated by using different types of cuves: always for "Hand" data, 6 times for

''Hump" data, and 4 times for "Sine" data out of the 10 solutions. Al1 mode1 fits by Super

QALS are better than those by the standard two-way PCA. Particularly in the "Hand" and

the "Hump" data cases, most recovery correlations were also much better than those of

the PCA solutions. Thus, the analyses of the fallible data have shown that the two-way

Shifted Factor model can recover factor loadings and scores much better than the

standard two-way Factor analysis does. The recovered factor loadings of the fallible

"Hand" (except for 25% error level case) and the fallible "Hump" data were al1 very

close to the true loadings (at l e s t 0.9 in the recovery correlations). Considering that the

improvement of model fits was not so big, the much better improvement in the recovery

of parameters shows that the super iterative QALS helped the program more in

recovering the parameters than in increasing the model fit. In al1 analyses reported in

Table 2 (across different error levels and across different types of data), the additional

39

fitted variance by Super QALS over and above that by the standard two-way PCA was

6% at best. However, the recovery of factor loadings of the fallible "Hand" (5% and

10%) and "Hump" (5%, IO%, and 25%) data were dramatically improved. This

differential improvement clearly shows that the two-way Shifted Factor mode1 is much

better in recovenng the parameters than in fitting the additional variance from the error in

the data. This is presumably because of the uniqueness property of the Shified Factor

model.

The recovery correlations for the shifts were always much lower than were the

recoveries for the factor loadings. This was at first puzzling but an explanation was found

recently. The lower recovery of shifts have been found to be due to a factorwise (Le.,

within a factor) "mean" offset in the shift estimates. When the recovered shift estimates

are compared with their tme shift values, one finds a few outliers that are quite far from

the their true values while the other shift estimates deviated from the tnie values by a

small constant amount. The mean offsets in the shift estimates seem to have happened in

order to compensate for the outliers, because of an anchoring procedure of the shifi

estimates that is applied in both BF and FLS. This procedure is designed to anchor the

mean of the shifi estimates within a given factor on a constant value. The factorwise

anchoring of the shift values does make sense in the sense that the mean difference of the

shift estimates arnong factors will not change the recovered data as far as the constant

amount of the mean shift is compensated for in the estimate of the sequential factor

loadings (A). This is because the constant shift offset is comrnon to al1 levels of the non-

sequential mode (Mode B) o n that factor. The anchoring procedure takes out the rounded

40

mean of the shift estirnates within a factor rather than the rnean of them because al1 shift

estimates must be integers in BF. Thus, after being anchored, the rounded mean of each

column of the shift matrix S must be zero. The mean offset made the A recovery

correlation looked worse than it was. The anchoring procedure could fix the factonirise

constant offset of shifts, and consequently, correct the underestimated A recovery

correlation without changing the mode1 fit and the recovery correlation for B. This is

discussed M e r in Appendix C.

Note that there were not matching best solutions for the fallible "Hump" and the

fallible "Sine" data: the 25% error level for the "Hump" data and al1 error ievels for the

"Sine" data. This means that the non-matching best solutions are likely to be a local

optimum solution rather than the best global optimum solution, because it is not possible

to Say that a single best solution is unique, which does not match with any of the other

solutions. Considenng that in the results of the fallible "Sine" data, there was only one

non-matching best solution, and that the recovery correlations were smaller in the fallible

"Sine" data than in the fallible "Hand" and the "Hump" data. the difficulty of the "Sine"

c w e s has been again confirmed.

The recovery correlations for factor loadings by Super QALS were smaller in one

or both factors than were the corresponding recoveries by the two-way PCA in the

analyses of the "Hand" 25% error data and of al1 fallible "Sine" data. These worse

recoveries seem to have been partly because of the difficult shape of the "Sine" curves

and partly because of the effect of the enor variance. The low recoveries of the "Hand"

25% and the "Hump" 25% data show the effect of the error variance that causes Super

QALS to have difficulty recovering the parameters.

The general pattern of the number of the perfect (or best) solutions and the

recovery correlations shows that the "Hand data are easier to recover the parameters

than are the "Hump" data, and that the "Hump" data are easier than are the "Sine" data

Of course, the smaller the error variance, the better the solution.

Three-way S-ynthetic Data Analvses

Data. As mentioned in the algorithm section, the three-way QALS prograrn requires a

fairly large amount of computation time in inverting the rearranged "long" fixed part of

bi-products (e.g., aiKk for the Mode B estimation) so many times in the brute force

estimation. Thus, the number of levels of the two non-sequential modes (Le., Mode B and

Mode C) was set quite small in this study: 5 for Mode B and 4 for Mode C. Except for

the true loadings of Mode B and Mode C, and the tme shifi values (because the number

of rows must be the same for B and S in TSFI), ail conditions were fixed in generating

three types of three-way synthetic data based on the simplest three-way Shified Factor

model, TSF 1. The m e factor loadings for Mode B and for Mode C were selected from

uniformly distributed random numbers and the true shifi values fiom uniformly

distributed integers ranging £iom -3 to 3. Unlike the two-way synthetic data analysis,

only one error level(25%) of fallible data was used for the three-way QALS analysis,

again because of the huge computation time of the three-way QALS prograrn. Normally

distributed random numbers were again used for the error part in the fallible three-way

data. The same three types of curves given in Figure 3 were used to generate the three

42

different types of resulting datasets. Thus, six different three-way synthetic datasets were

analyzed: two error levels (error-free and 25%) by three different shapes (the "Hand", the

"Hump", and the "Sine" curves).

Results. The six synthetic three-way datasets were analyzed by using the three-way super

iterative QALS program. Unlike the two-way synthetic data analysis, only four different

starting positions were used for each of the six datasets. Their results are surnmarized in

Table 3 in the same way as in the results of the two-way synthetic data analysis. The

solutions of the proportional three-way factor model, PARAFAC have been provided as a

reference to compare with the three-way Shifted Factor model (TSF 1) solutions. Again,

reported under the colurnn heading Uniq. are the nurnbers of perfect solution for the

error-fiee datasets and the nurnbers of matching best fitting solutions for the fallible

datasets. These nurnbers are out of four solutions that started fiorn different randorn

numben as the initial values for the estimate of pararneters. The remaining non-perfect or

non-matching sohtions have been averaged and reported in the parentheses below the

perfect or best solutions.

The three-way Super QALS program could recover the pararneters perfectly From

the error-free synthetic data: always for the "Hand" and the "Hump" data, and twice for

the "Sine" data out of the four solutions. These perfect solutions show that the shifting

idea also works for three-way data. Even in the f ' l ible case (the 25% error level), the

recovered pararneters are very close to their true values. In particular, al1 the four

solutions of the fallible "Hand" data exactly match the others, giving very strong

Table 3

Mode1 Fits (R') and the Recoveries of the Three-wav Super OALS Analyses

Fit Uniq. A B C S

"Hand" O

25

"Hurnp" O

25

"Sine" O

25

PAR 896 TSFl P PAR 688 TSFl 789

PAR 846 TSFl P PAR 672 TSFl 800

(754)

PAU 963 TSFl P

(918) PAR 746 TSFl 801

(807)

968 865 Always P

955 838 Always 951 981

966 869 Always P

889 871 1 930 970

(834)

Note: Error (%) means the proportion of error in the total variance. PAR and TSF1 stand for, respectively, the PARAFAC solution and the solution of three-way super iterative QALS for TSFl. The heading Uniq. represents the nurnber of perfect (or best) solutions out of 4. The averaged model fits and recovery correlations of the remaining non-perfect (or non-matching) solutions are given in the parentheses below those of their perfect (or best) solution. The decimal points are dropped fiom the model fits and the recovery correlations. The capital let-ter P represents the perfect recovery.

44

evidence that the factors underlying some three-way sequential data can be uniquely and

very successfully recovered by the three-way Shifted Factor mode1 even against a

considerable amount of error. However, note that the P M A C solutions are also quite

close to the tnie values. Considering that PARAFAC has not any provision for the shifts,

the three-way datasets already have a lot of information to detemine the orientation of

factors. This information must be provided by the differential factor weights across Ievels

of the third mode that do not exist in the standard two-way factor analysis. In other

words, the true parts in the two-way and the three-way fallible datasets are not the sarne

in texms of the arnount of information to fix the factor orientation, even though their

proportion in the total variance was fixed to be the same in the two-way and the three-

way fallible data. This is because the PARAFAC solution of the error-free three-way data

(Le., the mie part in the three-way fallible data) is always unique, whereas the two-way

PCA solution would not be unique without the extemal constraint, the orthogonality

arnong factors. Of course, it was the three-way QALS program but not the PARAFAC

program that could recover the parameters perfectly.

The recovery correlations for the shift estimates are again poorer than those for

the factor loadings (A, B, and C). These poorer recovenes must have been partly due to

the mean shift offset as mentioned earlier. The mean shift offset is iikely to exist in the

fallible "Hand" data case, because the solutions are al1 identical and because the

recoveries are perfect for Mode C and aimost perfect (0.999) for Mode B on the first

factor. Thus, the relatively low recovery (0.95 1) for Mode A and the rnuch lower

45

recovery (0.846) for the shifls on the same factor, strongly suggest that a mean shiA

ofEset has made the recovery correlations for Mode A and shifts underestirnated.

The difficulty of the "Sine" data in recovenng the parameters seems to be partly

relieved by an additional source of information to fix the factor orientation. The three-

way QALS program rnay have used to fix the factor orientation the differential factor

loadings across levels of the third mode (Mode C) as well as the differential amounts of

shifting across levels of Mode B. However, the "Sine" curves were still more difficult to

recover than were the other types of curves. The error-free "Sine" data could be

recovered perfectly twice out of the four solutions, whereas the error-fkee "Hand" and

"Hump" data were always perfectly recovered. For the fallible datasets, the best solution

of the "Hurnp" and the "Sine" data is likely to be a local optimum solution because there

was no matching solutions. even though their recovery correlations are very high. To see

if the three-way fallible "Hump" and "Sine" data can reach a unique solution, more

starting positions need to be tried.

The efficiency of the three-way Shifted Factor mode1 (TSFI) appeared in another

way. PARAFAC sometimes needed much more iterations (e-g., more than 500) to reach a

converged solution for the same data than did the three-way QALS program for TSFl

(e.g., within 30). This fact suggests that some sequential data, where differential shifls are

present, make it hard for PARAFAC to find the unique orientation of factors. The very

slowly converged PARAFAC solutions were more likely to reach a local minimum than

were the three-way QALS solutions. Most local minimum P W A C solutions could not

differentiate factors fiom each other, even though their mode1 fits were a s good as those

46

of the global minimum solutions by the three-way QALS program. The PARAFAC must

have required much more iterations than did the three-way QALS program because the

smooth "Sine" curves take less information to fix the factor orientation from the sarne

amount of the shift values than did the "Hand" and the "Hump" curves.

One practical problem in the BF estimation of shifis is that the trial of dl possible

combinations of shifts requires a huge arnount of computation time. in particular, the

three-way QALS program for TSF 1 will require a couple of hours to converge on a fast

(e.g., 200 MHz) persona1 computer when the two non-sequential modes have a relatively

large nurnber of levels (approximately more than 10) and there are more than 2 factors to

fit. inverting the long tentatively fixed matnx of bi-products to estimate a row of

B needed more than 95 percent of the computation t h e spent in the analysis of the three-

way synthetic data.

Nonlinear Optimization Analvsis

The same synthetic two-way and three-way datasets have been used to test the

nonlinear optimization programs for the Shifted Factor models. For the unconstrained

analysis of the synthetic data, one of the nonlinear minimizers built in MATLAB.

"hinu" has been tested. Because basic estimation method is always the same for any

parameters (Le., no matter what the parameter is: a factor loading or a shift), the

computation time was faster in the nonlinear optimization analysis than in the QALS

analysis. The nonlinear optimization program could recover the parameters perfectly

from the error-fkee two-way "Hurnp" data and from both of the error-fiee three-way data.

Thus, the uniqueness property of the two-way and the three-way Shifted Factor models

47

could again be confumed by another estimation method: the nonlinear and simultaneous

optimization of parameters. The parameters were more often recovered perfectly fiom the

tluee-way data than from the two-way data. This is, as mentioned earlier, because the

three-way error-fkee data (and the true part in the three-way failible data) have another

source of information to fix the factor orientation: differential factor loadings across

levels of the third mode.

The results of the nonlinear optimization analyses were obtained by setting a more

lenient stopping criterion of iterations than in the QALS analyses. The reason why the

stopping criterion must be made lenient was that the minimizer "hinu" started to give a

degenerate solution at some point when the stopping criterion was as stringent as that in

the QALS analyses. Consequently, the soliitions of the nonlinear optimization analyses

are less correct than are the solutions of the QALS analyses because a lenient criterion

can allow an insufficiently converged solution. To get more correct (or sufficiently

converged) solutions, a M e r shidy will be required. One potentially better nonlinear

minimizer could be "leastsq", because it has been reported to be more appropriate to

apply to the least squares type of problems than are the other optimizen. In a few very

tentative tests, "leastsq" behaved better than did "hinu".

Two-wav Svnthetic Data Analvsis

Two types of synthetic data (the "Hand" and the "Hump" data) were used for the

two-way nonlinear optimization analyses. Only one level of error (25%) was used as the

fallible data. The results of the two-way nonlinear optirnization analysis are sumrnarized

in Table 4.

Table 4

Mode1 Fits (R') and the Recoveries of the Two-wav Nonlinear Ootirnization Analyses

Fit Uniq. A B S Error (%) It2 (Bad) F1 F2 (Bad) FI F2 (Bad) FI F2 (Bad)

"Hand" O PCA 936 893 591 884 342 0PT2 984 (982) 1 992 873 (927) 786 978 (850) 437 835 (480)

25 PCA 768 888 483 905 289 OPT2 804 (797) 1 838 770 (616) 721 737 (536) 344 525 (442)

"Hump" O PCA 879 340 883 574 976 0PT2 999 (999) 3 P 999 (998) P (999) 940 975 (831)

25 PCA 711 331 884 495 963 0PT2 804 (782) 1 869 940 (672) 9 19 963 (636) 689 9 15 (45 1 )

Note: Error (%) means the proportion of error in the total variance. PCA and OPT2 stand for, respectively, the PCA solution and the two-way nonlinear optirnization solution. The colurnn heading Uniq. represents the nurnber of perfect or best solutions out o f 4 . The averaged model fits and recovery correlations of the rernaining non-perfect solutions are given in the parentheses next to those of their best solutions. The decimal points are dropped fiom the model fits ( R ~ ) and the recovery correlations. The capital letter P represents the perfect recovery.

49

Conservatively speaking, the non-linear optimization program could recover the

parameters perfectly from none of the N o types of error-fiee data. However, three

solutions of the enor-&e b'Hump" data matched out of four soiutions. Furthemore, the

recovery correlations of factor loadings (A and B) were almost perfect: at least 0.999.

Thus, in the three matching solutions, only the estimates of shifts seem to have made the

solutions not perfect. Thus, the nonlinear optimization analyses could confirm "almost"

the hypothesized uniqueness property of the two-way Shified Factor model. Note that

even in the fallible data case, the pararneten were recovered quite successfuIly from the

"Hump" data although there was no matching solutions. The "Hurnp" curves seem to

have behaved better than do the "Hand" curves, in terms of the recovery of parameters

when they were analyzed by the nonlinear optimization program.

Three-way S-ynthetic Data AnaIvsis

As in the above two-way nonlinear optimization analysis, the two types of

synthetic data (the "Hand" and the "Hump" data) were used for the three-way nonlinear

optimization analysis. Only one level of error (25%) was again used as the fallible data.

The results of the three-way nonlinear optimization analysis are summarized in Table 5.

The pararneten could be recovered perfectly from the error-fiee "Hand" data and

almost perfectly (at least 0.988 in the recovery correlations) fiom the error-free "Hump"

data. In the result of the error-fiee bbHump" data, the recovery correlation of the second

factor (0.993) seems to have made the corresponding recovery correlation in Mode A not

perfect. The results of the error-fiee data show that the parameters of three-way

sequential data can be recovered perfectly by the nonlinear optimization method.

Table 5

Mode1 Fits ( ~ ~ 1 and the Recoveries of the Three-wav Nonlinear ODtimization Analyses

Fit Uniq. A B C S

"Hand" O PAR 896 0PT3 P

(972) 25 PAR 688

0PT3 722 (709)

"Hump" O PAR 846 0PT3 999

(998) 25 PAR 746

0PT3 656 (629)

Note: Error (%) means the proportion of error in the total variance. PAR and 0PT3 stand for, respectively, the PARAFAC solution and the solution of the three-way nonlinear optimization analysis. The column heading Uniq. represents the nurnber of perfect (or best) solutions out of 3. The averaged mode1 fits and recovery correlations of the remaining non-perfect (or non-rnatching bad) solutions are given in the parentheses below those of their perfect (or best) solution. The decimal points are dropped fkom the mode1 fits (R') and the recovery correlations. The capital letter P represents the perfect mode1 fit or recovery.

The uniqueness property of the three-way Shifted Factor mode1 (TSFI) has been

confirmed again by the nonlinear optimization analysis.

Note that the recovery correlations of the three-way fallible data are worse than

those given by PARAFAC. They are not even unique because there is no matching

solutions. This undesirable result suggests that the current nonlinear optirnization

program requires fùrther study to be used properly. Considering that the QALS solutions

of the same fallible three-way data were always better than the correspondhg PARAFAC

solutions. the nonlinear optimization programs seem not to be used correctly. Othewise,

to optimize nonlinearïy and simultaneously the parameters of the Shifted Factor rnodels

may not a proper estimation method.

52

AN APPLICATION

Field and Graupe (1991) successhilly showed the shift problem of sequential

factors along the tirne axis in their evoked potential data analysis. Their data would have

been a nice exarnple which would be appropriate to fit the Shifted Factor model.

Unfortunately, their evoked potential data were not available for this study. Instead. a set

of evoked response measures7 in the rat brain was used for this study.

The evoked potential measures were obtained every 0.1 millisecond after an

elecûic pulse during 20 milliseconds, consisting of 200 levels in the time mis. The

evoked potential measures were M e r obtained in various different conditions (e.g.,

different paired-pulse intervals, different voltage levels of the paired pulses, and a dmg

administration), but at the same 200 time points. Al1 the different measuring conditions

formed 25 levels of Mode B. Only the measures at the first 80 time points were used in

this analysis, because the Mode B weights of the 25 raw data curves looked not

multiplicative in the later levels of the time mode, resulting in an 80 by 25 two-way

sequential dataset. The 25 data curves are plotted in Figure 4, where the vertical and

horizontal axes represent, respectively, voltage levels (pl) and the 80 time points.

Factor loadings in the measuring condition mode (Mode B) were constrained to

be non-negative because it would be reasonable to assume that the underlying relative

physiological influence (Le., the factor weights) on the measuring conditions is not

negative. Ln other words, a relative physiological influence under a specific measuring

7 It is gratefully appreciated that Dr. Stan Leung provided the data without any hesitation for this study.

Figure 3 . The 25 Evoked I>oteiiiial Data Curves The vertical axis represeiits t lie voltase @A) of the evoked pot eiitials. l'lie tiorizoiital axis represenis the successive t ime poiii~s (0.1 ins) aiter ilie electric pulse.

condition can be interpreted as a magnitude of a physiological event. Thris, it will be

reasonable to think that the magnitude of the latent event underlying the evoked

potentials varies fiom zero to positive infinity. As mentioned in the algorithm section,

one of the MATLAB built-in optimimrs "nnls" was used in order to constrain the Mode

B loadings not to be negative. As in the synthetic data analyses, the standard two-way

PCA solution of the evoked potential data has been used as a reference to compare with

the constrained QALS solution. Two factorss were suggested to underlie the electric

potential data evoked in the rat hippocarnpus in an informal discussion. To see if the two-

factor solution is most reasonable and interpretable, a few three-factor analyses have been

tried. However, the third factor in the three-factor solutions was highly correlated with

the first factor. The third factor seemed to share a considerable amount of its variance

with the first factor. Thus, two factors were decided to be the most reasonable nurnber of

factors to extract fiom the evoked potential data. The following result is based on the best

interpretable two-factor solution out of a few two-factor solutions that started fiom

different random positions.

The mode1 fits ( R ~ ) were 0.9740 and 0.9893, respectively for the PCA solution

and for the constrained QALS solution. Even though the additional fitting by the two-way

Shifted Factor model is not big (0.01 53 in R ~ ) , the recovered curves (factors in Mode A)

by the Shifted Factor model look quite different from those by PCA. The recovered

curves are reported in Figure Sa (the constrained QALS solution) and Figure 5b (the two-

8 A related discussion about the theoretical components and the nurnber of the components underlying the evoked potentials in the rat brain c m be found in Leung (1 978).

a. Recovered A by tlie two-way constrained QALS

b. Recovered .A by the two-way PCA and onhogonally rotated A'S, by 6, 10. and 30 degrees clockwise

Fisure 5 Recovered Sequeiitial Factors (Mode A) by the Two- way Shified Factor hlodel and by tlie Standard Two-way PCA. "F 1 -O" and "F3-0" in panel b stand for the unrotated PCA solut ion.

56

way PCA solution). The unit of the factor loadings is the sarne as that of the data curves

because the scale of the data is reflected in the time mode (Mode A). In the unrotated

PCA solution (FI -0 and F2-0 in Figure 5b), the orthogonal fint factor seems to account

for the general pattern of the 25 data c w e s . In other words, most covariation of the data

seems to be captured by the first component. The dominance of the first factor has been

also confirmed by the relative size of the factor contribution (Le., the variance accounted

for by each factor) in those two solutions. In the unrotated PCA solution, the variance

accounted for by the first component was approximately 11 times larger than that by the

second factor. However, the first factor accounted for an approximately 5 times larger

variance than did the second factor, in the constrained QALS solution. In Figure Sa, it is

easy to see that the first factor accounts for the slow and overall event, whereas the

second factor accounts for most of the fast and early event. Thus, the two-way Shifted

Factor model could better differentiate the two factors than could PCA. The PCA solution

has been orthogonally rotated by applying various degrees to see if the constrained QALS

solution is simply a rotated version of the PCA solution. However, any orthogonal

rotation could not have the PCA solution resemble the constrained QALS solution. In

Figure 5b, three rotated PCA solutions are plotted with the unrotated PCA solution;

clockwise rotations by 6, 10, and 30 degrees (respectively, represented by -6, -10, and -

30). None of thern resembles the recovered c w e s by the Shifted Factor model, showing

that the Shifted Factor solution is qualitatively different fiom the PCA solution.

The factor loadings of the measuring condition mode and the shift estimates are

provided in Table 6. The 25 measuring conditions are combinations of three aspects:

Table 6

Factor Loadings (BI on the Measuring Conditions and Shift Estimates (S)

Intensity Interval Drug W) (ms) WN) F1 F2 FI F2

Note: n i e column heading "Intensity (@)" represents the voltage of the electrical stimuli, which evoked potentials in the rat brain. The second heading "Interval (ms)" represents the time interval between the paired pulses. The next column heading " Drug (Y/N)" represents whether or not the dmg (propofol) is adrninistered. S and B stand for, respectively, the shift estimates and the factor loadings in the measuring condition mode. Bold characten are used to distinguish the measuring conditions after the second electric pulse of the paired pulses from those after the first pulse. The factor loadings of the second factor (syrnbolized F2) measured after the dmg administration are underlined to distinguish from the loadings for the other conditions.

58

intensity of the electric pulse, time interval of the paired pulses. and a drug administration

(propofol). The second pulse curve d e r 300 ,UA with 30 rns between paired pulses was

excluded fkom this study because its shape looked abnorrnal compared to the other

curves. That is why the condition of 300 @ with 30 ms interpulse interval has only one

level in Table 6 (row 19). The factor loadings in the measuring condition mode must not

be compared between the two factors (F1 and F2 in Table 6). This is because the relative

importance of each factor is reflected in the time mode, resulting in the scale-free factor

loadings in the measuring condition mode. Thus, the factor loadings given in Table 6

must be used only in comparing the relative importance of the 25 measuring conditions

on each factor. The pattern of the shift esthates shows that there is a systematic

difference between the fint and the second pulse conditions. Bold characten are used to

the second pulse conditions from the first pulse conditions. Most shift estimates are

positively shifted when measured after the second of the paired pulses, whereas they are

negatively shifled after the fint electric pulse. This systematic difference of the shifiing

direction is the sarne for both factors. Thus, both factors can be interpreted to be delayed

when the data were measured after the second electric pulse. They arose earlier when

measured afier the first electric pulse. With respect to the factor loadings in the

measuring condition mode, the second factor (the fast and early event) more strongly

appeared when the h g was administered than did it without the dmg. The loadings of

the second factor for the post-dmg conditions are underlined to distinguish from the other

Ioadings. On both factors, factor loadings are larger at the second pulse conditions than at

the first pulse conditions.

First Pulse Conditions 800 1 1 1 1 1 I I

600 -

400 -

-200 1 I I 1 I 1 !

10 20 30 40 50 60 70 80

Second Pulse Conditions 800 1 1 1 1 I 1 T I 1

Figure 6. The Data Curves ir i Each Condition of t l ie Paired Pulses

'ïhese systematic patterns of the shift estimates and the factor Ioadings in the

measuring condition mode have been reassured by comparing the data c w e s between

the paired pulses. Data c w e s are plotted in each of the pulse conditions in Figure 6. It is

not difficult to see the different positions of the two events along the time a i s .

Furthemore, the data curves in the second pulse condition look higher than in the first

pulse condition. This size difference between two pulse conditions confirms the

systematic overall pattern of both factor loadings in the measuring condition mode.

Unfominately, the two-way Shifted Factor model could not recover successfully

the suggested theoretical factors. In particular, the onset time of the two factors should be

in opposite direction to that of the two factors recovered by the two-way Shifted Factor

model. The f a t and early event has been supposed to start later than the second factor

recovered by the constrained QALS (F2 in Figure 5a). The slow and overall event has

been supposed to start from the very first level in the time mode, unlike the first factor

recovered by the constrained QALS (FI in Figure Sa). The failure of recovering the

theoretical factors might be partly due to "shape change" problem of the sequential

factors across levels of the non-sequential mode. That is, the sequential factors

underiying the evoked potential measures seem to change their shapes as well as to be

differentially shifted across levels of the non-sequential mode. The "shape change"

problem seems to be quite common to sequential data where the sequential factors are

supposed to be differentially shifted. In other sequential data (spectral data fiom

chemistry), the "shape change" problern of the sequential factors appeared more clearly.

In order to take care of this problem, a time warping method has been suggested. By

6 1

applying the time warping method to the Shified Factor models, the sequential factors

will be able to change flexibly their shape across levels of the non-sequential mode.

Further study is required to adopt the time warping method into the Shifted Factor

models.

DISCUSSION

The shifting of factors does not irretrievably destroy or degrade information. With

the proper algonthm, it is possible to recover the original factors and the shifts fiom the

shifted mixtures that make up the surface data. This has been successfully tested by

analyzing the error-free synthetic datasets by both the QALS and the nonlinear

optimization prograrns. The two-way and the simplest three-way Shifted Factor mode1

(TSF1 ) cm perfectly recover the model parameters from most random starting positions.

Furthemore, the cornparison of the Shifted Factor solution with the PCA or the

PARAFAC solution clearly shows that the Shifted Factor models provide more

improvement in recovering the parameters rather than in increasing the model fit when

there is a significant amount of error. This ability of the Shifted Factor models was

clearer in the two-way analysis than in the three-way analysis. This is because

PARAFAC already has another source of information to recover the parameters that the

two-way PCA does not have: the distinct factor variation across 1eveIs of the third mode.

Thus, it can be said that the differential shifting of the sequential factors that provides

additional information allows the two-way Shifted Factor mode1 to recover the

parameters.

The hypothesized uniqueness property of the Shifted Factor models has been

numerically demonstrated by analyzing synthetic data. The perfectly recovered Shifted

Factor solutions starting from a few different random starting positions demonstrate that

the Shifted Factor solution is unique and that the prograrns implementing the Shifted

Factor models work. Furthemore, the Shifted Factor solution c m be considered

63

"intrinsically" unique, because the uniqueness of the Shifted Factor solutions is provided

by the differential shifts imbedded in the sequential data rather than by an extemal

constraint such as the orthogonaiity of factors as is the case in the two-way PCA. With

fallible data, the likelihood of a unique global optimum is suggested by number of

matching solutions. Even though the number of best fitting and matching solutions varies

depending on the amount of error and the shape of sequential factors, many recovered

parameten from fallible data matched with others. This fact shows that the best fitting

and matching solutions are local optima rather than arbitrary alternative solutions From an

infinite set of alternatives. For the two-way PCA, this would not happen even with

fallible data. Taking account of the mean shift offset, the matching solutions would be

more numerous than reported in the results.

To avoid underestimating how much the estimates of sequential factors resemble

their true values. the estimates of factor loadings in Mode A can be adjusted by taking the

mean offset out. The recovery correlation for shifts c m also be corrected by weighting

each shifi estimate and its tnie value by the Mode B weight at that row. This is because

the effect of a shift on the recovered data is a fùnction of factor loadings on the non-

sequential mode. In other words, the contribution of an amount of shift varies according

to the factor weight on the non-sequential mode at that row.

When interpreting the estimate of shifis, one should note that shifi values are not

the same in nature as the multiplicative factor weights (Le., factor scores and loadings) in

the standard factor analysis models. They must be interpreted in connection to the

specific loading pattern of the corresponding sequential factors (particularly with respect

64

to the shape of the recovered sequential facton) and the corresponding factor loadings in

the non-sequential mode. The contribution of a specific amount of shifting to the fit or

residuals is a function of two variables: the degree of change of the adjacent factor

loadings in the sequential mode (Le., the shape of the sequential factors; in particular, the

average absolute steepness of dope of the curve) and the size of the factor loading at that

level in the non-sequential mode (i.e., the corresponding Mode B weight). The shift

parameter is also different f?om a standard factor loading in the sense that it does not

have the multiplicative (or proportional) relation with other Ioadings in reproducing the

data. Because of these distinct properties of the shift parameter, the interpretation of the

Shified Factor solution must not be made in the same way as for the solution of the

standard factor analysis. For the same reason, it must be guaranteed in sequential data

before fitting the Shifted Factor models that the adjacent loadings along the sequential

axis are somehow related to each other. Without this assumption, a recovered curve

(sequential factor) by the Shifted Factor mode1 will be rneaningless.

As shown in the results of the synthetic data analyses, the shape of sequential

facton has an important effect on the recovery of parameters. The more rapidly changing

(i.e., steeper slope of the sequential factor) and/or the more fluctuating the successive

factor loadings are along the sequential axis, the stronger the contribution of shift

parameters will be to differentiating the facton. However, the shape effect must be

interpreted together with the factor loading in the non-sequential mode (Mode B).

In the evoked potential data analysis, the sequential factors were found to change

their shape, as well as their position along the sequential mis, across levels of the non-

65

sequential mode. This shape change problem was more clearly found in a chemical

spectral dataset. If the Shifted Factor mode1 could be modified to account for the

differential shape change of the sequentiai factors, it might have another source of

information to fix the factor orientation as well as become a more general model that can

resolve the shape change "problem". A method cailed "time warping" has been suggested

to allow the sequential factors to differentially change their shape across levels of the

non-sequential mode. The time warping method must be applied in the latent factor level

but not in the observed variable level, as must the position adjustment. The idea of shape

changes in sequential factors must be M e r studied to see if it can be implemented in a

generalized Shifted Factor model.

With respect to the Super QALS program, the maximum number of iterations

must be carefully determined in the subprograms (BF and FLS). If it is too small, BF will

be unable to iterate enough to reach a minimum. However, if it is too large, FLS will be

unable to give the following BF a chance to try another starting position. Thus, a few

different maximum nurnbers of iterations rnust be tried for a specific datasei before

deciding the optimal level of the maximum number of iterations for each subprogram. As

mentioned earlier, to constrain the model could be very valuable in the analysis of some

sequential data. Further study must be devoted to the nonlinear optirnization method of

fitting the Shifted Factor model(s) so that one c m flexibly constrain the model

parameters.

REFERENCES

Carroll, I. D. & Pnizansky, S. (1984). The CANDECOMP-CANDELMC farnily of

models and methods for multidimensional data analysis. In H. G. Law, C. W.

Snyder, Jr., J. Hattie, and R. P. McDonald (Ed.), Research Methods for

Multirnode Data Analvsis (pp. 372-402). New York: Praeger, 1984.

Carroll, J. D. & Arabie, P. (1980). Multidimensional scaling. Annual Review of

Psvcholorzv. 3 1, 607-649.

Cattell, R. B. (1946). The Description and Measurement of Personalitv. New York:

Harcourt, Brace & World.

Cattell, R. B. (1 963). The stnicturing of change by Ftechnique and incremental R-

technique. In C. W. Harris (Ed.), Problems in measurïnn chance (pp. 167- 198).

Madison, Wisconsin: University of Wisconsin Press.

Field, A. S. & Graupe, D. (1991). Topographic cornponent (paralle1 factor) analysis of

multichannel evoked potentials: Practical issues in trilinear spatiotemporal

decomposition. Brain To~oeraphv. 3,407-423.

Green, P. E. & Carroll D. J. ( 1976). Mathematical Tools for Applied Multivariate

Analysis (p 167). New York: Academic Press.

Harshman, R. A. (1 994a). PARAFAC: Parallel factor analysis. Com~utational Statistic &

Data Analysis. 18, 39-72.

Harshman, R. A. (1 994b). Substituting statistical for physical decomposition: Are there

applications for parallel factor analysis (PARAFAC) in non-destructive

evaluation? In X. P. V. Malague (Eds.), Advances in Signal Processine - for

Nondestructive Evaluation of Matenals @p. 469-483). The Netherlands: Kluwer

Academic Publishers.

H a n h a n , R A. (1 997). A Shifted-Factor mode1 for analvsis of seauentially oreanized

data Research Bulletin, Department of Psychology. London, Canada: University - of Western Ontario.

Horst, P. (1965). Factor AnaIvsis of Data Matrices. New York: Holt, Rinehart, and

Winston.

Kniskal, J. B. (1978). Factor analysis and principal components: Bilinear methods. In W.

H. Kruskal and J. M. Tanur (Eds.), International Encyclopedia of Statistics (pp.

307-330). New York: Free Press.

Leung, L. S. (1978). Dynarnic mode1 of neuronal response in the rat hippocampus.

Biological Cvbernetics. 3 1 ,2 19-230.

MathWorks. (1996a). Using MATLAB. Natick: The MathWorks, Lnc.

MathWorks. (1996b). MATLAB: Chtimization Toolbox. Natick: The MathWorks, inc.

Molenaar, P. C. M. (1985). A dynamic factor mode1 for the analysis of multivariate tirne

series. Psvchometrika. 50, 1 8 1 -202.

Press, W. H., Teukolsky, S. A., Vetterling, W. T., & Flannery, B. P. (1 992). Numerical

Recines in FORTRAN: The Art of Scientific Comoutinq (2nd Ed.), (Chapter 10).

New York: Cambridge University Press.

APPENDIX A. Fractional Line Search (FLS) for the Shift Estimation

The purpose of FLS is to allow the estimate of shifts to be fractional values rather

than only integer values as assumed in BF, where they are the nurnbers of rows to shifi

along the sequential mis. FLS interpolates the factor loadings in the sequential mode

(Mode A) in obtaining estimates for fractional shifl values in order to increase further the

mode1 fit.

Once the iterative solution of BF has become sufficiently stable (changing less

than some critical level fiom one iteration to the next), the converged final solution is fed

as a starting position into the FLS estimation. FLS has a more stringent stopping criterion

than BF does. FLS is to find the least squares solution for each row of B that gives a

M e r minimized sum of squared residuals of the corresponding column of X. Unlike

BF, FLS estirnates only one element in a row of S and B (e.g., sjr and bjr) at a time given

the current values of the other shifls and the Mode B loadings in that row (e-g., the jth

row). This is obviously necessary because al1 possible combinations of fractional shifts

for a row of S and B are infinite.

The FLS estimation proceeds as follows. When the shift estimate at iteration i in

FLS is s j r , for the jth level of Mode B and the rdi factor, three shift values (s-0.5, s,

s+0.5) are tried in the next iteration t+l to find the best shift that will give a further

rninimized surn of squared residuals for x, but given that the other (R-1) shift values and

Mode B loadings in the same jth row are fixed. If one of the boundary shift values arnong

the three candidates (i.e., s-0.5 or sM.5) gives the smallest sum of squared residuals of x,

than that given by the middle value s, the shifi and the Mode B loading, respectively,

69

s,rtr+,, and bjrff+,, are updated with that boundary shifi value and the resulting estimate of

Mode B weight. When sjrff+,, is updated, the interval of the three shift candidates to ny in

iteration r+2 gets reduced to half. The decreasing interval is to restrict the possible range

of the final fractional shifi estimate in FLS from s-l to s+l where s is the initial integer

shift fed in FLS. Of course, the middle candidate d u e in iteration r+2 must move to the

updated shift in iteration t + l (e-g., s-0.5 or s+0.5). For instance, the three candidates in

iteration r+2 become s-0.75, s-0.5, s-0.25 when the updated shift in iteration r+l is s-0.5.

However, if none of the two boundary values gives the smallest sum of squared residuals

of 4 in iteration r+ 1, the interval of shift candidates does not change in iteration r +2 and

both the shifi and the corresponding Mode B Ioading remain the same. T echnically

speaking, the resulting three sums of squared residuals of x, are successively "bracketed"

to search the best shift by trying the evenly spaced three shifi candidates. For more details

about the bracketing procedure, see Press et al. (pp. 390-393, 1992). Af'ter the shifi and

the B estimation, the fiactional part of the fiactional shift values is used in linearly

interpolating the values in columns of A (or X when unshifting the data matrix) that are

already shi Red or unshifted by the integer part of the fractional shifts.

The allowed range of shifl estimates during the whole iterations in FLS is

constrained not to go beyond the integer boundaries (s-1, s+ 1 ), where s is the initial shift

value given from BF. This restriction is to keep the best shifi estimates in the integer

level, which have been estimated by the exhaustive trial-and-error in BF. Note that a

single shift sjr is estimated at a time in FLS rather than a combination of shifts for al1

factors at the jth column of the data, sj as it is the case in the BF. Necessarily, there is an

70

order effect o f factors in terms of the hctional shifi estimation and the resulting Mode B

estimation. The order effect c m be thought as a cost to the more flexible estimation of the

shifi values so as to fit the data over and above the fitting in BF.

APPENDK B. Oscillating Mode1 Fits in FLS and Super QALS

Fit values during the iterations in FLS sometimes oscillated by the every nth

iteration, where n varied case by case. For example, the fit values bounced back and fonh

between the same two nurnbers during the iterations in the analyses of synthetic data

when the oscillating syrnptom was strong. This syrnptom seems to be due to both the

order effect of factors in the shift and the B estimations in FLS and the "quasi" l es t

squares solution of A due to the incorrect estimate of the rth factor contribution to the

data.

Let suppose we have a simple example with 2 factors to extract. In iteration r in

FLS, each elernents in the jth row of S and B is estimated given A fiom iteration t - 1.

However, the second element in the jth row of S, s,? and the resulting Mode B loading 6,'

are also a function of the first elernent of shifis s,, and its Mode B loading b,, because of

the order effect of factors in FLS. Then in the same iteration, Mode A estimates are partly

a function of the corresponding shifts as well as Mode B loadings because of the

polishing (or subtracting) procedure for each factor contribution before estimating each

column of A. Of course, the order effect of factors arises not only in the shift estimation

but also in the size loading estimation (i.e., Mode B and Mode A loadings). If the

improvement of factor Ioadings is very small across iterations as is the case after a few

hundred iterations in the ordinary least squares analysis, synthetic data analyses, the

bouncing iterative fits might be mainly due to the order effect in the shifi estimation and

the polishing procedure in the Mode A estimation. Note that the Shified Factor mode1

always needs to shifi colurnns of A whenever producing or estimating a column of

sequential data, and the estimation of each factor contribution is also not an exception.

One possible interpretation o f the oscillating iterative fits and agreement with the

true parameters is as follows. In iteration t , the first shift in the jth row sjr, will have an

effect not only on the estimation of the second shift s,2(, but also on polishing X for the

second factor estimation of A. nie updated second factor of A, a ~ ~ ~ ) will influence the

estimation of the second shift in iteration t+ 1. This second shifi in iteration r+ 1

will then influence the estimation of the fint shift in iteration t+2, sj,/,+~,. Although the

shift estimation is always to minimize surn of squared residuals for a specific column of

data it is given the fixed shift for the other factor. Thus, to minimize the s u m of squared

residuals of a column of data is not necessarily compatible with the rninimization in the

least squares sense. The minimization in FLS always depends on the fixed shifi for the

other factor@). ïh is type of weak minimization in FLS can result in a pair of bouncing fit

values and agreement values with the rnie loadings, which might happen through the

influence path described above. This kind of oscillating or circulating solution c m be

considered as a new type of local minima in the sense thar it will not change across an

infinite nurnber of iterations although it is obviously not a minimum.

As a provision for the oscillation problern, a "super" iterative method has been

developed. The oscillating or non-converging solutions up to a prescnbed maximum

number of iterations in FLS are again fed into BF as a starting position. This "super"

level iteration of recurrent BF and FLS resolved most oscillating problems and

sometimes non-converging solutions of error-fiee data in a given nurnber of iterations

73

(e.g., 1000). The super iterative method seems to take a distinct advantage from both BF

and FLS. BF in the super iteration can estimate shifts without the order effect of factors

because of the exhaustive way of estimation. The solution by FLS is l e s accurate than is

the solution by BF because of the order effect and is "wrong" in a sense. Thus, the

"wrong" or less accurate FLS solution could be considered as a kind of different initial

starting position in the recurrent BF that gets the BF solution in another path to the final

solution. Thus, the less accurate FLS solution can give BF another chance to try a better

path and then to avoid reaching a local minimum. If the FLS solution is bad enough to

give BF a chance to find an efficient path to the grand minimum, the recurrent fitting by

BF but with a small maximum number of iterations will be better than a single BF fitting

with a fairly large maximum number of iterations. The recurrent fittiiig by BF in the

super iteration could be equivalent to tryng a number of different starting positions and

then picking the best arnong them. However, the less accurate FLS solution must not be

considered as a totally different starting position because it is still partly a least squares

solution.

APPENDIX C. Alternathg Estimation for TSF1

The idea of alternating estimation for each mode works for one of the three-way

Shifted Factor models, TSFl. Before descnbing the TSFl program, it will be usefùl to

take a bnef look at how the ALS program works for the ordinary three-way factor mode1

such as PARAFAC. When estimaiing one of the three loading matrices, the three-way

data array must be appropriately reshaped into a two-way big "strung out9' matrix. The

given predictors, which are sets of bi-products (e.g., aiçb for the Mode B estimation) of

the eiements in the two fixed matrices, also have to be arranged so that the row (or

column) position agrees with that of the adjoined data matrix. For instance, the I by J by

K three-way data array c m be reshaped into an IxK by J matrix for the Mode B

estimation by stringing K slices in the three-way data array out one below the other.

Then, the elements in the tentatively fixed part in the regression estimation (an I x K by R

matrix combined by the given but tentative A and C in this case) must be arranged so that

the row position of the fixed part of the parameters matches to that of the strung out

'btwo-way" data matrix. The way of reshaping of the three-way data array and the

required arrangement of the tentatively fixed bi-products hoids for the three-way Shifted

Factor models. Three-way Shi fied Factor models are. though, di fferent from P ARAF AC

in that A must be properly shifted for Mode B and Mode C estimations before combining

the tentatively fixed bi-products and that the strung out two-way data matrix for Mode A

estimation must be properly unshifted before using it in the regression estimation.

Harshrnan (1 994a) described more details about the rearrangements of both the data and

the fixed parameters.

75

For the estimation of each row of B, the columns of the tentatively fixed part of

the parameten ( I d by R rnatrix combined by using A and C) must fint be shifted by the

amount given in the corresponding row of S because TSF1 assumes that a specific shifi

s,, holds across al1 levels of Mode C but at the jth level of Mode B and for the rth factor.

Thus, B is estimated one row at a time by using the properly shified version of the fixed

part. However, when estimating the factor loadings of the other non-sequential mode C,

the shifting procedure is not the same as that for the Mode B estimation. The three-way

data c m be, for example, reshaped into an I d by K matrix. Consequently, the fixed bi-

products (ai&,,, colurnnwise or factonvise combination of A and B) must form an IxJ by

R matrix so that the row positions of the two rearranged matrices match each other.

Because TSF1 assumes that the shift amount varies across levels of Mode B, the bi-

products in the fixed part ai&, must be differentially shifted according to the row

subscript of loadings of Mode B. It will be easier to visualize the differential shifiing of

colurnns of bi-products if one collects hem in an I by J b y R three-way array. The shape

of the top surface of the three-way array of the fixed bi-products will be the same as that

of the shift rnatrix S. Thus each colurnn of the rearranged three-way fixed part can be

shifted by the arnount defined in S. Of course, the shifted version of the three-way array

fixed part must be reshaped into an IxJ by R two-way matrix before using it in the Mode

C estimation. Another difference between the C estimation and the B estimation is that

al1 elements in C are simultaneously estimated rather than by individual rows because the

size of shif€ is cornmon across al1 levels of Mode C.

76

A is estirnated in the sarne way as in the two-way QALS. That is, each column of

A (e.g., a,) is estimated separately by using the row-position corrected version of X that

is unshifted appropnately to line up the latent contributions of factor r. This is done by

applying the values defined in the rth column of S to restore the '?me" row position in

the data. Of course, the unshifted version of X for the rth factor must be reshaped into a

proper two-way maûix, for example, an I by JxK matrix to use for the regression

estimation of A.

77

APPENDIX D. Singular Value Decomposition for the A Estimation

As discussed in the section of the polishing procedure, the estimate of A is not an

exact least squares solution because of the inaccurate estimate of the rth factor

contribution in the data. If the factors are orthogonal to the others in both modes. the

contribution of the other factors does not have an effect on the estimation of the rth

column of A. Consequently, the polishing procedure in the A estimation will not be

required in the Shified Factor models. However, the orthogonal factors are not guaranteed

in the Shifted Factor models. This is because the Shifted Factor models do not need to

constrain the factors to be orthogonal in order to fix the factor orientation, as is the case

in the standard two-way factor analysis.

The ordinary regression has been used to estirnate a column of A given the

polished and then unshifted version of X and the rth column of B. Considenng that the

estimate of the rth factor contribution to the data is confounded with some of the other

factors' contribution, it may not be safe to use a column of B as the fixed parameters in

regression to estirnate the corresponding column of A. Because each row of B has been

estimated given the differentially shifted version of A that is not quite correct in the least

squares sense, the estimate of B may also be a little incorrect by the effect of incorrect

solution of A in the previous iteration. If the effect of the incorrect A solution is

sornehow accurnulated across iterations through the subsequent incorrect solution of B.

the solution will diverge from the correct solution more and more during iterations. To

relieve this conjectured "carry-over effect" of the incorect solution of A, singular value

decomposition (SVD) has been adopted for the A estimation. SVD is a similar type of

78

decornposition method to PCA except its different scaling convention. That is, the SVD

solution is the sarne as the unrotated PCA solution except their scaling convention: SVD

collects the relative scale of factors in a diagonal matrix whereas the unrotated PCA

solution reflects the scale of factors in the loading matrix. Thus, both the PCA and the

SVD solution are orthogonal among factors.

The orthogonality of SVD has been supposed to cut the chain, through which the

effect of the incorrect A is canied over, because the SVD solution for A is less related

with the B in the sense that its does not use the column of B (b,) when estimating the

corresponding column of A (a,). However, the SVD estimation for A has not always been

better than the regular regression: SVD was better in the three-way synthetic data

analysis but regression was better in the two-way synthetic data analysis. It, however, has

not been thoroughly studied what the advantage, if any, of SVD for the Mode A

estimation is, and why it happens.

APPENDIX E. Nonlinear Optimization Procedures

In the nonlinear optimization programS. the objective Function (Le., sum of

squared residuals of a column of the data or the whole data) is minimized by a numencal

'hill climbing' procedure, simultaneous optimizing al1 parameters of the Shifted Factor

model. MATLAB built-in optimization programs "hinu" and "leastsq" have been tested

for the unconstrained Shifted Factor analysis of synthetic datasets; and "constr" and

"nnls" for the constrained Shified Factor analysis of a real dataset. The default estimation

methods for these minimizers are Quasi-Newton, Levenberg-Marguardt, Quadratic

Prograrnming, and Sequential Quadratic Prograrnming for "hinu", "leastsq", "nnls", and

"constr", respectively (MathWorks, 1996b). "Leastsq" is reportedly more efficient for the

least squares problem, whereas "hinu" is more general for unconstrained multivariable

functions.

One of the constrained optimization programs, "nnls" estimates one set of

pararneters given the other set of parameters and the data but with the non-negativity

constraint. That is, it gives a non-negative least squares solution. Thus, "mls" is not for

the simultaneous estimation of al1 pararneters but for one mode estimation given the

factor loadings in the other mode(s) a s in QALS. It has thus been used inside the

constrained QALS for the real data analysis. That is, the constrained QALS uses "nnls",

rather than the regular regression, for the estimation of each row of B given a column of

data and the correctly shifted version of A. "Constr", however, is more appropnate to

apply to the simultaneous optimization problem than is "MIS" because it does not need

the known set of parameters (such as the fixed part in the regression equation) as "mls"

80

does. With "constr", it is possible to impose various constraints on the mode1 parameters,

e.g., non-negativity on factor loadings A and B andlor upper and lower bounds on the

shifi parameters S to restrict the range of shift estimates. Thus, "constr" seems to be an

appropriate program to constrained nonlinear and simultaneous optimization problems,

particularIy for some real data.

One common requirement in using these minimization programs is that the

objective function must be continuous. This continuity is a requirement to evaluate the

gradient of the objective fict ion to minimize. It could be appropriate to assume that the

variation of factor loading parameten (A and B) results in continuous fùnction values

because it is always possible to define a value between two factor loadings. Considenng

one element in a two-way data matrix X, .rd, the contribution of factor r, is a bi-

product of the rth factor score at the ith row of A and the rth factor loading at the jth row

of B. Given one of the two sets of R factor weights (e.g., a,), the sum of squared

residuals of.? will be a linear function of the estimate of the other set (e.g., bjr). Thus the

continuity of the objective function will be guaranteed because the factor weight is

infinitely continuous in theory. However, the continuity does not hold for the Shifted

Factor models as long as the size of shifts are defined to be a number of rows in a given

dataset. This linear interpolation of A or X by using the fractional shifis has resolved this

discontinuity problern. If one can define a fairly small amount of fiactional shift, the

resulting sum of squared residuals will be continuous. This is because, given air and b,,

for .TV, the sum of squared residual of xv is a continuous function of continuos the

fiactional shifts, which cm be any real nurnber.

APPENDIX F. MATLAB Prograrns of QALS

The two-way and the three-way subprograms in the super iterative QALS: BF and

FLS are provided, which were written in the MATLAB language (MATLAB ver. 5).

Each pmgram will require a few complementary programs to be run. They must not be

considered completed for a ninning.

iBF2WSIM is the f i r s t subroutine of the 2-way modified ALS shifted + factor program for simulated data. The true parameters Aoriq, 5 3oriq, and Loriq as weil as X should be defined as a global t varianle. It calls for the fractionai ! i n e search sabroutine, * II ; ,s2wsirn.m".

i 5 May 28, 1597, Sunqjin Hong 1.

[nas,nbs] = s i z e ( X I ; ~olBf=.OS;rnaxinum=50;aChangeMax=l;bChangeMax=l; inum=l; L=zeros (nbs, nfs) ; lagmin=-5; lagmax=5; c= lagmin: lagmax] ' ; i f nfs==2 3Generate al1 possi~le combinations of shifts f o r a column of

3x. Lpop=[kron(c,ones(length(c) ,1) 1 ,kron(ones(lenqth(c) , l ) , c ) J ;

else i f nfs==3 Lpop=[kronic,one~(length(c1~2,l)) , k r o n ( k r o n ~ o n e s ~ l e n g ~ h ~ c ~ ,l) ,c) ,. .. ones(length(c),l) 1 ,kron(one~(lengthic)~2,1) ,cl 1;

elseif n f s = = 4 Lpop=[kron(c,ones(length(c) *3,1) 1 , . . .

k r o n ( o n e s ( l e f i g t h ( ~ ) , l ) ~ k r o n ( ~ ~ ~ n e ~ ( i e n g t h ( ~ ) ~ Z , l ) 1 I I . . . kron(one~(length(c~~2,l),kron(c,ones~lenqth~c~,l! 1 1, . . . kroniones(length(c) - 3 , 1) ,cl 1 ;

elseif n f s = = 5 Lpop=[kron(c,ones(length(c) "4,111 ,.. .

kron(ones(length(c),l),kron(~,0ne~(~ength(c1~3,~) 1 1 , . . . kron(ones(length(c)^2,?i,kron(c,on~~,ones~1enqth~c~~2, 1) } 1 , . . . k r o n ( o n e s ( l e n g t h ( c ) A 3 I l ) , k r o n ~ ~ I ~ n e s ~ 1 e n g t h ~ c ~ , l ~ 11, .. . kron(ones(length(c) ̂4,l),c) 1 ;

end outsavefilename=[outfiIenarne, 'b2',start,int2strrunsn , .mat1]; disp ( [ 'Starc of 2-way Brute Force ' , int2str (runsnum) ] ;

while (aChangeMax>tolBFIbChangeMax>tolBF) &inum<-maxinum

if inum>maxinum disp('BF has reached its maximum number of iterations')

end

3 Estimation of S and L f3r j=l:nbs ssqresXj=10n6; EAssigned initial value of sum of squared

iresiduals of xj. r a z i=l: length (lpop)

Alag=laq (A, Lpop (i, : 1 1 ; SShFft A DY usinq a combinat ion of %shifts picked from the matrix of %al1 possible combination of shifts.

bj=X(:, j ) '*pinv(Alag) '; xjHat=Alag'bjl; ~sqresXjO=sum(fX(:,jl-xjHat).~2); %Cornpute sum of squared

$residuals of xj given by applying ith Jcombination of 3shifts.

if ssqresXjO<ssqresXj %If the ith combination gives smaller asum of squared residuals, update the %minimum ssqresx j, j th row of B, and %jth row of shift matrix L.

ssqresX j=ssqresXjO; Bhat(j,:)=bj; Lhat(j,:)=Lpop(i,:);

end end

end

$ PLnchor each colurrn of shift values so chat ics aean is close r c z e r c . laqoffset=rocnd (mean iLhati i ; Lhat=Lhat-ones inbsI 1 ; -l&qoffse:; A=laq (A, laqcffse~) ;Ahac=A;

i Estimation of A BinvPrFm=pinv (Bhat ' ) ; for r=l:nfs

for j=l:nbs Alag=lag (Ahat, Lhat ( j , : 1 ) ; JShift A to get the estimate fo j th

%colurnn of X. Alag ( : , r) =zeros (nas, i) ; %Assign zeros so as partial out the

3rth factor contribution. xjrhat=X ( :, j 1 -AlaqCBhat ( j , : 1 ' ; %Subtracte the contribution of

Sthe other factors from X . s=-Lhat (j, r) ; SAssigr! "-" to al1 shifts for che inverse

$shiftinq of X or undoing the shifting of X . if s<O 5Unshifc X for rth column of A .

XrUnlag(:, j)=[xjrhat(l-s:nasl ;zeros(-s,l)]; else

XrUnlag(:, j)=[zeros(s,l);xjrhat(I:nas-s) j ; end

end Ahac i : , r) =XrUnlaqCBinvPrim( :, r ) ;

end

3Scale A and B so tnat the scale of data goes to A. Bscaleinv=diag (msq (Bhat) . Y-. 5) ) ; Bhat=3hatCBscaleinv;Ahat=Ahat*inv (Bscaleinv) ;

% Fit values Ares=A-Ahat; Bres=S-Bhat;msA=msq (A) ;msB=msq (BI ; AperChange=abs (Ares) 'diag(msA. A ( - . 5 ) ) *lOO; BperChange=abs (Bres) *dia9 (msB. ( - . 5) 1 *lOO; aChange~ax=max(max(AperChange));bChangeMax~ax(max[BperCnange~ 1 ; ssqresL=sum(sum( (L-Lhat) . " 2 ) ) ;

Two-wav FLS

3fS2WSïX F s t h e second subrcucine of the 2-way sh i fced f a c t o r prcqr=T, w h i 5 rs 3 ne s tod Ln rhe f i r s t subroucize D f Z w s i m . 5 3 Ya:l 2 9 , 1 9 5 7 , SungjLs Bcng 4 3 - - . , , , rS=. - 31; ;rnaxinum=5O;aChanqeMax=L;i:ChangeMax=l; Fnum=L; 3=0nes ( * s , 2:s: * . 5; outsavefil2name= [oucfilename, ' f 2 ' , star:, inc2scz ( n n s n u n ) , ' ..mt ' 1 ; d i sp ( [ ' S:ar= of 2-way Fracciocal Li?-e S e a r c h ' , i n t 2 s t r ! rncsnum) 1) ;

i 5 Lnum>rnaxi~un d i s p ( ' F l S has reached ics naximum number 3f iterations . ' )

3 Zst i ,nat ion of 3 f o r j=i :nbs

for r = l : n f s sh i f t vec=Lha t ( j , : ) ; S S h i f t s for j t h column of d a t a . s - sh i f zvec (r) ; % S h i f t for r t h f a c t o r in the jth s h i f t vecEot. d=D ( j , r) ; %?ic!c OUL the correspondinq i n t e r r a l f o r s h i f t

3 for f a c t o r r and column j. s v e c = [ s - d , s , s r d ] ; %Three possible s h i f t s o u t of which one w i l l be

3picked ouc. f o r v=1:3

s h i f t v e c ( r i = s v e c ( v ) ; Alap- lag in ter (A, s h i f t - r e c ) ; % S h i f t A buc i n t e r ? o l a c i n g with the

% f r a c t i o n a l p a r t of s h i f t va lces .

. SI=:<, . :, 1' ' -?iriv ! X a c ' ; x i i a r=Alag*3: '; ssqrosX:uoc ,-Y) =su..? ( (X : , : 1 -x:nat) . ̂ 2 ) ; 3 S m cf square r e s r d u a l s f o r

%=Se t h r e e snif: a l t e z x t i - r e s fcr 3the !ch and r th s h l f z .

FZ min ;ssqrosXjvec: ==ssqresX jvec (1 ) 3 i , ! 5 ; %If rninLpum sum of squarec! r e s i d u a l s i s ;iTzer. by t h e

3 E i r s t s n i f r , reauce the Fncervai t o h a l f =a preven: S it f rom gettzng d ive rge .

Lha= ; 3 , z : =s-d; 3-d chen, update ~ h e snift . e l s e i f e n (ssqresX jvrec) ==ssqzesX jvec :3j

a ( ; ,=;=.5*-; :ha= l f , r : = s i c ! ;

end enc X l a ç - l a s i z t s r ,A, :bat (:, : ! 1 ; 3 h a t : f , : 1 = X : :, : ' *;ix-r ( .Uzg) ' ;

end

3scaiei~v=diag(msq:3hac) - - ( - . 5 ) 1; aha t=ahac-Bsca le inv ; Mat=Ahac'inv(Bscaleinv); %Sca le Ahat and 3 h t .

9 Fit -ralues Ares=A-Ahat; S r e s = M h a t ; L r e s = L - L h a t ; v q (BI ; AperChange=ans (Ares) *diag (msA. A ( - . 5 ) 1 '100; BcerChanqe=abs (Bres) *dFag ( m s B - A ( - .5 1 ) W O ; aChangeMax=rnax (max (AperChange ) ) ; bChangeMax=max ssqresL=sum(sun( (L-Lhatl . ̂ 2) ) ; ssqresLorig=surn (sm( (Lorig-Lhat) . -2 1 1 ; corrAqax ( a b s ( c o r r 2 s e t (Aor ig ,Ahat ! 1 l ;corzB=max f o r f =1: nbs

s h i f t v o c = l h a t (j, : ) ;

33F3SIY-SVD runs the 3-way s u p e r iteritive quasi-ALS shifzed f a c z ~ r 3 ?raqran wlrh simufated data of v a r m g e t r c r Levels :ha= 3 oscinates calumns of A by using svd. j

3 i 9rLqinal: Ju=e 3, 1997, S u n g j F ~ 3cnç 3 Revis~cc: A l q u s t 1 8 , 97. % 3 [nas,zSs,ncs]=size (X!;

+Set i n i r i a i values tolB~=.05;maxiter=50;aChangeMax=I;bChangeMax=~;cChangeMax=l; iternum=l; L=zeros (Anbs, n f s ) ; L h a t = L ; AC3d=zeros (nas, ncs , nfs 1 ;AC3dShif ted=AC3d; .=3d=zeros (nas, ribs,nfs) ;-3dShifted~AB3d; XrVnskifted-zeros (nas, zBsI ncs) ; iagmi~=-5;lagmax=5;~=[1agmin:Lagmax] ';

bRearrange X for Mode 9 and C estimation. XforB=reshape (permuta (X, [1,3,2] 1 ,nas*ncs,nbs) ; XforC=reshape ( X , nas*nbs, ncs) ;

if izermmxnoxi:ez ciisp c ' 3-way 3" has zeached its max F z e r a ï i o ~ rmiber' !

end

3 Zsc.aa-;co - -. L A W A . of 3 arrd 5 f o r k=l:~cs iCsn3ine A and C :O De a nas*ncs*nfs 3 - w a y array as t2.e

3fixed ?ar t . AC?d::, k, : ) =A*diaq:C(k,:)) ;

sr.d f 3 f j=I:n8s

ssqrosXj=1Oa6; f o r s=l:langth!L?op)

f o r r=l:zfs snif==Lpop i s, r) ; i f s,'iift>=c)

AC3dSkif=od(:, :, r)=(zercs(snif:,ncs);AC3d!l:nas-sii1f:, : ) ; ; olse

AC3dShifred(:, : ,r) =[AC3d(1-si:ift:nas, :, z ) ;zeros (-shif=,~cs) j ; a n c

end ADkset=reshape !AC3dShiftedt nas*ncs, nfs) ; bj3atTeap=Xfor3( : , j 1 '*pinv(ADksetl ) ; Xj HatTemp=ADkset*b jHatTempv ; ssqresXjTernp=sm( ( X f o r S ( : , j 1 -XjHatTempl . * 2 ) ; if ssqresXjTemp<ssqresXj %If the sth combination gives a

3smaller ssqres, Lhat(j, :)=Lpop(s, : ) ; %Update jth row of Lhat, ahat ( j , : 1 =b jHatTemp; %Update j th row of Bhac, and ssqresXj=ss~resXjTemp; 3Reset ssqres for Xj with the

%temporal minimur,. end

end

end

3 F i t * ra lues nsA-msq (4 ) ;ms3=msq (3) ;rnsC-msq (Cl ; AperChanqe=a=s (X - .Ua t ) *diag(msA." ( - - 5 ) 'LOO;aChangeMax=mx(mx:A~erChançe) ! ; 3perChznge=aCs (3-3haz) -diâg (=B. A ( - . 5 1 1 L O G ; bChangeMaxtmax (max (3pezChanqe 1 1 ;

C?erCSange=abs ( C-Char! *dia9 (rnsC. ( - - 5 ) '100; cCSanqeMax=max (max (C?erCharqe) :I ;

nsqresL=sun(sum( ( L-5,iat) . ̂2) i / (nbs'nfs) ; a~qresLorig=sum(sum((Lorig-Lhat).~2) ) / ( n b s * n f s ) ; carrA=n~ax!abs (corr2set (Ahat ,Aor ig) ) 1 ; corrBqax (ans (corr2sec (Bhat,Borig) l l ; corrc-max (abs (corr2set (Chat,Corig) ) ) ; f o r j=l:ibs

for r=l:nfs s h i f r = L h a t ( j , r) ; if shif=>=O

A s n i f t e d ( : , r ) = [ t e r o s (shift,l) ;Ahat{I:nas-shift, r) 1; e l s e

Ashiftedi:, r)=IA9at(l-shift:nas,r);zer~s (-shift, 1) 1;

end

3 3 3 3 i S % ? i % b 3 3 3 3 3 3 3 3 % i 3 3 % 0 3 3 3 3 3 END OF 3C3SI:d SVD 3 3 % 3 % t 3 3 4 3 5 3 % 3 3 i % ! ; 3 j ' f 7 1 + 3 3 % i i -

Three-wav FLS

5FS3ST!! ?.KI is the ç e c m d sub rouc ine ~f zhe 3-way X S s h i f t e d f a c t z r - 3 z r z g r m , whic? estimates columns of A by ~ s i n g svd. 3 3 Original: J w e 3, 1997, Sungjin Hong 3 Xe-ris~an: 4ugrst 19, 9 7 . 's 3 - ,,,?S=. - 7 01;maxiter=50; i=er?.um=I; 3 ~ V s e ~ = t e r o s ( 3 , n f s ) ;.ADkrVset=teros ( n a s ' n s , n fs ) *. 5 ; ou~saveEilename=[outfL1enme, ' f3svddr , int2str(nfsj, 'd- ' , . . -

s t a r t T ~ e c ( s t a r ~ ) , Fnt2str (runsnum) , ' .ma t ' ] ; dis? i [ ' S t a r t of 3-way FLS-SVD ' , e r r o r l e v e l , ' % ' , star tvec ( s t a r t l , . . .

'- ' , i nc2sc r (runsnum) 1 ) ; disp i ' No MSC RSQ A chng B chng C chng MSQ L MSQ LO A cor=

a c o r r C c o r r ' ) ;

if i~ernum>rnaxi ïer d isp( '3-way FLS has reached its i t e r a t i o n m a x ' i ;

end

3 Ancnor each z o i ~ n n o f shifts so t h a t i z s m e a n Fs CLOSE ta zer3 . laçoffse~=round(mearl i L h a t 1 1 ; Lhat=Lhat-cnes (nbs , 1) * l a g o f f s e ~ ; A=lag (4, lagof fsec) ;Snat=A;

3 E s t i m a t i o n of C for j=l:nbs

A B 3 d ( : , j, :l=Aediag(Bhatij, : ) 1 : end f o r j=l:&s

for r-l :nfs shFf~=Lhat: j, z ) ;shiftint=fIoor(shift} ;shifcrem=shift-shiftint: if s h i f t i n c > = O

aj rShiftod=[=eros (sniftint, 1) ;squeeze (AB3d (1:nas-

3 TL: v a i x e s asA=sm!A. "21 /nas ;mB=sum(3 . -21 /nbs;nsC=suniC. - 2 : / x s ; X p r C h a n ç e = a ~ s (A-.XiatI *diag (msA."( - .5 i 1 * 190; aChangeKax=ma:i [nax ;A?ezChariqe) ; ; 3perChange-a~s ( 8 - m a t ' d i a g ( m s B . (- . 5) -100; bC!Ia~geMax=rnax (max (3pezChafiqe) 1 ; CperChange=aCs ( C-Chat) *diag (msC - ( - . 5) 1 'x (max ( Q e r C h a ~ q e 1 1 ; msqresL=sum~swn( (2-Lhatl . & 2 ) / {nbs-nfs) ; nsqres lor ig=sum(sum( (Lorig-Lhat) . -21 1 / (nbsWnfs) ; corrA-max (abs ( c o r r 2 s e t (Ahatf Aor ig ) ) 1 ; corrB-max (abs ( c o r r 2 s e t (Bhat , Bo r ig ) 1 ) ; corrC=max ( a b s ( c o r r 2 s e t (Cha t f Corig) 1 1 ; for j=l:nbs

shiftvec=Lhat ( j , : 1 ; s h i f t i n c v e c = f l o o r (shiftvec) ; shiftremvec=shiftvec-shiftintvec; for r=l:nfs

s h i f t = s h i f t i n t v e c ( r ) ; i f s h i f r > = O

A s h i f t e d ( : , r ) = [ z e r o s ( sh i f t ,L ) ;Aha t ! I : na s - sh i f t , r l 1 ; e l s e

end

MPENDIX G. Mean Shift Offset and Shifi Outliers

An underestimation has been suspected to look the recovery correlations of the

shift estimates much smailer than those of the standard factor loadings (A, B. or C). [f the

Mode B loadings corresponding to the shifi outliers are not small, the deviation of most

shifts by a constant amount must be considered meaningful rather than an artificial offset.

This is because the contribution of a particular factor to the fit and residuals for a column

of data is also a function of the corresponding B weight. To see whether or not the shifi

deviation by a constant amount was an artificial offset, the solutions of the "Hurnp" data

with 5% e m r were investigated. Eight out of wenty columns of shift estimates were

involved in the mean offset. As an example, one case is provided in Table 7 to show how

the mean offsets compensated for the outliers to make the rounded rnean of shifts zero,

without a big change in the model fit. In Table 7, the a11 shift estimates other than those in

row 9.10, 13, 15, and 16, are deviated from their true values by 1. In the five rows where

the shift deviates are not 1. the estimates of Mode B loadings are al1 small as expected,

except row 16. Even though the loadings at row 16 are not small, their effect on the

model fit and residuals will be relatively small because the amount of shift deviation at

row 16 is small (-1). This example well shows that the recovery correlation for A on that

factor has been underestimated. When the mean offset of shifts was corrected, the

recovery correlation for A increased fkom 0.9461 to 0.9893. 'The eight columns of A out

of 20, where the mean shift offset is present, were then corrected by their mean shift

offset. M e r the correction, the number of the best and matching solutions has increased

i?6$'T 6 Z P 9 . 1 OOPO'T LZTZ'T ZLEL 'O 08TT-O OLL9 ' 0 9ETO'O - C ::6?": C C C . C I - 4 - € ' 5 tP60 ' 0 0190 'O ? g t - z

94

f?om 3 to 6 (out of 10 solutions) and the recovery correlations of A from -9388 and -9982

to .9877 and .9970, respectively for each factor. Note that the recovery correlations are

only about the best fining "and" matching soIutions. The reason why the second recovery

correlation goes down is that the second factor in the initial 3 unique solutions were not

involved in the mean shft offset.

A few shift outlien seern to have made the recovery correlations of shifts

underestimated by an considerable amount, even though their coaesponding Mode B

loadings m u t have been relatively srnaIl. The number of unique solutions reported in

Table 2 rnay also have been underestimated for the same reason. The mean offset of

shifts must be the major reason why the recovery correlations for the shifts are much

smaller than the recovery correlations for A and B. When the recovery correlations are

low both for factor loadings (A and B) and for shifts (S), the low recovery correlations

for shifis might be partly because of the artificial rnean shift offset and parrly because of

the di fficult shape of sequential factors such as the "Sine" curves. This might be the case

in the results of the "Sine" data because the recovery correlations for factor loadings are

lower than those of the "Hand" and the "Hump" data. To veriS, this will need further

investigation.

r I V I M U C C V H L U H I I V I Y

TEST TARGET (QA-3)

APPLIED - IMAGE. lnc = 1653 East Main Street - -. . , Rochester. NY 14609 USA -- -- , - Phone: 716/482-0300 -- -- - - Fax: 71 6/200-5989

O 1993. Appiied Image. Inc.. Ail Rights Reserved