Date: Tue, 20 Sep 2005 19:36:42 -0300
Reply-To: Hector Maletta <hmaletta@fibertel.com.ar>
Sender: "SPSSX(r) Discussion" <SPSSX-L@LISTSERV.UGA.EDU>
From: Hector Maletta <hmaletta@fibertel.com.ar>
Subject: Re: PCA and Rotation
In-Reply-To: <BAY107-F431FCDCA09F2E06DB8E9AED950@phx.gbl>
Content-Type: text/plain; charset="iso-8859-1"
Luis,
Some authors think different than others. As I told you before, rotating PCA
is usual and may be legitimate, though for certain purposes it may make no
sense.
Hector
> -----Original Message-----
> From: SPSSX(r) Discussion [mailto:SPSSX-L@LISTSERV.UGA.EDU]
> On Behalf Of Luis O.
> Sent: Tuesday, September 20, 2005 7:19 PM
> To: SPSSX-L@LISTSERV.UGA.EDU
> Subject: Re: PCA and Rotation
>
> Hi Hector, Paul, Manny, Stanley and William,
>
> Thank you for your help. Manny and William, I have just
> downloaded the papers you suggested, and will read them
> carefully this week.
>
> What I have learned through this debate is that there is a
> considerable debate over this issue, and it seems to me that
> there is a consensus that PCA and FA are two different
> methods in terms of the underlining statistical theory.
>
> Still, Paul's latest comment makes me a little puzzled: "If
> all I want to do is account for the maximum variance with the
> smallest number of indepedent components, then PCA with
> variamax rotation is okay". So if this is the case, PCA with
> the rotation is acceptable after all. That is, all depend
> upon what you plan to do in your research and analysis. Also,
> if you are not interested in latent structure of the
> variables, PCA is indeed a usuful method. Paul, please
> correct my statement if I misunderstand your explanation.
>
> Another point. In most of "classic" marketing research books,
> such as Multivariate Data Analysis by Hair, Anderson, Tatham
> and Blanck (Prentice
> Hall) explains that FA can be classified into two methods,
> CFA and PCA, and in BOTH cases, we can rotate the results
> (they even show a table of "VARIMAX-rotated component
> analysis factor matrix" on p. 126). Furthermore, Barbara G
> Tabachinick and Linda S Fidell provides a similar explanation
> in their Using Multivariate Statistics (4th ed.): "Principal
> component extraction with varimax rotation through SAS FACTOR
> is used in an initial run to estimate the number of factors
> from eigenvalues" (p. 633).
>
> If all the explanations I have learned in this discussion are
> true, it is rather surprising that the authors teach such
> procedure which is not considered as "correct" by many theorists.
>
> Luis
>
>
>
> >From: "Swank, Paul R" <Paul.R.Swank@uth.tmc.edu>
> >Reply-To: "Swank, Paul R" <Paul.R.Swank@uth.tmc.edu>
> >To: SPSSX-L@LISTSERV.UGA.EDU
> >Subject: Re: PCA and Rotation
> >Date: Tue, 20 Sep 2005 13:54:48 -0500
> >
> >PCA does not distinguish between common and specific factor variance
> >and does not explicity model the error variance. In studies
> where the
> >factor structure is known, PCA with the eigenvalue > 1 rule
> will often
> >incorrectly identify the number of factors present. Thirdly,
> rotation
> >is used for interpretation of the factors. Why rotate if not
> to interpret the factors?
> >Or components in the case of PCA. But since the components are not
> >common factors since they can contain specific variance, why try to
> >interpret them. If one wants to examine the underlying
> structure of the
> >data to find the dimensions represented, then one should
> look at common
> >factors with principal axes or maximum likelihood methods.
> Finally, in
> >the areas of biomedical and social sciences in which I work,
> there are
> >rarely uncorrelated factors. In fact there is a saying that all
> >psychological variables are correlated at least .30. This of
> course is
> >a rank generalization but it is true that many variables, and ther!
> > efore factors, are correlated. If the factors are not
> correlated then
> >the oblique solution will be similar to the orthogonal one
> so what is
> >lost by doing the oblique solution in the first place?
> >
> >Considering the variance accounted for by the components is a
> >characteristic of data reduction, not interpretation. If all
> I want to
> >do is account for the maximum variance with the smallest number of
> >indepedent components, then PCA with variamax rotation is
> okay. But if
> >one is attempting to understand the underlying dimensions then the
> >percentage of variance accounted for is less applicable. Of more
> >interest is accounting for all of the common factor
> variance. Then one
> >can eliminate factors that do not make sense or are
> otherwise not interpreatble.
> >
> >
> >Paul R. Swank, Ph.D.
> >Professor, Developmental Pediatrics
> >Director of Research, Center for Improving the Readiness of Children
> >for Learning and Education (C.I.R.C.L.E.) Medical School UT Health
> >Science Center at Houston
> >
> >-----Original Message-----
> >From: Hector Maletta [mailto:hmaletta@fibertel.com.ar]
> >Sent: Tuesday, September 20, 2005 11:17 AM
> >To: Swank, Paul R; SPSSX-L@LISTSERV.UGA.EDU
> >Subject: RE: PCA and Rotation
> >
> >I have not read Preacher and MacCallum's article. However,
> some remarks:
> >
> >1. From n variables one can extract up to n factors. If one
> does, and
> >uses PCA, one explains 100% of total variance in the n
> variables. With
> >other methods, n factors explain 100% of the commonality. If one
> >extracts only some of the factors, one explains only part of total
> >variance or total commonality.
> >
> >2. There is no objective criterion to stop extracting
> factors. One can
> >stop after one factor, two, three, 14 or 28 with equal theoretical
> >justification.
> >One can aim at explaining 20% of total variance, or 50%, or 90%,
> >depending on goals and circumstances. Therefore there is no
> under- or
> >over-factoring as such, only relative to some standard of
> comparison.
> >And the decision to extract fewer or more numerous factors
> can be made
> >with any of the factoring methods, not just with PCA.
> >
> >3. One criterion for determining the number of factors is
> stopping at
> >the last factor whose eigenvalue is >1, on the fragile grounds that
> >factors with eigenvalues >1 explain more variance than any observed
> >variable (whose "eigenvalue" is 1 by convention). This criterion is
> >essentially arbitrary, and has been criticized many times. For
> >different purposes you may want fewer or more factors.
> >
> >4. Another criterion is the so-called scree curve: stop when the
> >eigenvalue of a new factor differs only little from the
> preceding one
> >(i.e. when the slope of the eigenvalues curve becomes
> flatter). This is
> >still less justifiable than the above, but still widely used.
> >
> >5. In practical work one often needs (or has theoretical grounds to
> >seek) only the main one or just a few factors, and the
> extraction stops
> >when the analyst seems fit. In some specific applications
> one is trying
> >to capture as much as possible of total variance (or total
> commonality
> >as the case might be), and so one wants to use more factors. Since
> >factors are just shorthand constructs for the intercorrelation of
> >variables, it is all in the hands of the analyst, and one
> may take any
> >of these decisions without fear of divine retribution.
> >
> >Hector
> >
> >
> > > -----Original Message-----
> > > From: SPSSX(r) Discussion [mailto:SPSSX-L@LISTSERV.UGA.EDU] On
> > > Behalf Of Swank, Paul R
> > > Sent: Tuesday, September 20, 2005 1:17 PM
> > > To: SPSSX-L@LISTSERV.UGA.EDU
> > > Subject: Re: PCA and Rotation
> > >
> > > I highly recommend reading :
> > >
> > > Preacher, K. J., & MacCallum, R. C. (2003). Repairing Tom Swift's
> > > electric factor analysis machine. Understanding Statistics, 2(1),
> > > 13-32.
> > >
> > > It describes the "little jiffy", principal components analysis
> > > followed by a varimax rotation, as the most commonly used, yet
> > > incorrect, factoring procedure. It is uncorrect because PCA will
> > > typically underfactor, especially if using Kaiser's
> eigenvalue > 1
> > > criterion, it does not attempt to eliminate specific
> variance, and
> > > most of the time, factors will have some correaltion.
> > >
> > >
> > > Paul R. Swank, Ph.D.
> > > Professor, Developmental Pediatrics
> > > Director of Research, Center for Improving the Readiness
> of Children
> > > for Learning and Education (C.I.R.C.L.E.) Medical School
> UT Health
> > > Science Center at Houston
> > >
> > > -----Original Message-----
> > > From: SPSSX(r) Discussion [mailto:SPSSX-L@LISTSERV.UGA.EDU] On
> > > Behalf Of Luis O.
> > > Sent: Tuesday, September 20, 2005 8:53 AM
> > > To: SPSSX-L@LISTSERV.UGA.EDU
> > > Subject: Re: PCA and Rotation
> > >
> > > Hector, thank you for your clear explanations.
> > >
> > > Your comments bring me to the ultimate question: when we
> use SPSS's
> > > "Factor Analysis" with the "Principal Component"
> > > method and VARIMAX rotation. are we conducting FA or PCA?
> > >
> > > I realized that some researchers are arguing that SPSS's "Factor
> > > Analysis"
> > > with the default function of "Principal component" is causing a
> > > great confusion in distinguishing between FA and PCA.
> > > Because of this, my stat friends recommend not to choose
> principal
> > > componet method in FA.
> > > Furthermore, their explanations are accidentally similar to what
> > > Paul
> > > suggests: the use of oblique rotation.
> > >
> > > Luis
> > >
> > >
> > > >From: "Hector Maletta" <hmaletta@fibertel.com.ar>
> > > >To: "'Luis O.'" <soka28806@hotmail.com>
> > > >Subject: RE: PCA and Rotation
> > > >Date: Tue, 20 Sep 2005 08:26:07 -0300
> > > >
> > > >Luis,
> > > >I do not think your friends are right. Historically rotation
> > > arose with
> > > >principal axes, but nothing hinders rotation for principal
> > > components.
> > > >Both PCA and PA are methods of data reduction. They represent or
> > > >summarize n variables through k<n factors or components.
> > > Your friends
> > > >are probably thinking that if you apply PCA to identify one
> > > >dominant factor underlying all your varibles, and explaining by
> > > itself a large
> > > >portion of total variance, it is pointless to rotate it. But
> > > PCA is not
> > > >restricted to that situation, and often two or more factors
> > > are needed
> > > >to explain a large part of variance. Those factors may
> be rotated
> > > >to make the structure more clear, i.e. to associate each factor
> > > >more closely with some group of variables that have a
> common meaning (e.g.
> > > >various linguistic tests associated mainly with one factor or
> > > >component).
> > > >
> > > >By the way, factor and component are the same. The
> vocabulary just
> > > >reflects how they were used by their introductors in the
> > > past, nearly 100 years ago.
> > > >
> > > >Hector
> > > >
> > > >
> > > > > -----Original Message-----
> > > > > From: Luis O. [mailto:soka28806@hotmail.com]
> > > > > Sent: Tuesday, September 20, 2005 5:43 AM
> > > > > To: hmaletta@fibertel.com.ar
> > > > > Subject: Re: PCA and Rotation
> > > > >
> > > > > Paul and Hector, thank you very much for your quick reply. I
> > > > > sincerely appreciaed your explanations.
> > > > >
> > > > > I want to ask one more question to Hector. Based on your
> > > > > explanations, you seem to be saying that in BOTH
> cases (PCA and
> > > > > Principal Axes), we can rotate the principal components
> > > > > (although you switched the word to "factor" in the later
> > > > > paragraphs) in order to "put one factor closer to one set of
> > > > > observed variables, and far from others, and the opposite for
> > > > > another factor". Is this correct? I am asking this
> > > question, because
> > > > > this is precisely the doubt I had. Some of my stat
> > > friends say that
> > > > > in PCA the rotation should not be used, because it is a data
> > > > > reduction procedure (as Paul explains), and provides a
> > > best linear
> > > > > combination of the variance.
> > > > > Furthermore, they are making a point in that if we rotate the
> > > > > principal components in PCA, they are no longer "principal"
> > > > > components.
> > > > >
> > > > > I would appreciate your further clarification.
> > > > >
> > > > > Luis
> > > > >
> > > > >
> > > > > >From: Hector Maletta <hmaletta@fibertel.com.ar>
> > > > > >Reply-To: Hector Maletta <hmaletta@fibertel.com.ar>
> > > > > >To: SPSSX-L@LISTSERV.UGA.EDU
> > > > > >Subject: Re: PCA and Rotation
> > > > > >Date: Mon, 19 Sep 2005 20:54:50 -0300
> > > > > >
> > > > > >Some clarification:
> > > > > >PCA is designed to maximize the first underlying factor,
> > > > > maximizing its
> > > > > >contribution, and (besides) when it extracts all
> > > possible factors
> > > > > >(number of factor equal to the number of variables) it
> > > > > accounts for the
> > > > > >entire variance of the variables. Other methods of
> extraction
> > > > > >do not maximize the first, and besides they try to
> explain only
> > > > > >the COMMON variance, leaving aside a portion of
> variance that
> > > > > >is regarded as unique to each observed variable.
> > > > > >Historically, PCA was used first to identify a single factor
> > > > > underlying
> > > > > >a set of related measures, supposedly measuring all the
> > > same trait
> > > > > >(intelligence, as it was), while Principal Axes was used
> > > > > based on the
> > > > > >theory that instead of a single Intelligence factor
> there were
> > > > > >underlying "intelligences" for varios "faculties of the
> > > > > mind" such as
> > > > > >linguistic, graphic or mathematical ability.
> > > > > >In BOTH cases, the position of the coordinate system on
> > > which the
> > > > > >underlying factors are measured is essentially arbitrary. By
> > > > > rotation,
> > > > > >the analyst may be able to put one factor closer to one set
> > > > > of observed
> > > > > >variables, and far from others, and the opposite for
> > > another factor.
> > > > > >This make happen by chance in the initial extraction, but
> > > > > >ordinarily doesn't, so rotation is used in order to get that
> > > > > >nice
> > > > > characteristic
> > > > > >of a factor linked to sets of interrelated variables.
> > > For instance,
> > > > > >using the same IQ example, by rotation one may get one
> > > > > factor strongly
> > > > > >associated with several linguistic tests, and only
> > > weakly related
> > > > > >to other tests, while another factor is strongly related to
> > > > > mathematical
> > > > > >tests and weakly to other tests, so one intuitively
> > > calls the first
> > > > > >factor "linguistic" and the second "mathematical".
> > > > > >These different factors may be independe from each other, or
> > > > > >correlated. It is perfectly possible that being good in
> > > > > >language implies being good also in math, and so at least to
> > > > > >some
> > > > > degree, some
> > > > > >correlation between linguistic and math factors is only to
> > > > > be expected.
> > > > > >Initial patterns of extraction ordinarily extract
> > > factors that are
> > > > > >orthogonal or uncorrelated to each other, because each
> > > successive
> > > > > >factor is extracted on the unexplained residuals left by the
> > > > > preceding
> > > > > >ones, but rotation can position the factor axes in ways that
> > > > > imply they
> > > > > >are correlated to each other.
> > > > > >Rotation preserving the independence of factors is called
> > > > > >orthogonal rotation. Rotation allowing them to be
> correlated to
> > > > > >each other is called oblique rotation.
> > > > > >So in Paul Swank responde there is a (probably involuntary)
> > > > > confusion
> > > > > >at the
> > > > > >end: oblique rotation yields correlated factors (though
> > > it does not
> > > > > >FORCE them to be correlated) while orthogonal
> rotation methods
> > > > > >FORCE rotated factors to be uncorrelated to each other.
> > > > > >
> > > > > >Hector
> > > > > >
> > > > > >
> > > > > > > -----Original Message-----
> > > > > > > From: SPSSX(r) Discussion
> > > [mailto:SPSSX-L@LISTSERV.UGA.EDU] On
> > > > > > > Behalf Of Swank, Paul R
> > > > > > > Sent: Monday, September 19, 2005 8:26 PM
> > > > > > > To: SPSSX-L@LISTSERV.UGA.EDU
> > > > > > > Subject: Re: PCA and Rotation
> > > > > > >
> > > > > > > Principal components is a data reduction procedure,
> > > not a way to
> > > > > > > identify interpretable factors. To do the latter, use
> > > > > principal axes
> > > > > > > analysis or some other factor algorithm that targets
> > > > > common factors.
> > > > > > > It makes sense to rotate these since you are
> intreseted in
> > > > > > > inpretable factors. However, I suggest an oblique
> > > > > rotation to ensure
> > > > > > > that the factors are no correlated before forcing
> them to be.
> > > > > > >
> > > > > > >
> > > > > > > Paul R. Swank, Ph.D.
> > > > > > > Professor, Developmental Pediatrics Director of
> > > Research, Center
> > > > > > > for Improving the Readiness
> > > > > of Children
> > > > > > > for Learning and Education (C.I.R.C.L.E.) Medical School
> > > > > UT Health
> > > > > > > Science Center at Houston
> > > > > > >
> > > > > > > -----Original Message-----
> > > > > > > From: SPSSX(r) Discussion
> > > [mailto:SPSSX-L@LISTSERV.UGA.EDU] On
> > > > > > > Behalf Of Luis O.
> > > > > > > Sent: Monday, September 19, 2005 5:14 PM
> > > > > > > To: SPSSX-L@LISTSERV.UGA.EDU
> > > > > > > Subject: PCA and Rotation
> > > > > > >
> > > > > > > Dear List Members,
> > > > > > >
> > > > > > > I am new in the list and want to ask a very basic
> > > > > question regarding
> > > > > > > the principal component analysis. I was doing some
> > > > > analysis by using
> > > > > > > principal component method and VArimax rotation. However,
> > > > > one of my
> > > > > > > friends told me that the stat book says that we
> should not
> > > > > > > rotate principal components.
> > > > > > > That is, the principal component analysis should not
> > > rotate the
> > > > > > > soluations, because, by theory, it produces a unique
> > > > > soluation. On
> > > > > > > the other hand, when I read some SPSS manuals, they
> > > > > usually tell you
> > > > > > > to use the principal component method with some
> > > rotation method.
> > > > > > > Which is correct?
> > > > > > >
> > > > > > > Luis O. Benavent
> > > > > > > Benavnet Talps Research
> > > > > > > Gandía, Spain
> > > > > > >
> > > > > > > __________ Información de NOD32 1.1222 (20050919)
> __________
> > > > > > >
> > > > > > > Este mensaje ha sido analizado con NOD32
> Antivirus System
> > > > > > > http://www.nod32.com
> > > > > > >
> > > > > > >
> > > > >
> > > > >
> > > > >
> > > > > __________ Información de NOD32 1.1224 (20050920) __________
> > > > >
> > > > > Este mensaje ha sido analizado con NOD32 Antivirus System
> > > > > http://www.nod32.com
> > > > >
> > > > >
> > > >
> > >
> > > __________ Información de NOD32 1.1224 (20050920) __________
> > >
> > > Este mensaje ha sido analizado con NOD32 Antivirus System
> > > http://www.nod32.com
> > >
> > >
>
> __________ Información de NOD32 1.1226 (20050920) __________
>
> Este mensaje ha sido analizado con NOD32 Antivirus System
> http://www.nod32.com
>
>
| 
