LISTSERV at the University of Georgia
Menubar Imagemap
Home Browse Manage Request Manuals Register
Previous messageNext messagePrevious in topicNext in topicPrevious by same authorNext by same authorPrevious page (September 2005)Back to main SPSSX-L pageJoin or leave SPSSX-L (or change settings)ReplyPost a new messageSearchProportional fontNon-proportional font
Date:         Wed, 21 Sep 2005 13:47:01 +1000
Reply-To:     paulandpen@optusnet.com.au
Sender:       "SPSSX(r) Discussion" <SPSSX-L@LISTSERV.UGA.EDU>
From:         Paul Dickson <paulandpen@optusnet.com.au>
Subject:      Re: PCA and Rotation
Comments: To: "Luis O ." <soka28806@hotmail.com>
Content-Type: text/plain

Hi Luis

Have a look at the link below. There is a test on Brian OÇonnors web page where he recommends using Velicers test to guide the choice of the number of factors in PCA that might be useful for you as well.

http://flash.lakeheadu.ca/~boconno2/nfactors.html

Cheers Paul D

> Luis O. <soka28806@hotmail.com> wrote: > > Hi Hector, Paul, Manny, Stanley and William, > > Thank you for your help. Manny and William, I have just downloaded the > papers you suggested, and will read them carefully this week. > > What I have learned through this debate is that there is a considerable > debate over this issue, and it seems to me that there is a consensus > that > PCA and FA are two different methods in terms of the underlining > statistical > theory. > > Still, Paul's latest comment makes me a little puzzled: "If all I want > to do > is account for the maximum variance with the smallest number of > indepedent > components, then PCA with variamax rotation is okay". So if this is the > case, PCA with the rotation is acceptable after all. That is, all depend > upon what you plan to do in your research and analysis. Also, if you are > not > interested in latent structure of the variables, PCA is indeed a usuful > method. Paul, please correct my statement if I misunderstand your > explanation. > > Another point. In most of "classic" marketing research books, such as > Multivariate Data Analysis by Hair, Anderson, Tatham and Blanck > (Prentice > Hall) explains that FA can be classified into two methods, CFA and PCA, > and > in BOTH cases, we can rotate the results (they even show a table of > "VARIMAX-rotated component analysis factor matrix" on p. 126). > Furthermore, > Barbara G Tabachinick and Linda S Fidell provides a similar explanation > in > their Using Multivariate Statistics (4th ed.): "Principal component > extraction with varimax rotation through SAS FACTOR is used in an > initial > run to estimate the number of factors from eigenvalues" (p. 633). > > If all the explanations I have learned in this discussion are true, it > is > rather surprising that the authors teach such procedure which is not > considered as "correct" by many theorists. > > Luis > > > > >From: "Swank, Paul R" <Paul.R.Swank@uth.tmc.edu> > >Reply-To: "Swank, Paul R" <Paul.R.Swank@uth.tmc.edu> > >To: SPSSX-L@LISTSERV.UGA.EDU > >Subject: Re: PCA and Rotation > >Date: Tue, 20 Sep 2005 13:54:48 -0500 > > > >PCA does not distinguish between common and specific factor variance > and > >does not explicity model the error variance. In studies where the > factor > >structure is known, PCA with the eigenvalue > 1 rule will often > incorrectly > >identify the number of factors present. Thirdly, rotation is used for > >interpretation of the factors. Why rotate if not to interpret the > factors? > >Or components in the case of PCA. But since the components are not > common > >factors since they can contain specific variance, why try to interpret > >them. If one wants to examine the underlying structure of the data to > find > >the dimensions represented, then one should look at common factors with > >principal axes or maximum likelihood methods. Finally, in the areas of > >biomedical and social sciences in which I work, there are rarely > >uncorrelated factors. In fact there is a saying that all psychological > >variables are correlated at least .30. This of course is a rank > >generalization but it is true that many variables, and ther! > > efore factors, are correlated. If the factors are not correlated then > the > >oblique solution will be similar to the orthogonal one so what is lost > by > >doing the oblique solution in the first place? > > > >Considering the variance accounted for by the components is a > >characteristic of data reduction, not interpretation. If all I want to > do > >is account for the maximum variance with the smallest number of > indepedent > >components, then PCA with variamax rotation is okay. But if one is > >attempting to understand the underlying dimensions then the percentage > of > >variance accounted for is less applicable. Of more interest is > accounting > >for all of the common factor variance. Then one can eliminate factors > that > >do not make sense or are otherwise not interpreatble. > > > > > >Paul R. Swank, Ph.D. > >Professor, Developmental Pediatrics > >Director of Research, Center for Improving the Readiness of Children > for > >Learning and Education (C.I.R.C.L.E.) > >Medical School > >UT Health Science Center at Houston > > > >-----Original Message----- > >From: Hector Maletta [mailto:hmaletta@fibertel.com.ar] > >Sent: Tuesday, September 20, 2005 11:17 AM > >To: Swank, Paul R; SPSSX-L@LISTSERV.UGA.EDU > >Subject: RE: PCA and Rotation > > > >I have not read Preacher and MacCallum's article. However, some > remarks: > > > >1. From n variables one can extract up to n factors. If one does, and > uses > >PCA, one explains 100% of total variance in the n variables. With other > >methods, n factors explain 100% of the commonality. If one extracts > only > >some of the factors, one explains only part of total variance or total > >commonality. > > > >2. There is no objective criterion to stop extracting factors. One can > stop > >after one factor, two, three, 14 or 28 with equal theoretical > >justification. > >One can aim at explaining 20% of total variance, or 50%, or 90%, > depending > >on goals and circumstances. Therefore there is no under- or > over-factoring > >as such, only relative to some standard of comparison. And the decision > to > >extract fewer or more numerous factors can be made with any of the > >factoring methods, not just with PCA. > > > >3. One criterion for determining the number of factors is stopping at > the > >last factor whose eigenvalue is >1, on the fragile grounds that factors > >with eigenvalues >1 explain more variance than any observed variable > (whose > >"eigenvalue" is 1 by convention). This criterion is essentially > arbitrary, > >and has been criticized many times. For different purposes you may want > >fewer or more factors. > > > >4. Another criterion is the so-called scree curve: stop when the > eigenvalue > >of a new factor differs only little from the preceding one (i.e. when > the > >slope of the eigenvalues curve becomes flatter). This is still less > >justifiable than the above, but still widely used. > > > >5. In practical work one often needs (or has theoretical grounds to > seek) > >only the main one or just a few factors, and the extraction stops when > the > >analyst seems fit. In some specific applications one is trying to > capture > >as much as possible of total variance (or total commonality as the case > >might be), and so one wants to use more factors. Since factors are just > >shorthand constructs for the intercorrelation of variables, it is all > in > >the hands of the analyst, and one may take any of these decisions > without > >fear of divine retribution. > > > >Hector > > > > > > > -----Original Message----- > > > From: SPSSX(r) Discussion [mailto:SPSSX-L@LISTSERV.UGA.EDU] On > Behalf > > > Of Swank, Paul R > > > Sent: Tuesday, September 20, 2005 1:17 PM > > > To: SPSSX-L@LISTSERV.UGA.EDU > > > Subject: Re: PCA and Rotation > > > > > > I highly recommend reading : > > > > > > Preacher, K. J., & MacCallum, R. C. (2003). Repairing Tom Swift's > > > electric factor analysis machine. Understanding Statistics, 2(1), > > > 13-32. > > > > > > It describes the "little jiffy", principal components analysis > > > followed by a varimax rotation, as the most commonly used, yet > > > incorrect, factoring procedure. It is uncorrect because PCA will > > > typically underfactor, especially if using Kaiser's eigenvalue > 1 > > > criterion, it does not attempt to eliminate specific variance, and > > > most of the time, factors will have some correaltion. > > > > > > > > > Paul R. Swank, Ph.D. > > > Professor, Developmental Pediatrics > > > Director of Research, Center for Improving the Readiness of Children > > > for Learning and Education (C.I.R.C.L.E.) Medical School UT Health > > > Science Center at Houston > > > > > > -----Original Message----- > > > From: SPSSX(r) Discussion [mailto:SPSSX-L@LISTSERV.UGA.EDU] On > Behalf > > > Of Luis O. > > > Sent: Tuesday, September 20, 2005 8:53 AM > > > To: SPSSX-L@LISTSERV.UGA.EDU > > > Subject: Re: PCA and Rotation > > > > > > Hector, thank you for your clear explanations. > > > > > > Your comments bring me to the ultimate question: when we use SPSS's > > > "Factor Analysis" with the "Principal Component" > > > method and VARIMAX rotation. are we conducting FA or PCA? > > > > > > I realized that some researchers are arguing that SPSS's "Factor > > > Analysis" > > > with the default function of "Principal component" is causing a > great > > > confusion in distinguishing between FA and PCA. > > > Because of this, my stat friends recommend not to choose principal > > > componet method in FA. > > > Furthermore, their explanations are accidentally similar to what > Paul > > > suggests: the use of oblique rotation. > > > > > > Luis > > > > > > > > > >From: "Hector Maletta" <hmaletta@fibertel.com.ar> > > > >To: "'Luis O.'" <soka28806@hotmail.com> > > > >Subject: RE: PCA and Rotation > > > >Date: Tue, 20 Sep 2005 08:26:07 -0300 > > > > > > > >Luis, > > > >I do not think your friends are right. Historically rotation > > > arose with > > > >principal axes, but nothing hinders rotation for principal > > > components. > > > >Both PCA and PA are methods of data reduction. They represent or > > > >summarize n variables through k<n factors or components. > > > Your friends > > > >are probably thinking that if you apply PCA to identify one > dominant > > > >factor underlying all your varibles, and explaining by > > > itself a large > > > >portion of total variance, it is pointless to rotate it. But > > > PCA is not > > > >restricted to that situation, and often two or more factors > > > are needed > > > >to explain a large part of variance. Those factors may be rotated > to > > > >make the structure more clear, i.e. to associate each factor more > > > >closely with some group of variables that have a common meaning > (e.g. > > > >various linguistic tests associated mainly with one factor or > > > >component). > > > > > > > >By the way, factor and component are the same. The vocabulary just > > > >reflects how they were used by their introductors in the > > > past, nearly 100 years ago. > > > > > > > >Hector > > > > > > > > > > > > > -----Original Message----- > > > > > From: Luis O. [mailto:soka28806@hotmail.com] > > > > > Sent: Tuesday, September 20, 2005 5:43 AM > > > > > To: hmaletta@fibertel.com.ar > > > > > Subject: Re: PCA and Rotation > > > > > > > > > > Paul and Hector, thank you very much for your quick reply. I > > > > > sincerely appreciaed your explanations. > > > > > > > > > > I want to ask one more question to Hector. Based on your > > > > > explanations, you seem to be saying that in BOTH cases (PCA and > > > > > Principal Axes), we can rotate the principal components > (although > > > > > you switched the word to "factor" in the later > > > > > paragraphs) in order to "put one factor closer to one set of > > > > > observed variables, and far from others, and the opposite for > > > > > another factor". Is this correct? I am asking this > > > question, because > > > > > this is precisely the doubt I had. Some of my stat > > > friends say that > > > > > in PCA the rotation should not be used, because it is a data > > > > > reduction procedure (as Paul explains), and provides a > > > best linear > > > > > combination of the variance. > > > > > Furthermore, they are making a point in that if we rotate the > > > > > principal components in PCA, they are no longer "principal" > > > > > components. > > > > > > > > > > I would appreciate your further clarification. > > > > > > > > > > Luis > > > > > > > > > > > > > > > >From: Hector Maletta <hmaletta@fibertel.com.ar> > > > > > >Reply-To: Hector Maletta <hmaletta@fibertel.com.ar> > > > > > >To: SPSSX-L@LISTSERV.UGA.EDU > > > > > >Subject: Re: PCA and Rotation > > > > > >Date: Mon, 19 Sep 2005 20:54:50 -0300 > > > > > > > > > > > >Some clarification: > > > > > >PCA is designed to maximize the first underlying factor, > > > > > maximizing its > > > > > >contribution, and (besides) when it extracts all > > > possible factors > > > > > >(number of factor equal to the number of variables) it > > > > > accounts for the > > > > > >entire variance of the variables. Other methods of extraction > do > > > > > >not maximize the first, and besides they try to explain only > the > > > > > >COMMON variance, leaving aside a portion of variance that is > > > > > >regarded as unique to each observed variable. > > > > > >Historically, PCA was used first to identify a single factor > > > > > underlying > > > > > >a set of related measures, supposedly measuring all the > > > same trait > > > > > >(intelligence, as it was), while Principal Axes was used > > > > > based on the > > > > > >theory that instead of a single Intelligence factor there were > > > > > >underlying "intelligences" for varios "faculties of the > > > > > mind" such as > > > > > >linguistic, graphic or mathematical ability. > > > > > >In BOTH cases, the position of the coordinate system on > > > which the > > > > > >underlying factors are measured is essentially arbitrary. By > > > > > rotation, > > > > > >the analyst may be able to put one factor closer to one set > > > > > of observed > > > > > >variables, and far from others, and the opposite for > > > another factor. > > > > > >This make happen by chance in the initial extraction, but > > > > > >ordinarily doesn't, so rotation is used in order to get that > nice > > > > > characteristic > > > > > >of a factor linked to sets of interrelated variables. > > > For instance, > > > > > >using the same IQ example, by rotation one may get one > > > > > factor strongly > > > > > >associated with several linguistic tests, and only > > > weakly related > > > > > >to other tests, while another factor is strongly related to > > > > > mathematical > > > > > >tests and weakly to other tests, so one intuitively > > > calls the first > > > > > >factor "linguistic" and the second "mathematical". > > > > > >These different factors may be independe from each other, or > > > > > >correlated. It is perfectly possible that being good in > language > > > > > >implies being good also in math, and so at least to some > > > > > degree, some > > > > > >correlation between linguistic and math factors is only to > > > > > be expected. > > > > > >Initial patterns of extraction ordinarily extract > > > factors that are > > > > > >orthogonal or uncorrelated to each other, because each > > > successive > > > > > >factor is extracted on the unexplained residuals left by the > > > > > preceding > > > > > >ones, but rotation can position the factor axes in ways that > > > > > imply they > > > > > >are correlated to each other. > > > > > >Rotation preserving the independence of factors is called > > > > > >orthogonal rotation. Rotation allowing them to be correlated to > > > > > >each other is called oblique rotation. > > > > > >So in Paul Swank responde there is a (probably involuntary) > > > > > confusion > > > > > >at the > > > > > >end: oblique rotation yields correlated factors (though > > > it does not > > > > > >FORCE them to be correlated) while orthogonal rotation methods > > > > > >FORCE rotated factors to be uncorrelated to each other. > > > > > > > > > > > >Hector > > > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > From: SPSSX(r) Discussion > > > [mailto:SPSSX-L@LISTSERV.UGA.EDU] On > > > > > > > Behalf Of Swank, Paul R > > > > > > > Sent: Monday, September 19, 2005 8:26 PM > > > > > > > To: SPSSX-L@LISTSERV.UGA.EDU > > > > > > > Subject: Re: PCA and Rotation > > > > > > > > > > > > > > Principal components is a data reduction procedure, > > > not a way to > > > > > > > identify interpretable factors. To do the latter, use > > > > > principal axes > > > > > > > analysis or some other factor algorithm that targets > > > > > common factors. > > > > > > > It makes sense to rotate these since you are intreseted in > > > > > > > inpretable factors. However, I suggest an oblique > > > > > rotation to ensure > > > > > > > that the factors are no correlated before forcing them to > be. > > > > > > > > > > > > > > > > > > > > > Paul R. Swank, Ph.D. > > > > > > > Professor, Developmental Pediatrics Director of > > > Research, Center > > > > > > > for Improving the Readiness > > > > > of Children > > > > > > > for Learning and Education (C.I.R.C.L.E.) Medical School > > > > > UT Health > > > > > > > Science Center at Houston > > > > > > > > > > > > > > -----Original Message----- > > > > > > > From: SPSSX(r) Discussion > > > [mailto:SPSSX-L@LISTSERV.UGA.EDU] On > > > > > > > Behalf Of Luis O. > > > > > > > Sent: Monday, September 19, 2005 5:14 PM > > > > > > > To: SPSSX-L@LISTSERV.UGA.EDU > > > > > > > Subject: PCA and Rotation > > > > > > > > > > > > > > Dear List Members, > > > > > > > > > > > > > > I am new in the list and want to ask a very basic > > > > > question regarding > > > > > > > the principal component analysis. I was doing some > > > > > analysis by using > > > > > > > principal component method and VArimax rotation. However, > > > > > one of my > > > > > > > friends told me that the stat book says that we should not > > > > > > > rotate principal components. > > > > > > > That is, the principal component analysis should not > > > rotate the > > > > > > > soluations, because, by theory, it produces a unique > > > > > soluation. On > > > > > > > the other hand, when I read some SPSS manuals, they > > > > > usually tell you > > > > > > > to use the principal component method with some > > > rotation method. > > > > > > > Which is correct? > > > > > > > > > > > > > > Luis O. Benavent > > > > > > > Benavnet Talps Research > > > > > > > Gandía, Spain > > > > > > > > > > > > > > __________ Información de NOD32 1.1222 (20050919) __________ > > > > > > > > > > > > > > Este mensaje ha sido analizado con NOD32 Antivirus System > > > > > > > http://www.nod32.com > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > __________ Información de NOD32 1.1224 (20050920) __________ > > > > > > > > > > Este mensaje ha sido analizado con NOD32 Antivirus System > > > > > http://www.nod32.com > > > > > > > > > > > > > > > > > > > > __________ Información de NOD32 1.1224 (20050920) __________ > > > > > > Este mensaje ha sido analizado con NOD32 Antivirus System > > > http://www.nod32.com > > > > > >


Back to: Top of message | Previous page | Main SPSSX-L page