```Date: Tue, 20 Sep 2005 13:11:27 -0300 Reply-To: Hector Maletta Sender: "SPSSX(r) Discussion" From: Hector Maletta Subject: Re: PCA and Rotation Comments: To: "Luis O." In-Reply-To: Content-Type: text/plain; charset="iso-8859-1" Luis, Do not get confused by mere names. Factor analysis includes Principal Component Analysis as one of its methods for factor extraction. In fact, PCA and FA are synonimous, but historically the introduction of factor analysis was done using PCA as a method for factor extraction, and thence the different denominations that still persist in textbooks. SPSS Factor procedure involves both. The procedure could be called PCA, and that would amount to the same. Besides, doing an oblique or orthogonal rotation is a different question altogether. If you rotate the factors or components in an oblique faction, you end up with a number of underlying or latent variables (the "factors") that are correlated to each other. Therefore, part of the variance in the original variables will be shared between two or more of these factors. Moreover, once you obtain an oblique solution, you could still perform a second-order factor analysis to find the common underlying factor/s that explain the correlations between your first-order underlying factors. (I hope you do not make the second-order factors correlated too, for this would suggest a third-order factorization and so on until you run out of degrees of freedom --or patience). Independent or orthogonal factors explain parts of the variance in the variables that do not overlap. After letting Factor 1 explain as much as it can, Factor 2 explains as much as it can of the residuals, and so on. So Factors 1 and 2 may be regarded as completely unrelated variables. Correlated factors (those resulting from an oblique rotation) cannot be identified with different unrelated variables; since they share part of their variability with each other, they may be seen in part as the expression of some common, deeper variable. If you want this game to end, you need to arrive either to one overarching factor that explains most of the variance, or to several UNRELATED factors each explaining a part of it. The classical example of intelligence is useful in this context. The initial work by Spearmann used PCA to identify a "general intelligence" factor, correlated with all intelligence tests and even with most of the individual items in those tests. The rest of the variance was attributed to random error and to unique factors associated with each individual test. Years later Thurstone developed the idea of multiple intelligences (linguistic, visual, logical, etc.), and used Principal Axes Factoring to extract several independent factors, later rotated in orthogonal or oblique fashion to make them pass through the appropriate tests (i.e. the "linguistic" factor was a factor that could be made, through rotation, to have high loadings on linguistic tests and low loadings on other tests). Since the same data set could be analyzed either way, yielding one general or several partial intelligence factors, analysts have long recognized that "factors" are mathematical constructs and not real objects. One uses them to summarize data and illustrate theory, and may choose one or another depending on external evidence, better data fit, or other considerations. Hector > -----Original Message----- > From: SPSSX(r) Discussion [mailto:SPSSX-L@LISTSERV.UGA.EDU] > On Behalf Of Luis O. > Sent: Tuesday, September 20, 2005 11:53 AM > To: SPSSX-L@LISTSERV.UGA.EDU > Subject: Re: PCA and Rotation > > Hector, thank you for your clear explanations. > > Your comments bring me to the ultimate question: when we use > SPSS's "Factor Analysis" with the "Principal Component" > method and VARIMAX rotation. are we conducting FA or PCA? > > I realized that some researchers are arguing that SPSS's > "Factor Analysis" > with the default function of "Principal component" is causing > a great confusion in distinguishing between FA and PCA. > Because of this, my stat friends recommend not to choose > principal componet method in FA. > Furthermore, their explanations are accidentally similar to what Paul > suggests: the use of oblique rotation. > > Luis > > > >From: "Hector Maletta" > >To: "'Luis O.'" > >Subject: RE: PCA and Rotation > >Date: Tue, 20 Sep 2005 08:26:07 -0300 > > > >Luis, > >I do not think your friends are right. Historically rotation > arose with > >principal axes, but nothing hinders rotation for principal > components. > >Both PCA and PA are methods of data reduction. They represent or > >summarize n variables through k Your friends > >are probably thinking that if you apply PCA to identify one dominant > >factor underlying all your varibles, and explaining by > itself a large > >portion of total variance, it is pointless to rotate it. But > PCA is not > >restricted to that situation, and often two or more factors > are needed > >to explain a large part of variance. Those factors may be rotated to > >make the structure more clear, i.e. to associate each factor more > >closely with some group of variables that have a common > meaning (e.g. > >various linguistic tests associated mainly with one factor or > >component). > > > >By the way, factor and component are the same. The vocabulary just > >reflects how they were used by their introductors in the > past, nearly 100 years ago. > > > >Hector > > > > > > > -----Original Message----- > > > From: Luis O. [mailto:soka28806@hotmail.com] > > > Sent: Tuesday, September 20, 2005 5:43 AM > > > To: hmaletta@fibertel.com.ar > > > Subject: Re: PCA and Rotation > > > > > > Paul and Hector, thank you very much for your quick reply. I > > > sincerely appreciaed your explanations. > > > > > > I want to ask one more question to Hector. Based on your > > > explanations, you seem to be saying that in BOTH cases (PCA and > > > Principal Axes), we can rotate the principal components (although > > > you switched the word to "factor" in the later > > > paragraphs) in order to "put one factor closer to one set of > > > observed variables, and far from others, and the opposite for > > > another factor". Is this correct? I am asking this > question, because > > > this is precisely the doubt I had. Some of my stat > friends say that > > > in PCA the rotation should not be used, because it is a data > > > reduction procedure (as Paul explains), and provides a > best linear > > > combination of the variance. > > > Furthermore, they are making a point in that if we rotate the > > > principal components in PCA, they are no longer "principal" > > > components. > > > > > > I would appreciate your further clarification. > > > > > > Luis > > > > > > > > > >From: Hector Maletta > > > >Reply-To: Hector Maletta > > > >To: SPSSX-L@LISTSERV.UGA.EDU > > > >Subject: Re: PCA and Rotation > > > >Date: Mon, 19 Sep 2005 20:54:50 -0300 > > > > > > > >Some clarification: > > > >PCA is designed to maximize the first underlying factor, > > > maximizing its > > > >contribution, and (besides) when it extracts all > possible factors > > > >(number of factor equal to the number of variables) it > > > accounts for the > > > >entire variance of the variables. Other methods of extraction do > > > >not maximize the first, and besides they try to explain only the > > > >COMMON variance, leaving aside a portion of variance that is > > > >regarded as unique to each observed variable. > > > >Historically, PCA was used first to identify a single factor > > > underlying > > > >a set of related measures, supposedly measuring all the > same trait > > > >(intelligence, as it was), while Principal Axes was used > > > based on the > > > >theory that instead of a single Intelligence factor there were > > > >underlying "intelligences" for varios "faculties of the > > > mind" such as > > > >linguistic, graphic or mathematical ability. > > > >In BOTH cases, the position of the coordinate system on > which the > > > >underlying factors are measured is essentially arbitrary. By > > > rotation, > > > >the analyst may be able to put one factor closer to one set > > > of observed > > > >variables, and far from others, and the opposite for > another factor. > > > >This make happen by chance in the initial extraction, but > > > >ordinarily doesn't, so rotation is used in order to get that nice > > > characteristic > > > >of a factor linked to sets of interrelated variables. > For instance, > > > >using the same IQ example, by rotation one may get one > > > factor strongly > > > >associated with several linguistic tests, and only > weakly related > > > >to other tests, while another factor is strongly related to > > > mathematical > > > >tests and weakly to other tests, so one intuitively > calls the first > > > >factor "linguistic" and the second "mathematical". > > > >These different factors may be independe from each other, or > > > >correlated. It is perfectly possible that being good in language > > > >implies being good also in math, and so at least to some > > > degree, some > > > >correlation between linguistic and math factors is only to > > > be expected. > > > >Initial patterns of extraction ordinarily extract > factors that are > > > >orthogonal or uncorrelated to each other, because each > successive > > > >factor is extracted on the unexplained residuals left by the > > > preceding > > > >ones, but rotation can position the factor axes in ways that > > > imply they > > > >are correlated to each other. > > > >Rotation preserving the independence of factors is called > > > >orthogonal rotation. Rotation allowing them to be correlated to > > > >each other is called oblique rotation. > > > >So in Paul Swank responde there is a (probably involuntary) > > > confusion > > > >at the > > > >end: oblique rotation yields correlated factors (though > it does not > > > >FORCE them to be correlated) while orthogonal rotation methods > > > >FORCE rotated factors to be uncorrelated to each other. > > > > > > > >Hector > > > > > > > > > > > > > -----Original Message----- > > > > > From: SPSSX(r) Discussion > [mailto:SPSSX-L@LISTSERV.UGA.EDU] On > > > > > Behalf Of Swank, Paul R > > > > > Sent: Monday, September 19, 2005 8:26 PM > > > > > To: SPSSX-L@LISTSERV.UGA.EDU > > > > > Subject: Re: PCA and Rotation > > > > > > > > > > Principal components is a data reduction procedure, > not a way to > > > > > identify interpretable factors. To do the latter, use > > > principal axes > > > > > analysis or some other factor algorithm that targets > > > common factors. > > > > > It makes sense to rotate these since you are intreseted in > > > > > inpretable factors. However, I suggest an oblique > > > rotation to ensure > > > > > that the factors are no correlated before forcing them to be. > > > > > > > > > > > > > > > Paul R. Swank, Ph.D. > > > > > Professor, Developmental Pediatrics Director of > Research, Center > > > > > for Improving the Readiness > > > of Children > > > > > for Learning and Education (C.I.R.C.L.E.) Medical School > > > UT Health > > > > > Science Center at Houston > > > > > > > > > > -----Original Message----- > > > > > From: SPSSX(r) Discussion > [mailto:SPSSX-L@LISTSERV.UGA.EDU] On > > > > > Behalf Of Luis O. > > > > > Sent: Monday, September 19, 2005 5:14 PM > > > > > To: SPSSX-L@LISTSERV.UGA.EDU > > > > > Subject: PCA and Rotation > > > > > > > > > > Dear List Members, > > > > > > > > > > I am new in the list and want to ask a very basic > > > question regarding > > > > > the principal component analysis. I was doing some > > > analysis by using > > > > > principal component method and VArimax rotation. However, > > > one of my > > > > > friends told me that the stat book says that we should not > > > > > rotate principal components. > > > > > That is, the principal component analysis should not > rotate the > > > > > soluations, because, by theory, it produces a unique > > > soluation. On > > > > > the other hand, when I read some SPSS manuals, they > > > usually tell you > > > > > to use the principal component method with some > rotation method. > > > > > Which is correct? > > > > > > > > > > Luis O. Benavent > > > > > Benavnet Talps Research > > > > > Gandía, Spain > > > > > > > > > > __________ Información de NOD32 1.1222 (20050919) __________ > > > > > > > > > > Este mensaje ha sido analizado con NOD32 Antivirus System > > > > > http://www.nod32.com > > > > > > > > > > > > > > > > > > > > > > __________ Información de NOD32 1.1224 (20050920) __________ > > > > > > Este mensaje ha sido analizado con NOD32 Antivirus System > > > http://www.nod32.com > > > > > > > > > > __________ Información de NOD32 1.1224 (20050920) __________ > > Este mensaje ha sido analizado con NOD32 Antivirus System > http://www.nod32.com > > ```

Back to: Top of message | Previous page | Main SPSSX-L page