Date:         Fri, 2 Dec 2005 06:25:26 -0800
Reply-To:     Paige Miller <paige.miller@ITT.COM>
Sender:       "SAS(r) Discussion" <SAS-L@LISTSERV.UGA.EDU>
From:         Paige Miller <paige.miller@ITT.COM>
Organization: http://groups.google.com
Subject:      Re: Influential observations
Comments: To: sas-l@uga.edu
In-Reply-To:  <43900BD7.B875.00C9.0@ndri.org>
Content-Type: text/plain; charset="iso-8859-1"

Peter Flom wrote:

> There's a very good discussion of PLS, ridge, and other methods in
> Hastie, Tibshirani, and Friedman (2001) Elements of statistical
> learning.
>
> Writing any details of these methods in e-mail is tricky (wouldn't it
> be nice if you could
> use LaTeX in e-mail?) but they do say
>
> "PLS, PCR  [principal components regression] and ridge regression tend
> to perform similarly.
> Ridge regression may be preferred because it shrink smoothly, rather
> than in discrete steps" (p 70)

I haven't read Hastie, Tibshirani and Friedman carefully, but perhaps
someone could enlighten me, as a user of statistical methods, why I
would care that one method "shrink(s) smoothly" while another method
shrinks in discrete steps? What is the practical relevance to me? And I
would like to point out that if I use either PLS or Ridge Regression,
the amount of shrinkage is rarely (never) a concern to me. Should it
be?

The paper by Frank and Friedman, which HTF relies upon for some of its
conclusions about PLS, has been criticized for not simulating data
which is typical of the data that PLS has been used on. This could
unfairly bias the results away from showing PLS is superior on those
types of data (although I do not claim that this was done intentionally
by Frank and Friedman). Thus, although I have no simulation study to
back up my belief, I do believe that PLS is superior on those types of
data, not just in its quantifiable aspects, but as I said before, also
in the non-quantifiable aspects of the procedure.

Finally, why does anyone do PCR any more? PLS is logically superior by
design if your goal is to find subspaces of the X-variables that
predict (subspaces of) the Y-variables.

--
Paige Miller
paige.miller@itt.com