Date:         Sat, 29 Apr 2006 12:41:47 +0200
Reply-To:     "Marc Halbrügge" <marc.halbruegge@gmx.de>
Sender:       "SPSSX(r) Discussion" <SPSSX-L@LISTSERV.UGA.EDU>
From:         "Marc Halbrügge" <marc.halbruegge@gmx.de>
Subject:      Re: correlation between covariates and DV
Comments: To: Richard Ristow <wrristow@mindspring.com>
Content-Type: text/plain; charset="us-ascii"

>
> >I would say, if the mean square error is smaller with the covariate in
> >the model then leave it in.
>
> Am I missing something? I believe the mean square error will be smaller
> if you add ANY variable to a linear model, whether the variable
> actually had anything to do with the DV or not. The simple way to see
> this is, the fitting minimizes the mean square error. When you add a
> variable, the old model is still available: it could be fit with a 0
> coefficient on the new variable. So, mean-square error can't INCREASE.
> It will decrease unless, let's see, I think, unless the new variable is
> uncorrelated with the residuals from the model without the new
> variable.
You're right. As soon as the correlation between the new variable and the
residuals differs from zero, the mean square error decreases.
If you would follow the proposal above and keep on adding variables, you
would finish with a model that fits very well on the data, but generalizes
poorly.


--
GMX Produkte empfehlen und ganz einfach Geld verdienen!
Satte Provisionen für GMX Partner: http://www.gmx.net/de/go/partner