Date: Tue, 14 Sep 2010 14:49:29 -0700
Reply-To: Bruce Weaver <bruce.weaver@hotmail.com>
Sender: "SPSSX(r) Discussion" <SPSSX-L@LISTSERV.UGA.EDU>
From: Bruce Weaver <bruce.weaver@hotmail.com>
Subject: Re: Chow test to compare regression on two groups
In-Reply-To: <7.0.1.0.2.20100914154043.034f0208@mindspring.com>
Content-Type: text/plain; charset=us-ascii
Richard Ristow wrote:
>
>
> --- snip ---
>
> Second, testing all six sex*coping
> interactions individually is an instance of multiple comparisons. That
> multiplies your chance of getting a significant result where there's no
> underlying effect ("Type I error").
> Applying the Bonferroni correction, which is admittedly conservative, you
> should report the sex*coping6 as significant
> only if you have p<.05/6, i.e. p<.008.
>
>
Richard's comment raises a point that I've thought about frequently. It
seems to me that when it comes to linear models that contain both
categorical and continuous explanatory variables, the world can be divided
into two camps:
1. the ANOVA camp, and
2. the regression camp.
The ANOVA camp tends to be made up of folks who do designed experiments
(e.g., experimental psychologists. The regression camp tends to be made up
of people who work with observational data. Personally, I started off in
the ANOVA camp, as a psychology student; but after starting to work for
people doing medical and health-related research, I've shifted somewhat (but
not completely) toward the regression camp.
I won't ramble on at great length, but here are a few of the differences
that I've noticed between these camps.
a) ANOVA folks who are doing designed experiments are content with MUCH
smaller sample sizes than are regression folks. In some experimental
fields, researchers would say that group sizes well below 10 are just dandy.
For regression, the rules of thumb suggest that much larger samples would be
needed (e.g.,
http://www.angelfire.com/wv/bwhomedir/notes/linreg_rule_of_thumb.txt).
b) In the basic ANCOVA model, ANOVA folks are much more concerned than
regression folks about equality of the groups on the covariate. In fact,
some ANOVA campers argue that the ANCOVA model should only be used when
there is random assignment to groups. But regression campers view ANCOVA as
a regression model with one continuous and one explanatory variable, and are
often unconcerned about whether or not there was random assignment to
groups.
c) ANOVA folks are much more concerned than regression folks about
correcting for multiple tests. E.g., suppose you had a study with several
treatments compared to a common control. ANOVA campers would likely use
Dunnett's test to control the family-wise error. Regression campers, on the
other hand, would create k-1 indicator variables for the treatments, and use
the t-tests on those k-1 coefficients without any correction.
I've always found this last example rather curious, given that running it as
an ANOVA or a regression doesn't change anything--it's exactly the same
model in both cases. I suppose it just goes to show how big a role
convention plays in all of the various fields.
Getting back to Richard's point, I do agree that regression campers ought to
be more concerned about the multiple testing problem than they often are.
Oh yes...one more difference just occurred to me.
d) ANOVA campers love the interaction. Traditionally, regression campers
have not been very fond of interactions, and have gone to great lengths to
avoid having them in their models. I suspect this is because they struggle
to interpret what the coefficients for the product terms mean, and they
often fail to understand that the coefficients for main effects are really
simple main effects, etc. Books like the one by Aiken & West can help a lot
in this regard, though.
Cheers,
Bruce
-----
--
Bruce Weaver
bweaver@lakeheadu.ca
http://sites.google.com/a/lakeheadu.ca/bweaver/
"When all else fails, RTFM."
NOTE: My Hotmail account is not monitored regularly.
To send me an e-mail, please use the address shown above.
--
View this message in context: http://spssx-discussion.1045642.n5.nabble.com/Chow-test-to-compare-regression-on-two-groups-tp2839448p2839846.html
Sent from the SPSSX Discussion mailing list archive at Nabble.com.
=====================
To manage your subscription to SPSSX-L, send a message to
LISTSERV@LISTSERV.UGA.EDU (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD
