Date:         Wed, 12 Mar 2003 10:54:21 -0800
Reply-To:     cassell.david@EPAMAIL.EPA.GOV
Sender:       "SAS(r) Discussion" <SAS-L@LISTSERV.UGA.EDU>
From:         "David L. Cassell" <cassell.david@EPAMAIL.EPA.GOV>
Subject:      Re: Non Sign. Main Effect/Significant Post Hoc
Content-type: text/plain; charset=us-ascii

Thompson Bill T Contr USAFSAM/FEC <Bill.Thompson@BROOKS.AF.MIL> wrote:
> In writing the code for performing a MANOVA I included the code for
> performing post hoc tests on the main effects before knowing if they
were
> significant.  Interestingly the results showed no main effect for my
> variable but when looking at the bonferroni (only 3 levels of indep.
var.)
> post hoc it showed significance.  Even more interesting is that in
another
> analysis the exact opposite occurred where there was a significant
main
> effect but post hoc analysis did not identify a significant
relationship.

I second everything Jim Groeneveld said.  You cannot expect that the
overall F test and individual values of multiple comparison tests will
give you the same answer, *particularly* when using something like
Bonferroni.  In fact, I don't recommend using Bonferroni here.  I would
recommend Tukey, without knowing more of your analysis plan.

If your data do not meet the underlying assumptions of your MANOVA, then
you can get really differing results.  In particular, if you have
unequal
variances, then individual pairwise comparisons may look quite unlike
one
overall test assuming homoskedasticity.  As Jim alluded, consider
plotting
out your data to see: (1) if your assumptions are being met; (2) if your
data really look like some levels have differing means than other
levels;
and (3) if there might be other effects or interactions driving some of
your process.

HTH,
David
--
David Cassell, CSC
Cassell.David@epa.gov
Senior computing specialist
mathematical statistician