Date:         Tue, 13 Sep 2005 12:59:33 -0700
Reply-To:     Dale McLerran <stringplayer_2@YAHOO.COM>
Sender:       "SAS(r) Discussion" <SAS-L@LISTSERV.UGA.EDU>
From:         Dale McLerran <stringplayer_2@YAHOO.COM>
Subject:      Re: GLM CONTRAST vs TTEST
In-Reply-To:  <Pine.GSO.4.58.0509131405060.21498@leukothea>
Content-Type: text/plain; charset=iso-8859-1

--- Kevin Roland Viel <kviel@EMORY.EDU> wrote:

> On Tue, 13 Sep 2005, Dale McLerran wrote:
>
> > So, you ran GLM without restricting the class variable X to the
> > two values X=1 or X=2.  But the t-test is restricted to these
> > two values.
> >
> > If you ran PROC GLM restricting X to X=1 or X=2, then you should
> > get the same p-values as are returned by the TTEST procedure.
> > So, why doesn't PROC GLM give consistent results for the X=1 vs
> > X=2 contrast?  Because if you include additional levels of X
> > in the GLM procedure, then the variance estimate is obtained
> > from the residuals of all data included in the analysis.  When
> > you restrict the data to only those observations which have
> > X=1 or X=2, then you compute your pooled variance estimate only
> > from a subset of the observations employed in your unrestricted
> > GLM code.  And since your F-test or t-test employs the residual
> > variance in the denominator, then including additional data
> > which affect the residual variance will change your test p-value.
> >
> >
> > What are your denominator df values for the t-test and the F-test
> > from GLM.  This will also affect the p-value.  The number of
> > observations employed to compute the residual variance determines
> > the denominator df, so you will get a different p-value even
> > if the residual variance does not change at all.
>
> Dale,
>
>   This, of course, was precisely what I wanted.  I will still get
> back to
> one of my favorite text ("Applied Linear Statistical Models") when I
> get
> some breathing room.
>
>   My superficial interpretation is that I should expect to gain power
> using the unrestricted GLM, barring violations of the underlying
> assumptions.  Considering this bare model, it would seem that it is
> offering superiority over a (series) of T-test(s).  Does this seem
> fair?
>

Assuming that the variance is the same across all groups, then
the increase in denominator df obtained from the combined groups
analysis would give more power than the t-test for any specific
contrast.  The increase in power may be modest if you have large
samples in each group to begin with.  And you might need large
samples to really address the variance homogeneity assumption.
If you have small samples in each group and cannot examine very
well the assumptions of normality and homogeneity of variance,
then you might need to use some nonparametric methods.

>   I'll be anxious to see this in mathematical terms.
>
>   I also would like to express my gratitude to Dave Vishal and
> Paul Swank, who emphasized the multiple testing concern.
>

Yes, that is a very important concern.  Obviously, you need the
omnibus test significant before examining specific contrasts.
If you have specific contrasts that are of interest a priori,
then it is not necessary to adjust for testing of multiple
hypotheses.  However, if you are looking to see which of the
contrasts yield the significan omnibus test, then it is
necessary to adjust for multiple comparisons.

Dale


---------------------------------------
Dale McLerran
Fred Hutchinson Cancer Research Center
mailto: dmclerra@NO_SPAMfhcrc.org
Ph:  (206) 667-2926
Fax: (206) 667-5977
---------------------------------------



__________________________________
Yahoo! Mail - PC Magazine Editors' Choice 2005
http://mail.yahoo.com