Date:         Mon, 25 Feb 2002 02:40:50 GMT
Reply-To:     Jeremy Fox <jerfox@STANFORD.EDU>
Sender:       "SAS(r) Discussion" <SAS-L@LISTSERV.UGA.EDU>
From:         Jeremy Fox <jerfox@STANFORD.EDU>
Subject:      Re: proc GLM?

Dale McLerran <stringplayer_2@yahoo.com> wrote:

: Here is the crux of the matter.  Even though you may be able to fit
: a model in which you can examine all manner of joint tests, you will
: be shot down at the door of just about any journal because your
: statistical model won't be understood.  It is too complex.  If it
: is not shot down because it is too complex to be understood, it will
: be shot down because you have provided yourself too much wiggle room
: to redefine just what are your joint distribution tests because there
: are so many joint tests that you could examine that noone is going
: to believe that you specified you tests a priori to fitting the
: model.

Hopefully most reputable authors would engage in experimentation using
multiple tests and estimates to see if their results are especially
sensitive to assumptions which are not essential to the question being
asked. If you are using some computationally intensive nonlinear
technique this might be difficult, but for linear techniques this
should be easy.

If journals want you to engage in a farce and pretend you only
estimated one model and didn't do anything else, then I feel sorry for
these moronic journal editors. In the end empirical work depends on
the honesty of the researcher performing it, not the silly policies of
journal editors.

: Ah, and here is another rub.  I set up my hypotheses right at the
: start and test them in a model.  You are only going to report the
: hypothesis "if it seems appropriate after estimation"?  I think
: you've got the cart before the horse.  Where is your hypothsis?

The critical aspect of the original poster's problem was to measure
the treatment effect in some experiment. Assigning responses to groups
doesn't seem to be the main issue on which you have to stand your
ground. You might decide based on theory that A and D are the
responses that measure the same thing, while after estimating my full
SUR model you find out that A and C were instead highly
correlated. Under my approach, I would ruminate on this new
information but wouldn't it slow me down in estimating treatment
effects. Under your approach, you would say that the mean of the
effects for A and D is different from your theoretical prediction, and
publish a paper with a negative result, without really exploring why
you are getting this negative result.

It could be just that your theoretical knowledge about the meaning of
A, C and D is wrong, but that doesn't really relate to the empirical
problem of measuring the effect of the treatment.

: p.s.  If there is further public discussion of this thread, I am
: likely not to participate beyond this posting, not because I would
: back away from anything which I have already stated, but I just have
: too much to do at present to belabor this further.  So, Jeremy, you
: may have the last word if you so choose.  (Unless someone else steps
: in with further response.)

All right, we can call it quits. I think we have both stated our
positions.

--
------------------------
Jeremy T. Fox
jerfox@stanford.edu