Date:         Tue, 18 Mar 2003 11:18:52 -0800
Reply-To:     cassell.david@EPAMAIL.EPA.GOV
Sender:       "SAS(r) Discussion" <SAS-L@LISTSERV.UGA.EDU>
From:         "David L. Cassell" <cassell.david@EPAMAIL.EPA.GOV>
Subject:      Re: FW: Outputting ONLY significant results
Content-type: text/plain; charset=us-ascii

Peter Flom <flom@NDRI.ORG> sagely replied:
> I agree fully that this is usually a bad idea.  I can't think of a
reason to
> do so in the ordinary course of data analysis.  And I'm not sure why
Doug wants
> to do this.  BUT, here is one possible application.
>
> Suppose you are doing a Monte Carlo study.  You want to see how many
variables
> some statistical technique regards as significant, and which ones it
regards as
> significant, to compare these results to what you know to be the
'correct' ones.
>
> If you ran, say, 1000 iterations of PROC REG with 10 IVs, 3 of which
'ought' to
> be significant, and 7 ought not be, then this might be one way to
reduce the output.
> Of course, there might be better ways to do this as well.
>
> I only thought of this because I am getting interested in doing some
Monte Carlo simulations myself

In this case, I recommend aggregating the results, but in a "X
significant at Y level
out of Z runs" sort of way.  You might want to look at my
randomization-test wrapper
to see how I approached this problem.  Try this URL:

   http://www.wuss.org/Conference/papers/DA07.pdf

David
--
David Cassell, CSC
Cassell.David@epa.gov
Senior computing specialist
mathematical statistician