Date: Fri, 12 Nov 2010 13:00:09 -0500
Reply-To: Michael Palij <mp26@nyu.edu>
Sender: "SPSSX(r) Discussion" <SPSSX-L@LISTSERV.UGA.EDU>
From: Michael Palij <mp26@nyu.edu>
Subject: Re: non-SPSS: appropriate statistical test
In-Reply-To: <1289583245997-3262459.post@n5.nabble.com>
Content-Type: text/plain; charset=us-ascii
As usual, Bruce gives good advice. ;-) What he is suggesting,
I believe, might be something like have Pearson r set
to values of 0.00, 0.10, 0.20, 0.30 and so on, and see
what happens to the obtained t-value (i.e., does it
become statistically significant). One might also use
Cohen's recommended values for small, medium, and
large effect sizes for r (his "Power Primer" article would
have the values -- a Google Scholar search may turn up
the article if one doesn't have access to PsycInfo/Articles).
I hesistated making such a recommendation because I
think that it may be possible to come up with an empirically
derived value for the Pearson r given that data (though this
might be difficult). Ultimately, what one decides to do
depends upon the question(s) one wants to answer and
how important it is to get to the "truth" of the situation.
-Mike Palij
New York University
mp26@nyu.edu
----- Original Message -----
From: Bruce Weaver <bruce.weaver@hotmail.com>
Date: Friday, November 12, 2010 12:39 pm
Subject: Re: non-SPSS: appropriate statistical test
To: SPSSX-L@LISTSERV.UGA.EDU
> As usual, Mike is giving solid advice. The only point at which I did
> a
> (small) double-take was where he said, "If you cannot calculate "r", you
> have to assume that it is equal to zero". I was reminded of situations
> where I did not know the value of some parameter, and was therefore advised
> to do a "sensitivity analysis". I.e., do the computation a few times
> with a
> range of plausible values plugged in for the unknown parameter to determine
> how sensitive the result was to the value of that parameter. That was
> when
> I was working for people doing medical research, and it seemed that
> the term
> "sensitivity analysis" was well known in those circles. I had never heard
> of it prior to that (my background to that point had been in experimental
> psychology). The obvious potential problem here is convincing other people
> that the values you plugged in are plausible. ;-)
>
> HTH.
>
>
>
> Mike Palij wrote:
> >
> > Not sure I'm a greater mind but here goes:
> >
> > (1) Simple stuff first: if you are doing t-tests, the general formula
> > for the t-test is the following:
> >
> > Obtained t=(M1 - M2)/sqrt[VarErr1 + VarErr2 - 2*r*SE1*SE2]
> >
> > Where M1=mean group1, M2=mean group2,VarErr1=Variance error group1,
> > VarErr2=Variance error group2, r=Pearson r between group1 and group2
> > valaues, SE1=standard error group1, SE2=standard error group2, and
> > 2=constant (the number 2).
> >
> > If you cannot calculate "r", you have to assume that it is equal to
> zero
> > which makes the t-test denominator = sqrt [ VarErr1 + VarErr1]. This
> > denominator will be larger than the denominator if "r" is known. The
> > good news is if the t-test is significant under the assumption of r=0.00,
> > then it has to be significant if you can calculate r (NOTE: r is typically
> > a positive value -- a negative r should cause you to re-examine your
> > data).
> > The bad news is if the t-test is non-significant, it could be so because
> > there is no real difference or you failed to find a significant difference
> > because you could not adjust (reduce) your denominator appropriately.
> >
> > So, treating your data as independent groups makes the test more
> > conservative
> > or less powerful. I am open to correction on these points.
> >
> > (2) It seems to me that you should be able to get an estimate of the
> > Pearson r through bootstrapping or some other simulation procedure.
> > If there is a positive correlation between time 1 and time 2, then,
> > assuming
> > data consisting only of 0 and 1, time1 zeros should co-occur with time2
> > zeros at a greater than chance level and the same holds for ones.
> even if
> > they are not matched up properly. I haven't thought this through but
> > perhaps someone more familiar with bootstrapping with correlation
> > has more wisdom.
> >
> > -Mike Palij
> > New York University
> > mp26@nyu.edu
> >
> >
> >
> > ----- Original Message -----
> > From: J P
> > To: SPSSX-L@LISTSERV.UGA.EDU
> > Sent: Friday, November 12, 2010 9:19 AM
> > Subject: non-SPSS: appropriate statistical test
> >
> >
> > Colleaguees,
> >
> > This is not a SPSS question (at least not yet).
> >
> > I am seeking advice on the appropriate test for comparing two
> > non-independent samples when the non-independence cannot be modeled.
> >
> > The proportions are drawn from the same employees pop (~ 700, response
> > rate of ~50%) employee population, surveyd one year apart. An
> example of
> > an actual comparison is 98.4% vs 96.1% between time1 and time2.
> >
> > The problem, as I see it, is the two samples are not independent but
> > there is no ID so neither a dependent t-test nor a mixed model can be
> > used. I found a test for comparing proportions from two independent
> > groups.
> >
> > What is the risk of violating the assumption of independence? inflated
> > type 1 error?
> >
> > As far as I know there is no appropriate test for this situation,
> but I
> > thought I'd check with minds greater than mine...
> >
> > Thank you,
> >
> > John
> >
> >
> >
> >
> >
> >
>
>
> -----
> --
> Bruce Weaver
> bweaver@lakeheadu.ca
> http://sites.google.com/a/lakeheadu.ca/bweaver/
>
> "When all else fails, RTFM."
>
> NOTE: My Hotmail account is not monitored regularly.
> To send me an e-mail, please use the address shown above.
>
> --
> View this message in context: http://spssx-discussion.1045642.n5.nabble.com/comparing-groups-in-two-different-datasets-tp3255523p3262459.html
> Sent from the SPSSX Discussion mailing list archive at Nabble.com.
>
> =====================
> To manage your subscription to SPSSX-L, send a message to
> LISTSERV@LISTSERV.UGA.EDU (not to SPSSX-L), with no body text except the
> command. To leave the list, send the command
> SIGNOFF SPSSX-L
> For a list of commands to manage subscriptions, send the command
> INFO REFCARD
=====================
To manage your subscription to SPSSX-L, send a message to
LISTSERV@LISTSERV.UGA.EDU (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD
| 
