Date: Thu, 27 Dec 2007 00:26:20 -0800
Reply-To: Albert-jan Roskam <fomcl@yahoo.com>
Sender: "SPSSX(r) Discussion" <SPSSX-L@LISTSERV.UGA.EDU>
From: Albert-jan Roskam <fomcl@yahoo.com>
Subject: Re: Cherrypicking & publication bias
In-Reply-To: <006501c843e4$1f042930$6e413c0a@YOURF03EC04B22>
Content-Type: text/plain; charset=iso-8859-1
Hi all,
Thanks to everybody who responded to my mail!
Somebody sent me a very interesting article off-list.
It's in PLoS Medicine (open source):
Why most published research findings are false.
Ioannidis JP. (2005) PLoS Med. 2005 Aug;2(8):e124.
Epub 2005 Aug 30.
http://www.ncbi.nlm.nih.gov/sites/entrez?Db=pubmed&Cmd=ShowDetailView&TermToSearch=16060722&ordinalpos=4&itool=EntrezSystem2.PEntrez.Pubmed.Pubmed_ResultsPanel.Pubmed_RVDocSum
If that link is too l-o-n-g, try:
http://tinyurl.com/2te7ub
** PubMed Abstract:
There is increasing concern that most current
published research findings are false. The probability
that a research claim is true may depend on study
power and bias, the number of other studies on the
same question, and, importantly, the ratio of true to
no relationships among the relationships probed in
each scientific field. In this framework, a research
finding is less likely to be true when the studies
conducted in a field are smaller; when effect sizes
are smaller; when there is a greater number and lesser
preselection of tested relationships; where there is
greater flexibility in designs, definitions, outcomes,
and analytical modes; when there is greater financial
and other interest and prejudice; and when more teams
are involved in a scientific field in chase of
statistical significance. Simulations show that for
most study designs and settings, it is more likely for
a research claim to be false than true. Moreover, for
many current scientific fields, claimed research
findings may often be simply accurate measures of the
prevailing bias. In this essay, I discuss the
implications of these problems for the conduct and
interpretation of research.
PMID: 16060722 [PubMed - indexed for MEDLINE]
Cheers!!
Albert-Jan
--- Anthony Babinec <tbabinec@sbcglobal.net> wrote:
> While not a central theme of the book,
> "cherry-picking" comes up in Rex Kline's
> book "Beyond Significance Testing." His
> review of the misconceptions and misuses
> surrounding p-values is very good.
>
> Anthony Babinec
> tbabinec@sbcglobal.net
>
> -----Original Message-----
> From: SPSSX(r) Discussion
> [mailto:SPSSX-L@LISTSERV.UGA.EDU] On Behalf Of
> Albert-jan Roskam
> Sent: Friday, December 21, 2007 7:53 AM
> To: SPSSX-L@LISTSERV.UGA.EDU
> Subject: Cherrypicking & publication bias
>
> Hi again listers,
>
> Another, completely different post this time. I was
> wondering if you could recommend some texts about
> "cherrypicking" and "publication bias". Cherry
> picking
> may be defined as "the act of pointing at individual
> cases or data that seem to confirm a particular
> position, while ignoring a significant portion of
> related cases or data that may contradict that
> position". Publication bias refers to the "tendency
> for researchers and editors to handle experimental
> results that are positive (they found something)
> differently from results that are negative (found
> that
> something did not happen) or inconclusive."
>
> In a time where the number of publications sometimes
> appears to be more important than the actual
> *contents*of those papers, where the researcher's
> daily bread so heavily depends on how often s/he has
> a
> paper accepted, those two phenomena may (in my view)
> become a serious threat to science. This would
> result
> in a 'polished' version of reality, esp. in
> meta-analyses. I was wondering if any of you could
> recommend some reading materials, quantifications,
> etc. (published or unpublished! ;-) about this. I am
> aware of some papers, I believe in the Lancet and
> BMJ
> that did not find evidence for publication bias. One
> disclosure, however: these were written by the
> editors
> of those journals!
>
> Thanks in advance and merry x-mas!
>
> Albert-Jan
>
>
>
>
> Cheers!
> Albert-Jan
>
>
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> Did you know that 87.166253% of all statistics claim
> a precision of results
> that is not justified by the method employed?
> [HELMUT RICHTER]
>
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>
>
>
>
____________________________________________________________________________
> ________
> Be a better friend, newshound, and
> know-it-all with Yahoo! Mobile. Try it now.
>
http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ
>
> =====================
> To manage your subscription to SPSSX-L, send a
> message to
> LISTSERV@LISTSERV.UGA.EDU (not to SPSSX-L), with no
> body text except the
> command. To leave the list, send the command
> SIGNOFF SPSSX-L
> For a list of commands to manage subscriptions, send
> the command
> INFO REFCARD
>
> =====================
> To manage your subscription to SPSSX-L, send a
> message to
> LISTSERV@LISTSERV.UGA.EDU (not to SPSSX-L), with no
> body text except the
> command. To leave the list, send the command
> SIGNOFF SPSSX-L
> For a list of commands to manage subscriptions, send
> the command
> INFO REFCARD
>
____________________________________________________________________________________
Be a better friend, newshound, and
know-it-all with Yahoo! Mobile. Try it now. http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ
=====================
To manage your subscription to SPSSX-L, send a message to
LISTSERV@LISTSERV.UGA.EDU (not to SPSSX-L), with no body text except the
command. To leave the list, send the command
SIGNOFF SPSSX-L
For a list of commands to manage subscriptions, send the command
INFO REFCARD
| 
