Date: Fri, 20 Apr 2007 13:30:29 -0500
Reply-To: "Swank, Paul R" <Paul.R.Swank@UTH.TMC.EDU>
Sender: "SAS(r) Discussion" <SAS-L@LISTSERV.UGA.EDU>
From: "Swank, Paul R" <Paul.R.Swank@UTH.TMC.EDU>
Subject: Re: Kendall coefficient of concordance
In-Reply-To: <1390147.1177092395501.JavaMail.root@mswamui-swiss.atl.sa.earthlink.net>
Content-Type: text/plain; charset="us-ascii"
however, nothing is going to work very well when most of the responses
fall into one cell. Kappa, which compares agreement against chance
agreement will not be very high here because the chance of agreement is
very high when most of the reponses are in one cell.
Paul R. Swank, Ph.D. Professor
Director of Reseach
Children's Learning Institute
University of Texas Health Science Center-Houston
-----Original Message-----
From: SAS(r) Discussion [mailto:SAS-L@LISTSERV.UGA.EDU] On Behalf Of
Peter Flom
Sent: Friday, April 20, 2007 1:07 PM
To: SAS-L@LISTSERV.UGA.EDU
Subject: Re: Kendall coefficient of concordance
>From: Michelle Secic <consult@SECICSTATS.COM> wrote I'm looking at
>concordance of two raters on 3 possible findings and the 3 x 3
>crosstable of the findings have lots of zeros and the Kendall stat via
>%magree results in 0.5 even though almost all the data (n>250) fall in
>a single cell of agreement. Please let me know if you know of any
>references that talk about unreliabile Kendell stat with lots of zeros,
>or something like that. I'm trying to find a reference for this...
>Thanks so much!!!
Kendall's coefficient isn't unreliable, it just answers one particular
question. If you want to aks a different question, then you want a
different statistic. There are a vast number of statistics to measure
agreement between raters, most notably Cohen's kappa:
http://en.wikipedia.org/wiki/Cohen's_kappa
This, and many other measures, are discussed in many books on
nonparametric statistics. e.g.
Agresti: Categorical Data Analysis
Fleiss: Statistical Methods for Rates and Proportions
You can also search Lex Jansen's page: http://www.lexjansen.com/sugi/
for keywords like 'agreement'.
HTH
Peter
