Date: Tue, 5 Sep 2006 10:06:53 +0100
Reply-To: Margaret MacDougall <email@example.com>
Sender: "SPSSX(r) Discussion" <SPSSX-L@LISTSERV.UGA.EDU>
From: Margaret MacDougall <firstname.lastname@example.org>
Subject: Re: Measures of agreement for individuals categories when the
categories are ordinal
Content-Type: text/plain; charset=iso-8859-1
Thank you for this kind reply. Having had a brief look at how the relevant program works, I am somewhat discouraged by the fact that I am required to enter individual scores by hand in order to obtain my results. The data is currently in an SPSS spreadhseet and there are 1718 entries.
Marta García-Granero <email@example.com> wrote:
There is a good freeware program called Kappa.exe (from PEPI 4.0
collection of DOS programs) that will compute kappa for ordinal
MM> I have been using an intra-class correlation coefficient to
MM> analyse my data which is on an ordinal scale from 1 to 7. The
MM> analysis involves a two-way mixed effects model in which overall
MM> absolute agreement is being measured. I would like to complement
MM> the results to date with further results relating to the level of
MM> agreement for each category individually (under the assumption
MM> that there are two raters). As I understand from my reading,
MM> there are a number of definitions of Kappa statistics which allow
MM> for the assessment of chance-corrected inter-rater agreement over
MM> grade A only, say. However, it appears that the related
MM> calculations involve the assumption that there are only two
MM> categories (in the above example: 'grade A' or 'other grade').
MM> The generalization 'other grade' removes the capacity to assess
MM> the extent to which individual examiners disagree on an ordinal
MM> scale when one examiner assings the grade A but the other does
MM> not. I wonder therefore if anyone is
MM> aware of alternative chance-corrected approaches to
MM> assessing agreement between two raters for a single category
MM> whereby whenever the raters disagree, the extent of disagreement
MM> is taken into consideration.
All New Yahoo! Mail – Tired of Vi@gr@! come-ons? Let our SpamGuard protect you.