| Date: | Fri, 21 May 2004 18:57:41 -0400 |
|---|
| Reply-To: | Stephen Salbod <ssalbod@pace.edu> |
|---|
| Sender: | "SPSSX(r) Discussion" <SPSSX-L@LISTSERV.UGA.EDU> |
|---|
| From: | Stephen Salbod <ssalbod@pace.edu> |
|---|
| Subject: | Inter-rater Reliability Problem |
|---|
| Content-Type: | text/plain; charset="us-ascii"; format=flowed |
|---|
A colleague asked me for a measure of inter-rater reliability for a
situation similar to the following:
Given 2 raters who rate a sentence from 10 participates
on five related factors.
# r1f1 r1f2 r1f3 r1f4 r1f5 r2f1 r2f2 r2f3 r2f4 r2f5
1 1 1 1 0 1 1 1 0 0 0
2 1 1 1 1 1 1 1 0 0 1
3 1 1 1 0 1 1 1 0 0 0
4 1 1 1 0 1 1 1 0 1 1
5 0 1 1 0 1 1 1 0 0 1
6 1 0 1 1 0 1 1 0 0 0
7 0 1 1 0 1 1 1 1 1 1
8 0 0 1 0 0 1 1 0 0 1
9 1 1 1 1 1 1 1 1 0 1
10 1 1 0 0 1 1 1 0 0 1
(The actual problem has 100 participates and 40 factors.)
Is the solution related to the ANOVA approach to inter-rater reliability?
If it is can someone recommend a reference on this approach.
Thank you,
Stephen Salbod, Pace University
| 
