Date:         Fri, 21 Sep 2001 12:29:48 -0700
Reply-To:     John Uebersax <jsuebersax@YAHOO.COM>
Sender:       "SAS(r) Discussion" <SAS-L@LISTSERV.UGA.EDU>
From:         John Uebersax <jsuebersax@YAHOO.COM>
Organization: http://groups.google.com/
Subject:      Re: inter rater problem from a different angle
Content-Type: text/plain; charset=ISO-8859-1

Hi Chris,

This formulation of the problem is much better--it answers most of
questions about your first post.

It still isn't clear what the ratings consist of.  Do ratings consist
of terms from a defined set of possible terms--such as "poor",
"moderate", and "good"?

Kappa statistics can be used to test the null-hypothesis of agreement
at chance levels.  A problem with these data are that they are not
independent--i.e., you really don't have 22 independent data points,
so the usuall assumptions of the kappa test are violated.  You could
calculate kappa, but the usual formula for it's standard error would
not apply.

--------------------------------------------------------------------------------
John Uebersax, PhD             (805) 384-7688
Thousand Oaks, California      (805) 383-1726 (fax)
email: jsuebersax@yahoo.com

Agreement Stats:   http://ourworld.compuserve.com/homepages/jsuebersax/agree.htm
Latent Structure:  http://ourworld.compuserve.com/homepages/jsuebersax
Existential Psych: http://members.aol.com/spiritualpsych
--------------------------------------------------------------------------------